Compare commits
33 Commits
da9b911f1e
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
 curo1305
|
0f760c379d | ||
|
curo1305
|
f13ef88711 | ||
|
curo1305
|
0d8e0366c6 | ||
|
curo1305
|
3a66aeeec5 | ||
|
curo1305
|
248b2bb9d7 | ||
|
curo1305
|
cfec3bb906 | ||
|
curo1305
|
4c35d7a2a4 | ||
|
curo1305
|
2f3efb9bf9 | ||
|
curo1305
|
5349f21752 | ||
|
curo1305
|
50d2348b36 | ||
|
curo1305
|
d345ace86d | ||
|
curo1305
|
c59718171c | ||
|
curo1305
|
99d22660f9 | ||
|
curo1305
|
fcfc06cda9 | ||
 curo
|
f5bc28cda2 | ||
|
curo1305
|
1c8b35399c | ||
|
curo1305
|
ebf97b6f4a | ||
|
curo1305
|
fec3953009 | ||
|
curo1305
|
6e5e5c08bf | ||
|
curo1305
|
05d79d3d21 | ||
|
curo1305
|
75b7ae6062 | ||
|
curo1305
|
479108779f | ||
|
curo1305
|
c5976882be | ||
|
curo1305
|
64808e0928 | ||
|
curo1305
|
94901fc30f | ||
|
curo
|
08e7caac4c | ||
|
curo1305
|
f16c290b92 | ||
|
curo1305
|
c45236651b | ||
|
curo1305
|
003fbee20f | ||
|
curo1305
|
18a638bc3a | ||
|
curo1305
|
00466a9801 | ||
|
curo1305
|
2d7207b62f | ||
|
curo1305
|
608b0b7fe8 |
@@ -5,8 +5,21 @@
|
||||
"Bash(git add:*)",
|
||||
"Bash(git commit -m ':*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git pull:*)",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(git merge:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git config:*)",
|
||||
"Bash(git mv:*)",
|
||||
"Bash(docker compose:*)",
|
||||
"Bash(docker run:*)"
|
||||
"Bash(docker run:*)",
|
||||
"Bash(docker inspect:*)",
|
||||
"Bash(docker ps:*)",
|
||||
"Bash(curl:*)",
|
||||
"Bash(lsof:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
@@ -22,5 +22,9 @@ resume.txt
|
||||
# Test fixtures — drop PDFs here for local testing, never commit them
|
||||
features/doc-service/tests/pdfs/*.pdf
|
||||
|
||||
# Feature branch test stacks — never commit these
|
||||
docker-compose.feat-*.yml
|
||||
|
||||
# Don't sync .un files
|
||||
*.un~
|
||||
dev-watch/**/*.pdf
|
||||
|
||||
@@ -1,19 +1,43 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides permanent, authoritative guidance to Claude Code for every session. All sections below reflect the actual codebase state and must be kept up-to-date as the project evolves.
|
||||
This file provides permanent, authoritative guidance to Claude Code for every session. It covers project-wide concerns only. Service-specific details live in sub-files — read them only when working in that service:
|
||||
|
||||
- `backend/CLAUDE.md` — auth/users/admin/settings/plugins endpoints; DB models; JWT/bcrypt/sanitization security; naming conventions
|
||||
- `frontend/CLAUDE.md` — routes, components, API client patterns, XSS prevention
|
||||
- `features/ai-service/CLAUDE.md` — /chat, /health, /queue endpoints; queue service
|
||||
- `features/doc-service/CLAUDE.md` — document/category/share endpoints; DB models; PDF limits; file watcher
|
||||
- `features/storage-service/CLAUDE.md` — storage API, pluggable backend drivers (local/S3/WebDAV), migration
|
||||
|
||||
---
|
||||
|
||||
## Merge checklist
|
||||
|
||||
Before merging any feature branch into `main`, every test relevant to the changed area in `tests/ALL_TESTS.md` (and the relevant service-specific file) must be marked passing. The test suite covers all 20 feature areas across five service files:
|
||||
|
||||
- `tests/backend_tests.md` — §1–9, §18
|
||||
- `tests/frontend_tests.md` — §19
|
||||
- `tests/doc-service_tests.md` — §10–16
|
||||
- `tests/ai-service_tests.md` — §17
|
||||
- `tests/storage-service_tests.md` — §20
|
||||
|
||||
Do not merge without it.
|
||||
|
||||
---
|
||||
|
||||
## CLAUDE.md self-update checkpoint
|
||||
|
||||
**After every change to the codebase**, before committing, check whether CLAUDE.md needs updating:
|
||||
**After every change to the codebase**, before committing, check which CLAUDE.md files need updating:
|
||||
|
||||
- New route added → update **All API Endpoints** and **Frontend Routes** tables
|
||||
- New DB model or column → update **Database Models**
|
||||
- New migration → update **Migration chains**
|
||||
- New file or directory → update **File & Folder Tree**
|
||||
- New limit or default value changed → update **Default Values & Limits**
|
||||
- New dependency, auth mechanism, or security pattern → update **Security Standards**
|
||||
- New Docker service, volume, network, or env var → update **Docker Infrastructure**
|
||||
- Stack version changed → update **Stack**
|
||||
- New route added → update **API Endpoints** in `backend/CLAUDE.md`, `features/doc-service/CLAUDE.md`, or `features/ai-service/CLAUDE.md`; update **Frontend Routes** in `frontend/CLAUDE.md`
|
||||
- New DB model or column → update **Database Models** in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
|
||||
- New migration → update **Migration chain** table in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
|
||||
- New file or directory → update **File & Folder Tree** in the relevant sub-file; update the high-level tree in this root file only if a top-level directory changes
|
||||
- New limit or default value changed → update **Default Values & Limits** in the relevant sub-file
|
||||
- New dependency, auth mechanism, or security pattern → update **Security Standards** in the relevant sub-file
|
||||
- New Docker service, volume, network, or env var → update **Docker Infrastructure** in this file
|
||||
- Stack version changed → update **Stack** in this file
|
||||
|
||||
- New feature or endpoint added → add test rows to **both** `tests/ALL_TESTS.md` (in the relevant section) **and** the matching service-specific file (`tests/backend_tests.md`, `tests/frontend_tests.md`, `tests/doc-service_tests.md`, `tests/ai-service_tests.md`, or `tests/storage-service_tests.md`). Use the same test number and format as existing rows.
|
||||
|
||||
This check is mandatory — treat it the same as updating STATUS.md.
|
||||
|
||||
@@ -36,22 +60,6 @@ This check is mandatory — treat it the same as updating STATUS.md.
|
||||
|
||||
All test, build, and package-manager commands run **inside Docker** — never on the host. See the memory note: "Testing inside Docker only".
|
||||
|
||||
### Migrations (run in Docker)
|
||||
|
||||
```bash
|
||||
docker compose exec backend alembic revision --autogenerate -m "describe change"
|
||||
docker compose exec backend alembic upgrade head
|
||||
docker compose exec backend alembic downgrade -1
|
||||
```
|
||||
|
||||
### Lint (run in Docker)
|
||||
|
||||
```bash
|
||||
docker compose exec backend ruff check . && ruff format .
|
||||
docker compose exec frontend npm run typecheck
|
||||
docker compose exec frontend npm run lint
|
||||
```
|
||||
|
||||
### Full stack
|
||||
|
||||
```bash
|
||||
@@ -63,13 +71,15 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
For service-specific commands (migrations, lint), see `backend/CLAUDE.md` and `frontend/CLAUDE.md`.
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
/
|
||||
├── CLAUDE.md ← This file — authoritative session context
|
||||
├── CLAUDE.md ← This file — project-wide context
|
||||
├── README.md ← Project overview, containers table, Current State
|
||||
├── TODO.md ← Task list
|
||||
├── .env.example ← Template for backend/.env
|
||||
@@ -78,103 +88,18 @@ docker compose up --build -d
|
||||
├── .githooks/pre-commit ← Runs scripts/security_check.py before every commit
|
||||
├── scripts/security_check.py ← Static analysis: secrets, weak crypto, SQLi, JWT
|
||||
├── changelog/YYYY-MM-DD_<slug>.md ← Per-date change logs
|
||||
├── tests/ALL_TESTS.md ← Full test suite (all 19 areas); must pass before merging to main
|
||||
├── tests/backend_tests.md ← Backend-only tests (§1–9, §18)
|
||||
├── tests/frontend_tests.md ← Frontend-only tests (§19)
|
||||
├── tests/doc-service_tests.md ← Doc-service tests (§10–16)
|
||||
├── tests/ai-service_tests.md ← AI-service tests (§17)
|
||||
├── dev-watch/ ← Dev bind-mount for file watcher testing (.gitkeep only)
|
||||
│
|
||||
├── backend/ ← FastAPI gateway (port 8000, internal)
|
||||
│ ├── app/
|
||||
│ │ ├── main.py ← App factory, router registration, lifespan (health loop)
|
||||
│ │ ├── database.py ← AsyncEngine, AsyncSessionLocal, Base
|
||||
│ │ ├── deps.py ← get_current_user, get_current_admin
|
||||
│ │ ├── core/
|
||||
│ │ │ ├── config.py ← All settings via pydantic-settings (reads .env)
|
||||
│ │ │ ├── security.py ← JWT sign/verify (RS256), bcrypt hash/verify
|
||||
│ │ │ ├── sanitize.py ← Input sanitization helpers (see Security Standards)
|
||||
│ │ │ └── app_config.py ← Per-service config load/save to /config volume
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── __init__.py ← Imports all models (required for Alembic autogenerate)
|
||||
│ │ │ ├── user.py ← User model (see Database Models)
|
||||
│ │ │ ├── profile.py ← Profile model
|
||||
│ │ │ └── group.py ← Group, GroupMembership models
|
||||
│ │ ├── schemas/
|
||||
│ │ │ ├── user.py ← UserCreate/Out, Token, DashboardPrefsOut/Update
|
||||
│ │ │ ├── profile.py ← ProfileRead, ProfileUpdate
|
||||
│ │ │ └── group.py ← GroupCreate/Update/Out/DetailOut, GroupMemberOut
|
||||
│ │ ├── routers/
|
||||
│ │ │ ├── auth.py ← POST /register, POST /login
|
||||
│ │ │ ├── users.py ← GET /me, GET+PATCH /me/preferences
|
||||
│ │ │ ├── profile.py ← GET+PUT /me (profile)
|
||||
│ │ │ ├── admin.py ← User admin CRUD (admin-only)
|
||||
│ │ │ ├── groups.py ← Group CRUD + member management (admin-only)
|
||||
│ │ │ ├── settings.py ← AI, doc limits, system prompts (admin-only)
|
||||
│ │ │ ├── services.py ← GET /services (health status)
|
||||
│ │ │ ├── categories_proxy.py ← Transparent proxy → doc-service /categories/*
|
||||
│ │ │ └── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ └── services/
|
||||
│ │ └── service_health.py ← Background 30s health-check loop
|
||||
│ ├── alembic/
|
||||
│ │ ├── env.py ← Async migration runner
|
||||
│ │ └── versions/ ← Migration chain (see Migrations section)
|
||||
│ ├── scripts/seed.py ← Seed test user
|
||||
│ ├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
│ └── STATUS.md
|
||||
│
|
||||
├── backend/ ← FastAPI gateway (port 8000, internal); see backend/CLAUDE.md
|
||||
├── features/
|
||||
│ ├── ai-service/ ← AI provider intermediary (port 8010, internal)
|
||||
│ │ ├── app/
|
||||
│ │ │ ├── main.py ← FastAPI, queue worker lifespan
|
||||
│ │ │ ├── routers/chat.py ← POST /chat (sync, NORMAL priority queue)
|
||||
│ │ │ ├── routers/health.py ← GET /health
|
||||
│ │ │ ├── routers/queue.py ← GET /queue/status, /pause, /resume, /cancel/{id}
|
||||
│ │ │ ├── providers/base.py ← AIProvider abstract class
|
||||
│ │ │ ├── providers/anthropic_provider.py
|
||||
│ │ │ ├── providers/openai_compat.py ← Ollama / LM Studio
|
||||
│ │ │ └── services/queue.py ← Priority queue (CRITICAL > HIGH > NORMAL)
|
||||
│ │ ├── Dockerfile
|
||||
│ │ └── STATUS.md
|
||||
│ │
|
||||
│ └── doc-service/ ← PDF extraction microservice (port 8001, internal)
|
||||
│ ├── app/
|
||||
│ │ ├── main.py
|
||||
│ │ ├── database.py ← Same PostgreSQL instance as backend
|
||||
│ │ ├── deps.py ← get_user_id (reads x-user-id header)
|
||||
│ │ ├── models/
|
||||
│ │ │ ├── document.py ← Document model (see Database Models)
|
||||
│ │ │ ├── category.py ← DocumentCategory model
|
||||
│ │ │ └── category_assignment.py ← CategoryAssignment (composite PK)
|
||||
│ │ ├── schemas/
|
||||
│ │ │ ├── document.py ← DocumentOut, DocumentPage, DocumentStatusOut, etc.
|
||||
│ │ │ └── category.py ← CategoryOut, CategoryCreate, CategoryUpdate
|
||||
│ │ ├── routers/
|
||||
│ │ │ ├── documents.py ← Full document CRUD + file serving + reprocess
|
||||
│ │ │ └── categories.py ← Category CRUD
|
||||
│ │ └── services/
|
||||
│ │ ├── storage.py ← File I/O
|
||||
│ │ ├── ai_client.py ← classify_document() → ai-service:8010/chat
|
||||
│ │ └── config_reader.py
|
||||
│ ├── alembic/versions/ ← Doc-service migration chain
|
||||
│ ├── Dockerfile
|
||||
│ └── STATUS.md
|
||||
│
|
||||
└── frontend/ ← React SPA (port 5173 dev / 80 prod)
|
||||
├── src/
|
||||
│ ├── main.tsx ← React root, QueryClientProvider, BrowserRouter
|
||||
│ ├── App.tsx ← Route tree, PrivateRoute, AdminRoute
|
||||
│ ├── api/client.ts ← Axios instance + ALL API functions (single source of truth)
|
||||
│ ├── hooks/
|
||||
│ │ ├── useAuth.ts ← Token state (localStorage), login/logout
|
||||
│ │ └── useTheme.ts ← Theme toggle
|
||||
│ ├── components/
|
||||
│ │ ├── AppShell.tsx ← Layout: Sidebar + scrollable main
|
||||
│ │ ├── Sidebar.tsx ← Collapsible nav (icons ↔ icons+labels)
|
||||
│ │ ├── ThemeToggle.tsx ← Light/dark mode toggle
|
||||
│ │ └── ui/ ← shadcn/ui components (Button, Input, …)
|
||||
│ ├── pages/ ← One file per route (see Routes section)
|
||||
│ ├── lib/utils.ts ← cn() = clsx + tailwind-merge
|
||||
│ └── styles/theme.css ← CSS custom properties, Tailwind setup
|
||||
├── vite.config.ts ← /api/* proxied to backend:8000
|
||||
├── tailwind.config.ts
|
||||
├── components.json ← shadcn/ui config
|
||||
├── Dockerfile ← Multi-stage: Node build → nginx-unprivileged
|
||||
└── STATUS.md
|
||||
│ ├── ai-service/ ← AI provider intermediary (port 8010, internal); see features/ai-service/CLAUDE.md
|
||||
│ └── doc-service/ ← PDF extraction microservice (port 8001, internal); see features/doc-service/CLAUDE.md
|
||||
└── frontend/ ← React SPA (port 5173 dev / 80 prod); see frontend/CLAUDE.md
|
||||
```
|
||||
|
||||
---
|
||||
@@ -195,8 +120,8 @@ Browser (:5173 dev / :80 prod)
|
||||
/users /groups /documents/categories/*
|
||||
/profile /settings
|
||||
/services │ │
|
||||
JSON volume proxy (injects x-user-id)
|
||||
(/config) │
|
||||
JSON volume proxy (injects x-user-id,
|
||||
(/config) x-user-groups) │
|
||||
doc-service:8001
|
||||
│
|
||||
ai-service:8010
|
||||
@@ -212,303 +137,25 @@ Browser (:5173 dev / :80 prod)
|
||||
|
||||
---
|
||||
|
||||
## Database Models
|
||||
|
||||
### Backend (`users`, `profiles`, `groups`, `group_memberships`)
|
||||
|
||||
**`users`**
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `email` | String | UNIQUE, indexed, NOT NULL | lowercased before storing |
|
||||
| `hashed_password` | String | NOT NULL | bcrypt 13 rounds |
|
||||
| `full_name` | String | nullable | sanitized max 128 chars |
|
||||
| `is_active` | Boolean | default=True | soft-delete flag |
|
||||
| `is_superuser` | Boolean | default=False | admin role; never exposed as-is (serialised as `is_admin`) |
|
||||
| `dashboard_app_ids` | JSON | NOT NULL, default=[] | list of pinned service IDs |
|
||||
|
||||
Relationship: `profile` (one-to-one, cascade all+delete-orphan)
|
||||
|
||||
**`profiles`**
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `user_id` | String | FK→users.id UNIQUE, cascade delete | one-to-one |
|
||||
| `phone` | String(20) | nullable | validated format |
|
||||
| `date_of_birth` | Date | nullable | 1900+ and not future |
|
||||
| `position` | String(128) | nullable | job title |
|
||||
| `address` | String(255) | nullable | |
|
||||
| `updated_at` | DateTime(tz) | server_default=now(), onupdate=now() | |
|
||||
|
||||
**`groups`**
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `name` | String(128) | UNIQUE indexed, NOT NULL |
|
||||
| `description` | String(512) | nullable |
|
||||
| `created_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
**`group_memberships`**
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `group_id` | String | FK→groups.id, indexed, CASCADE |
|
||||
| `user_id` | String | FK→users.id, indexed, CASCADE |
|
||||
| `joined_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
Unique constraint: `(group_id, user_id)`
|
||||
|
||||
### Doc-service (`documents`, `document_categories`, `document_category_assignments`)
|
||||
|
||||
**`documents`**
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | |
|
||||
| `user_id` | String | indexed | not FK — trusts x-user-id header |
|
||||
| `filename` | String | NOT NULL | |
|
||||
| `file_path` | String | NOT NULL | absolute path under /data/documents |
|
||||
| `file_size` | Integer | NOT NULL | bytes |
|
||||
| `status` | String | default="pending" | pending / processing / done / failed |
|
||||
| `title` | String(500) | nullable | AI-extracted |
|
||||
| `document_type` | String | nullable | invoice / bill / receipt / order / expense / revenue / unknown |
|
||||
| `raw_text` | Text | nullable | first 500 k chars |
|
||||
| `extracted_data` | Text | nullable | JSON string |
|
||||
| `tags` | Text | nullable | JSON array string |
|
||||
| `error_message` | String(500) | nullable | |
|
||||
| `created_at` | DateTime(tz) | server_default=now() | |
|
||||
| `processed_at` | DateTime(tz) | nullable | |
|
||||
|
||||
**`document_categories`**
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `user_id` | String | indexed |
|
||||
| `name` | String(128) | NOT NULL |
|
||||
| `created_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
**`document_category_assignments`** (composite PK)
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `document_id` | String | PK + FK→documents.id CASCADE |
|
||||
| `category_id` | String | PK + FK→document_categories.id CASCADE |
|
||||
|
||||
### Migration chains
|
||||
|
||||
**Backend** (must be applied in order):
|
||||
|
||||
| Rev ID | Slug |
|
||||
|--------|------|
|
||||
| `38efeff7c45a` | `create_users_table` |
|
||||
| `676084df61d1` | `add_profiles_table` |
|
||||
| `a3f9c2d14e87` | `add_groups_and_group_memberships` |
|
||||
| `c7e8f9a0b1d2` | `add_dashboard_app_ids_to_users` |
|
||||
|
||||
**Doc-service**:
|
||||
|
||||
| Rev ID | Slug |
|
||||
|--------|------|
|
||||
| `0001` | `create_doc_tables` |
|
||||
| `0002` | `add_document_title` |
|
||||
|
||||
---
|
||||
|
||||
## All API Endpoints
|
||||
|
||||
### Auth (`/api/auth`) — public
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/auth/register` | — | Create account; returns `UserOut`; enforces password policy |
|
||||
| POST | `/api/auth/login` | — | OAuth2 password flow; returns `{access_token, token_type}` |
|
||||
|
||||
### Users (`/api/users`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/users/me` | user | Current user info → `UserOut` |
|
||||
| GET | `/api/users/me/preferences` | user | Dashboard pinned app IDs → `{app_ids}` |
|
||||
| PATCH | `/api/users/me/preferences` | user | Save pinned app IDs (max 50, slug-safe) |
|
||||
|
||||
### Profile (`/api/profile`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/profile/me` | user | Fetch profile; auto-creates if missing |
|
||||
| PUT | `/api/profile/me` | user | Update profile fields |
|
||||
|
||||
### Admin — Users (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/users` | List all users → `list[UserAdminOut]` |
|
||||
| POST | `/api/admin/users` | Create user (with optional is_admin) |
|
||||
| DELETE | `/api/admin/users/{user_id}` | Delete user (204) |
|
||||
| PATCH | `/api/admin/users/{user_id}/active` | Toggle active status |
|
||||
|
||||
### Admin — Groups (`/api/admin/groups`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/groups` | List groups with member count |
|
||||
| POST | `/api/admin/groups` | Create group |
|
||||
| GET | `/api/admin/groups/{id}` | Group detail + members |
|
||||
| PATCH | `/api/admin/groups/{id}` | Update name / description |
|
||||
| DELETE | `/api/admin/groups/{id}` | Delete (cascades memberships) |
|
||||
| POST | `/api/admin/groups/{id}/members/{user_id}` | Add member |
|
||||
| DELETE | `/api/admin/groups/{id}/members/{user_id}` | Remove member |
|
||||
|
||||
### Settings (`/api/settings`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/settings/ai` | AI config (keys masked) |
|
||||
| PATCH | `/api/settings/ai` | Update AI provider / credentials |
|
||||
| POST | `/api/settings/ai/test` | Test AI connection |
|
||||
| GET | `/api/settings/documents/limits` | PDF upload limits |
|
||||
| PATCH | `/api/settings/documents/limits` | Update max PDF size |
|
||||
| GET | `/api/settings/system-prompts` | All editable system prompts |
|
||||
| PATCH | `/api/settings/system-prompts/{service_id}` | Update system prompt |
|
||||
|
||||
### Services (`/api/services`) — authenticated
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/services` | Health status of all registered services → `list[ServiceStatus]` |
|
||||
|
||||
### Documents (`/api/documents/*`) — authenticated, proxied to doc-service
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/api/documents/upload` | Upload PDF (202, background processing) |
|
||||
| GET | `/api/documents` | Paginated list (filterable: search, status, type, category, sort) |
|
||||
| GET | `/api/documents/{id}` | Document detail |
|
||||
| GET | `/api/documents/{id}/status` | Processing status only |
|
||||
| PATCH | `/api/documents/{id}/type` | Update document type |
|
||||
| PATCH | `/api/documents/{id}/tags` | Update tags |
|
||||
| PATCH | `/api/documents/{id}/title` | Update title |
|
||||
| POST | `/api/documents/{id}/reprocess` | Re-run AI extraction |
|
||||
| DELETE | `/api/documents/{id}` | Delete document (204) |
|
||||
| GET | `/api/documents/{id}/file` | Download PDF (streaming) |
|
||||
| POST | `/api/documents/{id}/categories/{cat_id}` | Assign category |
|
||||
| DELETE | `/api/documents/{id}/categories/{cat_id}` | Remove category |
|
||||
|
||||
### Categories (`/api/documents/categories/*`) — authenticated, proxied to doc-service
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/documents/categories` | List user's categories |
|
||||
| POST | `/api/documents/categories` | Create category (triggers background AI reanalysis) |
|
||||
| PATCH | `/api/documents/categories/{id}` | Rename |
|
||||
| DELETE | `/api/documents/categories/{id}` | Delete (204) |
|
||||
|
||||
### AI-service (internal only — not exposed to browser)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/chat` | Chat request (queued at NORMAL priority) |
|
||||
| GET | `/health` | Health check |
|
||||
| GET | `/queue/status` | Queue state |
|
||||
| POST | `/queue/pause` | Pause queue |
|
||||
| POST | `/queue/resume` | Resume queue |
|
||||
| POST | `/queue/cancel/{job_id}` | Cancel job |
|
||||
|
||||
---
|
||||
|
||||
## Frontend Routes
|
||||
|
||||
| Path | Component | Guard |
|
||||
|------|-----------|-------|
|
||||
| `/login` | `LoginPage` | Public |
|
||||
| `/` | `DashboardPage` | PrivateRoute |
|
||||
| `/apps` | `AppsPage` | PrivateRoute |
|
||||
| `/apps/documents` | `DocumentsPage` | PrivateRoute |
|
||||
| `/apps/documents/settings/admin` | `DocumentAdminSettingsPage` | AdminRoute |
|
||||
| `/apps/ai/settings/admin` | `AIAdminSettingsPage` | AdminRoute |
|
||||
| `/profile` | `ProfilePage` | PrivateRoute |
|
||||
| `/settings` | `SettingsPage` | PrivateRoute |
|
||||
| `/admin` | `AdminPage` (→ `/admin/users`) | AdminRoute |
|
||||
| `/admin/users` | `AdminUsersPage` | AdminRoute |
|
||||
| `/admin/groups` | `AdminGroupsPage` | AdminRoute |
|
||||
| `*` | redirect to `/` | — |
|
||||
|
||||
`PrivateRoute` — checks `token` from `useAuth`, redirects to `/login` if absent.
|
||||
`AdminRoute` — checks token AND queries `GET /api/users/me` for `is_admin`; waits for query to avoid flash; redirects to `/login` (not `/`) if not admin.
|
||||
|
||||
---
|
||||
|
||||
## Security Standards
|
||||
|
||||
These standards are **non-negotiable**. Every change must comply.
|
||||
|
||||
### JWT
|
||||
|
||||
- **Algorithm**: RS256 (4096-bit RSA key pair, generated by `scripts/generate_jwt_keys.py`)
|
||||
- **Keys**: PEM-encoded in `backend/.env` as `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` (gitignored)
|
||||
- **Expiry**: 8 hours (`EXPIRE_MINUTES=480`) — never set longer; no refresh tokens
|
||||
- **Claims**: `{sub: user_id, exp, iat}` — user_id is a UUID string
|
||||
- **Validation**: `decode_access_token()` in `core/security.py`; called by `get_current_user`
|
||||
- **Never**: set algorithm to `"none"`, disable `verify_exp`, or hardcode secrets in code
|
||||
|
||||
### Password hashing
|
||||
|
||||
- **Algorithm**: bcrypt, **13 rounds** (`bcrypt.gensalt(rounds=13)`)
|
||||
- **Timing**: ~300 ms per hash (intentional brute-force resistance)
|
||||
- **Never** use MD5, SHA1, or plain SHA256 for password storage
|
||||
|
||||
### Password policy (enforced in `UserCreate` schema)
|
||||
|
||||
All of the following must pass:
|
||||
- ≥ 8 characters
|
||||
- ≥ 1 uppercase (A–Z)
|
||||
- ≥ 1 lowercase (a–z)
|
||||
- ≥ 1 digit (0–9)
|
||||
- ≥ 1 special character: `!@#$%^&*()\-_=+[]{}|;:'"<>?/\`~`
|
||||
- No common words (password, secret, login, admin, test, qwerty, welcome, …)
|
||||
|
||||
### Input sanitization
|
||||
|
||||
Every user-supplied string stored in the database **must** pass through `core/sanitize.py`:
|
||||
|
||||
```python
|
||||
sanitize_str(value, max_len=255)
|
||||
# → strips whitespace; rejects null bytes (\x00); rejects control chars
|
||||
# (0x01–0x1F, 0x7F except \t \n \r); enforces max_len; returns None for ""
|
||||
|
||||
normalize_email(value) # lowercase + strip
|
||||
validate_phone(value) # sanitize_str(max=20) + regex ^\+?[\d\s\-()\[\]]{7,20}$
|
||||
validate_date_of_birth(v) # must be ≥ 1900, not future
|
||||
```
|
||||
|
||||
Apply via Pydantic `@field_validator` on all request schemas.
|
||||
|
||||
### XSS prevention
|
||||
|
||||
- React JSX text interpolation (`{value}`) is HTML-escaped by the DOM renderer — **never** use `dangerouslySetInnerHTML` with user-supplied content.
|
||||
- Server-side `sanitize_str` provides defense-in-depth (control char stripping, max length).
|
||||
|
||||
### SQL injection prevention
|
||||
|
||||
- Use SQLAlchemy ORM (bound parameters) — **never** raw SQL strings.
|
||||
- If `text()` is needed, use `bindparam()` for all user-supplied values.
|
||||
- **Never** use f-strings, `.format()`, or `%`-formatting for SQL.
|
||||
|
||||
### Admin route security
|
||||
|
||||
- Use `get_current_admin` dependency (checks `is_superuser`).
|
||||
- Return **404** (not 403) for unauthorized access — hides both endpoint existence and permission model.
|
||||
These standards are **non-negotiable**. Every change must comply. Implementation-specific security rules (JWT, bcrypt, input sanitization, XSS, SQLi, admin routes) are in the relevant sub-CLAUDE.md files.
|
||||
|
||||
### Network isolation
|
||||
|
||||
- `backend-net`: all containers except frontend; not reachable from host in prod.
|
||||
- `frontend-net`: only frontend; single host port (80 prod / 5173 dev).
|
||||
- DB, backend, doc-service, ai-service have **no** host port bindings in prod.
|
||||
- DB, backend, doc-service, ai-service, storage-service have **no** host port bindings in prod.
|
||||
|
||||
### Storage rule (non-negotiable)
|
||||
|
||||
**No service may write to a filesystem path for persistent data.** All file/blob storage must go through the storage-service HTTP API (`PUT/GET/DELETE /objects/{bucket}/{key}`). Config JSON files must be stored in the `config` bucket. Uploaded files must be stored in the `documents` bucket. Violation is a security and architecture defect.
|
||||
|
||||
The only two persistent storage mechanisms in the project are:
|
||||
1. **PostgreSQL** — structured/relational data
|
||||
2. **storage-service** — all file/blob/config data (local filesystem by default; switchable to S3-compatible or WebDAV)
|
||||
|
||||
New services and features must follow this pattern. See `features/storage-service/CLAUDE.md` for the API reference.
|
||||
|
||||
### Pre-commit security hook
|
||||
|
||||
@@ -526,165 +173,14 @@ Apply via Pydantic `@field_validator` on all request schemas.
|
||||
|
||||
---
|
||||
|
||||
## Frontend Patterns & Conventions
|
||||
|
||||
### API client (`src/api/client.ts`)
|
||||
|
||||
Single Axios instance — **all** API calls live here, nowhere else:
|
||||
|
||||
```typescript
|
||||
const api = axios.create({ baseURL: "/api" });
|
||||
api.interceptors.request.use((config) => {
|
||||
const token = localStorage.getItem("token");
|
||||
if (token) config.headers.Authorization = `Bearer ${token}`;
|
||||
return config;
|
||||
});
|
||||
```
|
||||
|
||||
Adding a new API call:
|
||||
1. Define a TypeScript interface for the response if it's new.
|
||||
2. Add a named export function (`getX`, `createX`, `updateX`, `deleteX`).
|
||||
3. Use `api.get<T>(...)`, `api.post<T>(...)`, etc.; always `.then((r) => r.data)`.
|
||||
|
||||
### TanStack Query conventions
|
||||
|
||||
**Query keys** (flat arrays, lowercase):
|
||||
```typescript
|
||||
["me"] // current user
|
||||
["services"] // service health list
|
||||
["dashboard-prefs"] // user dashboard preferences
|
||||
["categories"] // document categories
|
||||
["documents", params] // document list (params object for cache isolation)
|
||||
["document", id] // single document
|
||||
```
|
||||
|
||||
**Mutation pattern**:
|
||||
```typescript
|
||||
const mutation = useMutation({
|
||||
mutationFn: apiFunction,
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["affected-key"] });
|
||||
// additional side effects (close dialog, reset form, etc.)
|
||||
},
|
||||
});
|
||||
// Usage:
|
||||
mutation.mutate(data);
|
||||
mutation.isPending // show spinner / disable button
|
||||
mutation.isError // show error message
|
||||
```
|
||||
|
||||
**Polling**:
|
||||
```typescript
|
||||
useQuery({ queryKey: ["services"], queryFn: getServices,
|
||||
refetchInterval: 30_000, refetchIntervalInBackground: true });
|
||||
```
|
||||
|
||||
### Route guards
|
||||
|
||||
```typescript
|
||||
// PrivateRoute — redirect to /login if no token
|
||||
// AdminRoute — redirect to /login if no token OR not admin
|
||||
// (waits for getMe() query to avoid flash; uses 404 semantics)
|
||||
```
|
||||
|
||||
### Component patterns
|
||||
|
||||
- Functional components only.
|
||||
- Local `useState` for UI-only state (edit mode, pending values, open/closed).
|
||||
- Server state via `useQuery` / `useMutation` — no duplicated local copies.
|
||||
- `cn()` from `lib/utils.ts` for conditional Tailwind classes.
|
||||
- `lucide-react` for all icons.
|
||||
- Never use `dangerouslySetInnerHTML` with user-supplied content.
|
||||
|
||||
---
|
||||
|
||||
## Naming & Code Conventions
|
||||
|
||||
### Database
|
||||
|
||||
- **Tables**: lowercase, plural, snake_case (`users`, `group_memberships`, `document_category_assignments`)
|
||||
- **Columns**: lowercase, snake_case
|
||||
- **ORM models**: PascalCase, singular (`User`, `Group`, `GroupMembership`, `Document`)
|
||||
- Primary keys: `id` (String UUID, auto-generated)
|
||||
- Timestamps: `created_at` / `updated_at` / `joined_at` / `processed_at` — always timezone-aware
|
||||
|
||||
### Pydantic schemas
|
||||
|
||||
| Suffix | Purpose |
|
||||
|--------|---------|
|
||||
| `Create` | POST request body (user-supplied input) |
|
||||
| `Update` | PATCH request body (partial update) |
|
||||
| `Out` | API response (safe subset of model) |
|
||||
| `AdminOut` | Extended response for admin endpoints |
|
||||
| `Read` | GET response (same as `Out`, used for profiles) |
|
||||
|
||||
Always set `model_config = {"from_attributes": True}` on response schemas.
|
||||
Use `validation_alias` when the ORM field name differs from the JSON key (e.g., `is_superuser` → `is_admin`).
|
||||
|
||||
### HTTP status codes
|
||||
|
||||
| Code | Use |
|
||||
|------|-----|
|
||||
| 200 | Successful GET / PATCH / PUT |
|
||||
| 201 | Successful POST that creates a resource |
|
||||
| 202 | Accepted (async processing started, e.g., document upload) |
|
||||
| 204 | Successful DELETE or action with no response body |
|
||||
| 400 | Bad request (duplicates, invalid data beyond Pydantic) |
|
||||
| 401 | Missing / invalid JWT |
|
||||
| 404 | Not found **and** admin routes when not admin |
|
||||
| 413 | Payload too large (file exceeds limit) |
|
||||
| 415 | Unsupported media type (not a PDF) |
|
||||
| 422 | Pydantic validation failure (FastAPI default) |
|
||||
| 502 | Downstream service unreachable |
|
||||
| 503 | Service unavailable (queue stopped, AI error) |
|
||||
| 504 | Gateway timeout |
|
||||
|
||||
### Backend code style
|
||||
|
||||
- Async/await for **all** I/O (DB, HTTP, file).
|
||||
- `raise HTTPException(status_code=..., detail="...")` for all errors.
|
||||
- Response models always declared in route decorator: `@router.get("/path", response_model=XOut)`.
|
||||
- Background tasks via `BackgroundTasks` param; tasks open their own `AsyncSessionLocal` session.
|
||||
- Commit + refresh pattern after mutations:
|
||||
```python
|
||||
await db.commit()
|
||||
await db.refresh(obj)
|
||||
```
|
||||
|
||||
### Frontend code style
|
||||
|
||||
- TypeScript strict mode — no `any`.
|
||||
- API response types inferred from interfaces in `client.ts` only.
|
||||
- Error messages displayed inline (no alert); loading shown as disabled state or "…" text.
|
||||
- All user-facing text: safe via React JSX rendering (not innerHTML).
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
## Default Values & Limits (cross-cutting)
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| JWT expiry | 480 min (8 h) | `core/security.py` |
|
||||
| Bcrypt rounds | 13 | `core/security.py` |
|
||||
| Token localStorage key | `"token"` | `useAuth.ts` |
|
||||
| Health check interval | 30 s | `service_health.py` |
|
||||
| Service poll (frontend) | 30 s | `AppsPage.tsx`, `DashboardPage.tsx` |
|
||||
| Max dashboard pinned apps | 50 | `schemas/user.py` |
|
||||
| App ID max length | 64 chars | `schemas/user.py` |
|
||||
| App ID allowed chars | `[a-zA-Z0-9_\-]` | `schemas/user.py` |
|
||||
| full_name max length | 128 chars | `schemas/user.py` |
|
||||
| Group name max length | 128 chars | `schemas/group.py` |
|
||||
| Group description max | 512 chars | `schemas/group.py` |
|
||||
| Phone max length | 20 chars | `sanitize.py` |
|
||||
| Position max length | 128 chars | `schemas/profile.py` |
|
||||
| Address max length | 255 chars | `schemas/profile.py` |
|
||||
| Document title max | 500 chars | `models/document.py` |
|
||||
| Category name max | 128 chars | `models/category.py` |
|
||||
| PDF max size (default) | 20 MB | admin settings (configurable) |
|
||||
| Raw text cap | 500 k chars | `doc-service` AI client |
|
||||
| Documents per_page | 1–100, default 20 | `routers/documents.py` |
|
||||
| AI service timeout | 60 s | `ai_client.py` |
|
||||
| AI service max retries | 2 | `ai_client.py` |
|
||||
|
||||
All other per-service defaults are in the relevant sub-CLAUDE.md file.
|
||||
|
||||
---
|
||||
|
||||
@@ -695,9 +191,10 @@ Use `validation_alias` when the ORM field name differs from the JSON key (e.g.,
|
||||
| Service | Image base | Internal port | User | Volumes | Network |
|
||||
|---------|-----------|---------------|------|---------|---------|
|
||||
| `db` | postgres:16-alpine | 5432 | 70:70 | `postgres_data` | backend-net |
|
||||
| `backend` | python:3.12-slim | 8000 | 1001:1001 | `app_config` | backend-net |
|
||||
| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | `app_config` | backend-net |
|
||||
| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `doc_data`, `app_config` | backend-net |
|
||||
| `backend` | python:3.12-slim | 8000 | 1001:1001 | — | backend-net |
|
||||
| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | — | backend-net |
|
||||
| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `watch_data` | backend-net |
|
||||
| `storage-service` | python:3.12-slim | 8020 | 1001:1001 | `storage_data` | backend-net |
|
||||
| `frontend` | nginx-unprivileged:alpine | 8080 | 1001:1001 | — | backend-net, frontend-net |
|
||||
|
||||
### Volumes
|
||||
@@ -705,14 +202,14 @@ Use `validation_alias` when the ORM field name differs from the JSON key (e.g.,
|
||||
| Volume | Mount path | Contains |
|
||||
|--------|-----------|---------|
|
||||
| `postgres_data` | `/var/lib/postgresql/data` | PostgreSQL data |
|
||||
| `doc_data` | `/data/documents` | Uploaded PDF files |
|
||||
| `app_config` | `/config` | Per-service runtime config JSON files |
|
||||
| `storage_data` | `/data/storage` | All file/blob storage: PDFs (`documents/`) and config JSONs (`config/`) |
|
||||
| `watch_data` | `/data/watch` | Watch directory (bind-mount NAS/Nextcloud via docker-compose.override.yml) |
|
||||
|
||||
### Networks
|
||||
|
||||
| Network | Host-accessible | Members |
|
||||
|---------|----------------|---------|
|
||||
| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, frontend |
|
||||
| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, storage-service, frontend |
|
||||
| `frontend-net` | Yes (port 80 → frontend:8080) | frontend |
|
||||
|
||||
### Environment variables (required in `backend/.env`)
|
||||
@@ -728,6 +225,7 @@ Injected by docker-compose (not in `.env`):
|
||||
```
|
||||
DOC_SERVICE_URL=http://doc-service:8001
|
||||
AI_SERVICE_URL=http://ai-service:8010
|
||||
STORAGE_SERVICE_URL=http://storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
@@ -794,8 +292,9 @@ Each entry must include:
|
||||
4. Add router in `backend/app/routers/`, mount it in `main.py`
|
||||
5. Add API function(s) to `frontend/src/api/client.ts`
|
||||
6. Add page component in `frontend/src/pages/`, register route in `App.tsx`
|
||||
7. Update `STATUS.md` for affected services
|
||||
8. Add changelog entry
|
||||
7. If the resource involves file or blob data: store it via `PUT /objects/{bucket}/{key}` on `storage-service:8020`. Never write to the local filesystem. See `features/storage-service/CLAUDE.md` for the API.
|
||||
8. Update `STATUS.md` for affected services
|
||||
9. Add changelog entry
|
||||
|
||||
---
|
||||
|
||||
@@ -805,6 +304,111 @@ Always run `git push` immediately after every `git commit`.
|
||||
|
||||
---
|
||||
|
||||
### Feature branch & isolated test environment
|
||||
|
||||
Every non-trivial implementation (anything beyond a one-line fix or doc change) **must** follow this workflow:
|
||||
|
||||
#### 0 — Mandatory planning phase (REQUIRED before any code changes)
|
||||
|
||||
Before touching any code, present a written plan and **wait for explicit user approval**. Do not open files to edit, do not create branches, do not write code until the user says the plan is approved.
|
||||
|
||||
The plan must include:
|
||||
- **What** is changing and **why**
|
||||
- **Which files** will be created or modified (with paths)
|
||||
- **Database / migration impact** (if any)
|
||||
- **API contract changes** (new endpoints, changed schemas)
|
||||
- **Frontend route / component changes**
|
||||
- **Risks or non-obvious decisions**
|
||||
|
||||
Only proceed to step 1 after the user responds with explicit approval (e.g. "looks good", "go ahead", "approved").
|
||||
|
||||
#### 1 — Create a feature branch
|
||||
After the planning phase is approved, branch off `main`. Name the branch after the title of the change — use lowercase words separated by hyphens, descriptive enough to understand at a glance what the branch does:
|
||||
```bash
|
||||
git checkout main && git pull
|
||||
git checkout -b feat/<descriptive-title> # e.g. feat/user-profile-avatar-upload, feat/document-bulk-delete
|
||||
```
|
||||
|
||||
#### 2 — Spin up an isolated Docker stack for the feature
|
||||
The feature stack always uses port `5173` (same as the main dev stack). Stop the main stack before starting a feature stack, and restart it when done.
|
||||
|
||||
**Stop the main dev stack first:**
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml down
|
||||
```
|
||||
|
||||
**Create a per-feature override file** at `docker-compose.feat-<slug>.yml` (gitignored):
|
||||
```yaml
|
||||
# docker-compose.feat-<slug>.yml — feature test stack, never committed to main
|
||||
services:
|
||||
frontend:
|
||||
container_name: frontend-<slug>
|
||||
backend:
|
||||
container_name: backend-<slug>
|
||||
doc-service:
|
||||
container_name: doc-service-<slug>
|
||||
ai-service:
|
||||
container_name: ai-service-<slug>
|
||||
db:
|
||||
container_name: db-<slug>
|
||||
|
||||
networks:
|
||||
backend-net:
|
||||
name: backend-net-<slug>
|
||||
frontend-net:
|
||||
name: frontend-net-<slug>
|
||||
```
|
||||
|
||||
**Start the feature stack**:
|
||||
```bash
|
||||
docker compose -f docker-compose.yml \
|
||||
-f docker-compose.dev.yml \
|
||||
-f docker-compose.feat-<slug>.yml \
|
||||
--project-name <slug> up --build
|
||||
```
|
||||
|
||||
The feature frontend is now reachable at `http://localhost:5173`.
|
||||
|
||||
#### 3 — Develop on the feature branch
|
||||
All code changes happen on `feat/<slug>`. Commit and push normally:
|
||||
```bash
|
||||
git add <files>
|
||||
git commit -m "feat: <description>"
|
||||
git push -u origin feat/<slug>
|
||||
```
|
||||
|
||||
#### 4 — Confirm functionality
|
||||
Before merging, verify all of the following on `http://localhost:5173`:
|
||||
- [ ] Login and registration work end-to-end
|
||||
- [ ] The specific feature works as intended
|
||||
- [ ] No regressions visible in the UI
|
||||
- [ ] Backend logs show no unexpected errors: `docker compose -p <slug> logs backend`
|
||||
- [ ] Migrations (if any) applied cleanly: `docker compose -p <slug> exec backend alembic upgrade head`
|
||||
|
||||
#### 5 — Merge to main
|
||||
Once all checks pass:
|
||||
```bash
|
||||
git checkout main
|
||||
git merge --no-ff feat/<slug> -m "Merge feat/<slug>: <description>"
|
||||
git push
|
||||
git branch -d feat/<slug>
|
||||
git push origin --delete feat/<slug>
|
||||
```
|
||||
|
||||
#### 6 — Tear down the feature stack and restart main dev stack
|
||||
```bash
|
||||
docker compose -f docker-compose.yml \
|
||||
-f docker-compose.dev.yml \
|
||||
-f docker-compose.feat-<slug>.yml \
|
||||
--project-name <slug> down --volumes --remove-orphans
|
||||
rm docker-compose.feat-<slug>.yml
|
||||
|
||||
# Restart the main dev stack on :5173
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Infrastructure change protocol
|
||||
|
||||
After **any** change to Dockerfiles, `docker-compose*.yml`, `nginx.conf`, or setup scripts:
|
||||
|
||||
@@ -20,10 +20,11 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
- All input sanitized before reaching the DB (null-byte rejection, length caps, format validation)
|
||||
- **PDF Documents app** (`/apps/documents`): upload PDFs, async text extraction (pdfplumber), AI classification via ai-service, per-user categories, file download
|
||||
- **AI Service** (`ai-service:8010`): shared AI intermediary container; routes prompts to Anthropic / Ollama / LM Studio; stateless; all feature containers talk to it via `POST /chat`
|
||||
- Admin settings: `/apps/ai/settings/admin` (provider, credentials, test connection); `/apps/documents/settings/admin` (upload limits only)
|
||||
- Config stored in shared Docker volume: `/config/ai_service_config.json` and `/config/doc_service_config.json`
|
||||
- **Storage Service** (`storage-service:8020`): unified file/blob storage with pluggable backends (local filesystem default; S3-compatible and WebDAV built in); backend switchable via admin UI with zero-data-loss migration
|
||||
- Admin settings: AI provider, doc upload limits, storage backend switching with live migration progress
|
||||
- Config stored in storage-service (`config` bucket); PDFs stored in storage-service (`documents` bucket) — no shared filesystem volumes
|
||||
- `/apps` launcher hub — one card per installed app with Open + Settings links
|
||||
- 5 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `frontend`
|
||||
- 6 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `storage-service`, `frontend`
|
||||
- All containers run as non-root users (UID 1001 for app containers, UID 70 for db)
|
||||
- Network-isolated: only the frontend exposes a host port (80/5173); all backend services are unreachable from outside Docker
|
||||
- Dev environment seeds a test user automatically on startup (`test@example.com` / `Test123!`)
|
||||
@@ -38,6 +39,7 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
| `backend` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | FastAPI management API + proxy to doc-service |
|
||||
| `ai-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Shared AI intermediary (routes to LM Studio / Ollama / Anthropic) |
|
||||
| `doc-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | PDF extraction microservice (calls ai-service) |
|
||||
| `storage-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Unified file/blob storage (local / S3-compatible / WebDAV) |
|
||||
| `frontend` | custom (nginxinc/nginx-unprivileged:alpine) | 80 (prod) / 5173 (dev) | backend-net + frontend-net | 1001:1001 | React UI + nginx reverse proxy |
|
||||
|
||||
**Networks:**
|
||||
@@ -46,8 +48,8 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
|
||||
**Volumes:**
|
||||
- `postgres_data` — PostgreSQL data files
|
||||
- `doc_data` — uploaded PDF files (mounted into doc-service at `/data/documents`)
|
||||
- `app_config` — per-service runtime config JSON files (mounted into backend, ai-service, and doc-service at `/config`)
|
||||
- `storage_data` — all file/blob storage: uploaded PDFs (`documents/` bucket) and service config JSON files (`config/` bucket); mounted into storage-service at `/data/storage`
|
||||
- `watch_data` — file watcher input directory; mounted into doc-service at `/data/watch`
|
||||
|
||||
The frontend nginx proxies `/api/*` to `backend:8000` via `backend-net`. The backend proxies `/api/documents/*` and `/api/documents/categories/*` to `doc-service:8001`. The backend test-connection endpoint proxies to `ai-service:8010`. No backend service or database port is ever exposed to the host.
|
||||
|
||||
@@ -126,10 +128,8 @@ Copy `.env.example` to `backend/.env` and adjust:
|
||||
| `JWT_PRIVATE_KEY` | — | RS256 private key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `JWT_PUBLIC_KEY` | — | RS256 public key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `CORS_ORIGINS` | `["http://localhost:5173"]` | Allowed frontend origins |
|
||||
| `APP_CONFIG_DIR` | `/config` | Directory for per-service runtime config JSON files |
|
||||
| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of the doc-service (set by docker-compose) |
|
||||
|
||||
`doc-service` reads `DATABASE_URL`, `DATA_DIR`, and `CONFIG_PATH` from its own environment (set in `docker-compose.yml`).
|
||||
| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of doc-service (set by docker-compose) |
|
||||
| `STORAGE_SERVICE_URL` | `http://storage-service:8020` | Internal URL of storage-service (set by docker-compose) |
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
@@ -0,0 +1,365 @@
|
||||
# backend — Claude context
|
||||
|
||||
FastAPI async gateway, port 8000 (internal). Handles auth, user/group management, settings, and proxies document/category requests to `doc-service:8001`. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
All commands run inside Docker — never on the host.
|
||||
|
||||
### Migrations
|
||||
|
||||
```bash
|
||||
docker compose exec backend alembic revision --autogenerate -m "describe change"
|
||||
docker compose exec backend alembic upgrade head
|
||||
docker compose exec backend alembic downgrade -1
|
||||
```
|
||||
|
||||
### Lint
|
||||
|
||||
```bash
|
||||
docker compose exec backend ruff check . && ruff format .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── main.py ← App factory, router registration, lifespan (health loop)
|
||||
│ ├── database.py ← AsyncEngine, AsyncSessionLocal, Base
|
||||
│ ├── deps.py ← get_current_user, get_current_admin, get_service_admin(id), check_plugin_access (also get_user_groups in doc-service)
|
||||
│ ├── core/
|
||||
│ │ ├── config.py ← All settings via pydantic-settings (reads .env)
|
||||
│ │ ├── security.py ← JWT sign/verify (RS256), bcrypt hash/verify
|
||||
│ │ ├── sanitize.py ← Input sanitization helpers (see Security Standards)
|
||||
│ │ ├── app_config.py ← Per-service config load/save via storage-service; theme files in config/themes/{id}.json
|
||||
│ │ └── config_storage.py ← Thin async HTTP helpers: read_json/write_json/delete_key/list_keys → storage-service config bucket
|
||||
│ ├── models/
|
||||
│ │ ├── __init__.py ← Imports all models (required for Alembic autogenerate)
|
||||
│ │ ├── user.py ← User model
|
||||
│ │ ├── profile.py ← Profile model
|
||||
│ │ └── group.py ← Group, GroupMembership models
|
||||
│ ├── schemas/
|
||||
│ │ ├── user.py ← UserCreate/Out, Token, DashboardPrefsOut/Update
|
||||
│ │ ├── profile.py ← ProfileRead, ProfileUpdate
|
||||
│ │ └── group.py ← GroupCreate/Update/Out/DetailOut, GroupMemberOut
|
||||
│ ├── routers/
|
||||
│ │ ├── auth.py ← POST /register, POST /login
|
||||
│ │ ├── users.py ← GET /me, GET+PATCH /me/preferences, PATCH /me/color-mode, GET /me/groups
|
||||
│ │ ├── profile.py ← GET+PUT /me (profile)
|
||||
│ │ ├── admin.py ← User admin CRUD (admin-only)
|
||||
│ │ ├── groups.py ← Group CRUD + member management (admin-only)
|
||||
│ │ ├── settings.py ← AI, doc limits, system prompts, appearance, themes (admin-only)
|
||||
│ │ ├── services.py ← GET /services (health status)
|
||||
│ │ ├── plugins.py ← Generic plugin proxy (GET/PATCH /api/plugins/*)
|
||||
│ │ ├── categories_proxy.py ← Transparent proxy → doc-service /categories/*
|
||||
│ │ ├── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ └── storage_config.py ← Admin proxy → storage-service config + migration endpoints
|
||||
│ └── services/
|
||||
│ ├── service_health.py ← Background 30s health-check loop; caches /plugin/manifest per service
|
||||
│ └── group_bootstrap.py ← Ensures {service-id}-admin group exists for every registered service at startup
|
||||
├── alembic/
|
||||
│ ├── env.py ← Async migration runner
|
||||
│ └── versions/ ← Migration chain (see Database Models)
|
||||
├── scripts/seed.py ← Seed test user
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Models
|
||||
|
||||
### `users`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `email` | String | UNIQUE, indexed, NOT NULL | lowercased before storing |
|
||||
| `hashed_password` | String | NOT NULL | bcrypt 13 rounds |
|
||||
| `full_name` | String | nullable | sanitized max 128 chars |
|
||||
| `is_active` | Boolean | default=True | soft-delete flag |
|
||||
| `is_superuser` | Boolean | default=False | admin role; never exposed as-is (serialised as `is_admin`) |
|
||||
| `dashboard_app_ids` | JSON | NOT NULL, default=[] | list of pinned service IDs |
|
||||
| `color_mode` | String | nullable, default=NULL | user's preferred mode: "light" / "dark" / "system" / NULL (use admin default) |
|
||||
|
||||
Relationship: `profile` (one-to-one, cascade all+delete-orphan)
|
||||
|
||||
### `profiles`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `user_id` | String | FK→users.id UNIQUE, cascade delete | one-to-one |
|
||||
| `phone` | String(20) | nullable | validated format |
|
||||
| `date_of_birth` | Date | nullable | 1900+ and not future |
|
||||
| `position` | String(128) | nullable | job title |
|
||||
| `address` | String(255) | nullable | |
|
||||
| `updated_at` | DateTime(tz) | server_default=now(), onupdate=now() | |
|
||||
|
||||
### `groups`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `name` | String(128) | UNIQUE indexed, NOT NULL |
|
||||
| `description` | String(512) | nullable |
|
||||
| `created_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
### `group_memberships`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `group_id` | String | FK→groups.id, indexed, CASCADE |
|
||||
| `user_id` | String | FK→users.id, indexed, CASCADE |
|
||||
| `is_group_admin` | Boolean | NOT NULL, default=false | grants group-admin rights (manage group categories, delete shared docs) |
|
||||
| `joined_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
Unique constraint: `(group_id, user_id)`
|
||||
|
||||
### Migration chain (must be applied in order)
|
||||
|
||||
| Rev ID | Slug |
|
||||
|--------|------|
|
||||
| `38efeff7c45a` | `create_users_table` |
|
||||
| `676084df61d1` | `add_profiles_table` |
|
||||
| `a3f9c2d14e87` | `add_groups_and_group_memberships` |
|
||||
| `c7e8f9a0b1d2` | `add_dashboard_app_ids_to_users` |
|
||||
| `dd6ad2f2c211` | `add_color_mode_to_users` |
|
||||
| `e1f2a3b4c5d6` | `add_group_member_is_admin` |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Auth (`/api/auth`) — public
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/auth/register` | — | Create account; returns `UserOut`; enforces password policy |
|
||||
| POST | `/api/auth/login` | — | OAuth2 password flow; returns `{access_token, token_type}` |
|
||||
|
||||
### Users (`/api/users`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/users/me` | user | Current user info → `UserOut` |
|
||||
| GET | `/api/users/me/preferences` | user | Dashboard pinned app IDs → `{app_ids}` |
|
||||
| PATCH | `/api/users/me/preferences` | user | Save pinned app IDs (max 50, slug-safe) |
|
||||
| PATCH | `/api/users/me/color-mode` | user | Save colour mode preference ("light"/"dark"/"system") |
|
||||
| GET | `/api/users/me/groups` | user | Groups current user belongs to → `list[UserGroupOut]` |
|
||||
|
||||
### Profile (`/api/profile`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/profile/me` | user | Fetch profile; auto-creates if missing |
|
||||
| PUT | `/api/profile/me` | user | Update profile fields |
|
||||
|
||||
### Admin — Users (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/users` | List all users → `list[UserAdminOut]` |
|
||||
| POST | `/api/admin/users` | Create user (with optional is_admin) |
|
||||
| DELETE | `/api/admin/users/{user_id}` | Delete user (204) |
|
||||
| PATCH | `/api/admin/users/{user_id}/active` | Toggle active status |
|
||||
|
||||
### Admin — Groups (`/api/admin/groups`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/groups` | List groups with member count |
|
||||
| POST | `/api/admin/groups` | Create group |
|
||||
| GET | `/api/admin/groups/{id}` | Group detail + members |
|
||||
| PATCH | `/api/admin/groups/{id}` | Update name / description |
|
||||
| DELETE | `/api/admin/groups/{id}` | Delete (cascades memberships) |
|
||||
| POST | `/api/admin/groups/{id}/members/{user_id}` | Add member |
|
||||
| DELETE | `/api/admin/groups/{id}/members/{user_id}` | Remove member |
|
||||
| PATCH | `/api/admin/groups/{id}/members/{user_id}/admin` | Set/unset group admin role (body: `{ is_group_admin: bool }`) |
|
||||
|
||||
### Settings (`/api/settings`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/settings/ai` | AI config (keys masked) |
|
||||
| PATCH | `/api/settings/ai` | Update AI provider / credentials |
|
||||
| POST | `/api/settings/ai/test` | Test AI connection |
|
||||
| GET | `/api/settings/documents/limits` | PDF upload limits |
|
||||
| PATCH | `/api/settings/documents/limits` | Update max PDF size |
|
||||
| GET | `/api/settings/system-prompts` | All editable system prompts |
|
||||
| PATCH | `/api/settings/system-prompts/{service_id}` | Update system prompt |
|
||||
| GET | `/api/settings/appearance` | Active theme + default mode (auth) |
|
||||
| PATCH | `/api/settings/appearance` | Update active theme + default mode (admin) |
|
||||
| GET | `/api/settings/themes` | List all themes — built-in + custom (auth) |
|
||||
| POST | `/api/settings/themes` | Create custom theme (admin) |
|
||||
| PATCH | `/api/settings/themes/{id}` | Update custom theme label/colours (admin) |
|
||||
| DELETE | `/api/settings/themes/{id}` | Delete custom theme (admin, 204) |
|
||||
|
||||
### Services (`/api/services`) — authenticated
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/services` | Health status of all registered services → `list[ServiceStatus]` |
|
||||
|
||||
### Plugins (`/api/plugins`) — authenticated, auth-per-plugin
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/plugins` | List plugins accessible to current user |
|
||||
| GET | `/api/plugins/{id}/manifest` | Plugin manifest with settings JSON Schema (auth-gated) |
|
||||
| GET | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
|
||||
| PATCH | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
|
||||
|
||||
Auth: is_superuser OR member of group listed in manifest `required_groups`. Returns 404 (not 403) to hide existence.
|
||||
|
||||
### Admin — Storage (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/storage-config` | Current backend driver + health → proxied from storage-service `/health` |
|
||||
| PATCH | `/api/admin/storage-config` | Reconfigure backend without data migration (same-backend credential update) |
|
||||
| POST | `/api/admin/storage-config/migrate` | Start async migration to a new backend (copy → verify → switch → cleanup) |
|
||||
| GET | `/api/admin/storage-config/migrate/status` | Poll migration progress: `{state, total, done, failed, errors[]}` |
|
||||
| DELETE | `/api/admin/storage-config/migrate` | Cancel a running migration; old backend remains active |
|
||||
|
||||
### Documents and Categories — proxied
|
||||
|
||||
`/api/documents/*` and `/api/documents/categories/*` are transparently proxied to `doc-service:8001`. The backend injects `x-user-id`, `x-user-groups`, and `x-user-is-admin` headers. See `features/doc-service/CLAUDE.md` for the internal endpoint list.
|
||||
|
||||
---
|
||||
|
||||
## Security Standards
|
||||
|
||||
These standards are **non-negotiable**. Every change must comply.
|
||||
|
||||
### JWT
|
||||
|
||||
- **Algorithm**: RS256 (4096-bit RSA key pair, generated by `scripts/generate_jwt_keys.py`)
|
||||
- **Keys**: PEM-encoded in `backend/.env` as `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` (gitignored)
|
||||
- **Expiry**: 8 hours (`EXPIRE_MINUTES=480`) — never set longer; no refresh tokens
|
||||
- **Claims**: `{sub: user_id, exp, iat}` — user_id is a UUID string
|
||||
- **Validation**: `decode_access_token()` in `core/security.py`; called by `get_current_user`
|
||||
- **Never**: set algorithm to `"none"`, disable `verify_exp`, or hardcode secrets in code
|
||||
|
||||
### Password hashing
|
||||
|
||||
- **Algorithm**: bcrypt, **13 rounds** (`bcrypt.gensalt(rounds=13)`)
|
||||
- **Timing**: ~300 ms per hash (intentional brute-force resistance)
|
||||
- **Never** use MD5, SHA1, or plain SHA256 for password storage
|
||||
|
||||
### Password policy (enforced in `UserCreate` schema)
|
||||
|
||||
All of the following must pass:
|
||||
- ≥ 8 characters
|
||||
- ≥ 1 uppercase (A–Z)
|
||||
- ≥ 1 lowercase (a–z)
|
||||
- ≥ 1 digit (0–9)
|
||||
- ≥ 1 special character: `!@#$%^&*()\-_=+[]{}|;:'"<>?/\`~`
|
||||
- No common words (password, secret, login, admin, test, qwerty, welcome, …)
|
||||
|
||||
### Input sanitization
|
||||
|
||||
Every user-supplied string stored in the database **must** pass through `core/sanitize.py`:
|
||||
|
||||
```python
|
||||
sanitize_str(value, max_len=255)
|
||||
# → strips whitespace; rejects null bytes (\x00); rejects control chars
|
||||
# (0x01–0x1F, 0x7F except \t \n \r); enforces max_len; returns None for ""
|
||||
|
||||
normalize_email(value) # lowercase + strip
|
||||
validate_phone(value) # sanitize_str(max=20) + regex ^\+?[\d\s\-()\[\]]{7,20}$
|
||||
validate_date_of_birth(v) # must be ≥ 1900, not future
|
||||
```
|
||||
|
||||
Apply via Pydantic `@field_validator` on all request schemas.
|
||||
|
||||
### SQL injection prevention
|
||||
|
||||
- Use SQLAlchemy ORM (bound parameters) — **never** raw SQL strings.
|
||||
- If `text()` is needed, use `bindparam()` for all user-supplied values.
|
||||
- **Never** use f-strings, `.format()`, or `%`-formatting for SQL.
|
||||
|
||||
### Admin route security
|
||||
|
||||
- Use `get_current_admin` dependency (checks `is_superuser`).
|
||||
- Return **404** (not 403) for unauthorized access — hides both endpoint existence and permission model.
|
||||
|
||||
---
|
||||
|
||||
## Naming & Code Conventions
|
||||
|
||||
### Database
|
||||
|
||||
- **Tables**: lowercase, plural, snake_case (`users`, `group_memberships`, `document_category_assignments`)
|
||||
- **Columns**: lowercase, snake_case
|
||||
- **ORM models**: PascalCase, singular (`User`, `Group`, `GroupMembership`, `Document`)
|
||||
- Primary keys: `id` (String UUID, auto-generated)
|
||||
- Timestamps: `created_at` / `updated_at` / `joined_at` / `processed_at` — always timezone-aware
|
||||
|
||||
### Pydantic schemas
|
||||
|
||||
| Suffix | Purpose |
|
||||
|--------|---------|
|
||||
| `Create` | POST request body (user-supplied input) |
|
||||
| `Update` | PATCH request body (partial update) |
|
||||
| `Out` | API response (safe subset of model) |
|
||||
| `AdminOut` | Extended response for admin endpoints |
|
||||
| `Read` | GET response (same as `Out`, used for profiles) |
|
||||
|
||||
Always set `model_config = {"from_attributes": True}` on response schemas.
|
||||
Use `validation_alias` when the ORM field name differs from the JSON key (e.g., `is_superuser` → `is_admin`).
|
||||
|
||||
### HTTP status codes
|
||||
|
||||
| Code | Use |
|
||||
|------|-----|
|
||||
| 200 | Successful GET / PATCH / PUT |
|
||||
| 201 | Successful POST that creates a resource |
|
||||
| 202 | Accepted (async processing started, e.g., document upload) |
|
||||
| 204 | Successful DELETE or action with no response body |
|
||||
| 400 | Bad request (duplicates, invalid data beyond Pydantic) |
|
||||
| 401 | Missing / invalid JWT |
|
||||
| 404 | Not found **and** admin routes when not admin |
|
||||
| 413 | Payload too large (file exceeds limit) |
|
||||
| 415 | Unsupported media type (not a PDF) |
|
||||
| 422 | Pydantic validation failure (FastAPI default) |
|
||||
| 502 | Downstream service unreachable |
|
||||
| 503 | Service unavailable (queue stopped, AI error) |
|
||||
| 504 | Gateway timeout |
|
||||
|
||||
### Backend code style
|
||||
|
||||
- Async/await for **all** I/O (DB, HTTP, file).
|
||||
- `raise HTTPException(status_code=..., detail="...")` for all errors.
|
||||
- Response models always declared in route decorator: `@router.get("/path", response_model=XOut)`.
|
||||
- Background tasks via `BackgroundTasks` param; tasks open their own `AsyncSessionLocal` session.
|
||||
- Commit + refresh pattern after mutations:
|
||||
```python
|
||||
await db.commit()
|
||||
await db.refresh(obj)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| JWT expiry | 480 min (8 h) | `core/security.py` |
|
||||
| Bcrypt rounds | 13 | `core/security.py` |
|
||||
| User `color_mode` default | NULL (falls back to admin default_mode, then system) | `models/user.py` |
|
||||
| Max dashboard pinned apps | 50 | `schemas/user.py` |
|
||||
| App ID max length | 64 chars | `schemas/user.py` |
|
||||
| App ID allowed chars | `[a-zA-Z0-9_\-]` | `schemas/user.py` |
|
||||
| full_name max length | 128 chars | `schemas/user.py` |
|
||||
| Group name max length | 128 chars | `schemas/group.py` |
|
||||
| Group description max | 512 chars | `schemas/group.py` |
|
||||
| Phone max length | 20 chars | `sanitize.py` |
|
||||
| Position max length | 128 chars | `schemas/profile.py` |
|
||||
| Address max length | 255 chars | `schemas/profile.py` |
|
||||
+44
-8
@@ -27,6 +27,7 @@ JWT signing uses a 4096-bit RSA key pair (`RS256`). Keys are generated by `scrip
|
||||
| `GET` | `/api/users/me` | Current user info |
|
||||
| `GET` | `/api/users/me/preferences` | User's dashboard preferences (`app_ids` list) |
|
||||
| `PATCH` | `/api/users/me/preferences` | Update pinned app IDs (max 50; validated as safe slugs) |
|
||||
| `GET` | `/api/users/me/groups` | List groups the current user belongs to (for share picker) |
|
||||
|
||||
### Profile (`/api/profile`)
|
||||
|
||||
@@ -66,13 +67,27 @@ A background task (`service_health.py`) polls each service's `/health` endpoint
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/settings/ai` | AI service config (masked — API keys redacted) |
|
||||
| `PATCH` | `/api/settings/ai` | Update AI provider / credentials |
|
||||
| `POST` | `/api/settings/ai/test` | Test AI connection (proxies a minimal /chat call) |
|
||||
| `GET` | `/api/settings/documents/limits` | Doc service upload limits |
|
||||
| `PATCH` | `/api/settings/documents/limits` | Update max PDF size |
|
||||
| `GET` | `/api/settings/ai` | AI service config (masked) — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/ai` | Update AI provider / credentials — same access |
|
||||
| `POST` | `/api/settings/ai/test` | Test AI connection — same access |
|
||||
| `GET` | `/api/settings/documents/limits` | Doc service upload limits — superuser OR `doc-service-admin` member |
|
||||
| `PATCH` | `/api/settings/documents/limits` | Update max PDF size — same access |
|
||||
| `GET` | `/api/settings/system-prompts` | All editable system prompts — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/system-prompts/{id}` | Update system prompt — same access |
|
||||
|
||||
Settings are persisted to JSON files on the `app_config` Docker named volume and read by the respective feature services.
|
||||
Settings are persisted to the `config` bucket of `storage-service:8020` via `core/config_storage.py`. All config I/O is async HTTP; no filesystem volumes are used.
|
||||
|
||||
Access to service-specific settings endpoints is enforced by `get_service_admin(service_id)` in `deps.py` — grants access to superusers OR members of the `{service_id}-admin` group.
|
||||
|
||||
### Storage config (`/api/admin`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/storage-config` | Current backend driver + health (proxied from storage-service) |
|
||||
| `PATCH` | `/api/admin/storage-config` | Reconfigure backend without migration |
|
||||
| `POST` | `/api/admin/storage-config/migrate` | Start async migration to a new backend |
|
||||
| `GET` | `/api/admin/storage-config/migrate/status` | Poll migration progress |
|
||||
| `DELETE` | `/api/admin/storage-config/migrate` | Cancel running migration |
|
||||
|
||||
### Feature proxies
|
||||
|
||||
@@ -82,6 +97,23 @@ All `/api/documents/*` and `/api/documents/categories/*` requests are transparen
|
||||
- Strips hop-by-hop headers + `content-length`, `accept-encoding`, `content-type`
|
||||
- Returns `Response` (not `StreamingResponse`) to avoid content-length/chunked conflicts
|
||||
|
||||
### Plugin system (`/api/plugins`)
|
||||
|
||||
Generic extension/plugin infrastructure — **zero feature-specific code in backend**. Feature containers self-describe via `GET /plugin/manifest`.
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/plugins` | user | List plugins accessible to current user |
|
||||
| `GET` | `/api/plugins/{id}/manifest` | user | Cached manifest for a plugin (404 if not accessible) |
|
||||
| `GET` | `/api/plugins/{id}/settings` | user | Proxy to feature `GET /plugin/settings` |
|
||||
| `PATCH` | `/api/plugins/{id}/settings` | user | Proxy to feature `PATCH /plugin/settings` |
|
||||
|
||||
Access is controlled by the manifest: `allow_superuser` for admins; `required_groups` for group members. `check_plugin_access(plugin_id, user, db)` in `deps.py` enforces this.
|
||||
|
||||
During each health poll, `service_health.py` also fetches `GET /plugin/manifest` from healthy services and caches it. New feature containers that expose `/plugin/manifest` automatically appear in the plugin list — no backend code changes required.
|
||||
|
||||
**Service admin group bootstrap:** On every startup, `group_bootstrap.py` creates a `{service-id}-admin` group for every registered service (idempotent). Admins add users to these groups via the Admin → Groups UI to delegate service-level administration.
|
||||
|
||||
### Database models
|
||||
|
||||
| Model | Table | Notes |
|
||||
@@ -107,8 +139,11 @@ Browser (port 5173 dev / 80 prod)
|
||||
┌───────────┼────────────┬──────────────┐
|
||||
/auth /settings /documents/* /services
|
||||
/users (JSON │ │
|
||||
/admin volume) └── proxy → health-check loop
|
||||
/profile doc-service:8001 (30s poll)
|
||||
/admin /storage- └── proxy → health-check loop
|
||||
/profile config doc-service:8001 (30s poll)
|
||||
(proxy)
|
||||
│
|
||||
storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
@@ -137,6 +172,7 @@ Browser (port 5173 dev / 80 prod)
|
||||
## Future work
|
||||
|
||||
- [x] Groups system: `groups`, `group_memberships` tables; admin CRUD; add/remove members
|
||||
- [x] Generic plugin infrastructure: manifest contract, `/api/plugins` proxy router, `check_plugin_access`
|
||||
- [ ] App permissions registry: `group_app_permissions` table; AppsPage filtered by group grants
|
||||
- [ ] Doc sharing via group membership
|
||||
- [ ] App permissions registry: `user_app_permissions (user_id, app_key)`; AppsPage filtered by grants
|
||||
|
||||
@@ -0,0 +1,25 @@
|
||||
"""add_color_mode_to_users
|
||||
|
||||
Revision ID: dd6ad2f2c211
|
||||
Revises: c7e8f9a0b1d2
|
||||
Create Date: 2026-04-17 23:42:58.222958
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = 'dd6ad2f2c211'
|
||||
down_revision: Union[str, None] = 'c7e8f9a0b1d2'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column('users', sa.Column('color_mode', sa.String(), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column('users', 'color_mode')
|
||||
@@ -0,0 +1,32 @@
|
||||
"""add is_group_admin to group_memberships
|
||||
|
||||
Revision ID: e1f2a3b4c5d6
|
||||
Revises: dd6ad2f2c211
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "e1f2a3b4c5d6"
|
||||
down_revision: Union[str, None] = "dd6ad2f2c211"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"group_memberships",
|
||||
sa.Column(
|
||||
"is_group_admin",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default=sa.text("false"),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("group_memberships", "is_group_admin")
|
||||
+239
-43
@@ -1,20 +1,25 @@
|
||||
"""
|
||||
Per-service runtime config helpers.
|
||||
|
||||
Config files live on the shared `app_config` Docker volume at /config/.
|
||||
Each service has its own JSON file.
|
||||
All config files are stored in the 'config' bucket of the storage-service.
|
||||
Every function is async — callers must await them.
|
||||
|
||||
Atomic write pattern: write to .tmp in same dir, then os.replace() so
|
||||
services never read a partial file.
|
||||
Key layout in the config bucket:
|
||||
ai_service_config.json
|
||||
doc_service_config.json
|
||||
appearance_config.json
|
||||
themes/{id}.json
|
||||
"""
|
||||
import copy
|
||||
import json
|
||||
import os
|
||||
from pathlib import Path
|
||||
import logging
|
||||
import re
|
||||
from copy import deepcopy
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
_CONFIG_DIR = Path(os.environ.get("APP_CONFIG_DIR", "/config"))
|
||||
from app.core import config_storage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── AI service config schemas ──────────────────────────────────────────────────
|
||||
|
||||
@@ -107,59 +112,50 @@ def _mask_ai_config(data: dict) -> dict:
|
||||
|
||||
# ── Load / Save ────────────────────────────────────────────────────────────────
|
||||
|
||||
def _config_path(service: str) -> Path:
|
||||
return _CONFIG_DIR / f"{service}_config.json"
|
||||
|
||||
|
||||
def load_service_config(service: str) -> dict:
|
||||
path = _config_path(service)
|
||||
if not path.exists():
|
||||
async def load_service_config(service: str) -> dict:
|
||||
data = await config_storage.read_json(f"{service}_config.json")
|
||||
if data is None:
|
||||
if service == "ai_service":
|
||||
return AIServiceConfig().model_dump()
|
||||
if service == "doc_service":
|
||||
return DocServiceConfig().model_dump()
|
||||
return {}
|
||||
with path.open() as f:
|
||||
return json.load(f)
|
||||
return data
|
||||
|
||||
|
||||
def save_service_config(service: str, data: dict) -> None:
|
||||
path = _config_path(service)
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
tmp = path.with_suffix(".tmp")
|
||||
tmp.write_text(json.dumps(data, indent=2))
|
||||
os.replace(tmp, path)
|
||||
async def save_service_config(service: str, data: dict) -> None:
|
||||
await config_storage.write_json(f"{service}_config.json", data)
|
||||
|
||||
|
||||
# AI service helpers
|
||||
|
||||
def load_ai_service_config() -> AIServiceConfig:
|
||||
raw = load_service_config("ai_service")
|
||||
async def load_ai_service_config() -> AIServiceConfig:
|
||||
raw = await load_service_config("ai_service")
|
||||
return AIServiceConfig.model_validate(raw)
|
||||
|
||||
|
||||
def save_ai_service_config(config: AIServiceConfig) -> None:
|
||||
save_service_config("ai_service", config.model_dump())
|
||||
async def save_ai_service_config(config: AIServiceConfig) -> None:
|
||||
await save_service_config("ai_service", config.model_dump())
|
||||
|
||||
|
||||
def load_ai_service_config_masked() -> dict:
|
||||
raw = load_service_config("ai_service")
|
||||
async def load_ai_service_config_masked() -> dict:
|
||||
raw = await load_service_config("ai_service")
|
||||
return _mask_ai_config(raw)
|
||||
|
||||
|
||||
# Doc service helpers
|
||||
|
||||
def load_doc_service_config() -> DocServiceConfig:
|
||||
raw = load_service_config("doc_service")
|
||||
async def load_doc_service_config() -> DocServiceConfig:
|
||||
raw = await load_service_config("doc_service")
|
||||
return DocServiceConfig.model_validate(raw)
|
||||
|
||||
|
||||
def save_doc_service_config(config: DocServiceConfig) -> None:
|
||||
save_service_config("doc_service", config.model_dump())
|
||||
async def save_doc_service_config(config: DocServiceConfig) -> None:
|
||||
await save_service_config("doc_service", config.model_dump())
|
||||
|
||||
|
||||
def load_doc_service_config_masked() -> dict:
|
||||
return load_service_config("doc_service")
|
||||
async def load_doc_service_config_masked() -> dict:
|
||||
return await load_service_config("doc_service")
|
||||
|
||||
|
||||
def _merge_api_key(new_key: str, existing_key: str) -> str:
|
||||
@@ -171,18 +167,16 @@ def _merge_api_key(new_key: str, existing_key: str) -> str:
|
||||
|
||||
# ── System prompts helpers ─────────────────────────────────────────────────────
|
||||
|
||||
# Registry of all services that have editable system prompts.
|
||||
# key = service identifier, value = human-readable label
|
||||
SYSTEM_PROMPT_SERVICES: dict[str, str] = {
|
||||
"doc_service": "Document Service",
|
||||
}
|
||||
|
||||
|
||||
def load_all_system_prompts() -> dict:
|
||||
async def load_all_system_prompts() -> dict:
|
||||
"""Return {service_id: {label, system, user_template, default_system, default_user_template}}."""
|
||||
result: dict = {}
|
||||
for service_id, label in SYSTEM_PROMPT_SERVICES.items():
|
||||
config = load_service_config(service_id)
|
||||
config = await load_service_config(service_id)
|
||||
prompts = config.get("system_prompts", {})
|
||||
defaults = _get_service_prompt_defaults(service_id)
|
||||
result[service_id] = {
|
||||
@@ -195,15 +189,14 @@ def load_all_system_prompts() -> dict:
|
||||
return result
|
||||
|
||||
|
||||
def save_service_system_prompts(service_id: str, system: str, user_template: str) -> None:
|
||||
"""Persist updated system prompts into the service's config file."""
|
||||
async def save_service_system_prompts(service_id: str, system: str, user_template: str) -> None:
|
||||
if service_id not in SYSTEM_PROMPT_SERVICES:
|
||||
raise ValueError(f"Unknown service: {service_id!r}")
|
||||
config = load_service_config(service_id)
|
||||
config = await load_service_config(service_id)
|
||||
config.setdefault("system_prompts", {})
|
||||
config["system_prompts"]["system"] = system
|
||||
config["system_prompts"]["user_template"] = user_template
|
||||
save_service_config(service_id, config)
|
||||
await save_service_config(service_id, config)
|
||||
|
||||
|
||||
def _get_service_prompt_defaults(service_id: str) -> dict:
|
||||
@@ -211,3 +204,206 @@ def _get_service_prompt_defaults(service_id: str) -> dict:
|
||||
d = DocServiceSystemPrompts()
|
||||
return {"system": d.system, "user_template": d.user_template}
|
||||
return {"system": "", "user_template": ""}
|
||||
|
||||
|
||||
# ── Appearance config ──────────────────────────────────────────────────────────
|
||||
|
||||
class AppearanceConfig(BaseModel):
|
||||
theme: str = "default"
|
||||
default_mode: str = "system"
|
||||
|
||||
|
||||
async def load_appearance_config() -> AppearanceConfig:
|
||||
data = await config_storage.read_json("appearance_config.json")
|
||||
if data is None:
|
||||
return AppearanceConfig()
|
||||
return AppearanceConfig.model_validate(data)
|
||||
|
||||
|
||||
async def save_appearance_config(config: AppearanceConfig) -> None:
|
||||
await config_storage.write_json("appearance_config.json", config.model_dump())
|
||||
|
||||
|
||||
# ── Theme file management ──────────────────────────────────────────────────────
|
||||
|
||||
# 9 required colour tokens per mode
|
||||
_REQUIRED_TOKENS = frozenset({
|
||||
"primary", "primary_hover", "accent", "accent_hover",
|
||||
"background", "surface", "border", "text_primary", "text_muted",
|
||||
})
|
||||
|
||||
_RGB_RE = re.compile(r"^\d{1,3} \d{1,3} \d{1,3}$")
|
||||
|
||||
_BUILTIN_THEMES: list[dict] = [
|
||||
{
|
||||
"id": "default",
|
||||
"label": "Default",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "37 99 235",
|
||||
"primary_hover": "29 78 216",
|
||||
"accent": "234 179 8",
|
||||
"accent_hover": "202 138 4",
|
||||
"background": "248 250 252",
|
||||
"surface": "255 255 255",
|
||||
"border": "226 232 240",
|
||||
"text_primary": "15 23 42",
|
||||
"text_muted": "100 116 139",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "59 130 246",
|
||||
"primary_hover": "37 99 235",
|
||||
"accent": "250 204 21",
|
||||
"accent_hover": "234 179 8",
|
||||
"background": "15 23 42",
|
||||
"surface": "30 41 59",
|
||||
"border": "51 65 85",
|
||||
"text_primary": "203 213 225",
|
||||
"text_muted": "148 163 184",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "pastel",
|
||||
"label": "Pastel",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "124 58 237",
|
||||
"primary_hover": "109 40 217",
|
||||
"accent": "236 72 153",
|
||||
"accent_hover": "219 39 119",
|
||||
"background": "253 244 255",
|
||||
"surface": "250 245 255",
|
||||
"border": "233 213 255",
|
||||
"text_primary": "30 27 75",
|
||||
"text_muted": "107 114 128",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "167 139 250",
|
||||
"primary_hover": "196 181 253",
|
||||
"accent": "244 114 182",
|
||||
"accent_hover": "251 164 200",
|
||||
"background": "30 20 51",
|
||||
"surface": "45 27 78",
|
||||
"border": "76 53 117",
|
||||
"text_primary": "233 213 255",
|
||||
"text_muted": "196 181 253",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "high-contrast",
|
||||
"label": "High Contrast",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "30 58 138",
|
||||
"primary_hover": "30 64 175",
|
||||
"accent": "220 38 38",
|
||||
"accent_hover": "185 28 28",
|
||||
"background": "255 255 255",
|
||||
"surface": "255 255 255",
|
||||
"border": "156 163 175",
|
||||
"text_primary": "0 0 0",
|
||||
"text_muted": "75 85 99",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "96 165 250",
|
||||
"primary_hover": "147 197 253",
|
||||
"accent": "248 113 113",
|
||||
"accent_hover": "252 165 165",
|
||||
"background": "0 0 0",
|
||||
"surface": "10 10 10",
|
||||
"border": "55 65 81",
|
||||
"text_primary": "255 255 255",
|
||||
"text_muted": "156 163 175",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "ocean",
|
||||
"label": "Ocean Blue",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "29 78 216",
|
||||
"primary_hover": "30 58 138",
|
||||
"accent": "8 145 178",
|
||||
"accent_hover": "14 116 144",
|
||||
"background": "239 246 255",
|
||||
"surface": "219 234 254",
|
||||
"border": "147 197 253",
|
||||
"text_primary": "30 58 138",
|
||||
"text_muted": "59 130 246",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "96 165 250",
|
||||
"primary_hover": "147 197 253",
|
||||
"accent": "34 211 238",
|
||||
"accent_hover": "103 232 249",
|
||||
"background": "10 22 40",
|
||||
"surface": "15 36 68",
|
||||
"border": "29 78 216",
|
||||
"text_primary": "219 234 254",
|
||||
"text_muted": "147 197 253",
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
async def seed_builtin_themes() -> None:
|
||||
"""Write built-in theme files to storage-service if they are not already there."""
|
||||
existing_keys = await config_storage.list_keys(prefix="themes/")
|
||||
existing_ids = {k.removeprefix("themes/").removesuffix(".json") for k in existing_keys}
|
||||
for theme in _BUILTIN_THEMES:
|
||||
if theme["id"] not in existing_ids:
|
||||
await config_storage.write_json(f"themes/{theme['id']}.json", theme)
|
||||
logger.info("Built-in themes seeded (%d themes)", len(_BUILTIN_THEMES))
|
||||
|
||||
|
||||
async def load_all_themes() -> list[dict]:
|
||||
"""Return all themes from storage-service, built-ins first then custom by label."""
|
||||
keys = await config_storage.list_keys(prefix="themes/")
|
||||
themes: list[dict] = []
|
||||
for key in keys:
|
||||
data = await config_storage.read_json(key)
|
||||
if data:
|
||||
themes.append(data)
|
||||
|
||||
builtin_ids = [t["id"] for t in _BUILTIN_THEMES]
|
||||
|
||||
def sort_key(t: dict) -> tuple:
|
||||
tid = t.get("id", "")
|
||||
try:
|
||||
return (0, builtin_ids.index(tid))
|
||||
except ValueError:
|
||||
return (1, t.get("label", tid).lower())
|
||||
|
||||
return sorted(themes, key=sort_key)
|
||||
|
||||
|
||||
async def load_theme_by_id(theme_id: str) -> dict | None:
|
||||
"""Return a single theme dict, or None if not found."""
|
||||
return await config_storage.read_json(f"themes/{theme_id}.json")
|
||||
|
||||
|
||||
async def save_theme(theme: dict) -> None:
|
||||
"""Write a theme to storage-service."""
|
||||
await config_storage.write_json(f"themes/{theme['id']}.json", theme)
|
||||
|
||||
|
||||
async def delete_theme(theme_id: str) -> None:
|
||||
"""Delete a custom theme. Raises ValueError for built-ins, KeyError if not found."""
|
||||
data = await config_storage.read_json(f"themes/{theme_id}.json")
|
||||
if data is None:
|
||||
raise FileNotFoundError(theme_id)
|
||||
if data.get("builtin"):
|
||||
raise ValueError("Cannot delete a built-in theme")
|
||||
await config_storage.delete_key(f"themes/{theme_id}.json")
|
||||
|
||||
|
||||
def validate_theme_tokens(colors: dict) -> list[str]:
|
||||
"""Return a list of validation error messages, empty if valid."""
|
||||
errors = []
|
||||
missing = _REQUIRED_TOKENS - set(colors.keys())
|
||||
if missing:
|
||||
errors.append(f"Missing tokens: {', '.join(sorted(missing))}")
|
||||
for key, val in colors.items():
|
||||
if key in _REQUIRED_TOKENS and not _RGB_RE.match(str(val)):
|
||||
errors.append(f"Token '{key}' must be an RGB triplet like '37 99 235', got: {val!r}")
|
||||
return errors
|
||||
|
||||
@@ -17,6 +17,7 @@ class Settings(BaseSettings):
|
||||
|
||||
DOC_SERVICE_URL: str = "http://doc-service:8001"
|
||||
AI_SERVICE_URL: str = "http://ai-service:8010"
|
||||
STORAGE_SERVICE_URL: str = "http://storage-service:8020"
|
||||
|
||||
@field_validator("JWT_PRIVATE_KEY", "JWT_PUBLIC_KEY", mode="before")
|
||||
@classmethod
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""
|
||||
Async HTTP client for the 'config' bucket in storage-service.
|
||||
|
||||
All JSON config files (AI settings, doc settings, appearance, themes, …) are stored
|
||||
in the 'config' bucket under the storage-service. This module provides thin
|
||||
async helpers so app_config.py does not depend on the filesystem at all.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_BUCKET = "config"
|
||||
_TIMEOUT = 10.0
|
||||
|
||||
|
||||
def _url(key: str) -> str:
|
||||
return f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}/{key}"
|
||||
|
||||
|
||||
async def read_json(key: str) -> dict | None:
|
||||
"""Return parsed JSON from the config bucket, or None if the key does not exist."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(_url(key))
|
||||
if resp.status_code == 404:
|
||||
return None
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def write_json(key: str, data: dict) -> None:
|
||||
"""Serialise *data* to JSON and PUT it into the config bucket."""
|
||||
payload = json.dumps(data, indent=2).encode()
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.put(
|
||||
_url(key),
|
||||
content=payload,
|
||||
headers={"Content-Type": "application/octet-stream"},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
async def delete_key(key: str) -> None:
|
||||
"""Delete a key from the config bucket. No-op if it does not exist."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.delete(_url(key))
|
||||
if resp.status_code not in (204, 404):
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
async def list_keys(prefix: str = "") -> list[str]:
|
||||
"""List all keys in the config bucket, optionally filtered by *prefix*."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}")
|
||||
resp.raise_for_status()
|
||||
keys: list[str] = resp.json().get("keys", [])
|
||||
if prefix:
|
||||
keys = [k for k in keys if k.startswith(prefix)]
|
||||
return keys
|
||||
@@ -43,3 +43,69 @@ async def get_current_admin(
|
||||
detail="Not found",
|
||||
)
|
||||
return current_user
|
||||
|
||||
|
||||
def get_service_admin(service_id: str):
|
||||
"""
|
||||
Dependency factory that grants access to service-specific admin endpoints.
|
||||
|
||||
Access is granted if the user is a global superuser OR a member of the
|
||||
'{service_id}-admin' group. Returns 404 (not 403) to hide both the
|
||||
endpoint existence and the permission model.
|
||||
|
||||
Usage:
|
||||
@router.get("/ai")
|
||||
async def get_ai_settings(_: User = Depends(get_service_admin("ai-service"))):
|
||||
"""
|
||||
async def _dep(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> User:
|
||||
if current_user.is_superuser:
|
||||
return current_user
|
||||
if await check_plugin_access(service_id, current_user, db):
|
||||
return current_user
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
|
||||
|
||||
return _dep
|
||||
|
||||
|
||||
async def check_plugin_access(
|
||||
plugin_id: str,
|
||||
current_user: User,
|
||||
db: AsyncSession,
|
||||
) -> bool:
|
||||
"""
|
||||
Return True if the user may access the given plugin's settings.
|
||||
|
||||
Access is granted when any of these conditions holds:
|
||||
1. The user is a superuser AND the manifest allows superuser access.
|
||||
2. The user is a member of one of the groups listed in manifest.access.required_groups.
|
||||
|
||||
Returns False (not raises) so callers can decide how to respond.
|
||||
"""
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.services.service_health import get_cached_manifest
|
||||
|
||||
manifest = get_cached_manifest(plugin_id)
|
||||
if manifest is None:
|
||||
return False
|
||||
|
||||
access = manifest.get("access", {})
|
||||
|
||||
if current_user.is_superuser and access.get("allow_superuser", True):
|
||||
return True
|
||||
|
||||
for group_name in access.get("required_groups", []):
|
||||
result = await db.execute(
|
||||
select(GroupMembership)
|
||||
.join(Group, Group.id == GroupMembership.group_id)
|
||||
.where(
|
||||
Group.name == group_name,
|
||||
GroupMembership.user_id == current_user.id,
|
||||
)
|
||||
)
|
||||
if result.scalar_one_or_none() is not None:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
+12
-1
@@ -4,18 +4,27 @@ from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from app.core.app_config import seed_builtin_themes
|
||||
from app.core.config import settings
|
||||
from app.routers import admin, auth, categories_proxy, documents_proxy, groups, profile, services, users
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.routers import admin, auth, categories_proxy, documents_proxy, groups, plugins, profile, services, users
|
||||
from app.routers import settings as settings_router
|
||||
from app.routers import storage_config
|
||||
from app.services.group_bootstrap import ensure_service_admin_groups
|
||||
from app.services.service_health import check_all, health_check_loop, register_services
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
await seed_builtin_themes()
|
||||
register_services(
|
||||
doc_service_url=settings.DOC_SERVICE_URL,
|
||||
ai_service_url=settings.AI_SERVICE_URL,
|
||||
storage_service_url=settings.STORAGE_SERVICE_URL,
|
||||
)
|
||||
# Create <service-id>-admin groups for every registered service (idempotent)
|
||||
async with AsyncSessionLocal() as db:
|
||||
await ensure_service_admin_groups(db)
|
||||
# Run an initial check immediately so the first API response is accurate
|
||||
await check_all()
|
||||
task = asyncio.create_task(health_check_loop())
|
||||
@@ -44,6 +53,8 @@ app.include_router(admin.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(groups.router, prefix="/api/admin/groups", tags=["admin"])
|
||||
app.include_router(settings_router.router, prefix="/api/settings", tags=["settings"])
|
||||
app.include_router(services.router, prefix="/api/services", tags=["services"])
|
||||
app.include_router(storage_config.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(plugins.router, prefix="/api/plugins", tags=["plugins"])
|
||||
# categories_proxy MUST be registered before documents_proxy —
|
||||
# otherwise /api/documents/{path:path} swallows /api/documents/categories/*
|
||||
app.include_router(
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import DateTime, ForeignKey, String, UniqueConstraint
|
||||
from sqlalchemy import Boolean, DateTime, ForeignKey, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.database import Base
|
||||
@@ -35,6 +35,9 @@ class GroupMembership(Base):
|
||||
user_id: Mapped[str] = mapped_column(
|
||||
String, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
||||
)
|
||||
is_group_admin: Mapped[bool] = mapped_column(
|
||||
Boolean, nullable=False, default=False, server_default="false"
|
||||
)
|
||||
joined_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
|
||||
@@ -23,6 +23,8 @@ class User(Base):
|
||||
is_superuser: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
# List of service IDs pinned to the user's home dashboard.
|
||||
dashboard_app_ids: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
# User's preferred colour mode: "light", "dark", "system", or None (use admin default).
|
||||
color_mode: Mapped[str | None] = mapped_column(String, nullable=True, default=None)
|
||||
|
||||
profile: Mapped["Profile"] = relationship(
|
||||
"Profile", back_populates="user", uselist=False, cascade="all, delete-orphan"
|
||||
|
||||
@@ -9,8 +9,12 @@ import os
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import GroupMembership
|
||||
from app.models.user import User
|
||||
|
||||
DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
|
||||
@@ -35,13 +39,26 @@ _HOP_BY_HOP = frozenset([
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
def _forward_headers(request: Request, user_id: str) -> dict:
|
||||
async def _forward_headers(
|
||||
request: Request, user_id: str, is_admin: bool, db: AsyncSession
|
||||
) -> dict:
|
||||
headers = {
|
||||
k: v
|
||||
for k, v in request.headers.items()
|
||||
if k.lower() not in _HOP_BY_HOP
|
||||
}
|
||||
headers["x-user-id"] = user_id
|
||||
headers["x-user-is-admin"] = "true" if is_admin else "false"
|
||||
|
||||
mem_result = await db.execute(
|
||||
select(GroupMembership.group_id, GroupMembership.is_group_admin)
|
||||
.where(GroupMembership.user_id == user_id)
|
||||
)
|
||||
rows = mem_result.all()
|
||||
group_ids = [row[0] for row in rows]
|
||||
admin_group_ids = [row[0] for row in rows if row[1]]
|
||||
headers["x-user-groups"] = ",".join(group_ids)
|
||||
headers["x-user-admin-groups"] = ",".join(admin_group_ids)
|
||||
return headers
|
||||
|
||||
|
||||
@@ -50,10 +67,11 @@ def _forward_headers(request: Request, user_id: str) -> dict:
|
||||
async def proxy_categories(
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
path: str = "",
|
||||
) -> Response:
|
||||
url = f"/categories/{path}" if path else "/categories"
|
||||
headers = _forward_headers(request, str(current_user.id))
|
||||
headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
|
||||
body = await request.body()
|
||||
|
||||
try:
|
||||
|
||||
@@ -3,14 +3,21 @@ Proxy all /api/documents/* requests to doc-service:8001/documents/*.
|
||||
|
||||
Uses a module-level AsyncClient for connection pooling.
|
||||
Strips hop-by-hop headers that must not be forwarded.
|
||||
Injects X-User-Id and X-User-Groups headers so the doc-service
|
||||
can enforce ownership and group-sharing access without querying the
|
||||
backend database directly.
|
||||
"""
|
||||
import os
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import GroupMembership
|
||||
from app.models.user import User
|
||||
|
||||
DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
|
||||
@@ -43,13 +50,29 @@ _HOP_BY_HOP = frozenset([
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
def _forward_headers(request: Request, user_id: str) -> dict:
|
||||
async def _forward_headers(
|
||||
request: Request, user_id: str, is_admin: bool, db: AsyncSession
|
||||
) -> dict:
|
||||
headers = {
|
||||
k: v
|
||||
for k, v in request.headers.items()
|
||||
if k.lower() not in _HOP_BY_HOP
|
||||
}
|
||||
headers["x-user-id"] = user_id
|
||||
headers["x-user-is-admin"] = "true" if is_admin else "false"
|
||||
|
||||
# Inject group memberships and group-admin status so the doc-service can
|
||||
# evaluate ownership, sharing access, and group-admin permissions.
|
||||
mem_result = await db.execute(
|
||||
select(GroupMembership.group_id, GroupMembership.is_group_admin)
|
||||
.where(GroupMembership.user_id == user_id)
|
||||
)
|
||||
rows = mem_result.all()
|
||||
group_ids = [row[0] for row in rows]
|
||||
admin_group_ids = [row[0] for row in rows if row[1]]
|
||||
headers["x-user-groups"] = ",".join(group_ids)
|
||||
headers["x-user-admin-groups"] = ",".join(admin_group_ids)
|
||||
|
||||
return headers
|
||||
|
||||
|
||||
@@ -58,10 +81,11 @@ def _forward_headers(request: Request, user_id: str) -> dict:
|
||||
async def proxy_documents(
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
path: str = "",
|
||||
) -> Response:
|
||||
url = f"/documents/{path}" if path else "/documents"
|
||||
headers = _forward_headers(request, str(current_user.id))
|
||||
headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
|
||||
body = await request.body()
|
||||
|
||||
try:
|
||||
|
||||
@@ -7,7 +7,7 @@ from app.database import get_db
|
||||
from app.deps import get_current_admin
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.models.user import User
|
||||
from app.schemas.group import GroupCreate, GroupDetailOut, GroupOut, GroupUpdate, GroupMemberOut
|
||||
from app.schemas.group import GroupCreate, GroupDetailOut, GroupMemberAdminUpdate, GroupMemberOut, GroupOut, GroupUpdate
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -111,6 +111,7 @@ async def get_group(
|
||||
email=user.email,
|
||||
full_name=user.full_name,
|
||||
is_active=user.is_active,
|
||||
is_group_admin=membership.is_group_admin,
|
||||
joined_at=membership.joined_at,
|
||||
)
|
||||
for membership, user in rows
|
||||
@@ -197,6 +198,26 @@ async def add_member(
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.patch("/{group_id}/members/{user_id}/admin", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def set_member_admin(
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
body: GroupMemberAdminUpdate,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
result = await db.execute(
|
||||
select(GroupMembership).where(
|
||||
GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
|
||||
)
|
||||
)
|
||||
membership = result.scalar_one_or_none()
|
||||
if not membership:
|
||||
raise HTTPException(status_code=404, detail="User is not a member of this group")
|
||||
membership.is_group_admin = body.is_group_admin
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.delete("/{group_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def remove_member(
|
||||
group_id: str,
|
||||
|
||||
@@ -0,0 +1,125 @@
|
||||
"""
|
||||
Generic plugin proxy.
|
||||
|
||||
Feature containers advertise themselves via GET /plugin/manifest. The backend
|
||||
health-poller caches those manifests. This router exposes them to the browser
|
||||
through auth-gated endpoints so the frontend never needs to know about specific
|
||||
features.
|
||||
|
||||
Routes:
|
||||
GET /api/plugins → list accessible plugins for current user
|
||||
GET /api/plugins/{id}/manifest → cached manifest (404 if not accessible)
|
||||
GET /api/plugins/{id}/settings → proxy to feature /plugin/settings
|
||||
PATCH /api/plugins/{id}/settings → proxy to feature /plugin/settings
|
||||
"""
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import check_plugin_access, get_current_user
|
||||
from app.models.user import User
|
||||
from app.services.service_health import get_cached_manifest, get_registry, get_service_url
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_HOP_BY_HOP = frozenset([
|
||||
"connection",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"te",
|
||||
"trailers",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
"host",
|
||||
"accept-encoding",
|
||||
])
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
async def _proxy(plugin_id: str, method: str, path: str, body: bytes | None,
|
||||
content_type: str | None = None) -> Response:
|
||||
"""Forward a request to the feature container's plugin endpoint."""
|
||||
url = get_service_url(plugin_id)
|
||||
if url is None:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
|
||||
headers: dict[str, str] = {}
|
||||
if content_type:
|
||||
headers["content-type"] = content_type
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(base_url=url, timeout=30.0) as client:
|
||||
resp = await client.request(method, path, content=body, headers=headers)
|
||||
except httpx.RequestError as exc:
|
||||
raise HTTPException(status_code=502, detail=f"Plugin service unreachable: {exc}")
|
||||
|
||||
resp_headers = {k: v for k, v in resp.headers.items() if k.lower() not in _STRIP_RESPONSE}
|
||||
return Response(
|
||||
content=resp.content,
|
||||
status_code=resp.status_code,
|
||||
headers=resp_headers,
|
||||
media_type=resp.headers.get("content-type", "application/json"),
|
||||
)
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def list_plugins(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
"""Return the list of plugins the current user may access."""
|
||||
accessible = []
|
||||
for svc in get_registry():
|
||||
manifest = get_cached_manifest(svc.id)
|
||||
if manifest is None:
|
||||
continue
|
||||
if await check_plugin_access(svc.id, current_user, db):
|
||||
accessible.append({
|
||||
"id": manifest["id"],
|
||||
"name": manifest["name"],
|
||||
"icon": manifest.get("icon", "package"),
|
||||
"version": manifest.get("version", ""),
|
||||
})
|
||||
return accessible
|
||||
|
||||
|
||||
@router.get("/{plugin_id}/manifest")
|
||||
async def get_plugin_manifest(
|
||||
plugin_id: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> dict:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
manifest = get_cached_manifest(plugin_id)
|
||||
if manifest is None:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
return manifest
|
||||
|
||||
|
||||
@router.get("/{plugin_id}/settings")
|
||||
async def get_plugin_settings(
|
||||
plugin_id: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Response:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
return await _proxy(plugin_id, "GET", "/plugin/settings", None)
|
||||
|
||||
|
||||
@router.patch("/{plugin_id}/settings")
|
||||
async def update_plugin_settings(
|
||||
plugin_id: str,
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Response:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
body = await request.body()
|
||||
content_type = request.headers.get("content-type", "application/json")
|
||||
return await _proxy(plugin_id, "PATCH", "/plugin/settings", body, content_type)
|
||||
+161
-25
@@ -2,9 +2,9 @@
|
||||
Admin-only settings API for per-service runtime configuration.
|
||||
|
||||
All endpoints require the caller to be an admin (Depends(get_current_admin)).
|
||||
Config files live on the shared app_config volume (/config/).
|
||||
Config files are stored in the 'config' bucket of storage-service.
|
||||
"""
|
||||
import asyncio
|
||||
import re as _re
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
@@ -12,22 +12,32 @@ from pydantic import BaseModel
|
||||
|
||||
from app.core.app_config import (
|
||||
SYSTEM_PROMPT_SERVICES,
|
||||
AppearanceConfig,
|
||||
_merge_api_key,
|
||||
delete_theme,
|
||||
load_ai_service_config,
|
||||
load_ai_service_config_masked,
|
||||
load_all_system_prompts,
|
||||
load_all_themes,
|
||||
load_appearance_config,
|
||||
load_doc_service_config,
|
||||
load_doc_service_config_masked,
|
||||
load_theme_by_id,
|
||||
save_ai_service_config,
|
||||
save_appearance_config,
|
||||
save_doc_service_config,
|
||||
save_service_system_prompts,
|
||||
save_theme,
|
||||
validate_theme_tokens,
|
||||
)
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin
|
||||
from app.deps import get_current_admin, get_current_user, get_service_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_THEME_ID_RE = _re.compile(r"^[a-z0-9_-]{1,64}$")
|
||||
|
||||
|
||||
# ── Pydantic request bodies ────────────────────────────────────────────────────
|
||||
|
||||
@@ -53,26 +63,56 @@ class SystemPromptUpdate(BaseModel):
|
||||
user_template: str
|
||||
|
||||
|
||||
class AppearanceUpdate(BaseModel):
|
||||
theme: str
|
||||
default_mode: str
|
||||
|
||||
|
||||
class ThemeColors(BaseModel):
|
||||
primary: str
|
||||
primary_hover: str
|
||||
accent: str
|
||||
accent_hover: str
|
||||
background: str
|
||||
surface: str
|
||||
border: str
|
||||
text_primary: str
|
||||
text_muted: str
|
||||
|
||||
|
||||
class ThemeCreate(BaseModel):
|
||||
id: str
|
||||
label: str
|
||||
light: ThemeColors
|
||||
dark: ThemeColors
|
||||
|
||||
|
||||
class ThemeUpdate(BaseModel):
|
||||
label: str | None = None
|
||||
light: ThemeColors | None = None
|
||||
dark: ThemeColors | None = None
|
||||
|
||||
|
||||
# ── AI settings ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/ai")
|
||||
async def get_ai_settings(
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
return load_ai_service_config_masked()
|
||||
return await load_ai_service_config_masked()
|
||||
|
||||
|
||||
@router.patch("/ai")
|
||||
async def update_ai_settings(
|
||||
body: AIProviderUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
valid_providers = ("anthropic", "ollama", "lmstudio")
|
||||
if body.provider not in valid_providers:
|
||||
raise HTTPException(status_code=422, detail=f"provider must be one of {valid_providers}")
|
||||
|
||||
config = load_ai_service_config()
|
||||
config = await load_ai_service_config()
|
||||
config.provider = body.provider
|
||||
|
||||
# Anthropic
|
||||
@@ -101,13 +141,13 @@ async def update_ai_settings(
|
||||
body.lmstudio_api_key, config.lmstudio.api_key
|
||||
)
|
||||
|
||||
await asyncio.to_thread(save_ai_service_config, config)
|
||||
return load_ai_service_config_masked()
|
||||
await save_ai_service_config(config)
|
||||
return await load_ai_service_config_masked()
|
||||
|
||||
|
||||
@router.post("/ai/test")
|
||||
async def test_ai_connection(
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
"""Proxy a minimal chat request to ai-service to verify the connection."""
|
||||
try:
|
||||
@@ -133,23 +173,23 @@ async def test_ai_connection(
|
||||
|
||||
@router.get("/documents/limits")
|
||||
async def get_documents_limits(
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("doc-service")),
|
||||
) -> dict:
|
||||
return load_doc_service_config_masked()
|
||||
return await load_doc_service_config_masked()
|
||||
|
||||
|
||||
@router.patch("/documents/limits")
|
||||
async def update_documents_limits(
|
||||
body: LimitsUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("doc-service")),
|
||||
) -> dict:
|
||||
if body.max_pdf_mb < 1 or body.max_pdf_mb > 200:
|
||||
raise HTTPException(status_code=422, detail="max_pdf_mb must be between 1 and 200")
|
||||
|
||||
config = load_doc_service_config()
|
||||
config = await load_doc_service_config()
|
||||
config.documents.max_pdf_bytes = body.max_pdf_mb * 1024 * 1024
|
||||
await asyncio.to_thread(save_doc_service_config, config)
|
||||
return load_doc_service_config_masked()
|
||||
await save_doc_service_config(config)
|
||||
return await load_doc_service_config_masked()
|
||||
|
||||
|
||||
# ── System prompts ─────────────────────────────────────────────────────────────
|
||||
@@ -157,22 +197,118 @@ async def update_documents_limits(
|
||||
|
||||
@router.get("/system-prompts")
|
||||
async def get_system_prompts(
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
"""Return all editable system prompts, keyed by service id."""
|
||||
return await asyncio.to_thread(load_all_system_prompts)
|
||||
return await load_all_system_prompts()
|
||||
|
||||
|
||||
@router.patch("/system-prompts/{service_id}")
|
||||
async def update_system_prompt(
|
||||
service_id: str,
|
||||
body: SystemPromptUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
"""Update the system prompts for a single service."""
|
||||
if service_id not in SYSTEM_PROMPT_SERVICES:
|
||||
raise HTTPException(status_code=404, detail=f"No system prompts registered for {service_id!r}")
|
||||
await asyncio.to_thread(
|
||||
save_service_system_prompts, service_id, body.system, body.user_template
|
||||
)
|
||||
return await asyncio.to_thread(load_all_system_prompts)
|
||||
await save_service_system_prompts(service_id, body.system, body.user_template)
|
||||
return await load_all_system_prompts()
|
||||
|
||||
|
||||
# ── Appearance (global default — auth read, admin write) ───────────────────────
|
||||
|
||||
|
||||
@router.get("/appearance")
|
||||
async def get_appearance(
|
||||
_: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
config = await load_appearance_config()
|
||||
return config.model_dump()
|
||||
|
||||
|
||||
@router.patch("/appearance")
|
||||
async def update_appearance(
|
||||
body: AppearanceUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
if body.default_mode not in ("light", "dark", "system"):
|
||||
raise HTTPException(status_code=422, detail="default_mode must be 'light', 'dark', or 'system'")
|
||||
themes = await load_all_themes()
|
||||
theme_ids = {t["id"] for t in themes}
|
||||
if body.theme not in theme_ids:
|
||||
raise HTTPException(status_code=422, detail=f"Unknown theme: {body.theme!r}")
|
||||
config = AppearanceConfig(theme=body.theme, default_mode=body.default_mode)
|
||||
await save_appearance_config(config)
|
||||
return config.model_dump()
|
||||
|
||||
|
||||
# ── Theme CRUD ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/themes")
|
||||
async def list_themes(
|
||||
_: User = Depends(get_current_user),
|
||||
) -> list:
|
||||
return await load_all_themes()
|
||||
|
||||
|
||||
@router.post("/themes", status_code=201)
|
||||
async def create_theme(
|
||||
body: ThemeCreate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
if not _THEME_ID_RE.match(body.id):
|
||||
raise HTTPException(status_code=422, detail="Theme ID must match [a-z0-9_-]{1,64}")
|
||||
existing = {t["id"] for t in await load_all_themes()}
|
||||
if body.id in existing:
|
||||
raise HTTPException(status_code=400, detail=f"Theme ID already in use: {body.id!r}")
|
||||
light = body.light.model_dump()
|
||||
dark = body.dark.model_dump()
|
||||
for mode, colors in (("light", light), ("dark", dark)):
|
||||
errors = validate_theme_tokens(colors)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"{mode}: {'; '.join(errors)}")
|
||||
theme = {"id": body.id, "label": body.label, "builtin": False, "light": light, "dark": dark}
|
||||
await save_theme(theme)
|
||||
return theme
|
||||
|
||||
|
||||
@router.patch("/themes/{theme_id}")
|
||||
async def update_theme(
|
||||
theme_id: str,
|
||||
body: ThemeUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
theme = await load_theme_by_id(theme_id)
|
||||
if theme is None:
|
||||
raise HTTPException(status_code=404, detail="Theme not found")
|
||||
if theme.get("builtin"):
|
||||
raise HTTPException(status_code=400, detail="Cannot edit a built-in theme")
|
||||
if body.label is not None:
|
||||
theme["label"] = body.label
|
||||
if body.light is not None:
|
||||
light = body.light.model_dump()
|
||||
errors = validate_theme_tokens(light)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"light: {'; '.join(errors)}")
|
||||
theme["light"] = light
|
||||
if body.dark is not None:
|
||||
dark = body.dark.model_dump()
|
||||
errors = validate_theme_tokens(dark)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"dark: {'; '.join(errors)}")
|
||||
theme["dark"] = dark
|
||||
await save_theme(theme)
|
||||
return theme
|
||||
|
||||
|
||||
@router.delete("/themes/{theme_id}", status_code=204)
|
||||
async def remove_theme(
|
||||
theme_id: str,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
try:
|
||||
await delete_theme(theme_id)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="Theme not found")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""
|
||||
Admin-only endpoints for storage-service backend configuration.
|
||||
|
||||
GET /admin/storage-config — current backend driver + health
|
||||
PATCH /admin/storage-config — update backend config (no data migration)
|
||||
POST /admin/storage-config/migrate — start migration to a new backend
|
||||
GET /admin/storage-config/migrate/status — poll migration progress
|
||||
DELETE /admin/storage-config/migrate — cancel in-progress migration
|
||||
|
||||
All endpoints proxy to storage-service:8020.
|
||||
"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_STORAGE_BASE = settings.STORAGE_SERVICE_URL
|
||||
|
||||
|
||||
class BackendConfigUpdate(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
class MigrateRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
def _storage_url(path: str) -> str:
|
||||
return f"{_STORAGE_BASE}{path}"
|
||||
|
||||
|
||||
async def _proxy_get(path: str) -> dict:
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.get(_storage_url(path))
|
||||
if resp.status_code == 404:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config")
|
||||
async def get_storage_config(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Return current backend driver and health status."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url("/health"))
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.patch("/storage-config", status_code=204)
|
||||
async def update_storage_config(
|
||||
body: BackendConfigUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""
|
||||
Reconfigure the active backend without migrating data.
|
||||
Use when changing credentials for the same backend type, or reverting to local.
|
||||
To move data to a new backend, use POST /admin/storage-config/migrate instead.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.patch(
|
||||
_storage_url("/backend-config"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="Migration in progress — cannot reconfigure now")
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
@router.post("/storage-config/migrate", status_code=202)
|
||||
async def start_migration(
|
||||
body: MigrateRequest,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""
|
||||
Start an async migration to a new backend.
|
||||
|
||||
Flow: validate new backend → copy all objects → verify → switch → delete old objects.
|
||||
The old backend stays active until 100% of objects are verified on the new one.
|
||||
Poll GET /admin/storage-config/migrate/status to track progress.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
_storage_url("/migrate"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="A migration is already in progress")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config/migrate/status")
|
||||
async def migration_status(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Poll migration progress. State: idle → validating → migrating → switching → cleaning → done."""
|
||||
return await _proxy_get("/migrate/status")
|
||||
|
||||
|
||||
@router.delete("/storage-config/migrate", status_code=204)
|
||||
async def cancel_migration(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""Cancel a running migration. The old backend remains active."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.delete(_storage_url("/migrate"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="No cancellable migration in progress")
|
||||
resp.raise_for_status()
|
||||
@@ -1,10 +1,13 @@
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.models.user import User
|
||||
from app.schemas.user import DashboardPrefsOut, DashboardPrefsUpdate, UserOut
|
||||
from app.schemas.user import ColorModeUpdate, DashboardPrefsOut, DashboardPrefsUpdate, UserGroupOut, UserOut
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -29,3 +32,33 @@ async def update_preferences(
|
||||
await db.commit()
|
||||
await db.refresh(current_user)
|
||||
return DashboardPrefsOut(app_ids=current_user.dashboard_app_ids or [])
|
||||
|
||||
|
||||
@router.get("/me/groups", response_model=list[UserGroupOut])
|
||||
async def get_my_groups(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Return all groups the current user belongs to, including their admin status."""
|
||||
result = await db.execute(
|
||||
select(Group, GroupMembership.is_group_admin)
|
||||
.join(GroupMembership, GroupMembership.group_id == Group.id)
|
||||
.where(GroupMembership.user_id == current_user.id)
|
||||
.order_by(Group.name)
|
||||
)
|
||||
return [
|
||||
UserGroupOut(id=g.id, name=g.name, description=g.description, is_group_admin=is_admin)
|
||||
for g, is_admin in result.all()
|
||||
]
|
||||
|
||||
|
||||
@router.patch("/me/color-mode", response_model=UserOut)
|
||||
async def update_color_mode(
|
||||
body: ColorModeUpdate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
current_user.color_mode = body.color_mode
|
||||
await db.commit()
|
||||
await db.refresh(current_user)
|
||||
return current_user
|
||||
|
||||
@@ -18,11 +18,16 @@ class GroupMemberOut(BaseModel):
|
||||
email: str
|
||||
full_name: str | None
|
||||
is_active: bool
|
||||
is_group_admin: bool = False
|
||||
joined_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class GroupMemberAdminUpdate(BaseModel):
|
||||
is_group_admin: bool
|
||||
|
||||
|
||||
class GroupOut(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, EmailStr, Field, field_validator
|
||||
|
||||
@@ -71,6 +72,7 @@ class UserOut(BaseModel):
|
||||
# validation_alias reads is_superuser from the ORM object; the JSON key
|
||||
# in the response is the field name "is_admin" (not the alias).
|
||||
is_admin: bool = Field(validation_alias="is_superuser", default=False)
|
||||
color_mode: str | None = None
|
||||
|
||||
model_config = {"from_attributes": True, "populate_by_name": True}
|
||||
|
||||
@@ -104,6 +106,27 @@ class DashboardPrefsOut(BaseModel):
|
||||
app_ids: list[str]
|
||||
|
||||
|
||||
class ColorModeUpdate(BaseModel):
|
||||
color_mode: str
|
||||
|
||||
@field_validator("color_mode")
|
||||
@classmethod
|
||||
def validate_mode(cls, v: str) -> str:
|
||||
if v not in ("light", "dark", "system"):
|
||||
raise ValueError("color_mode must be 'light', 'dark', or 'system'")
|
||||
return v
|
||||
|
||||
|
||||
class UserGroupOut(BaseModel):
|
||||
"""A group the current user belongs to — used for the share picker."""
|
||||
id: str
|
||||
name: str
|
||||
description: str | None
|
||||
is_group_admin: bool = False
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class DashboardPrefsUpdate(BaseModel):
|
||||
app_ids: list[str] = Field(default_factory=list)
|
||||
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
"""
|
||||
Ensure that every registered service has a corresponding admin group.
|
||||
|
||||
Called once at startup after register_services(). Idempotent — safe to run
|
||||
on every restart, creates nothing if groups already exist.
|
||||
|
||||
Naming convention: "{service_id}-admin" (e.g. "doc-service-admin")
|
||||
"""
|
||||
import logging
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.group import Group
|
||||
from app.services.service_health import get_registry
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def ensure_service_admin_groups(db: AsyncSession) -> None:
|
||||
"""Create a <service-id>-admin group for each registered service if absent."""
|
||||
for svc in get_registry():
|
||||
group_name = f"{svc.id}-admin"
|
||||
result = await db.execute(select(Group).where(Group.name == group_name))
|
||||
if result.scalar_one_or_none() is not None:
|
||||
continue
|
||||
|
||||
import uuid
|
||||
group = Group(
|
||||
id=str(uuid.uuid4()),
|
||||
name=group_name,
|
||||
description=f"Administrators for the {svc.name} service.",
|
||||
)
|
||||
db.add(group)
|
||||
logger.info("[bootstrap] Created admin group %r for service %r", group_name, svc.id)
|
||||
|
||||
await db.commit()
|
||||
@@ -2,8 +2,9 @@
|
||||
Background health-checker for registered feature services.
|
||||
|
||||
Polls each service's /health endpoint every POLL_INTERVAL seconds and stores
|
||||
the result in an in-memory dict. The REST layer reads from that dict — no DB,
|
||||
no blocking calls on the request path.
|
||||
the result in an in-memory dict. Also fetches /plugin/manifest when available
|
||||
and caches it so the plugin proxy can serve it without per-request network calls.
|
||||
The REST layer reads from that dict — no DB, no blocking calls on the request path.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
@@ -35,10 +36,13 @@ _REGISTRY: list[ServiceDefinition] = []
|
||||
# id → True/False/None (None = not yet checked)
|
||||
_health: dict[str, bool | None] = {}
|
||||
|
||||
# id → plugin manifest dict, or None if the service has no plugin manifest
|
||||
_manifests: dict[str, dict | None] = {}
|
||||
|
||||
def register_services(doc_service_url: str, ai_service_url: str) -> None:
|
||||
|
||||
def register_services(doc_service_url: str, ai_service_url: str, storage_service_url: str) -> None:
|
||||
"""Called once during app startup to populate the registry from config."""
|
||||
global _REGISTRY, _health
|
||||
global _REGISTRY, _health, _manifests
|
||||
|
||||
_REGISTRY = [
|
||||
ServiceDefinition(
|
||||
@@ -48,7 +52,7 @@ def register_services(doc_service_url: str, ai_service_url: str) -> None:
|
||||
internal_url=doc_service_url,
|
||||
health_path="/health",
|
||||
app_path="/apps/documents",
|
||||
settings_path="/apps/documents/settings/admin",
|
||||
settings_path="/apps/documents/settings",
|
||||
),
|
||||
ServiceDefinition(
|
||||
id="ai-service",
|
||||
@@ -57,11 +61,21 @@ def register_services(doc_service_url: str, ai_service_url: str) -> None:
|
||||
internal_url=ai_service_url,
|
||||
health_path="/health",
|
||||
app_path="",
|
||||
settings_path="/apps/ai/settings/admin",
|
||||
settings_path="/apps/ai/settings",
|
||||
),
|
||||
ServiceDefinition(
|
||||
id="storage-service",
|
||||
name="Storage",
|
||||
description="Unified file storage. Manages all uploaded files with pluggable backends (local, S3, WebDAV).",
|
||||
internal_url=storage_service_url,
|
||||
health_path="/health",
|
||||
app_path="",
|
||||
settings_path="/admin/storage",
|
||||
),
|
||||
]
|
||||
|
||||
_health = {svc.id: None for svc in _REGISTRY}
|
||||
_manifests = {svc.id: None for svc in _REGISTRY}
|
||||
logger.info("Service registry initialised with %d services", len(_REGISTRY))
|
||||
|
||||
|
||||
@@ -88,6 +102,25 @@ async def _check_service(svc: ServiceDefinition) -> None:
|
||||
else:
|
||||
logger.warning("Service %s is now UNHEALTHY", svc.id)
|
||||
|
||||
# Opportunistically fetch plugin manifest when the service is healthy
|
||||
if healthy:
|
||||
await _fetch_manifest(svc)
|
||||
|
||||
|
||||
async def _fetch_manifest(svc: ServiceDefinition) -> None:
|
||||
"""Try to GET /plugin/manifest from the service; cache result (or None)."""
|
||||
url = f"{svc.internal_url}/plugin/manifest"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
resp = await client.get(url)
|
||||
if resp.status_code == 200:
|
||||
_manifests[svc.id] = resp.json()
|
||||
else:
|
||||
_manifests[svc.id] = None
|
||||
except Exception:
|
||||
# Service doesn't have a plugin manifest — not an error
|
||||
_manifests[svc.id] = None
|
||||
|
||||
|
||||
async def check_all() -> None:
|
||||
"""Run health checks for all registered services concurrently."""
|
||||
@@ -125,3 +158,21 @@ def get_all_statuses() -> list[dict]:
|
||||
}
|
||||
for svc in _REGISTRY
|
||||
]
|
||||
|
||||
|
||||
def get_cached_manifest(service_id: str) -> dict | None:
|
||||
"""Return the cached plugin manifest for a service, or None if unavailable."""
|
||||
return _manifests.get(service_id)
|
||||
|
||||
|
||||
def get_service_url(service_id: str) -> str | None:
|
||||
"""Return the internal URL for a registered service, or None if unknown."""
|
||||
for svc in _REGISTRY:
|
||||
if svc.id == service_id:
|
||||
return svc.internal_url
|
||||
return None
|
||||
|
||||
|
||||
def get_registry() -> list[ServiceDefinition]:
|
||||
"""Return the current service registry (always up-to-date after register_services)."""
|
||||
return _REGISTRY
|
||||
|
||||
@@ -0,0 +1,34 @@
|
||||
# 2026-04-18 — Category scopes, group admin role, and permission model
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduces three category scopes (personal / group / system), a PascalCase-with-dashes naming convention, a group-admin role on group memberships, and a full permission model for who can create, rename, and delete categories and documents.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/alembic/versions/e1f2a3b4c5d6_add_group_member_is_admin.py` — adds `is_group_admin BOOLEAN` to `group_memberships`
|
||||
- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — adds `can_delete BOOLEAN` to `document_shares` (backfill from feat/document-delete-permissions)
|
||||
- `features/doc-service/alembic/versions/0006_add_category_scope.py` — adds `scope VARCHAR(16)` and `group_id VARCHAR` to `document_categories`; data-migrates watch categories to scope='system'
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/models/group.py` — added `is_group_admin` to `GroupMembership`
|
||||
- `backend/app/schemas/group.py` — added `is_group_admin` to `GroupMemberOut`; new `GroupMemberAdminUpdate`
|
||||
- `backend/app/schemas/user.py` — added `is_group_admin` to `UserGroupOut`
|
||||
- `backend/app/routers/users.py` — `get_my_groups` now joins `GroupMembership` to include `is_group_admin`
|
||||
- `backend/app/routers/groups.py` — `get_group` includes `is_group_admin`; new `PATCH /{id}/members/{user_id}/admin` endpoint
|
||||
- `backend/app/routers/categories_proxy.py` — injects `x-user-is-admin` and `x-user-admin-groups` headers
|
||||
- `backend/app/routers/documents_proxy.py` — injects `x-user-admin-groups` header (was already injecting `x-user-is-admin`)
|
||||
- `features/doc-service/app/models/category.py` — added `scope`, `group_id` columns
|
||||
- `features/doc-service/app/schemas/category.py` — `CategoryOut` includes `scope`/`group_id`; `CategoryCreate` accepts `group_id`
|
||||
- `features/doc-service/app/deps.py` — added `get_user_is_admin`, `get_user_admin_groups`
|
||||
- `features/doc-service/app/routers/categories.py` — full rewrite: name validation regex, scope-based list/create, `_check_can_manage_cat` permission helper, scope-aware rename/delete
|
||||
- `features/doc-service/app/routers/documents.py` — `delete_document` enforces is_admin/can_delete/group-admin hierarchy; `remove_category` requires doc ownership; `assign_category` accepts group/system categories
|
||||
- `frontend/src/api/client.ts` — `CategoryOut` gains `scope`/`group_id`; `createCategory` accepts optional `groupId`; `UserGroupOut`/`GroupMemberOut` gain `is_group_admin`; new `adminSetGroupMemberAdmin()`; `ApiError` exported
|
||||
- `frontend/src/components/ManageCategoriesDialog.tsx` — categories grouped by scope; lock icons for unmanageable categories; rename/delete gated by scope permissions; inline rename error display
|
||||
- `frontend/src/components/SourcePanel.tsx` — categories shown in sections (Mine / Group name / System); scope picker on new category form; client-side name validation
|
||||
- `frontend/src/pages/AdminGroupsPage.tsx` — group admin checkbox column in members table
|
||||
- `backend/CLAUDE.md` — updated `group_memberships` model, migration chain, endpoints
|
||||
- `features/doc-service/CLAUDE.md` — updated `document_categories` model, `document_shares` model, migration chain, deps note
|
||||
@@ -0,0 +1,43 @@
|
||||
# 2026-04-18 — Doc Service Redesign: Scalable UX + Group-Based Sharing
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Complete redesign of the document management UX for scale (10 → 100 000 documents, 2 → 1 000 categories) and group-based document sharing. Replaced the monolithic DocumentsPage with a three-column layout (Sidebar + SourcePanel + main), a slide-over detail panel, a filter chip system, multi-file upload queue, and bulk actions. Added the full backend sharing stack: `document_shares` table, share CRUD endpoints, a shared-with-me view, and X-User-Groups header injection in the gateway proxy.
|
||||
|
||||
---
|
||||
|
||||
## Files Added
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `features/doc-service/app/models/document_share.py` | DocumentShare ORM model (document_id, group_id, shared_by_user_id) |
|
||||
| `features/doc-service/app/schemas/share.py` | DocumentShareOut, DocumentShareCreate, SharedDocumentOut schemas |
|
||||
| `features/doc-service/alembic/versions/0004_add_document_shares.py` | Migration creating document_shares table with indexes |
|
||||
| `frontend/src/components/SourcePanel.tsx` | Left panel (240px): views (All/Mine/Shared) + searchable category tree + new category form |
|
||||
| `frontend/src/components/ManageCategoriesDialog.tsx` | Category CRUD modal (inline rename, delete with confirm, search) |
|
||||
| `frontend/src/components/DocumentSlideOver.tsx` | Right slide-over (480px): metadata, inline title edit, type picker, AI suggestions, categories combobox, tags, sharing section, raw text, actions |
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `features/doc-service/app/models/__init__.py` | Import DocumentShare |
|
||||
| `features/doc-service/app/deps.py` | Added `get_user_groups` dependency (reads X-User-Groups header) |
|
||||
| `features/doc-service/app/schemas/document.py` | Added `share_count: int = 0` to DocumentOut |
|
||||
| `features/doc-service/app/routers/documents.py` | Complete rewrite: added share CRUD endpoints, shared-with-me endpoint, N+1-safe share_count, recipient download access, X-User-Groups enforcement |
|
||||
| `features/doc-service/STATUS.md` | Added sharing section, migration 0004, updated future work |
|
||||
| `backend/app/routers/documents_proxy.py` | Injects X-User-Groups header (queries GroupMembership per request) |
|
||||
| `backend/app/routers/categories_proxy.py` | Same X-User-Groups injection pattern |
|
||||
| `backend/app/routers/users.py` | Added GET /me/groups endpoint |
|
||||
| `backend/app/schemas/user.py` | Added UserGroupOut schema |
|
||||
| `backend/STATUS.md` | Added /me/groups endpoint |
|
||||
| `frontend/src/api/client.ts` | Added share_count to DocumentOut, SharedDocumentOut, DocumentShareOut, listSharedWithMe, getDocumentShares, addDocumentShare, removeDocumentShare, getMyGroups |
|
||||
| `frontend/src/components/Sidebar.tsx` | Removed per-category NavLinks; Documents is now a single NavLink under Apps |
|
||||
| `frontend/src/components/AppShell.tsx` | Renders SourcePanel between Sidebar and main on /apps/documents route |
|
||||
| `frontend/src/pages/DocumentsPage.tsx` | Complete rewrite: three-panel layout, view param (all/mine/shared), smart polling, drag-and-drop, multi-file upload queue, filter chip system, bulk actions bar |
|
||||
| `frontend/STATUS.md` | Complete rewrite reflecting all new components and patterns |
|
||||
| `CLAUDE.md` | Updated file tree, Database Models (DocumentShare), Migration chains (0004), API endpoints (shares, shared-with-me, /me/groups), TanStack Query keys, request flow diagram |
|
||||
@@ -0,0 +1,25 @@
|
||||
# 2026-04-18 — Document delete permissions + three-dots menu fix
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Added a proper permission model for document deletion: owners and admins can always delete; group members can delete only when the share was explicitly granted `can_delete=true`. Fixed silent delete failures (watch docs returning 404 with no user feedback) and fixed the three-dots context menu being clipped by `overflow-hidden` on the table container.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Added
|
||||
- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — migration: adds `can_delete BOOLEAN NOT NULL DEFAULT false` to `document_shares`
|
||||
|
||||
### Modified
|
||||
- `features/doc-service/app/models/document_share.py` — added `can_delete: Mapped[bool]` column
|
||||
- `features/doc-service/app/schemas/share.py` — added `can_delete` to `DocumentShareOut` and `DocumentShareCreate`; added `viewer_can_delete` to `SharedDocumentOut`
|
||||
- `features/doc-service/app/schemas/document.py` — added `viewer_can_delete: bool = False` to `DocumentOut`
|
||||
- `features/doc-service/app/deps.py` — added `get_user_is_admin()` dep reading `x-user-is-admin` header
|
||||
- `features/doc-service/app/routers/documents.py` — added `_get_deletable_doc_ids()` helper; updated list/get/delete endpoints with permission logic; updated `add_share` to store `can_delete`; updated shared-with-me to include `viewer_can_delete`
|
||||
- `backend/app/routers/documents_proxy.py` — `_forward_headers()` now injects `x-user-is-admin` header
|
||||
- `frontend/src/api/client.ts` — `DocumentOut`: added `viewer_can_delete`; `DocumentShareOut`: added `can_delete`; `addDocumentShare`: accepts `canDelete` param
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — `RowActionsMenu`: replaced absolute dropdown with `createPortal` to fix clipping; delete button now uses `doc.viewer_can_delete`; added `onError` handler for silent failures
|
||||
- `frontend/src/components/DocumentSlideOver.tsx` — sharing section: shows trash icon badge on shares with `can_delete=true`; added "Allow group members to delete" checkbox before group picker; delete button uses `doc.viewer_can_delete`
|
||||
- `features/doc-service/CLAUDE.md` — updated `document_shares` table docs + migration chain
|
||||
- `backend/CLAUDE.md` — noted `x-user-is-admin` header injection
|
||||
@@ -0,0 +1,44 @@
|
||||
# 2026-04-18 — Generic Plugin Architecture + Watch Directory Feature
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented a generic plugin/extension infrastructure that allows feature containers to self-describe their settings via a manifest contract, with no feature-specific code required in the backend or frontend. Built the watch-directory feature entirely inside the doc-service container as the first plugin consumer.
|
||||
|
||||
## Files Added
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `backend/app/routers/plugins.py` | Generic plugin proxy: `GET/PATCH /api/plugins`, `/api/plugins/{id}/manifest`, `/api/plugins/{id}/settings` |
|
||||
| `frontend/src/components/PluginSchemaForm.tsx` | JSON Schema → React form renderer (boolean/string/number/readOnly) |
|
||||
| `frontend/src/pages/PluginSettingsPage.tsx` | Generic plugin settings page driven by manifest |
|
||||
| `features/doc-service/app/routers/plugin.py` | Doc-service plugin endpoints: `/plugin/manifest`, `/plugin/settings` |
|
||||
| `features/doc-service/app/services/file_watcher.py` | watchdog-based PDF watcher with startup scan, folder-to-category mapping, no-remove policy |
|
||||
| `features/doc-service/alembic/versions/0003_add_watch_columns.py` | Migration: source, watch_path, suggested_folder, suggested_filename |
|
||||
| `dev-watch/.gitkeep` | Dev bind-mount directory for local file watcher testing |
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `backend/app/services/service_health.py` | Also fetches and caches `/plugin/manifest` from healthy services |
|
||||
| `backend/app/deps.py` | Added `check_plugin_access(plugin_id, user, db)` helper |
|
||||
| `backend/app/main.py` | Mounted `/api/plugins` router |
|
||||
| `frontend/src/api/client.ts` | Added plugin API functions and suggestion confirm/reject functions; extended `DocumentOut` with new fields |
|
||||
| `frontend/src/components/Sidebar.tsx` | Added dynamic "Extensions" section populated from `/api/plugins` |
|
||||
| `frontend/src/App.tsx` | Added `/settings/plugins/:id` route |
|
||||
| `features/doc-service/app/models/document.py` | Added 4 new columns: source, watch_path, suggested_folder, suggested_filename |
|
||||
| `features/doc-service/app/schemas/document.py` | Exposed 4 new fields in `DocumentOut` |
|
||||
| `features/doc-service/app/services/config_reader.py` | Added storage config defaults, `get_storage_config()`, `save_storage_config()` |
|
||||
| `features/doc-service/app/routers/documents.py` | Watch-user visibility (`OR user_id = "watch"`); 4 suggestion endpoints |
|
||||
| `features/doc-service/app/routers/categories.py` | Watch-owned categories included in list |
|
||||
| `features/doc-service/app/main.py` | Lifespan watcher start/stop; plugin router mounted |
|
||||
| `features/doc-service/pyproject.toml` | Added `watchdog>=4.0` |
|
||||
| `features/doc-service/Dockerfile` | Pre-create `/data/watch` |
|
||||
| `docker-compose.yml` | Added `watch_data` named volume; mounted to doc-service |
|
||||
| `docker-compose.dev.yml` | Dev bind-mount `./dev-watch:/data/watch` |
|
||||
| `CLAUDE.md` | Updated all affected sections (models, migrations, endpoints, routes, tree, query keys, volumes) |
|
||||
| `backend/STATUS.md` | Plugin system section added |
|
||||
| `features/doc-service/STATUS.md` | Watch feature, plugin endpoints, migration 0003, updated architecture diagram |
|
||||
| `frontend/STATUS.md` | Extensions sidebar, PluginSchemaForm, PluginSettingsPage, new API functions |
|
||||
@@ -0,0 +1,24 @@
|
||||
# 2026-04-18 — Service admin groups + combined settings pages
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduced per-service admin groups that are auto-created at startup, consolidated doc-service and AI-service settings each onto a single page, and collapsed the dual "Settings + Extension" app card buttons into one Settings button visible to admins and service-group members.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/app/services/group_bootstrap.py` — Idempotent startup task: creates `{service_id}-admin` group for every registered service if absent.
|
||||
- `features/ai-service/app/routers/plugin.py` — `GET /plugin/manifest` for ai-service (exposes access rules: `ai-service-admin` group).
|
||||
- `frontend/src/pages/DocServiceSettingsPage.tsx` — Combined doc-service settings page: Upload Limits + Watch Directory (rendered via `PluginSchemaForm`).
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/main.py` — Lifespan now calls `ensure_service_admin_groups(db)` after `register_services()`.
|
||||
- `backend/app/deps.py` — Added `get_service_admin(service_id)` factory dependency: grants access to superusers or `{service_id}-admin` group members; returns 404 otherwise.
|
||||
- `backend/app/routers/settings.py` — AI settings (`/ai`, `/ai/test`, `/system-prompts`) and doc limits (`/documents/limits`) now use `get_service_admin(...)` instead of `get_current_admin` — service group members can access them.
|
||||
- `backend/app/services/service_health.py` — `settings_path` for doc-service changed to `/apps/documents/settings`; ai-service to `/apps/ai/settings` (removed `/admin` suffix).
|
||||
- `features/ai-service/app/main.py` — Mounts new `plugin.router` so backend poller can discover ai-service manifest.
|
||||
- `frontend/src/App.tsx` — Added `ServiceAdminRoute` component (checks token + is_admin OR plugin list contains serviceId). Updated doc/AI settings routes to new paths under `ServiceAdminRoute`.
|
||||
- `frontend/src/pages/AppsPage.tsx` — Replaced two-button layout (Settings + Extension) with single Settings button; visible when `user.is_admin || pluginIds.has(svc.id)`.
|
||||
- `backend/STATUS.md`, `frontend/STATUS.md`, `CLAUDE.md` — Updated to reflect all changes above.
|
||||
@@ -0,0 +1,12 @@
|
||||
# 2026-04-19 — Merge checklist update + admin delete bug fix
|
||||
|
||||
**Timestamp:** 2026-04-19T00:15:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Updated `tests/MERGE_CHECKLIST.md` with all new tests for the two recently merged features (document delete permissions and category scopes / group-admin role). While running the new test 12.16b, discovered and fixed a bug where the doc-service delete endpoint returned 404 for admins deleting non-owned documents.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
- **Modified** `tests/MERGE_CHECKLIST.md` — added 18 new tests: 4.9–4.10 (group admin role), 12.16b–12.16e (delete permissions), 13.11–13.14 (can_delete sharing), 14.7–14.17 (category scopes, PascalCase naming), 19.11 (three-dots portal fix); updated 12.16 and 14.5 descriptions
|
||||
- **Modified** `features/doc-service/app/routers/documents.py` — fixed `delete_document` to bypass group-membership filter for admins; previously admins got 404 on any document they didn't own or that wasn't a watch doc
|
||||
@@ -0,0 +1,60 @@
|
||||
# 2026-04-20 — Dedicated storage-service with pluggable backends
|
||||
|
||||
**Timestamp:** 2026-04-20T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduced a dedicated `storage-service` container (port 8020) as the single file/blob persistence layer for the entire stack. All services now route file and config I/O through this service's HTTP API. The service supports pluggable storage backends (local filesystem by default; S3-compatible and WebDAV built in) with a zero-data-loss migration flow. The `doc_data` and `app_config` Docker volumes were removed.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/storage-service/app/main.py` — FastAPI app, lifespan (backend init)
|
||||
- `features/storage-service/app/core/config.py` — Settings (DATA_DIR, STORAGE_BACKEND, S3_*, WEBDAV_*)
|
||||
- `features/storage-service/app/routers/health.py` — GET /health
|
||||
- `features/storage-service/app/routers/objects.py` — PUT/GET/DELETE /objects/{bucket}/{key:path}, GET /objects/{bucket}
|
||||
- `features/storage-service/app/routers/migrate.py` — POST/GET/DELETE /migrate, PATCH /backend-config
|
||||
- `features/storage-service/app/services/backend_manager.py` — Driver factory, singleton, atomic switch
|
||||
- `features/storage-service/app/services/migration.py` — Async migration: copy → verify → switch → cleanup
|
||||
- `features/storage-service/app/services/backends/base.py` — AbstractStorageBackend ABC
|
||||
- `features/storage-service/app/services/backends/local.py` — LocalFSBackend (path traversal guard)
|
||||
- `features/storage-service/app/services/backends/s3.py` — S3Backend (aiobotocore, endpoint_url configurable)
|
||||
- `features/storage-service/app/services/backends/webdav.py` — WebDAVBackend (aiohttp + defusedxml)
|
||||
- `features/storage-service/scripts/start.sh` — prod uvicorn start
|
||||
- `features/storage-service/scripts/start_dev.sh` — dev uvicorn --reload start
|
||||
- `features/storage-service/pyproject.toml` — Dependencies
|
||||
- `features/storage-service/Dockerfile` — python:3.12-slim, non-root user 1001, port 8020
|
||||
- `features/storage-service/CLAUDE.md` — API reference, bucket docs, driver docs
|
||||
- `features/storage-service/STATUS.md` — Service status
|
||||
- `backend/app/core/config_storage.py` — Thin async helpers: read_json/write_json/delete_key/list_keys
|
||||
- `backend/app/routers/storage_config.py` — Admin proxy endpoints for storage config + migration
|
||||
- `features/doc-service/alembic/versions/0008_rename_file_path_to_storage_key.py` — DB migration
|
||||
- `frontend/src/pages/StorageAdminPage.tsx` — Admin UI: backend status, driver form, migration progress
|
||||
- `tests/storage-service_tests.md` — §20 storage-service test suite
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — Added storage-service, storage_data volume; removed doc_data, app_config; added depends_on service_healthy
|
||||
- `docker-compose.dev.yml` — Added storage-service dev override
|
||||
- `backend/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `backend/app/core/app_config.py` — Full async rewrite using config_storage HTTP helpers (no filesystem)
|
||||
- `backend/app/routers/settings.py` — Removed all asyncio.to_thread wrappers; direct await calls
|
||||
- `backend/app/main.py` — Register storage_config router; update register_services call
|
||||
- `backend/app/services/service_health.py` — Register storage-service
|
||||
- `features/doc-service/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `features/doc-service/app/models/document.py` — file_path → storage_key
|
||||
- `features/doc-service/app/services/storage.py` — Complete rewrite: HTTP client calls to storage-service
|
||||
- `features/doc-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `features/doc-service/app/services/file_watcher.py` — Uses save_upload() → storage-service
|
||||
- `features/doc-service/app/routers/documents.py` — storage_key refs, pdfplumber(io.BytesIO), streaming from storage-service
|
||||
- `features/ai-service/app/core/config.py` — Added STORAGE_SERVICE_URL; removed CONFIG_PATH
|
||||
- `features/ai-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `frontend/src/api/client.ts` — Added StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions
|
||||
- `frontend/src/App.tsx` — Added /admin/storage route (AdminRoute → StorageAdminPage)
|
||||
- `tests/ALL_TESTS.md` — Updated to 20 feature areas; added §20 storage-service tests
|
||||
- `CLAUDE.md` — Added storage-service to Services/Volumes/Networks tables; storage enforcement rule; §20 test file
|
||||
- `backend/CLAUDE.md` — Added config_storage.py, storage_config.py to tree; added admin storage endpoints
|
||||
- `frontend/CLAUDE.md` — Added StorageAdminPage to tree; added /admin/storage route
|
||||
- `features/doc-service/CLAUDE.md` — Updated storage.py description; file_path → storage_key; added migration 0008
|
||||
- `features/ai-service/CLAUDE.md` — Added config_reader.py description
|
||||
- `backend/STATUS.md` — Added storage-config endpoints; updated settings persistence note
|
||||
- `frontend/STATUS.md` — Added /admin/storage route; added StorageAdminPage description
|
||||
@@ -0,0 +1 @@
|
||||
# Watch directory for development testing
|
||||
@@ -29,8 +29,14 @@ services:
|
||||
volumes:
|
||||
- ./features/ai-service:/app
|
||||
|
||||
storage-service:
|
||||
command: sh scripts/start_dev.sh
|
||||
volumes:
|
||||
- ./features/storage-service:/app
|
||||
|
||||
doc-service:
|
||||
command: sh scripts/start_dev.sh
|
||||
env_file: ./features/doc-service/.env
|
||||
volumes:
|
||||
- ./features/doc-service:/app
|
||||
- ./dev-watch:/data/watch # bind-mount local folder for easy testing
|
||||
|
||||
+34
-11
@@ -19,6 +19,27 @@ services:
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── Storage service (unified blob storage) ──────────────────────────────────
|
||||
storage-service:
|
||||
build:
|
||||
context: ./features/storage-service
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
STORAGE_BACKEND: local
|
||||
DATA_DIR: /data/storage
|
||||
volumes:
|
||||
- storage_data:/data/storage
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8020/health')\""]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── Backend (management) ────────────────────────────────────────────────────
|
||||
backend:
|
||||
build:
|
||||
@@ -32,11 +53,12 @@ services:
|
||||
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
|
||||
DOC_SERVICE_URL: http://doc-service:8001
|
||||
AI_SERVICE_URL: http://ai-service:8010
|
||||
volumes:
|
||||
- app_config:/config
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
@@ -49,9 +71,10 @@ services:
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
CONFIG_PATH: /config/ai_service_config.json
|
||||
volumes:
|
||||
- app_config:/config
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
depends_on:
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
@@ -65,17 +88,17 @@ services:
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
|
||||
DATA_DIR: /data/documents
|
||||
CONFIG_PATH: /config/doc_service_config.json
|
||||
AI_SERVICE_URL: http://ai-service:8010
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
volumes:
|
||||
- doc_data:/data/documents
|
||||
- app_config:/config
|
||||
- watch_data:/data/watch
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
ai-service:
|
||||
condition: service_started
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
@@ -97,8 +120,8 @@ services:
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
doc_data: # PDF files persisted across restarts
|
||||
app_config: # Per-service runtime config JSON files
|
||||
storage_data: # All file/blob storage — managed by storage-service (documents + config)
|
||||
watch_data: # Watch directory — bind-mount your NAS/Nextcloud here via docker-compose.override.yml
|
||||
|
||||
networks:
|
||||
# backend-net: db ↔ backend ↔ doc-service. No host ports bound.
|
||||
|
||||
@@ -0,0 +1,51 @@
|
||||
# ai-service — Claude context
|
||||
|
||||
AI provider intermediary, port 8010 (internal only — never proxied to the browser). Accepts chat requests from `doc-service` (and potentially other callers). Manages a priority queue and abstracts over multiple AI providers (Anthropic, Ollama/LM Studio). See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
features/ai-service/
|
||||
├── app/
|
||||
│ ├── main.py ← FastAPI, queue worker lifespan
|
||||
│ ├── core/
|
||||
│ │ └── config.py ← Settings via pydantic-settings
|
||||
│ ├── providers/
|
||||
│ │ ├── base.py ← AIProvider abstract class
|
||||
│ │ ├── anthropic_provider.py ← Anthropic API integration
|
||||
│ │ └── openai_compat.py ← Ollama / LM Studio compatibility
|
||||
│ ├── routers/
|
||||
│ │ ├── chat.py ← POST /chat (sync, NORMAL priority queue)
|
||||
│ │ ├── health.py ← GET /health
|
||||
│ │ ├── queue.py ← GET /queue/status, /pause, /resume, /cancel/{id}
|
||||
│ │ └── plugin.py ← GET /plugin/manifest (access rules for ai-service-admin group)
|
||||
│ └── services/
|
||||
│ ├── queue.py ← Priority queue (CRITICAL > HIGH > NORMAL)
|
||||
│ └── config_reader.py ← Reads ai_service_config.json from storage-service config bucket (30 s TTL cache)
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints (internal only)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/chat` | Chat request (queued at NORMAL priority) |
|
||||
| GET | `/health` | Health check |
|
||||
| GET | `/queue/status` | Queue state |
|
||||
| POST | `/queue/pause` | Pause queue |
|
||||
| POST | `/queue/resume` | Resume queue |
|
||||
| POST | `/queue/cancel/{job_id}` | Cancel job |
|
||||
| GET | `/plugin/manifest` | Plugin manifest (access rules for ai-service-admin group) |
|
||||
|
||||
These endpoints are only reachable on `backend-net`. The backend does not expose them to the browser.
|
||||
|
||||
---
|
||||
|
||||
## Note on timeout and retry configuration
|
||||
|
||||
Caller-side timeout and retry settings live in `features/doc-service/app/services/ai_client.py` — see `features/doc-service/CLAUDE.md` for the values.
|
||||
@@ -15,8 +15,7 @@ FROM python:3.12-slim
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# Pre-create the config directory with correct ownership
|
||||
RUN mkdir -p /config && chown -R appuser:appuser /config
|
||||
# No filesystem directories needed — all config goes through storage-service.
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -3,7 +3,7 @@ from pydantic_settings import BaseSettings
|
||||
|
||||
class Settings(BaseSettings):
|
||||
PROJECT_NAME: str = "ai-service"
|
||||
CONFIG_PATH: str = "/config/ai_service_config.json"
|
||||
STORAGE_SERVICE_URL: str = "http://storage-service:8020"
|
||||
|
||||
model_config = {"env_file": ".env", "extra": "ignore"}
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ from contextlib import asynccontextmanager
|
||||
from fastapi import FastAPI
|
||||
|
||||
from app.core.config import settings
|
||||
from app.routers import chat, health
|
||||
from app.routers import chat, health, plugin
|
||||
from app.routers import queue as queue_router
|
||||
from app.services.config_reader import load_ai_config
|
||||
from app.services.queue import queue_service
|
||||
@@ -33,3 +33,4 @@ app = FastAPI(title=settings.PROJECT_NAME, lifespan=lifespan)
|
||||
app.include_router(chat.router, tags=["chat"])
|
||||
app.include_router(health.router, tags=["health"])
|
||||
app.include_router(queue_router.router)
|
||||
app.include_router(plugin.router, tags=["plugin"])
|
||||
|
||||
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Plugin manifest endpoint for the AI service.
|
||||
|
||||
Exposes GET /plugin/manifest so the backend health-poller can discover the
|
||||
service's access rules and register it in the plugin system.
|
||||
|
||||
No settings schema is exposed here — the AI service settings are complex
|
||||
(provider selection, conditional fields) and are rendered by a bespoke page
|
||||
rather than the generic PluginSchemaForm.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_MANIFEST = {
|
||||
"id": "ai-service",
|
||||
"name": "AI Service",
|
||||
"icon": "cpu",
|
||||
"version": "1.0",
|
||||
"access": {
|
||||
"allow_superuser": True,
|
||||
"required_groups": ["ai-service-admin"],
|
||||
},
|
||||
# No settings_schema — the frontend uses a custom settings page
|
||||
"settings_schema": None,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/plugin/manifest")
|
||||
async def get_manifest() -> dict:
|
||||
return _MANIFEST
|
||||
@@ -1,5 +1,5 @@
|
||||
"""
|
||||
Reads ai_service_config.json from the shared config volume.
|
||||
Reads ai_service_config.json from the storage-service config bucket.
|
||||
30-second TTL cache + env var overrides (dev credentials stay out of git).
|
||||
|
||||
Env var overrides (all optional):
|
||||
@@ -8,15 +8,17 @@ Env var overrides (all optional):
|
||||
OLLAMA_BASE_URL, OLLAMA_MODEL, OLLAMA_API_KEY
|
||||
ANTHROPIC_API_KEY, ANTHROPIC_MODEL
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from copy import deepcopy
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
_CONFIG_KEY = "ai_service_config.json"
|
||||
|
||||
_DEFAULT_CONFIG: dict = {
|
||||
"provider": "lmstudio",
|
||||
"timeout_seconds": 60,
|
||||
@@ -31,12 +33,18 @@ _cache_at: float = 0.0
|
||||
_CACHE_TTL = 30.0
|
||||
|
||||
|
||||
def _read_config_sync() -> dict:
|
||||
path = Path(settings.CONFIG_PATH)
|
||||
if not path.exists():
|
||||
return _apply_env_overrides(deepcopy(_DEFAULT_CONFIG))
|
||||
with open(path) as f:
|
||||
return _apply_env_overrides(json.load(f))
|
||||
def _storage_url() -> str:
|
||||
return f"{settings.STORAGE_SERVICE_URL}/objects/config/{_CONFIG_KEY}"
|
||||
|
||||
|
||||
async def _fetch_config() -> dict:
|
||||
"""Fetch config from storage-service. Returns defaults if not found."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url())
|
||||
if resp.status_code == 404:
|
||||
return deepcopy(_DEFAULT_CONFIG)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
def _apply_env_overrides(config: dict) -> dict:
|
||||
@@ -75,7 +83,8 @@ async def load_ai_config() -> dict:
|
||||
now = time.monotonic()
|
||||
if _cache is not None and (now - _cache_at) < _CACHE_TTL:
|
||||
return _cache
|
||||
data = await asyncio.to_thread(_read_config_sync)
|
||||
raw = await _fetch_config()
|
||||
data = _apply_env_overrides(raw)
|
||||
_cache = data
|
||||
_cache_at = now
|
||||
return data
|
||||
|
||||
@@ -0,0 +1,184 @@
|
||||
# doc-service — Claude context
|
||||
|
||||
PDF extraction microservice, port 8001 (internal). Shares the same PostgreSQL instance as the backend. Receives proxied requests from `backend:8000`, which injects `x-user-id` and `x-user-groups` headers — doc-service trusts these headers directly. Calls `ai-service:8010` for document classification. All file/blob storage goes through `storage-service:8020` — no files are written directly to the filesystem. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
All commands run inside Docker — never on the host.
|
||||
|
||||
```bash
|
||||
docker compose exec doc-service alembic revision --autogenerate -m "describe change"
|
||||
docker compose exec doc-service alembic upgrade head
|
||||
docker compose exec doc-service alembic downgrade -1
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
features/doc-service/
|
||||
├── app/
|
||||
│ ├── main.py ← FastAPI, lifespan (file watcher start/stop)
|
||||
│ ├── database.py ← Same PostgreSQL instance as backend
|
||||
│ ├── deps.py ← get_user_id, get_user_groups, get_user_is_admin, get_user_admin_groups (injected headers)
|
||||
│ ├── models/
|
||||
│ │ ├── document.py ← Document model
|
||||
│ │ ├── category.py ← DocumentCategory model
|
||||
│ │ ├── category_assignment.py ← CategoryAssignment (composite PK)
|
||||
│ │ └── document_share.py ← DocumentShare model (group-based sharing)
|
||||
│ ├── schemas/
|
||||
│ │ ├── document.py ← DocumentOut, DocumentPage, DocumentStatusOut, etc.
|
||||
│ │ ├── category.py ← CategoryOut, CategoryCreate, CategoryUpdate
|
||||
│ │ └── share.py ← DocumentShareOut, DocumentShareCreate, SharedDocumentOut
|
||||
│ ├── routers/
|
||||
│ │ ├── documents.py ← Full CRUD + file serving + reprocess + suggestions + sharing
|
||||
│ │ ├── categories.py ← Category CRUD (includes watch-owned categories)
|
||||
│ │ └── plugin.py ← GET /plugin/manifest, GET+PATCH /plugin/settings
|
||||
│ └── services/
|
||||
│ ├── storage.py ← Storage client: save_upload/download_file/delete_file → storage-service:8020 documents bucket
|
||||
│ ├── ai_client.py ← classify_document() → ai-service:8010/chat
|
||||
│ ├── config_reader.py ← Config load/save via storage-service config bucket (doc_service_config.json)
|
||||
│ └── file_watcher.py ← watchdog-based PDF watcher + startup scan + ingestion
|
||||
├── alembic/versions/ ← Migration chain
|
||||
│ ├── 0003_add_watch_columns.py ← source, watch_path, suggested_folder, suggested_filename
|
||||
│ ├── 0004_add_document_shares.py ← document_shares table (group-based sharing)
|
||||
│ └── 0008_rename_file_path_to_storage_key.py ← file_path → storage_key; strips /data/documents/ prefix from existing rows
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Models
|
||||
|
||||
### `documents`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | |
|
||||
| `user_id` | String | indexed | not FK — trusts x-user-id header |
|
||||
| `filename` | String | NOT NULL | |
|
||||
| `storage_key` | String | NOT NULL | storage-service key: `{user_id}/{doc_id}.pdf` (documents bucket) |
|
||||
| `file_size` | Integer | NOT NULL | bytes |
|
||||
| `status` | String | default="pending" | pending / processing / done / failed |
|
||||
| `title` | String(500) | nullable | AI-extracted |
|
||||
| `document_type` | String | nullable | invoice / bill / receipt / order / expense / revenue / unknown |
|
||||
| `raw_text` | Text | nullable | first 500 k chars |
|
||||
| `extracted_data` | Text | nullable | JSON string |
|
||||
| `tags` | Text | nullable | JSON array string |
|
||||
| `error_message` | String(500) | nullable | |
|
||||
| `created_at` | DateTime(tz) | server_default=now() | |
|
||||
| `processed_at` | DateTime(tz) | nullable | |
|
||||
| `source` | String(16) | default="upload" | "upload" or "watch" |
|
||||
| `watch_path` | String | nullable | original absolute path in watch directory |
|
||||
| `suggested_folder` | String(128) | nullable | AI-suggested category (pending user confirm) |
|
||||
| `suggested_filename` | String(500) | nullable | AI-suggested title/rename (pending user confirm) |
|
||||
|
||||
### `document_categories`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | |
|
||||
| `user_id` | String | indexed | owner; "watch" for system categories |
|
||||
| `name` | String(128) | NOT NULL | PascalCase-with-dashes convention enforced on create/rename |
|
||||
| `scope` | String(16) | NOT NULL, default="personal" | "personal" / "group" / "system" |
|
||||
| `group_id` | String | nullable, indexed | set when scope="group" |
|
||||
| `created_at` | DateTime(tz) | server_default=now() | |
|
||||
|
||||
### `document_category_assignments` (composite PK)
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `document_id` | String | PK + FK→documents.id CASCADE |
|
||||
| `category_id` | String | PK + FK→document_categories.id CASCADE |
|
||||
|
||||
### `document_shares`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | |
|
||||
| `document_id` | String | indexed, NOT NULL | not FK — trusts proxy |
|
||||
| `group_id` | String | indexed, NOT NULL | group from backend |
|
||||
| `shared_by_user_id` | String | NOT NULL | owner who shared |
|
||||
| `can_delete` | Boolean | NOT NULL, default=false | allows group members to delete the doc |
|
||||
| `created_at` | DateTime(tz) | server_default=now() | |
|
||||
|
||||
Unique constraint: `(document_id, group_id)`
|
||||
|
||||
### Migration chain
|
||||
|
||||
| Rev ID | Slug |
|
||||
|--------|------|
|
||||
| `0001` | `create_doc_tables` |
|
||||
| `0002` | `add_document_title` |
|
||||
| `0003` | `add_watch_columns` |
|
||||
| `0004` | `add_document_shares` |
|
||||
| `0005` | `add_share_can_delete` |
|
||||
| `0006` | `add_category_scope` |
|
||||
| `0007` | `capitalize_system_category_names` |
|
||||
| `0008` | `rename_file_path_to_storage_key` |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints (internal — reached via backend proxy)
|
||||
|
||||
All these endpoints are proxied from `backend:8000`. The backend injects `x-user-id` and `x-user-groups` before forwarding.
|
||||
|
||||
### Documents
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/documents/upload` | Upload PDF (202, background processing) |
|
||||
| GET | `/documents` | Paginated list (filterable: search, status, type, category, sort) |
|
||||
| GET | `/documents/{id}` | Document detail |
|
||||
| GET | `/documents/{id}/status` | Processing status only |
|
||||
| PATCH | `/documents/{id}/type` | Update document type |
|
||||
| PATCH | `/documents/{id}/tags` | Update tags |
|
||||
| PATCH | `/documents/{id}/title` | Update title |
|
||||
| POST | `/documents/{id}/reprocess` | Re-run AI extraction |
|
||||
| DELETE | `/documents/{id}` | Delete document (204) |
|
||||
| GET | `/documents/{id}/file` | Download PDF (streaming) |
|
||||
| POST | `/documents/{id}/categories/{cat_id}` | Assign category |
|
||||
| DELETE | `/documents/{id}/categories/{cat_id}` | Remove category |
|
||||
| POST | `/documents/{id}/suggestions/folder/confirm` | Confirm AI folder suggestion |
|
||||
| POST | `/documents/{id}/suggestions/folder/reject` | Reject AI folder suggestion |
|
||||
| POST | `/documents/{id}/suggestions/filename/confirm` | Confirm AI filename suggestion |
|
||||
| POST | `/documents/{id}/suggestions/filename/reject` | Reject AI filename suggestion |
|
||||
| GET | `/documents/shared-with-me` | Documents shared with current user via their groups |
|
||||
| GET | `/documents/{id}/shares` | List groups the document is shared with (owner only) |
|
||||
| POST | `/documents/{id}/shares` | Share with a group (owner only; group must be in user's groups) |
|
||||
| DELETE | `/documents/{id}/shares/{group_id}` | Stop sharing with a group (owner only) |
|
||||
|
||||
### Categories
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/categories` | List user's categories |
|
||||
| POST | `/categories` | Create category (triggers background AI reanalysis) |
|
||||
| PATCH | `/categories/{id}` | Rename |
|
||||
| DELETE | `/categories/{id}` | Delete (204) |
|
||||
|
||||
### Plugin
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/plugin/manifest` | Plugin manifest with settings JSON Schema |
|
||||
| GET | `/plugin/settings` | Current plugin settings |
|
||||
| PATCH | `/plugin/settings` | Update plugin settings |
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| Document title max | 500 chars | `models/document.py` |
|
||||
| Category name max | 128 chars | `models/category.py` |
|
||||
| PDF max size (default) | 20 MB | admin settings (configurable) |
|
||||
| Raw text cap | 500 k chars | `services/ai_client.py` |
|
||||
| Documents per_page | 1–100, default 20 | `routers/documents.py` |
|
||||
| AI service timeout | 60 s | `services/ai_client.py` |
|
||||
| AI service max retries | 2 | `services/ai_client.py` |
|
||||
@@ -15,9 +15,9 @@ FROM python:3.12-slim
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# Pre-create data and config dirs with correct ownership.
|
||||
# Named volumes mounted over these paths will inherit ownership on first creation.
|
||||
RUN mkdir -p /data/documents /config && chown -R appuser:appuser /data /config
|
||||
# Pre-create watch dir with correct ownership.
|
||||
# /data/documents and /config are no longer used — all file/config storage goes through storage-service.
|
||||
RUN mkdir -p /data/watch && chown -R appuser:appuser /data
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -2,11 +2,11 @@
|
||||
|
||||
## What it is
|
||||
|
||||
PDF document management microservice. Handles upload, storage, async AI-powered extraction, tagging, categorisation, and retrieval of PDF documents on a per-user basis.
|
||||
PDF document management microservice. Handles upload, storage, async AI-powered extraction, tagging, categorisation, and retrieval of PDF documents on a per-user basis. Also supports automatic ingestion from a mounted watch directory (NAS, Nextcloud, Syncthing, etc.).
|
||||
|
||||
Port: `8001` (internal only, not exposed to host). All traffic arrives via the backend proxy (`backend/app/routers/documents_proxy.py`), which injects the authenticated `x-user-id` header.
|
||||
|
||||
Database: shared PostgreSQL instance, isolated via `alembic_version_doc_service` Alembic version table. Storage: `/data/documents/` (Docker named volume `doc_data`).
|
||||
Database: shared PostgreSQL instance, isolated via `alembic_version_doc_service` Alembic version table. Storage: `/data/documents/` (Docker named volume `doc_data`). Watch directory: `/data/watch` (named volume `watch_data` in prod; bind-mount in dev via `docker-compose.dev.yml`).
|
||||
|
||||
---
|
||||
|
||||
@@ -31,13 +31,25 @@ Database: shared PostgreSQL instance, isolated via `alembic_version_doc_service`
|
||||
| `PATCH` | `/documents/{id}/type` | Update document type |
|
||||
| `PATCH` | `/documents/{id}/tags` | Replace tag list (dedup, preserve order) |
|
||||
| `PATCH` | `/documents/{id}/title` | Update editable title |
|
||||
| `GET` | `/documents/categories` | List all categories for the user |
|
||||
| `GET` | `/documents/categories` | List all categories (user + watch) |
|
||||
| `POST` | `/documents/categories` | Create a category; triggers re-analysis of documents in similar categories |
|
||||
| `POST` | `/documents/{id}/reprocess` | Reset status to pending and re-run AI extraction; 409 if already pending/processing |
|
||||
| `PATCH` | `/documents/categories/{id}` | Rename a category |
|
||||
| `DELETE` | `/documents/categories/{id}` | Delete a category |
|
||||
| `POST` | `/documents/{id}/categories/{cat_id}` | Assign category to document |
|
||||
| `DELETE` | `/documents/{id}/categories/{cat_id}` | Remove category from document |
|
||||
| `POST` | `/documents/{id}/suggestions/folder/confirm` | Apply AI folder suggestion → create/find category + assign |
|
||||
| `POST` | `/documents/{id}/suggestions/folder/reject` | Clear AI folder suggestion |
|
||||
| `POST` | `/documents/{id}/suggestions/filename/confirm` | Apply AI filename suggestion → set title |
|
||||
| `POST` | `/documents/{id}/suggestions/filename/reject` | Clear AI filename suggestion |
|
||||
|
||||
### Plugin endpoints (internal — backend calls only)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/plugin/manifest` | Static manifest: metadata, JSON Schema for settings, access rules |
|
||||
| `GET` | `/plugin/settings` | Current watch/storage config values |
|
||||
| `PATCH` | `/plugin/settings` | Update watch/storage config (persisted to `/config/doc_service_config.json`) |
|
||||
|
||||
### Pagination & filtering (`GET /documents`)
|
||||
|
||||
@@ -60,7 +72,7 @@ Response: `{ items: [...], total: N, page: N, pages: N }`
|
||||
|
||||
```
|
||||
id UUID
|
||||
user_id string (from x-user-id header)
|
||||
user_id string (from x-user-id header; "watch" for watch-ingested docs)
|
||||
filename original filename
|
||||
title AI-suggested editable title (nullable)
|
||||
file_size bytes
|
||||
@@ -71,9 +83,15 @@ tags JSON array string — editable tags
|
||||
error_message set if status=failed
|
||||
created_at upload timestamp
|
||||
processed_at when extraction finished
|
||||
source "upload" (default) or "watch"
|
||||
watch_path original absolute path in watch directory (nullable)
|
||||
suggested_folder AI-suggested category name, pending user confirm (nullable)
|
||||
suggested_filename AI-suggested title/rename, pending user confirm (nullable)
|
||||
categories many-to-many via category_assignments
|
||||
```
|
||||
|
||||
Watch-ingested documents (`user_id = "watch"`) are visible to all authenticated users.
|
||||
|
||||
### AI extraction (via ai-service)
|
||||
|
||||
System prompt and user prompt template are loaded at runtime from `doc_service_config.json` (`system_prompts` key). Defaults are built into the service and used as fallback if the config key is absent. Changes made via the AI Settings UI take effect within 30 seconds (config cache TTL).
|
||||
@@ -93,12 +111,43 @@ Prompt sends the first 50 000 chars of extracted text. Expected JSON response in
|
||||
```
|
||||
Env override: `DOC_MAX_PDF_MB`
|
||||
|
||||
### Watch directory feature
|
||||
|
||||
Controlled via plugin settings (UI accessible to superusers and `doc-service-admin` group members):
|
||||
|
||||
- `watch_enabled` — toggle file watching (default: false)
|
||||
- `watch_path` — mount point (read-only, `/data/watch`; override via Docker volume)
|
||||
- `ai_folder_suggestion` — AI suggests a category for each ingested doc (user confirms)
|
||||
- `ai_folder_default` — default category when AI suggestion is disabled
|
||||
- `ai_rename_suggestion` — AI suggests a title for each ingested doc (user confirms)
|
||||
|
||||
On startup scan, the watcher walks the watch directory and ingests any PDFs not already in the database (idempotency check by `watch_path`). Subfolders are automatically mapped to categories (e.g. `watch/invoices/bill.pdf` → category "invoices"). No-remove policy: deleting a file from the watch directory does not delete the document record.
|
||||
|
||||
### Document sharing (`document_shares`)
|
||||
|
||||
Group-based sharing allows a document owner to share a document with all members of any group they belong to. Recipients can view and download the shared document; they cannot edit, re-analyse, delete, or re-share it.
|
||||
|
||||
The gateway injects `X-User-Groups: <group_id1>,<group_id2>,...` alongside the existing `X-User-Id` header, so doc-service can evaluate group access without querying the backend DB.
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/documents/shared-with-me` | Any user | Documents shared with the user via their groups; excludes own docs |
|
||||
| `GET` | `/documents/{id}/shares` | Owner only | List all groups the document is shared with |
|
||||
| `POST` | `/documents/{id}/shares` | Owner only | Share with a group (`{group_id}` in body); group must be in X-User-Groups |
|
||||
| `DELETE` | `/documents/{id}/shares/{group_id}` | Owner only | Stop sharing with that group |
|
||||
|
||||
`DocumentOut` now includes `share_count: int` — the number of groups the document is shared with.
|
||||
|
||||
`GET /documents/{id}/file` also allows access to shared documents (recipients can download).
|
||||
|
||||
### Database migrations
|
||||
|
||||
| Revision | Description |
|
||||
|----------|-------------|
|
||||
| 0001 | Initial schema (documents, categories, category_assignments) |
|
||||
| 0002 | Add `title` column to documents |
|
||||
| 0003 | Add `source`, `watch_path`, `suggested_folder`, `suggested_filename` columns |
|
||||
| 0004 | Add `document_shares` table (document_id, group_id, shared_by_user_id, created_at) |
|
||||
|
||||
Run automatically on container start via `alembic upgrade head`.
|
||||
|
||||
@@ -109,18 +158,26 @@ Run automatically on container start via `alembic upgrade head`.
|
||||
```
|
||||
backend (proxy) → doc-service:8001
|
||||
│
|
||||
documents.py router
|
||||
│
|
||||
┌────────────┼────────────────────┐
|
||||
documents.py categories.py plugin.py
|
||||
│ │ (internal only)
|
||||
┌────────┴────────┐
|
||||
upload list/get/patch
|
||||
upload list/get/patch/suggest
|
||||
│
|
||||
save_upload() pdfplumber extraction
|
||||
save_upload() pdfplumber extraction
|
||||
│ │
|
||||
Document(status=pending) ai_client.classify_document()
|
||||
Document(status=pending) ai_client.classify_document()
|
||||
│ │
|
||||
BackgroundTask ai-service:8010/chat
|
||||
BackgroundTask ai-service:8010/chat
|
||||
│ │
|
||||
process_document() JSON result → update doc row
|
||||
process_document() JSON result → update doc row
|
||||
|
||||
file_watcher.py (watchdog Observer, daemon thread)
|
||||
│
|
||||
├── _PdfEventHandler.on_created / on_moved
|
||||
│ └── asyncio.run_coroutine_threadsafe(ingest_file, loop)
|
||||
│
|
||||
└── _scan_existing() on startup (catches offline gaps)
|
||||
```
|
||||
|
||||
---
|
||||
@@ -129,8 +186,8 @@ backend (proxy) → doc-service:8001
|
||||
|
||||
- **Re-process** — no endpoint to re-trigger AI extraction on an existing document (e.g. after changing the AI model or prompt)
|
||||
- **Advanced field-level search** — `search` param matches text fields via ILIKE but does not query into `extracted_data` JSON (e.g. filter by `vendor` or `due_date`)
|
||||
- **Bulk operations** — no bulk category assign/remove, no bulk delete
|
||||
- **Document sharing** — documents are strictly per-user; no group sharing yet
|
||||
- **Bulk operations** — no bulk category assign/remove endpoint (frontend handles bulk delete/share individually)
|
||||
- **Advanced field-level search** — `search` matches text fields via ILIKE but does not query into `extracted_data` JSON
|
||||
- **Pagination in categories** — categories are returned as a full list (no pagination)
|
||||
- **File type** — only PDF supported
|
||||
- **Concurrent uploads** — no rate limiting per user
|
||||
@@ -140,10 +197,14 @@ backend (proxy) → doc-service:8001
|
||||
## Future work
|
||||
|
||||
- [x] `POST /documents/{id}/reprocess` — re-run AI extraction
|
||||
- [x] Watch directory feature with file watcher, startup scan, folder-to-category mapping, AI suggestion toggles
|
||||
- [x] Plugin manifest endpoint (`/plugin/manifest`, `/plugin/settings`) for generic settings UI
|
||||
- [ ] Advanced filter: query `extracted_data` JSON fields (vendor, due_date, amount) — requires PostgreSQL `jsonb` column or indexed virtual columns
|
||||
- [ ] Bulk operations endpoint
|
||||
- [ ] Document sharing via groups (blocked on groups/permissions system in backend)
|
||||
- [x] Document sharing via groups — `document_shares` table + share endpoints + shared-with-me view
|
||||
- [x] Frontend UI for suggestion badges (suggested_folder / suggested_filename confirm/reject buttons in slide-over)
|
||||
- [ ] Advanced filter: query `extracted_data` JSON fields (vendor, due_date, amount)
|
||||
- [ ] Support additional file types (images via OCR, DOCX)
|
||||
- [ ] Rate limiting on upload endpoint
|
||||
- [ ] Soft delete with restore
|
||||
- [ ] Category rename / delete with cascade handling
|
||||
- [ ] Edit rights for shared recipients (V2)
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
"""add watch directory columns to documents
|
||||
|
||||
Revision ID: 0003
|
||||
Revises: 0002
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0003"
|
||||
down_revision: Union[str, None] = "0002"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column("documents", sa.Column("source", sa.String(16), nullable=False, server_default="upload"))
|
||||
op.add_column("documents", sa.Column("watch_path", sa.String(), nullable=True))
|
||||
op.add_column("documents", sa.Column("suggested_folder", sa.String(128), nullable=True))
|
||||
op.add_column("documents", sa.Column("suggested_filename", sa.String(500), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("documents", "suggested_filename")
|
||||
op.drop_column("documents", "suggested_folder")
|
||||
op.drop_column("documents", "watch_path")
|
||||
op.drop_column("documents", "source")
|
||||
@@ -0,0 +1,42 @@
|
||||
"""add document_shares table
|
||||
|
||||
Revision ID: 0004
|
||||
Revises: 0003
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0004"
|
||||
down_revision: Union[str, None] = "0003"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
"document_shares",
|
||||
sa.Column("id", sa.String(), nullable=False),
|
||||
sa.Column("document_id", sa.String(), nullable=False),
|
||||
sa.Column("group_id", sa.String(), nullable=False),
|
||||
sa.Column("shared_by_user_id", sa.String(), nullable=False),
|
||||
sa.Column(
|
||||
"created_at",
|
||||
sa.DateTime(timezone=True),
|
||||
server_default=sa.text("now()"),
|
||||
nullable=False,
|
||||
),
|
||||
sa.PrimaryKeyConstraint("id"),
|
||||
sa.UniqueConstraint("document_id", "group_id", name="uq_document_group_share"),
|
||||
)
|
||||
op.create_index("ix_document_shares_document_id", "document_shares", ["document_id"])
|
||||
op.create_index("ix_document_shares_group_id", "document_shares", ["group_id"])
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_document_shares_group_id", table_name="document_shares")
|
||||
op.drop_index("ix_document_shares_document_id", table_name="document_shares")
|
||||
op.drop_table("document_shares")
|
||||
@@ -0,0 +1,32 @@
|
||||
"""add can_delete to document_shares
|
||||
|
||||
Revision ID: 0005
|
||||
Revises: 0004
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0005"
|
||||
down_revision: Union[str, None] = "0004"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"document_shares",
|
||||
sa.Column(
|
||||
"can_delete",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default=sa.text("false"),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("document_shares", "can_delete")
|
||||
@@ -0,0 +1,42 @@
|
||||
"""add scope and group_id to document_categories
|
||||
|
||||
Revision ID: 0006
|
||||
Revises: 0005
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "0006"
|
||||
down_revision: Union[str, None] = "0005"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"document_categories",
|
||||
sa.Column(
|
||||
"scope",
|
||||
sa.String(16),
|
||||
nullable=False,
|
||||
server_default="personal",
|
||||
),
|
||||
)
|
||||
op.add_column(
|
||||
"document_categories",
|
||||
sa.Column("group_id", sa.String(), nullable=True),
|
||||
)
|
||||
op.create_index("ix_document_categories_group_id", "document_categories", ["group_id"])
|
||||
|
||||
# Migrate existing watch-owned categories to system scope
|
||||
op.execute("UPDATE document_categories SET scope = 'system' WHERE user_id = 'watch'")
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index("ix_document_categories_group_id", table_name="document_categories")
|
||||
op.drop_column("document_categories", "group_id")
|
||||
op.drop_column("document_categories", "scope")
|
||||
@@ -0,0 +1,44 @@
|
||||
"""capitalize existing system category names to PascalCase-with-dashes
|
||||
|
||||
Revision ID: 0007
|
||||
Revises: 0006
|
||||
Create Date: 2026-04-18
|
||||
|
||||
Converts names like "invoices" → "Invoices", "vendor-invoices" → "Vendor-Invoices"
|
||||
for all categories with scope='system' (watch-ingested).
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import re
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy import text
|
||||
|
||||
|
||||
revision: str = "0007"
|
||||
down_revision: Union[str, None] = "0006"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def _to_pascal(name: str) -> str:
|
||||
return "-".join(p.capitalize() for p in re.split(r"[-_\s]+", name) if p)
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
conn = op.get_bind()
|
||||
rows = conn.execute(
|
||||
text("SELECT id, name FROM document_categories WHERE scope = 'system'")
|
||||
).fetchall()
|
||||
for row in rows:
|
||||
new_name = _to_pascal(row.name)
|
||||
if new_name != row.name:
|
||||
conn.execute(
|
||||
text("UPDATE document_categories SET name = :name WHERE id = :id"),
|
||||
{"name": new_name[:128], "id": row.id},
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
pass # names before migration are unknown; downgrade is a no-op
|
||||
@@ -0,0 +1,56 @@
|
||||
"""rename file_path to storage_key and strip filesystem prefix from existing rows
|
||||
|
||||
Revision ID: 0008
|
||||
Revises: 0007
|
||||
Create Date: 2026-04-20
|
||||
|
||||
Renames the documents.file_path column to storage_key.
|
||||
Existing rows have paths like '/data/documents/{user_id}/{doc_id}.pdf' or
|
||||
'/data/documents/watch/{doc_id}.pdf'. The migration strips the leading
|
||||
'/data/documents/' prefix so the value becomes a plain storage key
|
||||
(e.g. '{user_id}/{doc_id}.pdf') that the storage-service uses as the object key.
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
revision: str = "0008"
|
||||
down_revision: Union[str, None] = "0007"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
with op.batch_alter_table("documents") as batch_op:
|
||||
batch_op.alter_column(
|
||||
"file_path",
|
||||
new_column_name="storage_key",
|
||||
existing_type=sa.String(),
|
||||
existing_nullable=False,
|
||||
)
|
||||
|
||||
# Strip the '/data/documents/' filesystem prefix from pre-migration rows.
|
||||
op.execute(
|
||||
sa.text(
|
||||
"UPDATE documents SET storage_key = REPLACE(storage_key, '/data/documents/', '')"
|
||||
" WHERE storage_key LIKE '/data/documents/%'"
|
||||
)
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# Restore the filesystem prefix so old code can still find the files.
|
||||
op.execute(
|
||||
sa.text(
|
||||
"UPDATE documents SET storage_key = '/data/documents/' || storage_key"
|
||||
" WHERE storage_key NOT LIKE '/data/documents/%'"
|
||||
)
|
||||
)
|
||||
with op.batch_alter_table("documents") as batch_op:
|
||||
batch_op.alter_column(
|
||||
"storage_key",
|
||||
new_column_name="file_path",
|
||||
existing_type=sa.String(),
|
||||
existing_nullable=False,
|
||||
)
|
||||
@@ -7,6 +7,7 @@ class Settings(BaseSettings):
|
||||
DATA_DIR: str = "/data/documents"
|
||||
CONFIG_PATH: str = "/config/doc_service_config.json"
|
||||
AI_SERVICE_URL: str = "http://ai-service:8010"
|
||||
STORAGE_SERVICE_URL: str = "http://storage-service:8020"
|
||||
|
||||
class Config:
|
||||
env_file = ".env"
|
||||
|
||||
@@ -10,3 +10,33 @@ async def get_user_id(x_user_id: str = Header(...)) -> str:
|
||||
if not x_user_id:
|
||||
raise HTTPException(status_code=400, detail="Missing X-User-Id header")
|
||||
return x_user_id
|
||||
|
||||
|
||||
async def get_user_groups(x_user_groups: str = Header(default="")) -> list[str]:
|
||||
"""
|
||||
Extract the group IDs injected by the main backend proxy.
|
||||
Comma-separated list of group UUIDs the current user belongs to.
|
||||
Returns an empty list if the header is absent or empty.
|
||||
"""
|
||||
if not x_user_groups:
|
||||
return []
|
||||
return [g.strip() for g in x_user_groups.split(",") if g.strip()]
|
||||
|
||||
|
||||
async def get_user_is_admin(x_user_is_admin: str = Header(default="false")) -> bool:
|
||||
"""
|
||||
Extract the superuser flag injected by the main backend proxy.
|
||||
Returns True only when the header value is exactly "true".
|
||||
"""
|
||||
return x_user_is_admin.lower() == "true"
|
||||
|
||||
|
||||
async def get_user_admin_groups(x_user_admin_groups: str = Header(default="")) -> list[str]:
|
||||
"""
|
||||
Extract the group IDs for which the current user is a group admin.
|
||||
Injected by the main backend proxy from GroupMembership.is_group_admin.
|
||||
Returns an empty list if absent or empty.
|
||||
"""
|
||||
if not x_user_admin_groups:
|
||||
return []
|
||||
return [g.strip() for g in x_user_admin_groups.split(",") if g.strip()]
|
||||
|
||||
@@ -1,15 +1,45 @@
|
||||
import asyncio
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
from app.core.config import settings
|
||||
from app.routers import categories, documents
|
||||
from app.routers import plugin as plugin_router
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
loop = asyncio.get_running_loop()
|
||||
watcher = None
|
||||
|
||||
try:
|
||||
from app.services.config_reader import get_storage_config
|
||||
storage_config = await get_storage_config()
|
||||
if storage_config.get("watch_enabled"):
|
||||
from app.services.file_watcher import FileWatcherService
|
||||
watcher = FileWatcherService(loop)
|
||||
await watcher.start(storage_config["watch_path"], storage_config)
|
||||
except Exception as exc:
|
||||
logger.warning("[doc-service] File watcher could not start: %s", exc)
|
||||
|
||||
yield
|
||||
|
||||
if watcher is not None:
|
||||
await watcher.stop()
|
||||
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME, lifespan=lifespan)
|
||||
|
||||
# No CORS — this service is only reachable from the main backend on backend-net.
|
||||
# All browser traffic goes through the main backend proxy.
|
||||
|
||||
app.include_router(documents.router, prefix="/documents", tags=["documents"])
|
||||
app.include_router(categories.router, prefix="/categories", tags=["categories"])
|
||||
app.include_router(plugin_router.router, prefix="/plugin", tags=["plugin"])
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
from app.models.document import Document
|
||||
from app.models.category import DocumentCategory
|
||||
from app.models.category_assignment import CategoryAssignment
|
||||
from app.models.document_share import DocumentShare
|
||||
|
||||
__all__ = ["Document", "DocumentCategory", "CategoryAssignment"]
|
||||
__all__ = ["Document", "DocumentCategory", "CategoryAssignment", "DocumentShare"]
|
||||
|
||||
@@ -13,6 +13,8 @@ class DocumentCategory(Base):
|
||||
id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
user_id: Mapped[str] = mapped_column(String, nullable=False, index=True)
|
||||
name: Mapped[str] = mapped_column(String(128), nullable=False)
|
||||
scope: Mapped[str] = mapped_column(String(16), nullable=False, default="personal", server_default="personal")
|
||||
group_id: Mapped[str | None] = mapped_column(String, nullable=True, index=True)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
@@ -13,7 +13,7 @@ class Document(Base):
|
||||
id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
user_id: Mapped[str] = mapped_column(String, nullable=False, index=True)
|
||||
filename: Mapped[str] = mapped_column(String, nullable=False)
|
||||
file_path: Mapped[str] = mapped_column(String, nullable=False)
|
||||
storage_key: Mapped[str] = mapped_column(String, nullable=False)
|
||||
file_size: Mapped[int] = mapped_column(Integer, nullable=False)
|
||||
status: Mapped[str] = mapped_column(String, nullable=False, default="pending")
|
||||
title: Mapped[str | None] = mapped_column(String(500), nullable=True)
|
||||
@@ -27,6 +27,12 @@ class Document(Base):
|
||||
)
|
||||
processed_at: Mapped[datetime | None] = mapped_column(DateTime(timezone=True), nullable=True)
|
||||
|
||||
# Watch-directory ingestion fields (migration 0003)
|
||||
source: Mapped[str] = mapped_column(String(16), nullable=False, default="upload")
|
||||
watch_path: Mapped[str | None] = mapped_column(String, nullable=True)
|
||||
suggested_folder: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
suggested_filename: Mapped[str | None] = mapped_column(String(500), nullable=True)
|
||||
|
||||
category_assignments: Mapped[list["CategoryAssignment"]] = relationship(
|
||||
"CategoryAssignment", back_populates="document", cascade="all, delete-orphan"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,24 @@
|
||||
import uuid
|
||||
from datetime import datetime
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, String, UniqueConstraint, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class DocumentShare(Base):
|
||||
__tablename__ = "document_shares"
|
||||
|
||||
id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
document_id: Mapped[str] = mapped_column(String, nullable=False, index=True)
|
||||
group_id: Mapped[str] = mapped_column(String, nullable=False, index=True)
|
||||
shared_by_user_id: Mapped[str] = mapped_column(String, nullable=False)
|
||||
can_delete: Mapped[bool] = mapped_column(Boolean, nullable=False, default=False, server_default="false")
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True), server_default=func.now(), nullable=False
|
||||
)
|
||||
|
||||
__table_args__ = (
|
||||
UniqueConstraint("document_id", "group_id", name="uq_document_group_share"),
|
||||
)
|
||||
@@ -1,12 +1,15 @@
|
||||
import difflib
|
||||
import json
|
||||
import re
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from sqlalchemy import or_
|
||||
|
||||
from app.database import AsyncSessionLocal, get_db
|
||||
from app.deps import get_user_id
|
||||
from app.deps import get_user_admin_groups, get_user_groups, get_user_id, get_user_is_admin
|
||||
from app.models.category import DocumentCategory
|
||||
from app.models.category_assignment import CategoryAssignment
|
||||
from app.models.document import Document
|
||||
@@ -15,16 +18,36 @@ from app.services.ai_client import classify_document
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Sentinel user_id for watch-ingested categories — must match documents.py
|
||||
_WATCH_USER_ID = "watch"
|
||||
|
||||
_SIMILARITY_THRESHOLD = 0.4
|
||||
|
||||
# PascalCase-with-dashes: each word starts with a capital, words joined by '-'
|
||||
# Valid: Invoices, Vendor-Invoices, Q1-Reports
|
||||
# Invalid: invoices, Invoice Reports, Invoice-reports
|
||||
_NAME_RE = re.compile(r'^[A-Z][a-zA-Z0-9]*(-[A-Z][a-zA-Z0-9]*)*$')
|
||||
|
||||
|
||||
def _validate_name(name: str) -> None:
|
||||
if not _NAME_RE.match(name):
|
||||
raise HTTPException(
|
||||
status_code=422,
|
||||
detail=(
|
||||
"Category name must start with a capital letter. "
|
||||
"Multiple words are joined with dashes, each starting with a capital "
|
||||
"(e.g. Invoices, Vendor-Invoices, Q1-Reports)."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def _name_similarity(a: str, b: str) -> float:
|
||||
"""Return similarity score (0–1) between two category names."""
|
||||
a_low = a.lower()
|
||||
b_low = b.lower()
|
||||
# Word overlap is a strong signal
|
||||
a_words = set(a_low.split())
|
||||
b_words = set(b_low.split())
|
||||
a_words = set(a_low.split("-"))
|
||||
b_words = set(b_low.split("-"))
|
||||
if a_words & b_words:
|
||||
return 0.9
|
||||
# Fallback: character sequence ratio
|
||||
@@ -76,14 +99,44 @@ async def _reanalyze_documents_for_new_category(
|
||||
pass
|
||||
|
||||
|
||||
async def _check_can_manage_cat(
|
||||
cat: DocumentCategory,
|
||||
user_id: str,
|
||||
is_admin: bool,
|
||||
user_admin_groups: list[str],
|
||||
) -> None:
|
||||
"""Raise 403/404 if the current user may not rename or delete this category."""
|
||||
if is_admin:
|
||||
return # superuser can manage anything
|
||||
if cat.scope == "system":
|
||||
raise HTTPException(status_code=403, detail="Only admins can modify system categories")
|
||||
if cat.scope == "personal" and cat.user_id != user_id:
|
||||
raise HTTPException(status_code=404, detail="Category not found")
|
||||
if cat.scope == "group" and cat.group_id not in user_admin_groups:
|
||||
raise HTTPException(status_code=403, detail="Only group admins can modify group categories")
|
||||
|
||||
|
||||
@router.get("", response_model=list[CategoryOut])
|
||||
async def list_categories(
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[DocumentCategory]:
|
||||
"""
|
||||
Return all categories visible to the current user:
|
||||
- personal (owned by the user)
|
||||
- group (any group the user belongs to)
|
||||
- system (watch-ingested, scope='system')
|
||||
"""
|
||||
result = await db.execute(
|
||||
select(DocumentCategory)
|
||||
.where(DocumentCategory.user_id == user_id)
|
||||
.where(
|
||||
or_(
|
||||
DocumentCategory.user_id == user_id, # personal
|
||||
DocumentCategory.group_id.in_(user_groups), # group
|
||||
DocumentCategory.scope == "system", # system
|
||||
)
|
||||
)
|
||||
.order_by(DocumentCategory.name)
|
||||
)
|
||||
return result.scalars().all()
|
||||
@@ -94,18 +147,32 @@ async def create_category(
|
||||
body: CategoryCreate,
|
||||
background_tasks: BackgroundTasks,
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentCategory:
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise HTTPException(status_code=422, detail="Category name cannot be empty")
|
||||
_validate_name(name)
|
||||
|
||||
if body.group_id:
|
||||
# User must be a member of the target group
|
||||
if body.group_id not in user_groups:
|
||||
raise HTTPException(status_code=403, detail="You are not a member of that group")
|
||||
cat = DocumentCategory(
|
||||
user_id=user_id,
|
||||
name=name[:128],
|
||||
scope="group",
|
||||
group_id=body.group_id,
|
||||
)
|
||||
else:
|
||||
cat = DocumentCategory(user_id=user_id, name=name[:128], scope="personal")
|
||||
|
||||
cat = DocumentCategory(user_id=user_id, name=name[:128])
|
||||
db.add(cat)
|
||||
await db.commit()
|
||||
await db.refresh(cat)
|
||||
|
||||
# Find existing categories with similar names
|
||||
# Find existing personal categories with similar names for background AI reanalysis
|
||||
result = await db.execute(
|
||||
select(DocumentCategory)
|
||||
.where(DocumentCategory.user_id == user_id)
|
||||
@@ -134,12 +201,18 @@ async def rename_category(
|
||||
cat_id: str,
|
||||
body: CategoryUpdate,
|
||||
user_id: str = Depends(get_user_id),
|
||||
is_admin: bool = Depends(get_user_is_admin),
|
||||
user_admin_groups: list[str] = Depends(get_user_admin_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentCategory:
|
||||
cat = await _get_user_cat(cat_id, user_id, db)
|
||||
cat = await _fetch_visible_cat(cat_id, user_id, db)
|
||||
await _check_can_manage_cat(cat, user_id, is_admin, user_admin_groups)
|
||||
|
||||
name = body.name.strip()
|
||||
if not name:
|
||||
raise HTTPException(status_code=422, detail="Category name cannot be empty")
|
||||
_validate_name(name)
|
||||
|
||||
cat.name = name[:128]
|
||||
await db.commit()
|
||||
await db.refresh(cat)
|
||||
@@ -150,19 +223,20 @@ async def rename_category(
|
||||
async def delete_category(
|
||||
cat_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
is_admin: bool = Depends(get_user_is_admin),
|
||||
user_admin_groups: list[str] = Depends(get_user_admin_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
cat = await _get_user_cat(cat_id, user_id, db)
|
||||
cat = await _fetch_visible_cat(cat_id, user_id, db)
|
||||
await _check_can_manage_cat(cat, user_id, is_admin, user_admin_groups)
|
||||
await db.delete(cat)
|
||||
await db.commit()
|
||||
|
||||
|
||||
async def _get_user_cat(cat_id: str, user_id: str, db: AsyncSession) -> DocumentCategory:
|
||||
async def _fetch_visible_cat(cat_id: str, user_id: str, db: AsyncSession) -> DocumentCategory:
|
||||
"""Fetch a category that the user can see (personal/group/system). 404 if absent."""
|
||||
result = await db.execute(
|
||||
select(DocumentCategory).where(
|
||||
DocumentCategory.id == cat_id,
|
||||
DocumentCategory.user_id == user_id,
|
||||
)
|
||||
select(DocumentCategory).where(DocumentCategory.id == cat_id)
|
||||
)
|
||||
cat = result.scalar_one_or_none()
|
||||
if cat is None:
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
import asyncio
|
||||
import io
|
||||
import json
|
||||
import math
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
import aiofiles
|
||||
import pdfplumber
|
||||
from fastapi import APIRouter, BackgroundTasks, Depends, HTTPException, Query, UploadFile
|
||||
from fastapi.responses import StreamingResponse
|
||||
@@ -13,26 +13,43 @@ from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from app.database import AsyncSessionLocal, get_db
|
||||
from app.deps import get_user_id
|
||||
from app.deps import get_user_admin_groups, get_user_groups, get_user_id, get_user_is_admin
|
||||
from app.models.category import DocumentCategory
|
||||
from app.models.category_assignment import CategoryAssignment
|
||||
from app.models.document import Document
|
||||
from app.schemas.document import DocumentOut, DocumentPage, DocumentStatusOut, DocumentTypeUpdate, TagsUpdate, TitleUpdate
|
||||
from app.models.document_share import DocumentShare
|
||||
from app.schemas.document import (
|
||||
DocumentOut,
|
||||
DocumentPage,
|
||||
DocumentStatusOut,
|
||||
DocumentTypeUpdate,
|
||||
TagsUpdate,
|
||||
TitleUpdate,
|
||||
)
|
||||
from app.schemas.share import DocumentShareCreate, DocumentShareOut, SharedDocumentOut
|
||||
from app.services.ai_client import AIServiceError, classify_document
|
||||
from app.services.config_reader import load_doc_config
|
||||
from app.services.storage import delete_file, get_upload_path, save_upload
|
||||
from app.services.storage import delete_file, download_file as storage_download, save_upload
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_DEFAULT_MAX_BYTES = 20 * 1024 * 1024
|
||||
|
||||
# Sentinel user_id used for watch-directory-ingested documents.
|
||||
# These documents are visible to all authenticated users.
|
||||
_WATCH_USER_ID = "watch"
|
||||
|
||||
|
||||
# ── Helpers ───────────────────────────────────────────────────────────────────
|
||||
|
||||
async def _get_user_doc(doc_id: str, user_id: str, db: AsyncSession) -> Document:
|
||||
"""Fetch a document owned by user_id OR a watch-ingested document (visible to all)."""
|
||||
result = await db.execute(
|
||||
select(Document)
|
||||
.where(Document.id == doc_id, Document.user_id == user_id)
|
||||
.where(
|
||||
Document.id == doc_id,
|
||||
or_(Document.user_id == user_id, Document.user_id == _WATCH_USER_ID),
|
||||
)
|
||||
.options(
|
||||
selectinload(Document.category_assignments)
|
||||
.selectinload(CategoryAssignment.category)
|
||||
@@ -44,7 +61,38 @@ async def _get_user_doc(doc_id: str, user_id: str, db: AsyncSession) -> Document
|
||||
return doc
|
||||
|
||||
|
||||
def _doc_with_categories(doc: Document) -> DocumentOut:
|
||||
async def _get_share_counts(doc_ids: list[str], db: AsyncSession) -> dict[str, int]:
|
||||
"""Return a mapping of doc_id → share count for the given document IDs."""
|
||||
if not doc_ids:
|
||||
return {}
|
||||
rows = await db.execute(
|
||||
select(DocumentShare.document_id, func.count(DocumentShare.id))
|
||||
.where(DocumentShare.document_id.in_(doc_ids))
|
||||
.group_by(DocumentShare.document_id)
|
||||
)
|
||||
return {row[0]: row[1] for row in rows.all()}
|
||||
|
||||
|
||||
async def _get_deletable_doc_ids(
|
||||
doc_ids: list[str], user_groups: list[str], db: AsyncSession
|
||||
) -> set[str]:
|
||||
"""Return doc IDs for which the user has delete permission via a group share."""
|
||||
if not doc_ids or not user_groups:
|
||||
return set()
|
||||
rows = await db.execute(
|
||||
select(DocumentShare.document_id)
|
||||
.where(
|
||||
DocumentShare.document_id.in_(doc_ids),
|
||||
DocumentShare.group_id.in_(user_groups),
|
||||
DocumentShare.can_delete.is_(True),
|
||||
)
|
||||
)
|
||||
return {row[0] for row in rows.all()}
|
||||
|
||||
|
||||
def _doc_with_categories(
|
||||
doc: Document, share_count: int = 0, viewer_can_delete: bool = False
|
||||
) -> DocumentOut:
|
||||
from app.schemas.document import CategoryOut
|
||||
cats = [CategoryOut(id=a.category.id, name=a.category.name) for a in doc.category_assignments]
|
||||
return DocumentOut(
|
||||
@@ -61,13 +109,19 @@ def _doc_with_categories(doc: Document) -> DocumentOut:
|
||||
created_at=doc.created_at,
|
||||
processed_at=doc.processed_at,
|
||||
categories=cats,
|
||||
source=doc.source,
|
||||
watch_path=doc.watch_path,
|
||||
suggested_folder=doc.suggested_folder,
|
||||
suggested_filename=doc.suggested_filename,
|
||||
share_count=share_count,
|
||||
viewer_can_delete=viewer_can_delete,
|
||||
)
|
||||
|
||||
|
||||
def _extract_pdf_text(file_path: str) -> str:
|
||||
def _extract_pdf_text(pdf_bytes: bytes) -> str:
|
||||
"""Synchronous — must be called via asyncio.to_thread."""
|
||||
text_parts = []
|
||||
with pdfplumber.open(file_path) as pdf:
|
||||
with pdfplumber.open(io.BytesIO(pdf_bytes)) as pdf:
|
||||
for page in pdf.pages:
|
||||
page_text = page.extract_text()
|
||||
if page_text:
|
||||
@@ -92,7 +146,8 @@ async def process_document(doc_id: str) -> None:
|
||||
await db.commit()
|
||||
|
||||
try:
|
||||
text = await asyncio.to_thread(_extract_pdf_text, doc.file_path)
|
||||
pdf_bytes = await storage_download(doc.storage_key)
|
||||
text = await asyncio.to_thread(_extract_pdf_text, pdf_bytes)
|
||||
result = await classify_document(text)
|
||||
|
||||
doc.raw_text = text[:500_000] # cap stored text at 500k chars
|
||||
@@ -133,13 +188,13 @@ async def upload_document(
|
||||
)
|
||||
|
||||
doc_id = str(uuid.uuid4())
|
||||
dest = await save_upload(file_data, user_id, doc_id)
|
||||
storage_key = await save_upload(file_data, user_id, doc_id)
|
||||
|
||||
doc = Document(
|
||||
id=doc_id,
|
||||
user_id=user_id,
|
||||
filename=file.filename or "upload.pdf",
|
||||
file_path=str(dest),
|
||||
storage_key=storage_key,
|
||||
file_size=len(file_data),
|
||||
status="pending",
|
||||
)
|
||||
@@ -149,8 +204,6 @@ async def upload_document(
|
||||
background_tasks.add_task(process_document, doc_id)
|
||||
|
||||
# Re-query with selectinload so category_assignments is eagerly loaded.
|
||||
# A new doc has no categories yet, but we need the relationship populated
|
||||
# to avoid MissingGreenlet in the async session.
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
return _doc_with_categories(doc)
|
||||
|
||||
@@ -177,13 +230,15 @@ async def list_documents(
|
||||
search: str | None = Query(default=None),
|
||||
category_id: str | None = Query(default=None),
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
is_admin: bool = Depends(get_user_is_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentPage:
|
||||
sort_col = _SORT_COLUMNS.get(sort, Document.created_at)
|
||||
sort_expr = sort_col.desc() if order == "desc" else sort_col.asc()
|
||||
|
||||
# Build filter conditions once and reuse for both count + items queries.
|
||||
conditions = [Document.user_id == user_id]
|
||||
# Watch-ingested documents (user_id = "watch") are visible to all users.
|
||||
conditions = [or_(Document.user_id == user_id, Document.user_id == _WATCH_USER_ID)]
|
||||
if status:
|
||||
conditions.append(Document.status == status)
|
||||
if document_type:
|
||||
@@ -220,7 +275,21 @@ async def list_documents(
|
||||
.offset((page - 1) * per_page)
|
||||
.limit(per_page)
|
||||
)
|
||||
items = [_doc_with_categories(d) for d in items_result.scalars().all()]
|
||||
docs = items_result.scalars().all()
|
||||
|
||||
doc_ids = [d.id for d in docs]
|
||||
share_counts = await _get_share_counts(doc_ids, db)
|
||||
|
||||
if is_admin:
|
||||
deletable_ids = set(doc_ids)
|
||||
else:
|
||||
deletable_ids = {d.id for d in docs if d.user_id == user_id}
|
||||
deletable_ids |= await _get_deletable_doc_ids(doc_ids, user_groups, db)
|
||||
|
||||
items = [
|
||||
_doc_with_categories(d, share_counts.get(d.id, 0), viewer_can_delete=d.id in deletable_ids)
|
||||
for d in docs
|
||||
]
|
||||
|
||||
return DocumentPage(
|
||||
items=items,
|
||||
@@ -230,14 +299,137 @@ async def list_documents(
|
||||
)
|
||||
|
||||
|
||||
# NOTE: This route must be registered BEFORE /{doc_id} to avoid path collision.
|
||||
@router.get("/shared-with-me", response_model=DocumentPage)
|
||||
async def list_shared_with_me(
|
||||
page: int = Query(default=1, ge=1),
|
||||
per_page: int = Query(default=20, ge=1, le=100),
|
||||
sort: str = Query(default="created_at"),
|
||||
order: str = Query(default="desc", pattern="^(asc|desc)$"),
|
||||
search: str | None = Query(default=None),
|
||||
document_type: str | None = Query(default=None),
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentPage:
|
||||
"""Return documents shared with the current user via any of their groups.
|
||||
Excludes documents the user owns (those appear in their regular list).
|
||||
"""
|
||||
if not user_groups:
|
||||
return DocumentPage(items=[], total=0, page=page, pages=1)
|
||||
|
||||
sort_col = _SORT_COLUMNS.get(sort, Document.created_at)
|
||||
sort_expr = sort_col.desc() if order == "desc" else sort_col.asc()
|
||||
|
||||
shared_doc_ids_subq = (
|
||||
select(DocumentShare.document_id)
|
||||
.where(DocumentShare.group_id.in_(user_groups))
|
||||
.scalar_subquery()
|
||||
)
|
||||
|
||||
conditions = [
|
||||
Document.id.in_(shared_doc_ids_subq),
|
||||
Document.user_id != user_id, # exclude own docs
|
||||
]
|
||||
if document_type:
|
||||
conditions.append(Document.document_type == document_type)
|
||||
if search:
|
||||
like = f"%{search}%"
|
||||
conditions.append(
|
||||
or_(
|
||||
Document.title.ilike(like),
|
||||
Document.filename.ilike(like),
|
||||
Document.tags.ilike(like),
|
||||
Document.document_type.ilike(like),
|
||||
)
|
||||
)
|
||||
|
||||
count_result = await db.execute(
|
||||
select(func.count(Document.id)).where(*conditions)
|
||||
)
|
||||
total = count_result.scalar_one()
|
||||
|
||||
items_result = await db.execute(
|
||||
select(Document)
|
||||
.where(*conditions)
|
||||
.options(
|
||||
selectinload(Document.category_assignments)
|
||||
.selectinload(CategoryAssignment.category)
|
||||
)
|
||||
.order_by(sort_expr)
|
||||
.offset((page - 1) * per_page)
|
||||
.limit(per_page)
|
||||
)
|
||||
docs = items_result.scalars().all()
|
||||
|
||||
# For each doc, find which share (group) brought it in (pick first match)
|
||||
share_rows_result = await db.execute(
|
||||
select(DocumentShare)
|
||||
.where(
|
||||
DocumentShare.document_id.in_([d.id for d in docs]),
|
||||
DocumentShare.group_id.in_(user_groups),
|
||||
)
|
||||
)
|
||||
share_rows = share_rows_result.scalars().all()
|
||||
# Map doc_id → first share row found
|
||||
share_map: dict[str, DocumentShare] = {}
|
||||
for share in share_rows:
|
||||
if share.document_id not in share_map:
|
||||
share_map[share.document_id] = share
|
||||
|
||||
from app.schemas.document import CategoryOut
|
||||
|
||||
items: list[SharedDocumentOut] = []
|
||||
for doc in docs:
|
||||
cats = [CategoryOut(id=a.category.id, name=a.category.name) for a in doc.category_assignments]
|
||||
share = share_map.get(doc.id)
|
||||
items.append(
|
||||
SharedDocumentOut(
|
||||
id=doc.id,
|
||||
user_id=doc.user_id,
|
||||
filename=doc.filename,
|
||||
title=doc.title,
|
||||
file_size=doc.file_size,
|
||||
status=doc.status,
|
||||
document_type=doc.document_type,
|
||||
extracted_data=doc.extracted_data,
|
||||
tags=doc.tags,
|
||||
error_message=doc.error_message,
|
||||
created_at=doc.created_at,
|
||||
processed_at=doc.processed_at,
|
||||
categories=cats,
|
||||
source=doc.source,
|
||||
shared_by_user_id=share.shared_by_user_id if share else "",
|
||||
shared_via_group_id=share.group_id if share else "",
|
||||
viewer_can_delete=bool(share and share.can_delete),
|
||||
)
|
||||
)
|
||||
|
||||
return DocumentPage(
|
||||
items=items, # type: ignore[arg-type]
|
||||
total=total,
|
||||
page=page,
|
||||
pages=max(1, math.ceil(total / per_page)),
|
||||
)
|
||||
|
||||
|
||||
@router.get("/{doc_id}", response_model=DocumentOut)
|
||||
async def get_document(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
is_admin: bool = Depends(get_user_is_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentOut:
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
return _doc_with_categories(doc)
|
||||
counts = await _get_share_counts([doc.id], db)
|
||||
if is_admin:
|
||||
viewer_can_delete = True
|
||||
elif doc.user_id == user_id:
|
||||
viewer_can_delete = True
|
||||
else:
|
||||
viewer_can_delete = bool(await _get_deletable_doc_ids([doc.id], user_groups, db))
|
||||
return _doc_with_categories(doc, counts.get(doc.id, 0), viewer_can_delete=viewer_can_delete)
|
||||
|
||||
|
||||
@router.get("/{doc_id}/status", response_model=DocumentStatusOut)
|
||||
@@ -247,7 +439,10 @@ async def get_document_status(
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Document:
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
select(Document).where(
|
||||
Document.id == doc_id,
|
||||
or_(Document.user_id == user_id, Document.user_id == _WATCH_USER_ID),
|
||||
)
|
||||
)
|
||||
doc = result.scalar_one_or_none()
|
||||
if doc is None:
|
||||
@@ -327,15 +522,64 @@ async def reprocess_document(
|
||||
async def delete_document(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
is_admin: bool = Depends(get_user_is_admin),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
user_admin_groups: list[str] = Depends(get_user_admin_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
if is_admin:
|
||||
# Admins can delete any document — fetch unconditionally.
|
||||
result = await db.execute(select(Document).where(Document.id == doc_id))
|
||||
else:
|
||||
# Fetch the document (owner, watch, or shared with user's groups)
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
select(Document).where(
|
||||
Document.id == doc_id,
|
||||
or_(
|
||||
Document.user_id == user_id,
|
||||
Document.user_id == _WATCH_USER_ID,
|
||||
Document.id.in_(
|
||||
select(DocumentShare.document_id).where(
|
||||
DocumentShare.group_id.in_(user_groups)
|
||||
)
|
||||
) if user_groups else False,
|
||||
),
|
||||
)
|
||||
)
|
||||
doc = result.scalar_one_or_none()
|
||||
if doc is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
delete_file(doc.file_path)
|
||||
|
||||
is_owner = doc.user_id == user_id
|
||||
|
||||
if not is_owner and not is_admin:
|
||||
# Check: shared with a group where the user has can_delete=True
|
||||
can_delete_via_share = False
|
||||
if user_groups:
|
||||
share_result = await db.execute(
|
||||
select(DocumentShare).where(
|
||||
DocumentShare.document_id == doc_id,
|
||||
DocumentShare.group_id.in_(user_groups),
|
||||
DocumentShare.can_delete.is_(True),
|
||||
)
|
||||
)
|
||||
can_delete_via_share = share_result.scalar_one_or_none() is not None
|
||||
|
||||
# Check: user is a group admin for a group the doc is shared with
|
||||
can_delete_as_group_admin = False
|
||||
if user_admin_groups:
|
||||
admin_share_result = await db.execute(
|
||||
select(DocumentShare).where(
|
||||
DocumentShare.document_id == doc_id,
|
||||
DocumentShare.group_id.in_(user_admin_groups),
|
||||
)
|
||||
)
|
||||
can_delete_as_group_admin = admin_share_result.scalar_one_or_none() is not None
|
||||
|
||||
if not can_delete_via_share and not can_delete_as_group_admin:
|
||||
raise HTTPException(status_code=403, detail="Not allowed to delete this document")
|
||||
|
||||
await delete_file(doc.storage_key)
|
||||
await db.delete(doc)
|
||||
await db.commit()
|
||||
|
||||
@@ -344,22 +588,35 @@ async def delete_document(
|
||||
async def download_file(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> StreamingResponse:
|
||||
# Allow access if: owner, watch doc, or shared with any of user's groups
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
select(Document).where(
|
||||
Document.id == doc_id,
|
||||
or_(
|
||||
Document.user_id == user_id,
|
||||
Document.user_id == _WATCH_USER_ID,
|
||||
Document.id.in_(
|
||||
select(DocumentShare.document_id).where(
|
||||
DocumentShare.group_id.in_(user_groups)
|
||||
)
|
||||
) if user_groups else False,
|
||||
),
|
||||
)
|
||||
)
|
||||
doc = result.scalar_one_or_none()
|
||||
if doc is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
async def file_generator():
|
||||
async with aiofiles.open(doc.file_path, "rb") as f:
|
||||
while chunk := await f.read(64 * 1024):
|
||||
yield chunk
|
||||
try:
|
||||
pdf_bytes = await storage_download(doc.storage_key)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="File not found in storage")
|
||||
|
||||
return StreamingResponse(
|
||||
file_generator(),
|
||||
iter([pdf_bytes]),
|
||||
media_type="application/pdf",
|
||||
headers={"Content-Disposition": f'inline; filename="{doc.filename}"'},
|
||||
)
|
||||
@@ -372,24 +629,32 @@ async def assign_category(
|
||||
doc_id: str,
|
||||
cat_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
# Verify both belong to this user
|
||||
doc_result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
select(Document).where(
|
||||
Document.id == doc_id,
|
||||
or_(Document.user_id == user_id, Document.user_id == _WATCH_USER_ID),
|
||||
)
|
||||
)
|
||||
if doc_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
# Accept personal, group (user is a member), or system categories
|
||||
cat_result = await db.execute(
|
||||
select(DocumentCategory).where(
|
||||
DocumentCategory.id == cat_id, DocumentCategory.user_id == user_id
|
||||
DocumentCategory.id == cat_id,
|
||||
or_(
|
||||
DocumentCategory.user_id == user_id,
|
||||
DocumentCategory.group_id.in_(user_groups) if user_groups else False,
|
||||
DocumentCategory.scope == "system",
|
||||
),
|
||||
)
|
||||
)
|
||||
if cat_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=404, detail="Category not found")
|
||||
|
||||
# Upsert — ignore if already assigned
|
||||
existing = await db.execute(
|
||||
select(CategoryAssignment).where(
|
||||
CategoryAssignment.document_id == doc_id,
|
||||
@@ -408,6 +673,13 @@ async def remove_category(
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
# Only the document owner may remove a category assignment
|
||||
doc_result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
)
|
||||
if doc_result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=403, detail="Only the document owner can remove category assignments")
|
||||
|
||||
result = await db.execute(
|
||||
select(CategoryAssignment).where(
|
||||
CategoryAssignment.document_id == doc_id,
|
||||
@@ -418,3 +690,169 @@ async def remove_category(
|
||||
if assignment:
|
||||
await db.delete(assignment)
|
||||
await db.commit()
|
||||
|
||||
|
||||
# ── AI suggestion confirmation ────────────────────────────────────────────────
|
||||
|
||||
@router.post("/{doc_id}/suggestions/folder/confirm", status_code=204)
|
||||
async def confirm_folder_suggestion(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
if not doc.suggested_folder:
|
||||
raise HTTPException(status_code=400, detail="No folder suggestion pending")
|
||||
|
||||
cat_result = await db.execute(
|
||||
select(DocumentCategory).where(
|
||||
DocumentCategory.user_id == _WATCH_USER_ID,
|
||||
DocumentCategory.name == doc.suggested_folder,
|
||||
)
|
||||
)
|
||||
cat = cat_result.scalar_one_or_none()
|
||||
if cat is None:
|
||||
cat = DocumentCategory(user_id=_WATCH_USER_ID, name=doc.suggested_folder[:128])
|
||||
db.add(cat)
|
||||
await db.commit()
|
||||
await db.refresh(cat)
|
||||
|
||||
exists = await db.execute(
|
||||
select(CategoryAssignment).where(
|
||||
CategoryAssignment.document_id == doc_id,
|
||||
CategoryAssignment.category_id == cat.id,
|
||||
)
|
||||
)
|
||||
if exists.scalar_one_or_none() is None:
|
||||
db.add(CategoryAssignment(document_id=doc_id, category_id=cat.id))
|
||||
|
||||
doc.suggested_folder = None
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.post("/{doc_id}/suggestions/folder/reject", status_code=204)
|
||||
async def reject_folder_suggestion(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
doc.suggested_folder = None
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.post("/{doc_id}/suggestions/filename/confirm", status_code=204)
|
||||
async def confirm_filename_suggestion(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
if not doc.suggested_filename:
|
||||
raise HTTPException(status_code=400, detail="No filename suggestion pending")
|
||||
doc.title = doc.suggested_filename
|
||||
doc.suggested_filename = None
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.post("/{doc_id}/suggestions/filename/reject", status_code=204)
|
||||
async def reject_filename_suggestion(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
doc = await _get_user_doc(doc_id, user_id, db)
|
||||
doc.suggested_filename = None
|
||||
await db.commit()
|
||||
|
||||
|
||||
# ── Document sharing ──────────────────────────────────────────────────────────
|
||||
|
||||
@router.get("/{doc_id}/shares", response_model=list[DocumentShareOut])
|
||||
async def list_document_shares(
|
||||
doc_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[DocumentShare]:
|
||||
"""List all group shares for a document. Owner only."""
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
)
|
||||
if result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
shares_result = await db.execute(
|
||||
select(DocumentShare).where(DocumentShare.document_id == doc_id)
|
||||
)
|
||||
return shares_result.scalars().all()
|
||||
|
||||
|
||||
@router.post("/{doc_id}/shares", response_model=DocumentShareOut, status_code=201)
|
||||
async def add_document_share(
|
||||
doc_id: str,
|
||||
body: DocumentShareCreate,
|
||||
user_id: str = Depends(get_user_id),
|
||||
user_groups: list[str] = Depends(get_user_groups),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> DocumentShare:
|
||||
"""Share a document with a group. The sharing user must own the document
|
||||
and must be a member of the target group."""
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
)
|
||||
if result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
if body.group_id not in user_groups:
|
||||
raise HTTPException(
|
||||
status_code=403,
|
||||
detail="You can only share with groups you belong to",
|
||||
)
|
||||
|
||||
# Idempotent — return existing share if already shared with this group
|
||||
existing = await db.execute(
|
||||
select(DocumentShare).where(
|
||||
DocumentShare.document_id == doc_id,
|
||||
DocumentShare.group_id == body.group_id,
|
||||
)
|
||||
)
|
||||
share = existing.scalar_one_or_none()
|
||||
if share is not None:
|
||||
return share
|
||||
|
||||
share = DocumentShare(
|
||||
document_id=doc_id,
|
||||
group_id=body.group_id,
|
||||
shared_by_user_id=user_id,
|
||||
can_delete=body.can_delete,
|
||||
)
|
||||
db.add(share)
|
||||
await db.commit()
|
||||
await db.refresh(share)
|
||||
return share
|
||||
|
||||
|
||||
@router.delete("/{doc_id}/shares/{group_id}", status_code=204)
|
||||
async def remove_document_share(
|
||||
doc_id: str,
|
||||
group_id: str,
|
||||
user_id: str = Depends(get_user_id),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
"""Remove a group share. Owner only."""
|
||||
result = await db.execute(
|
||||
select(Document).where(Document.id == doc_id, Document.user_id == user_id)
|
||||
)
|
||||
if result.scalar_one_or_none() is None:
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
share_result = await db.execute(
|
||||
select(DocumentShare).where(
|
||||
DocumentShare.document_id == doc_id,
|
||||
DocumentShare.group_id == group_id,
|
||||
)
|
||||
)
|
||||
share = share_result.scalar_one_or_none()
|
||||
if share:
|
||||
await db.delete(share)
|
||||
await db.commit()
|
||||
|
||||
@@ -0,0 +1,97 @@
|
||||
"""
|
||||
Plugin manifest and settings endpoints for doc-service.
|
||||
|
||||
These are internal-only — they are called by the main backend's generic plugin
|
||||
proxy, never directly by the browser. No authentication is applied here because
|
||||
the backend enforces access control before forwarding the request.
|
||||
|
||||
Endpoints:
|
||||
GET /plugin/manifest → static manifest with JSON Schema for settings
|
||||
GET /plugin/settings → current storage config values
|
||||
PATCH /plugin/settings → update storage config (partial update)
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.services.config_reader import get_storage_config, save_storage_config
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_MANIFEST: dict = {
|
||||
"id": "doc-service",
|
||||
"name": "Document Service",
|
||||
"icon": "file-text",
|
||||
"version": "1.0",
|
||||
"access": {
|
||||
"allow_superuser": True,
|
||||
"required_groups": ["doc-service-admin"],
|
||||
},
|
||||
"settings_schema": {
|
||||
"type": "object",
|
||||
"title": "Storage & Watch",
|
||||
"properties": {
|
||||
"watch_enabled": {
|
||||
"type": "boolean",
|
||||
"title": "Enable file watching",
|
||||
"description": (
|
||||
"Automatically ingest PDF files added to the mounted watch directory. "
|
||||
"Requires a service restart to take effect after toggling."
|
||||
),
|
||||
},
|
||||
"watch_path": {
|
||||
"type": "string",
|
||||
"title": "Watch path",
|
||||
"readOnly": True,
|
||||
"description": "Configured via Docker volume mount — edit docker-compose to change.",
|
||||
},
|
||||
"ai_folder_suggestion": {
|
||||
"type": "boolean",
|
||||
"title": "AI folder suggestion",
|
||||
"description": (
|
||||
"AI suggests a category for each ingested document. "
|
||||
"You must confirm the suggestion before it is applied."
|
||||
),
|
||||
},
|
||||
"ai_folder_default": {
|
||||
"type": "string",
|
||||
"title": "Default import category",
|
||||
"description": "Category assigned automatically when AI folder suggestion is disabled.",
|
||||
},
|
||||
"ai_rename_suggestion": {
|
||||
"type": "boolean",
|
||||
"title": "AI rename suggestion",
|
||||
"description": (
|
||||
"AI suggests a document title for each ingested file. "
|
||||
"You must confirm before it is applied."
|
||||
),
|
||||
},
|
||||
},
|
||||
},
|
||||
}
|
||||
|
||||
|
||||
class StorageSettingsUpdate(BaseModel):
|
||||
watch_enabled: bool | None = None
|
||||
ai_folder_suggestion: bool | None = None
|
||||
ai_folder_default: str | None = None
|
||||
ai_rename_suggestion: bool | None = None
|
||||
# watch_path is intentionally excluded — it cannot be changed via API
|
||||
|
||||
|
||||
@router.get("/manifest")
|
||||
async def get_manifest() -> dict:
|
||||
return _MANIFEST
|
||||
|
||||
|
||||
@router.get("/settings")
|
||||
async def get_settings() -> dict:
|
||||
return await get_storage_config()
|
||||
|
||||
|
||||
@router.patch("/settings")
|
||||
async def update_settings(body: StorageSettingsUpdate) -> dict:
|
||||
update = body.model_dump(exclude_none=True)
|
||||
if "ai_folder_default" in update:
|
||||
update["ai_folder_default"] = update["ai_folder_default"][:128].strip() or "imports"
|
||||
await save_storage_config(update)
|
||||
return await get_storage_config()
|
||||
@@ -7,6 +7,8 @@ class CategoryOut(BaseModel):
|
||||
id: str
|
||||
user_id: str
|
||||
name: str
|
||||
scope: str = "personal"
|
||||
group_id: str | None = None
|
||||
created_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
@@ -14,6 +16,7 @@ class CategoryOut(BaseModel):
|
||||
|
||||
class CategoryCreate(BaseModel):
|
||||
name: str
|
||||
group_id: str | None = None
|
||||
|
||||
|
||||
class CategoryUpdate(BaseModel):
|
||||
|
||||
@@ -23,6 +23,12 @@ class DocumentOut(BaseModel):
|
||||
created_at: datetime
|
||||
processed_at: datetime | None
|
||||
categories: list[CategoryOut] = []
|
||||
source: str = "upload"
|
||||
watch_path: str | None = None
|
||||
suggested_folder: str | None = None
|
||||
suggested_filename: str | None = None
|
||||
share_count: int = 0
|
||||
viewer_can_delete: bool = False
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
|
||||
class DocumentShareOut(BaseModel):
|
||||
id: str
|
||||
document_id: str
|
||||
group_id: str
|
||||
shared_by_user_id: str
|
||||
can_delete: bool
|
||||
created_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class DocumentShareCreate(BaseModel):
|
||||
group_id: str
|
||||
can_delete: bool = False
|
||||
|
||||
|
||||
class SharedDocumentOut(BaseModel):
|
||||
"""DocumentOut fields plus sharing context for the 'Shared with me' view."""
|
||||
id: str
|
||||
user_id: str
|
||||
filename: str
|
||||
title: str | None
|
||||
file_size: int
|
||||
status: str
|
||||
document_type: str | None
|
||||
extracted_data: str | None
|
||||
tags: str | None
|
||||
error_message: str | None
|
||||
created_at: datetime
|
||||
processed_at: datetime | None
|
||||
categories: list = []
|
||||
source: str = "upload"
|
||||
share_count: int = 0
|
||||
viewer_can_delete: bool = False
|
||||
# Sharing context
|
||||
shared_by_user_id: str
|
||||
shared_via_group_id: str
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
@@ -1,19 +1,28 @@
|
||||
"""
|
||||
Reads doc_service_config.json from the shared config volume.
|
||||
Reads doc_service_config.json from the storage-service config bucket.
|
||||
30-second TTL cache + env var overrides.
|
||||
|
||||
Env var overrides (all optional):
|
||||
DOC_MAX_PDF_MB — max upload size in megabytes (e.g. "50")
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import os
|
||||
import time
|
||||
from copy import deepcopy
|
||||
from pathlib import Path
|
||||
|
||||
import httpx
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
_CONFIG_KEY = "doc_service_config.json"
|
||||
|
||||
_DEFAULT_STORAGE_CONFIG: dict = {
|
||||
"watch_enabled": False,
|
||||
"watch_path": "/data/watch",
|
||||
"ai_folder_suggestion": False,
|
||||
"ai_folder_default": "imports",
|
||||
"ai_rename_suggestion": False,
|
||||
}
|
||||
|
||||
_DEFAULT_SYSTEM_PROMPT = (
|
||||
"You are a financial document analysis assistant. "
|
||||
"Given the text extracted from a PDF document, return ONLY a JSON object "
|
||||
@@ -43,6 +52,7 @@ _DEFAULT_USER_TEMPLATE = (
|
||||
|
||||
_DEFAULT_CONFIG: dict = {
|
||||
"documents": {"max_pdf_bytes": 20 * 1024 * 1024},
|
||||
"storage": _DEFAULT_STORAGE_CONFIG,
|
||||
"system_prompts": {
|
||||
"system": _DEFAULT_SYSTEM_PROMPT,
|
||||
"user_template": _DEFAULT_USER_TEMPLATE,
|
||||
@@ -54,14 +64,30 @@ _cache_at: float = 0.0
|
||||
_CACHE_TTL = 30.0
|
||||
|
||||
|
||||
def _read_config_sync() -> dict:
|
||||
path = Path(settings.CONFIG_PATH)
|
||||
if not path.exists():
|
||||
base = deepcopy(_DEFAULT_CONFIG)
|
||||
else:
|
||||
with open(path) as f:
|
||||
base = json.load(f)
|
||||
return _apply_env_overrides(base)
|
||||
def _storage_url() -> str:
|
||||
return f"{settings.STORAGE_SERVICE_URL}/objects/config/{_CONFIG_KEY}"
|
||||
|
||||
|
||||
async def _fetch_config() -> dict:
|
||||
"""Fetch config from storage-service. Returns defaults if not found."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url())
|
||||
if resp.status_code == 404:
|
||||
return deepcopy(_DEFAULT_CONFIG)
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def _write_config(data: dict) -> None:
|
||||
import json
|
||||
payload = json.dumps(data, indent=2).encode()
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.put(
|
||||
_storage_url(),
|
||||
content=payload,
|
||||
headers={"Content-Type": "application/octet-stream"},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
def _apply_env_overrides(config: dict) -> dict:
|
||||
@@ -80,7 +106,26 @@ async def load_doc_config() -> dict:
|
||||
now = time.monotonic()
|
||||
if _cache is not None and (now - _cache_at) < _CACHE_TTL:
|
||||
return _cache
|
||||
data = await asyncio.to_thread(_read_config_sync)
|
||||
raw = await _fetch_config()
|
||||
data = _apply_env_overrides(raw)
|
||||
_cache = data
|
||||
_cache_at = now
|
||||
return data
|
||||
|
||||
|
||||
async def get_storage_config() -> dict:
|
||||
"""Return storage config block, filling in defaults for any missing keys."""
|
||||
config = await load_doc_config()
|
||||
result = deepcopy(_DEFAULT_STORAGE_CONFIG)
|
||||
result.update(config.get("storage", {}))
|
||||
return result
|
||||
|
||||
|
||||
async def save_storage_config(data: dict) -> None:
|
||||
"""Merge data into the storage config block and persist to storage-service."""
|
||||
global _cache, _cache_at
|
||||
raw = await _fetch_config()
|
||||
raw.setdefault("storage", {}).update(data)
|
||||
await _write_config(raw)
|
||||
_cache = None
|
||||
_cache_at = 0.0
|
||||
|
||||
@@ -0,0 +1,261 @@
|
||||
"""
|
||||
File-system watcher for the watch directory.
|
||||
|
||||
Uses the watchdog library to monitor a configured directory for new PDF files.
|
||||
When a PDF is detected, it is automatically ingested into the document service
|
||||
(uploaded to storage-service, a DB record is created, and the AI pipeline runs).
|
||||
|
||||
Key design decisions:
|
||||
- No-remove policy: on_deleted and on_moved events are intentionally ignored.
|
||||
The watcher never deletes, moves, or modifies files on the watched volume.
|
||||
- Watch documents use user_id="watch" as a sentinel so they are visible to
|
||||
all authenticated users in the document list.
|
||||
- Subfolder names map to categories: a file at invoices/bill.pdf is assigned
|
||||
to an "Invoices" category (auto-created if needed; folder name is converted
|
||||
to PascalCase-with-dashes: "vendor-invoices" → "Vendor-Invoices").
|
||||
- Suggestions: if ai_folder_suggestion or ai_rename_suggestion are enabled,
|
||||
the relevant fields are set on the document after AI processing so users
|
||||
can confirm/reject from the UI.
|
||||
- Thread → async bridge: watchdog runs in a daemon thread; asyncio coroutines
|
||||
are dispatched from that thread via run_coroutine_threadsafe.
|
||||
"""
|
||||
import asyncio
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import uuid
|
||||
from pathlib import Path
|
||||
|
||||
from watchdog.events import FileSystemEventHandler
|
||||
from watchdog.observers.polling import PollingObserver
|
||||
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.models.category import DocumentCategory
|
||||
from app.models.category_assignment import CategoryAssignment
|
||||
from app.models.document import Document
|
||||
from app.services.storage import save_upload
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# Must match _WATCH_USER_ID in app/routers/documents.py
|
||||
WATCH_USER_ID = "watch"
|
||||
|
||||
|
||||
# ── Ingestion logic ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def ingest_file(path_str: str, watch_root: Path, config: dict) -> None:
|
||||
"""
|
||||
Ingest a single PDF file from the watch directory.
|
||||
|
||||
Idempotent: skips files that already have a non-failed document record.
|
||||
"""
|
||||
from sqlalchemy import select
|
||||
|
||||
path = Path(path_str)
|
||||
if not path.exists() or not path.is_file():
|
||||
return
|
||||
|
||||
async with AsyncSessionLocal() as db:
|
||||
# Idempotency check — skip if already tracked (and not failed)
|
||||
existing_result = await db.execute(
|
||||
select(Document).where(Document.watch_path == path_str)
|
||||
)
|
||||
existing = existing_result.scalar_one_or_none()
|
||||
if existing is not None and existing.status != "failed":
|
||||
return
|
||||
|
||||
# Determine category from the first subfolder component
|
||||
try:
|
||||
rel = path.relative_to(watch_root)
|
||||
folder_name = (
|
||||
"-".join(p.capitalize() for p in re.split(r"[-_\s]+", rel.parts[0]) if p)
|
||||
if len(rel.parts) > 1 else None
|
||||
)
|
||||
except ValueError:
|
||||
folder_name = None
|
||||
|
||||
# Read file bytes
|
||||
try:
|
||||
file_data = path.read_bytes()
|
||||
except OSError as exc:
|
||||
logger.warning("[watcher] Cannot read %s: %s", path_str, exc)
|
||||
return
|
||||
|
||||
# Upload to storage-service under documents/watch/{doc_id}.pdf
|
||||
doc_id = existing.id if existing is not None else str(uuid.uuid4())
|
||||
storage_key = await save_upload(file_data, WATCH_USER_ID, doc_id)
|
||||
|
||||
if existing is not None:
|
||||
# Re-ingest a previously failed document
|
||||
existing.storage_key = storage_key
|
||||
existing.file_size = len(file_data)
|
||||
existing.status = "pending"
|
||||
existing.error_message = None
|
||||
await db.commit()
|
||||
else:
|
||||
doc = Document(
|
||||
id=doc_id,
|
||||
user_id=WATCH_USER_ID,
|
||||
source="watch",
|
||||
watch_path=path_str,
|
||||
filename=path.name,
|
||||
storage_key=storage_key,
|
||||
file_size=len(file_data),
|
||||
status="pending",
|
||||
)
|
||||
db.add(doc)
|
||||
await db.commit()
|
||||
|
||||
# Auto-assign category from subfolder name
|
||||
if folder_name:
|
||||
cat_result = await db.execute(
|
||||
select(DocumentCategory).where(
|
||||
DocumentCategory.user_id == WATCH_USER_ID,
|
||||
DocumentCategory.name == folder_name,
|
||||
)
|
||||
)
|
||||
cat = cat_result.scalar_one_or_none()
|
||||
if cat is None:
|
||||
cat = DocumentCategory(user_id=WATCH_USER_ID, name=folder_name[:128])
|
||||
db.add(cat)
|
||||
await db.commit()
|
||||
await db.refresh(cat)
|
||||
|
||||
exists_assign = await db.execute(
|
||||
select(CategoryAssignment).where(
|
||||
CategoryAssignment.document_id == doc_id,
|
||||
CategoryAssignment.category_id == cat.id,
|
||||
)
|
||||
)
|
||||
if exists_assign.scalar_one_or_none() is None:
|
||||
db.add(CategoryAssignment(document_id=doc_id, category_id=cat.id))
|
||||
await db.commit()
|
||||
|
||||
# Run AI pipeline (opens its own session internally)
|
||||
from app.routers.documents import process_document
|
||||
await process_document(doc_id)
|
||||
|
||||
# Set AI suggestions if enabled
|
||||
if config.get("ai_folder_suggestion") or config.get("ai_rename_suggestion"):
|
||||
await _apply_suggestions(doc_id, config)
|
||||
|
||||
|
||||
async def _apply_suggestions(doc_id: str, config: dict) -> None:
|
||||
"""Populate suggested_folder / suggested_filename after AI processing."""
|
||||
from sqlalchemy import select
|
||||
|
||||
async with AsyncSessionLocal() as db:
|
||||
result = await db.execute(select(Document).where(Document.id == doc_id))
|
||||
doc = result.scalar_one_or_none()
|
||||
if doc is None or doc.status != "done" or not doc.extracted_data:
|
||||
return
|
||||
|
||||
try:
|
||||
extracted = json.loads(doc.extracted_data)
|
||||
except Exception:
|
||||
return
|
||||
|
||||
changed = False
|
||||
if config.get("ai_folder_suggestion"):
|
||||
suggestions = extracted.get("suggested_categories", [])
|
||||
if suggestions:
|
||||
doc.suggested_folder = str(suggestions[0])[:128]
|
||||
changed = True
|
||||
|
||||
if config.get("ai_rename_suggestion"):
|
||||
title = extracted.get("title")
|
||||
if title:
|
||||
doc.suggested_filename = str(title)[:500]
|
||||
changed = True
|
||||
|
||||
if changed:
|
||||
await db.commit()
|
||||
|
||||
|
||||
# ── Watchdog event handler ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class _PdfEventHandler(FileSystemEventHandler):
|
||||
def __init__(
|
||||
self,
|
||||
watch_root: Path,
|
||||
loop: asyncio.AbstractEventLoop,
|
||||
config: dict,
|
||||
) -> None:
|
||||
super().__init__()
|
||||
self._watch_root = watch_root
|
||||
self._loop = loop
|
||||
self._config = config
|
||||
|
||||
def _dispatch_ingest(self, path_str: str) -> None:
|
||||
if path_str.lower().endswith(".pdf"):
|
||||
asyncio.run_coroutine_threadsafe(
|
||||
ingest_file(path_str, self._watch_root, self._config),
|
||||
self._loop,
|
||||
)
|
||||
|
||||
def on_created(self, event): # type: ignore[override]
|
||||
if not event.is_directory:
|
||||
self._dispatch_ingest(event.src_path)
|
||||
|
||||
def on_moved(self, event): # type: ignore[override]
|
||||
# Handles atomic rename/move (e.g. Nextcloud or Syncthing completing a sync)
|
||||
if not event.is_directory:
|
||||
self._dispatch_ingest(event.dest_path)
|
||||
|
||||
# on_deleted / on_modified: intentionally not overridden — no-remove policy
|
||||
|
||||
|
||||
# ── Service ───────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class FileWatcherService:
|
||||
"""Manages the watchdog Observer lifecycle within the FastAPI lifespan."""
|
||||
|
||||
def __init__(self, loop: asyncio.AbstractEventLoop) -> None:
|
||||
self._loop = loop
|
||||
self._observer: Observer | None = None
|
||||
self._watch_root: Path | None = None
|
||||
self._config: dict = {}
|
||||
|
||||
async def start(self, watch_path: str, config: dict) -> None:
|
||||
self._watch_root = Path(watch_path)
|
||||
self._config = config
|
||||
|
||||
if not self._watch_root.exists():
|
||||
logger.warning(
|
||||
"[watcher] Watch path %s does not exist — file watching disabled",
|
||||
watch_path,
|
||||
)
|
||||
return
|
||||
|
||||
handler = _PdfEventHandler(self._watch_root, self._loop, config)
|
||||
self._observer = PollingObserver()
|
||||
self._observer.schedule(handler, watch_path, recursive=True)
|
||||
self._observer.start()
|
||||
logger.info("[watcher] started, watching %s", watch_path)
|
||||
|
||||
# Run startup scan as a background task so startup is not blocked
|
||||
asyncio.create_task(self._scan_existing())
|
||||
|
||||
async def _scan_existing(self) -> None:
|
||||
"""Ingest any PDFs already present in the watch directory."""
|
||||
if self._watch_root is None:
|
||||
return
|
||||
logger.info("[watcher] scanning existing files in %s", self._watch_root)
|
||||
count = 0
|
||||
for pdf_path in sorted(self._watch_root.rglob("*.pdf")):
|
||||
try:
|
||||
await ingest_file(str(pdf_path), self._watch_root, self._config)
|
||||
count += 1
|
||||
except Exception as exc:
|
||||
logger.warning("[watcher] scan error for %s: %s", pdf_path, exc)
|
||||
logger.info("[watcher] startup scan complete — processed %d file(s)", count)
|
||||
|
||||
async def stop(self) -> None:
|
||||
if self._observer is not None:
|
||||
self._observer.stop()
|
||||
await asyncio.to_thread(self._observer.join)
|
||||
self._observer = None
|
||||
logger.info("[watcher] stopped")
|
||||
@@ -1,27 +1,61 @@
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
"""
|
||||
Storage client for the storage-service HTTP API.
|
||||
|
||||
import aiofiles
|
||||
All persistent file I/O goes through storage-service:8020.
|
||||
The bucket for all document PDFs is 'documents'.
|
||||
Keys follow the pattern:
|
||||
uploaded: {user_id}/{doc_id}.pdf
|
||||
watch-ingested: watch/{doc_id}.pdf
|
||||
"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
def get_upload_path(user_id: str, doc_id: str) -> Path:
|
||||
"""Return /data/documents/{user_id}/{doc_id}.pdf, creating the directory if needed."""
|
||||
user_dir = Path(settings.DATA_DIR) / user_id
|
||||
user_dir.mkdir(parents=True, exist_ok=True)
|
||||
return user_dir / f"{doc_id}.pdf"
|
||||
_BUCKET = "documents"
|
||||
|
||||
|
||||
async def save_upload(file_data: bytes, user_id: str, doc_id: str) -> Path:
|
||||
dest = get_upload_path(user_id, doc_id)
|
||||
async with aiofiles.open(dest, "wb") as f:
|
||||
await f.write(file_data)
|
||||
return dest
|
||||
def _storage_url(key: str) -> str:
|
||||
return f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}/{key}"
|
||||
|
||||
|
||||
def delete_file(file_path: str) -> None:
|
||||
def build_storage_key(user_id: str, doc_id: str) -> str:
|
||||
"""Return the canonical storage key for a document."""
|
||||
return f"{user_id}/{doc_id}.pdf"
|
||||
|
||||
|
||||
async def save_upload(file_data: bytes, user_id: str, doc_id: str) -> str:
|
||||
"""Upload bytes to storage-service. Returns the storage key."""
|
||||
key = build_storage_key(user_id, doc_id)
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.put(
|
||||
_storage_url(key),
|
||||
content=file_data,
|
||||
headers={"Content-Type": "application/octet-stream"},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
return key
|
||||
|
||||
|
||||
async def download_file(storage_key: str) -> bytes:
|
||||
"""Download bytes from storage-service by storage key."""
|
||||
async with httpx.AsyncClient(timeout=60.0) as client:
|
||||
resp = await client.get(_storage_url(storage_key))
|
||||
if resp.status_code == 404:
|
||||
raise FileNotFoundError(f"Object not found: {storage_key}")
|
||||
resp.raise_for_status()
|
||||
return resp.content
|
||||
|
||||
|
||||
async def delete_file(storage_key: str) -> None:
|
||||
"""Delete an object from storage-service. Swallows errors — deletion failure must not 500."""
|
||||
try:
|
||||
Path(file_path).unlink(missing_ok=True)
|
||||
except OSError:
|
||||
pass # log but do not raise — deletion failure must not 500
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.delete(_storage_url(storage_key))
|
||||
if resp.status_code not in (204, 404):
|
||||
logger.warning("storage-service DELETE returned %s for key %s", resp.status_code, storage_key)
|
||||
except Exception as exc:
|
||||
logger.warning("Could not delete %s from storage-service: %s", storage_key, exc)
|
||||
|
||||
@@ -17,6 +17,7 @@ dependencies = [
|
||||
"pdfplumber>=0.11",
|
||||
"aiofiles>=23.0",
|
||||
"python-multipart>=0.0.9",
|
||||
"watchdog>=4.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
|
||||
@@ -0,0 +1,115 @@
|
||||
# storage-service — Claude context
|
||||
|
||||
Unified file/blob storage microservice, port 8020 (internal). All services must use this service's
|
||||
HTTP API for any file persistence — no service may write to a Docker volume directly. See root
|
||||
`CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## Architecture rule (enforced)
|
||||
|
||||
**No service may write to a filesystem path for persistent data.**
|
||||
All file/blob storage must go through the storage-service HTTP API.
|
||||
Violation is a security/architecture defect.
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
features/storage-service/
|
||||
├── app/
|
||||
│ ├── main.py ← FastAPI, lifespan (backend init)
|
||||
│ ├── core/config.py ← Settings (DATA_DIR, STORAGE_BACKEND, S3_*, WEBDAV_*)
|
||||
│ ├── routers/
|
||||
│ │ ├── health.py ← GET /health
|
||||
│ │ ├── objects.py ← PUT/GET/DELETE /objects/{bucket}/{key:path}, GET /objects/{bucket}
|
||||
│ │ └── migrate.py ← POST /migrate, GET /migrate/status, DELETE /migrate, PATCH /backend-config
|
||||
│ └── services/
|
||||
│ ├── backend_manager.py ← build_backend(), initialize_backend(), get_backend(), switch_backend()
|
||||
│ ├── migration.py ← run_migration(), get_status(), cancel(); KNOWN_BUCKETS
|
||||
│ └── backends/
|
||||
│ ├── base.py ← AbstractStorageBackend (ABC)
|
||||
│ ├── local.py ← LocalFSBackend — /data/storage/{bucket}/{key}
|
||||
│ ├── s3.py ← S3Backend — aiobotocore, endpoint_url configurable
|
||||
│ └── webdav.py ← WebDAVBackend — aiohttp + WebDAV PROPFIND/PUT/GET/DELETE
|
||||
├── scripts/
|
||||
│ ├── start.sh ← prod start (uvicorn port 8020)
|
||||
│ └── start_dev.sh ← dev start (uvicorn --reload)
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## HTTP API
|
||||
|
||||
### Objects
|
||||
|
||||
| Method | Path | Body | Response |
|
||||
|--------|------|------|----------|
|
||||
| PUT | `/objects/{bucket}/{key:path}` | Raw bytes | 204 |
|
||||
| GET | `/objects/{bucket}/{key:path}` | — | 200 Raw bytes / 404 |
|
||||
| DELETE | `/objects/{bucket}/{key:path}` | — | 204 |
|
||||
| GET | `/objects/{bucket}` | — | `{"bucket": "...", "keys": [...]}` |
|
||||
|
||||
Keys may contain `/` (e.g. `user123/abc.pdf`). Path traversal (`..`) returns 400.
|
||||
|
||||
### Migration
|
||||
|
||||
| Method | Path | Body | Response |
|
||||
|--------|------|------|----------|
|
||||
| POST | `/migrate` | `{"driver": "s3", "config": {...}}` | 202 / 400 / 409 |
|
||||
| GET | `/migrate/status` | — | `{state, total, done, failed, errors[]}` |
|
||||
| DELETE | `/migrate` | — | 204 / 409 |
|
||||
| PATCH | `/backend-config` | `{"driver": "...", "config": {...}}` | 204 / 400 / 409 |
|
||||
|
||||
Migration states: `idle → validating → migrating → switching → cleaning → done` (or `failed`/`cancelled`)
|
||||
|
||||
### Health
|
||||
|
||||
| Method | Path | Response |
|
||||
|--------|------|----------|
|
||||
| GET | `/health` | `{"status": "ok", "backend": "local"}` |
|
||||
|
||||
---
|
||||
|
||||
## Buckets
|
||||
|
||||
| Bucket | Contents | Key format |
|
||||
|--------|----------|------------|
|
||||
| `documents` | Uploaded PDFs | `{user_id}/{doc_id}.pdf` or `watch/{doc_id}.pdf` |
|
||||
| `config` | JSON config files | `{service_name}_config.json` |
|
||||
|
||||
To add a new bucket: add it to `KNOWN_BUCKETS` in `services/migration.py` so it is included in migrations.
|
||||
|
||||
---
|
||||
|
||||
## Backend drivers
|
||||
|
||||
| Driver | Config fields | Notes |
|
||||
|--------|---------------|-------|
|
||||
| `local` | `data_dir` (optional) | Default. Files under `/data/storage/`. Zero external deps. |
|
||||
| `s3` | `endpoint_url`, `access_key`, `secret_key`, `region` | Works with MinIO, AWS S3, Backblaze B2, Cloudflare R2. Set `endpoint_url=""` for real AWS. |
|
||||
| `webdav` | `url`, `username`, `password`, `root_path` | Nextcloud: set root_path to `/remote.php/dav/files/{username}` |
|
||||
|
||||
---
|
||||
|
||||
## Adding a new backend driver
|
||||
|
||||
1. Create `app/services/backends/your_driver.py` implementing `AbstractStorageBackend`
|
||||
2. Add a branch in `build_backend()` in `backend_manager.py`
|
||||
3. Add config fields to `app/core/config.py` if env-based config is needed
|
||||
4. Document driver name + config fields in this file
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| Default backend | `local` | `STORAGE_BACKEND` env var |
|
||||
| Local data dir | `/data/storage` | `DATA_DIR` env var |
|
||||
| S3 region default | `us-east-1` | `S3_REGION` env var |
|
||||
| Migration error cap in response | 50 | `migration.py` |
|
||||
| Port | 8020 | `scripts/start.sh` |
|
||||
@@ -0,0 +1,32 @@
|
||||
# ── Stage 1: dependency installation ─────────────────────────────────────────
|
||||
FROM python:3.12-slim AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install --upgrade pip
|
||||
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --prefix=/install .
|
||||
|
||||
# ── Stage 2: runtime ──────────────────────────────────────────────────────────
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Create non-root user (UID/GID 1001)
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# Pre-create data dir with correct ownership.
|
||||
# Named volume mounted over this path will inherit ownership on first creation.
|
||||
RUN mkdir -p /data/storage && chown -R appuser:appuser /data
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /install /usr/local
|
||||
COPY --chown=appuser:appuser app ./app
|
||||
COPY --chown=appuser:appuser scripts ./scripts
|
||||
|
||||
USER appuser
|
||||
|
||||
EXPOSE 8020
|
||||
|
||||
CMD ["sh", "scripts/start.sh"]
|
||||
@@ -0,0 +1,83 @@
|
||||
# Storage Service — Status
|
||||
|
||||
## What it is
|
||||
Unified file/blob storage microservice, port 8020 (internal). All services store and retrieve files
|
||||
through its HTTP API — no service writes to a Docker volume directly. Uses a pluggable backend
|
||||
driver (local FS by default; S3-compatible and WebDAV available). Backend is switchable at runtime
|
||||
via admin settings with automatic data migration.
|
||||
|
||||
## Current functionality
|
||||
|
||||
### Object API (`/objects`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| PUT | `/objects/{bucket}/{key}` | Upload raw bytes |
|
||||
| GET | `/objects/{bucket}/{key}` | Download raw bytes |
|
||||
| DELETE | `/objects/{bucket}/{key}` | Delete object |
|
||||
| GET | `/objects/{bucket}` | List all keys in bucket |
|
||||
|
||||
### Migration API (`/migrate`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/migrate` | Start migration to a new backend (validates, copies, switches, cleans) |
|
||||
| GET | `/migrate/status` | Poll migration progress |
|
||||
| DELETE | `/migrate` | Cancel in-progress migration |
|
||||
| PATCH | `/backend-config` | Reconfigure backend without migrating data |
|
||||
|
||||
### Health
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/health` | `{"status": "ok", "backend": "<driver>"}` |
|
||||
|
||||
### Buckets
|
||||
|
||||
| Bucket | Contents |
|
||||
|--------|----------|
|
||||
| `documents` | Uploaded PDFs (keyed as `{user_id}/{doc_id}.pdf` or `watch/{doc_id}.pdf`) |
|
||||
| `config` | JSON config files (replaces `app_config` volume — Phase 3) |
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
backend / doc-service / future-svc
|
||||
│ HTTP
|
||||
▼
|
||||
storage-service:8020
|
||||
│
|
||||
backend_manager
|
||||
│
|
||||
┌──────┴──────────────────┐
|
||||
│ │
|
||||
LocalFSBackend S3Backend / WebDAVBackend
|
||||
/data/storage/ (configured via admin UI)
|
||||
{bucket}/{key}
|
||||
```
|
||||
|
||||
Migration flow:
|
||||
```
|
||||
POST /migrate { driver, config }
|
||||
→ test_connection() (validate)
|
||||
→ list all objects in KNOWN_BUCKETS (enumerate)
|
||||
→ GET old / PUT new / exists verify (copy + verify, per object)
|
||||
→ if 0 failures: switch_backend() (atomic switch)
|
||||
→ DELETE old objects (cleanup)
|
||||
```
|
||||
|
||||
## Known limitations / not implemented
|
||||
|
||||
- Migration state is in-memory — a container restart during migration loses progress (restart restarts from scratch)
|
||||
- No presigned URL support (direct client downloads go through the API)
|
||||
- rclone backends (Google Drive, OneDrive, Dropbox) not yet implemented
|
||||
- No per-object metadata or content-type headers
|
||||
- No multipart upload for very large files (> available RAM)
|
||||
|
||||
## Future work
|
||||
|
||||
- [ ] rclone-based backend adapter (GDrive, OneDrive, Dropbox)
|
||||
- [ ] Presigned URL generation for direct browser downloads
|
||||
- [ ] Persist migration state to DB so restarts can resume
|
||||
- [ ] Streaming upload/download to avoid buffering entire file in memory
|
||||
- [ ] Per-bucket access policies
|
||||
@@ -0,0 +1,23 @@
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
model_config = SettingsConfigDict(env_file=".env", extra="ignore")
|
||||
|
||||
DATA_DIR: str = "/data/storage"
|
||||
STORAGE_BACKEND: str = "local" # local | s3 | webdav
|
||||
|
||||
# S3-compatible (MinIO, AWS S3, Backblaze B2, Cloudflare R2, …)
|
||||
S3_ENDPOINT_URL: str = "" # leave empty for real AWS S3
|
||||
S3_ACCESS_KEY: str = ""
|
||||
S3_SECRET_KEY: str = ""
|
||||
S3_REGION: str = "us-east-1"
|
||||
|
||||
# WebDAV (Nextcloud, …)
|
||||
WEBDAV_URL: str = ""
|
||||
WEBDAV_USERNAME: str = ""
|
||||
WEBDAV_PASSWORD: str = ""
|
||||
WEBDAV_ROOT_PATH: str = "/"
|
||||
|
||||
|
||||
settings = Settings()
|
||||
@@ -0,0 +1,29 @@
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
from app.core.config import settings
|
||||
from app.routers import health, objects, migrate
|
||||
from app.services.backend_manager import initialize_backend
|
||||
|
||||
logging.basicConfig(
|
||||
level=logging.INFO,
|
||||
format="%(asctime)s [%(levelname)s] %(name)s: %(message)s",
|
||||
)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
initialize_backend()
|
||||
logger.info("storage-service started (backend=%s)", settings.STORAGE_BACKEND)
|
||||
yield
|
||||
logger.info("storage-service shutting down")
|
||||
|
||||
|
||||
app = FastAPI(title="Storage Service", lifespan=lifespan)
|
||||
|
||||
app.include_router(health.router)
|
||||
app.include_router(objects.router)
|
||||
app.include_router(migrate.router)
|
||||
@@ -0,0 +1,10 @@
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.services.backend_manager import get_backend
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
async def health():
|
||||
return {"status": "ok", "backend": get_backend().driver_name}
|
||||
@@ -0,0 +1,88 @@
|
||||
import logging
|
||||
|
||||
from fastapi import APIRouter, BackgroundTasks, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.services import migration
|
||||
from app.services.backend_manager import build_backend, switch_backend
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class MigrateRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
class BackendConfigRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
@router.post("/migrate", status_code=202)
|
||||
async def start_migration(body: MigrateRequest, background_tasks: BackgroundTasks):
|
||||
"""
|
||||
Validate the new backend, then start an async migration job that:
|
||||
1. Copies all objects from the current backend to the new one
|
||||
2. Verifies every object
|
||||
3. Atomically switches the active backend
|
||||
4. Deletes all objects from the old backend
|
||||
|
||||
Returns 409 if a migration is already in progress.
|
||||
Returns 400 if the new backend config fails validation.
|
||||
"""
|
||||
if migration.is_in_progress():
|
||||
raise HTTPException(status_code=409, detail="A migration is already in progress")
|
||||
|
||||
# Reset status and enter validating state before any async work
|
||||
migration._status.state = "validating"
|
||||
migration._status.total = 0
|
||||
migration._status.done = 0
|
||||
migration._status.failed = 0
|
||||
migration._status.errors.clear()
|
||||
|
||||
try:
|
||||
new_backend = build_backend(body.driver, body.config)
|
||||
await new_backend.test_connection()
|
||||
except Exception as exc:
|
||||
migration._status.state = "idle"
|
||||
raise HTTPException(status_code=400, detail=f"Backend validation failed: {exc}")
|
||||
|
||||
background_tasks.add_task(migration.run_migration, new_backend)
|
||||
return {"status": "started", "driver": body.driver}
|
||||
|
||||
|
||||
@router.get("/migrate/status")
|
||||
async def migration_status():
|
||||
"""Poll this to track migration progress."""
|
||||
return migration.get_status()
|
||||
|
||||
|
||||
@router.delete("/migrate", status_code=204)
|
||||
async def cancel_migration():
|
||||
"""
|
||||
Request cancellation of a running migration.
|
||||
The old backend remains active. Returns 409 if no migration is running.
|
||||
"""
|
||||
cancelled = await migration.cancel()
|
||||
if not cancelled:
|
||||
raise HTTPException(status_code=409, detail="No cancellable migration in progress")
|
||||
|
||||
|
||||
@router.patch("/backend-config", status_code=204)
|
||||
async def update_backend_config(body: BackendConfigRequest):
|
||||
"""
|
||||
Reconfigure the active backend without migrating data (e.g. update S3 credentials
|
||||
for the same endpoint, or switch back to local after a failed migration).
|
||||
|
||||
Use POST /migrate when you need data to be moved to the new backend.
|
||||
"""
|
||||
if migration.is_in_progress():
|
||||
raise HTTPException(status_code=409, detail="Cannot reconfigure while migration is in progress")
|
||||
try:
|
||||
new_backend = build_backend(body.driver, body.config)
|
||||
await new_backend.test_connection()
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=400, detail=f"Backend validation failed: {exc}")
|
||||
switch_backend(new_backend)
|
||||
@@ -0,0 +1,60 @@
|
||||
from fastapi import APIRouter, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
|
||||
from app.services.backend_manager import get_backend
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _validate_key(key: str) -> str:
|
||||
"""Reject path traversal. Key may contain '/' for nested objects (e.g. user/doc.pdf)."""
|
||||
parts = key.split("/")
|
||||
if ".." in parts:
|
||||
raise HTTPException(status_code=400, detail="Invalid key: path traversal not allowed")
|
||||
return key
|
||||
|
||||
|
||||
@router.put("/objects/{bucket}/{key:path}", status_code=204)
|
||||
async def put_object(bucket: str, key: str, request: Request):
|
||||
"""Upload raw bytes. Body is read as-is (application/octet-stream)."""
|
||||
_validate_key(key)
|
||||
data = await request.body()
|
||||
try:
|
||||
await get_backend().put(bucket, key, data)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc))
|
||||
|
||||
|
||||
@router.get("/objects/{bucket}/{key:path}")
|
||||
async def get_object(bucket: str, key: str):
|
||||
"""Download raw bytes."""
|
||||
_validate_key(key)
|
||||
try:
|
||||
data = await get_backend().get(bucket, key)
|
||||
except KeyError:
|
||||
raise HTTPException(status_code=404, detail="Object not found")
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc))
|
||||
return Response(content=data, media_type="application/octet-stream")
|
||||
|
||||
|
||||
@router.delete("/objects/{bucket}/{key:path}", status_code=204)
|
||||
async def delete_object(bucket: str, key: str):
|
||||
"""Delete an object. No-op if it does not exist."""
|
||||
_validate_key(key)
|
||||
try:
|
||||
await get_backend().delete(bucket, key)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc))
|
||||
|
||||
|
||||
@router.get("/objects/{bucket}")
|
||||
async def list_objects(bucket: str):
|
||||
"""List all keys in a bucket."""
|
||||
try:
|
||||
keys = await get_backend().list_keys(bucket)
|
||||
except Exception as exc:
|
||||
raise HTTPException(status_code=500, detail=str(exc))
|
||||
return {"bucket": bucket, "keys": keys}
|
||||
@@ -0,0 +1,70 @@
|
||||
import logging
|
||||
|
||||
from app.core.config import settings
|
||||
from app.services.backends.base import AbstractStorageBackend
|
||||
from app.services.backends.local import LocalFSBackend
|
||||
from app.services.backends.s3 import S3Backend
|
||||
from app.services.backends.webdav import WebDAVBackend
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_active_backend: AbstractStorageBackend | None = None
|
||||
|
||||
|
||||
def build_backend(driver: str, config: dict) -> AbstractStorageBackend:
|
||||
"""Construct a backend instance from a driver name + config dict."""
|
||||
if driver == "local":
|
||||
return LocalFSBackend(data_dir=config.get("data_dir", settings.DATA_DIR))
|
||||
if driver == "s3":
|
||||
return S3Backend(
|
||||
endpoint_url=config.get("endpoint_url", ""),
|
||||
access_key=config.get("access_key", ""),
|
||||
secret_key=config.get("secret_key", ""),
|
||||
region=config.get("region", "us-east-1"),
|
||||
)
|
||||
if driver == "webdav":
|
||||
return WebDAVBackend(
|
||||
url=config.get("url", ""),
|
||||
username=config.get("username", ""),
|
||||
password=config.get("password", ""),
|
||||
root_path=config.get("root_path", "/"),
|
||||
)
|
||||
raise ValueError(f"Unknown driver: {driver!r}. Valid options: local, s3, webdav")
|
||||
|
||||
|
||||
def initialize_backend() -> None:
|
||||
"""Build the initial backend from environment variables at startup."""
|
||||
global _active_backend
|
||||
driver = settings.STORAGE_BACKEND
|
||||
config: dict = {}
|
||||
if driver == "s3":
|
||||
config = {
|
||||
"endpoint_url": settings.S3_ENDPOINT_URL,
|
||||
"access_key": settings.S3_ACCESS_KEY,
|
||||
"secret_key": settings.S3_SECRET_KEY,
|
||||
"region": settings.S3_REGION,
|
||||
}
|
||||
elif driver == "webdav":
|
||||
config = {
|
||||
"url": settings.WEBDAV_URL,
|
||||
"username": settings.WEBDAV_USERNAME,
|
||||
"password": settings.WEBDAV_PASSWORD,
|
||||
"root_path": settings.WEBDAV_ROOT_PATH,
|
||||
}
|
||||
# local needs no extra config — DATA_DIR is read from settings inside build_backend
|
||||
_active_backend = build_backend(driver, config)
|
||||
logger.info("Storage backend initialized: %s", driver)
|
||||
|
||||
|
||||
def get_backend() -> AbstractStorageBackend:
|
||||
if _active_backend is None:
|
||||
raise RuntimeError("Backend not initialized — call initialize_backend() at startup")
|
||||
return _active_backend
|
||||
|
||||
|
||||
def switch_backend(new_backend: AbstractStorageBackend) -> None:
|
||||
"""Replace the active backend. Called by the migration job after all data is verified."""
|
||||
global _active_backend
|
||||
old_name = _active_backend.driver_name if _active_backend else "none"
|
||||
_active_backend = new_backend
|
||||
logger.info("Storage backend switched: %s → %s", old_name, new_backend.driver_name)
|
||||
@@ -0,0 +1,34 @@
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
|
||||
class AbstractStorageBackend(ABC):
|
||||
"""Common interface every storage backend must implement."""
|
||||
|
||||
@property
|
||||
@abstractmethod
|
||||
def driver_name(self) -> str:
|
||||
"""Short identifier returned in /health: 'local', 's3', or 'webdav'."""
|
||||
|
||||
@abstractmethod
|
||||
async def put(self, bucket: str, key: str, data: bytes) -> None:
|
||||
"""Store *data* under bucket/key. Creates bucket/intermediate dirs as needed."""
|
||||
|
||||
@abstractmethod
|
||||
async def get(self, bucket: str, key: str) -> bytes:
|
||||
"""Return the stored bytes. Raises KeyError if the object does not exist."""
|
||||
|
||||
@abstractmethod
|
||||
async def delete(self, bucket: str, key: str) -> None:
|
||||
"""Delete the object. No-op if it does not exist."""
|
||||
|
||||
@abstractmethod
|
||||
async def list_keys(self, bucket: str) -> list[str]:
|
||||
"""Return all keys stored in *bucket*. Returns [] if bucket is empty/absent."""
|
||||
|
||||
@abstractmethod
|
||||
async def exists(self, bucket: str, key: str) -> bool:
|
||||
"""Return True if the object exists."""
|
||||
|
||||
@abstractmethod
|
||||
async def test_connection(self) -> None:
|
||||
"""Verify the backend is reachable and writable. Raise on failure."""
|
||||
@@ -0,0 +1,67 @@
|
||||
import asyncio
|
||||
from pathlib import Path
|
||||
|
||||
import aiofiles
|
||||
|
||||
from .base import AbstractStorageBackend
|
||||
|
||||
|
||||
class LocalFSBackend(AbstractStorageBackend):
|
||||
"""Stores objects as files under <data_dir>/<bucket>/<key>."""
|
||||
|
||||
def __init__(self, data_dir: str) -> None:
|
||||
self._root = Path(data_dir)
|
||||
self._root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
@property
|
||||
def driver_name(self) -> str:
|
||||
return "local"
|
||||
|
||||
def _resolve(self, bucket: str, key: str) -> Path:
|
||||
safe_key = key.lstrip("/")
|
||||
if ".." in safe_key.split("/"):
|
||||
raise ValueError(f"Invalid key: {key!r}")
|
||||
return self._root / bucket / safe_key
|
||||
|
||||
async def put(self, bucket: str, key: str, data: bytes) -> None:
|
||||
dest = self._resolve(bucket, key)
|
||||
await asyncio.to_thread(dest.parent.mkdir, parents=True, exist_ok=True)
|
||||
async with aiofiles.open(dest, "wb") as f:
|
||||
await f.write(data)
|
||||
|
||||
async def get(self, bucket: str, key: str) -> bytes:
|
||||
path = self._resolve(bucket, key)
|
||||
if not path.exists():
|
||||
raise KeyError(f"{bucket}/{key}")
|
||||
async with aiofiles.open(path, "rb") as f:
|
||||
return await f.read()
|
||||
|
||||
async def delete(self, bucket: str, key: str) -> None:
|
||||
path = self._resolve(bucket, key)
|
||||
try:
|
||||
await asyncio.to_thread(path.unlink, missing_ok=True)
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
async def list_keys(self, bucket: str) -> list[str]:
|
||||
bucket_dir = self._root / bucket
|
||||
if not bucket_dir.exists():
|
||||
return []
|
||||
|
||||
def _scan() -> list[str]:
|
||||
return [
|
||||
str(p.relative_to(bucket_dir))
|
||||
for p in bucket_dir.rglob("*")
|
||||
if p.is_file()
|
||||
]
|
||||
|
||||
return await asyncio.to_thread(_scan)
|
||||
|
||||
async def exists(self, bucket: str, key: str) -> bool:
|
||||
return self._resolve(bucket, key).exists()
|
||||
|
||||
async def test_connection(self) -> None:
|
||||
self._root.mkdir(parents=True, exist_ok=True)
|
||||
probe = self._root / ".health_probe"
|
||||
probe.write_bytes(b"ok")
|
||||
probe.unlink()
|
||||
@@ -0,0 +1,99 @@
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from aiobotocore.session import get_session
|
||||
|
||||
from .base import AbstractStorageBackend
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
class S3Backend(AbstractStorageBackend):
|
||||
"""
|
||||
S3-compatible backend. Works with AWS S3, MinIO, Backblaze B2, Cloudflare R2, etc.
|
||||
Set endpoint_url to the service URL for non-AWS providers; leave empty for real AWS.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
endpoint_url: str,
|
||||
access_key: str,
|
||||
secret_key: str,
|
||||
region: str = "us-east-1",
|
||||
) -> None:
|
||||
self._endpoint_url = endpoint_url or None
|
||||
self._access_key = access_key
|
||||
self._secret_key = secret_key
|
||||
self._region = region
|
||||
self._session = get_session()
|
||||
|
||||
@property
|
||||
def driver_name(self) -> str:
|
||||
return "s3"
|
||||
|
||||
@asynccontextmanager
|
||||
async def _client(self):
|
||||
async with self._session.create_client(
|
||||
"s3",
|
||||
endpoint_url=self._endpoint_url,
|
||||
aws_access_key_id=self._access_key,
|
||||
aws_secret_access_key=self._secret_key,
|
||||
region_name=self._region,
|
||||
) as client:
|
||||
yield client
|
||||
|
||||
async def _ensure_bucket(self, client, bucket: str) -> None:
|
||||
try:
|
||||
await client.head_bucket(Bucket=bucket)
|
||||
except Exception:
|
||||
try:
|
||||
if self._region == "us-east-1":
|
||||
await client.create_bucket(Bucket=bucket)
|
||||
else:
|
||||
await client.create_bucket(
|
||||
Bucket=bucket,
|
||||
CreateBucketConfiguration={"LocationConstraint": self._region},
|
||||
)
|
||||
except Exception as exc:
|
||||
logger.debug("Bucket create skipped (may already exist): %s", exc)
|
||||
|
||||
async def put(self, bucket: str, key: str, data: bytes) -> None:
|
||||
async with self._client() as client:
|
||||
await self._ensure_bucket(client, bucket)
|
||||
await client.put_object(Bucket=bucket, Key=key, Body=data)
|
||||
|
||||
async def get(self, bucket: str, key: str) -> bytes:
|
||||
async with self._client() as client:
|
||||
try:
|
||||
response = await client.get_object(Bucket=bucket, Key=key)
|
||||
return await response["Body"].read()
|
||||
except Exception as exc:
|
||||
raise KeyError(f"{bucket}/{key}") from exc
|
||||
|
||||
async def delete(self, bucket: str, key: str) -> None:
|
||||
async with self._client() as client:
|
||||
await client.delete_object(Bucket=bucket, Key=key)
|
||||
|
||||
async def list_keys(self, bucket: str) -> list[str]:
|
||||
async with self._client() as client:
|
||||
try:
|
||||
paginator = client.get_paginator("list_objects_v2")
|
||||
keys: list[str] = []
|
||||
async for page in paginator.paginate(Bucket=bucket):
|
||||
for obj in page.get("Contents", []):
|
||||
keys.append(obj["Key"])
|
||||
return keys
|
||||
except Exception:
|
||||
return []
|
||||
|
||||
async def exists(self, bucket: str, key: str) -> bool:
|
||||
async with self._client() as client:
|
||||
try:
|
||||
await client.head_object(Bucket=bucket, Key=key)
|
||||
return True
|
||||
except Exception:
|
||||
return False
|
||||
|
||||
async def test_connection(self) -> None:
|
||||
async with self._client() as client:
|
||||
await client.list_buckets()
|
||||
@@ -0,0 +1,121 @@
|
||||
import base64
|
||||
from urllib.parse import quote
|
||||
|
||||
import defusedxml.ElementTree as ET
|
||||
|
||||
import aiohttp
|
||||
|
||||
from .base import AbstractStorageBackend
|
||||
|
||||
|
||||
class WebDAVBackend(AbstractStorageBackend):
|
||||
"""
|
||||
WebDAV backend. Compatible with Nextcloud and any standard WebDAV server.
|
||||
root_path should be the WebDAV root on the server, e.g. '/remote.php/dav/files/username'.
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
url: str,
|
||||
username: str,
|
||||
password: str,
|
||||
root_path: str = "/",
|
||||
) -> None:
|
||||
self._base = url.rstrip("/")
|
||||
self._root = root_path.rstrip("/")
|
||||
creds = base64.b64encode(f"{username}:{password}".encode()).decode()
|
||||
self._auth = f"Basic {creds}"
|
||||
|
||||
@property
|
||||
def driver_name(self) -> str:
|
||||
return "webdav"
|
||||
|
||||
def _url(self, *parts: str) -> str:
|
||||
encoded = "/".join(quote(p, safe="") for p in parts)
|
||||
return f"{self._base}{self._root}/{encoded}"
|
||||
|
||||
def _headers(self, extra: dict | None = None) -> dict[str, str]:
|
||||
h = {"Authorization": self._auth}
|
||||
if extra:
|
||||
h.update(extra)
|
||||
return h
|
||||
|
||||
async def _ensure_collection(self, session: aiohttp.ClientSession, *parts: str) -> None:
|
||||
"""MKCOL is idempotent — ignore 405 (already exists)."""
|
||||
url = self._url(*parts)
|
||||
async with session.request("MKCOL", url, headers=self._headers()) as resp:
|
||||
if resp.status not in (200, 201, 405):
|
||||
pass # best-effort; PUT will fail if directory is truly missing
|
||||
|
||||
async def put(self, bucket: str, key: str, data: bytes) -> None:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
await self._ensure_collection(session, bucket)
|
||||
parts = key.split("/")
|
||||
for i in range(1, len(parts)):
|
||||
await self._ensure_collection(session, bucket, *parts[:i])
|
||||
url = self._url(bucket, key)
|
||||
async with session.put(url, data=data, headers=self._headers()) as resp:
|
||||
if resp.status not in (200, 201, 204):
|
||||
raise OSError(f"WebDAV PUT {url} → {resp.status}")
|
||||
|
||||
async def get(self, bucket: str, key: str) -> bytes:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
url = self._url(bucket, key)
|
||||
async with session.get(url, headers=self._headers()) as resp:
|
||||
if resp.status == 404:
|
||||
raise KeyError(f"{bucket}/{key}")
|
||||
if resp.status != 200:
|
||||
raise OSError(f"WebDAV GET {url} → {resp.status}")
|
||||
return await resp.read()
|
||||
|
||||
async def delete(self, bucket: str, key: str) -> None:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
url = self._url(bucket, key)
|
||||
async with session.delete(url, headers=self._headers()) as resp:
|
||||
if resp.status not in (200, 204, 404):
|
||||
raise OSError(f"WebDAV DELETE {url} → {resp.status}")
|
||||
|
||||
async def list_keys(self, bucket: str) -> list[str]:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
url = self._url(bucket)
|
||||
headers = self._headers({"Depth": "infinity", "Content-Type": "application/xml"})
|
||||
body = '<?xml version="1.0"?><d:propfind xmlns:d="DAV:"><d:prop><d:resourcetype/></d:prop></d:propfind>'
|
||||
async with session.request("PROPFIND", url, headers=headers, data=body) as resp:
|
||||
if resp.status == 404:
|
||||
return []
|
||||
if resp.status != 207:
|
||||
return []
|
||||
xml_body = await resp.text()
|
||||
|
||||
ns = {"d": "DAV:"}
|
||||
try:
|
||||
root = ET.fromstring(xml_body)
|
||||
except ET.ParseError:
|
||||
return []
|
||||
|
||||
prefix = f"{self._base}{self._root}/{quote(bucket, safe='')}/"
|
||||
keys: list[str] = []
|
||||
for response in root.findall("d:response", ns):
|
||||
href = response.findtext("d:href", namespaces=ns) or ""
|
||||
prop = response.find(".//d:prop", ns)
|
||||
if prop is not None:
|
||||
rt = prop.find("d:resourcetype", ns)
|
||||
if rt is not None and rt.find("d:collection", ns) is not None:
|
||||
continue # skip directories
|
||||
if href.startswith(prefix):
|
||||
keys.append(href[len(prefix):])
|
||||
return keys
|
||||
|
||||
async def exists(self, bucket: str, key: str) -> bool:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
url = self._url(bucket, key)
|
||||
async with session.request("HEAD", url, headers=self._headers()) as resp:
|
||||
return resp.status == 200
|
||||
|
||||
async def test_connection(self) -> None:
|
||||
async with aiohttp.ClientSession() as session:
|
||||
root_url = f"{self._base}{self._root}/"
|
||||
headers = self._headers({"Depth": "0"})
|
||||
async with session.request("PROPFIND", root_url, headers=headers) as resp:
|
||||
if resp.status not in (200, 207):
|
||||
raise ConnectionError(f"WebDAV root PROPFIND → {resp.status}")
|
||||
@@ -0,0 +1,139 @@
|
||||
"""
|
||||
Backend migration service.
|
||||
|
||||
Flow:
|
||||
1. POST /migrate → validate new backend (test_connection)
|
||||
2. Background task enumerates all objects in all known buckets
|
||||
3. Each object is copied old → new, then verified
|
||||
4. Only after 100 % success: atomically switch active backend
|
||||
5. Delete all objects from old backend
|
||||
6. If any copy fails: old backend stays active; state = "failed"
|
||||
7. DELETE /migrate cancels a running migration (old backend stays active)
|
||||
"""
|
||||
|
||||
import logging
|
||||
from dataclasses import dataclass, field
|
||||
from typing import Literal
|
||||
|
||||
from app.services.backends.base import AbstractStorageBackend
|
||||
from app.services.backend_manager import get_backend, switch_backend
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# All logical buckets the service knows about — enumerated during migration.
|
||||
KNOWN_BUCKETS = ["documents", "config"]
|
||||
|
||||
MigrationState = Literal[
|
||||
"idle", "validating", "migrating", "switching", "cleaning", "done", "failed", "cancelled"
|
||||
]
|
||||
|
||||
|
||||
@dataclass
|
||||
class _MigrationStatus:
|
||||
state: MigrationState = "idle"
|
||||
total: int = 0
|
||||
done: int = 0
|
||||
failed: int = 0
|
||||
errors: list[str] = field(default_factory=list)
|
||||
|
||||
|
||||
_status = _MigrationStatus()
|
||||
_cancel_requested: bool = False
|
||||
|
||||
|
||||
def get_status() -> dict:
|
||||
return {
|
||||
"state": _status.state,
|
||||
"total": _status.total,
|
||||
"done": _status.done,
|
||||
"failed": _status.failed,
|
||||
"errors": _status.errors[:50], # cap to avoid huge responses
|
||||
}
|
||||
|
||||
|
||||
def is_in_progress() -> bool:
|
||||
return _status.state in ("validating", "migrating", "switching", "cleaning")
|
||||
|
||||
|
||||
async def cancel() -> bool:
|
||||
global _cancel_requested
|
||||
if _status.state == "migrating":
|
||||
_cancel_requested = True
|
||||
return True
|
||||
return False
|
||||
|
||||
|
||||
async def run_migration(new_backend: AbstractStorageBackend) -> None:
|
||||
"""
|
||||
Background task: copy all objects to new_backend, verify, switch, clean old.
|
||||
Called after the caller has already validated new_backend.test_connection().
|
||||
"""
|
||||
global _cancel_requested
|
||||
_cancel_requested = False
|
||||
old_backend = get_backend()
|
||||
|
||||
_status.state = "migrating"
|
||||
_status.done = 0
|
||||
_status.failed = 0
|
||||
_status.errors.clear()
|
||||
|
||||
try:
|
||||
# Collect all objects across every known bucket
|
||||
all_objects: list[tuple[str, str]] = []
|
||||
for bucket in KNOWN_BUCKETS:
|
||||
try:
|
||||
keys = await old_backend.list_keys(bucket)
|
||||
for key in keys:
|
||||
all_objects.append((bucket, key))
|
||||
except Exception as exc:
|
||||
logger.warning("Could not list bucket %r: %s", bucket, exc)
|
||||
|
||||
_status.total = len(all_objects)
|
||||
logger.info("Migration: %d objects to migrate across %d buckets", len(all_objects), len(KNOWN_BUCKETS))
|
||||
|
||||
for bucket, key in all_objects:
|
||||
if _cancel_requested:
|
||||
_status.state = "cancelled"
|
||||
logger.info("Migration cancelled (%d/%d done)", _status.done, _status.total)
|
||||
return
|
||||
|
||||
try:
|
||||
data = await old_backend.get(bucket, key)
|
||||
await new_backend.put(bucket, key, data)
|
||||
if not await new_backend.exists(bucket, key):
|
||||
raise OSError("Verification failed: object absent after PUT")
|
||||
_status.done += 1
|
||||
except Exception as exc:
|
||||
_status.failed += 1
|
||||
entry = f"{bucket}/{key}: {exc}"
|
||||
_status.errors.append(entry)
|
||||
logger.warning("Migration copy failed — %s", entry)
|
||||
|
||||
if _status.failed > 0:
|
||||
_status.state = "failed"
|
||||
logger.error(
|
||||
"Migration failed: %d/%d objects could not be copied; old backend remains active",
|
||||
_status.failed,
|
||||
_status.total,
|
||||
)
|
||||
return
|
||||
|
||||
# All objects verified — atomically switch
|
||||
_status.state = "switching"
|
||||
switch_backend(new_backend)
|
||||
|
||||
# Remove all objects from old backend (best-effort)
|
||||
_status.state = "cleaning"
|
||||
for bucket, key in all_objects:
|
||||
try:
|
||||
await old_backend.delete(bucket, key)
|
||||
except Exception as exc:
|
||||
logger.warning("Cleanup failed for %s/%s: %s", bucket, key, exc)
|
||||
|
||||
_status.state = "done"
|
||||
logger.info("Migration complete: %d objects moved to %s", _status.total, new_backend.driver_name)
|
||||
|
||||
except Exception as exc:
|
||||
_status.state = "failed"
|
||||
_status.errors.append(f"Unexpected error: {exc}")
|
||||
logger.exception("Migration aborted with unexpected error")
|
||||
@@ -0,0 +1,25 @@
|
||||
[build-system]
|
||||
requires = ["setuptools>=45"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "storage-service"
|
||||
version = "0.1.0"
|
||||
requires-python = ">=3.11"
|
||||
dependencies = [
|
||||
"fastapi>=0.111",
|
||||
"uvicorn[standard]>=0.29",
|
||||
"pydantic-settings>=2.2",
|
||||
"aiofiles>=23.0",
|
||||
"aiobotocore>=2.13",
|
||||
"aiohttp>=3.9",
|
||||
"defusedxml>=0.7",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"ruff>=0.4",
|
||||
]
|
||||
|
||||
[tool.ruff]
|
||||
line-length = 100
|
||||
Executable
+5
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "[storage-service] starting uvicorn..."
|
||||
exec uvicorn app.main:app --host 0.0.0.0 --port 8020
|
||||
+5
@@ -0,0 +1,5 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "[storage-service] starting uvicorn (dev)..."
|
||||
exec uvicorn app.main:app --host 0.0.0.0 --port 8020 --reload
|
||||
@@ -0,0 +1,190 @@
|
||||
# frontend — Claude context
|
||||
|
||||
React 18 SPA built with Vite, port 5173 dev / 80 prod, served by nginx-unprivileged in production. All `/api/*` requests are proxied to `backend:8000`. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
All commands run inside Docker — never on the host.
|
||||
|
||||
```bash
|
||||
docker compose exec frontend npm run typecheck
|
||||
docker compose exec frontend npm run lint
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
frontend/
|
||||
├── src/
|
||||
│ ├── main.tsx ← React root, QueryClientProvider, BrowserRouter
|
||||
│ ├── App.tsx ← Route tree, PrivateRoute, AdminRoute
|
||||
│ ├── api/client.ts ← Native fetch wrapper (`request()`) + ALL API functions (single source of truth); no Axios
|
||||
│ ├── hooks/
|
||||
│ │ ├── useAuth.ts ← Token state (localStorage), login/logout
|
||||
│ │ └── useTheme.ts ← Theme toggle
|
||||
│ ├── components/
|
||||
│ │ ├── AppShell.tsx ← Layout: Sidebar + SourcePanel (on /apps/documents) + main
|
||||
│ │ ├── Sidebar.tsx ← Collapsible nav (icons ↔ icons+labels)
|
||||
│ │ ├── SourcePanel.tsx ← Views + searchable category tree (docs route only)
|
||||
│ │ ├── ManageCategoriesDialog.tsx ← Category CRUD modal (rename, delete)
|
||||
│ │ ├── DocumentSlideOver.tsx ← Right slide-over: detail, edit, share, AI suggestions
|
||||
│ │ ├── ThemeToggle.tsx ← Light/dark mode toggle
|
||||
│ │ ├── PluginSchemaForm.tsx ← JSON Schema → React form (boolean/string/number/readOnly)
|
||||
│ │ └── ui/ ← shadcn/ui components (Button, Input, …)
|
||||
│ ├── pages/ ← One file per route
|
||||
│ │ ├── DocServiceSettingsPage.tsx ← Combined doc-service settings: upload limits + watch directory
|
||||
│ │ ├── PluginSettingsPage.tsx ← Generic plugin settings page driven by manifest
|
||||
│ │ └── StorageAdminPage.tsx ← Admin storage backend config + live migration progress
|
||||
│ ├── lib/utils.ts ← cn() = clsx + tailwind-merge
|
||||
│ └── styles/theme.css ← CSS custom properties, Tailwind setup
|
||||
├── vite.config.ts ← /api/* proxied to backend:8000
|
||||
├── tailwind.config.ts
|
||||
├── components.json ← shadcn/ui config
|
||||
├── Dockerfile ← Multi-stage: Node build → nginx-unprivileged
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Frontend Routes
|
||||
|
||||
| Path | Component | Guard |
|
||||
|------|-----------|-------|
|
||||
| `/login` | `LoginPage` | Public |
|
||||
| `/` | `DashboardPage` | PrivateRoute |
|
||||
| `/apps` | `AppsPage` | PrivateRoute |
|
||||
| `/apps/documents` | `DocumentsPage` | PrivateRoute |
|
||||
| `/apps/documents/settings` | `DocServiceSettingsPage` | ServiceAdminRoute (is_admin OR doc-service-admin member) |
|
||||
| `/apps/ai/settings` | `AIAdminSettingsPage` | ServiceAdminRoute (is_admin OR ai-service-admin member) |
|
||||
| `/profile` | `ProfilePage` | PrivateRoute |
|
||||
| `/settings` | `SettingsPage` | PrivateRoute |
|
||||
| `/settings/plugins/:id` | `PluginSettingsPage` | PrivateRoute (auth enforced per-plugin by backend) |
|
||||
| `/admin` | `AdminPage` (→ `/admin/users`) | AdminRoute |
|
||||
| `/admin/users` | `AdminUsersPage` | AdminRoute |
|
||||
| `/admin/groups` | `AdminGroupsPage` | AdminRoute |
|
||||
| `/admin/appearance` | `AdminAppearancePage` | AdminRoute |
|
||||
| `/admin/storage` | `StorageAdminPage` | AdminRoute |
|
||||
| `*` | redirect to `/` | — |
|
||||
|
||||
`PrivateRoute` — checks `token` from `useAuth`, redirects to `/login` if absent.
|
||||
`AdminRoute` — checks token AND queries `GET /api/users/me` for `is_admin`; waits for query to avoid flash; redirects to `/login` (not `/`) if not admin.
|
||||
|
||||
---
|
||||
|
||||
## Security Standards
|
||||
|
||||
### XSS prevention
|
||||
|
||||
- React JSX text interpolation (`{value}`) is HTML-escaped by the DOM renderer — **never** use `dangerouslySetInnerHTML` with user-supplied content.
|
||||
- Server-side `sanitize_str` provides defense-in-depth (control char stripping, max length).
|
||||
|
||||
---
|
||||
|
||||
## Frontend Patterns & Conventions
|
||||
|
||||
### API client (`src/api/client.ts`)
|
||||
|
||||
**No Axios** — uses a thin native `fetch` wrapper. All API calls live here, nowhere else.
|
||||
|
||||
The core `request()` function handles:
|
||||
- Prepends `/api` base URL
|
||||
- Injects `Authorization: Bearer {token}` from `localStorage` on every request
|
||||
- **Global 401 handler**: clears `localStorage` token and redirects to `/login` via `window.location.href` — this is the expired-session redirect
|
||||
- Throws `ApiError(status, detail)` on non-2xx responses (detail parsed from JSON body)
|
||||
- Returns `undefined` on 204 No Content
|
||||
- Supports `blob: true` for file download/preview responses
|
||||
|
||||
```typescript
|
||||
// The internal api object — use these methods in exported functions:
|
||||
api.get<T>(path, params?) // GET with optional query params object
|
||||
api.post<T>(path, json?) // POST with JSON body
|
||||
api.postForm<T>(path, URLSearchParams) // POST with form-encoded body (login)
|
||||
api.postFile<T>(path, FormData) // POST with multipart body (file upload)
|
||||
api.patch<T>(path, json?) // PATCH with JSON body
|
||||
api.delete<T>(path) // DELETE
|
||||
api.getBlob(path) // GET → Blob (download / view)
|
||||
```
|
||||
|
||||
Adding a new API call:
|
||||
1. Define a TypeScript interface for the response if it's new.
|
||||
2. Add a named export function (`getX`, `createX`, `updateX`, `deleteX`).
|
||||
3. Use the appropriate `api.*` method — return the promise directly (no `.then((r) => r.data)`).
|
||||
|
||||
Error handling in components: catch blocks receive an `ApiError` instance with `.status` and `.message` (the detail string).
|
||||
|
||||
### TanStack Query conventions
|
||||
|
||||
**Query keys** (flat arrays, lowercase):
|
||||
```typescript
|
||||
["me"] // current user
|
||||
["services"] // service health list
|
||||
["dashboard-prefs"] // user dashboard preferences
|
||||
["categories"] // document categories
|
||||
["documents", params] // document list (params object for cache isolation)
|
||||
["documents-shared", params] // shared-with-me list
|
||||
["document", id] // single document
|
||||
["document-shares", id] // share list for a specific document
|
||||
["my-groups"] // current user's group memberships (for share picker)
|
||||
["plugins"] // accessible plugin list (filtered by user access)
|
||||
["plugin-manifest", id] // plugin manifest (cached)
|
||||
["plugin-settings", id] // plugin current settings
|
||||
```
|
||||
|
||||
**Mutation pattern**:
|
||||
```typescript
|
||||
const mutation = useMutation({
|
||||
mutationFn: apiFunction,
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["affected-key"] });
|
||||
// additional side effects (close dialog, reset form, etc.)
|
||||
},
|
||||
});
|
||||
// Usage:
|
||||
mutation.mutate(data);
|
||||
mutation.isPending // show spinner / disable button
|
||||
mutation.isError // show error message
|
||||
```
|
||||
|
||||
**Polling**:
|
||||
```typescript
|
||||
useQuery({ queryKey: ["services"], queryFn: getServices,
|
||||
refetchInterval: 30_000, refetchIntervalInBackground: true });
|
||||
```
|
||||
|
||||
### Route guards
|
||||
|
||||
```typescript
|
||||
// PrivateRoute — redirect to /login if no token
|
||||
// AdminRoute — redirect to /login if no token OR not admin
|
||||
// (waits for getMe() query to avoid flash; uses 404 semantics)
|
||||
```
|
||||
|
||||
### Component patterns
|
||||
|
||||
- Functional components only.
|
||||
- Local `useState` for UI-only state (edit mode, pending values, open/closed).
|
||||
- Server state via `useQuery` / `useMutation` — no duplicated local copies.
|
||||
- `cn()` from `lib/utils.ts` for conditional Tailwind classes.
|
||||
- `lucide-react` for all icons.
|
||||
- Never use `dangerouslySetInnerHTML` with user-supplied content.
|
||||
|
||||
---
|
||||
|
||||
## Naming & Code Conventions
|
||||
|
||||
- TypeScript strict mode — no `any`.
|
||||
- API response types inferred from interfaces in `client.ts` only.
|
||||
- Error messages displayed inline (no alert); loading shown as disabled state or "…" text.
|
||||
- All user-facing text: safe via React JSX rendering (not innerHTML).
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| Token localStorage key | `"token"` | `useAuth.ts` |
|
||||
+116
-100
@@ -16,13 +16,15 @@ All API calls go through `src/api/client.ts` (single Axios instance, JWT injecte
|
||||
| `/` | `DashboardPage` | Required |
|
||||
| `/apps` | `AppsPage` | Required |
|
||||
| `/apps/documents` | `DocumentsPage` | Required |
|
||||
| `/apps/documents/settings/admin` | `DocumentAdminSettingsPage` | Admin only |
|
||||
| `/apps/ai/settings/admin` | `AIAdminSettingsPage` | Admin only |
|
||||
| `/apps/documents/settings` | `DocServiceSettingsPage` | ServiceAdminRoute (is_admin OR doc-service-admin) |
|
||||
| `/apps/ai/settings` | `AIAdminSettingsPage` | ServiceAdminRoute (is_admin OR ai-service-admin) |
|
||||
| `/admin` | `AdminPage` (redirects to `/admin/users`) | Admin only |
|
||||
| `/admin/users` | `AdminUsersPage` | Admin only |
|
||||
| `/admin/groups` | `AdminGroupsPage` | Admin only |
|
||||
| `/admin/storage` | `StorageAdminPage` | Admin only |
|
||||
| `/profile` | `ProfilePage` | Required |
|
||||
| `/settings` | `SettingsPage` (placeholder) | Required |
|
||||
| `/settings/plugins/:id` | `PluginSettingsPage` | Required (per-plugin access control) |
|
||||
|
||||
`PrivateRoute` redirects to `/login` when no token. `AdminRoute` redirects to `/` when not admin.
|
||||
|
||||
@@ -39,127 +41,133 @@ All API calls go through `src/api/client.ts` (single Axios instance, JWT injecte
|
||||
### Home dashboard (`/`)
|
||||
|
||||
Personalised landing page per user:
|
||||
- Time-aware greeting with the user's display name (`full_name` or email). React JSX text rendering HTML-escapes all values — no `dangerouslySetInnerHTML` is used anywhere on this page.
|
||||
- Grid of **pinned app cards** drawn from `GET /api/services`, filtered to the user's saved list.
|
||||
- **Customize mode** (pencil button): shows all services; `+` / `−` toggle buttons on each card; changes committed with **Save** via `PATCH /api/users/me/preferences`.
|
||||
- Empty-state prompt when no apps are pinned.
|
||||
- Time-aware greeting with the user's display name
|
||||
- Grid of **pinned app cards** from `GET /api/services`, filtered to user's saved list
|
||||
- **Customize mode** (pencil button): shows all services; `+` / `−` toggle; commits via `PATCH /api/users/me/preferences`
|
||||
|
||||
### Apps page (`/apps`)
|
||||
|
||||
Cards are rendered dynamically from `GET /api/services` (polled every 30 s via TanStack Query):
|
||||
- **healthy=true + app_path set** — clickable card with "Available" badge
|
||||
- **healthy=true + no app_path** — non-clickable card (e.g. AI Service — no user UI)
|
||||
- **healthy=false** — non-clickable, dimmed card with "Unavailable" badge and explanation text
|
||||
- Admin settings link shown for admins regardless of health status
|
||||
Cards from `GET /api/services` (polled every 30 s):
|
||||
- healthy + app_path → clickable card with "Available" badge
|
||||
- healthy + no app_path → non-clickable card
|
||||
- unhealthy → dimmed, non-clickable, "Unavailable"
|
||||
- Settings button visible to admins and service-admin group members
|
||||
|
||||
### Sidebar navigation
|
||||
|
||||
`Apps` is an expandable accordion in the sidebar:
|
||||
- **Documents** sub-item (expandable) — lists all user categories beneath it; clicking a category navigates to `/apps/documents?category_id=<id>`
|
||||
- AI Service is not listed (no openable UI)
|
||||
- Sections auto-open when navigating to their route
|
||||
- In collapsed (icons-only) mode, clicking the Apps icon navigates to `/apps`
|
||||
`Apps` expandable accordion: **Documents** single NavLink to `/apps/documents`. Category navigation moved to SourcePanel (only visible on `/apps/documents` route). Admin section (Users, Groups, Appearance) for admins. Collapsible to icon-only mode.
|
||||
|
||||
### Documents page (`/apps/documents`)
|
||||
### Documents page (`/apps/documents`) — three-column layout
|
||||
|
||||
**Upload:** PDF file input, 202 response, error display.
|
||||
**SourcePanel** (240px, left): Appears only on `/apps/documents`.
|
||||
- Views: All Documents / Mine / Shared with me (URL param `?view=`)
|
||||
- Category tree with client-side search (searchable when > 4 categories)
|
||||
- Inline new category form
|
||||
- "Manage categories" button opens `ManageCategoriesDialog`
|
||||
|
||||
**Filter bar:**
|
||||
- Search input (400ms debounce) — matches title, filename, tags, document_type
|
||||
- Status dropdown (all / pending / processing / done / failed)
|
||||
- Type dropdown (all / invoice / bill / receipt / order / expense / revenue / unknown)
|
||||
- Sort selector (upload date / processed date / title / filename / file size / type / status)
|
||||
- Asc/Desc toggle
|
||||
- "Clear filters" button (appears when any filter is active)
|
||||
**Toolbar:** Debounced search input (400ms) + filter chips system.
|
||||
- Filter chips: Status, Document type, Category (each adds a removable chip)
|
||||
- "Add filter" button opens a two-step picker (dimension → value)
|
||||
- Sort via clickable column headers (↑/↓ chevron)
|
||||
|
||||
**Pagination:** Prev/Next with "X–Y of Z" count. Only shown when total > per_page.
|
||||
**Compact table rows:**
|
||||
- Columns: checkbox | title/filename | type | status dot | categories (2 + overflow) | sharing icon | date | size | 3-dot actions
|
||||
- Row click opens `DocumentSlideOver`
|
||||
- Shared-with-me rows show a primary border accent
|
||||
|
||||
**Document row (collapsed):**
|
||||
- Inline title editor (pencil icon, Enter to save, Esc to cancel; shows filename in italic when no title)
|
||||
- Status badge (colour-coded)
|
||||
- Document type label
|
||||
- File size
|
||||
- View button (opens PDF in new tab via blob URL — auth-gated)
|
||||
- Download button
|
||||
- Delete button (confirm dialog)
|
||||
**DocumentSlideOver** (480px, right slide-over):
|
||||
- Metadata (status dot, size, dates, source)
|
||||
- Inline title edit (pencil icon)
|
||||
- Type picker (chips for each doc type)
|
||||
- **AI Suggestions** — folder and filename confirm/reject buttons (was missing before, now implemented)
|
||||
- Extracted data key-value table
|
||||
- Categories multi-select combobox (search-to-filter)
|
||||
- AI-suggested categories with Assign / Create & Assign actions
|
||||
- Tags chip editor (add/remove inline)
|
||||
- **Sharing section** (owner only): lists groups the doc is shared with; group picker combobox (filtered to user's own groups); remove share button
|
||||
- Raw text section (collapsed by default)
|
||||
- Re-analyse / Delete actions (owner only)
|
||||
|
||||
**Document row (expanded):**
|
||||
- **Extracted data table** — all AI-extracted JSON fields (excludes `tags`, `suggested_categories`)
|
||||
- **Error message** — shown if status=failed
|
||||
- **Categories** — assigned chips with remove; dropdown to assign existing; AI-suggested chips with Accept / Create & Assign / Dismiss
|
||||
- **Status polling** — auto-refetches every 3s while status is pending/processing; invalidates document list on done/failed
|
||||
**Bulk actions bar** (floating, bottom center, owner view only):
|
||||
- Appears when any rows are checked
|
||||
- Share with group (opens group picker → shares all selected)
|
||||
- Delete (confirm dialog)
|
||||
- Clear selection
|
||||
|
||||
### AI Admin Settings (`/apps/ai/settings/admin`)
|
||||
**Upload experience:**
|
||||
- Full-page drag-and-drop overlay (activates on `dragenter`)
|
||||
- Multi-file upload (iterates all selected/dropped files)
|
||||
- Bottom-right upload queue panel (collapsible toast) with per-file status + "Review →" link after upload
|
||||
|
||||
- Provider selector (lmstudio / ollama / anthropic)
|
||||
- Per-provider fields (base URL, model, API key)
|
||||
- Test Connection button (`POST /api/settings/ai/test`)
|
||||
- Save button
|
||||
**Document sharing:**
|
||||
- Owner shares doc with any of their own groups from the slide-over
|
||||
- Recipient sees shared docs in "Shared with me" view
|
||||
- Recipient can View + Download only (no edit/delete/share)
|
||||
- `share_count` indicator (Users icon) in table rows
|
||||
|
||||
### Document Admin Settings (`/apps/documents/settings/admin`)
|
||||
**Polling:** List query refetches every 3s automatically when any visible doc is pending/processing (single query, not per-document). Uses TanStack Query `refetchInterval` function.
|
||||
|
||||
- Upload Limits section only (max PDF size in MB)
|
||||
- Save button
|
||||
### AI Service Settings (`/apps/ai/settings`)
|
||||
|
||||
Provider selector, per-provider fields, Test Connection, Save.
|
||||
|
||||
### Document Service Settings (`/apps/documents/settings`)
|
||||
|
||||
Upload limits + watch directory config.
|
||||
|
||||
### Admin — Storage page (`/admin/storage`)
|
||||
|
||||
Current backend status (green/red health dot). Driver selector (local/S3/WebDAV) with conditional credential fields. "Test & Migrate" button triggers an async migration that copies all objects to the new backend, verifies, then switches atomically. Live progress bar with 2s polling (states: validating → migrating → switching → cleaning → done). Cancel button during in-progress migrations.
|
||||
|
||||
### Admin — Users page (`/admin/users`)
|
||||
|
||||
- User list with role and active status
|
||||
- Inline active status toggle
|
||||
- Create user form (email, name, password, admin flag)
|
||||
- Delete user
|
||||
User list, toggle active, create user, delete user.
|
||||
|
||||
### Admin — Groups page (`/admin/groups`)
|
||||
|
||||
- Group list with name, description, member count
|
||||
- Create group (name, optional description)
|
||||
- Edit group name / description inline panel
|
||||
- Delete group (with confirmation)
|
||||
- Expand group row to manage members: view members, remove members, add non-members from dropdown
|
||||
Group list, create, edit name/description, delete, add/remove members.
|
||||
|
||||
### Profile page (`/profile`)
|
||||
|
||||
- Display and edit personal information
|
||||
Display and edit personal information.
|
||||
|
||||
---
|
||||
|
||||
## API client (`src/api/client.ts`)
|
||||
|
||||
Key functions:
|
||||
**No Axios** — replaced with a native `fetch` wrapper (`request()`). Global 401 handler clears the token and redirects to `/login`, fixing the expired-session blank-page bug. All exported function signatures are unchanged.
|
||||
|
||||
Key document-related functions:
|
||||
|
||||
| Function | Description |
|
||||
|----------|-------------|
|
||||
| `listDocuments(params)` | `GET /documents` — returns `DocumentPage`; supports `category_id` filter |
|
||||
| `listDocuments(params)` | `GET /documents` — paginated with filters |
|
||||
| `listSharedWithMe(params)` | `GET /documents/shared-with-me` |
|
||||
| `uploadDocument(file)` | `POST /documents/upload` |
|
||||
| `deleteDocument(id)` | `DELETE /documents/{id}` |
|
||||
| `downloadDocument(id, filename)` | Blob URL download |
|
||||
| `viewDocument(id)` | Blob URL → `window.open`, auto-revoke after 60s |
|
||||
| `getDocumentStatus(id)` | Poll endpoint |
|
||||
| `listCategories()` | All categories for user |
|
||||
| `createCategory(name)` | Create category |
|
||||
| `assignCategory(docId, catId)` | Assign |
|
||||
| `removeCategory(docId, catId)` | Remove |
|
||||
| `updateDocumentTags(id, tags)` | `PATCH /documents/{id}/tags` |
|
||||
| `updateDocumentTitle(id, title)` | `PATCH /documents/{id}/title` |
|
||||
| `getAISettings()` | `GET /settings/ai` (masked) |
|
||||
| `updateAISettings(data)` | `PATCH /settings/ai` |
|
||||
| `testAIConnection()` | `POST /settings/ai/test` |
|
||||
| `getDocumentLimits()` | `GET /settings/documents/limits` |
|
||||
| `adminListGroups()` | `GET /admin/groups` |
|
||||
| `adminCreateGroup(data)` | `POST /admin/groups` |
|
||||
| `adminGetGroup(id)` | `GET /admin/groups/{id}` with members |
|
||||
| `adminUpdateGroup(id, data)` | `PATCH /admin/groups/{id}` |
|
||||
| `adminDeleteGroup(id)` | `DELETE /admin/groups/{id}` |
|
||||
| `adminAddGroupMember(gId, uId)` | `POST /admin/groups/{gId}/members/{uId}` |
|
||||
| `adminRemoveGroupMember(gId, uId)` | `DELETE /admin/groups/{gId}/members/{uId}` |
|
||||
| `updateDocumentLimits(data)` | `PATCH /settings/documents/limits` |
|
||||
| `viewDocument(id)` | Blob URL → new tab, 60s revoke |
|
||||
| `getDocumentShares(docId)` | `GET /documents/{id}/shares` |
|
||||
| `addDocumentShare(docId, groupId)` | `POST /documents/{id}/shares` |
|
||||
| `removeDocumentShare(docId, groupId)` | `DELETE /documents/{id}/shares/{group_id}` |
|
||||
| `getMyGroups()` | `GET /users/me/groups` (for share picker) |
|
||||
| `confirmFolderSuggestion(docId)` | Apply AI folder suggestion |
|
||||
| `rejectFolderSuggestion(docId)` | Dismiss AI folder suggestion |
|
||||
| `confirmFilenameSuggestion(docId)` | Apply AI filename suggestion |
|
||||
| `rejectFilenameSuggestion(docId)` | Dismiss AI filename suggestion |
|
||||
|
||||
---
|
||||
|
||||
## State management
|
||||
|
||||
- **TanStack Query** — all server state; `queryKey: ["documents", params]` for cache isolation per filter/page combination
|
||||
- **No global store** — local `useState` for UI-only state (editing mode, filter params, etc.)
|
||||
- **Token** — `localStorage`, read by `useAuth` hook, injected by Axios interceptor
|
||||
- **TanStack Query** — all server state
|
||||
- `["documents", params]` — owned doc list (refetchInterval when pending/processing)
|
||||
- `["documents-shared", params]` — shared-with-me list
|
||||
- `["categories"]` — all user categories (shared across SourcePanel + DocumentSlideOver)
|
||||
- `["document-shares", docId]` — shares for a specific document
|
||||
- `["my-groups"]` — current user's group memberships
|
||||
- **URL search params** — `view`, `page`, `sort`, `order`, `search`, `status`, `document_type`, `category_id`
|
||||
- **Local `useState`** — UI-only state (drag, upload queue, active doc ID, selected IDs, slide-over open)
|
||||
|
||||
---
|
||||
|
||||
@@ -167,34 +175,42 @@ Key functions:
|
||||
|
||||
| Component | Path | Description |
|
||||
|-----------|------|-------------|
|
||||
| `AppShell` | `src/components/AppShell.tsx` | Layout wrapper: Sidebar + scrollable main content |
|
||||
| `Sidebar` | `src/components/Sidebar.tsx` | Collapsible left nav (icons-only ↔ icons+labels) |
|
||||
| `ThemeToggle` | `src/components/ThemeToggle.tsx` | Sun/moon ghost icon button; persists to localStorage |
|
||||
| `Button` | `src/components/ui/button.tsx` | shadcn/ui Button (default, ghost, outline, destructive) |
|
||||
| `AppShell` | `src/components/AppShell.tsx` | Layout: Sidebar + SourcePanel (on /apps/documents) + main |
|
||||
| `Sidebar` | `src/components/Sidebar.tsx` | Collapsible left nav (categories removed, replaced by SourcePanel) |
|
||||
| `SourcePanel` | `src/components/SourcePanel.tsx` | Views + searchable category tree (docs route only) |
|
||||
| `ManageCategoriesDialog` | `src/components/ManageCategoriesDialog.tsx` | Category CRUD modal |
|
||||
| `DocumentSlideOver` | `src/components/DocumentSlideOver.tsx` | Right slide-over: detail, edit, share, AI suggestions |
|
||||
| `ThemeToggle` | `src/components/ThemeToggle.tsx` | Sun/moon toggle |
|
||||
| `PluginSchemaForm` | `src/components/PluginSchemaForm.tsx` | JSON Schema → React form |
|
||||
| `Button` | `src/components/ui/button.tsx` | shadcn/ui Button |
|
||||
| `Input` | `src/components/ui/input.tsx` | shadcn/ui Input |
|
||||
|
||||
---
|
||||
|
||||
## Known limitations / not implemented
|
||||
|
||||
- **JWT in `localStorage`** — XSS risk; migrate to `httpOnly` cookie when backend supports it
|
||||
- **No toast / notification system** — errors shown inline; success is silent
|
||||
- **No loading skeletons** — "Loading…" text only
|
||||
- **No app permission UI** per group — groups exist but permission grants are not yet implemented
|
||||
- **No app permission UI** — all apps visible to all authenticated users
|
||||
- **No loading skeletons** — spinner only
|
||||
- **Raw text not in DocumentOut** — slide-over shows a placeholder; full text requires direct backend API call
|
||||
|
||||
---
|
||||
|
||||
## Future work
|
||||
|
||||
- [x] UI component library: shadcn/ui + Tailwind CSS — installed and wired up
|
||||
- [x] AppShell + Sidebar replacing inline Nav component
|
||||
- [x] Light/dark theme context with OS preference detection
|
||||
- [ ] Toast notification system (upload success, save feedback, errors)
|
||||
- [x] SourcePanel with views + searchable category navigation
|
||||
- [x] DocumentSlideOver replacing expand-in-row
|
||||
- [x] Filter chip system
|
||||
- [x] Multi-file upload with queue panel + drag-and-drop
|
||||
- [x] Bulk actions bar (share, delete)
|
||||
- [x] Document sharing UI (Sharing section + Shared with me view)
|
||||
- [x] AI suggestion confirm/reject UI (folder + filename)
|
||||
- [x] Groups admin UI
|
||||
- [x] Replace Axios with native fetch; add global 401 → `/login` redirect for expired sessions
|
||||
- [x] Admin storage page with live migration progress bar
|
||||
- [ ] Toast notification system
|
||||
- [ ] Loading skeletons
|
||||
- [ ] `POST /queue/jobs` integration — show AI processing queue status / progress per document
|
||||
- [ ] Re-process document button (`POST /documents/{id}/reprocess` — needs backend endpoint first)
|
||||
- [ ] Advanced filter: extracted data fields (vendor, due date, amount) — needs backend support
|
||||
- [x] Groups admin UI — list, create, edit, delete, add/remove members
|
||||
- [ ] App permissions UI per group (blocked on backend group_app_permissions)
|
||||
- [ ] Document sharing UI (blocked on backend)
|
||||
- [ ] Cmd+K global search (`CommandDialog`)
|
||||
- [ ] Advanced filter: extracted data fields (needs backend support)
|
||||
- [ ] `httpOnly` cookie auth (requires backend change)
|
||||
- [ ] Bulk document operations (select multiple, bulk delete / bulk categorise)
|
||||
- [ ] TanStack Virtual for category list > 200 items
|
||||
|
||||
@@ -13,8 +13,7 @@
|
||||
"dependencies": {
|
||||
"@radix-ui/react-slot": "^1.0.2",
|
||||
"@tanstack/react-query": "^5.40.0",
|
||||
"axios": "^1.7.2",
|
||||
"class-variance-authority": "^0.7.0",
|
||||
"class-variance-authority": "^0.7.0",
|
||||
"clsx": "^2.1.0",
|
||||
"lucide-react": "^0.400.0",
|
||||
"react": "^18.3.1",
|
||||
|
||||
+55
-8
@@ -1,7 +1,7 @@
|
||||
import { Routes, Route, Navigate } from "react-router-dom";
|
||||
import { useQuery } from "@tanstack/react-query";
|
||||
import { useAuth } from "./hooks/useAuth";
|
||||
import { getMe } from "./api/client";
|
||||
import { getMe, getPlugins } from "./api/client";
|
||||
import AppShell from "./components/AppShell";
|
||||
import LoginPage from "./pages/LoginPage";
|
||||
import DashboardPage from "./pages/DashboardPage";
|
||||
@@ -10,10 +10,13 @@ import AppsPage from "./pages/AppsPage";
|
||||
import AdminPage from "./pages/AdminPage";
|
||||
import AdminUsersPage from "./pages/AdminUsersPage";
|
||||
import AdminGroupsPage from "./pages/AdminGroupsPage";
|
||||
import AdminAppearancePage from "./pages/AdminAppearancePage";
|
||||
import DocumentsPage from "./pages/DocumentsPage";
|
||||
import DocumentAdminSettingsPage from "./pages/DocumentAdminSettingsPage";
|
||||
import DocServiceSettingsPage from "./pages/DocServiceSettingsPage";
|
||||
import AIAdminSettingsPage from "./pages/AIAdminSettingsPage";
|
||||
import SettingsPage from "./pages/SettingsPage";
|
||||
import PluginSettingsPage from "./pages/PluginSettingsPage";
|
||||
import StorageAdminPage from "./pages/StorageAdminPage";
|
||||
|
||||
function PrivateRoute({ children }: { children: React.ReactNode }) {
|
||||
const { token } = useAuth();
|
||||
@@ -29,13 +32,46 @@ function AdminRoute({ children }: { children: React.ReactNode }) {
|
||||
const { data: user, isLoading } = useQuery({ queryKey: ["me"], queryFn: getMe });
|
||||
|
||||
if (!token) return <Navigate to="/login" replace />;
|
||||
// Wait for the me query before deciding — prevents a flash redirect
|
||||
if (isLoading) return null;
|
||||
// Redirect to /login (not /) so the route appears not to exist
|
||||
if (!user?.is_admin) return <Navigate to="/login" replace />;
|
||||
return <AppShell>{children}</AppShell>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Route guard for service-specific settings pages.
|
||||
*
|
||||
* Grants access if the user is a global admin OR the plugin (service) list
|
||||
* returned by the backend includes the given serviceId — which means the user
|
||||
* is a member of that service's admin group.
|
||||
*/
|
||||
function ServiceAdminRoute({
|
||||
children,
|
||||
serviceId,
|
||||
}: {
|
||||
children: React.ReactNode;
|
||||
serviceId: string;
|
||||
}) {
|
||||
const { token } = useAuth();
|
||||
const { data: user, isLoading: userLoading } = useQuery({
|
||||
queryKey: ["me"],
|
||||
queryFn: getMe,
|
||||
});
|
||||
const { data: plugins = [], isLoading: pluginsLoading } = useQuery({
|
||||
queryKey: ["plugins"],
|
||||
queryFn: getPlugins,
|
||||
retry: false,
|
||||
});
|
||||
|
||||
if (!token) return <Navigate to="/login" replace />;
|
||||
if (userLoading || pluginsLoading) return null;
|
||||
|
||||
const hasAccess =
|
||||
user?.is_admin || plugins.some((p) => p.id === serviceId);
|
||||
|
||||
if (!hasAccess) return <Navigate to="/login" replace />;
|
||||
return <AppShell>{children}</AppShell>;
|
||||
}
|
||||
|
||||
export default function App() {
|
||||
return (
|
||||
<Routes>
|
||||
@@ -45,18 +81,29 @@ export default function App() {
|
||||
<Route path="/apps" element={<PrivateRoute><AppsPage /></PrivateRoute>} />
|
||||
<Route path="/apps/documents" element={<PrivateRoute><DocumentsPage /></PrivateRoute>} />
|
||||
<Route
|
||||
path="/apps/documents/settings/admin"
|
||||
element={<AdminRoute><DocumentAdminSettingsPage /></AdminRoute>}
|
||||
path="/apps/documents/settings"
|
||||
element={
|
||||
<ServiceAdminRoute serviceId="doc-service">
|
||||
<DocServiceSettingsPage />
|
||||
</ServiceAdminRoute>
|
||||
}
|
||||
/>
|
||||
<Route
|
||||
path="/apps/ai/settings/admin"
|
||||
element={<AdminRoute><AIAdminSettingsPage /></AdminRoute>}
|
||||
path="/apps/ai/settings"
|
||||
element={
|
||||
<ServiceAdminRoute serviceId="ai-service">
|
||||
<AIAdminSettingsPage />
|
||||
</ServiceAdminRoute>
|
||||
}
|
||||
/>
|
||||
<Route path="/profile" element={<PrivateRoute><ProfilePage /></PrivateRoute>} />
|
||||
<Route path="/settings" element={<PrivateRoute><SettingsPage /></PrivateRoute>} />
|
||||
<Route path="/settings/plugins/:id" element={<PrivateRoute><PluginSettingsPage /></PrivateRoute>} />
|
||||
<Route path="/admin" element={<AdminRoute><AdminPage /></AdminRoute>} />
|
||||
<Route path="/admin/users" element={<AdminRoute><AdminUsersPage /></AdminRoute>} />
|
||||
<Route path="/admin/groups" element={<AdminRoute><AdminGroupsPage /></AdminRoute>} />
|
||||
<Route path="/admin/appearance" element={<AdminRoute><AdminAppearancePage /></AdminRoute>} />
|
||||
<Route path="/admin/storage" element={<AdminRoute><StorageAdminPage /></AdminRoute>} />
|
||||
|
||||
{/* Catch-all */}
|
||||
<Route path="*" element={<Navigate to="/" replace />} />
|
||||
|
||||
+392
-74
@@ -1,46 +1,151 @@
|
||||
import axios from "axios";
|
||||
const BASE = "/api";
|
||||
|
||||
const api = axios.create({ baseURL: "/api" });
|
||||
// ---------------------------------------------------------------------------
|
||||
// Core fetch wrapper
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
api.interceptors.request.use((config) => {
|
||||
export class ApiError extends Error {
|
||||
status: number;
|
||||
detail: string;
|
||||
|
||||
constructor(status: number, detail: string) {
|
||||
super(detail);
|
||||
this.status = status;
|
||||
this.detail = detail;
|
||||
}
|
||||
}
|
||||
|
||||
async function request<T>(
|
||||
method: string,
|
||||
path: string,
|
||||
options: {
|
||||
json?: unknown;
|
||||
form?: URLSearchParams;
|
||||
body?: FormData;
|
||||
params?: Record<string, string | number | undefined>;
|
||||
blob?: boolean;
|
||||
} = {}
|
||||
): Promise<T> {
|
||||
const { json, form, body, params, blob: asBlob } = options;
|
||||
|
||||
// Build URL with optional query params
|
||||
let url = `${BASE}${path}`;
|
||||
if (params) {
|
||||
const qs = new URLSearchParams();
|
||||
for (const [k, v] of Object.entries(params)) {
|
||||
if (v !== undefined && v !== "") qs.set(k, String(v));
|
||||
}
|
||||
const str = qs.toString();
|
||||
if (str) url += `?${str}`;
|
||||
}
|
||||
|
||||
// Build headers
|
||||
const headers: Record<string, string> = {};
|
||||
const token = localStorage.getItem("token");
|
||||
if (token) config.headers.Authorization = `Bearer ${token}`;
|
||||
return config;
|
||||
});
|
||||
if (token) headers["Authorization"] = `Bearer ${token}`;
|
||||
if (json !== undefined) headers["Content-Type"] = "application/json";
|
||||
if (form !== undefined) headers["Content-Type"] = "application/x-www-form-urlencoded";
|
||||
// FormData: intentionally omit Content-Type so browser sets multipart boundary
|
||||
|
||||
// Build body
|
||||
let requestBody: BodyInit | undefined;
|
||||
if (json !== undefined) requestBody = JSON.stringify(json);
|
||||
else if (form !== undefined) requestBody = form;
|
||||
else if (body !== undefined) requestBody = body;
|
||||
|
||||
const response = await fetch(url, { method, headers, body: requestBody });
|
||||
|
||||
// Global 401 handler — expired or invalid token
|
||||
if (response.status === 401) {
|
||||
localStorage.removeItem("token");
|
||||
if (window.location.pathname !== "/login") {
|
||||
window.location.href = "/login";
|
||||
}
|
||||
throw new ApiError(401, "Session expired");
|
||||
}
|
||||
|
||||
// Parse error responses
|
||||
if (!response.ok) {
|
||||
let detail = `HTTP ${response.status}`;
|
||||
try {
|
||||
const err = await response.json();
|
||||
if (typeof err.detail === "string") detail = err.detail;
|
||||
else if (Array.isArray(err.detail)) detail = err.detail.map((e: { msg: string }) => e.msg).join(", ");
|
||||
} catch {
|
||||
// non-JSON error body — keep default message
|
||||
}
|
||||
throw new ApiError(response.status, detail);
|
||||
}
|
||||
|
||||
// 204 No Content
|
||||
if (response.status === 204) return undefined as T;
|
||||
|
||||
if (asBlob) return response.blob() as Promise<T>;
|
||||
|
||||
const text = await response.text();
|
||||
return text ? (JSON.parse(text) as T) : (undefined as T);
|
||||
}
|
||||
|
||||
const api = {
|
||||
get: <T>(path: string, params?: Record<string, string | number | undefined>) =>
|
||||
request<T>("GET", path, { params }),
|
||||
post: <T>(path: string, json?: unknown) =>
|
||||
request<T>("POST", path, { json }),
|
||||
postForm: <T>(path: string, form: URLSearchParams) =>
|
||||
request<T>("POST", path, { form }),
|
||||
postFile: <T>(path: string, body: FormData) =>
|
||||
request<T>("POST", path, { body }),
|
||||
patch: <T>(path: string, json?: unknown) =>
|
||||
request<T>("PATCH", path, { json }),
|
||||
delete: <T>(path: string) =>
|
||||
request<T>("DELETE", path),
|
||||
getBlob: (path: string) =>
|
||||
request<Blob>("GET", path, { blob: true }),
|
||||
};
|
||||
|
||||
export default api;
|
||||
|
||||
// --- Auth ---
|
||||
// ---------------------------------------------------------------------------
|
||||
// Auth
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const login = (email: string, password: string) =>
|
||||
api
|
||||
.post<{ access_token: string }>("/auth/login", new URLSearchParams({ username: email, password }))
|
||||
.then((r) => r.data.access_token);
|
||||
api.postForm<{ access_token: string }>(
|
||||
"/auth/login",
|
||||
new URLSearchParams({ username: email, password })
|
||||
).then((r) => r.access_token);
|
||||
|
||||
export const register = (email: string, password: string, full_name?: string) =>
|
||||
api.post("/auth/register", { email, password, full_name }).then((r) => r.data);
|
||||
api.post("/auth/register", { email, password, full_name });
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Users
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Users ---
|
||||
export interface UserData {
|
||||
id: string;
|
||||
email: string;
|
||||
full_name: string | null;
|
||||
is_active: boolean;
|
||||
is_admin: boolean;
|
||||
color_mode: string | null;
|
||||
}
|
||||
|
||||
export const getMe = () => api.get<UserData>("/users/me").then((r) => r.data);
|
||||
export const getMe = () => api.get<UserData>("/users/me");
|
||||
|
||||
export interface DashboardPrefs {
|
||||
app_ids: string[];
|
||||
}
|
||||
|
||||
export const getDashboardPrefs = () =>
|
||||
api.get<DashboardPrefs>("/users/me/preferences").then((r) => r.data);
|
||||
export const getDashboardPrefs = () => api.get<DashboardPrefs>("/users/me/preferences");
|
||||
|
||||
export const updateDashboardPrefs = (app_ids: string[]) =>
|
||||
api.patch<DashboardPrefs>("/users/me/preferences", { app_ids }).then((r) => r.data);
|
||||
api.patch<DashboardPrefs>("/users/me/preferences", { app_ids });
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Admin — Users
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Admin ---
|
||||
export interface AdminUserCreate {
|
||||
email: string;
|
||||
password: string;
|
||||
@@ -48,19 +153,21 @@ export interface AdminUserCreate {
|
||||
is_admin?: boolean;
|
||||
}
|
||||
|
||||
export const adminGetUsers = () =>
|
||||
api.get<UserData[]>("/admin/users").then((r) => r.data);
|
||||
export const adminGetUsers = () => api.get<UserData[]>("/admin/users");
|
||||
|
||||
export const adminCreateUser = (data: AdminUserCreate) =>
|
||||
api.post<UserData>("/admin/users", data).then((r) => r.data);
|
||||
api.post<UserData>("/admin/users", data);
|
||||
|
||||
export const adminDeleteUser = (userId: string) =>
|
||||
api.delete(`/admin/users/${userId}`);
|
||||
|
||||
export const adminToggleActive = (userId: string) =>
|
||||
api.patch<UserData>(`/admin/users/${userId}/active`).then((r) => r.data);
|
||||
api.patch<UserData>(`/admin/users/${userId}/active`);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Profile
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Profile ---
|
||||
export interface ProfileData {
|
||||
id: string;
|
||||
user_id: string;
|
||||
@@ -78,18 +185,22 @@ export interface ProfileUpdate {
|
||||
address?: string | null;
|
||||
}
|
||||
|
||||
export const getProfile = () =>
|
||||
api.get<ProfileData>("/profile/me").then((r) => r.data);
|
||||
export const getProfile = () => api.get<ProfileData>("/profile/me");
|
||||
|
||||
export const updateProfile = (data: ProfileUpdate) =>
|
||||
api.put<ProfileData>("/profile/me", data).then((r) => r.data);
|
||||
api.patch<ProfileData>("/profile/me", data);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Documents
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Documents ---
|
||||
export type DocumentStatus = "pending" | "processing" | "done" | "failed";
|
||||
|
||||
export interface CategoryOut {
|
||||
id: string;
|
||||
name: string;
|
||||
scope: "personal" | "group" | "system";
|
||||
group_id: string | null;
|
||||
}
|
||||
|
||||
export interface DocumentOut {
|
||||
@@ -106,6 +217,26 @@ export interface DocumentOut {
|
||||
created_at: string;
|
||||
processed_at: string | null;
|
||||
categories: CategoryOut[];
|
||||
source: string;
|
||||
watch_path: string | null;
|
||||
suggested_folder: string | null;
|
||||
suggested_filename: string | null;
|
||||
share_count: number;
|
||||
viewer_can_delete: boolean;
|
||||
}
|
||||
|
||||
export interface SharedDocumentOut extends DocumentOut {
|
||||
shared_by_user_id: string;
|
||||
shared_via_group_id: string;
|
||||
}
|
||||
|
||||
export interface DocumentShareOut {
|
||||
id: string;
|
||||
document_id: string;
|
||||
group_id: string;
|
||||
shared_by_user_id: string;
|
||||
can_delete: boolean;
|
||||
created_at: string;
|
||||
}
|
||||
|
||||
export interface DocumentPage {
|
||||
@@ -135,29 +266,40 @@ export interface DocumentStatusOut {
|
||||
}
|
||||
|
||||
export const listDocuments = (params: DocumentListParams = {}) =>
|
||||
api.get<DocumentPage>("/documents", { params }).then((r) => r.data);
|
||||
api.get<DocumentPage>("/documents", params as Record<string, string | number | undefined>);
|
||||
|
||||
export const listSharedWithMe = (params: DocumentListParams = {}) =>
|
||||
api.get<DocumentPage>("/documents/shared-with-me", params as Record<string, string | number | undefined>);
|
||||
|
||||
export const getDocumentShares = (docId: string) =>
|
||||
api.get<DocumentShareOut[]>(`/documents/${docId}/shares`);
|
||||
|
||||
export const addDocumentShare = (docId: string, groupId: string, canDelete = false) =>
|
||||
api.post<DocumentShareOut>(`/documents/${docId}/shares`, { group_id: groupId, can_delete: canDelete });
|
||||
|
||||
export const removeDocumentShare = (docId: string, groupId: string) =>
|
||||
api.delete(`/documents/${docId}/shares/${groupId}`);
|
||||
|
||||
export const getDocument = (id: string) =>
|
||||
api.get<DocumentOut>(`/documents/${id}`).then((r) => r.data);
|
||||
api.get<DocumentOut>(`/documents/${id}`);
|
||||
|
||||
export const getDocumentStatus = (id: string) =>
|
||||
api.get<DocumentStatusOut>(`/documents/${id}/status`).then((r) => r.data);
|
||||
api.get<DocumentStatusOut>(`/documents/${id}/status`);
|
||||
|
||||
export const uploadDocument = (file: File) => {
|
||||
const form = new FormData();
|
||||
form.append("file", file);
|
||||
return api.post<DocumentOut>("/documents/upload", form).then((r) => r.data);
|
||||
return api.postFile<DocumentOut>("/documents/upload", form);
|
||||
};
|
||||
|
||||
export const updateDocumentType = (id: string, document_type: string) =>
|
||||
api.patch<DocumentOut>(`/documents/${id}/type`, { document_type }).then((r) => r.data);
|
||||
api.patch<DocumentOut>(`/documents/${id}/type`, { document_type });
|
||||
|
||||
export const deleteDocument = (id: string) =>
|
||||
api.delete(`/documents/${id}`);
|
||||
export const deleteDocument = (id: string) => api.delete(`/documents/${id}`);
|
||||
|
||||
export const downloadDocument = async (id: string, filename: string) => {
|
||||
const response = await api.get(`/documents/${id}/file`, { responseType: "blob" });
|
||||
const url = URL.createObjectURL(response.data);
|
||||
const blob = await api.getBlob(`/documents/${id}/file`);
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement("a");
|
||||
a.href = url;
|
||||
a.download = filename;
|
||||
@@ -166,22 +308,21 @@ export const downloadDocument = async (id: string, filename: string) => {
|
||||
};
|
||||
|
||||
export const viewDocument = async (id: string): Promise<void> => {
|
||||
const response = await api.get(`/documents/${id}/file`, { responseType: "blob" });
|
||||
const url = URL.createObjectURL(response.data);
|
||||
const blob = await api.getBlob(`/documents/${id}/file`);
|
||||
const url = URL.createObjectURL(blob);
|
||||
const win = window.open(url, "_blank");
|
||||
// Revoke after a generous delay — the new tab needs time to load the blob
|
||||
setTimeout(() => URL.revokeObjectURL(url), 60_000);
|
||||
if (!win) alert("Pop-up blocked. Please allow pop-ups for this site to preview PDFs.");
|
||||
};
|
||||
|
||||
export const updateDocumentTags = (id: string, tags: string[]) =>
|
||||
api.patch<DocumentOut>(`/documents/${id}/tags`, { tags }).then((r) => r.data);
|
||||
api.patch<DocumentOut>(`/documents/${id}/tags`, { tags });
|
||||
|
||||
export const updateDocumentTitle = (id: string, title: string) =>
|
||||
api.patch<DocumentOut>(`/documents/${id}/title`, { title }).then((r) => r.data);
|
||||
api.patch<DocumentOut>(`/documents/${id}/title`, { title });
|
||||
|
||||
export const reprocessDocument = (id: string) =>
|
||||
api.post<DocumentOut>(`/documents/${id}/reprocess`).then((r) => r.data);
|
||||
api.post<DocumentOut>(`/documents/${id}/reprocess`);
|
||||
|
||||
export const assignCategory = (docId: string, catId: string) =>
|
||||
api.post(`/documents/${docId}/categories/${catId}`);
|
||||
@@ -189,20 +330,74 @@ export const assignCategory = (docId: string, catId: string) =>
|
||||
export const removeCategory = (docId: string, catId: string) =>
|
||||
api.delete(`/documents/${docId}/categories/${catId}`);
|
||||
|
||||
// --- Categories ---
|
||||
export const listCategories = () =>
|
||||
api.get<CategoryOut[]>("/documents/categories").then((r) => r.data);
|
||||
// ---------------------------------------------------------------------------
|
||||
// Categories
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const createCategory = (name: string) =>
|
||||
api.post<CategoryOut>("/documents/categories", { name }).then((r) => r.data);
|
||||
export const listCategories = () => api.get<CategoryOut[]>("/documents/categories");
|
||||
|
||||
export const createCategory = (name: string, groupId?: string) =>
|
||||
api.post<CategoryOut>("/documents/categories", { name, group_id: groupId ?? null });
|
||||
|
||||
export const renameCategory = (id: string, name: string) =>
|
||||
api.patch<CategoryOut>(`/documents/categories/${id}`, { name }).then((r) => r.data);
|
||||
api.patch<CategoryOut>(`/documents/categories/${id}`, { name });
|
||||
|
||||
export const deleteCategory = (id: string) =>
|
||||
api.delete(`/documents/categories/${id}`);
|
||||
export const deleteCategory = (id: string) => api.delete(`/documents/categories/${id}`);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Appearance & Themes
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ThemeColors {
|
||||
primary: string;
|
||||
primary_hover: string;
|
||||
accent: string;
|
||||
accent_hover: string;
|
||||
background: string;
|
||||
surface: string;
|
||||
border: string;
|
||||
text_primary: string;
|
||||
text_muted: string;
|
||||
}
|
||||
|
||||
export interface ThemeDefinition {
|
||||
id: string;
|
||||
label: string;
|
||||
builtin: boolean;
|
||||
light: ThemeColors;
|
||||
dark: ThemeColors;
|
||||
}
|
||||
|
||||
export interface AppearanceSettings {
|
||||
theme: string;
|
||||
default_mode: string;
|
||||
}
|
||||
|
||||
export const getAppearanceSettings = () =>
|
||||
api.get<AppearanceSettings>("/settings/appearance");
|
||||
|
||||
export const updateAppearanceSettings = (data: AppearanceSettings) =>
|
||||
api.patch<AppearanceSettings>("/settings/appearance", data);
|
||||
|
||||
export const getThemes = () => api.get<ThemeDefinition[]>("/settings/themes");
|
||||
|
||||
export const createTheme = (data: Omit<ThemeDefinition, "builtin">) =>
|
||||
api.post<ThemeDefinition>("/settings/themes", data);
|
||||
|
||||
export const updateTheme = (
|
||||
id: string,
|
||||
data: { label?: string; light?: ThemeColors; dark?: ThemeColors }
|
||||
) => api.patch<ThemeDefinition>(`/settings/themes/${id}`, data);
|
||||
|
||||
export const deleteTheme = (id: string) => api.delete(`/settings/themes/${id}`);
|
||||
|
||||
export const updateColorMode = (color_mode: string) =>
|
||||
api.patch<UserData>("/users/me/color-mode", { color_mode });
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Settings (admin only)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Settings (admin only) ---
|
||||
export interface AIProviderUpdate {
|
||||
provider: string;
|
||||
anthropic_api_key?: string;
|
||||
@@ -216,25 +411,39 @@ export interface AIProviderUpdate {
|
||||
}
|
||||
|
||||
export const getAISettings = () =>
|
||||
api.get<Record<string, unknown>>("/settings/ai").then((r) => r.data);
|
||||
api.get<Record<string, unknown>>("/settings/ai");
|
||||
|
||||
export const updateAISettings = (data: AIProviderUpdate) =>
|
||||
api.patch<Record<string, unknown>>("/settings/ai", data).then((r) => r.data);
|
||||
api.patch<Record<string, unknown>>("/settings/ai", data);
|
||||
|
||||
export const testAIConnection = () =>
|
||||
api.post<{ ok: boolean; provider: string; response?: string; error?: string }>(
|
||||
"/settings/ai/test"
|
||||
).then((r) => r.data);
|
||||
|
||||
export const updateDocumentLimits = (max_pdf_mb: number) =>
|
||||
api.patch<Record<string, unknown>>("/settings/documents/limits", { max_pdf_mb }).then(
|
||||
(r) => r.data
|
||||
);
|
||||
|
||||
export const getDocumentLimits = () =>
|
||||
api.get<Record<string, unknown>>("/settings/documents/limits").then((r) => r.data);
|
||||
export const updateDocumentLimits = (max_pdf_mb: number) =>
|
||||
api.patch<Record<string, unknown>>("/settings/documents/limits", { max_pdf_mb });
|
||||
|
||||
export const getDocumentLimits = () =>
|
||||
api.get<Record<string, unknown>>("/settings/documents/limits");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// User groups (current user's own memberships)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface UserGroupOut {
|
||||
id: string;
|
||||
name: string;
|
||||
description: string | null;
|
||||
is_group_admin: boolean;
|
||||
}
|
||||
|
||||
export const getMyGroups = () => api.get<UserGroupOut[]>("/users/me/groups");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Groups (admin only)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- Groups (admin only) ---
|
||||
export interface GroupOut {
|
||||
id: string;
|
||||
name: string;
|
||||
@@ -249,6 +458,7 @@ export interface GroupMemberOut {
|
||||
full_name: string | null;
|
||||
is_active: boolean;
|
||||
joined_at: string;
|
||||
is_group_admin: boolean;
|
||||
}
|
||||
|
||||
export interface GroupDetailOut extends GroupOut {
|
||||
@@ -265,17 +475,16 @@ export interface GroupUpdate {
|
||||
description?: string | null;
|
||||
}
|
||||
|
||||
export const adminListGroups = () =>
|
||||
api.get<GroupOut[]>("/admin/groups").then((r) => r.data);
|
||||
export const adminListGroups = () => api.get<GroupOut[]>("/admin/groups");
|
||||
|
||||
export const adminCreateGroup = (data: GroupCreate) =>
|
||||
api.post<GroupOut>("/admin/groups", data).then((r) => r.data);
|
||||
api.post<GroupOut>("/admin/groups", data);
|
||||
|
||||
export const adminGetGroup = (groupId: string) =>
|
||||
api.get<GroupDetailOut>(`/admin/groups/${groupId}`).then((r) => r.data);
|
||||
api.get<GroupDetailOut>(`/admin/groups/${groupId}`);
|
||||
|
||||
export const adminUpdateGroup = (groupId: string, data: GroupUpdate) =>
|
||||
api.patch<GroupOut>(`/admin/groups/${groupId}`, data).then((r) => r.data);
|
||||
api.patch<GroupOut>(`/admin/groups/${groupId}`, data);
|
||||
|
||||
export const adminDeleteGroup = (groupId: string) =>
|
||||
api.delete(`/admin/groups/${groupId}`);
|
||||
@@ -286,7 +495,13 @@ export const adminAddGroupMember = (groupId: string, userId: string) =>
|
||||
export const adminRemoveGroupMember = (groupId: string, userId: string) =>
|
||||
api.delete(`/admin/groups/${groupId}/members/${userId}`);
|
||||
|
||||
// --- Services ---
|
||||
export const adminSetGroupMemberAdmin = (groupId: string, userId: string, isAdmin: boolean) =>
|
||||
api.patch(`/admin/groups/${groupId}/members/${userId}/admin`, { is_group_admin: isAdmin });
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Services
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface ServiceStatus {
|
||||
id: string;
|
||||
name: string;
|
||||
@@ -296,10 +511,12 @@ export interface ServiceStatus {
|
||||
settings_path: string;
|
||||
}
|
||||
|
||||
export const getServices = () =>
|
||||
api.get<ServiceStatus[]>("/services").then((r) => r.data);
|
||||
export const getServices = () => api.get<ServiceStatus[]>("/services");
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// System Prompts (admin only)
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
// --- System Prompts (admin only) ---
|
||||
export interface ServiceSystemPrompt {
|
||||
label: string;
|
||||
system: string;
|
||||
@@ -311,12 +528,113 @@ export interface ServiceSystemPrompt {
|
||||
export type SystemPromptsData = Record<string, ServiceSystemPrompt>;
|
||||
|
||||
export const getSystemPrompts = () =>
|
||||
api.get<SystemPromptsData>("/settings/system-prompts").then((r) => r.data);
|
||||
api.get<SystemPromptsData>("/settings/system-prompts");
|
||||
|
||||
export const updateSystemPrompt = (
|
||||
serviceId: string,
|
||||
data: { system: string; user_template: string }
|
||||
) =>
|
||||
api
|
||||
.patch<SystemPromptsData>(`/settings/system-prompts/${serviceId}`, data)
|
||||
.then((r) => r.data);
|
||||
) => api.patch<SystemPromptsData>(`/settings/system-prompts/${serviceId}`, data);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Document suggestions
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export const confirmFolderSuggestion = (docId: string) =>
|
||||
api.post(`/documents/${docId}/suggestions/folder/confirm`);
|
||||
|
||||
export const rejectFolderSuggestion = (docId: string) =>
|
||||
api.post(`/documents/${docId}/suggestions/folder/reject`);
|
||||
|
||||
export const confirmFilenameSuggestion = (docId: string) =>
|
||||
api.post(`/documents/${docId}/suggestions/filename/confirm`);
|
||||
|
||||
export const rejectFilenameSuggestion = (docId: string) =>
|
||||
api.post(`/documents/${docId}/suggestions/filename/reject`);
|
||||
|
||||
// ---------------------------------------------------------------------------
|
||||
// Plugins
|
||||
// ---------------------------------------------------------------------------
|
||||
|
||||
export interface PluginOut {
|
||||
id: string;
|
||||
name: string;
|
||||
icon: string;
|
||||
version: string;
|
||||
}
|
||||
|
||||
export interface PluginSchemaProperty {
|
||||
type: string;
|
||||
title: string;
|
||||
description?: string;
|
||||
readOnly?: boolean;
|
||||
}
|
||||
|
||||
export interface PluginManifest {
|
||||
id: string;
|
||||
name: string;
|
||||
icon: string;
|
||||
version: string;
|
||||
access: {
|
||||
allow_superuser: boolean;
|
||||
required_groups: string[];
|
||||
};
|
||||
settings_schema: {
|
||||
type: string;
|
||||
title?: string;
|
||||
properties: Record<string, PluginSchemaProperty>;
|
||||
};
|
||||
}
|
||||
|
||||
// ── Storage admin ──────────────────────────────────────────────────────────────
|
||||
|
||||
export interface StorageStatus {
|
||||
status: string;
|
||||
backend: string;
|
||||
}
|
||||
|
||||
export interface MigrationStatus {
|
||||
state:
|
||||
| "idle"
|
||||
| "validating"
|
||||
| "migrating"
|
||||
| "switching"
|
||||
| "cleaning"
|
||||
| "done"
|
||||
| "failed"
|
||||
| "cancelled";
|
||||
total: number;
|
||||
done: number;
|
||||
failed: number;
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
export interface StorageBackendConfig {
|
||||
driver: string;
|
||||
config: Record<string, string>;
|
||||
}
|
||||
|
||||
export const getStorageConfig = () => api.get<StorageStatus>("/admin/storage-config");
|
||||
|
||||
export const updateStorageConfig = (body: StorageBackendConfig) =>
|
||||
api.patch<void>("/admin/storage-config", body);
|
||||
|
||||
export const startStorageMigration = (body: StorageBackendConfig) =>
|
||||
api.post<{ status: string; driver: string }>("/admin/storage-config/migrate", body);
|
||||
|
||||
export const getMigrationStatus = () =>
|
||||
api.get<MigrationStatus>("/admin/storage-config/migrate/status");
|
||||
|
||||
export const cancelMigration = () => api.delete<void>("/admin/storage-config/migrate");
|
||||
|
||||
// ── Plugins ────────────────────────────────────────────────────────────────────
|
||||
|
||||
export const getPlugins = () => api.get<PluginOut[]>("/plugins");
|
||||
|
||||
export const getPluginManifest = (id: string) =>
|
||||
api.get<PluginManifest>(`/plugins/${id}/manifest`);
|
||||
|
||||
export const getPluginSettings = (id: string) =>
|
||||
api.get<Record<string, unknown>>(`/plugins/${id}/settings`);
|
||||
|
||||
export const updatePluginSettings = (id: string, data: Record<string, unknown>) =>
|
||||
api.patch<Record<string, unknown>>(`/plugins/${id}/settings`, data);
|
||||
|
||||
@@ -1,13 +1,19 @@
|
||||
import { useLocation } from "react-router-dom";
|
||||
import Sidebar from "@/components/Sidebar";
|
||||
import SourcePanel from "@/components/SourcePanel";
|
||||
|
||||
interface AppShellProps {
|
||||
children: React.ReactNode;
|
||||
}
|
||||
|
||||
export default function AppShell({ children }: AppShellProps) {
|
||||
const location = useLocation();
|
||||
const showSourcePanel = location.pathname === "/apps/documents";
|
||||
|
||||
return (
|
||||
<div className="flex h-screen bg-background">
|
||||
<Sidebar />
|
||||
{showSourcePanel && <SourcePanel />}
|
||||
<main className="flex-1 overflow-y-auto p-6">{children}</main>
|
||||
</div>
|
||||
);
|
||||
|
||||
@@ -0,0 +1,792 @@
|
||||
import { useState, useRef, useEffect, useCallback } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import {
|
||||
X, Download, Eye, RefreshCw, Trash2, Check, Pencil, Plus,
|
||||
ChevronDown, ChevronRight, Users, UserMinus,
|
||||
} from "lucide-react";
|
||||
import {
|
||||
DocumentOut, CategoryOut,
|
||||
updateDocumentTitle, updateDocumentTags, updateDocumentType,
|
||||
assignCategory, removeCategory, deleteDocument,
|
||||
downloadDocument, viewDocument, reprocessDocument,
|
||||
confirmFolderSuggestion, rejectFolderSuggestion,
|
||||
confirmFilenameSuggestion, rejectFilenameSuggestion,
|
||||
listCategories,
|
||||
getDocumentShares, addDocumentShare, removeDocumentShare,
|
||||
getMyGroups,
|
||||
} from "@/api/client";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
interface Props {
|
||||
doc: DocumentOut | null;
|
||||
isOwner: boolean; // false for "shared with me" view
|
||||
onClose: () => void;
|
||||
onDeleted: (id: string) => void;
|
||||
}
|
||||
|
||||
const STATUS_COLORS: Record<string, string> = {
|
||||
pending: "bg-orange-400",
|
||||
processing: "bg-blue-400",
|
||||
done: "bg-emerald-400",
|
||||
failed: "bg-red-400",
|
||||
};
|
||||
|
||||
const DOC_TYPES = ["invoice", "bill", "receipt", "order", "expense", "revenue", "unknown"];
|
||||
|
||||
function formatBytes(bytes: number) {
|
||||
if (bytes < 1024) return `${bytes} B`;
|
||||
if (bytes < 1024 * 1024) return `${(bytes / 1024).toFixed(1)} KB`;
|
||||
return `${(bytes / (1024 * 1024)).toFixed(1)} MB`;
|
||||
}
|
||||
|
||||
function formatDate(iso: string | null) {
|
||||
if (!iso) return "—";
|
||||
return new Date(iso).toLocaleDateString(undefined, { year: "numeric", month: "short", day: "numeric" });
|
||||
}
|
||||
|
||||
// ── Category combobox ─────────────────────────────────────────────────────────
|
||||
|
||||
function CategoryCombobox({
|
||||
categories, assigned, onAssign,
|
||||
}: { categories: CategoryOut[]; assigned: CategoryOut[]; onAssign: (id: string) => void }) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const [search, setSearch] = useState("");
|
||||
const ref = useRef<HTMLDivElement>(null);
|
||||
|
||||
const assignedIds = new Set(assigned.map((c) => c.id));
|
||||
const unassigned = categories.filter((c) => !assignedIds.has(c.id));
|
||||
const filtered = search
|
||||
? unassigned.filter((c) => c.name.toLowerCase().includes(search.toLowerCase()))
|
||||
: unassigned;
|
||||
|
||||
useEffect(() => {
|
||||
const handler = (e: MouseEvent) => {
|
||||
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
|
||||
};
|
||||
document.addEventListener("mousedown", handler);
|
||||
return () => document.removeEventListener("mousedown", handler);
|
||||
}, []);
|
||||
|
||||
if (unassigned.length === 0) return null;
|
||||
|
||||
return (
|
||||
<div className="relative" ref={ref}>
|
||||
<button
|
||||
onClick={() => setOpen((o) => !o)}
|
||||
className="flex items-center gap-1 text-xs text-muted hover:text-foreground transition-colors border border-dashed border-border rounded px-2 py-0.5"
|
||||
>
|
||||
<Plus className="h-3 w-3" /> Add category
|
||||
</button>
|
||||
{open && (
|
||||
<div className="absolute left-0 top-full mt-1 z-20 bg-surface border border-border rounded-lg shadow-lg w-52">
|
||||
{categories.length > 5 && (
|
||||
<div className="p-2 border-b border-border">
|
||||
<Input
|
||||
value={search}
|
||||
onChange={(e) => setSearch(e.target.value)}
|
||||
placeholder="Search…"
|
||||
className="h-7 text-xs"
|
||||
autoFocus
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<div className="max-h-48 overflow-y-auto py-1">
|
||||
{filtered.map((cat) => (
|
||||
<button
|
||||
key={cat.id}
|
||||
className="w-full text-left px-3 py-1.5 text-sm hover:bg-muted/20 transition-colors"
|
||||
onClick={() => { onAssign(cat.id); setOpen(false); setSearch(""); }}
|
||||
>
|
||||
{cat.name}
|
||||
</button>
|
||||
))}
|
||||
{filtered.length === 0 && (
|
||||
<p className="text-xs text-muted px-3 py-2">No categories</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ── Group picker combobox ─────────────────────────────────────────────────────
|
||||
|
||||
function GroupCombobox({
|
||||
groups, sharedGroupIds, onShare,
|
||||
}: { groups: { id: string; name: string }[]; sharedGroupIds: Set<string>; onShare: (id: string) => void }) {
|
||||
const [open, setOpen] = useState(false);
|
||||
const [search, setSearch] = useState("");
|
||||
const ref = useRef<HTMLDivElement>(null);
|
||||
|
||||
const available = groups.filter((g) => !sharedGroupIds.has(g.id));
|
||||
const filtered = search
|
||||
? available.filter((g) => g.name.toLowerCase().includes(search.toLowerCase()))
|
||||
: available;
|
||||
|
||||
useEffect(() => {
|
||||
const handler = (e: MouseEvent) => {
|
||||
if (ref.current && !ref.current.contains(e.target as Node)) setOpen(false);
|
||||
};
|
||||
document.addEventListener("mousedown", handler);
|
||||
return () => document.removeEventListener("mousedown", handler);
|
||||
}, []);
|
||||
|
||||
if (available.length === 0) return null;
|
||||
|
||||
return (
|
||||
<div className="relative" ref={ref}>
|
||||
<button
|
||||
onClick={() => setOpen((o) => !o)}
|
||||
className="flex items-center gap-1 text-xs text-muted hover:text-foreground transition-colors border border-dashed border-border rounded px-2 py-0.5 mt-1"
|
||||
>
|
||||
<Users className="h-3 w-3" /> Share with a group
|
||||
</button>
|
||||
{open && (
|
||||
<div className="absolute left-0 top-full mt-1 z-20 bg-surface border border-border rounded-lg shadow-lg w-52">
|
||||
{groups.length > 5 && (
|
||||
<div className="p-2 border-b border-border">
|
||||
<Input
|
||||
value={search}
|
||||
onChange={(e) => setSearch(e.target.value)}
|
||||
placeholder="Search groups…"
|
||||
className="h-7 text-xs"
|
||||
autoFocus
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
<div className="max-h-48 overflow-y-auto py-1">
|
||||
{filtered.map((g) => (
|
||||
<button
|
||||
key={g.id}
|
||||
className="w-full text-left px-3 py-1.5 text-sm hover:bg-muted/20 transition-colors"
|
||||
onClick={() => { onShare(g.id); setOpen(false); setSearch(""); }}
|
||||
>
|
||||
{g.name}
|
||||
</button>
|
||||
))}
|
||||
{filtered.length === 0 && (
|
||||
<p className="text-xs text-muted px-3 py-2">No groups available</p>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// ── Main component ────────────────────────────────────────────────────────────
|
||||
|
||||
export default function DocumentSlideOver({ doc, isOwner, onClose, onDeleted }: Props) {
|
||||
const queryClient = useQueryClient();
|
||||
const [rawOpen, setRawOpen] = useState(false);
|
||||
const [editingTitle, setEditingTitle] = useState(false);
|
||||
const [titleValue, setTitleValue] = useState("");
|
||||
const [editingType, setEditingType] = useState(false);
|
||||
const [tagInput, setTagInput] = useState("");
|
||||
const [canDeleteNew, setCanDeleteNew] = useState(false);
|
||||
const titleInputRef = useRef<HTMLInputElement>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (doc) {
|
||||
setTitleValue(doc.title ?? "");
|
||||
setEditingTitle(false);
|
||||
setEditingType(false);
|
||||
setRawOpen(false);
|
||||
setTagInput("");
|
||||
}
|
||||
}, [doc?.id]);
|
||||
|
||||
useEffect(() => {
|
||||
if (editingTitle) titleInputRef.current?.focus();
|
||||
}, [editingTitle]);
|
||||
|
||||
const { data: allCategories = [] } = useQuery({
|
||||
queryKey: ["categories"],
|
||||
queryFn: listCategories,
|
||||
});
|
||||
|
||||
const { data: myGroups = [] } = useQuery({
|
||||
queryKey: ["my-groups"],
|
||||
queryFn: getMyGroups,
|
||||
enabled: isOwner,
|
||||
});
|
||||
|
||||
const { data: shares = [] } = useQuery({
|
||||
queryKey: ["document-shares", doc?.id],
|
||||
queryFn: () => getDocumentShares(doc!.id),
|
||||
enabled: isOwner && !!doc,
|
||||
});
|
||||
|
||||
const invalidateDoc = useCallback(() => {
|
||||
queryClient.invalidateQueries({ queryKey: ["documents"] });
|
||||
}, [queryClient]);
|
||||
|
||||
const titleMut = useMutation({
|
||||
mutationFn: (title: string) => updateDocumentTitle(doc!.id, title),
|
||||
onSuccess: () => { invalidateDoc(); setEditingTitle(false); },
|
||||
});
|
||||
|
||||
const typeMut = useMutation({
|
||||
mutationFn: (document_type: string) => updateDocumentType(doc!.id, document_type),
|
||||
onSuccess: () => { invalidateDoc(); setEditingType(false); },
|
||||
});
|
||||
|
||||
const addTagMut = useMutation({
|
||||
mutationFn: (tags: string[]) => updateDocumentTags(doc!.id, tags),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const assignCatMut = useMutation({
|
||||
mutationFn: (catId: string) => assignCategory(doc!.id, catId),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const removeCatMut = useMutation({
|
||||
mutationFn: (catId: string) => removeCategory(doc!.id, catId),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const reprocessMut = useMutation({
|
||||
mutationFn: () => reprocessDocument(doc!.id),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const deleteMut = useMutation({
|
||||
mutationFn: () => deleteDocument(doc!.id),
|
||||
onSuccess: () => { invalidateDoc(); onDeleted(doc!.id); },
|
||||
});
|
||||
|
||||
const confirmFolderMut = useMutation({
|
||||
mutationFn: () => confirmFolderSuggestion(doc!.id),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const rejectFolderMut = useMutation({
|
||||
mutationFn: () => rejectFolderSuggestion(doc!.id),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const confirmFilenameMut = useMutation({
|
||||
mutationFn: () => confirmFilenameSuggestion(doc!.id),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const rejectFilenameMut = useMutation({
|
||||
mutationFn: () => rejectFilenameSuggestion(doc!.id),
|
||||
onSuccess: invalidateDoc,
|
||||
});
|
||||
|
||||
const addShareMut = useMutation({
|
||||
mutationFn: ({ groupId, canDelete }: { groupId: string; canDelete: boolean }) =>
|
||||
addDocumentShare(doc!.id, groupId, canDelete),
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["document-shares", doc!.id] });
|
||||
setCanDeleteNew(false);
|
||||
},
|
||||
});
|
||||
|
||||
const removeShareMut = useMutation({
|
||||
mutationFn: (groupId: string) => removeDocumentShare(doc!.id, groupId),
|
||||
onSuccess: () => queryClient.invalidateQueries({ queryKey: ["document-shares", doc!.id] }),
|
||||
});
|
||||
|
||||
if (!doc) return null;
|
||||
|
||||
const extractedData = (() => {
|
||||
try { return doc.extracted_data ? JSON.parse(doc.extracted_data) : null; }
|
||||
catch { return null; }
|
||||
})();
|
||||
|
||||
const tags: string[] = (() => {
|
||||
try { return doc.tags ? JSON.parse(doc.tags) : []; }
|
||||
catch { return []; }
|
||||
})();
|
||||
|
||||
const suggestedCategories: string[] = (() => {
|
||||
try { return extractedData?.suggested_categories ?? []; }
|
||||
catch { return []; }
|
||||
})();
|
||||
|
||||
const displayKeys = extractedData
|
||||
? Object.entries(extractedData).filter(([k]) => !["tags", "suggested_categories"].includes(k))
|
||||
: [];
|
||||
|
||||
const sharedGroupIds = new Set(shares.map((s) => s.group_id));
|
||||
|
||||
function addTag() {
|
||||
const tag = tagInput.trim();
|
||||
if (!tag || tags.includes(tag)) return;
|
||||
addTagMut.mutate([...tags, tag]);
|
||||
setTagInput("");
|
||||
}
|
||||
|
||||
function removeTag(tag: string) {
|
||||
addTagMut.mutate(tags.filter((t) => t !== tag));
|
||||
}
|
||||
|
||||
return (
|
||||
<div
|
||||
className="fixed inset-0 z-40"
|
||||
onClick={(e) => { if (e.target === e.currentTarget) onClose(); }}
|
||||
>
|
||||
{/* Overlay backdrop (subtle) */}
|
||||
<div className="absolute inset-0 bg-background/20" />
|
||||
|
||||
{/* Panel */}
|
||||
<div className="absolute inset-y-0 right-0 w-[480px] bg-surface border-l border-border shadow-2xl flex flex-col overflow-hidden z-50">
|
||||
{/* Header */}
|
||||
<div className="flex items-center gap-3 px-5 py-4 border-b border-border shrink-0">
|
||||
<div className="flex-1 min-w-0">
|
||||
<p className="text-xs text-muted truncate">{doc.filename}</p>
|
||||
</div>
|
||||
<div className="flex items-center gap-1 shrink-0">
|
||||
{isOwner && (
|
||||
<>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => viewDocument(doc.id)}
|
||||
className="h-7 px-2 gap-1 text-xs"
|
||||
>
|
||||
<Eye className="h-3.5 w-3.5" /> View
|
||||
</Button>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
onClick={() => downloadDocument(doc.id, doc.filename)}
|
||||
className="h-7 px-2 gap-1 text-xs"
|
||||
>
|
||||
<Download className="h-3.5 w-3.5" /> Download
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
{!isOwner && (
|
||||
<>
|
||||
<Button variant="ghost" size="sm" onClick={() => viewDocument(doc.id)} className="h-7 px-2 gap-1 text-xs">
|
||||
<Eye className="h-3.5 w-3.5" /> View
|
||||
</Button>
|
||||
<Button variant="ghost" size="sm" onClick={() => downloadDocument(doc.id, doc.filename)} className="h-7 px-2 gap-1 text-xs">
|
||||
<Download className="h-3.5 w-3.5" /> Download
|
||||
</Button>
|
||||
</>
|
||||
)}
|
||||
<button onClick={onClose} className="ml-1 text-muted hover:text-foreground transition-colors">
|
||||
<X className="h-5 w-5" />
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Scrollable body */}
|
||||
<div className="flex-1 overflow-y-auto px-5 py-4 space-y-5">
|
||||
|
||||
{/* Metadata row */}
|
||||
<div className="flex items-center gap-3 flex-wrap text-xs text-muted">
|
||||
<span className="flex items-center gap-1.5">
|
||||
<span className={cn("h-2 w-2 rounded-full", STATUS_COLORS[doc.status] ?? "bg-gray-400")} />
|
||||
{doc.status}
|
||||
</span>
|
||||
<span>{formatBytes(doc.file_size)}</span>
|
||||
<span>Uploaded {formatDate(doc.created_at)}</span>
|
||||
{doc.processed_at && <span>Processed {formatDate(doc.processed_at)}</span>}
|
||||
{doc.source === "watch" && (
|
||||
<span className="text-primary/60">Watch-ingested</span>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Error */}
|
||||
{doc.status === "failed" && doc.error_message && (
|
||||
<div className="text-xs text-red-500 bg-red-500/10 rounded-md px-3 py-2">
|
||||
Error: {doc.error_message}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Title */}
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">Title</p>
|
||||
{isOwner && editingTitle ? (
|
||||
<form
|
||||
onSubmit={(e) => { e.preventDefault(); titleMut.mutate(titleValue); }}
|
||||
className="flex gap-2"
|
||||
>
|
||||
<Input
|
||||
ref={titleInputRef}
|
||||
value={titleValue}
|
||||
onChange={(e) => setTitleValue(e.target.value)}
|
||||
className="h-8 text-sm flex-1"
|
||||
disabled={titleMut.isPending}
|
||||
onKeyDown={(e) => { if (e.key === "Escape") setEditingTitle(false); }}
|
||||
/>
|
||||
<button type="submit" disabled={titleMut.isPending} className="text-primary disabled:opacity-50">
|
||||
<Check className="h-4 w-4" />
|
||||
</button>
|
||||
<button type="button" onClick={() => setEditingTitle(false)} className="text-muted hover:text-foreground">
|
||||
<X className="h-4 w-4" />
|
||||
</button>
|
||||
</form>
|
||||
) : (
|
||||
<div className="flex items-center gap-2 group">
|
||||
<span className={cn("text-sm", !doc.title && "text-muted italic")}>
|
||||
{doc.title ?? "No title"}
|
||||
</span>
|
||||
{isOwner && (
|
||||
<button
|
||||
onClick={() => { setTitleValue(doc.title ?? ""); setEditingTitle(true); }}
|
||||
className="text-muted opacity-0 group-hover:opacity-100 hover:text-foreground transition-all"
|
||||
>
|
||||
<Pencil className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Document type */}
|
||||
{isOwner && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">Type</p>
|
||||
{editingType ? (
|
||||
<div className="flex gap-1 flex-wrap">
|
||||
{DOC_TYPES.map((t) => (
|
||||
<button
|
||||
key={t}
|
||||
onClick={() => typeMut.mutate(t)}
|
||||
disabled={typeMut.isPending}
|
||||
className={cn(
|
||||
"text-xs px-2 py-0.5 rounded border transition-colors",
|
||||
doc.document_type === t
|
||||
? "border-primary text-primary bg-primary/10"
|
||||
: "border-border text-muted hover:border-foreground hover:text-foreground"
|
||||
)}
|
||||
>
|
||||
{t}
|
||||
</button>
|
||||
))}
|
||||
<button onClick={() => setEditingType(false)} className="text-muted hover:text-foreground">
|
||||
<X className="h-4 w-4" />
|
||||
</button>
|
||||
</div>
|
||||
) : (
|
||||
<div className="flex items-center gap-2 group">
|
||||
<span className={cn("text-sm", !doc.document_type && "text-muted italic")}>
|
||||
{doc.document_type ?? "Unknown"}
|
||||
</span>
|
||||
<button
|
||||
onClick={() => setEditingType(true)}
|
||||
className="text-muted opacity-0 group-hover:opacity-100 hover:text-foreground transition-all"
|
||||
>
|
||||
<Pencil className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* AI Suggestions */}
|
||||
{isOwner && (doc.suggested_folder || doc.suggested_filename) && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">
|
||||
AI Suggestions
|
||||
</p>
|
||||
<div className="space-y-2">
|
||||
{doc.suggested_folder && (
|
||||
<div className="flex items-center gap-2 text-sm bg-amber-500/10 border border-amber-500/20 rounded-md px-3 py-2">
|
||||
<span className="flex-1">
|
||||
<span className="text-xs text-muted mr-1">Folder:</span>
|
||||
{doc.suggested_folder}
|
||||
</span>
|
||||
<button
|
||||
onClick={() => confirmFolderMut.mutate()}
|
||||
disabled={confirmFolderMut.isPending || rejectFolderMut.isPending}
|
||||
className="text-xs text-emerald-600 hover:text-emerald-700 font-medium disabled:opacity-50"
|
||||
>
|
||||
✓ Apply
|
||||
</button>
|
||||
<button
|
||||
onClick={() => rejectFolderMut.mutate()}
|
||||
disabled={confirmFolderMut.isPending || rejectFolderMut.isPending}
|
||||
className="text-muted hover:text-red-500 transition-colors disabled:opacity-50"
|
||||
>
|
||||
<X className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
{doc.suggested_filename && (
|
||||
<div className="flex items-center gap-2 text-sm bg-amber-500/10 border border-amber-500/20 rounded-md px-3 py-2">
|
||||
<span className="flex-1">
|
||||
<span className="text-xs text-muted mr-1">Title:</span>
|
||||
{doc.suggested_filename}
|
||||
</span>
|
||||
<button
|
||||
onClick={() => confirmFilenameMut.mutate()}
|
||||
disabled={confirmFilenameMut.isPending || rejectFilenameMut.isPending}
|
||||
className="text-xs text-emerald-600 hover:text-emerald-700 font-medium disabled:opacity-50"
|
||||
>
|
||||
✓ Apply
|
||||
</button>
|
||||
<button
|
||||
onClick={() => rejectFilenameMut.mutate()}
|
||||
disabled={confirmFilenameMut.isPending || rejectFilenameMut.isPending}
|
||||
className="text-muted hover:text-red-500 transition-colors disabled:opacity-50"
|
||||
>
|
||||
<X className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Extracted data */}
|
||||
{displayKeys.length > 0 && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">
|
||||
Extracted Data
|
||||
</p>
|
||||
<table className="w-full text-xs">
|
||||
<tbody>
|
||||
{displayKeys.map(([k, v]) => (
|
||||
<tr key={k} className="border-b border-border/50 last:border-0">
|
||||
<td className="py-1.5 pr-3 text-muted font-medium w-1/3 align-top">
|
||||
{k.replace(/_/g, " ")}
|
||||
</td>
|
||||
<td className="py-1.5 text-foreground align-top break-words">
|
||||
{v === null || v === undefined ? (
|
||||
<span className="text-muted">—</span>
|
||||
) : Array.isArray(v) ? (
|
||||
<span className="font-mono">{JSON.stringify(v)}</span>
|
||||
) : (
|
||||
String(v)
|
||||
)}
|
||||
</td>
|
||||
</tr>
|
||||
))}
|
||||
</tbody>
|
||||
</table>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Categories */}
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">Categories</p>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{doc.categories.map((cat) => (
|
||||
<span
|
||||
key={cat.id}
|
||||
className="flex items-center gap-1 text-xs bg-primary/10 text-primary rounded-full px-2.5 py-0.5"
|
||||
>
|
||||
{cat.name}
|
||||
{isOwner && (
|
||||
<button
|
||||
onClick={() => removeCatMut.mutate(cat.id)}
|
||||
disabled={removeCatMut.isPending}
|
||||
className="hover:text-red-500 transition-colors disabled:opacity-50"
|
||||
>
|
||||
<X className="h-3 w-3" />
|
||||
</button>
|
||||
)}
|
||||
</span>
|
||||
))}
|
||||
{isOwner && (
|
||||
<CategoryCombobox
|
||||
categories={allCategories}
|
||||
assigned={doc.categories}
|
||||
onAssign={(id) => assignCatMut.mutate(id)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* AI-suggested categories */}
|
||||
{isOwner && suggestedCategories.length > 0 && (
|
||||
<div className="mt-2">
|
||||
<p className="text-xs text-muted mb-1">Suggested by AI:</p>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{suggestedCategories
|
||||
.filter((name) => !doc.categories.some((c) => c.name.toLowerCase() === name.toLowerCase()))
|
||||
.map((name) => {
|
||||
const exists = allCategories.find(
|
||||
(c) => c.name.toLowerCase() === name.toLowerCase()
|
||||
);
|
||||
return (
|
||||
<span
|
||||
key={name}
|
||||
className="flex items-center gap-1 text-xs bg-amber-500/10 text-amber-700 dark:text-amber-400 rounded-full px-2.5 py-0.5 border border-amber-500/20"
|
||||
>
|
||||
{name}
|
||||
{exists ? (
|
||||
<button
|
||||
onClick={() => assignCatMut.mutate(exists.id)}
|
||||
className="text-emerald-600 hover:text-emerald-700"
|
||||
title="Assign"
|
||||
>
|
||||
<Check className="h-3 w-3" />
|
||||
</button>
|
||||
) : (
|
||||
<button
|
||||
onClick={async () => {
|
||||
// Create the category then assign it
|
||||
const { createCategory } = await import("@/api/client");
|
||||
const cat = await createCategory(name);
|
||||
queryClient.invalidateQueries({ queryKey: ["categories"] });
|
||||
assignCatMut.mutate(cat.id);
|
||||
}}
|
||||
className="text-emerald-600 hover:text-emerald-700 text-[10px] font-medium"
|
||||
title="Create & assign"
|
||||
>
|
||||
+
|
||||
</button>
|
||||
)}
|
||||
</span>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Tags */}
|
||||
{isOwner && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">Tags</p>
|
||||
<div className="flex flex-wrap gap-1.5 mb-2">
|
||||
{tags.map((tag) => (
|
||||
<span
|
||||
key={tag}
|
||||
className="flex items-center gap-1 text-xs bg-muted/20 rounded-full px-2.5 py-0.5"
|
||||
>
|
||||
{tag}
|
||||
<button onClick={() => removeTag(tag)} className="text-muted hover:text-red-500">
|
||||
<X className="h-3 w-3" />
|
||||
</button>
|
||||
</span>
|
||||
))}
|
||||
</div>
|
||||
<form
|
||||
onSubmit={(e) => { e.preventDefault(); addTag(); }}
|
||||
className="flex gap-2"
|
||||
>
|
||||
<Input
|
||||
value={tagInput}
|
||||
onChange={(e) => setTagInput(e.target.value)}
|
||||
placeholder="Add tag…"
|
||||
className="h-7 text-xs flex-1"
|
||||
disabled={addTagMut.isPending}
|
||||
/>
|
||||
<button
|
||||
type="submit"
|
||||
disabled={!tagInput.trim() || addTagMut.isPending}
|
||||
className="text-primary disabled:opacity-50"
|
||||
>
|
||||
<Plus className="h-4 w-4" />
|
||||
</button>
|
||||
</form>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Sharing */}
|
||||
{isOwner && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider mb-1.5">Sharing</p>
|
||||
{shares.length === 0 && (
|
||||
<p className="text-xs text-muted mb-1">Not shared with any groups</p>
|
||||
)}
|
||||
<div className="space-y-1 mb-1">
|
||||
{shares.map((share) => {
|
||||
const group = myGroups.find((g) => g.id === share.group_id);
|
||||
return (
|
||||
<div key={share.id} className="flex items-center gap-2 text-sm">
|
||||
<Users className="h-3.5 w-3.5 text-muted shrink-0" />
|
||||
<span className="flex-1 text-sm">{group?.name ?? share.group_id}</span>
|
||||
{share.can_delete && (
|
||||
<span title="Group members can delete this document">
|
||||
<Trash2 className="h-3 w-3 text-muted shrink-0" />
|
||||
</span>
|
||||
)}
|
||||
<button
|
||||
onClick={() => removeShareMut.mutate(share.group_id)}
|
||||
disabled={removeShareMut.isPending}
|
||||
className="text-muted hover:text-red-500 transition-colors disabled:opacity-50"
|
||||
title="Stop sharing"
|
||||
>
|
||||
<UserMinus className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
</div>
|
||||
<label className="flex items-center gap-1.5 text-xs text-muted cursor-pointer mt-1 mb-0.5 select-none">
|
||||
<input
|
||||
type="checkbox"
|
||||
checked={canDeleteNew}
|
||||
onChange={(e) => setCanDeleteNew(e.target.checked)}
|
||||
className="h-3 w-3 accent-primary"
|
||||
/>
|
||||
Allow group members to delete
|
||||
</label>
|
||||
<GroupCombobox
|
||||
groups={myGroups}
|
||||
sharedGroupIds={sharedGroupIds}
|
||||
onShare={(id) => addShareMut.mutate({ groupId: id, canDelete: canDeleteNew })}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Owner/permitted actions */}
|
||||
{(isOwner || doc.viewer_can_delete) && (
|
||||
<div className="flex gap-2 pt-1">
|
||||
{isOwner && (
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => reprocessMut.mutate()}
|
||||
disabled={reprocessMut.isPending || doc.status === "pending" || doc.status === "processing"}
|
||||
className="gap-1.5"
|
||||
>
|
||||
<RefreshCw className={cn("h-3.5 w-3.5", reprocessMut.isPending && "animate-spin")} />
|
||||
Re-analyse
|
||||
</Button>
|
||||
)}
|
||||
{doc.viewer_can_delete && (
|
||||
<Button
|
||||
variant="outline"
|
||||
size="sm"
|
||||
onClick={() => {
|
||||
if (confirm(`Delete "${doc.title ?? doc.filename}"? This cannot be undone.`)) {
|
||||
deleteMut.mutate();
|
||||
}
|
||||
}}
|
||||
disabled={deleteMut.isPending}
|
||||
className="gap-1.5 text-red-500 border-red-200 hover:bg-red-50 dark:hover:bg-red-900/20"
|
||||
>
|
||||
<Trash2 className="h-3.5 w-3.5" />
|
||||
Delete
|
||||
</Button>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Raw text */}
|
||||
{doc.source && (
|
||||
<div>
|
||||
<button
|
||||
onClick={() => setRawOpen((o) => !o)}
|
||||
className="flex items-center gap-1.5 text-xs text-muted hover:text-foreground transition-colors"
|
||||
>
|
||||
{rawOpen ? <ChevronDown className="h-3.5 w-3.5" /> : <ChevronRight className="h-3.5 w-3.5" />}
|
||||
Extracted text
|
||||
</button>
|
||||
{rawOpen && (
|
||||
<pre className="mt-2 text-xs bg-muted/10 border border-border rounded-md p-3 overflow-y-auto max-h-64 whitespace-pre-wrap break-words font-mono">
|
||||
{/* raw_text not in DocumentOut — show message */}
|
||||
(Raw text is stored server-side; use the backend API to retrieve it.)
|
||||
</pre>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,245 @@
|
||||
import { useState, useRef, useEffect } from "react";
|
||||
import { useMutation, useQuery, useQueryClient } from "@tanstack/react-query";
|
||||
import { X, Pencil, Trash2, Check, Lock } from "lucide-react";
|
||||
import {
|
||||
listCategories,
|
||||
renameCategory,
|
||||
deleteCategory,
|
||||
getMyGroups,
|
||||
getMe,
|
||||
type CategoryOut,
|
||||
ApiError,
|
||||
} from "@/api/client";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { Button } from "@/components/ui/button";
|
||||
|
||||
interface Props {
|
||||
onClose: () => void;
|
||||
}
|
||||
|
||||
export default function ManageCategoriesDialog({ onClose }: Props) {
|
||||
const queryClient = useQueryClient();
|
||||
|
||||
const { data: categories = [] } = useQuery({
|
||||
queryKey: ["categories"],
|
||||
queryFn: listCategories,
|
||||
});
|
||||
|
||||
const { data: myGroups = [] } = useQuery({
|
||||
queryKey: ["my-groups"],
|
||||
queryFn: getMyGroups,
|
||||
});
|
||||
|
||||
const { data: me } = useQuery({ queryKey: ["me"], queryFn: getMe });
|
||||
const isSuperuser = me?.is_admin ?? false;
|
||||
|
||||
// Set of group IDs for which the current user is a group admin
|
||||
const adminGroupIds = new Set(myGroups.filter((g) => g.is_group_admin).map((g) => g.id));
|
||||
|
||||
const [editingId, setEditingId] = useState<string | null>(null);
|
||||
const [editValue, setEditValue] = useState("");
|
||||
const [editError, setEditError] = useState<string | null>(null);
|
||||
const [search, setSearch] = useState("");
|
||||
const editInputRef = useRef<HTMLInputElement>(null);
|
||||
|
||||
useEffect(() => {
|
||||
if (editingId) editInputRef.current?.focus();
|
||||
}, [editingId]);
|
||||
|
||||
const renameMut = useMutation({
|
||||
mutationFn: ({ id, name }: { id: string; name: string }) => renameCategory(id, name),
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["categories"] });
|
||||
setEditingId(null);
|
||||
setEditError(null);
|
||||
},
|
||||
onError: (err) => {
|
||||
setEditError(err instanceof ApiError ? err.message : "Rename failed");
|
||||
},
|
||||
});
|
||||
|
||||
const deleteMut = useMutation({
|
||||
mutationFn: deleteCategory,
|
||||
onSuccess: () => queryClient.invalidateQueries({ queryKey: ["categories"] }),
|
||||
});
|
||||
|
||||
function canManage(cat: CategoryOut): boolean {
|
||||
if (isSuperuser) return true;
|
||||
if (cat.scope === "personal") return true;
|
||||
if (cat.scope === "group") return cat.group_id != null && adminGroupIds.has(cat.group_id);
|
||||
return false; // system — non-admin cannot manage
|
||||
}
|
||||
|
||||
const filtered = search
|
||||
? categories.filter((c) => c.name.toLowerCase().includes(search.toLowerCase()))
|
||||
: categories;
|
||||
|
||||
// Group by scope
|
||||
const personal = filtered.filter((c) => c.scope === "personal");
|
||||
const system = filtered.filter((c) => c.scope === "system");
|
||||
|
||||
// Group-scoped categories grouped by group_id
|
||||
const groupCats = filtered.filter((c) => c.scope === "group");
|
||||
const groupMap = new Map<string, { name: string; cats: CategoryOut[] }>();
|
||||
for (const cat of groupCats) {
|
||||
if (!cat.group_id) continue;
|
||||
if (!groupMap.has(cat.group_id)) {
|
||||
const grp = myGroups.find((g) => g.id === cat.group_id);
|
||||
groupMap.set(cat.group_id, { name: grp?.name ?? cat.group_id, cats: [] });
|
||||
}
|
||||
groupMap.get(cat.group_id)!.cats.push(cat);
|
||||
}
|
||||
|
||||
function startEdit(id: string, name: string) {
|
||||
setEditingId(id);
|
||||
setEditValue(name);
|
||||
setEditError(null);
|
||||
}
|
||||
|
||||
function submitEdit(id: string) {
|
||||
const name = editValue.trim();
|
||||
if (!name) return;
|
||||
if (name === categories.find((c) => c.id === id)?.name) {
|
||||
setEditingId(null);
|
||||
return;
|
||||
}
|
||||
renameMut.mutate({ id, name });
|
||||
}
|
||||
|
||||
function renderCat(cat: CategoryOut) {
|
||||
const manageable = canManage(cat);
|
||||
return (
|
||||
<div
|
||||
key={cat.id}
|
||||
className="flex items-center gap-2 py-1.5 px-2 rounded-md hover:bg-muted/10 group"
|
||||
>
|
||||
{editingId === cat.id ? (
|
||||
<div className="flex flex-col flex-1 gap-1">
|
||||
<form
|
||||
className="flex items-center gap-2"
|
||||
onSubmit={(e) => { e.preventDefault(); submitEdit(cat.id); }}
|
||||
>
|
||||
<Input
|
||||
ref={editInputRef}
|
||||
value={editValue}
|
||||
onChange={(e) => { setEditValue(e.target.value); setEditError(null); }}
|
||||
className="h-7 text-sm flex-1"
|
||||
disabled={renameMut.isPending}
|
||||
onKeyDown={(e) => { if (e.key === "Escape") { setEditingId(null); setEditError(null); } }}
|
||||
/>
|
||||
<button
|
||||
type="submit"
|
||||
disabled={!editValue.trim() || renameMut.isPending}
|
||||
className="text-primary hover:text-primary/80 disabled:opacity-50"
|
||||
>
|
||||
<Check className="h-4 w-4" />
|
||||
</button>
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => { setEditingId(null); setEditError(null); }}
|
||||
className="text-muted hover:text-foreground"
|
||||
>
|
||||
<X className="h-4 w-4" />
|
||||
</button>
|
||||
</form>
|
||||
{editError && (
|
||||
<p className="text-xs text-red-500 pl-0.5">{editError}</p>
|
||||
)}
|
||||
</div>
|
||||
) : (
|
||||
<>
|
||||
{!manageable && (
|
||||
<Lock className="h-3.5 w-3.5 text-muted flex-shrink-0" />
|
||||
)}
|
||||
<span className="flex-1 text-sm truncate">{cat.name}</span>
|
||||
{manageable && (
|
||||
<div className="flex items-center gap-1 opacity-0 group-hover:opacity-100 transition-opacity">
|
||||
<button
|
||||
onClick={() => startEdit(cat.id, cat.name)}
|
||||
className="text-muted hover:text-foreground transition-colors p-0.5"
|
||||
title="Rename"
|
||||
>
|
||||
<Pencil className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
<button
|
||||
onClick={() => {
|
||||
if (confirm(`Delete category "${cat.name}"? Documents in it will be uncategorised.`)) {
|
||||
deleteMut.mutate(cat.id);
|
||||
}
|
||||
}}
|
||||
disabled={deleteMut.isPending}
|
||||
className="text-muted hover:text-red-500 transition-colors p-0.5 disabled:opacity-50"
|
||||
title="Delete"
|
||||
>
|
||||
<Trash2 className="h-3.5 w-3.5" />
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function renderSection(title: string, cats: CategoryOut[]) {
|
||||
if (cats.length === 0) return null;
|
||||
return (
|
||||
<div className="mb-3">
|
||||
<p className="text-xs font-semibold text-muted uppercase tracking-wider px-2 mb-1">{title}</p>
|
||||
{cats.map(renderCat)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
const hasAny = filtered.length > 0;
|
||||
|
||||
return (
|
||||
<div
|
||||
className="fixed inset-0 z-50 flex items-center justify-center bg-background/70"
|
||||
onClick={(e) => { if (e.target === e.currentTarget) onClose(); }}
|
||||
>
|
||||
<div className="bg-surface border border-border rounded-lg w-[520px] max-h-[80vh] flex flex-col shadow-xl">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between px-5 py-4 border-b border-border">
|
||||
<h2 className="text-base font-semibold">Manage Categories</h2>
|
||||
<button onClick={onClose} className="text-muted hover:text-foreground transition-colors">
|
||||
<X className="h-5 w-5" />
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Search */}
|
||||
{categories.length > 6 && (
|
||||
<div className="px-5 pt-4 pb-2">
|
||||
<Input
|
||||
placeholder="Search categories…"
|
||||
value={search}
|
||||
onChange={(e) => setSearch(e.target.value)}
|
||||
className="h-8 text-sm"
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* List */}
|
||||
<div className="flex-1 overflow-y-auto px-5 py-3 min-h-0">
|
||||
{!hasAny && (
|
||||
<p className="text-sm text-muted py-4 text-center">
|
||||
{search ? "No categories match" : "No categories yet"}
|
||||
</p>
|
||||
)}
|
||||
{renderSection("My Categories", personal)}
|
||||
{Array.from(groupMap.entries()).map(([, { name, cats }]) =>
|
||||
renderSection(`Group: ${name}`, cats)
|
||||
)}
|
||||
{renderSection("System", system)}
|
||||
</div>
|
||||
|
||||
{/* Footer */}
|
||||
<div className="px-5 py-4 border-t border-border">
|
||||
<Button variant="outline" size="sm" onClick={onClose}>
|
||||
Close
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,129 @@
|
||||
import { useState, useEffect } from "react";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import { Input } from "@/components/ui/input";
|
||||
import { cn } from "@/lib/utils";
|
||||
import type { PluginSchemaProperty } from "@/api/client";
|
||||
|
||||
interface PluginSchema {
|
||||
type: string;
|
||||
title?: string;
|
||||
properties: Record<string, PluginSchemaProperty>;
|
||||
}
|
||||
|
||||
interface PluginSchemaFormProps {
|
||||
schema: PluginSchema;
|
||||
values: Record<string, unknown>;
|
||||
onSave: (values: Record<string, unknown>) => void;
|
||||
isPending?: boolean;
|
||||
isError?: boolean;
|
||||
isSuccess?: boolean;
|
||||
/** When true, the built-in save button row is hidden (caller renders its own). */
|
||||
noSaveButton?: boolean;
|
||||
/** Expose current form state to the parent via callback on every change. */
|
||||
onChange?: (values: Record<string, unknown>) => void;
|
||||
}
|
||||
|
||||
function Toggle({ checked, onChange }: { checked: boolean; onChange: (v: boolean) => void }) {
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
role="switch"
|
||||
aria-checked={checked}
|
||||
onClick={() => onChange(!checked)}
|
||||
className={cn(
|
||||
"relative inline-flex h-6 w-11 shrink-0 items-center rounded-full transition-colors focus-visible:outline-none focus-visible:ring-2 focus-visible:ring-primary",
|
||||
checked ? "bg-primary" : "bg-muted/60 border border-border"
|
||||
)}
|
||||
>
|
||||
<span
|
||||
className={cn(
|
||||
"inline-block h-4 w-4 rounded-full bg-white shadow-sm transition-transform",
|
||||
checked ? "translate-x-6" : "translate-x-1"
|
||||
)}
|
||||
/>
|
||||
</button>
|
||||
);
|
||||
}
|
||||
|
||||
export default function PluginSchemaForm({
|
||||
schema,
|
||||
values,
|
||||
onSave,
|
||||
isPending,
|
||||
isError,
|
||||
isSuccess,
|
||||
noSaveButton,
|
||||
onChange,
|
||||
}: PluginSchemaFormProps) {
|
||||
const [form, setForm] = useState<Record<string, unknown>>(values);
|
||||
|
||||
useEffect(() => {
|
||||
setForm(values);
|
||||
}, [values]);
|
||||
|
||||
const setField = (key: string, value: unknown) => {
|
||||
const next = { ...form, [key]: value };
|
||||
setForm(next);
|
||||
onChange?.(next);
|
||||
};
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
{Object.entries(schema.properties).map(([key, prop]) => (
|
||||
<div key={key} className="space-y-1.5">
|
||||
<div className="flex items-center justify-between gap-4">
|
||||
<div className="min-w-0">
|
||||
<p className="text-sm font-medium text-foreground">{prop.title}</p>
|
||||
{prop.description && (
|
||||
<p className="text-xs text-muted mt-0.5">{prop.description}</p>
|
||||
)}
|
||||
</div>
|
||||
{prop.type === "boolean" && !prop.readOnly && (
|
||||
<Toggle
|
||||
checked={Boolean(form[key])}
|
||||
onChange={(v) => setField(key, v)}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{prop.type === "string" && prop.readOnly && (
|
||||
<p className="text-sm text-muted font-mono bg-muted/20 px-3 py-1.5 rounded-md border border-border">
|
||||
{String(form[key] ?? "")}
|
||||
</p>
|
||||
)}
|
||||
|
||||
{prop.type === "string" && !prop.readOnly && (
|
||||
<Input
|
||||
value={String(form[key] ?? "")}
|
||||
onChange={(e) => setField(key, e.target.value)}
|
||||
className="h-9"
|
||||
/>
|
||||
)}
|
||||
|
||||
{prop.type === "number" && !prop.readOnly && (
|
||||
<Input
|
||||
type="number"
|
||||
value={String(form[key] ?? "")}
|
||||
onChange={(e) => setField(key, Number(e.target.value))}
|
||||
className="h-9"
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
))}
|
||||
|
||||
{!noSaveButton && (
|
||||
<div className="flex items-center gap-3 pt-2">
|
||||
<Button onClick={() => onSave(form)} disabled={isPending} size="sm">
|
||||
{isPending ? "Saving…" : "Save changes"}
|
||||
</Button>
|
||||
{isError && (
|
||||
<span className="text-sm text-destructive">Failed to save. Please try again.</span>
|
||||
)}
|
||||
{isSuccess && !isPending && (
|
||||
<span className="text-sm text-green-600 dark:text-green-400">Saved successfully.</span>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -12,14 +12,14 @@ import {
|
||||
LogOut,
|
||||
UserCircle,
|
||||
FileText,
|
||||
Folder,
|
||||
Users,
|
||||
UsersRound,
|
||||
Palette,
|
||||
} from "lucide-react";
|
||||
import { Button } from "@/components/ui/button";
|
||||
import ThemeToggle from "@/components/ThemeToggle";
|
||||
import { useAuth } from "@/hooks/useAuth";
|
||||
import { getMe, listCategories } from "@/api/client";
|
||||
import { getMe } from "@/api/client";
|
||||
import { cn } from "@/lib/utils";
|
||||
|
||||
export default function Sidebar() {
|
||||
@@ -29,11 +29,9 @@ export default function Sidebar() {
|
||||
const { data: user } = useQuery({ queryKey: ["me"], queryFn: getMe });
|
||||
|
||||
const isAppsRoute = location.pathname.startsWith("/apps");
|
||||
const isDocsRoute = location.pathname.startsWith("/apps/documents");
|
||||
const isAdminRoute = location.pathname.startsWith("/admin");
|
||||
|
||||
const [appsOpen, setAppsOpen] = useState(isAppsRoute);
|
||||
const [docsOpen, setDocsOpen] = useState(isDocsRoute);
|
||||
const [adminOpen, setAdminOpen] = useState(isAdminRoute);
|
||||
|
||||
// Auto-open sections when navigating to their routes
|
||||
@@ -41,20 +39,10 @@ export default function Sidebar() {
|
||||
if (isAppsRoute) setAppsOpen(true);
|
||||
}, [isAppsRoute]);
|
||||
|
||||
useEffect(() => {
|
||||
if (isDocsRoute) setDocsOpen(true);
|
||||
}, [isDocsRoute]);
|
||||
|
||||
useEffect(() => {
|
||||
if (isAdminRoute) setAdminOpen(true);
|
||||
}, [isAdminRoute]);
|
||||
|
||||
const { data: categories = [] } = useQuery({
|
||||
queryKey: ["categories"],
|
||||
queryFn: listCategories,
|
||||
enabled: appsOpen && docsOpen && !!user,
|
||||
});
|
||||
|
||||
const navItemClass = (isActive: boolean) =>
|
||||
cn(
|
||||
"flex items-center rounded-lg transition-colors",
|
||||
@@ -73,15 +61,6 @@ export default function Sidebar() {
|
||||
: "text-muted hover:bg-muted/20 hover:text-foreground"
|
||||
);
|
||||
|
||||
const subSubItemClass = (isActive: boolean) =>
|
||||
cn(
|
||||
"flex items-center rounded-lg transition-colors text-sm",
|
||||
"pl-12 pr-3 py-1 gap-2",
|
||||
isActive
|
||||
? "bg-primary/10 text-primary"
|
||||
: "text-muted hover:bg-muted/20 hover:text-foreground"
|
||||
);
|
||||
|
||||
return (
|
||||
<aside
|
||||
className={cn(
|
||||
@@ -146,53 +125,13 @@ export default function Sidebar() {
|
||||
{/* Apps sub-items — only when sidebar is expanded and appsOpen */}
|
||||
{sidebarExpanded && appsOpen && (
|
||||
<div className="mt-0.5 space-y-0.5">
|
||||
{/* Documents service */}
|
||||
<div>
|
||||
<div
|
||||
className={cn(
|
||||
"flex items-center rounded-lg transition-colors text-sm",
|
||||
isDocsRoute
|
||||
? "bg-primary/10 text-primary"
|
||||
: "text-muted hover:bg-muted/20 hover:text-foreground"
|
||||
)}
|
||||
>
|
||||
<NavLink
|
||||
to="/apps/documents"
|
||||
end
|
||||
className="flex items-center gap-2 pl-8 pr-2 py-1.5 flex-1 min-w-0"
|
||||
className={({ isActive }) => subItemClass(isActive)}
|
||||
>
|
||||
<FileText className="h-4 w-4 shrink-0" />
|
||||
<span className="whitespace-nowrap">Documents</span>
|
||||
</NavLink>
|
||||
<button
|
||||
onClick={() => setDocsOpen((o) => !o)}
|
||||
className="px-2 py-1.5 rounded-r-lg"
|
||||
aria-label={docsOpen ? "Collapse documents" : "Expand documents"}
|
||||
>
|
||||
{docsOpen ? (
|
||||
<ChevronDown className="h-3.5 w-3.5 shrink-0" />
|
||||
) : (
|
||||
<ChevronRight className="h-3.5 w-3.5 shrink-0" />
|
||||
)}
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Categories */}
|
||||
{docsOpen && (
|
||||
<div className="mt-0.5 space-y-0.5">
|
||||
{categories.map((cat) => (
|
||||
<NavLink
|
||||
key={cat.id}
|
||||
to={`/apps/documents?category_id=${cat.id}`}
|
||||
className={({ isActive }) => subSubItemClass(isActive)}
|
||||
>
|
||||
<Folder className="h-3.5 w-3.5 shrink-0" />
|
||||
<span className="whitespace-nowrap truncate">{cat.name}</span>
|
||||
</NavLink>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
@@ -265,6 +204,13 @@ export default function Sidebar() {
|
||||
<UsersRound className="h-4 w-4 shrink-0" />
|
||||
<span className="whitespace-nowrap">Groups</span>
|
||||
</NavLink>
|
||||
<NavLink
|
||||
to="/admin/appearance"
|
||||
className={({ isActive }) => subItemClass(isActive)}
|
||||
>
|
||||
<Palette className="h-4 w-4 shrink-0" />
|
||||
<span className="whitespace-nowrap">Appearance</span>
|
||||
</NavLink>
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user