fix: remove obsolete /data/documents and /config dirs from Dockerfiles

doc-service and ai-service no longer use local filesystem directories —
all file and config I/O goes through storage-service. Update README and
CLAUDE.md to reflect 6-service architecture, new volumes, and add
storage-service step to the "Adding a new resource" checklist.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

.claude/agents/security-auditor.md

@@ -83,3 +83,6 @@ Key management: private key (`JWT_PRIVATE_KEY`) signs tokens and must never be e
 - Never use `text()` with string interpolation in SQLAlchemy queries
 - Never expose `hashed_password`, `is_superuser`, or internal IDs in API responses unless explicitly required
 - After any code change, verify the pre-commit hook still passes
+- **Never mount `/var/run/docker.sock` directly into the backend container** — Docker socket access must always go through `tecnativa/docker-socket-proxy` on an internal-only network with a minimal API whitelist. Raw socket access inside any app container is equivalent to root on the host.
+- **Never spawn `--privileged` containers** or containers with added capabilities for app workloads
+- **Expose the bare minimum of ports to the host** — only the frontend binds a host port (80 prod / 5173 dev). The database and backend must never have `ports:` in any compose file; they are reachable only via internal Docker networks. If a new service is added, default to no host port binding unless there is an explicit reason.

.claude/agents/ux-designer.md

@@ -1,47 +1,29 @@
 ---
 name: ux-designer
-description: Advisory UX/UI designer for this project. Use when you need feedback on user flows, page layout, navigation structure, component hierarchy, or visual consistency. Connects to a self-hosted Penpot instance via REST API once configured. Returns analysis and design recommendations — does not write code.
+description: Advisory UX/UI designer for this project. Use when you need feedback on user flows, page layout, navigation structure, component hierarchy, or visual consistency. Connects to Figma via REST API once configured. Returns analysis and design recommendations — does not write code.
 model: claude-sonnet-4-6
 tools:
   - Read
   - Grep
   - Glob
+  - Bash
   - WebFetch
   - WebSearch
 ---
 
 You are a senior UX/UI designer advising on this specific project. Your role is purely advisory — you analyse user flows, critique layouts, and produce design recommendations, but you do not write or modify source code files directly.
 
-## ⚠ Next session checklist — Penpot not yet set up
+## Figma connection — active
 
-The design tooling has been decided but not yet configured. At the start of
-any session involving UX/UI work, check this list and remind the user of
-outstanding steps before doing anything else:
+File key: `kcmvLytS31lSjP44YpBUSn`
 
-- [ ] **1. Spin up Penpot LXC on the server**
-  - Penpot runs via Docker Compose: https://github.com/penpot/penpot/tree/main/docker/images
-  - Recommended: separate LXC container, 2–4 GB RAM, Docker installed
-  - Default compose file: `docker-compose.yaml` in the penpot repo
-  - Expose via a subdomain (e.g. `penpot.yourdomain.com`) behind the same
-    reverse proxy / nginx proxy manager planned for the app
+The user provides a fresh personal access token at the start of each session:
 
-- [ ] **2. Create a Penpot account and project**
-  - Register on your self-hosted instance
-  - Create a project called `destroying_sap` (or the customer's brand name)
-  - Create a file for the app design
+> "Use Figma file kcmvLytS31lSjP44YpBUSn with token: <your-token>"
 
-- [ ] **3. Generate a Penpot access token**
-  - Penpot UI → Profile → Access tokens → Create token
-  - Keep the token — you will paste it into the agent prompt when invoking
-    this agent for design work
-
-- [ ] **4. Note your instance URL**
-  - The agent will call `https://penpot.yourdomain.com/api/rpc/command/...`
-  - Confirm the API is reachable (no auth firewall blocking it)
-
-- [ ] **5. Decide on a UI component library**
+- [ ] **Decide on a UI component library**
   - Pending decision: Tailwind + shadcn/ui, MUI, or plain CSS
-  - This decision affects both the Penpot design system (tokens, components)
+  - This decision affects both the Figma design system (tokens, components)
     and the frontend implementation
   - Recommendation: shadcn/ui — pairs well with Tailwind, ships unstyled
     accessible primitives, fits the white-label requirement (customer logo +
@@ -49,22 +31,24 @@ outstanding steps before doing anything else:
 
 ---
 
-## Using Penpot once set up
+## Calling the Figma API
 
-Provide the agent with your access token and instance URL at the start of
-the session:
+Use `Bash` with `curl` for all Figma API calls — `WebFetch` does not support
+custom headers and will get a 403.
 
-> "Use Penpot at https://penpot.yourdomain.com with token: <your-token>"
+```bash
+curl -s -H "X-Figma-Token: <token>" "https://api.figma.com/v1/files/<file_key>"
+```
 
-The agent will then use `WebFetch` to call the Penpot REST API directly —
-no MCP server or plugin required.
+Useful endpoints (base URL: `https://api.figma.com/v1`):
+- `GET  /me` — verify token, returns account info
+- `GET  /files/<file_key>` — full file contents (pages, frames, components)
+- `GET  /files/<file_key>/components` — published components
+- `GET  /files/<file_key>/styles` — published styles (colours, text, effects)
+- `GET  /teams/<team_id>/projects` — projects in a team
+- `GET  /projects/<project_id>/files` — files in a project
 
-Useful API endpoints:
-- `GET  /api/rpc/command/get-profile` — verify token works
-- `GET  /api/rpc/command/get-projects` — list projects
-- `GET  /api/rpc/command/get-files?project-id=<id>` — list design files
-- `GET  /api/rpc/command/get-file?file-id=<id>` — read full file contents
-- `POST /api/rpc/command/update-file` — write design changes
+All requests require the header `X-Figma-Token: <your-token>`.
 
 ---
 
@@ -72,7 +56,7 @@ Useful API endpoints:
 
 - **App type**: Employer/employee management SaaS — B2B, not consumer
 - **Current state**: Functional but unstyled — plain inline CSS, no design
-  system chosen yet (see checklist item 5 above)
+  system chosen yet (see checklist item 4 above)
 - **Pages**: Login (landing), Dashboard (/), Apps (/apps),
   Settings (/settings), Profile (/profile), Admin (/admin — admin only)
 - **Nav**: Home | Apps | Settings | [Admin] | Logout — on all protected pages

.claude/settings.local.json

@@ -5,8 +5,21 @@
       "Bash(git add:*)",
       "Bash(git commit -m ':*)",
       "Bash(git push:*)",
+      "Bash(git pull:*)",
+      "Bash(git checkout:*)",
+      "Bash(git merge:*)",
+      "Bash(git branch:*)",
+      "Bash(git log:*)",
+      "Bash(git diff:*)",
+      "Bash(git status:*)",
+      "Bash(git config:*)",
+      "Bash(git mv:*)",
       "Bash(docker compose:*)",
-      "Bash(docker run:*)"
+      "Bash(docker run:*)",
+      "Bash(docker inspect:*)",
+      "Bash(docker ps:*)",
+      "Bash(curl:*)",
+      "Bash(lsof:*)"
     ]
   }
 }

.githooks/pre-commit

@@ -4,6 +4,14 @@
 
 REPO_ROOT="$(git rev-parse --show-toplevel)"
 
+# Resolve Docker socket — the git hook environment may not inherit the active
+# Docker context, so we probe common socket paths explicitly.
+if [ -S "/Users/$USER/.docker/run/docker.sock" ]; then
+  export DOCKER_HOST="unix:///Users/$USER/.docker/run/docker.sock"
+elif [ -S "/var/run/docker.sock" ]; then
+  export DOCKER_HOST="unix:///var/run/docker.sock"
+fi
+
 # Collect staged files on the host and pass them into the container as arguments
 STAGED=$(git diff --cached --name-only --diff-filter=ACM)
 
@@ -20,10 +28,10 @@ docker run --rm \
   -w /repo \
   -e STAGED_FILES="$STAGED" \
   -u 1001:1001 \
-  -e HOME=/tmp \
   -e PIP_DISABLE_PIP_VERSION_CHECK=1 \
+  -e PIP_NO_CACHE_DIR=1 \
   python:3.12-slim \
-  sh -c "pip install --quiet --user --no-warn-script-location bandit && python scripts/security_check.py"
+  sh -c "python -m venv /tmp/venv && /tmp/venv/bin/pip install --quiet bandit && /tmp/venv/bin/python scripts/security_check.py"
 
 EXIT_CODE=$?
 

.gitignore

@@ -17,3 +17,14 @@ frontend/dist/
 
 # OS
 .DS_Store
+resume.txt
+
+# Test fixtures — drop PDFs here for local testing, never commit them
+features/doc-service/tests/pdfs/*.pdf
+
+# Feature branch test stacks — never commit these
+docker-compose.feat-*.yml
+
+# Don't sync .un files
+*.un~
+dev-watch/**/*.pdf

CLAUDE.md

@@ -1,150 +1,445 @@
 # CLAUDE.md
 
-This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
+This file provides permanent, authoritative guidance to Claude Code for every session. It covers project-wide concerns only. Service-specific details live in sub-files — read them only when working in that service:
+
+- `backend/CLAUDE.md` — auth/users/admin/settings/plugins endpoints; DB models; JWT/bcrypt/sanitization security; naming conventions
+- `frontend/CLAUDE.md` — routes, components, API client patterns, XSS prevention
+- `features/ai-service/CLAUDE.md` — /chat, /health, /queue endpoints; queue service
+- `features/doc-service/CLAUDE.md` — document/category/share endpoints; DB models; PDF limits; file watcher
+- `features/storage-service/CLAUDE.md` — storage API, pluggable backend drivers (local/S3/WebDAV), migration
+
+---
+
+## Merge checklist
+
+Before merging any feature branch into `main`, every test relevant to the changed area in `tests/ALL_TESTS.md` (and the relevant service-specific file) must be marked passing. The test suite covers all 20 feature areas across five service files:
+
+- `tests/backend_tests.md` — §1–9, §18
+- `tests/frontend_tests.md` — §19
+- `tests/doc-service_tests.md` — §10–16
+- `tests/ai-service_tests.md` — §17
+- `tests/storage-service_tests.md` — §20
+
+Do not merge without it.
+
+---
+
+## CLAUDE.md self-update checkpoint
+
+**After every change to the codebase**, before committing, check which CLAUDE.md files need updating:
+
+- New route added → update **API Endpoints** in `backend/CLAUDE.md`, `features/doc-service/CLAUDE.md`, or `features/ai-service/CLAUDE.md`; update **Frontend Routes** in `frontend/CLAUDE.md`
+- New DB model or column → update **Database Models** in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
+- New migration → update **Migration chain** table in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
+- New file or directory → update **File & Folder Tree** in the relevant sub-file; update the high-level tree in this root file only if a top-level directory changes
+- New limit or default value changed → update **Default Values & Limits** in the relevant sub-file
+- New dependency, auth mechanism, or security pattern → update **Security Standards** in the relevant sub-file
+- New Docker service, volume, network, or env var → update **Docker Infrastructure** in this file
+- Stack version changed → update **Stack** in this file
+
+- New feature or endpoint added → add test rows to **both** `tests/ALL_TESTS.md` (in the relevant section) **and** the matching service-specific file (`tests/backend_tests.md`, `tests/frontend_tests.md`, `tests/doc-service_tests.md`, `tests/ai-service_tests.md`, or `tests/storage-service_tests.md`). Use the same test number and format as existing rows.
+
+This check is mandatory — treat it the same as updating STATUS.md.
+
+---
 
 ## Stack
 
 | Layer | Tech |
 |---|---|
-| Backend | FastAPI (async), SQLAlchemy 2 (async), Alembic, PostgreSQL |
-| Auth | JWT via `python-jose`, bcrypt via `bcrypt` (direct, no passlib) |
+| Backend | FastAPI (async), SQLAlchemy 2 (async), Alembic, PostgreSQL 16 |
+| Auth | JWT RS256 via `python-jose`, bcrypt via `bcrypt` (direct, 13 rounds) |
 | Frontend | React 18, TypeScript, Vite, React Router v6, TanStack Query, Axios |
-| Dev DB | PostgreSQL 16 via Docker Compose |
+| UI Library | shadcn/ui (Radix primitives + Tailwind CSS v3) |
+| Styling | Tailwind CSS v3, CSS custom properties for theme tokens |
+| Containerisation | Docker Compose (5 services, non-root users, named volumes) |
+
+---
 
 ## Commands
 
-### Backend (run from `backend/`)
-
-```bash
-# Install
-python -m venv .venv && source .venv/bin/activate
-pip install -e ".[dev]"
-
-# Run dev server
-uvicorn app.main:app --reload
-
-# Lint / format
-ruff check . && ruff format .
-
-# Tests
-pytest
-pytest tests/test_auth.py   # single file
-
-# Migrations
-alembic revision --autogenerate -m "describe change"
-alembic upgrade head
-alembic downgrade -1
-```
-
-### Frontend (run from `frontend/`)
-
-```bash
-npm install
-npm run dev        # Vite dev server at :5173, proxies /api → :8000
-npm run build
-npm run typecheck
-npm run lint
-```
-
-### Full stack via Docker
+All test, build, and package-manager commands run **inside Docker** — never on the host. See the memory note: "Testing inside Docker only".
+
+### Full stack
 
 ```bash
+# Dev stack (hot-reload, Vite on :5173)
 cp .env.example backend/.env
-docker compose up --build
+docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
+
+# Prod stack
+docker compose up --build -d
 ```
 
+For service-specific commands (migrations, lint), see `backend/CLAUDE.md` and `frontend/CLAUDE.md`.
+
+---
+
+## File & Folder Tree
+
+```
+/
+├── CLAUDE.md                              ← This file — project-wide context
+├── README.md                              ← Project overview, containers table, Current State
+├── TODO.md                                ← Task list
+├── .env.example                           ← Template for backend/.env
+├── docker-compose.yml                     ← Production (5 services, named volumes)
+├── docker-compose.dev.yml                 ← Dev overrides (hot-reload, host ports)
+├── .githooks/pre-commit                   ← Runs scripts/security_check.py before every commit
+├── scripts/security_check.py             ← Static analysis: secrets, weak crypto, SQLi, JWT
+├── changelog/YYYY-MM-DD_<slug>.md        ← Per-date change logs
+├── tests/ALL_TESTS.md                     ← Full test suite (all 19 areas); must pass before merging to main
+├── tests/backend_tests.md                 ← Backend-only tests (§1–9, §18)
+├── tests/frontend_tests.md               ← Frontend-only tests (§19)
+├── tests/doc-service_tests.md            ← Doc-service tests (§10–16)
+├── tests/ai-service_tests.md             ← AI-service tests (§17)
+├── dev-watch/                             ← Dev bind-mount for file watcher testing (.gitkeep only)
+│
+├── backend/                               ← FastAPI gateway (port 8000, internal); see backend/CLAUDE.md
+├── features/
+│   ├── ai-service/                        ← AI provider intermediary (port 8010, internal); see features/ai-service/CLAUDE.md
+│   └── doc-service/                       ← PDF extraction microservice (port 8001, internal); see features/doc-service/CLAUDE.md
+└── frontend/                              ← React SPA (port 5173 dev / 80 prod); see frontend/CLAUDE.md
+```
+
+---
+
 ## Architecture
 
 ### Request flow
 
 ```
-Browser → Vite dev server (:5173)
-  /api/* → proxy → FastAPI (:8000)
-    → router → dependency injection (get_db, get_current_user)
-      → SQLAlchemy async session → PostgreSQL
+Browser (:5173 dev / :80 prod)
+  │
+  └── Vite dev proxy / nginx
+        │
+        └── /api/*  ──→  backend:8000  (FastAPI)
+                              │
+              ┌───────────────┼───────────────────┐
+           /auth           /admin              /documents/*
+           /users          /groups             /documents/categories/*
+           /profile        /settings
+           /services            │                   │
+                           JSON volume         proxy (injects x-user-id,
+                           (/config)           x-user-groups)           │
+                                             doc-service:8001
+                                                    │
+                                             ai-service:8010
+                                             (classify, chat)
 ```
 
-### Backend layout
-
-- `app/main.py` — FastAPI app, CORS, router registration
-- `app/core/config.py` — all settings via `pydantic-settings` (reads `.env`)
-- `app/core/security.py` — password hashing and JWT encode/decode
-- `app/database.py` — async engine, `AsyncSessionLocal`, `Base` (all models inherit from here)
-- `app/models/` — SQLAlchemy ORM models; import them all in `__init__.py` so Alembic detects them
-- `app/schemas/` — Pydantic request/response models (separate from ORM models)
-- `app/routers/` — one file per resource; mount in `main.py`
-- `app/deps.py` — FastAPI dependencies: `get_current_user` validates JWT and returns `User`
-
-### Frontend layout
-
-- `src/api/client.ts` — single Axios instance; all API calls live here, token injected via interceptor
-- `src/hooks/useAuth.ts` — token state (localStorage), `login`, `logout`; consumed by pages and `App.tsx`
-- `src/pages/` — one file per route; data fetching via TanStack Query
-- `src/App.tsx` — route tree; `PrivateRoute` wrapper redirects to `/login` when no token
-
 ### Auth flow
 
-1. `POST /api/auth/login` returns a JWT bearer token
-2. Token stored in `localStorage`, attached to every request by the Axios interceptor
-3. Protected routes call `GET /api/users/me`; `get_current_user` dep validates the token on the server
+1. `POST /api/auth/login` → RS256 JWT (8 h), stored in `localStorage`
+2. Axios interceptor injects `Authorization: Bearer {token}` on every request
+3. `get_current_user` dep validates token on every protected route
+4. Admin routes additionally check `user.is_superuser`; return 404 (not 403) if not admin
 
-## Git convention
+---
 
-Always run `git push` immediately after every `git commit`.
+## Security Standards
 
-## Infrastructure change protocol
+These standards are **non-negotiable**. Every change must comply. Implementation-specific security rules (JWT, bcrypt, input sanitization, XSS, SQLi, admin routes) are in the relevant sub-CLAUDE.md files.
 
-After **any** change to Dockerfiles, `docker-compose*.yml`, `nginx.conf`, setup scripts, or installation / usage procedures:
+### Network isolation
 
-1. **Update `README.md`** — keep the Containers table, ports, image names, and Current State section accurate.
-2. **Spin up the dev stack** and verify that login and registration work end-to-end:
-   ```bash
-   docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
-   ```
-3. **Spin up the prod stack** and run the same checks:
-   ```bash
-   docker compose up --build -d
-   ```
-4. Confirm each container is running as a non-root user (`docker inspect <container> --format '{{.Config.User}}'`).
-5. **Tear down after testing** — stop and remove all containers, networks, and volumes:
-   ```bash
-   docker compose down --volumes --remove-orphans
-   # or for dev stack:
-   docker compose -f docker-compose.yml -f docker-compose.dev.yml down --volumes --remove-orphans
-   ```
+- `backend-net`: all containers except frontend; not reachable from host in prod.
+- `frontend-net`: only frontend; single host port (80 prod / 5173 dev).
+- DB, backend, doc-service, ai-service, storage-service have **no** host port bindings in prod.
 
-## Security hook
+### Storage rule (non-negotiable)
 
-A pre-commit hook lives in `.githooks/pre-commit` and runs `scripts/security_check.py` inside a Docker container. It is registered via `git config core.hooksPath .githooks` (already set in this repo).
+**No service may write to a filesystem path for persistent data.** All file/blob storage must go through the storage-service HTTP API (`PUT/GET/DELETE /objects/{bucket}/{key}`). Config JSON files must be stored in the `config` bucket. Uploaded files must be stored in the `documents` bucket. Violation is a security and architecture defect.
 
-The hook checks staged files for:
-- Hardcoded credentials / secrets
-- Dangerous patterns (`eval`, `exec`, `shell=True`, `pickle`)
-- Weak cryptography (MD5, SHA1, DES)
-- SQL injection risk
-- Debug flags left in code
-- `bandit` static analysis on all Python files
+The only two persistent storage mechanisms in the project are:
+1. **PostgreSQL** — structured/relational data
+2. **storage-service** — all file/blob/config data (local filesystem by default; switchable to S3-compatible or WebDAV)
 
-New clones must run `git config core.hooksPath .githooks` to activate the hook.
+New services and features must follow this pattern. See `features/storage-service/CLAUDE.md` for the API reference.
 
-## Changelog convention
+### Pre-commit security hook
 
-Every time files are added or modified, append an entry to the relevant file in `changelog/` (one file per date, named `YYYY-MM-DD_<slug>.md`). If a file for today already exists, append to it rather than creating a new one.
+`.githooks/pre-commit` runs `scripts/security_check.py` on every staged commit. It blocks commits that contain:
+
+1. Hardcoded credentials / private keys / AWS creds
+2. `eval()`, `exec()`, `shell=True`, `pickle.loads()`, `yaml.load()` without SafeLoader
+3. MD5, SHA1, DES, `random.random()` / `random.randint()` for security use
+4. SQL f-strings / format strings / concatenation passed to `execute()`/`query()`
+5. JWT algorithm `"none"`, `verify_exp=False`, expiry > 9999 min, hardcoded secrets
+6. `debug=True`, `print()` with passwords
+7. `bandit` static analysis failures
+
+**Never** bypass with `--no-verify` unless explicitly instructed by the user.
+
+---
+
+## Default Values & Limits (cross-cutting)
+
+| Parameter | Value | Location |
+|-----------|-------|----------|
+| Health check interval | 30 s | `service_health.py` |
+| Service poll (frontend) | 30 s | `AppsPage.tsx`, `DashboardPage.tsx` |
+
+All other per-service defaults are in the relevant sub-CLAUDE.md file.
+
+---
+
+## Docker Infrastructure
+
+### Services
+
+| Service | Image base | Internal port | User | Volumes | Network |
+|---------|-----------|---------------|------|---------|---------|
+| `db` | postgres:16-alpine | 5432 | 70:70 | `postgres_data` | backend-net |
+| `backend` | python:3.12-slim | 8000 | 1001:1001 | — | backend-net |
+| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | — | backend-net |
+| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `watch_data` | backend-net |
+| `storage-service` | python:3.12-slim | 8020 | 1001:1001 | `storage_data` | backend-net |
+| `frontend` | nginx-unprivileged:alpine | 8080 | 1001:1001 | — | backend-net, frontend-net |
+
+### Volumes
+
+| Volume | Mount path | Contains |
+|--------|-----------|---------|
+| `postgres_data` | `/var/lib/postgresql/data` | PostgreSQL data |
+| `storage_data` | `/data/storage` | All file/blob storage: PDFs (`documents/`) and config JSONs (`config/`) |
+| `watch_data` | `/data/watch` | Watch directory (bind-mount NAS/Nextcloud via docker-compose.override.yml) |
+
+### Networks
+
+| Network | Host-accessible | Members |
+|---------|----------------|---------|
+| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, storage-service, frontend |
+| `frontend-net` | Yes (port 80 → frontend:8080) | frontend |
+
+### Environment variables (required in `backend/.env`)
+
+```
+DATABASE_URL=postgresql+asyncpg://<user>:<pass>@db:5432/destroying_sap
+CORS_ORIGINS=["http://localhost:5173"]
+JWT_PRIVATE_KEY=<PEM, newlines as \n>
+JWT_PUBLIC_KEY=<PEM, newlines as \n>
+```
+
+Injected by docker-compose (not in `.env`):
+```
+DOC_SERVICE_URL=http://doc-service:8001
+AI_SERVICE_URL=http://ai-service:8010
+STORAGE_SERVICE_URL=http://storage-service:8020
+```
+
+---
+
+## Workflows
+
+### STATUS.md workflow
+
+Every directory with runnable code has a `STATUS.md`. These are the canonical **resume point** for each session.
+
+**At the start of every conversation:**
+1. Read the `STATUS.md` for every directory you will touch.
+2. If it does not exist for a directory you are working in, create it using the structure below.
+
+This applies equally to subagents.
+
+**After making changes**, update affected `STATUS.md` files:
+- Add new endpoints / models / routes.
+- Move completed items off the **Future work** checklist.
+- Add new items to **Known limitations** or **Future work**.
+- Keep the **What it is** summary accurate.
+
+**Structure:**
+```markdown
+# <Service Name> — Status
+
+## What it is
+One paragraph: purpose, port, database/storage, how traffic arrives.
+
+## Current functionality
+Subsections per router / feature area. Tables for endpoints.
+
+## Architecture
+ASCII diagram of call graph / data flow.
+
+## Known limitations / not implemented
+Bullet list of known gaps.
+
+## Future work
+- [ ] Planned improvements
+```
+
+Maintained in: `backend/`, `features/ai-service/`, `features/doc-service/`, `frontend/`
+
+---
+
+### Changelog convention
+
+Every time files are added or modified, append to `changelog/YYYY-MM-DD_<slug>.md`. If today's file exists, append; otherwise create new.
 
 Each entry must include:
-- A heading with the date and a short description
+- A heading with date and short description
 - `**Timestamp:**` in ISO-8601 format
 - A **Summary** sentence
 - A **Files Added / Modified / Deleted** list with one-line descriptions
 
-The `README.md` **Current State** section should be kept up to date whenever significant features are added or removed.
+---
+
+### Adding a new resource (checklist)
+
+1. Add ORM model in `backend/app/models/`, import it in `models/__init__.py`
+2. Run migration: `docker compose exec backend alembic revision --autogenerate -m "add <resource>"` then `alembic upgrade head`
+3. Add Pydantic schemas in `backend/app/schemas/`
+4. Add router in `backend/app/routers/`, mount it in `main.py`
+5. Add API function(s) to `frontend/src/api/client.ts`
+6. Add page component in `frontend/src/pages/`, register route in `App.tsx`
+7. If the resource involves file or blob data: store it via `PUT /objects/{bucket}/{key}` on `storage-service:8020`. Never write to the local filesystem. See `features/storage-service/CLAUDE.md` for the API.
+8. Update `STATUS.md` for affected services
+9. Add changelog entry
 
 ---
 
-### Adding a new resource
+### Git convention
 
-1. Add ORM model in `app/models/`, import it in `app/models/__init__.py`
-2. Run `alembic revision --autogenerate -m "add <resource>"` + `alembic upgrade head`
-3. Add Pydantic schemas in `app/schemas/`
-4. Add router in `app/routers/`, mount it in `app/main.py`
-5. Add API function(s) to `src/api/client.ts`, add page/component, register route in `App.tsx`
+Always run `git push` immediately after every `git commit`.
+
+---
+
+### Feature branch & isolated test environment
+
+Every non-trivial implementation (anything beyond a one-line fix or doc change) **must** follow this workflow:
+
+#### 0 — Mandatory planning phase (REQUIRED before any code changes)
+
+Before touching any code, present a written plan and **wait for explicit user approval**. Do not open files to edit, do not create branches, do not write code until the user says the plan is approved.
+
+The plan must include:
+- **What** is changing and **why**
+- **Which files** will be created or modified (with paths)
+- **Database / migration impact** (if any)
+- **API contract changes** (new endpoints, changed schemas)
+- **Frontend route / component changes**
+- **Risks or non-obvious decisions**
+
+Only proceed to step 1 after the user responds with explicit approval (e.g. "looks good", "go ahead", "approved").
+
+#### 1 — Create a feature branch
+After the planning phase is approved, branch off `main`. Name the branch after the title of the change — use lowercase words separated by hyphens, descriptive enough to understand at a glance what the branch does:
+```bash
+git checkout main && git pull
+git checkout -b feat/<descriptive-title>   # e.g. feat/user-profile-avatar-upload, feat/document-bulk-delete
+```
+
+#### 2 — Spin up an isolated Docker stack for the feature
+The feature stack always uses port `5173` (same as the main dev stack). Stop the main stack before starting a feature stack, and restart it when done.
+
+**Stop the main dev stack first:**
+```bash
+docker compose -f docker-compose.yml -f docker-compose.dev.yml down
+```
+
+**Create a per-feature override file** at `docker-compose.feat-<slug>.yml` (gitignored):
+```yaml
+# docker-compose.feat-<slug>.yml  — feature test stack, never committed to main
+services:
+  frontend:
+    container_name: frontend-<slug>
+  backend:
+    container_name: backend-<slug>
+  doc-service:
+    container_name: doc-service-<slug>
+  ai-service:
+    container_name: ai-service-<slug>
+  db:
+    container_name: db-<slug>
+
+networks:
+  backend-net:
+    name: backend-net-<slug>
+  frontend-net:
+    name: frontend-net-<slug>
+```
+
+**Start the feature stack**:
+```bash
+docker compose -f docker-compose.yml \
+               -f docker-compose.dev.yml \
+               -f docker-compose.feat-<slug>.yml \
+  --project-name <slug> up --build
+```
+
+The feature frontend is now reachable at `http://localhost:5173`.
+
+#### 3 — Develop on the feature branch
+All code changes happen on `feat/<slug>`. Commit and push normally:
+```bash
+git add <files>
+git commit -m "feat: <description>"
+git push -u origin feat/<slug>
+```
+
+#### 4 — Confirm functionality
+Before merging, verify all of the following on `http://localhost:5173`:
+- [ ] Login and registration work end-to-end
+- [ ] The specific feature works as intended
+- [ ] No regressions visible in the UI
+- [ ] Backend logs show no unexpected errors: `docker compose -p <slug> logs backend`
+- [ ] Migrations (if any) applied cleanly: `docker compose -p <slug> exec backend alembic upgrade head`
+
+#### 5 — Merge to main
+Once all checks pass:
+```bash
+git checkout main
+git merge --no-ff feat/<slug> -m "Merge feat/<slug>: <description>"
+git push
+git branch -d feat/<slug>
+git push origin --delete feat/<slug>
+```
+
+#### 6 — Tear down the feature stack and restart main dev stack
+```bash
+docker compose -f docker-compose.yml \
+               -f docker-compose.dev.yml \
+               -f docker-compose.feat-<slug>.yml \
+  --project-name <slug> down --volumes --remove-orphans
+rm docker-compose.feat-<slug>.yml
+
+# Restart the main dev stack on :5173
+docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d
+```
+
+---
+
+### Infrastructure change protocol
+
+After **any** change to Dockerfiles, `docker-compose*.yml`, `nginx.conf`, or setup scripts:
+
+1. **Update `README.md`** — containers table, ports, image names, Current State section.
+2. **Dev stack** — verify login and registration end-to-end:
+   ```bash
+   docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
+   ```
+3. **Prod stack** — run the same checks:
+   ```bash
+   docker compose up --build -d
+   ```
+4. Confirm non-root users:
+   ```bash
+   docker inspect <container> --format '{{.Config.User}}'
+   ```
+5. **Tear down** after testing:
+   ```bash
+   docker compose down --volumes --remove-orphans
+   ```
+
+---
+
+### Security hook
+
+`.githooks/pre-commit` (registered via `git config core.hooksPath .githooks`). Runs `scripts/security_check.py` in Docker. New clones must run:
+```bash
+git config core.hooksPath .githooks
+```
+
+See **Security Standards → Pre-commit security hook** for the full list of checks.
+
+**Never** bypass with `--no-verify`.

README.md

@@ -7,34 +7,51 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
 | Layer | Tech |
 |---|---|
 | Backend | FastAPI (async), SQLAlchemy 2, Alembic, PostgreSQL 16 |
-| Auth | JWT bearer tokens, bcrypt password hashing |
+| Auth | JWT bearer tokens (RS256), bcrypt password hashing |
 | Frontend | React 18, TypeScript, Vite, React Router v6, TanStack Query |
 
 ## Current State
 
-- User registration and login (JWT auth)
-- Protected dashboard with nav bar (Dashboard | Profile | Logout)
+- User registration and login (JWT RS256 auth, 8-hour expiry)
+- Protected dashboard with nav bar
 - `/api/users/me` — authenticated user info
 - `/api/profile/me` — GET/PUT personal profile (position, phone, date of birth, address)
-- Profile data stored in a dedicated `profiles` table; auto-created on first access
-- Admin role flag (`is_superuser`) stored in `users` table; exposed as `is_admin` in API (false for regular users, true for admins)
-- Admin-only user management at `/admin`: list all users, add users, delete users, toggle active status
+- Admin-only user management at `/admin`: list, add, delete, toggle active
 - All input sanitized before reaching the DB (null-byte rejection, length caps, format validation)
-- 3 separate Docker containers: `db` (PostgreSQL), `backend` (FastAPI), `frontend` (nginx)
-- All containers run as non-root users (UID 1001 for backend and frontend, UID 70 for db)
+- **PDF Documents app** (`/apps/documents`): upload PDFs, async text extraction (pdfplumber), AI classification via ai-service, per-user categories, file download
+- **AI Service** (`ai-service:8010`): shared AI intermediary container; routes prompts to Anthropic / Ollama / LM Studio; stateless; all feature containers talk to it via `POST /chat`
+- **Storage Service** (`storage-service:8020`): unified file/blob storage with pluggable backends (local filesystem default; S3-compatible and WebDAV built in); backend switchable via admin UI with zero-data-loss migration
+- Admin settings: AI provider, doc upload limits, storage backend switching with live migration progress
+- Config stored in storage-service (`config` bucket); PDFs stored in storage-service (`documents` bucket) — no shared filesystem volumes
+- `/apps` launcher hub — one card per installed app with Open + Settings links
+- 6 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `storage-service`, `frontend`
+- All containers run as non-root users (UID 1001 for app containers, UID 70 for db)
+- Network-isolated: only the frontend exposes a host port (80/5173); all backend services are unreachable from outside Docker
 - Dev environment seeds a test user automatically on startup (`test@example.com` / `Test123!`)
 - Password policy: min 8 chars, upper + lowercase, digit, special character, no common words
 - Pre-commit security hook (`scripts/security_check.py`) runs inside Docker on every commit
 
 ## Containers
 
-| Container | Image | Port | User (UID:GID) | Description |
-|---|---|---|---|---|
-| `db` | postgres:16-alpine | 5432 | 70:70 (postgres) | PostgreSQL database |
-| `backend` | custom (python:3.12-slim) | 8000 | 1001:1001 (appuser) | FastAPI management API |
-| `frontend` | custom (nginxinc/nginx-unprivileged:alpine) | 80 | 1001:1001 (appuser) | React UI served by nginx (internal port 8080) |
+| Container | Image | Host port | Network | User (UID:GID) | Description |
+|---|---|---|---|---|---|
+| `db` | postgres:16-alpine | none | backend-net | 70:70 | PostgreSQL database |
+| `backend` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | FastAPI management API + proxy to doc-service |
+| `ai-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Shared AI intermediary (routes to LM Studio / Ollama / Anthropic) |
+| `doc-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | PDF extraction microservice (calls ai-service) |
+| `storage-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Unified file/blob storage (local / S3-compatible / WebDAV) |
+| `frontend` | custom (nginxinc/nginx-unprivileged:alpine) | 80 (prod) / 5173 (dev) | backend-net + frontend-net | 1001:1001 | React UI + nginx reverse proxy |
 
-The frontend nginx container proxies `/api/*` to the backend container internally — no CORS headers needed in production.
+**Networks:**
+- `backend-net` — all backend services; no host ports bound; outbound internet access allowed (needed for cloud AI API calls)
+- `frontend-net` — frontend only; this is where the single host port (80/5173) is bound
+
+**Volumes:**
+- `postgres_data` — PostgreSQL data files
+- `storage_data` — all file/blob storage: uploaded PDFs (`documents/` bucket) and service config JSON files (`config/` bucket); mounted into storage-service at `/data/storage`
+- `watch_data` — file watcher input directory; mounted into doc-service at `/data/watch`
+
+The frontend nginx proxies `/api/*` to `backend:8000` via `backend-net`. The backend proxies `/api/documents/*` and `/api/documents/categories/*` to `doc-service:8001`. The backend test-connection endpoint proxies to `ai-service:8010`. No backend service or database port is ever exposed to the host.
 
 ## Installation
 
@@ -47,12 +64,15 @@ The frontend nginx container proxies `/api/*` to the backend container internall
 ```bash
 git clone <repo>
 cd destroying_sap
-cp .env.example backend/.env   # edit SECRET_KEY at minimum
+cp .env.example backend/.env
+python scripts/generate_jwt_keys.py  # paste output into backend/.env
 docker compose up --build -d
 ```
 
 - Frontend: http://localhost
-- API docs: http://localhost:8000/docs
+- API docs: not directly accessible from host (backend port not exposed)
+
+After first start, configure the AI provider at `/apps/documents/settings/admin` (admin login required).
 
 ### Development (hot reload)
 
@@ -61,7 +81,7 @@ docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
 ```
 
 - Frontend (Vite): http://localhost:5173
-- Backend (uvicorn --reload): http://localhost:8000
+- Backend: reachable by frontend via Docker network only (not exposed to host)
 
 ### Local (no Docker)
 
@@ -75,14 +95,24 @@ docker compose up db -d
 
 ```bash
 cd backend
-python -m venv .venv && source .venv/bin/activate   # Windows: .venv\Scripts\activate
+python -m venv .venv && source .venv/bin/activate
 pip install -e ".[dev]"
 cp ../.env.example .env
 alembic upgrade head
 uvicorn app.main:app --reload
 ```
 
-**3. Frontend**
+**3. doc-service**
+
+```bash
+cd features/doc-service
+python -m venv .venv && source .venv/bin/activate
+pip install -e .
+alembic upgrade head
+uvicorn app.main:app --host 0.0.0.0 --port 8001 --reload
+```
+
+**4. Frontend**
 
 ```bash
 cd frontend && npm install && npm run dev
@@ -95,8 +125,11 @@ Copy `.env.example` to `backend/.env` and adjust:
 | Variable | Default | Description |
 |---|---|---|
 | `DATABASE_URL` | `postgresql+asyncpg://postgres:password@localhost:5432/destroying_sap` | Async PostgreSQL URL |
-| `SECRET_KEY` | `change-me-in-production` | JWT signing key |
+| `JWT_PRIVATE_KEY` | — | RS256 private key PEM (generate with `scripts/generate_jwt_keys.py`) |
+| `JWT_PUBLIC_KEY` | — | RS256 public key PEM (generate with `scripts/generate_jwt_keys.py`) |
 | `CORS_ORIGINS` | `["http://localhost:5173"]` | Allowed frontend origins |
+| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of doc-service (set by docker-compose) |
+| `STORAGE_SERVICE_URL` | `http://storage-service:8020` | Internal URL of storage-service (set by docker-compose) |
 
 ## Development
 
@@ -110,7 +143,11 @@ cd backend && pytest
 # Frontend type check + lint
 cd frontend && npm run typecheck && npm run lint
 
-# New DB migration (after changing models)
+# New DB migration — main backend
 cd backend && alembic revision --autogenerate -m "describe change"
 cd backend && alembic upgrade head
+
+# New DB migration — doc-service
+cd features/doc-service && alembic revision --autogenerate -m "describe change"
+cd features/doc-service && alembic upgrade head
 ```

TODO.md

@@ -1,12 +1,12 @@
 # TODO
 
-## UX/UI — Penpot setup
+## UX/UI — Figma setup
 
-- [ ] **Spin up Penpot LXC** — separate LXC container on the server (~2–4 GB RAM), Docker Compose from https://github.com/penpot/penpot; expose via subdomain behind nginx proxy manager
-- [ ] **Create Penpot project** — register on the self-hosted instance, create project `destroying_sap`, create initial design file
-- [ ] **Generate Penpot access token** — Profile → Access tokens; used by the `ux-designer` agent via WebFetch REST API calls
-- [ ] **Decide on UI component library** — shadcn/ui (recommended: Tailwind-based, unstyled accessible primitives, white-label friendly) vs MUI vs other; decision affects both Penpot design system and frontend implementation
-- [ ] **Connect ux-designer agent** — confirm Penpot API reachable, provide instance URL + token to agent at session start
+- [x] **Create a Figma account** — signed up at https://figma.com
+- [x] **Create Figma project and file** — file key `kcmvLytS31lSjP44YpBUSn` confirmed active
+- [x] **Generate Figma personal access token** — generated and verified (HTTP 200); provide a fresh token at each session start
+- [x] **Connect ux-designer agent** — agent updated to use `Bash`/`curl` with `X-Figma-Token` header; connection confirmed working
+- [ ] **Decide on UI component library** — shadcn/ui (recommended: Tailwind-based, unstyled accessible primitives, white-label friendly) vs MUI vs other; decision affects both Figma design system and frontend implementation
 
 ## Auth / session security
 
@@ -19,15 +19,37 @@
 
 - [ ] **Permissions registry** — admin-managed table that controls which apps each user can access. Schema: `user_app_permissions (user_id FK, app_key)`. Admin UI lets the admin grant/revoke per-app access per user. The Apps page only shows apps the current user has been granted access to.
 
+## PDF Documents app (`features/doc-service`)
+
+- [x] **doc-service container** — FastAPI microservice on `backend-net`; never exposed to host or frontend directly
+- [x] **PDF upload + async extraction** — background task with pdfplumber + pluggable AI (Anthropic / Ollama / LM Studio)
+- [x] **Per-app settings page** — `/apps/documents/settings/admin`; AI provider config, max file size; admin only
+- [x] **Per-user categories** — create/rename/delete categories; assign multiple categories per document
+- [x] **Alembic isolation** — `alembic_version_doc_service` version table; no collision with main backend migrations
+- [x] **Runtime config file** — `/config/doc_service_config.json` on shared Docker volume; editable from frontend; 30s TTL cache in doc-service
+- [ ] **Re-process document** — UI button to re-trigger AI extraction on an existing document (after changing AI provider/model)
+- [ ] **Bulk category operations** — assign/remove a category from multiple documents at once
+- [ ] **Search / filter documents** — filter by status, document type, category, date range
+
 ## Frontend features
 
 - [x] **Logout button** — visible when logged in, clears token and redirects to `/login`
 - [x] **Profile page** (`/profile`) — shows personal information for the logged-in user
 - [x] **Edit & save profile** — form to update personal details, stored in a dedicated `profiles` table (separate from `users`, same PostgreSQL container)
 
+## App container architecture (future)
+
+Design decision: each installable app (billing, PDF, email, etc.) runs in its own isolated Docker/Podman container, spawned and managed by the backend via the Docker API. Key rules to implement:
+
+- [ ] **Docker socket proxy** — backend must never mount `/var/run/docker.sock` directly; use `tecnativa/docker-socket-proxy` on an internal-only network, with only the required API endpoints whitelisted (CONTAINERS, IMAGES, NETWORKS, POST). Raw socket access = root on the host.
+- [ ] **Network isolation per app** — each spawned app container gets its own Docker bridge network; app containers never talk to each other directly; only the backend can reach them
+- [ ] **No privileged app containers** — all spawned containers run without `--privileged`, without extra capabilities, with resource limits (CPU, memory)
+- [ ] **Image allowlist** — backend may only spawn containers from a pre-approved image list; never pull or build arbitrary images at runtime
+- [ ] **Consider Podman** — evaluate rootless Podman as replacement for Docker daemon; daemonless model eliminates the socket entirely; Docker SDK compatible
+
 ## Infrastructure
 
-- [ ] **Docker port hardening** — expose only port 80 externally; backend (8000) and db (5432) must not be reachable from outside the Docker network. Prepare for deployment behind Traefik or nginx proxy manager (SSL termination, reverse proxy, no direct container exposure).
+- [x] **Docker port hardening** — only port 80 (prod) / 5173 (dev) exposed on the host via `frontend-net`; backend and db have no host port bindings and sit on `internal: true` `backend-net`
 
 ## Infrastructure (existing)
 

backend/CLAUDE.md

@@ -0,0 +1,365 @@
+# backend — Claude context
+
+FastAPI async gateway, port 8000 (internal). Handles auth, user/group management, settings, and proxies document/category requests to `doc-service:8001`. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
+
+---
+
+## Commands
+
+All commands run inside Docker — never on the host.
+
+### Migrations
+
+```bash
+docker compose exec backend alembic revision --autogenerate -m "describe change"
+docker compose exec backend alembic upgrade head
+docker compose exec backend alembic downgrade -1
+```
+
+### Lint
+
+```bash
+docker compose exec backend ruff check . && ruff format .
+```
+
+---
+
+## File & Folder Tree
+
+```
+backend/
+├── app/
+│   ├── main.py                        ← App factory, router registration, lifespan (health loop)
+│   ├── database.py                    ← AsyncEngine, AsyncSessionLocal, Base
+│   ├── deps.py                        ← get_current_user, get_current_admin, get_service_admin(id), check_plugin_access (also get_user_groups in doc-service)
+│   ├── core/
+│   │   ├── config.py                  ← All settings via pydantic-settings (reads .env)
+│   │   ├── security.py                ← JWT sign/verify (RS256), bcrypt hash/verify
+│   │   ├── sanitize.py                ← Input sanitization helpers (see Security Standards)
+│   │   ├── app_config.py              ← Per-service config load/save via storage-service; theme files in config/themes/{id}.json
+│   │   └── config_storage.py          ← Thin async HTTP helpers: read_json/write_json/delete_key/list_keys → storage-service config bucket
+│   ├── models/
+│   │   ├── __init__.py                ← Imports all models (required for Alembic autogenerate)
+│   │   ├── user.py                    ← User model
+│   │   ├── profile.py                 ← Profile model
+│   │   └── group.py                   ← Group, GroupMembership models
+│   ├── schemas/
+│   │   ├── user.py                    ← UserCreate/Out, Token, DashboardPrefsOut/Update
+│   │   ├── profile.py                 ← ProfileRead, ProfileUpdate
+│   │   └── group.py                   ← GroupCreate/Update/Out/DetailOut, GroupMemberOut
+│   ├── routers/
+│   │   ├── auth.py                    ← POST /register, POST /login
+│   │   ├── users.py                   ← GET /me, GET+PATCH /me/preferences, PATCH /me/color-mode, GET /me/groups
+│   │   ├── profile.py                 ← GET+PUT /me (profile)
+│   │   ├── admin.py                   ← User admin CRUD (admin-only)
+│   │   ├── groups.py                  ← Group CRUD + member management (admin-only)
+│   │   ├── settings.py                ← AI, doc limits, system prompts, appearance, themes (admin-only)
+│   │   ├── services.py                ← GET /services (health status)
+│   │   ├── plugins.py                 ← Generic plugin proxy (GET/PATCH /api/plugins/*)
+│   │   ├── categories_proxy.py        ← Transparent proxy → doc-service /categories/*
+│   │   ├── documents_proxy.py         ← Transparent proxy → doc-service /documents/*
+│   │   └── storage_config.py          ← Admin proxy → storage-service config + migration endpoints
+│   └── services/
+│       ├── service_health.py          ← Background 30s health-check loop; caches /plugin/manifest per service
+│       └── group_bootstrap.py         ← Ensures {service-id}-admin group exists for every registered service at startup
+├── alembic/
+│   ├── env.py                         ← Async migration runner
+│   └── versions/                      ← Migration chain (see Database Models)
+├── scripts/seed.py                    ← Seed test user
+├── Dockerfile                         ← python:3.12-slim, non-root user 1001
+└── STATUS.md
+```
+
+---
+
+## Database Models
+
+### `users`
+
+| Column | Type | Constraints | Notes |
+|--------|------|-------------|-------|
+| `id` | String | PK, UUID | auto-generated |
+| `email` | String | UNIQUE, indexed, NOT NULL | lowercased before storing |
+| `hashed_password` | String | NOT NULL | bcrypt 13 rounds |
+| `full_name` | String | nullable | sanitized max 128 chars |
+| `is_active` | Boolean | default=True | soft-delete flag |
+| `is_superuser` | Boolean | default=False | admin role; never exposed as-is (serialised as `is_admin`) |
+| `dashboard_app_ids` | JSON | NOT NULL, default=[] | list of pinned service IDs |
+| `color_mode` | String | nullable, default=NULL | user's preferred mode: "light" / "dark" / "system" / NULL (use admin default) |
+
+Relationship: `profile` (one-to-one, cascade all+delete-orphan)
+
+### `profiles`
+
+| Column | Type | Constraints | Notes |
+|--------|------|-------------|-------|
+| `id` | String | PK, UUID | auto-generated |
+| `user_id` | String | FK→users.id UNIQUE, cascade delete | one-to-one |
+| `phone` | String(20) | nullable | validated format |
+| `date_of_birth` | Date | nullable | 1900+ and not future |
+| `position` | String(128) | nullable | job title |
+| `address` | String(255) | nullable | |
+| `updated_at` | DateTime(tz) | server_default=now(), onupdate=now() | |
+
+### `groups`
+
+| Column | Type | Constraints |
+|--------|------|-------------|
+| `id` | String | PK, UUID |
+| `name` | String(128) | UNIQUE indexed, NOT NULL |
+| `description` | String(512) | nullable |
+| `created_at` | DateTime(tz) | server_default=now() |
+
+### `group_memberships`
+
+| Column | Type | Constraints |
+|--------|------|-------------|
+| `id` | String | PK, UUID |
+| `group_id` | String | FK→groups.id, indexed, CASCADE |
+| `user_id` | String | FK→users.id, indexed, CASCADE |
+| `is_group_admin` | Boolean | NOT NULL, default=false | grants group-admin rights (manage group categories, delete shared docs) |
+| `joined_at` | DateTime(tz) | server_default=now() |
+
+Unique constraint: `(group_id, user_id)`
+
+### Migration chain (must be applied in order)
+
+| Rev ID | Slug |
+|--------|------|
+| `38efeff7c45a` | `create_users_table` |
+| `676084df61d1` | `add_profiles_table` |
+| `a3f9c2d14e87` | `add_groups_and_group_memberships` |
+| `c7e8f9a0b1d2` | `add_dashboard_app_ids_to_users` |
+| `dd6ad2f2c211` | `add_color_mode_to_users` |
+| `e1f2a3b4c5d6` | `add_group_member_is_admin` |
+
+---
+
+## API Endpoints
+
+### Auth (`/api/auth`) — public
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| POST | `/api/auth/register` | — | Create account; returns `UserOut`; enforces password policy |
+| POST | `/api/auth/login` | — | OAuth2 password flow; returns `{access_token, token_type}` |
+
+### Users (`/api/users`) — authenticated
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| GET | `/api/users/me` | user | Current user info → `UserOut` |
+| GET | `/api/users/me/preferences` | user | Dashboard pinned app IDs → `{app_ids}` |
+| PATCH | `/api/users/me/preferences` | user | Save pinned app IDs (max 50, slug-safe) |
+| PATCH | `/api/users/me/color-mode` | user | Save colour mode preference ("light"/"dark"/"system") |
+| GET | `/api/users/me/groups` | user | Groups current user belongs to → `list[UserGroupOut]` |
+
+### Profile (`/api/profile`) — authenticated
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| GET | `/api/profile/me` | user | Fetch profile; auto-creates if missing |
+| PUT | `/api/profile/me` | user | Update profile fields |
+
+### Admin — Users (`/api/admin`) — admin-only
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/admin/users` | List all users → `list[UserAdminOut]` |
+| POST | `/api/admin/users` | Create user (with optional is_admin) |
+| DELETE | `/api/admin/users/{user_id}` | Delete user (204) |
+| PATCH | `/api/admin/users/{user_id}/active` | Toggle active status |
+
+### Admin — Groups (`/api/admin/groups`) — admin-only
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/admin/groups` | List groups with member count |
+| POST | `/api/admin/groups` | Create group |
+| GET | `/api/admin/groups/{id}` | Group detail + members |
+| PATCH | `/api/admin/groups/{id}` | Update name / description |
+| DELETE | `/api/admin/groups/{id}` | Delete (cascades memberships) |
+| POST | `/api/admin/groups/{id}/members/{user_id}` | Add member |
+| DELETE | `/api/admin/groups/{id}/members/{user_id}` | Remove member |
+| PATCH | `/api/admin/groups/{id}/members/{user_id}/admin` | Set/unset group admin role (body: `{ is_group_admin: bool }`) |
+
+### Settings (`/api/settings`) — admin-only
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/settings/ai` | AI config (keys masked) |
+| PATCH | `/api/settings/ai` | Update AI provider / credentials |
+| POST | `/api/settings/ai/test` | Test AI connection |
+| GET | `/api/settings/documents/limits` | PDF upload limits |
+| PATCH | `/api/settings/documents/limits` | Update max PDF size |
+| GET | `/api/settings/system-prompts` | All editable system prompts |
+| PATCH | `/api/settings/system-prompts/{service_id}` | Update system prompt |
+| GET | `/api/settings/appearance` | Active theme + default mode (auth) |
+| PATCH | `/api/settings/appearance` | Update active theme + default mode (admin) |
+| GET | `/api/settings/themes` | List all themes — built-in + custom (auth) |
+| POST | `/api/settings/themes` | Create custom theme (admin) |
+| PATCH | `/api/settings/themes/{id}` | Update custom theme label/colours (admin) |
+| DELETE | `/api/settings/themes/{id}` | Delete custom theme (admin, 204) |
+
+### Services (`/api/services`) — authenticated
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/services` | Health status of all registered services → `list[ServiceStatus]` |
+
+### Plugins (`/api/plugins`) — authenticated, auth-per-plugin
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/plugins` | List plugins accessible to current user |
+| GET | `/api/plugins/{id}/manifest` | Plugin manifest with settings JSON Schema (auth-gated) |
+| GET | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
+| PATCH | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
+
+Auth: is_superuser OR member of group listed in manifest `required_groups`. Returns 404 (not 403) to hide existence.
+
+### Admin — Storage (`/api/admin`) — admin-only
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/api/admin/storage-config` | Current backend driver + health → proxied from storage-service `/health` |
+| PATCH | `/api/admin/storage-config` | Reconfigure backend without data migration (same-backend credential update) |
+| POST | `/api/admin/storage-config/migrate` | Start async migration to a new backend (copy → verify → switch → cleanup) |
+| GET | `/api/admin/storage-config/migrate/status` | Poll migration progress: `{state, total, done, failed, errors[]}` |
+| DELETE | `/api/admin/storage-config/migrate` | Cancel a running migration; old backend remains active |
+
+### Documents and Categories — proxied
+
+`/api/documents/*` and `/api/documents/categories/*` are transparently proxied to `doc-service:8001`. The backend injects `x-user-id`, `x-user-groups`, and `x-user-is-admin` headers. See `features/doc-service/CLAUDE.md` for the internal endpoint list.
+
+---
+
+## Security Standards
+
+These standards are **non-negotiable**. Every change must comply.
+
+### JWT
+
+- **Algorithm**: RS256 (4096-bit RSA key pair, generated by `scripts/generate_jwt_keys.py`)
+- **Keys**: PEM-encoded in `backend/.env` as `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` (gitignored)
+- **Expiry**: 8 hours (`EXPIRE_MINUTES=480`) — never set longer; no refresh tokens
+- **Claims**: `{sub: user_id, exp, iat}` — user_id is a UUID string
+- **Validation**: `decode_access_token()` in `core/security.py`; called by `get_current_user`
+- **Never**: set algorithm to `"none"`, disable `verify_exp`, or hardcode secrets in code
+
+### Password hashing
+
+- **Algorithm**: bcrypt, **13 rounds** (`bcrypt.gensalt(rounds=13)`)
+- **Timing**: ~300 ms per hash (intentional brute-force resistance)
+- **Never** use MD5, SHA1, or plain SHA256 for password storage
+
+### Password policy (enforced in `UserCreate` schema)
+
+All of the following must pass:
+- ≥ 8 characters
+- ≥ 1 uppercase (A–Z)
+- ≥ 1 lowercase (a–z)
+- ≥ 1 digit (0–9)
+- ≥ 1 special character: `!@#$%^&*()\-_=+[]{}|;:'"<>?/\`~`
+- No common words (password, secret, login, admin, test, qwerty, welcome, …)
+
+### Input sanitization
+
+Every user-supplied string stored in the database **must** pass through `core/sanitize.py`:
+
+```python
+sanitize_str(value, max_len=255)
+# → strips whitespace; rejects null bytes (\x00); rejects control chars
+#   (0x01–0x1F, 0x7F except \t \n \r); enforces max_len; returns None for ""
+
+normalize_email(value)       # lowercase + strip
+validate_phone(value)        # sanitize_str(max=20) + regex ^\+?[\d\s\-()\[\]]{7,20}$
+validate_date_of_birth(v)    # must be ≥ 1900, not future
+```
+
+Apply via Pydantic `@field_validator` on all request schemas.
+
+### SQL injection prevention
+
+- Use SQLAlchemy ORM (bound parameters) — **never** raw SQL strings.
+- If `text()` is needed, use `bindparam()` for all user-supplied values.
+- **Never** use f-strings, `.format()`, or `%`-formatting for SQL.
+
+### Admin route security
+
+- Use `get_current_admin` dependency (checks `is_superuser`).
+- Return **404** (not 403) for unauthorized access — hides both endpoint existence and permission model.
+
+---
+
+## Naming & Code Conventions
+
+### Database
+
+- **Tables**: lowercase, plural, snake_case (`users`, `group_memberships`, `document_category_assignments`)
+- **Columns**: lowercase, snake_case
+- **ORM models**: PascalCase, singular (`User`, `Group`, `GroupMembership`, `Document`)
+- Primary keys: `id` (String UUID, auto-generated)
+- Timestamps: `created_at` / `updated_at` / `joined_at` / `processed_at` — always timezone-aware
+
+### Pydantic schemas
+
+| Suffix | Purpose |
+|--------|---------|
+| `Create` | POST request body (user-supplied input) |
+| `Update` | PATCH request body (partial update) |
+| `Out` | API response (safe subset of model) |
+| `AdminOut` | Extended response for admin endpoints |
+| `Read` | GET response (same as `Out`, used for profiles) |
+
+Always set `model_config = {"from_attributes": True}` on response schemas.
+Use `validation_alias` when the ORM field name differs from the JSON key (e.g., `is_superuser` → `is_admin`).
+
+### HTTP status codes
+
+| Code | Use |
+|------|-----|
+| 200 | Successful GET / PATCH / PUT |
+| 201 | Successful POST that creates a resource |
+| 202 | Accepted (async processing started, e.g., document upload) |
+| 204 | Successful DELETE or action with no response body |
+| 400 | Bad request (duplicates, invalid data beyond Pydantic) |
+| 401 | Missing / invalid JWT |
+| 404 | Not found **and** admin routes when not admin |
+| 413 | Payload too large (file exceeds limit) |
+| 415 | Unsupported media type (not a PDF) |
+| 422 | Pydantic validation failure (FastAPI default) |
+| 502 | Downstream service unreachable |
+| 503 | Service unavailable (queue stopped, AI error) |
+| 504 | Gateway timeout |
+
+### Backend code style
+
+- Async/await for **all** I/O (DB, HTTP, file).
+- `raise HTTPException(status_code=..., detail="...")` for all errors.
+- Response models always declared in route decorator: `@router.get("/path", response_model=XOut)`.
+- Background tasks via `BackgroundTasks` param; tasks open their own `AsyncSessionLocal` session.
+- Commit + refresh pattern after mutations:
+  ```python
+  await db.commit()
+  await db.refresh(obj)
+  ```
+
+---
+
+## Default Values & Limits
+
+| Parameter | Value | Location |
+|-----------|-------|----------|
+| JWT expiry | 480 min (8 h) | `core/security.py` |
+| Bcrypt rounds | 13 | `core/security.py` |
+| User `color_mode` default | NULL (falls back to admin default_mode, then system) | `models/user.py` |
+| Max dashboard pinned apps | 50 | `schemas/user.py` |
+| App ID max length | 64 chars | `schemas/user.py` |
+| App ID allowed chars | `[a-zA-Z0-9_\-]` | `schemas/user.py` |
+| full_name max length | 128 chars | `schemas/user.py` |
+| Group name max length | 128 chars | `schemas/group.py` |
+| Group description max | 512 chars | `schemas/group.py` |
+| Phone max length | 20 chars | `sanitize.py` |
+| Position max length | 128 chars | `schemas/profile.py` |
+| Address max length | 255 chars | `schemas/profile.py` |

backend/Dockerfile

@@ -24,9 +24,11 @@ COPY --from=builder /install /usr/local
 COPY --chown=appuser:appuser app ./app
 COPY --chown=appuser:appuser alembic ./alembic
 COPY --chown=appuser:appuser alembic.ini .
+COPY --chown=appuser:appuser scripts ./scripts
+RUN chmod +x scripts/start.sh
 
 USER appuser
 
 EXPOSE 8000
 
-CMD ["uvicorn", "app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+CMD ["sh", "scripts/start.sh"]

backend/STATUS.md

@@ -0,0 +1,183 @@
+# Backend — Status
+
+## What it is
+
+Central FastAPI gateway. Handles authentication, user management, admin settings, and proxies feature-service traffic. It is the only container that has host-level port exposure (`8000`, internal) — all browser traffic arrives via the Vite/nginx frontend proxy.
+
+Port: `8000` (on `backend-net`, no direct host binding in prod).
+Database: PostgreSQL 16 (`postgres_data` named volume).
+
+---
+
+## Current functionality
+
+### Auth (`/api/auth`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/api/auth/register` | Create account; password policy enforced (uppercase, special char, no "test") |
+| `POST` | `/api/auth/login` | OAuth2 password flow; returns RS256 JWT (8-hour expiry) |
+
+JWT signing uses a 4096-bit RSA key pair (`RS256`). Keys are generated by `scripts/generate_jwt_keys.py` and stored in `backend/.env` (gitignored). Token stored in `localStorage` on the client.
+
+### Users (`/api/users`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/users/me` | Current user info |
+| `GET` | `/api/users/me/preferences` | User's dashboard preferences (`app_ids` list) |
+| `PATCH` | `/api/users/me/preferences` | Update pinned app IDs (max 50; validated as safe slugs) |
+| `GET` | `/api/users/me/groups` | List groups the current user belongs to (for share picker) |
+
+### Profile (`/api/profile`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/profile` | Fetch profile (separate `profiles` table) |
+| `PUT` | `/api/profile` | Update profile fields |
+
+### Admin (`/api/admin`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/admin/users` | List all users (admin only) |
+| `PATCH` | `/api/admin/users/{id}` | Update user (role, active flag) |
+
+### Groups (`/api/admin/groups`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/admin/groups` | List all groups with member count |
+| `POST` | `/api/admin/groups` | Create a new group |
+| `GET` | `/api/admin/groups/{id}` | Get group detail with member list |
+| `PATCH` | `/api/admin/groups/{id}` | Update group name / description |
+| `DELETE` | `/api/admin/groups/{id}` | Delete group (cascades memberships) |
+| `POST` | `/api/admin/groups/{id}/members/{user_id}` | Add user to group |
+| `DELETE` | `/api/admin/groups/{id}/members/{user_id}` | Remove user from group |
+
+### Services (`/api/services`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/services` | Returns health status of all registered feature services |
+
+A background task (`service_health.py`) polls each service's `/health` endpoint every 30 s and stores the result in memory. The first check runs immediately on startup. Any authenticated user may call `GET /api/services`; the frontend uses it to drive app card visibility.
+
+### Settings (`/api/settings`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/settings/ai` | AI service config (masked) — superuser OR `ai-service-admin` member |
+| `PATCH` | `/api/settings/ai` | Update AI provider / credentials — same access |
+| `POST` | `/api/settings/ai/test` | Test AI connection — same access |
+| `GET` | `/api/settings/documents/limits` | Doc service upload limits — superuser OR `doc-service-admin` member |
+| `PATCH` | `/api/settings/documents/limits` | Update max PDF size — same access |
+| `GET` | `/api/settings/system-prompts` | All editable system prompts — superuser OR `ai-service-admin` member |
+| `PATCH` | `/api/settings/system-prompts/{id}` | Update system prompt — same access |
+
+Settings are persisted to the `config` bucket of `storage-service:8020` via `core/config_storage.py`. All config I/O is async HTTP; no filesystem volumes are used.
+
+Access to service-specific settings endpoints is enforced by `get_service_admin(service_id)` in `deps.py` — grants access to superusers OR members of the `{service_id}-admin` group.
+
+### Storage config (`/api/admin`)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/api/admin/storage-config` | Current backend driver + health (proxied from storage-service) |
+| `PATCH` | `/api/admin/storage-config` | Reconfigure backend without migration |
+| `POST` | `/api/admin/storage-config/migrate` | Start async migration to a new backend |
+| `GET` | `/api/admin/storage-config/migrate/status` | Poll migration progress |
+| `DELETE` | `/api/admin/storage-config/migrate` | Cancel running migration |
+
+### Feature proxies
+
+All `/api/documents/*` and `/api/documents/categories/*` requests are transparently proxied to `doc-service:8001` via `httpx.AsyncClient`. The proxy:
+- Validates the JWT (`get_current_user`)
+- Injects `x-user-id` header (UUID from `users.id`)
+- Strips hop-by-hop headers + `content-length`, `accept-encoding`, `content-type`
+- Returns `Response` (not `StreamingResponse`) to avoid content-length/chunked conflicts
+
+### Plugin system (`/api/plugins`)
+
+Generic extension/plugin infrastructure — **zero feature-specific code in backend**. Feature containers self-describe via `GET /plugin/manifest`.
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/api/plugins` | user | List plugins accessible to current user |
+| `GET` | `/api/plugins/{id}/manifest` | user | Cached manifest for a plugin (404 if not accessible) |
+| `GET` | `/api/plugins/{id}/settings` | user | Proxy to feature `GET /plugin/settings` |
+| `PATCH` | `/api/plugins/{id}/settings` | user | Proxy to feature `PATCH /plugin/settings` |
+
+Access is controlled by the manifest: `allow_superuser` for admins; `required_groups` for group members. `check_plugin_access(plugin_id, user, db)` in `deps.py` enforces this.
+
+During each health poll, `service_health.py` also fetches `GET /plugin/manifest` from healthy services and caches it. New feature containers that expose `/plugin/manifest` automatically appear in the plugin list — no backend code changes required.
+
+**Service admin group bootstrap:** On every startup, `group_bootstrap.py` creates a `{service-id}-admin` group for every registered service (idempotent). Admins add users to these groups via the Admin → Groups UI to delegate service-level administration.
+
+### Database models
+
+| Model | Table | Notes |
+|-------|-------|-------|
+| `User` | `users` | email, hashed_password, role (`user`\|`admin`), is_active, dashboard_app_ids (JSON) |
+| `Profile` | `profiles` | one-to-one with User; full_name, phone, etc. |
+| `Group` | `groups` | name (unique), description, created_at |
+| `GroupMembership` | `group_memberships` | group_id + user_id (unique pair); joined_at |
+
+Alembic migrations in `backend/alembic/versions/` — version table: `alembic_version`.
+
+---
+
+## Architecture
+
+```
+Browser (port 5173 dev / 80 prod)
+    │
+    └── Vite dev proxy / nginx
+            │
+            └── /api/*  →  backend:8000  (FastAPI)
+                                │
+                    ┌───────────┼────────────┬──────────────┐
+                 /auth       /settings    /documents/*   /services
+                 /users       (JSON        │              │
+                 /admin       /storage-    └── proxy →   health-check loop
+                 /profile      config       doc-service:8001  (30s poll)
+                             (proxy)
+                                  │
+                             storage-service:8020
+```
+
+---
+
+## Security notes
+
+- JWT stored in `localStorage` — XSS risk. Migration to `httpOnly` cookie planned.
+- No refresh token — after 8h the user must log in again.
+- Admin routes use `get_current_admin` dependency (checks `role == "admin"`).
+- All backend routes require authentication except `/api/auth/*`.
+- `backend-net` is marked `internal: true` — containers on it cannot reach the internet directly.
+
+---
+
+## Known limitations / not implemented
+
+- **No refresh tokens** — 8h hard expiry; adding refresh requires `httpOnly` cookie + rotation
+- **No `httpOnly` cookie** — JWT in `localStorage` is XSS-exposed
+- **App permissions** — no per-user, per-app access control. Currently all authenticated users can use all apps. Planned: `user_app_permissions` table, admin UI to grant/revoke
+- **Groups / sharing** — groups + memberships exist; app permission hooks not yet wired up
+- **Email verification** — accounts are active immediately after registration
+- **Password reset** — no flow implemented
+
+---
+
+## Future work
+
+- [x] Groups system: `groups`, `group_memberships` tables; admin CRUD; add/remove members
+- [x] Generic plugin infrastructure: manifest contract, `/api/plugins` proxy router, `check_plugin_access`
+- [ ] App permissions registry: `group_app_permissions` table; AppsPage filtered by group grants
+- [ ] Doc sharing via group membership
+- [ ] App permissions registry: `user_app_permissions (user_id, app_key)`; AppsPage filtered by grants
+- [ ] `httpOnly` cookie migration for JWT
+- [ ] Refresh token flow (paired with cookie migration)
+- [ ] Email verification on registration
+- [ ] Password reset flow
+- [ ] Rate limiting on auth endpoints

backend/alembic/versions/a3f9c2d14e87_add_groups_and_group_memberships.py

@@ -0,0 +1,51 @@
+"""add groups and group_memberships tables
+
+Revision ID: a3f9c2d14e87
+Revises: 676084df61d1
+Create Date: 2026-04-17 12:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+revision: str = 'a3f9c2d14e87'
+down_revision: Union[str, None] = '676084df61d1'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        'groups',
+        sa.Column('id', sa.String(), nullable=False),
+        sa.Column('name', sa.String(length=128), nullable=False),
+        sa.Column('description', sa.String(length=512), nullable=True),
+        sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
+        sa.PrimaryKeyConstraint('id'),
+    )
+    op.create_index(op.f('ix_groups_name'), 'groups', ['name'], unique=True)
+
+    op.create_table(
+        'group_memberships',
+        sa.Column('id', sa.String(), nullable=False),
+        sa.Column('group_id', sa.String(), nullable=False),
+        sa.Column('user_id', sa.String(), nullable=False),
+        sa.Column('joined_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
+        sa.ForeignKeyConstraint(['group_id'], ['groups.id'], ondelete='CASCADE'),
+        sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='CASCADE'),
+        sa.PrimaryKeyConstraint('id'),
+        sa.UniqueConstraint('group_id', 'user_id', name='uq_group_user'),
+    )
+    op.create_index(op.f('ix_group_memberships_group_id'), 'group_memberships', ['group_id'], unique=False)
+    op.create_index(op.f('ix_group_memberships_user_id'), 'group_memberships', ['user_id'], unique=False)
+
+
+def downgrade() -> None:
+    op.drop_index(op.f('ix_group_memberships_user_id'), table_name='group_memberships')
+    op.drop_index(op.f('ix_group_memberships_group_id'), table_name='group_memberships')
+    op.drop_table('group_memberships')
+    op.drop_index(op.f('ix_groups_name'), table_name='groups')
+    op.drop_table('groups')

backend/alembic/versions/c7e8f9a0b1d2_add_dashboard_app_ids_to_users.py

@@ -0,0 +1,28 @@
+"""add dashboard_app_ids to users
+
+Revision ID: c7e8f9a0b1d2
+Revises: a3f9c2d14e87
+Create Date: 2026-04-17 14:00:00.000000
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+
+revision: str = 'c7e8f9a0b1d2'
+down_revision: Union[str, None] = 'a3f9c2d14e87'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        'users',
+        sa.Column('dashboard_app_ids', sa.JSON(), nullable=False, server_default='[]'),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column('users', 'dashboard_app_ids')

backend/alembic/versions/dd6ad2f2c211_add_color_mode_to_users.py

@@ -0,0 +1,25 @@
+"""add_color_mode_to_users
+
+Revision ID: dd6ad2f2c211
+Revises: c7e8f9a0b1d2
+Create Date: 2026-04-17 23:42:58.222958
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+from sqlalchemy.dialects import postgresql
+
+revision: str = 'dd6ad2f2c211'
+down_revision: Union[str, None] = 'c7e8f9a0b1d2'
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column('users', sa.Column('color_mode', sa.String(), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column('users', 'color_mode')

backend/alembic/versions/e1f2a3b4c5d6_add_group_member_is_admin.py

@@ -0,0 +1,32 @@
+"""add is_group_admin to group_memberships
+
+Revision ID: e1f2a3b4c5d6
+Revises: dd6ad2f2c211
+Create Date: 2026-04-18
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "e1f2a3b4c5d6"
+down_revision: Union[str, None] = "dd6ad2f2c211"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "group_memberships",
+        sa.Column(
+            "is_group_admin",
+            sa.Boolean(),
+            nullable=False,
+            server_default=sa.text("false"),
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("group_memberships", "is_group_admin")

backend/app/core/app_config.py

@@ -0,0 +1,409 @@
+"""
+Per-service runtime config helpers.
+
+All config files are stored in the 'config' bucket of the storage-service.
+Every function is async — callers must await them.
+
+Key layout in the config bucket:
+  ai_service_config.json
+  doc_service_config.json
+  appearance_config.json
+  themes/{id}.json
+"""
+import copy
+import logging
+import re
+from copy import deepcopy
+
+from pydantic import BaseModel
+
+from app.core import config_storage
+
+logger = logging.getLogger(__name__)
+
+# ── AI service config schemas ──────────────────────────────────────────────────
+
+
+class AnthropicConfig(BaseModel):
+    api_key: str = ""
+    model: str = "claude-haiku-4-5-20251001"
+
+
+class OllamaConfig(BaseModel):
+    base_url: str = "http://host.docker.internal:11434/v1"
+    model: str = "llama3.2"
+    api_key: str = "ollama"
+
+
+class LMStudioConfig(BaseModel):
+    base_url: str = "http://host.docker.internal:1234/v1"
+    model: str = "local-model"
+    api_key: str = "lm-studio"
+
+
+class AIServiceConfig(BaseModel):
+    provider: str = "lmstudio"
+    timeout_seconds: int = 60
+    max_retries: int = 2
+    anthropic: AnthropicConfig = AnthropicConfig()
+    ollama: OllamaConfig = OllamaConfig()
+    lmstudio: LMStudioConfig = LMStudioConfig()
+
+
+# ── Doc service config schemas ─────────────────────────────────────────────────
+
+_DOC_SYSTEM_PROMPT_DEFAULT = (
+    "You are a financial document analysis assistant. "
+    "Given the text extracted from a PDF document, return ONLY a JSON object "
+    "with no markdown, no code fences, and no explanation."
+)
+
+_DOC_USER_TEMPLATE_DEFAULT = (
+    'Analyze the following document text and return a JSON object with exactly these keys:\n'
+    'title (a short, descriptive human-readable title for this document, e.g. "ACME Corp Invoice April 2026", "Office Supplies Receipt", "Q1 Flower Delivery Order"),\n'
+    'document_type (one of: invoice, bill, receipt, order, expense, revenue, unknown),\n'
+    'total_amount (string or null),\n'
+    'currency (string or null),\n'
+    'vendor_name (string or null),\n'
+    'customer_name (string or null),\n'
+    'billing_address (string or null),\n'
+    'customer_address (string or null),\n'
+    'invoice_number (string or null),\n'
+    'invoice_date (string or null),\n'
+    'due_date (string or null),\n'
+    'tags (array of short keyword strings describing the document),\n'
+    'line_items (array of objects, each with keys: description, amount),\n'
+    'suggested_categories (array of 2 to 5 short category name strings a user might want to file this document under, e.g. "Utilities", "Travel", "Software Subscriptions", "Client Invoices").\n'
+    '\n'
+    'Document text:\n'
+    '{text}'
+)
+
+
+class DocumentsConfig(BaseModel):
+    max_pdf_bytes: int = 20 * 1024 * 1024
+
+
+class DocServiceSystemPrompts(BaseModel):
+    system: str = _DOC_SYSTEM_PROMPT_DEFAULT
+    user_template: str = _DOC_USER_TEMPLATE_DEFAULT
+
+
+class DocServiceConfig(BaseModel):
+    documents: DocumentsConfig = DocumentsConfig()
+    system_prompts: DocServiceSystemPrompts = DocServiceSystemPrompts()
+
+
+# ── Masking ────────────────────────────────────────────────────────────────────
+
+def _mask_key(key: str) -> str:
+    if not key or len(key) <= 8:
+        return "••••"
+    return key[:7] + "••••"
+
+
+def _mask_ai_config(data: dict) -> dict:
+    masked = copy.deepcopy(data)
+    for provider in ("anthropic", "ollama", "lmstudio"):
+        if provider in masked and "api_key" in masked[provider]:
+            masked[provider]["api_key"] = _mask_key(masked[provider]["api_key"])
+    return masked
+
+
+# ── Load / Save ────────────────────────────────────────────────────────────────
+
+async def load_service_config(service: str) -> dict:
+    data = await config_storage.read_json(f"{service}_config.json")
+    if data is None:
+        if service == "ai_service":
+            return AIServiceConfig().model_dump()
+        if service == "doc_service":
+            return DocServiceConfig().model_dump()
+        return {}
+    return data
+
+
+async def save_service_config(service: str, data: dict) -> None:
+    await config_storage.write_json(f"{service}_config.json", data)
+
+
+# AI service helpers
+
+async def load_ai_service_config() -> AIServiceConfig:
+    raw = await load_service_config("ai_service")
+    return AIServiceConfig.model_validate(raw)
+
+
+async def save_ai_service_config(config: AIServiceConfig) -> None:
+    await save_service_config("ai_service", config.model_dump())
+
+
+async def load_ai_service_config_masked() -> dict:
+    raw = await load_service_config("ai_service")
+    return _mask_ai_config(raw)
+
+
+# Doc service helpers
+
+async def load_doc_service_config() -> DocServiceConfig:
+    raw = await load_service_config("doc_service")
+    return DocServiceConfig.model_validate(raw)
+
+
+async def save_doc_service_config(config: DocServiceConfig) -> None:
+    await save_service_config("doc_service", config.model_dump())
+
+
+async def load_doc_service_config_masked() -> dict:
+    return await load_service_config("doc_service")
+
+
+def _merge_api_key(new_key: str, existing_key: str) -> str:
+    """If new_key is empty or a masked value, keep the existing key."""
+    if not new_key or "••••" in new_key:
+        return existing_key
+    return new_key
+
+
+# ── System prompts helpers ─────────────────────────────────────────────────────
+
+SYSTEM_PROMPT_SERVICES: dict[str, str] = {
+    "doc_service": "Document Service",
+}
+
+
+async def load_all_system_prompts() -> dict:
+    """Return {service_id: {label, system, user_template, default_system, default_user_template}}."""
+    result: dict = {}
+    for service_id, label in SYSTEM_PROMPT_SERVICES.items():
+        config = await load_service_config(service_id)
+        prompts = config.get("system_prompts", {})
+        defaults = _get_service_prompt_defaults(service_id)
+        result[service_id] = {
+            "label": label,
+            "system": prompts.get("system", defaults["system"]),
+            "user_template": prompts.get("user_template", defaults["user_template"]),
+            "default_system": defaults["system"],
+            "default_user_template": defaults["user_template"],
+        }
+    return result
+
+
+async def save_service_system_prompts(service_id: str, system: str, user_template: str) -> None:
+    if service_id not in SYSTEM_PROMPT_SERVICES:
+        raise ValueError(f"Unknown service: {service_id!r}")
+    config = await load_service_config(service_id)
+    config.setdefault("system_prompts", {})
+    config["system_prompts"]["system"] = system
+    config["system_prompts"]["user_template"] = user_template
+    await save_service_config(service_id, config)
+
+
+def _get_service_prompt_defaults(service_id: str) -> dict:
+    if service_id == "doc_service":
+        d = DocServiceSystemPrompts()
+        return {"system": d.system, "user_template": d.user_template}
+    return {"system": "", "user_template": ""}
+
+
+# ── Appearance config ──────────────────────────────────────────────────────────
+
+class AppearanceConfig(BaseModel):
+    theme: str = "default"
+    default_mode: str = "system"
+
+
+async def load_appearance_config() -> AppearanceConfig:
+    data = await config_storage.read_json("appearance_config.json")
+    if data is None:
+        return AppearanceConfig()
+    return AppearanceConfig.model_validate(data)
+
+
+async def save_appearance_config(config: AppearanceConfig) -> None:
+    await config_storage.write_json("appearance_config.json", config.model_dump())
+
+
+# ── Theme file management ──────────────────────────────────────────────────────
+
+# 9 required colour tokens per mode
+_REQUIRED_TOKENS = frozenset({
+    "primary", "primary_hover", "accent", "accent_hover",
+    "background", "surface", "border", "text_primary", "text_muted",
+})
+
+_RGB_RE = re.compile(r"^\d{1,3} \d{1,3} \d{1,3}$")
+
+_BUILTIN_THEMES: list[dict] = [
+    {
+        "id": "default",
+        "label": "Default",
+        "builtin": True,
+        "light": {
+            "primary":       "37 99 235",
+            "primary_hover": "29 78 216",
+            "accent":        "234 179 8",
+            "accent_hover":  "202 138 4",
+            "background":    "248 250 252",
+            "surface":       "255 255 255",
+            "border":        "226 232 240",
+            "text_primary":  "15 23 42",
+            "text_muted":    "100 116 139",
+        },
+        "dark": {
+            "primary":       "59 130 246",
+            "primary_hover": "37 99 235",
+            "accent":        "250 204 21",
+            "accent_hover":  "234 179 8",
+            "background":    "15 23 42",
+            "surface":       "30 41 59",
+            "border":        "51 65 85",
+            "text_primary":  "203 213 225",
+            "text_muted":    "148 163 184",
+        },
+    },
+    {
+        "id": "pastel",
+        "label": "Pastel",
+        "builtin": True,
+        "light": {
+            "primary":       "124 58 237",
+            "primary_hover": "109 40 217",
+            "accent":        "236 72 153",
+            "accent_hover":  "219 39 119",
+            "background":    "253 244 255",
+            "surface":       "250 245 255",
+            "border":        "233 213 255",
+            "text_primary":  "30 27 75",
+            "text_muted":    "107 114 128",
+        },
+        "dark": {
+            "primary":       "167 139 250",
+            "primary_hover": "196 181 253",
+            "accent":        "244 114 182",
+            "accent_hover":  "251 164 200",
+            "background":    "30 20 51",
+            "surface":       "45 27 78",
+            "border":        "76 53 117",
+            "text_primary":  "233 213 255",
+            "text_muted":    "196 181 253",
+        },
+    },
+    {
+        "id": "high-contrast",
+        "label": "High Contrast",
+        "builtin": True,
+        "light": {
+            "primary":       "30 58 138",
+            "primary_hover": "30 64 175",
+            "accent":        "220 38 38",
+            "accent_hover":  "185 28 28",
+            "background":    "255 255 255",
+            "surface":       "255 255 255",
+            "border":        "156 163 175",
+            "text_primary":  "0 0 0",
+            "text_muted":    "75 85 99",
+        },
+        "dark": {
+            "primary":       "96 165 250",
+            "primary_hover": "147 197 253",
+            "accent":        "248 113 113",
+            "accent_hover":  "252 165 165",
+            "background":    "0 0 0",
+            "surface":       "10 10 10",
+            "border":        "55 65 81",
+            "text_primary":  "255 255 255",
+            "text_muted":    "156 163 175",
+        },
+    },
+    {
+        "id": "ocean",
+        "label": "Ocean Blue",
+        "builtin": True,
+        "light": {
+            "primary":       "29 78 216",
+            "primary_hover": "30 58 138",
+            "accent":        "8 145 178",
+            "accent_hover":  "14 116 144",
+            "background":    "239 246 255",
+            "surface":       "219 234 254",
+            "border":        "147 197 253",
+            "text_primary":  "30 58 138",
+            "text_muted":    "59 130 246",
+        },
+        "dark": {
+            "primary":       "96 165 250",
+            "primary_hover": "147 197 253",
+            "accent":        "34 211 238",
+            "accent_hover":  "103 232 249",
+            "background":    "10 22 40",
+            "surface":       "15 36 68",
+            "border":        "29 78 216",
+            "text_primary":  "219 234 254",
+            "text_muted":    "147 197 253",
+        },
+    },
+]
+
+
+async def seed_builtin_themes() -> None:
+    """Write built-in theme files to storage-service if they are not already there."""
+    existing_keys = await config_storage.list_keys(prefix="themes/")
+    existing_ids = {k.removeprefix("themes/").removesuffix(".json") for k in existing_keys}
+    for theme in _BUILTIN_THEMES:
+        if theme["id"] not in existing_ids:
+            await config_storage.write_json(f"themes/{theme['id']}.json", theme)
+    logger.info("Built-in themes seeded (%d themes)", len(_BUILTIN_THEMES))
+
+
+async def load_all_themes() -> list[dict]:
+    """Return all themes from storage-service, built-ins first then custom by label."""
+    keys = await config_storage.list_keys(prefix="themes/")
+    themes: list[dict] = []
+    for key in keys:
+        data = await config_storage.read_json(key)
+        if data:
+            themes.append(data)
+
+    builtin_ids = [t["id"] for t in _BUILTIN_THEMES]
+
+    def sort_key(t: dict) -> tuple:
+        tid = t.get("id", "")
+        try:
+            return (0, builtin_ids.index(tid))
+        except ValueError:
+            return (1, t.get("label", tid).lower())
+
+    return sorted(themes, key=sort_key)
+
+
+async def load_theme_by_id(theme_id: str) -> dict | None:
+    """Return a single theme dict, or None if not found."""
+    return await config_storage.read_json(f"themes/{theme_id}.json")
+
+
+async def save_theme(theme: dict) -> None:
+    """Write a theme to storage-service."""
+    await config_storage.write_json(f"themes/{theme['id']}.json", theme)
+
+
+async def delete_theme(theme_id: str) -> None:
+    """Delete a custom theme. Raises ValueError for built-ins, KeyError if not found."""
+    data = await config_storage.read_json(f"themes/{theme_id}.json")
+    if data is None:
+        raise FileNotFoundError(theme_id)
+    if data.get("builtin"):
+        raise ValueError("Cannot delete a built-in theme")
+    await config_storage.delete_key(f"themes/{theme_id}.json")
+
+
+def validate_theme_tokens(colors: dict) -> list[str]:
+    """Return a list of validation error messages, empty if valid."""
+    errors = []
+    missing = _REQUIRED_TOKENS - set(colors.keys())
+    if missing:
+        errors.append(f"Missing tokens: {', '.join(sorted(missing))}")
+    for key, val in colors.items():
+        if key in _REQUIRED_TOKENS and not _RGB_RE.match(str(val)):
+            errors.append(f"Token '{key}' must be an RGB triplet like '37 99 235', got: {val!r}")
+    return errors

backend/app/core/config.py

@@ -15,6 +15,10 @@ class Settings(BaseSettings):
 
     CORS_ORIGINS: list[str] = ["http://localhost:5173"]
 
+    DOC_SERVICE_URL: str = "http://doc-service:8001"
+    AI_SERVICE_URL: str = "http://ai-service:8010"
+    STORAGE_SERVICE_URL: str = "http://storage-service:8020"
+
     @field_validator("JWT_PRIVATE_KEY", "JWT_PUBLIC_KEY", mode="before")
     @classmethod
     def expand_newlines(cls, v: str) -> str:

backend/app/core/config_storage.py

@@ -0,0 +1,63 @@
+"""
+Async HTTP client for the 'config' bucket in storage-service.
+
+All JSON config files (AI settings, doc settings, appearance, themes, …) are stored
+in the 'config' bucket under the storage-service.  This module provides thin
+async helpers so app_config.py does not depend on the filesystem at all.
+"""
+import json
+import logging
+
+import httpx
+
+from app.core.config import settings
+
+logger = logging.getLogger(__name__)
+
+_BUCKET = "config"
+_TIMEOUT = 10.0
+
+
+def _url(key: str) -> str:
+    return f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}/{key}"
+
+
+async def read_json(key: str) -> dict | None:
+    """Return parsed JSON from the config bucket, or None if the key does not exist."""
+    async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
+        resp = await client.get(_url(key))
+        if resp.status_code == 404:
+            return None
+        resp.raise_for_status()
+        return resp.json()
+
+
+async def write_json(key: str, data: dict) -> None:
+    """Serialise *data* to JSON and PUT it into the config bucket."""
+    payload = json.dumps(data, indent=2).encode()
+    async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
+        resp = await client.put(
+            _url(key),
+            content=payload,
+            headers={"Content-Type": "application/octet-stream"},
+        )
+        resp.raise_for_status()
+
+
+async def delete_key(key: str) -> None:
+    """Delete a key from the config bucket.  No-op if it does not exist."""
+    async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
+        resp = await client.delete(_url(key))
+        if resp.status_code not in (204, 404):
+            resp.raise_for_status()
+
+
+async def list_keys(prefix: str = "") -> list[str]:
+    """List all keys in the config bucket, optionally filtered by *prefix*."""
+    async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
+        resp = await client.get(f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}")
+        resp.raise_for_status()
+        keys: list[str] = resp.json().get("keys", [])
+    if prefix:
+        keys = [k for k in keys if k.startswith(prefix)]
+    return keys

backend/app/core/security.py

@@ -6,8 +6,11 @@ from jose import jwt
 from app.core.config import settings
 
 
+_BCRYPT_ROUNDS = 13  # ~300 ms on modern hardware; increase over time as CPUs get faster
+
+
 def hash_password(password: str) -> str:
-    return bcrypt.hashpw(password.encode(), bcrypt.gensalt()).decode()
+    return bcrypt.hashpw(password.encode(), bcrypt.gensalt(rounds=_BCRYPT_ROUNDS)).decode()
 
 
 def verify_password(plain: str, hashed: str) -> bool:

backend/app/deps.py

@@ -43,3 +43,69 @@ async def get_current_admin(
             detail="Not found",
         )
     return current_user
+
+
+def get_service_admin(service_id: str):
+    """
+    Dependency factory that grants access to service-specific admin endpoints.
+
+    Access is granted if the user is a global superuser OR a member of the
+    '{service_id}-admin' group.  Returns 404 (not 403) to hide both the
+    endpoint existence and the permission model.
+
+    Usage:
+        @router.get("/ai")
+        async def get_ai_settings(_: User = Depends(get_service_admin("ai-service"))):
+    """
+    async def _dep(
+        current_user: User = Depends(get_current_user),
+        db: AsyncSession = Depends(get_db),
+    ) -> User:
+        if current_user.is_superuser:
+            return current_user
+        if await check_plugin_access(service_id, current_user, db):
+            return current_user
+        raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
+
+    return _dep
+
+
+async def check_plugin_access(
+    plugin_id: str,
+    current_user: User,
+    db: AsyncSession,
+) -> bool:
+    """
+    Return True if the user may access the given plugin's settings.
+
+    Access is granted when any of these conditions holds:
+      1. The user is a superuser AND the manifest allows superuser access.
+      2. The user is a member of one of the groups listed in manifest.access.required_groups.
+
+    Returns False (not raises) so callers can decide how to respond.
+    """
+    from app.models.group import Group, GroupMembership
+    from app.services.service_health import get_cached_manifest
+
+    manifest = get_cached_manifest(plugin_id)
+    if manifest is None:
+        return False
+
+    access = manifest.get("access", {})
+
+    if current_user.is_superuser and access.get("allow_superuser", True):
+        return True
+
+    for group_name in access.get("required_groups", []):
+        result = await db.execute(
+            select(GroupMembership)
+            .join(Group, Group.id == GroupMembership.group_id)
+            .where(
+                Group.name == group_name,
+                GroupMembership.user_id == current_user.id,
+            )
+        )
+        if result.scalar_one_or_none() is not None:
+            return True
+
+    return False

backend/app/main.py

@@ -1,10 +1,42 @@
+import asyncio
+from contextlib import asynccontextmanager
+
 from fastapi import FastAPI
 from fastapi.middleware.cors import CORSMiddleware
 
+from app.core.app_config import seed_builtin_themes
 from app.core.config import settings
-from app.routers import admin, auth, profile, users
+from app.database import AsyncSessionLocal
+from app.routers import admin, auth, categories_proxy, documents_proxy, groups, plugins, profile, services, users
+from app.routers import settings as settings_router
+from app.routers import storage_config
+from app.services.group_bootstrap import ensure_service_admin_groups
+from app.services.service_health import check_all, health_check_loop, register_services
 
-app = FastAPI(title=settings.PROJECT_NAME, version="0.1.0")
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    await seed_builtin_themes()
+    register_services(
+        doc_service_url=settings.DOC_SERVICE_URL,
+        ai_service_url=settings.AI_SERVICE_URL,
+        storage_service_url=settings.STORAGE_SERVICE_URL,
+    )
+    # Create <service-id>-admin groups for every registered service (idempotent)
+    async with AsyncSessionLocal() as db:
+        await ensure_service_admin_groups(db)
+    # Run an initial check immediately so the first API response is accurate
+    await check_all()
+    task = asyncio.create_task(health_check_loop())
+    yield
+    task.cancel()
+    try:
+        await task
+    except asyncio.CancelledError:
+        pass
+
+
+app = FastAPI(title=settings.PROJECT_NAME, version="0.1.0", lifespan=lifespan)
 
 app.add_middleware(
     CORSMiddleware,
@@ -18,6 +50,19 @@ app.include_router(auth.router, prefix="/api/auth", tags=["auth"])
 app.include_router(users.router, prefix="/api/users", tags=["users"])
 app.include_router(profile.router, prefix="/api/profile", tags=["profile"])
 app.include_router(admin.router, prefix="/api/admin", tags=["admin"])
+app.include_router(groups.router, prefix="/api/admin/groups", tags=["admin"])
+app.include_router(settings_router.router, prefix="/api/settings", tags=["settings"])
+app.include_router(services.router, prefix="/api/services", tags=["services"])
+app.include_router(storage_config.router, prefix="/api/admin", tags=["admin"])
+app.include_router(plugins.router, prefix="/api/plugins", tags=["plugins"])
+# categories_proxy MUST be registered before documents_proxy —
+# otherwise /api/documents/{path:path} swallows /api/documents/categories/*
+app.include_router(
+    categories_proxy.router,
+    prefix="/api/documents/categories",
+    tags=["categories"],
+)
+app.include_router(documents_proxy.router, prefix="/api/documents", tags=["documents"])
 
 
 @app.get("/api/health")

backend/app/models/__init__.py

@@ -1,4 +1,5 @@
+from app.models.group import Group, GroupMembership
 from app.models.profile import Profile
 from app.models.user import User
 
-__all__ = ["User", "Profile"]
+__all__ = ["User", "Profile", "Group", "GroupMembership"]

backend/app/models/group.py

@@ -0,0 +1,47 @@
+import uuid
+from datetime import datetime, timezone
+
+from sqlalchemy import Boolean, DateTime, ForeignKey, String, UniqueConstraint
+from sqlalchemy.orm import Mapped, mapped_column, relationship
+
+from app.database import Base
+
+
+class Group(Base):
+    __tablename__ = "groups"
+
+    id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    name: Mapped[str] = mapped_column(String(128), unique=True, nullable=False, index=True)
+    description: Mapped[str | None] = mapped_column(String(512), nullable=True)
+    created_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True),
+        default=lambda: datetime.now(timezone.utc),
+        nullable=False,
+    )
+
+    memberships: Mapped[list["GroupMembership"]] = relationship(
+        "GroupMembership", back_populates="group", cascade="all, delete-orphan"
+    )
+
+
+class GroupMembership(Base):
+    __tablename__ = "group_memberships"
+    __table_args__ = (UniqueConstraint("group_id", "user_id", name="uq_group_user"),)
+
+    id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
+    group_id: Mapped[str] = mapped_column(
+        String, ForeignKey("groups.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    user_id: Mapped[str] = mapped_column(
+        String, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
+    )
+    is_group_admin: Mapped[bool] = mapped_column(
+        Boolean, nullable=False, default=False, server_default="false"
+    )
+    joined_at: Mapped[datetime] = mapped_column(
+        DateTime(timezone=True),
+        default=lambda: datetime.now(timezone.utc),
+        nullable=False,
+    )
+
+    group: Mapped["Group"] = relationship("Group", back_populates="memberships")

backend/app/models/user.py

@@ -1,7 +1,7 @@
 import uuid
 from typing import TYPE_CHECKING
 
-from sqlalchemy import Boolean, String
+from sqlalchemy import Boolean, JSON, String
 from sqlalchemy.orm import Mapped, mapped_column, relationship
 
 from app.database import Base
@@ -21,6 +21,10 @@ class User(Base):
     # Role flag — True = admin, False = regular user.
     # Never exposed in API responses; set only by direct DB or admin tooling.
     is_superuser: Mapped[bool] = mapped_column(Boolean, default=False)
+    # List of service IDs pinned to the user's home dashboard.
+    dashboard_app_ids: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
+    # User's preferred colour mode: "light", "dark", "system", or None (use admin default).
+    color_mode: Mapped[str | None] = mapped_column(String, nullable=True, default=None)
 
     profile: Mapped["Profile"] = relationship(
         "Profile", back_populates="user", uselist=False, cascade="all, delete-orphan"

backend/app/routers/categories_proxy.py

@@ -0,0 +1,99 @@
+"""
+Proxy /api/documents/categories/* → doc-service:8001/categories/*.
+
+Must be registered BEFORE the documents catch-all proxy in main.py,
+otherwise /api/documents/{path:path} swallows category requests.
+"""
+import os
+
+import httpx
+from fastapi import APIRouter, Depends, HTTPException, Request
+from fastapi.responses import Response
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.deps import get_current_user
+from app.models.group import GroupMembership
+from app.models.user import User
+
+DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
+
+_client = httpx.AsyncClient(base_url=DOC_SERVICE_URL, timeout=30.0)
+
+router = APIRouter()
+
+_HOP_BY_HOP = frozenset([
+    "connection",
+    "keep-alive",
+    "proxy-authenticate",
+    "proxy-authorization",
+    "te",
+    "trailers",
+    "transfer-encoding",
+    "upgrade",
+    "host",
+    "accept-encoding",
+])
+
+_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
+
+
+async def _forward_headers(
+    request: Request, user_id: str, is_admin: bool, db: AsyncSession
+) -> dict:
+    headers = {
+        k: v
+        for k, v in request.headers.items()
+        if k.lower() not in _HOP_BY_HOP
+    }
+    headers["x-user-id"] = user_id
+    headers["x-user-is-admin"] = "true" if is_admin else "false"
+
+    mem_result = await db.execute(
+        select(GroupMembership.group_id, GroupMembership.is_group_admin)
+        .where(GroupMembership.user_id == user_id)
+    )
+    rows = mem_result.all()
+    group_ids = [row[0] for row in rows]
+    admin_group_ids = [row[0] for row in rows if row[1]]
+    headers["x-user-groups"] = ",".join(group_ids)
+    headers["x-user-admin-groups"] = ",".join(admin_group_ids)
+    return headers
+
+
+@router.api_route("", methods=["GET", "POST"])
+@router.api_route("/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
+async def proxy_categories(
+    request: Request,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+    path: str = "",
+) -> Response:
+    url = f"/categories/{path}" if path else "/categories"
+    headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
+    body = await request.body()
+
+    try:
+        response = await _client.request(
+            method=request.method,
+            url=url,
+            headers=headers,
+            content=body,
+            params=dict(request.query_params),
+        )
+    except httpx.RequestError as exc:
+        raise HTTPException(status_code=502, detail=f"doc-service unreachable: {exc}")
+
+    resp_headers = {
+        k: v
+        for k, v in response.headers.items()
+        if k.lower() not in _STRIP_RESPONSE
+    }
+
+    return Response(
+        content=response.content,
+        status_code=response.status_code,
+        headers=resp_headers,
+        media_type=response.headers.get("content-type"),
+    )

backend/app/routers/documents_proxy.py

@@ -0,0 +1,113 @@
+"""
+Proxy all /api/documents/* requests to doc-service:8001/documents/*.
+
+Uses a module-level AsyncClient for connection pooling.
+Strips hop-by-hop headers that must not be forwarded.
+Injects X-User-Id and X-User-Groups headers so the doc-service
+can enforce ownership and group-sharing access without querying the
+backend database directly.
+"""
+import os
+
+import httpx
+from fastapi import APIRouter, Depends, HTTPException, Request
+from fastapi.responses import Response
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.deps import get_current_user
+from app.models.group import GroupMembership
+from app.models.user import User
+
+DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
+
+_client = httpx.AsyncClient(base_url=DOC_SERVICE_URL, timeout=120.0)
+
+router = APIRouter()
+
+# Headers that must not be forwarded in either direction.
+# Also strip accept-encoding so doc-service never compresses responses —
+# httpx decompresses transparently but the content-encoding header would
+# then mismatch the already-decompressed body we forward to the browser.
+_HOP_BY_HOP = frozenset([
+    "connection",
+    "keep-alive",
+    "proxy-authenticate",
+    "proxy-authorization",
+    "te",
+    "trailers",
+    "transfer-encoding",
+    "upgrade",
+    "host",
+    "accept-encoding",
+])
+
+# Additional response headers we let FastAPI recalculate rather than forward.
+# content-length is set automatically from the response body size.
+# content-type is set via the media_type argument, so strip it from headers
+# to avoid duplicates.
+_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
+
+
+async def _forward_headers(
+    request: Request, user_id: str, is_admin: bool, db: AsyncSession
+) -> dict:
+    headers = {
+        k: v
+        for k, v in request.headers.items()
+        if k.lower() not in _HOP_BY_HOP
+    }
+    headers["x-user-id"] = user_id
+    headers["x-user-is-admin"] = "true" if is_admin else "false"
+
+    # Inject group memberships and group-admin status so the doc-service can
+    # evaluate ownership, sharing access, and group-admin permissions.
+    mem_result = await db.execute(
+        select(GroupMembership.group_id, GroupMembership.is_group_admin)
+        .where(GroupMembership.user_id == user_id)
+    )
+    rows = mem_result.all()
+    group_ids = [row[0] for row in rows]
+    admin_group_ids = [row[0] for row in rows if row[1]]
+    headers["x-user-groups"] = ",".join(group_ids)
+    headers["x-user-admin-groups"] = ",".join(admin_group_ids)
+
+    return headers
+
+
+@router.api_route("", methods=["GET", "POST"])
+@router.api_route("/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
+async def proxy_documents(
+    request: Request,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+    path: str = "",
+) -> Response:
+    url = f"/documents/{path}" if path else "/documents"
+    headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
+    body = await request.body()
+
+    try:
+        response = await _client.request(
+            method=request.method,
+            url=url,
+            headers=headers,
+            content=body,
+            params=dict(request.query_params),
+        )
+    except httpx.RequestError as exc:
+        raise HTTPException(status_code=502, detail=f"doc-service unreachable: {exc}")
+
+    resp_headers = {
+        k: v
+        for k, v in response.headers.items()
+        if k.lower() not in _STRIP_RESPONSE
+    }
+
+    return Response(
+        content=response.content,
+        status_code=response.status_code,
+        headers=resp_headers,
+        media_type=response.headers.get("content-type"),
+    )

backend/app/routers/groups.py

@@ -0,0 +1,237 @@
+from fastapi import APIRouter, Depends, HTTPException, status
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.orm import selectinload
+
+from app.database import get_db
+from app.deps import get_current_admin
+from app.models.group import Group, GroupMembership
+from app.models.user import User
+from app.schemas.group import GroupCreate, GroupDetailOut, GroupMemberAdminUpdate, GroupMemberOut, GroupOut, GroupUpdate
+
+router = APIRouter()
+
+
+def _to_group_out(group: Group) -> GroupOut:
+    return GroupOut(
+        id=group.id,
+        name=group.name,
+        description=group.description,
+        created_at=group.created_at,
+        member_count=len(group.memberships),
+    )
+
+
+def _to_group_detail(group: Group) -> GroupDetailOut:
+    members = []
+    for m in group.memberships:
+        # memberships are loaded with joined user via selectinload
+        user = m.__dict__.get("_user") or getattr(m, "user", None)
+        if user is None:
+            continue
+        members.append(GroupMemberOut(
+            id=user.id,
+            email=user.email,
+            full_name=user.full_name,
+            is_active=user.is_active,
+            joined_at=m.joined_at,
+        ))
+    return GroupDetailOut(
+        id=group.id,
+        name=group.name,
+        description=group.description,
+        created_at=group.created_at,
+        member_count=len(members),
+        members=members,
+    )
+
+
+@router.get("", response_model=list[GroupOut])
+async def list_groups(
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> list[GroupOut]:
+    result = await db.execute(
+        select(Group).options(selectinload(Group.memberships)).order_by(Group.name)
+    )
+    groups = list(result.scalars().all())
+    return [_to_group_out(g) for g in groups]
+
+
+@router.post("", response_model=GroupOut, status_code=status.HTTP_201_CREATED)
+async def create_group(
+    body: GroupCreate,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> GroupOut:
+    existing = await db.execute(select(Group).where(Group.name == body.name))
+    if existing.scalar_one_or_none():
+        raise HTTPException(status_code=400, detail="A group with that name already exists")
+
+    group = Group(name=body.name, description=body.description)
+    db.add(group)
+    await db.commit()
+    await db.refresh(group)
+    # refresh doesn't load relationships — load fresh with memberships
+    result = await db.execute(
+        select(Group).options(selectinload(Group.memberships)).where(Group.id == group.id)
+    )
+    group = result.scalar_one()
+    return _to_group_out(group)
+
+
+@router.get("/{group_id}", response_model=GroupDetailOut)
+async def get_group(
+    group_id: str,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> GroupDetailOut:
+    result = await db.execute(
+        select(Group)
+        .options(
+            selectinload(Group.memberships).joinedload(GroupMembership.group),
+        )
+        .where(Group.id == group_id)
+    )
+    group = result.scalar_one_or_none()
+    if not group:
+        raise HTTPException(status_code=404, detail="Group not found")
+
+    # Load members separately for simplicity
+    mem_result = await db.execute(
+        select(GroupMembership, User)
+        .join(User, User.id == GroupMembership.user_id)
+        .where(GroupMembership.group_id == group_id)
+        .order_by(User.email)
+    )
+    rows = mem_result.all()
+    members = [
+        GroupMemberOut(
+            id=user.id,
+            email=user.email,
+            full_name=user.full_name,
+            is_active=user.is_active,
+            is_group_admin=membership.is_group_admin,
+            joined_at=membership.joined_at,
+        )
+        for membership, user in rows
+    ]
+    return GroupDetailOut(
+        id=group.id,
+        name=group.name,
+        description=group.description,
+        created_at=group.created_at,
+        member_count=len(members),
+        members=members,
+    )
+
+
+@router.patch("/{group_id}", response_model=GroupOut)
+async def update_group(
+    group_id: str,
+    body: GroupUpdate,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> GroupOut:
+    result = await db.execute(
+        select(Group).options(selectinload(Group.memberships)).where(Group.id == group_id)
+    )
+    group = result.scalar_one_or_none()
+    if not group:
+        raise HTTPException(status_code=404, detail="Group not found")
+
+    if body.name is not None and body.name != group.name:
+        dupe = await db.execute(select(Group).where(Group.name == body.name))
+        if dupe.scalar_one_or_none():
+            raise HTTPException(status_code=400, detail="A group with that name already exists")
+        group.name = body.name
+    if body.description is not None:
+        group.description = body.description
+
+    await db.commit()
+    await db.refresh(group)
+    result2 = await db.execute(
+        select(Group).options(selectinload(Group.memberships)).where(Group.id == group.id)
+    )
+    group = result2.scalar_one()
+    return _to_group_out(group)
+
+
+@router.delete("/{group_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def delete_group(
+    group_id: str,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> None:
+    result = await db.execute(select(Group).where(Group.id == group_id))
+    group = result.scalar_one_or_none()
+    if not group:
+        raise HTTPException(status_code=404, detail="Group not found")
+    await db.delete(group)
+    await db.commit()
+
+
+@router.post("/{group_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def add_member(
+    group_id: str,
+    user_id: str,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> None:
+    group_result = await db.execute(select(Group).where(Group.id == group_id))
+    if not group_result.scalar_one_or_none():
+        raise HTTPException(status_code=404, detail="Group not found")
+
+    user_result = await db.execute(select(User).where(User.id == user_id))
+    if not user_result.scalar_one_or_none():
+        raise HTTPException(status_code=404, detail="User not found")
+
+    existing = await db.execute(
+        select(GroupMembership).where(
+            GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
+        )
+    )
+    if existing.scalar_one_or_none():
+        raise HTTPException(status_code=400, detail="User is already a member of this group")
+
+    db.add(GroupMembership(group_id=group_id, user_id=user_id))
+    await db.commit()
+
+
+@router.patch("/{group_id}/members/{user_id}/admin", status_code=status.HTTP_204_NO_CONTENT)
+async def set_member_admin(
+    group_id: str,
+    user_id: str,
+    body: GroupMemberAdminUpdate,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> None:
+    result = await db.execute(
+        select(GroupMembership).where(
+            GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
+        )
+    )
+    membership = result.scalar_one_or_none()
+    if not membership:
+        raise HTTPException(status_code=404, detail="User is not a member of this group")
+    membership.is_group_admin = body.is_group_admin
+    await db.commit()
+
+
+@router.delete("/{group_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
+async def remove_member(
+    group_id: str,
+    user_id: str,
+    _admin: User = Depends(get_current_admin),
+    db: AsyncSession = Depends(get_db),
+) -> None:
+    result = await db.execute(
+        select(GroupMembership).where(
+            GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
+        )
+    )
+    membership = result.scalar_one_or_none()
+    if not membership:
+        raise HTTPException(status_code=404, detail="User is not a member of this group")
+    await db.delete(membership)
+    await db.commit()

backend/app/routers/plugins.py

@@ -0,0 +1,125 @@
+"""
+Generic plugin proxy.
+
+Feature containers advertise themselves via GET /plugin/manifest.  The backend
+health-poller caches those manifests.  This router exposes them to the browser
+through auth-gated endpoints so the frontend never needs to know about specific
+features.
+
+Routes:
+  GET  /api/plugins                    → list accessible plugins for current user
+  GET  /api/plugins/{id}/manifest      → cached manifest (404 if not accessible)
+  GET  /api/plugins/{id}/settings      → proxy to feature /plugin/settings
+  PATCH /api/plugins/{id}/settings     → proxy to feature /plugin/settings
+"""
+import httpx
+from fastapi import APIRouter, Depends, HTTPException, Request
+from fastapi.responses import Response
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.database import get_db
+from app.deps import check_plugin_access, get_current_user
+from app.models.user import User
+from app.services.service_health import get_cached_manifest, get_registry, get_service_url
+
+router = APIRouter()
+
+_HOP_BY_HOP = frozenset([
+    "connection",
+    "keep-alive",
+    "proxy-authenticate",
+    "proxy-authorization",
+    "te",
+    "trailers",
+    "transfer-encoding",
+    "upgrade",
+    "host",
+    "accept-encoding",
+])
+_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
+
+
+async def _proxy(plugin_id: str, method: str, path: str, body: bytes | None,
+                 content_type: str | None = None) -> Response:
+    """Forward a request to the feature container's plugin endpoint."""
+    url = get_service_url(plugin_id)
+    if url is None:
+        raise HTTPException(status_code=404, detail="Not found")
+
+    headers: dict[str, str] = {}
+    if content_type:
+        headers["content-type"] = content_type
+
+    try:
+        async with httpx.AsyncClient(base_url=url, timeout=30.0) as client:
+            resp = await client.request(method, path, content=body, headers=headers)
+    except httpx.RequestError as exc:
+        raise HTTPException(status_code=502, detail=f"Plugin service unreachable: {exc}")
+
+    resp_headers = {k: v for k, v in resp.headers.items() if k.lower() not in _STRIP_RESPONSE}
+    return Response(
+        content=resp.content,
+        status_code=resp.status_code,
+        headers=resp_headers,
+        media_type=resp.headers.get("content-type", "application/json"),
+    )
+
+
+@router.get("")
+async def list_plugins(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+) -> list[dict]:
+    """Return the list of plugins the current user may access."""
+    accessible = []
+    for svc in get_registry():
+        manifest = get_cached_manifest(svc.id)
+        if manifest is None:
+            continue
+        if await check_plugin_access(svc.id, current_user, db):
+            accessible.append({
+                "id": manifest["id"],
+                "name": manifest["name"],
+                "icon": manifest.get("icon", "package"),
+                "version": manifest.get("version", ""),
+            })
+    return accessible
+
+
+@router.get("/{plugin_id}/manifest")
+async def get_plugin_manifest(
+    plugin_id: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+) -> dict:
+    if not await check_plugin_access(plugin_id, current_user, db):
+        raise HTTPException(status_code=404, detail="Not found")
+    manifest = get_cached_manifest(plugin_id)
+    if manifest is None:
+        raise HTTPException(status_code=404, detail="Not found")
+    return manifest
+
+
+@router.get("/{plugin_id}/settings")
+async def get_plugin_settings(
+    plugin_id: str,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+) -> Response:
+    if not await check_plugin_access(plugin_id, current_user, db):
+        raise HTTPException(status_code=404, detail="Not found")
+    return await _proxy(plugin_id, "GET", "/plugin/settings", None)
+
+
+@router.patch("/{plugin_id}/settings")
+async def update_plugin_settings(
+    plugin_id: str,
+    request: Request,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+) -> Response:
+    if not await check_plugin_access(plugin_id, current_user, db):
+        raise HTTPException(status_code=404, detail="Not found")
+    body = await request.body()
+    content_type = request.headers.get("content-type", "application/json")
+    return await _proxy(plugin_id, "PATCH", "/plugin/settings", body, content_type)

backend/app/routers/services.py

@@ -0,0 +1,22 @@
+"""
+GET /api/services — returns health status for all registered feature services.
+Available to any authenticated user so the frontend can drive app visibility.
+"""
+from fastapi import APIRouter, Depends
+
+from app.deps import get_current_user
+from app.models.user import User
+from app.services.service_health import get_all_statuses
+
+router = APIRouter()
+
+
+@router.get("")
+async def list_services(_: User = Depends(get_current_user)) -> list[dict]:
+    """
+    Returns each registered service with its current health status.
+
+    healthy=true  → service responded 200 on its last /health poll
+    healthy=false → unreachable, timed out, or not yet polled
+    """
+    return get_all_statuses()

backend/app/routers/settings.py

@@ -0,0 +1,314 @@
+"""
+Admin-only settings API for per-service runtime configuration.
+
+All endpoints require the caller to be an admin (Depends(get_current_admin)).
+Config files are stored in the 'config' bucket of storage-service.
+"""
+import re as _re
+
+import httpx
+from fastapi import APIRouter, Depends, HTTPException
+from pydantic import BaseModel
+
+from app.core.app_config import (
+    SYSTEM_PROMPT_SERVICES,
+    AppearanceConfig,
+    _merge_api_key,
+    delete_theme,
+    load_ai_service_config,
+    load_ai_service_config_masked,
+    load_all_system_prompts,
+    load_all_themes,
+    load_appearance_config,
+    load_doc_service_config,
+    load_doc_service_config_masked,
+    load_theme_by_id,
+    save_ai_service_config,
+    save_appearance_config,
+    save_doc_service_config,
+    save_service_system_prompts,
+    save_theme,
+    validate_theme_tokens,
+)
+from app.core.config import settings
+from app.deps import get_current_admin, get_current_user, get_service_admin
+from app.models.user import User
+
+router = APIRouter()
+
+_THEME_ID_RE = _re.compile(r"^[a-z0-9_-]{1,64}$")
+
+
+# ── Pydantic request bodies ────────────────────────────────────────────────────
+
+
+class AIProviderUpdate(BaseModel):
+    provider: str
+    anthropic_api_key: str = ""
+    anthropic_model: str = ""
+    ollama_base_url: str = ""
+    ollama_model: str = ""
+    ollama_api_key: str = ""
+    lmstudio_base_url: str = ""
+    lmstudio_model: str = ""
+    lmstudio_api_key: str = ""
+
+
+class LimitsUpdate(BaseModel):
+    max_pdf_mb: int
+
+
+class SystemPromptUpdate(BaseModel):
+    system: str
+    user_template: str
+
+
+class AppearanceUpdate(BaseModel):
+    theme: str
+    default_mode: str
+
+
+class ThemeColors(BaseModel):
+    primary: str
+    primary_hover: str
+    accent: str
+    accent_hover: str
+    background: str
+    surface: str
+    border: str
+    text_primary: str
+    text_muted: str
+
+
+class ThemeCreate(BaseModel):
+    id: str
+    label: str
+    light: ThemeColors
+    dark: ThemeColors
+
+
+class ThemeUpdate(BaseModel):
+    label: str | None = None
+    light: ThemeColors | None = None
+    dark: ThemeColors | None = None
+
+
+# ── AI settings ────────────────────────────────────────────────────────────────
+
+
+@router.get("/ai")
+async def get_ai_settings(
+    _: User = Depends(get_service_admin("ai-service")),
+) -> dict:
+    return await load_ai_service_config_masked()
+
+
+@router.patch("/ai")
+async def update_ai_settings(
+    body: AIProviderUpdate,
+    _: User = Depends(get_service_admin("ai-service")),
+) -> dict:
+    valid_providers = ("anthropic", "ollama", "lmstudio")
+    if body.provider not in valid_providers:
+        raise HTTPException(status_code=422, detail=f"provider must be one of {valid_providers}")
+
+    config = await load_ai_service_config()
+    config.provider = body.provider
+
+    # Anthropic
+    if body.anthropic_api_key:
+        config.anthropic.api_key = _merge_api_key(
+            body.anthropic_api_key, config.anthropic.api_key
+        )
+    if body.anthropic_model:
+        config.anthropic.model = body.anthropic_model
+
+    # Ollama
+    if body.ollama_base_url:
+        config.ollama.base_url = body.ollama_base_url
+    if body.ollama_model:
+        config.ollama.model = body.ollama_model
+    if body.ollama_api_key:
+        config.ollama.api_key = _merge_api_key(body.ollama_api_key, config.ollama.api_key)
+
+    # LM Studio
+    if body.lmstudio_base_url:
+        config.lmstudio.base_url = body.lmstudio_base_url
+    if body.lmstudio_model:
+        config.lmstudio.model = body.lmstudio_model
+    if body.lmstudio_api_key:
+        config.lmstudio.api_key = _merge_api_key(
+            body.lmstudio_api_key, config.lmstudio.api_key
+        )
+
+    await save_ai_service_config(config)
+    return await load_ai_service_config_masked()
+
+
+@router.post("/ai/test")
+async def test_ai_connection(
+    _: User = Depends(get_service_admin("ai-service")),
+) -> dict:
+    """Proxy a minimal chat request to ai-service to verify the connection."""
+    try:
+        async with httpx.AsyncClient(timeout=30.0) as client:
+            resp = await client.post(
+                f"{settings.AI_SERVICE_URL}/chat",
+                json={
+                    "messages": [{"role": "user", "content": "Reply with: ok"}],
+                    "max_tokens": 16,
+                    "temperature": 0,
+                },
+            )
+        if resp.status_code == 200:
+            data = resp.json()
+            return {"ok": True, "provider": data.get("provider"), "response": data.get("content")}
+        return {"ok": False, "error": f"ai-service returned {resp.status_code}: {resp.text[:200]}"}
+    except Exception as exc:
+        return {"ok": False, "error": str(exc)}
+
+
+# ── Document limits ────────────────────────────────────────────────────────────
+
+
+@router.get("/documents/limits")
+async def get_documents_limits(
+    _: User = Depends(get_service_admin("doc-service")),
+) -> dict:
+    return await load_doc_service_config_masked()
+
+
+@router.patch("/documents/limits")
+async def update_documents_limits(
+    body: LimitsUpdate,
+    _: User = Depends(get_service_admin("doc-service")),
+) -> dict:
+    if body.max_pdf_mb < 1 or body.max_pdf_mb > 200:
+        raise HTTPException(status_code=422, detail="max_pdf_mb must be between 1 and 200")
+
+    config = await load_doc_service_config()
+    config.documents.max_pdf_bytes = body.max_pdf_mb * 1024 * 1024
+    await save_doc_service_config(config)
+    return await load_doc_service_config_masked()
+
+
+# ── System prompts ─────────────────────────────────────────────────────────────
+
+
+@router.get("/system-prompts")
+async def get_system_prompts(
+    _: User = Depends(get_service_admin("ai-service")),
+) -> dict:
+    return await load_all_system_prompts()
+
+
+@router.patch("/system-prompts/{service_id}")
+async def update_system_prompt(
+    service_id: str,
+    body: SystemPromptUpdate,
+    _: User = Depends(get_service_admin("ai-service")),
+) -> dict:
+    if service_id not in SYSTEM_PROMPT_SERVICES:
+        raise HTTPException(status_code=404, detail=f"No system prompts registered for {service_id!r}")
+    await save_service_system_prompts(service_id, body.system, body.user_template)
+    return await load_all_system_prompts()
+
+
+# ── Appearance (global default — auth read, admin write) ───────────────────────
+
+
+@router.get("/appearance")
+async def get_appearance(
+    _: User = Depends(get_current_user),
+) -> dict:
+    config = await load_appearance_config()
+    return config.model_dump()
+
+
+@router.patch("/appearance")
+async def update_appearance(
+    body: AppearanceUpdate,
+    _: User = Depends(get_current_admin),
+) -> dict:
+    if body.default_mode not in ("light", "dark", "system"):
+        raise HTTPException(status_code=422, detail="default_mode must be 'light', 'dark', or 'system'")
+    themes = await load_all_themes()
+    theme_ids = {t["id"] for t in themes}
+    if body.theme not in theme_ids:
+        raise HTTPException(status_code=422, detail=f"Unknown theme: {body.theme!r}")
+    config = AppearanceConfig(theme=body.theme, default_mode=body.default_mode)
+    await save_appearance_config(config)
+    return config.model_dump()
+
+
+# ── Theme CRUD ─────────────────────────────────────────────────────────────────
+
+
+@router.get("/themes")
+async def list_themes(
+    _: User = Depends(get_current_user),
+) -> list:
+    return await load_all_themes()
+
+
+@router.post("/themes", status_code=201)
+async def create_theme(
+    body: ThemeCreate,
+    _: User = Depends(get_current_admin),
+) -> dict:
+    if not _THEME_ID_RE.match(body.id):
+        raise HTTPException(status_code=422, detail="Theme ID must match [a-z0-9_-]{1,64}")
+    existing = {t["id"] for t in await load_all_themes()}
+    if body.id in existing:
+        raise HTTPException(status_code=400, detail=f"Theme ID already in use: {body.id!r}")
+    light = body.light.model_dump()
+    dark = body.dark.model_dump()
+    for mode, colors in (("light", light), ("dark", dark)):
+        errors = validate_theme_tokens(colors)
+        if errors:
+            raise HTTPException(status_code=422, detail=f"{mode}: {'; '.join(errors)}")
+    theme = {"id": body.id, "label": body.label, "builtin": False, "light": light, "dark": dark}
+    await save_theme(theme)
+    return theme
+
+
+@router.patch("/themes/{theme_id}")
+async def update_theme(
+    theme_id: str,
+    body: ThemeUpdate,
+    _: User = Depends(get_current_admin),
+) -> dict:
+    theme = await load_theme_by_id(theme_id)
+    if theme is None:
+        raise HTTPException(status_code=404, detail="Theme not found")
+    if theme.get("builtin"):
+        raise HTTPException(status_code=400, detail="Cannot edit a built-in theme")
+    if body.label is not None:
+        theme["label"] = body.label
+    if body.light is not None:
+        light = body.light.model_dump()
+        errors = validate_theme_tokens(light)
+        if errors:
+            raise HTTPException(status_code=422, detail=f"light: {'; '.join(errors)}")
+        theme["light"] = light
+    if body.dark is not None:
+        dark = body.dark.model_dump()
+        errors = validate_theme_tokens(dark)
+        if errors:
+            raise HTTPException(status_code=422, detail=f"dark: {'; '.join(errors)}")
+        theme["dark"] = dark
+    await save_theme(theme)
+    return theme
+
+
+@router.delete("/themes/{theme_id}", status_code=204)
+async def remove_theme(
+    theme_id: str,
+    _: User = Depends(get_current_admin),
+) -> None:
+    try:
+        await delete_theme(theme_id)
+    except FileNotFoundError:
+        raise HTTPException(status_code=404, detail="Theme not found")
+    except ValueError as exc:
+        raise HTTPException(status_code=400, detail=str(exc))

backend/app/routers/storage_config.py

@@ -0,0 +1,126 @@
+"""
+Admin-only endpoints for storage-service backend configuration.
+
+GET  /admin/storage-config           — current backend driver + health
+PATCH /admin/storage-config          — update backend config (no data migration)
+POST /admin/storage-config/migrate   — start migration to a new backend
+GET  /admin/storage-config/migrate/status — poll migration progress
+DELETE /admin/storage-config/migrate — cancel in-progress migration
+
+All endpoints proxy to storage-service:8020.
+"""
+import logging
+
+import httpx
+from fastapi import APIRouter, Depends, HTTPException
+from pydantic import BaseModel
+
+from app.core.config import settings
+from app.deps import get_current_admin
+from app.models.user import User
+
+router = APIRouter()
+logger = logging.getLogger(__name__)
+
+_STORAGE_BASE = settings.STORAGE_SERVICE_URL
+
+
+class BackendConfigUpdate(BaseModel):
+    driver: str
+    config: dict = {}
+
+
+class MigrateRequest(BaseModel):
+    driver: str
+    config: dict = {}
+
+
+def _storage_url(path: str) -> str:
+    return f"{_STORAGE_BASE}{path}"
+
+
+async def _proxy_get(path: str) -> dict:
+    async with httpx.AsyncClient(timeout=15.0) as client:
+        resp = await client.get(_storage_url(path))
+        if resp.status_code == 404:
+            raise HTTPException(status_code=404, detail="Not found")
+        resp.raise_for_status()
+        return resp.json()
+
+
+@router.get("/storage-config")
+async def get_storage_config(
+    _: User = Depends(get_current_admin),
+) -> dict:
+    """Return current backend driver and health status."""
+    async with httpx.AsyncClient(timeout=10.0) as client:
+        resp = await client.get(_storage_url("/health"))
+        resp.raise_for_status()
+        return resp.json()
+
+
+@router.patch("/storage-config", status_code=204)
+async def update_storage_config(
+    body: BackendConfigUpdate,
+    _: User = Depends(get_current_admin),
+) -> None:
+    """
+    Reconfigure the active backend without migrating data.
+    Use when changing credentials for the same backend type, or reverting to local.
+    To move data to a new backend, use POST /admin/storage-config/migrate instead.
+    """
+    async with httpx.AsyncClient(timeout=15.0) as client:
+        resp = await client.patch(
+            _storage_url("/backend-config"),
+            json={"driver": body.driver, "config": body.config},
+        )
+        if resp.status_code == 400:
+            raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
+        if resp.status_code == 409:
+            raise HTTPException(status_code=409, detail="Migration in progress — cannot reconfigure now")
+        resp.raise_for_status()
+
+
+@router.post("/storage-config/migrate", status_code=202)
+async def start_migration(
+    body: MigrateRequest,
+    _: User = Depends(get_current_admin),
+) -> dict:
+    """
+    Start an async migration to a new backend.
+
+    Flow: validate new backend → copy all objects → verify → switch → delete old objects.
+    The old backend stays active until 100% of objects are verified on the new one.
+    Poll GET /admin/storage-config/migrate/status to track progress.
+    """
+    async with httpx.AsyncClient(timeout=30.0) as client:
+        resp = await client.post(
+            _storage_url("/migrate"),
+            json={"driver": body.driver, "config": body.config},
+        )
+        if resp.status_code == 400:
+            raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
+        if resp.status_code == 409:
+            raise HTTPException(status_code=409, detail="A migration is already in progress")
+        resp.raise_for_status()
+        return resp.json()
+
+
+@router.get("/storage-config/migrate/status")
+async def migration_status(
+    _: User = Depends(get_current_admin),
+) -> dict:
+    """Poll migration progress. State: idle → validating → migrating → switching → cleaning → done."""
+    return await _proxy_get("/migrate/status")
+
+
+@router.delete("/storage-config/migrate", status_code=204)
+async def cancel_migration(
+    _: User = Depends(get_current_admin),
+) -> None:
+    """Cancel a running migration. The old backend remains active."""
+    async with httpx.AsyncClient(timeout=10.0) as client:
+        resp = await client.delete(_storage_url("/migrate"))
+        if resp.status_code == 409:
+            raise HTTPException(status_code=409, detail="No cancellable migration in progress")
+        resp.raise_for_status()

backend/app/routers/users.py

@@ -1,8 +1,13 @@
-from fastapi import APIRouter, Depends
+from fastapi import APIRouter, Depends, HTTPException
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+from sqlalchemy.orm import selectinload
 
+from app.database import get_db
 from app.deps import get_current_user
+from app.models.group import Group, GroupMembership
 from app.models.user import User
-from app.schemas.user import UserOut
+from app.schemas.user import ColorModeUpdate, DashboardPrefsOut, DashboardPrefsUpdate, UserGroupOut, UserOut
 
 router = APIRouter()
 
@@ -10,3 +15,50 @@ router = APIRouter()
 @router.get("/me", response_model=UserOut)
 async def get_me(current_user: User = Depends(get_current_user)):
     return current_user
+
+
+@router.get("/me/preferences", response_model=DashboardPrefsOut)
+async def get_preferences(current_user: User = Depends(get_current_user)):
+    return DashboardPrefsOut(app_ids=current_user.dashboard_app_ids or [])
+
+
+@router.patch("/me/preferences", response_model=DashboardPrefsOut)
+async def update_preferences(
+    body: DashboardPrefsUpdate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_user.dashboard_app_ids = body.app_ids
+    await db.commit()
+    await db.refresh(current_user)
+    return DashboardPrefsOut(app_ids=current_user.dashboard_app_ids or [])
+
+
+@router.get("/me/groups", response_model=list[UserGroupOut])
+async def get_my_groups(
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    """Return all groups the current user belongs to, including their admin status."""
+    result = await db.execute(
+        select(Group, GroupMembership.is_group_admin)
+        .join(GroupMembership, GroupMembership.group_id == Group.id)
+        .where(GroupMembership.user_id == current_user.id)
+        .order_by(Group.name)
+    )
+    return [
+        UserGroupOut(id=g.id, name=g.name, description=g.description, is_group_admin=is_admin)
+        for g, is_admin in result.all()
+    ]
+
+
+@router.patch("/me/color-mode", response_model=UserOut)
+async def update_color_mode(
+    body: ColorModeUpdate,
+    current_user: User = Depends(get_current_user),
+    db: AsyncSession = Depends(get_db),
+):
+    current_user.color_mode = body.color_mode
+    await db.commit()
+    await db.refresh(current_user)
+    return current_user

backend/app/schemas/group.py

@@ -0,0 +1,42 @@
+from datetime import datetime
+
+from pydantic import BaseModel, Field
+
+
+class GroupCreate(BaseModel):
+    name: str = Field(..., min_length=1, max_length=128)
+    description: str | None = Field(None, max_length=512)
+
+
+class GroupUpdate(BaseModel):
+    name: str | None = Field(None, min_length=1, max_length=128)
+    description: str | None = Field(None, max_length=512)
+
+
+class GroupMemberOut(BaseModel):
+    id: str
+    email: str
+    full_name: str | None
+    is_active: bool
+    is_group_admin: bool = False
+    joined_at: datetime
+
+    model_config = {"from_attributes": True}
+
+
+class GroupMemberAdminUpdate(BaseModel):
+    is_group_admin: bool
+
+
+class GroupOut(BaseModel):
+    id: str
+    name: str
+    description: str | None
+    created_at: datetime
+    member_count: int = 0
+
+    model_config = {"from_attributes": True}
+
+
+class GroupDetailOut(GroupOut):
+    members: list[GroupMemberOut] = []

backend/app/schemas/user.py

@@ -1,4 +1,5 @@
 import re
+from datetime import datetime
 
 from pydantic import BaseModel, EmailStr, Field, field_validator
 
@@ -71,6 +72,7 @@ class UserOut(BaseModel):
     # validation_alias reads is_superuser from the ORM object; the JSON key
     # in the response is the field name "is_admin" (not the alias).
     is_admin: bool = Field(validation_alias="is_superuser", default=False)
+    color_mode: str | None = None
 
     model_config = {"from_attributes": True, "populate_by_name": True}
 
@@ -96,3 +98,45 @@ class UserAdminCreate(UserCreate):
 class Token(BaseModel):
     access_token: str
     token_type: str = "bearer"
+
+
+# ── Dashboard preferences ──────────────────────────────────────────────────────
+
+class DashboardPrefsOut(BaseModel):
+    app_ids: list[str]
+
+
+class ColorModeUpdate(BaseModel):
+    color_mode: str
+
+    @field_validator("color_mode")
+    @classmethod
+    def validate_mode(cls, v: str) -> str:
+        if v not in ("light", "dark", "system"):
+            raise ValueError("color_mode must be 'light', 'dark', or 'system'")
+        return v
+
+
+class UserGroupOut(BaseModel):
+    """A group the current user belongs to — used for the share picker."""
+    id: str
+    name: str
+    description: str | None
+    is_group_admin: bool = False
+
+    model_config = {"from_attributes": True}
+
+
+class DashboardPrefsUpdate(BaseModel):
+    app_ids: list[str] = Field(default_factory=list)
+
+    @field_validator("app_ids")
+    @classmethod
+    def validate_app_ids(cls, v: list[str]) -> list[str]:
+        if len(v) > 50:
+            raise ValueError("Cannot pin more than 50 apps")
+        for item in v:
+            # Service IDs are alphanumeric slugs or UUIDs — no HTML/script allowed.
+            if not re.match(r'^[a-zA-Z0-9_\-]{1,64}$', item):
+                raise ValueError(f"Invalid app ID: {item!r}")
+        return v

backend/app/services/group_bootstrap.py

@@ -0,0 +1,37 @@
+"""
+Ensure that every registered service has a corresponding admin group.
+
+Called once at startup after register_services().  Idempotent — safe to run
+on every restart, creates nothing if groups already exist.
+
+Naming convention: "{service_id}-admin"  (e.g. "doc-service-admin")
+"""
+import logging
+
+from sqlalchemy import select
+from sqlalchemy.ext.asyncio import AsyncSession
+
+from app.models.group import Group
+from app.services.service_health import get_registry
+
+logger = logging.getLogger(__name__)
+
+
+async def ensure_service_admin_groups(db: AsyncSession) -> None:
+    """Create a <service-id>-admin group for each registered service if absent."""
+    for svc in get_registry():
+        group_name = f"{svc.id}-admin"
+        result = await db.execute(select(Group).where(Group.name == group_name))
+        if result.scalar_one_or_none() is not None:
+            continue
+
+        import uuid
+        group = Group(
+            id=str(uuid.uuid4()),
+            name=group_name,
+            description=f"Administrators for the {svc.name} service.",
+        )
+        db.add(group)
+        logger.info("[bootstrap] Created admin group %r for service %r", group_name, svc.id)
+
+    await db.commit()

backend/app/services/service_health.py

@@ -0,0 +1,178 @@
+"""
+Background health-checker for registered feature services.
+
+Polls each service's /health endpoint every POLL_INTERVAL seconds and stores
+the result in an in-memory dict.  Also fetches /plugin/manifest when available
+and caches it so the plugin proxy can serve it without per-request network calls.
+The REST layer reads from that dict — no DB, no blocking calls on the request path.
+"""
+import asyncio
+import logging
+from dataclasses import dataclass
+
+import httpx
+
+logger = logging.getLogger(__name__)
+
+POLL_INTERVAL = 30  # seconds
+
+
+@dataclass
+class ServiceDefinition:
+    id: str
+    name: str
+    description: str
+    internal_url: str          # e.g. http://doc-service:8001
+    health_path: str = "/health"
+    app_path: str = ""         # frontend route; empty = no open button
+    settings_path: str = ""    # frontend admin-settings route
+
+
+# ── Registry ──────────────────────────────────────────────────────────────────
+# Add new services here.  The internal_url is filled in at startup from settings.
+
+_REGISTRY: list[ServiceDefinition] = []
+
+# id → True/False/None (None = not yet checked)
+_health: dict[str, bool | None] = {}
+
+# id → plugin manifest dict, or None if the service has no plugin manifest
+_manifests: dict[str, dict | None] = {}
+
+
+def register_services(doc_service_url: str, ai_service_url: str, storage_service_url: str) -> None:
+    """Called once during app startup to populate the registry from config."""
+    global _REGISTRY, _health, _manifests
+
+    _REGISTRY = [
+        ServiceDefinition(
+            id="doc-service",
+            name="Documents",
+            description="Upload PDF files, extract data, and organise them with categories.",
+            internal_url=doc_service_url,
+            health_path="/health",
+            app_path="/apps/documents",
+            settings_path="/apps/documents/settings",
+        ),
+        ServiceDefinition(
+            id="ai-service",
+            name="AI Service",
+            description="Shared AI provider for all features. Configure model, credentials, and connection.",
+            internal_url=ai_service_url,
+            health_path="/health",
+            app_path="",
+            settings_path="/apps/ai/settings",
+        ),
+        ServiceDefinition(
+            id="storage-service",
+            name="Storage",
+            description="Unified file storage. Manages all uploaded files with pluggable backends (local, S3, WebDAV).",
+            internal_url=storage_service_url,
+            health_path="/health",
+            app_path="",
+            settings_path="/admin/storage",
+        ),
+    ]
+
+    _health = {svc.id: None for svc in _REGISTRY}
+    _manifests = {svc.id: None for svc in _REGISTRY}
+    logger.info("Service registry initialised with %d services", len(_REGISTRY))
+
+
+# ── Health check logic ────────────────────────────────────────────────────────
+
+
+async def _check_service(svc: ServiceDefinition) -> None:
+    url = f"{svc.internal_url}{svc.health_path}"
+    prev = _health.get(svc.id)
+    try:
+        async with httpx.AsyncClient(timeout=5.0) as client:
+            resp = await client.get(url)
+        healthy = resp.status_code == 200
+    except Exception as exc:
+        logger.debug("Health check failed for %s: %s", svc.id, exc)
+        healthy = False
+
+    _health[svc.id] = healthy
+
+    # Log only on transitions so the logs stay quiet during normal operation
+    if prev != healthy:
+        if healthy:
+            logger.info("Service %s is now HEALTHY", svc.id)
+        else:
+            logger.warning("Service %s is now UNHEALTHY", svc.id)
+
+    # Opportunistically fetch plugin manifest when the service is healthy
+    if healthy:
+        await _fetch_manifest(svc)
+
+
+async def _fetch_manifest(svc: ServiceDefinition) -> None:
+    """Try to GET /plugin/manifest from the service; cache result (or None)."""
+    url = f"{svc.internal_url}/plugin/manifest"
+    try:
+        async with httpx.AsyncClient(timeout=5.0) as client:
+            resp = await client.get(url)
+        if resp.status_code == 200:
+            _manifests[svc.id] = resp.json()
+        else:
+            _manifests[svc.id] = None
+    except Exception:
+        # Service doesn't have a plugin manifest — not an error
+        _manifests[svc.id] = None
+
+
+async def check_all() -> None:
+    """Run health checks for all registered services concurrently."""
+    await asyncio.gather(*[_check_service(svc) for svc in _REGISTRY])
+
+
+async def health_check_loop() -> None:
+    """Runs forever; polls every POLL_INTERVAL seconds.
+
+    Exceptions inside a single polling round are caught so the loop cannot
+    be killed by a transient error.
+    """
+    while True:
+        try:
+            await check_all()
+        except Exception:
+            logger.exception("Unexpected error during health check round; will retry")
+        await asyncio.sleep(POLL_INTERVAL)
+
+
+# ── Public read API ───────────────────────────────────────────────────────────
+
+
+def get_all_statuses() -> list[dict]:
+    """Return the current health snapshot for all registered services."""
+    return [
+        {
+            "id": svc.id,
+            "name": svc.name,
+            "description": svc.description,
+            "app_path": svc.app_path,
+            "settings_path": svc.settings_path,
+            # None means not yet checked; treat as unhealthy for the UI
+            "healthy": bool(_health.get(svc.id)),
+        }
+        for svc in _REGISTRY
+    ]
+
+
+def get_cached_manifest(service_id: str) -> dict | None:
+    """Return the cached plugin manifest for a service, or None if unavailable."""
+    return _manifests.get(service_id)
+
+
+def get_service_url(service_id: str) -> str | None:
+    """Return the internal URL for a registered service, or None if unknown."""
+    for svc in _REGISTRY:
+        if svc.id == service_id:
+            return svc.internal_url
+    return None
+
+
+def get_registry() -> list[ServiceDefinition]:
+    """Return the current service registry (always up-to-date after register_services)."""
+    return _REGISTRY

backend/pyproject.toml

@@ -17,13 +17,15 @@ dependencies = [
     "python-jose[cryptography]>=3.3",
     "bcrypt>=4.0",
     "python-multipart>=0.0.9",
+    "httpx>=0.27",
+    "anthropic>=0.28",
+    "openai>=1.0",
 ]
 
 [project.optional-dependencies]
 dev = [
     "pytest>=8",
     "pytest-asyncio>=0.23",
-    "httpx>=0.27",
     "ruff>=0.4",
 ]
 

backend/scripts/start.sh

@@ -0,0 +1,8 @@
+#!/bin/sh
+set -e
+
+echo "[start] running migrations..."
+alembic upgrade head
+
+echo "[start] starting uvicorn..."
+exec uvicorn app.main:app --host 0.0.0.0 --port 8000

changelog/2026-04-14_doc-service-tests-suggestions.md

@@ -0,0 +1,66 @@
+# 2026-04-14 — Doc-service tests, AI category suggestions, LM Studio default
+
+**Timestamp:** 2026-04-14T00:00:00+00:00
+
+## Summary
+
+Added pytest test suite for doc-service, updated the AI prompt to return suggested categories, wired up a suggestions UI in DocumentsPage (per-suggestion Accept/Create&Assign/Dismiss), changed the default AI provider to LM Studio at host.docker.internal:1234, and created a gitignored test PDF directory.
+
+## Files Added
+
+- `features/doc-service/tests/__init__.py`
+- `features/doc-service/tests/conftest.py` — SQLite in-memory DB, tmp DATA_DIR, mock AI provider, minimal+invoice PDF builders, real_pdfs fixture (auto-skips if no PDFs present)
+- `features/doc-service/tests/test_categories.py` — full CRUD + per-user isolation
+- `features/doc-service/tests/test_documents.py` — upload, list, get, status, delete, category assignment, AI processing integration, live PDF tests
+- `features/doc-service/tests/pdfs/.gitkeep` — tracked empty directory; drop PDFs here for live testing
+
+## Files Modified
+
+- `.gitignore` — ignore `features/doc-service/tests/pdfs/*.pdf`
+- `features/doc-service/pyproject.toml` — added `aiosqlite>=0.20` to dev deps
+- `features/doc-service/app/services/ai/base.py` — added `suggested_categories` to AI prompt (2–5 category names per document)
+- `features/doc-service/app/services/config_reader.py` — default provider changed to `lmstudio`; URLs changed to `host.docker.internal:1234/v1` (Docker→host resolution on macOS/Windows)
+- `backend/app/core/app_config.py` — default `LMStudioConfig.base_url` = `http://host.docker.internal:1234/v1`; default provider = `lmstudio`
+- `frontend/src/pages/DocumentsPage.tsx` — added `SuggestionChip` component and `suggested_categories` section in DocumentRow: checks if suggestion already exists as a user category, shows "Assign" (existing) or "Create & Assign" (new), dismiss removes from local state
+
+---
+
+# 2026-04-14 — AI service container (shared AI intermediary)
+
+**Timestamp:** 2026-04-14T12:00:00+00:00
+
+## Summary
+
+Extracted all AI provider logic from doc-service into a new standalone `ai-service` container (port 8010). All feature containers now POST messages to ai-service instead of calling AI providers directly. Added tests for ai-service, updated backend settings routes to /api/settings/ai, added AI Service card to frontend AppsPage with dedicated settings page.
+
+## Files Added
+
+- `features/ai-service/` — full new microservice: Dockerfile, pyproject.toml, scripts/, app/ (providers, schemas, routers, services), tests/
+- `features/ai-service/.env` — gitignored, holds LM Studio API key for dev
+- `features/ai-service/.env.example`
+- `features/doc-service/app/services/prompts.py` — domain prompts extracted from deleted base.py
+- `features/doc-service/app/services/ai_client.py` — httpx client that calls ai-service /chat
+
+## Files Modified
+
+- `features/doc-service/app/routers/documents.py` — replaced provider call with classify_document()
+- `features/doc-service/app/services/config_reader.py` — removed AI config section (owned by ai-service now)
+- `features/doc-service/app/core/config.py` — added AI_SERVICE_URL setting
+- `features/doc-service/pyproject.toml` — removed anthropic/openai, added httpx
+- `features/doc-service/.env` — removed LMSTUDIO_* vars, added AI_SERVICE_URL
+- `features/doc-service/tests/conftest.py` — renamed mock_ai → mock_ai_service, patching classify_document
+- `features/doc-service/tests/test_documents.py` — mock_ai → mock_ai_service; added graceful 502 test
+- `backend/app/core/app_config.py` — AIServiceConfig split from DocServiceConfig; new load/save/mask helpers
+- `backend/app/core/config.py` — added AI_SERVICE_URL setting
+- `backend/app/routers/settings.py` — new /api/settings/ai routes; test endpoint proxies to ai-service via httpx
+- `docker-compose.yml` — added ai-service container; AI_SERVICE_URL env on backend + doc-service
+- `docker-compose.dev.yml` — added ai-service dev override with hot reload and .env
+- `frontend/src/api/client.ts` — renamed getDocumentSettings→getAISettings, updateDocumentAISettings→updateAISettings, testDocumentAIConnection→testAIConnection; added getDocumentLimits
+- `frontend/src/pages/AIAdminSettingsPage.tsx` — new page at /apps/ai/settings/admin
+- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — now shows Upload Limits only
+- `frontend/src/pages/AppsPage.tsx` — added AI Service card (admin settings link, no Open button)
+- `frontend/src/App.tsx` — added /apps/ai/settings/admin route
+
+## Files Deleted
+
+- `features/doc-service/app/services/ai/` — anthropic_provider.py, openai_compat.py, base.py, __init__.py

changelog/2026-04-14_doc-service.md

@@ -0,0 +1,96 @@
+# 2026-04-14 — PDF Document Service
+
+**Timestamp:** 2026-04-14T00:00:00+00:00
+
+## Summary
+
+Added `features/doc-service` — a FastAPI microservice that accepts PDF uploads, extracts text with pdfplumber, and uses a pluggable AI provider (Anthropic, Ollama, or LM Studio) to classify and extract structured data. Integrated it into the main backend via httpx proxy routers. Added an admin settings UI at `/apps/documents/settings/admin`. Updated the frontend route tree, Nav, and AppsPage.
+
+## Files Added
+
+- `features/doc-service/Dockerfile` — UID 1001, pre-creates `/data/documents` and `/config`
+- `features/doc-service/pyproject.toml` — service dependencies
+- `features/doc-service/alembic.ini` — separate `alembic_version_doc_service` table
+- `features/doc-service/.env.example`
+- `features/doc-service/scripts/start.sh` — migrations + uvicorn
+- `features/doc-service/scripts/start_dev.sh` — migrations + uvicorn --reload
+- `features/doc-service/alembic/env.py` — async migrations, VERSION_TABLE isolation
+- `features/doc-service/alembic/versions/0001_create_doc_tables.py` — documents, document_categories, document_category_assignments
+- `features/doc-service/app/main.py` — no CORS (internal service)
+- `features/doc-service/app/core/config.py` — DATABASE_URL, DATA_DIR, CONFIG_PATH settings
+- `features/doc-service/app/database.py` — async engine, AsyncSessionLocal, Base
+- `features/doc-service/app/deps.py` — get_user_id from X-User-Id header
+- `features/doc-service/app/models/document.py` — Document ORM model
+- `features/doc-service/app/models/category.py` — DocumentCategory ORM model
+- `features/doc-service/app/models/category_assignment.py` — CategoryAssignment composite PK
+- `features/doc-service/app/models/__init__.py`
+- `features/doc-service/app/schemas/document.py` — DocumentOut, DocumentStatusOut, DocumentTypeUpdate, CategoryOut
+- `features/doc-service/app/schemas/category.py` — CategoryCreate, CategoryOut, CategoryUpdate
+- `features/doc-service/app/routers/documents.py` — upload, list, get, status, patch type, delete, file download, category assignment
+- `features/doc-service/app/routers/categories.py` — CRUD for DocumentCategory
+- `features/doc-service/app/services/storage.py` — aiofiles write, path helpers, delete
+- `features/doc-service/app/services/config_reader.py` — load_doc_config() with 30s TTL cache
+- `features/doc-service/app/services/ai/__init__.py` — get_provider() factory
+- `features/doc-service/app/services/ai/base.py` — AIProvider ABC, shared prompts
+- `features/doc-service/app/services/ai/anthropic_provider.py` — AnthropicProvider
+- `features/doc-service/app/services/ai/openai_compat.py` — OpenAICompatProvider (Ollama + LM Studio)
+- `backend/app/core/app_config.py` — DocServiceConfig Pydantic model, load/save with atomic write, api_key masking
+- `backend/app/routers/settings.py` — GET/PATCH /api/settings/documents/*, admin only
+- `backend/app/routers/documents_proxy.py` — httpx proxy to doc-service /documents/*
+- `backend/app/routers/categories_proxy.py` — httpx proxy to doc-service /categories/*
+- `frontend/src/pages/DocumentsPage.tsx` — upload, list, status polling, categories, file download
+- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — AI provider config, connection test, upload limits
+
+## Files Modified
+
+- `backend/app/main.py` — registered settings_router, categories_proxy (before!), documents_proxy
+- `backend/pyproject.toml` — moved httpx to main deps, added anthropic>=0.28, openai>=1.0
+- `frontend/src/App.tsx` — added /apps/documents and /apps/documents/settings/admin routes, removed /settings
+- `frontend/src/components/Nav.tsx` — removed Settings link, added Profile link, logo links to /
+- `frontend/src/pages/AppsPage.tsx` — replaced stub with app launcher card grid
+- `frontend/src/api/client.ts` — added documents, categories, and settings API functions
+- `docker-compose.yml` — added doc-service service, doc_data + app_config volumes, removed internal:true from backend-net, added app_config volume to backend
+- `docker-compose.dev.yml` — added doc-service dev override with --reload
+- `TODO.md` — added PDF Documents app section
+
+## Files Deleted
+
+- `frontend/src/pages/SettingsPage.tsx` — stub replaced by per-app settings pages
+
+---
+
+# 2026-04-14 — Server-side pagination and filter bar
+
+**Timestamp:** 2026-04-14T12:00:00+00:00
+
+## Summary
+
+Added server-side pagination and a filter bar to the Documents feature.
+
+## Files Added / Modified / Deleted
+
+- **Modified** `features/doc-service/app/schemas/document.py` — Added `DocumentPage` schema (`items`, `total`, `page`, `pages`)
+- **Modified** `features/doc-service/app/routers/documents.py` — `GET /documents` now accepts `page`, `per_page`, `sort`, `order`, `status`, `document_type`, `search` query params; returns `DocumentPage`
+- **Modified** `frontend/src/api/client.ts` — `listDocuments` accepts `DocumentListParams`; added `DocumentPage` and `DocumentListParams` interfaces
+- **Modified** `frontend/src/pages/DocumentsPage.tsx` — Added `FilterBar` (search, status, type, sort, order) and `Pagination` controls; query key includes params for cache isolation
+
+---
+
+# 2026-04-14 — AI Service priority queue + model config update
+
+**Timestamp:** 2026-04-14T15:00:00+00:00
+
+## Summary
+
+Added a priority queue system to ai-service with start/pause/resume/stop controls. Updated LM Studio model to gemma-4-e4b-it.
+
+## Files Added / Modified / Deleted
+
+- **Created** `features/ai-service/app/services/queue.py` — in-memory `asyncio.PriorityQueue` with HIGH/NORMAL/LOW priorities, FIFO within same level, single async worker with pause/resume/stop
+- **Created** `features/ai-service/app/schemas/queue.py` — `QueueRequest`, `JobStatus`, `QueueStatus` Pydantic models
+- **Created** `features/ai-service/app/routers/queue.py` — `POST /queue/jobs`, `GET /queue/jobs/{id}`, `DELETE /queue/jobs/{id}`, `GET /queue/status`, `POST /queue/pause|resume|start|stop`
+- **Modified** `features/ai-service/app/routers/chat.py` — extracted `execute_chat()` (called by queue worker); `POST /chat` now submits to queue at NORMAL priority and awaits result
+- **Modified** `features/ai-service/app/main.py` — start/stop queue worker in lifespan; mount queue router
+- **Modified** `features/ai-service/app/services/config_reader.py` — default model updated to `gemma-4-e4b-it`
+- **Modified** `features/ai-service/pyproject.toml` — `httpx` moved to runtime deps
+- **Modified** `features/ai-service/.env` — model updated to `gemma-4-e4b-it`

changelog/2026-04-14_network-isolation.md

@@ -0,0 +1,20 @@
+# 2026-04-14 — Docker network isolation: backend and db ports removed from host
+
+**Timestamp:** 2026-04-14T00:00:00
+
+## Summary
+
+Replaced flat single-network Docker setup with two explicit networks. Only the frontend exposes a host port. The database and backend are unreachable from outside the Docker network.
+
+## Network architecture
+
+- `backend-net` (`internal: true`) — db, backend, and frontend reverse proxy; no gateway, no host routing
+- `frontend-net` — frontend only; binds port 80 (prod) or 5173 (dev) to the host
+
+## Files Modified
+
+- `docker-compose.yml` — removed `ports:` from `db` and `backend`; added `networks:` to all services; defined `backend-net` (internal) and `frontend-net`
+- `docker-compose.dev.yml` — no network changes needed (inherits from base); kept `5173:5173` on frontend
+- `.claude/agents/security-auditor.md` — added hard rule: only frontend exposes host ports; db and backend must never have `ports:` in any compose file
+- `TODO.md` — marked Docker port hardening as done
+- `README.md` — updated Containers table with network column; updated Installation section; removed stale SECRET_KEY env var; noted backend API docs are not directly accessible from host

changelog/2026-04-17_comprehensive-claude-md.md

@@ -0,0 +1,12 @@
+# 2026-04-17 — Comprehensive CLAUDE.md rewrite
+
+**Timestamp:** 2026-04-17T15:00:00+00:00
+
+## Summary
+
+Full codebase analysis used to rewrite CLAUDE.md with permanent reference material that is loaded into every Claude Code session. All conventions, standards, and structural decisions are now documented in a single authoritative file.
+
+## Files Added / Modified / Deleted
+
+### Modified
+- `CLAUDE.md` — Complete rewrite; added: full file/folder tree, all API endpoints table, database model reference (all columns + constraints), Pydantic schema conventions, security standards section (JWT, bcrypt, sanitization, XSS, SQLi, admin 404 pattern, network isolation, pre-commit hook), frontend patterns (Axios client, TanStack Query key/mutation conventions, route guard docs), naming conventions, HTTP status code reference, default values and limits table, Docker infrastructure (services, volumes, networks, env vars), and reorganised all workflow sections (STATUS.md, changelog, adding resources, git, infra change protocol, security hook)

changelog/2026-04-17_customizable-home-dashboard.md

@@ -0,0 +1,23 @@
+# 2026-04-17 — Customizable home dashboard and settings placeholder
+
+**Timestamp:** 2026-04-17T14:00:00+00:00
+
+## Summary
+
+Replaced the static dashboard page with a per-user customizable home screen. Each user can pin and unpin apps from the available services list. A time-aware greeting shows the user's display name (XSS-safe via React JSX text rendering). The Settings navigation item now routes to a placeholder page.
+
+## Files Added / Modified / Deleted
+
+### Added
+- `backend/alembic/versions/c7e8f9a0b1d2_add_dashboard_app_ids_to_users.py` — Migration adding `dashboard_app_ids` JSON column to `users` table (default `[]`; non-nullable)
+- `frontend/src/pages/SettingsPage.tsx` — Placeholder settings page at `/settings`
+
+### Modified
+- `backend/app/models/user.py` — Added `dashboard_app_ids: Mapped[list]` JSON column
+- `backend/app/schemas/user.py` — Added `DashboardPrefsOut` and `DashboardPrefsUpdate` schemas; `app_ids` validated as safe slugs (regex, max 50, max 64 chars each)
+- `backend/app/routers/users.py` — Added `GET /api/users/me/preferences` and `PATCH /api/users/me/preferences` endpoints
+- `frontend/src/api/client.ts` — Added `DashboardPrefs` interface, `getDashboardPrefs()`, `updateDashboardPrefs()`
+- `frontend/src/pages/DashboardPage.tsx` — Full rewrite: greeting, pinned app cards grid, customize/edit mode with add/remove toggles, save via TanStack Query mutation
+- `frontend/src/App.tsx` — Imported `SettingsPage`, registered `/settings` route
+- `backend/STATUS.md` — Updated Users endpoints table and models table
+- `frontend/STATUS.md` — Added home dashboard section, added `/settings` to routes table

changelog/2026-04-17_groups-and-admin-nav.md

@@ -0,0 +1,39 @@
+# 2026-04-17 — Groups management and Admin navigation split
+
+**Timestamp:** 2026-04-17T12:00:00Z
+
+## Summary
+
+Added a Groups system (backend models, API, migration) and split the Admin sidebar item into an expandable accordion with "Users" and "Groups" sub-navigation points.
+
+## Files Added / Modified / Deleted
+
+### Added
+- `backend/app/models/group.py` — `Group` and `GroupMembership` SQLAlchemy models
+- `backend/app/schemas/group.py` — Pydantic schemas: `GroupCreate`, `GroupUpdate`, `GroupOut`, `GroupDetailOut`, `GroupMemberOut`
+- `backend/app/routers/groups.py` — Admin CRUD endpoints for groups + member add/remove
+- `backend/alembic/versions/a3f9c2d14e87_add_groups_and_group_memberships.py` — Migration creating `groups` and `group_memberships` tables
+- `frontend/src/pages/AdminUsersPage.tsx` — User management page (extracted from AdminPage)
+- `frontend/src/pages/AdminGroupsPage.tsx` — Group management page with inline member panel
+
+### Modified
+- `backend/app/models/__init__.py` — Import `Group` and `GroupMembership`
+- `backend/app/main.py` — Mount `/api/admin/groups` router
+- `frontend/src/api/client.ts` — Added 7 group API functions and TypeScript types
+- `frontend/src/pages/AdminPage.tsx` — Now a simple redirect to `/admin/users`
+- `frontend/src/App.tsx` — Added routes `/admin/users` and `/admin/groups`
+- `frontend/src/components/Sidebar.tsx` — Admin item is now an expandable accordion with Users and Groups sub-items
+- `backend/STATUS.md` — Documented groups endpoints, models, updated future work
+- `frontend/STATUS.md` — Documented new routes, pages, API client functions
+
+---
+
+# 2026-04-17 — Explicit bcrypt work factor
+
+**Timestamp:** 2026-04-17T15:00:00Z
+
+## Summary
+Made the bcrypt cost factor explicit (13 rounds, ~300 ms) in `hash_password` so it is easy to audit and increase over time.
+
+## Files Modified
+- `backend/app/core/security.py` — added `_BCRYPT_ROUNDS = 13`; passed `rounds=` to `bcrypt.gensalt()`

changelog/2026-04-17_service-health-checks.md

@@ -0,0 +1,22 @@
+# 2026-04-17 — Service health checks and dynamic Apps page
+
+**Timestamp:** 2026-04-17T00:00:00Z
+
+## Summary
+
+Added a background health-check system to the backend that polls each registered feature service every 30 seconds. The Apps page now renders dynamically based on live service status — showing "Unavailable" when a service is registered but its container is unreachable.
+
+## Files Added / Modified / Deleted
+
+### Added
+- `backend/app/services/__init__.py` — package init
+- `backend/app/services/service_health.py` — service registry, background polling loop (`POLL_INTERVAL=30s`), `get_all_statuses()` read API
+- `backend/app/routers/services.py` — `GET /api/services` endpoint (requires auth)
+
+### Modified
+- `backend/app/core/config.py` — added `DOC_SERVICE_URL` setting (default `http://doc-service:8001`)
+- `backend/app/main.py` — added FastAPI `lifespan` context manager: registers services, runs initial health check, starts background polling loop; mounts `/api/services` router
+- `frontend/src/api/client.ts` — added `ServiceStatus` interface and `getServices()` API function
+- `frontend/src/pages/AppsPage.tsx` — replaced hardcoded `APPS` array with dynamic query to `GET /api/services`; adds "Unavailable" state with dimmed card and explanation text
+- `backend/STATUS.md` — documented `/api/services` endpoint and health-check architecture
+- `frontend/STATUS.md` — documented dynamic Apps page behaviour

changelog/2026-04-17_switch-penpot-to-figma.md

@@ -0,0 +1,122 @@
+# 2026-04-17 — Switch UX/UI design tool from Penpot to Figma
+
+**Timestamp:** 2026-04-17T00:00:00
+
+**Summary:** Replaced all references to Penpot with Figma across the project. The ux-designer agent now describes Figma REST API integration instead of the Penpot API.
+
+**Files Modified:**
+- `.claude/agents/ux-designer.md` — rewrote Penpot setup checklist and API docs to use Figma (personal access token, `X-Figma-Token` header, `api.figma.com/v1` endpoints)
+- `frontend/STATUS.md` — updated Known limitations and Future work to reference Figma instead of Penpot
+- `TODO.md` — replaced "UX/UI — Penpot setup" section with "UX/UI — Figma setup"
+
+**Files Added:**
+- `changelog/2026-04-17_switch-penpot-to-figma.md` — this entry
+
+---
+
+# 2026-04-17 — Adopt shadcn/ui + Tailwind CSS as UI layer
+
+**Timestamp:** 2026-04-17T00:00:00
+
+**Summary:** Confirmed shadcn/ui + Tailwind CSS as the UI component library and styling system. Updated CLAUDE.md stack table and frontend/STATUS.md to reflect this decision.
+
+**Files Modified:**
+- `CLAUDE.md` — added UI Library (shadcn/ui) and Styling (Tailwind CSS v3) rows to Stack table
+- `frontend/STATUS.md` — marked shadcn/ui checklist item as done; updated Known limitations note from "evaluation pending" to "adoption in progress"
+
+---
+
+# 2026-04-17 — Implement shadcn/ui + Tailwind CSS UI layer
+
+**Timestamp:** 2026-04-17T12:00:00
+
+**Summary:** Full implementation of the shadcn/ui + Tailwind CSS UI layer: design token system, theme context, new LoginPage, AppShell + Sidebar replacing the inline Nav component.
+
+**Files Added:**
+- `frontend/tailwind.config.ts` — Tailwind config with CSS-variable-based design tokens (primary, accent, background, surface, border, foreground, muted)
+- `frontend/postcss.config.js` — PostCSS config (tailwindcss + autoprefixer)
+- `frontend/components.json` — shadcn/ui init config (style: default, baseColor: slate, cssVariables: true)
+- `frontend/src/styles/theme.css` — Tailwind directives + full CSS custom property token set (light/dark mode)
+- `frontend/src/lib/utils.ts` — `cn()` utility (clsx + tailwind-merge)
+- `frontend/src/components/ui/button.tsx` — shadcn/ui Button (default, ghost, outline, destructive variants)
+- `frontend/src/components/ui/input.tsx` — shadcn/ui Input
+- `frontend/src/hooks/useTheme.ts` — theme hook (localStorage + OS preference detection)
+- `frontend/src/components/ThemeToggle.tsx` — sun/moon ghost icon button
+- `frontend/src/components/Sidebar.tsx` — collapsible left sidebar (expanded/collapsed states, NavLinks, admin-only item, user avatar, logout)
+- `frontend/src/components/AppShell.tsx` — layout wrapper (Sidebar + scrollable main)
+
+**Files Modified:**
+- `frontend/package.json` — added lucide-react, clsx, tailwind-merge, class-variance-authority, @radix-ui/react-slot, tailwindcss, autoprefixer, postcss
+- `frontend/vite.config.ts` — added `@/` path alias via fileURLToPath
+- `frontend/src/main.tsx` — import theme.css
+- `frontend/src/App.tsx` — PrivateRoute and AdminRoute now wrap children in AppShell; removed Nav import
+- `frontend/src/pages/LoginPage.tsx` — full redesign: two-column grid (form panel + hero panel), shadcn Input/Button, ThemeToggle
+- `frontend/src/pages/DashboardPage.tsx` — removed Nav, applied Tailwind headings
+- `frontend/src/pages/AppsPage.tsx` — removed Nav
+- `frontend/src/pages/ProfilePage.tsx` — removed Nav
+- `frontend/src/pages/AdminPage.tsx` — removed Nav
+- `frontend/src/pages/DocumentsPage.tsx` — removed Nav
+- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — removed Nav
+- `frontend/src/pages/AIAdminSettingsPage.tsx` — removed Nav
+- `frontend/STATUS.md` — added component inventory table; updated What it is; updated Future work checklist
+
+---
+
+# 2026-04-17 — Per-service system prompts with AI Settings tab view
+
+**Timestamp:** 2026-04-17T12:00:00
+
+**Summary:** Added runtime-editable system prompts per service, stored in each service's config file on the shared volume. The AI Service Settings page now has a tab view (General / System Prompts).
+
+**Files Added / Modified / Deleted**
+
+- `backend/app/core/app_config.py` — Added `DocServiceSystemPrompts` model, updated `DocServiceConfig`, added `load_all_system_prompts`, `save_service_system_prompts`, `SYSTEM_PROMPT_SERVICES` registry
+- `backend/app/routers/settings.py` — Added `SystemPromptUpdate` schema, `GET /system-prompts` and `PATCH /system-prompts/{service_id}` endpoints
+- `features/doc-service/app/services/config_reader.py` — Added `_DEFAULT_SYSTEM_PROMPT`, `_DEFAULT_USER_TEMPLATE`, and `system_prompts` key to `_DEFAULT_CONFIG`
+- `features/doc-service/app/services/ai_client.py` — Loads system prompt and user template dynamically from config at runtime; falls back to defaults
+- `frontend/src/api/client.ts` — Added `ServiceSystemPrompt`, `SystemPromptsData` types, `getSystemPrompts` and `updateSystemPrompt` API functions
+- `frontend/src/pages/AIAdminSettingsPage.tsx` — Refactored to tab view (General | System Prompts); System Prompts tab shows per-service editable textarea cards
+- `features/ai-service/STATUS.md` — Documented system prompts architecture
+- `features/doc-service/STATUS.md` — Documented runtime prompt loading
+
+---
+
+# 2026-04-17 — Apps page: card surface colour + whole-card click
+
+**Timestamp:** 2026-04-17T00:00:00
+
+**Summary:** Cards on the Apps page now render with the `--color-surface` token (distinct from the page background), and clickable cards (status = available + path set) are wrapped in a `<Link>` so the entire frame navigates to the app. The Settings link is unchanged and stops click propagation.
+
+**Files Modified:**
+- `frontend/src/pages/AppsPage.tsx` — Added `cardStyle`/`clickableCardStyle` objects using CSS custom properties; conditionally wraps card in `<Link>` vs `<div>`; removed standalone "Open" button; settings link gains `e.stopPropagation()`
+
+---
+
+# 2026-04-17 — Sidebar app sub-navigation, category filtering, and re-analysis on category creation
+
+**Timestamp:** 2026-04-17T00:00:00
+
+**Summary:** Added expandable Apps section to the sidebar with Documents → categories sub-navigation. Clicking a category filters the documents view. Removed tag UI from the document list (deferred). When a new category is created, similar existing categories are detected and affected documents are re-analysed in the background so the new category surfaces as a pending AI suggestion.
+
+**Files Modified:**
+- `frontend/src/components/Sidebar.tsx` — replaced flat Apps nav item with collapsible accordion; Documents sub-item expands to list all user categories; category links navigate to `/apps/documents?category_id=<id>`
+- `frontend/src/pages/DocumentsPage.tsx` — removed `TagEditor` component; added `useSearchParams` for `category_id` URL param; category filter chip shown in FilterBar with dismiss button; "Clear filters" now also clears category filter
+- `frontend/src/api/client.ts` — added `category_id` field to `DocumentListParams`
+- `features/doc-service/app/routers/documents.py` — added `category_id` query param to `GET /documents`; filters via subquery on `category_assignments`
+- `features/doc-service/app/routers/categories.py` — `POST /documents/categories` now finds similar categories by name (word overlap + SequenceMatcher) and triggers a background task to re-run AI extraction on affected documents, merging new `suggested_categories` into their `extracted_data`
+- `features/doc-service/STATUS.md` — updated endpoints table and filter params table
+- `frontend/STATUS.md` — updated sidebar and documents page sections
+
+---
+
+# 2026-04-17 — Re-analyse button for documents
+
+**Timestamp:** 2026-04-17T00:00:00
+
+**Summary:** Added a Re-analyse button to each document row that re-runs AI extraction on demand. The endpoint resets status to pending, clears any previous error, and enqueues the background processing task. The button is disabled while the document is already pending or processing.
+
+**Files Modified:**
+- `features/doc-service/app/routers/documents.py` — added `POST /documents/{id}/reprocess` endpoint
+- `frontend/src/api/client.ts` — added `reprocessDocument(id)` API function
+- `frontend/src/pages/DocumentsPage.tsx` — added `reprocessMut` mutation and Re-analyse button in document row header
+- `features/doc-service/STATUS.md` — marked reprocess as done, added endpoint to table

changelog/2026-04-18_category-scopes-group-admin.md

@@ -0,0 +1,34 @@
+# 2026-04-18 — Category scopes, group admin role, and permission model
+
+**Timestamp:** 2026-04-18T00:00:00Z
+
+## Summary
+
+Introduces three category scopes (personal / group / system), a PascalCase-with-dashes naming convention, a group-admin role on group memberships, and a full permission model for who can create, rename, and delete categories and documents.
+
+## Files Added
+
+- `backend/alembic/versions/e1f2a3b4c5d6_add_group_member_is_admin.py` — adds `is_group_admin BOOLEAN` to `group_memberships`
+- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — adds `can_delete BOOLEAN` to `document_shares` (backfill from feat/document-delete-permissions)
+- `features/doc-service/alembic/versions/0006_add_category_scope.py` — adds `scope VARCHAR(16)` and `group_id VARCHAR` to `document_categories`; data-migrates watch categories to scope='system'
+
+## Files Modified
+
+- `backend/app/models/group.py` — added `is_group_admin` to `GroupMembership`
+- `backend/app/schemas/group.py` — added `is_group_admin` to `GroupMemberOut`; new `GroupMemberAdminUpdate`
+- `backend/app/schemas/user.py` — added `is_group_admin` to `UserGroupOut`
+- `backend/app/routers/users.py` — `get_my_groups` now joins `GroupMembership` to include `is_group_admin`
+- `backend/app/routers/groups.py` — `get_group` includes `is_group_admin`; new `PATCH /{id}/members/{user_id}/admin` endpoint
+- `backend/app/routers/categories_proxy.py` — injects `x-user-is-admin` and `x-user-admin-groups` headers
+- `backend/app/routers/documents_proxy.py` — injects `x-user-admin-groups` header (was already injecting `x-user-is-admin`)
+- `features/doc-service/app/models/category.py` — added `scope`, `group_id` columns
+- `features/doc-service/app/schemas/category.py` — `CategoryOut` includes `scope`/`group_id`; `CategoryCreate` accepts `group_id`
+- `features/doc-service/app/deps.py` — added `get_user_is_admin`, `get_user_admin_groups`
+- `features/doc-service/app/routers/categories.py` — full rewrite: name validation regex, scope-based list/create, `_check_can_manage_cat` permission helper, scope-aware rename/delete
+- `features/doc-service/app/routers/documents.py` — `delete_document` enforces is_admin/can_delete/group-admin hierarchy; `remove_category` requires doc ownership; `assign_category` accepts group/system categories
+- `frontend/src/api/client.ts` — `CategoryOut` gains `scope`/`group_id`; `createCategory` accepts optional `groupId`; `UserGroupOut`/`GroupMemberOut` gain `is_group_admin`; new `adminSetGroupMemberAdmin()`; `ApiError` exported
+- `frontend/src/components/ManageCategoriesDialog.tsx` — categories grouped by scope; lock icons for unmanageable categories; rename/delete gated by scope permissions; inline rename error display
+- `frontend/src/components/SourcePanel.tsx` — categories shown in sections (Mine / Group name / System); scope picker on new category form; client-side name validation
+- `frontend/src/pages/AdminGroupsPage.tsx` — group admin checkbox column in members table
+- `backend/CLAUDE.md` — updated `group_memberships` model, migration chain, endpoints
+- `features/doc-service/CLAUDE.md` — updated `document_categories` model, `document_shares` model, migration chain, deps note

changelog/2026-04-18_doc-service-redesign.md

@@ -0,0 +1,43 @@
+# 2026-04-18 — Doc Service Redesign: Scalable UX + Group-Based Sharing
+
+**Timestamp:** 2026-04-18T00:00:00+00:00
+
+## Summary
+
+Complete redesign of the document management UX for scale (10 → 100 000 documents, 2 → 1 000 categories) and group-based document sharing. Replaced the monolithic DocumentsPage with a three-column layout (Sidebar + SourcePanel + main), a slide-over detail panel, a filter chip system, multi-file upload queue, and bulk actions. Added the full backend sharing stack: `document_shares` table, share CRUD endpoints, a shared-with-me view, and X-User-Groups header injection in the gateway proxy.
+
+---
+
+## Files Added
+
+| File | Description |
+|------|-------------|
+| `features/doc-service/app/models/document_share.py` | DocumentShare ORM model (document_id, group_id, shared_by_user_id) |
+| `features/doc-service/app/schemas/share.py` | DocumentShareOut, DocumentShareCreate, SharedDocumentOut schemas |
+| `features/doc-service/alembic/versions/0004_add_document_shares.py` | Migration creating document_shares table with indexes |
+| `frontend/src/components/SourcePanel.tsx` | Left panel (240px): views (All/Mine/Shared) + searchable category tree + new category form |
+| `frontend/src/components/ManageCategoriesDialog.tsx` | Category CRUD modal (inline rename, delete with confirm, search) |
+| `frontend/src/components/DocumentSlideOver.tsx` | Right slide-over (480px): metadata, inline title edit, type picker, AI suggestions, categories combobox, tags, sharing section, raw text, actions |
+
+---
+
+## Files Modified
+
+| File | Change |
+|------|--------|
+| `features/doc-service/app/models/__init__.py` | Import DocumentShare |
+| `features/doc-service/app/deps.py` | Added `get_user_groups` dependency (reads X-User-Groups header) |
+| `features/doc-service/app/schemas/document.py` | Added `share_count: int = 0` to DocumentOut |
+| `features/doc-service/app/routers/documents.py` | Complete rewrite: added share CRUD endpoints, shared-with-me endpoint, N+1-safe share_count, recipient download access, X-User-Groups enforcement |
+| `features/doc-service/STATUS.md` | Added sharing section, migration 0004, updated future work |
+| `backend/app/routers/documents_proxy.py` | Injects X-User-Groups header (queries GroupMembership per request) |
+| `backend/app/routers/categories_proxy.py` | Same X-User-Groups injection pattern |
+| `backend/app/routers/users.py` | Added GET /me/groups endpoint |
+| `backend/app/schemas/user.py` | Added UserGroupOut schema |
+| `backend/STATUS.md` | Added /me/groups endpoint |
+| `frontend/src/api/client.ts` | Added share_count to DocumentOut, SharedDocumentOut, DocumentShareOut, listSharedWithMe, getDocumentShares, addDocumentShare, removeDocumentShare, getMyGroups |
+| `frontend/src/components/Sidebar.tsx` | Removed per-category NavLinks; Documents is now a single NavLink under Apps |
+| `frontend/src/components/AppShell.tsx` | Renders SourcePanel between Sidebar and main on /apps/documents route |
+| `frontend/src/pages/DocumentsPage.tsx` | Complete rewrite: three-panel layout, view param (all/mine/shared), smart polling, drag-and-drop, multi-file upload queue, filter chip system, bulk actions bar |
+| `frontend/STATUS.md` | Complete rewrite reflecting all new components and patterns |
+| `CLAUDE.md` | Updated file tree, Database Models (DocumentShare), Migration chains (0004), API endpoints (shares, shared-with-me, /me/groups), TanStack Query keys, request flow diagram |

changelog/2026-04-18_document-delete-permissions.md

@@ -0,0 +1,25 @@
+# 2026-04-18 — Document delete permissions + three-dots menu fix
+
+**Timestamp:** 2026-04-18T00:00:00Z
+
+## Summary
+
+Added a proper permission model for document deletion: owners and admins can always delete; group members can delete only when the share was explicitly granted `can_delete=true`. Fixed silent delete failures (watch docs returning 404 with no user feedback) and fixed the three-dots context menu being clipped by `overflow-hidden` on the table container.
+
+## Files Added / Modified / Deleted
+
+### Added
+- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — migration: adds `can_delete BOOLEAN NOT NULL DEFAULT false` to `document_shares`
+
+### Modified
+- `features/doc-service/app/models/document_share.py` — added `can_delete: Mapped[bool]` column
+- `features/doc-service/app/schemas/share.py` — added `can_delete` to `DocumentShareOut` and `DocumentShareCreate`; added `viewer_can_delete` to `SharedDocumentOut`
+- `features/doc-service/app/schemas/document.py` — added `viewer_can_delete: bool = False` to `DocumentOut`
+- `features/doc-service/app/deps.py` — added `get_user_is_admin()` dep reading `x-user-is-admin` header
+- `features/doc-service/app/routers/documents.py` — added `_get_deletable_doc_ids()` helper; updated list/get/delete endpoints with permission logic; updated `add_share` to store `can_delete`; updated shared-with-me to include `viewer_can_delete`
+- `backend/app/routers/documents_proxy.py` — `_forward_headers()` now injects `x-user-is-admin` header
+- `frontend/src/api/client.ts` — `DocumentOut`: added `viewer_can_delete`; `DocumentShareOut`: added `can_delete`; `addDocumentShare`: accepts `canDelete` param
+- `frontend/src/pages/DocumentsPage.tsx` — `RowActionsMenu`: replaced absolute dropdown with `createPortal` to fix clipping; delete button now uses `doc.viewer_can_delete`; added `onError` handler for silent failures
+- `frontend/src/components/DocumentSlideOver.tsx` — sharing section: shows trash icon badge on shares with `can_delete=true`; added "Allow group members to delete" checkbox before group picker; delete button uses `doc.viewer_can_delete`
+- `features/doc-service/CLAUDE.md` — updated `document_shares` table docs + migration chain
+- `backend/CLAUDE.md` — noted `x-user-is-admin` header injection

changelog/2026-04-18_plugin-arch-watch-feature.md

@@ -0,0 +1,44 @@
+# 2026-04-18 — Generic Plugin Architecture + Watch Directory Feature
+
+**Timestamp:** 2026-04-18T00:00:00Z
+
+## Summary
+
+Implemented a generic plugin/extension infrastructure that allows feature containers to self-describe their settings via a manifest contract, with no feature-specific code required in the backend or frontend. Built the watch-directory feature entirely inside the doc-service container as the first plugin consumer.
+
+## Files Added
+
+| File | Description |
+|------|-------------|
+| `backend/app/routers/plugins.py` | Generic plugin proxy: `GET/PATCH /api/plugins`, `/api/plugins/{id}/manifest`, `/api/plugins/{id}/settings` |
+| `frontend/src/components/PluginSchemaForm.tsx` | JSON Schema → React form renderer (boolean/string/number/readOnly) |
+| `frontend/src/pages/PluginSettingsPage.tsx` | Generic plugin settings page driven by manifest |
+| `features/doc-service/app/routers/plugin.py` | Doc-service plugin endpoints: `/plugin/manifest`, `/plugin/settings` |
+| `features/doc-service/app/services/file_watcher.py` | watchdog-based PDF watcher with startup scan, folder-to-category mapping, no-remove policy |
+| `features/doc-service/alembic/versions/0003_add_watch_columns.py` | Migration: source, watch_path, suggested_folder, suggested_filename |
+| `dev-watch/.gitkeep` | Dev bind-mount directory for local file watcher testing |
+
+## Files Modified
+
+| File | Description |
+|------|-------------|
+| `backend/app/services/service_health.py` | Also fetches and caches `/plugin/manifest` from healthy services |
+| `backend/app/deps.py` | Added `check_plugin_access(plugin_id, user, db)` helper |
+| `backend/app/main.py` | Mounted `/api/plugins` router |
+| `frontend/src/api/client.ts` | Added plugin API functions and suggestion confirm/reject functions; extended `DocumentOut` with new fields |
+| `frontend/src/components/Sidebar.tsx` | Added dynamic "Extensions" section populated from `/api/plugins` |
+| `frontend/src/App.tsx` | Added `/settings/plugins/:id` route |
+| `features/doc-service/app/models/document.py` | Added 4 new columns: source, watch_path, suggested_folder, suggested_filename |
+| `features/doc-service/app/schemas/document.py` | Exposed 4 new fields in `DocumentOut` |
+| `features/doc-service/app/services/config_reader.py` | Added storage config defaults, `get_storage_config()`, `save_storage_config()` |
+| `features/doc-service/app/routers/documents.py` | Watch-user visibility (`OR user_id = "watch"`); 4 suggestion endpoints |
+| `features/doc-service/app/routers/categories.py` | Watch-owned categories included in list |
+| `features/doc-service/app/main.py` | Lifespan watcher start/stop; plugin router mounted |
+| `features/doc-service/pyproject.toml` | Added `watchdog>=4.0` |
+| `features/doc-service/Dockerfile` | Pre-create `/data/watch` |
+| `docker-compose.yml` | Added `watch_data` named volume; mounted to doc-service |
+| `docker-compose.dev.yml` | Dev bind-mount `./dev-watch:/data/watch` |
+| `CLAUDE.md` | Updated all affected sections (models, migrations, endpoints, routes, tree, query keys, volumes) |
+| `backend/STATUS.md` | Plugin system section added |
+| `features/doc-service/STATUS.md` | Watch feature, plugin endpoints, migration 0003, updated architecture diagram |
+| `frontend/STATUS.md` | Extensions sidebar, PluginSchemaForm, PluginSettingsPage, new API functions |

changelog/2026-04-18_service-admin-groups-combined-settings.md

@@ -0,0 +1,24 @@
+# 2026-04-18 — Service admin groups + combined settings pages
+
+**Timestamp:** 2026-04-18T00:00:00Z
+
+## Summary
+
+Introduced per-service admin groups that are auto-created at startup, consolidated doc-service and AI-service settings each onto a single page, and collapsed the dual "Settings + Extension" app card buttons into one Settings button visible to admins and service-group members.
+
+## Files Added
+
+- `backend/app/services/group_bootstrap.py` — Idempotent startup task: creates `{service_id}-admin` group for every registered service if absent.
+- `features/ai-service/app/routers/plugin.py` — `GET /plugin/manifest` for ai-service (exposes access rules: `ai-service-admin` group).
+- `frontend/src/pages/DocServiceSettingsPage.tsx` — Combined doc-service settings page: Upload Limits + Watch Directory (rendered via `PluginSchemaForm`).
+
+## Files Modified
+
+- `backend/app/main.py` — Lifespan now calls `ensure_service_admin_groups(db)` after `register_services()`.
+- `backend/app/deps.py` — Added `get_service_admin(service_id)` factory dependency: grants access to superusers or `{service_id}-admin` group members; returns 404 otherwise.
+- `backend/app/routers/settings.py` — AI settings (`/ai`, `/ai/test`, `/system-prompts`) and doc limits (`/documents/limits`) now use `get_service_admin(...)` instead of `get_current_admin` — service group members can access them.
+- `backend/app/services/service_health.py` — `settings_path` for doc-service changed to `/apps/documents/settings`; ai-service to `/apps/ai/settings` (removed `/admin` suffix).
+- `features/ai-service/app/main.py` — Mounts new `plugin.router` so backend poller can discover ai-service manifest.
+- `frontend/src/App.tsx` — Added `ServiceAdminRoute` component (checks token + is_admin OR plugin list contains serviceId). Updated doc/AI settings routes to new paths under `ServiceAdminRoute`.
+- `frontend/src/pages/AppsPage.tsx` — Replaced two-button layout (Settings + Extension) with single Settings button; visible when `user.is_admin || pluginIds.has(svc.id)`.
+- `backend/STATUS.md`, `frontend/STATUS.md`, `CLAUDE.md` — Updated to reflect all changes above.

changelog/2026-04-19_checklist-update-delete-bug-fix.md

@@ -0,0 +1,12 @@
+# 2026-04-19 — Merge checklist update + admin delete bug fix
+
+**Timestamp:** 2026-04-19T00:15:00Z
+
+## Summary
+
+Updated `tests/MERGE_CHECKLIST.md` with all new tests for the two recently merged features (document delete permissions and category scopes / group-admin role). While running the new test 12.16b, discovered and fixed a bug where the doc-service delete endpoint returned 404 for admins deleting non-owned documents.
+
+## Files Added / Modified / Deleted
+
+- **Modified** `tests/MERGE_CHECKLIST.md` — added 18 new tests: 4.9–4.10 (group admin role), 12.16b–12.16e (delete permissions), 13.11–13.14 (can_delete sharing), 14.7–14.17 (category scopes, PascalCase naming), 19.11 (three-dots portal fix); updated 12.16 and 14.5 descriptions
+- **Modified** `features/doc-service/app/routers/documents.py` — fixed `delete_document` to bypass group-membership filter for admins; previously admins got 404 on any document they didn't own or that wasn't a watch doc

changelog/2026-04-20_storage-service.md

@@ -0,0 +1,60 @@
+# 2026-04-20 — Dedicated storage-service with pluggable backends
+
+**Timestamp:** 2026-04-20T00:00:00Z
+
+## Summary
+
+Introduced a dedicated `storage-service` container (port 8020) as the single file/blob persistence layer for the entire stack. All services now route file and config I/O through this service's HTTP API. The service supports pluggable storage backends (local filesystem by default; S3-compatible and WebDAV built in) with a zero-data-loss migration flow. The `doc_data` and `app_config` Docker volumes were removed.
+
+## Files Added
+
+- `features/storage-service/app/main.py` — FastAPI app, lifespan (backend init)
+- `features/storage-service/app/core/config.py` — Settings (DATA_DIR, STORAGE_BACKEND, S3_*, WEBDAV_*)
+- `features/storage-service/app/routers/health.py` — GET /health
+- `features/storage-service/app/routers/objects.py` — PUT/GET/DELETE /objects/{bucket}/{key:path}, GET /objects/{bucket}
+- `features/storage-service/app/routers/migrate.py` — POST/GET/DELETE /migrate, PATCH /backend-config
+- `features/storage-service/app/services/backend_manager.py` — Driver factory, singleton, atomic switch
+- `features/storage-service/app/services/migration.py` — Async migration: copy → verify → switch → cleanup
+- `features/storage-service/app/services/backends/base.py` — AbstractStorageBackend ABC
+- `features/storage-service/app/services/backends/local.py` — LocalFSBackend (path traversal guard)
+- `features/storage-service/app/services/backends/s3.py` — S3Backend (aiobotocore, endpoint_url configurable)
+- `features/storage-service/app/services/backends/webdav.py` — WebDAVBackend (aiohttp + defusedxml)
+- `features/storage-service/scripts/start.sh` — prod uvicorn start
+- `features/storage-service/scripts/start_dev.sh` — dev uvicorn --reload start
+- `features/storage-service/pyproject.toml` — Dependencies
+- `features/storage-service/Dockerfile` — python:3.12-slim, non-root user 1001, port 8020
+- `features/storage-service/CLAUDE.md` — API reference, bucket docs, driver docs
+- `features/storage-service/STATUS.md` — Service status
+- `backend/app/core/config_storage.py` — Thin async helpers: read_json/write_json/delete_key/list_keys
+- `backend/app/routers/storage_config.py` — Admin proxy endpoints for storage config + migration
+- `features/doc-service/alembic/versions/0008_rename_file_path_to_storage_key.py` — DB migration
+- `frontend/src/pages/StorageAdminPage.tsx` — Admin UI: backend status, driver form, migration progress
+- `tests/storage-service_tests.md` — §20 storage-service test suite
+
+## Files Modified
+
+- `docker-compose.yml` — Added storage-service, storage_data volume; removed doc_data, app_config; added depends_on service_healthy
+- `docker-compose.dev.yml` — Added storage-service dev override
+- `backend/app/core/config.py` — Added STORAGE_SERVICE_URL
+- `backend/app/core/app_config.py` — Full async rewrite using config_storage HTTP helpers (no filesystem)
+- `backend/app/routers/settings.py` — Removed all asyncio.to_thread wrappers; direct await calls
+- `backend/app/main.py` — Register storage_config router; update register_services call
+- `backend/app/services/service_health.py` — Register storage-service
+- `features/doc-service/app/core/config.py` — Added STORAGE_SERVICE_URL
+- `features/doc-service/app/models/document.py` — file_path → storage_key
+- `features/doc-service/app/services/storage.py` — Complete rewrite: HTTP client calls to storage-service
+- `features/doc-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
+- `features/doc-service/app/services/file_watcher.py` — Uses save_upload() → storage-service
+- `features/doc-service/app/routers/documents.py` — storage_key refs, pdfplumber(io.BytesIO), streaming from storage-service
+- `features/ai-service/app/core/config.py` — Added STORAGE_SERVICE_URL; removed CONFIG_PATH
+- `features/ai-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
+- `frontend/src/api/client.ts` — Added StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions
+- `frontend/src/App.tsx` — Added /admin/storage route (AdminRoute → StorageAdminPage)
+- `tests/ALL_TESTS.md` — Updated to 20 feature areas; added §20 storage-service tests
+- `CLAUDE.md` — Added storage-service to Services/Volumes/Networks tables; storage enforcement rule; §20 test file
+- `backend/CLAUDE.md` — Added config_storage.py, storage_config.py to tree; added admin storage endpoints
+- `frontend/CLAUDE.md` — Added StorageAdminPage to tree; added /admin/storage route
+- `features/doc-service/CLAUDE.md` — Updated storage.py description; file_path → storage_key; added migration 0008
+- `features/ai-service/CLAUDE.md` — Added config_reader.py description
+- `backend/STATUS.md` — Added storage-config endpoints; updated settings persistence note
+- `frontend/STATUS.md` — Added /admin/storage route; added StorageAdminPage description

dev-watch/.gitkeep

@@ -0,0 +1 @@
+# Watch directory for development testing

docker-compose.dev.yml

@@ -22,3 +22,21 @@ services:
     volumes:
       - ./frontend:/app
       - /app/node_modules
+
+  ai-service:
+    command: sh scripts/start_dev.sh
+    env_file: ./features/ai-service/.env   # gitignored — holds LM Studio / AI credentials
+    volumes:
+      - ./features/ai-service:/app
+
+  storage-service:
+    command: sh scripts/start_dev.sh
+    volumes:
+      - ./features/storage-service:/app
+
+  doc-service:
+    command: sh scripts/start_dev.sh
+    env_file: ./features/doc-service/.env
+    volumes:
+      - ./features/doc-service:/app
+      - ./dev-watch:/data/watch   # bind-mount local folder for easy testing

docker-compose.yml

@@ -9,8 +9,6 @@ services:
       POSTGRES_USER: ${POSTGRES_USER:-postgres}
       POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
       POSTGRES_DB: ${POSTGRES_DB:-destroying_sap}
-    ports:
-      - "5432:5432"
     volumes:
       - postgres_data:/var/lib/postgresql/data
     healthcheck:
@@ -18,6 +16,29 @@ services:
       interval: 5s
       timeout: 5s
       retries: 10
+    networks:
+      - backend-net
+
+  # ── Storage service (unified blob storage) ──────────────────────────────────
+  storage-service:
+    build:
+      context: ./features/storage-service
+      dockerfile: Dockerfile
+      network: host
+    user: "1001:1001"
+    restart: unless-stopped
+    environment:
+      STORAGE_BACKEND: local
+      DATA_DIR: /data/storage
+    volumes:
+      - storage_data:/data/storage
+    healthcheck:
+      test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8020/health')\""]
+      interval: 10s
+      timeout: 5s
+      retries: 5
+    networks:
+      - backend-net
 
   # ── Backend (management) ────────────────────────────────────────────────────
   backend:
@@ -30,11 +51,56 @@ services:
     env_file: ./backend/.env
     environment:
       DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
-    ports:
-      - "8000:8000"
+      DOC_SERVICE_URL: http://doc-service:8001
+      AI_SERVICE_URL: http://ai-service:8010
+      STORAGE_SERVICE_URL: http://storage-service:8020
     depends_on:
       db:
         condition: service_healthy
+      storage-service:
+        condition: service_healthy
+    networks:
+      - backend-net
+
+  # ── AI service (shared AI provider intermediary) ─────────────────────────────
+  ai-service:
+    build:
+      context: ./features/ai-service
+      dockerfile: Dockerfile
+      network: host
+    user: "1001:1001"
+    restart: unless-stopped
+    environment:
+      STORAGE_SERVICE_URL: http://storage-service:8020
+    depends_on:
+      storage-service:
+        condition: service_healthy
+    networks:
+      - backend-net
+
+  # ── Doc service (PDF extraction) ────────────────────────────────────────────
+  doc-service:
+    build:
+      context: ./features/doc-service
+      dockerfile: Dockerfile
+      network: host
+    user: "1001:1001"
+    restart: unless-stopped
+    environment:
+      DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
+      AI_SERVICE_URL: http://ai-service:8010
+      STORAGE_SERVICE_URL: http://storage-service:8020
+    volumes:
+      - watch_data:/data/watch
+    depends_on:
+      db:
+        condition: service_healthy
+      ai-service:
+        condition: service_started
+      storage-service:
+        condition: service_healthy
+    networks:
+      - backend-net
 
   # ── Frontend (UI) ────────────────────────────────────────────────────────────
   frontend:
@@ -48,6 +114,18 @@ services:
       - "80:8080"
     depends_on:
       - backend
+    networks:
+      - backend-net
+      - frontend-net
 
 volumes:
   postgres_data:
+  storage_data:  # All file/blob storage — managed by storage-service (documents + config)
+  watch_data:    # Watch directory — bind-mount your NAS/Nextcloud here via docker-compose.override.yml
+
+networks:
+  # backend-net: db ↔ backend ↔ doc-service. No host ports bound.
+  # internal:true removed — doc-service needs outbound access for cloud AI providers.
+  backend-net:
+  # External-facing: only the frontend binds a host port through this network.
+  frontend-net:

features/ai-service/.env.example

@@ -0,0 +1,12 @@
+AI_PROVIDER=lmstudio
+
+LMSTUDIO_BASE_URL=http://host.docker.internal:1234/v1
+LMSTUDIO_API_KEY=your-lmstudio-api-key
+LMSTUDIO_MODEL=local-model
+
+OLLAMA_BASE_URL=http://host.docker.internal:11434/v1
+OLLAMA_MODEL=llama3.2
+OLLAMA_API_KEY=ollama
+
+ANTHROPIC_API_KEY=sk-ant-your-key-here
+ANTHROPIC_MODEL=claude-haiku-4-5-20251001

features/ai-service/CLAUDE.md

@@ -0,0 +1,51 @@
+# ai-service — Claude context
+
+AI provider intermediary, port 8010 (internal only — never proxied to the browser). Accepts chat requests from `doc-service` (and potentially other callers). Manages a priority queue and abstracts over multiple AI providers (Anthropic, Ollama/LM Studio). See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
+
+---
+
+## File & Folder Tree
+
+```
+features/ai-service/
+├── app/
+│   ├── main.py                    ← FastAPI, queue worker lifespan
+│   ├── core/
+│   │   └── config.py              ← Settings via pydantic-settings
+│   ├── providers/
+│   │   ├── base.py                ← AIProvider abstract class
+│   │   ├── anthropic_provider.py  ← Anthropic API integration
+│   │   └── openai_compat.py       ← Ollama / LM Studio compatibility
+│   ├── routers/
+│   │   ├── chat.py                ← POST /chat (sync, NORMAL priority queue)
+│   │   ├── health.py              ← GET /health
+│   │   ├── queue.py               ← GET /queue/status, /pause, /resume, /cancel/{id}
+│   │   └── plugin.py              ← GET /plugin/manifest (access rules for ai-service-admin group)
+│   └── services/
+│       ├── queue.py               ← Priority queue (CRITICAL > HIGH > NORMAL)
+│       └── config_reader.py       ← Reads ai_service_config.json from storage-service config bucket (30 s TTL cache)
+├── Dockerfile                     ← python:3.12-slim, non-root user 1001
+└── STATUS.md
+```
+
+---
+
+## API Endpoints (internal only)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/chat` | Chat request (queued at NORMAL priority) |
+| GET | `/health` | Health check |
+| GET | `/queue/status` | Queue state |
+| POST | `/queue/pause` | Pause queue |
+| POST | `/queue/resume` | Resume queue |
+| POST | `/queue/cancel/{job_id}` | Cancel job |
+| GET | `/plugin/manifest` | Plugin manifest (access rules for ai-service-admin group) |
+
+These endpoints are only reachable on `backend-net`. The backend does not expose them to the browser.
+
+---
+
+## Note on timeout and retry configuration
+
+Caller-side timeout and retry settings live in `features/doc-service/app/services/ai_client.py` — see `features/doc-service/CLAUDE.md` for the values.

features/ai-service/Dockerfile

@@ -0,0 +1,32 @@
+# ── Stage 1: dependency installation ─────────────────────────────────────────
+FROM python:3.12-slim AS builder
+
+WORKDIR /app
+
+RUN pip install --upgrade pip
+
+COPY pyproject.toml .
+RUN pip install --prefix=/install .
+
+# ── Stage 2: runtime ──────────────────────────────────────────────────────────
+FROM python:3.12-slim
+
+# Create non-root user (UID/GID 1001)
+RUN groupadd --gid 1001 appuser && \
+    useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
+
+# No filesystem directories needed — all config goes through storage-service.
+
+WORKDIR /app
+
+COPY --from=builder /install /usr/local
+
+COPY --chown=appuser:appuser app ./app
+COPY --chown=appuser:appuser scripts ./scripts
+RUN chmod +x scripts/start.sh scripts/start_dev.sh
+
+USER appuser
+
+EXPOSE 8010
+
+CMD ["sh", "scripts/start.sh"]

features/ai-service/STATUS.md

@@ -0,0 +1,118 @@
+# AI Service — Status
+
+## What it is
+
+Shared AI intermediary container. All feature containers (doc-service, future services) POST prompts here. It routes requests to the configured model (LM Studio / Ollama / Anthropic) and returns a normalised response. It is **stateless** — no database, no conversation history. History and context are the caller's responsibility.
+
+Port: `8010` (internal only, not exposed to host).
+
+---
+
+## Current functionality
+
+### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/chat` | Synchronous chat: submits at NORMAL priority, blocks until done |
+| `GET` | `/health` | `{"status": "ok"}` |
+| `GET` | `/health/provider` | Active provider name, model, configured flag |
+| `POST` | `/queue/jobs` | Async enqueue — returns `job_id` immediately |
+| `GET` | `/queue/jobs/{id}` | Poll job: status, position, result, error |
+| `DELETE` | `/queue/jobs/{id}` | Cancel a pending job |
+| `GET` | `/queue/status` | Worker state: running, paused, queue_size, current_job_id |
+| `POST` | `/queue/pause` | Finish current job, stop picking new ones |
+| `POST` | `/queue/resume` | Unpause |
+| `POST` | `/queue/start` | Start (or restart) the worker task |
+| `POST` | `/queue/stop` | Stop worker (pending jobs stay queued) |
+
+### Priority queue
+
+- Three levels: `high` (1) > `normal` (3) > `low` (5)
+- FIFO within same priority level (monotonic sequence counter)
+- Single async worker — one LLM call at a time
+- Pause / resume / start / stop without restarting the container
+- `POST /chat` is a synchronous wrapper: enqueues at NORMAL, awaits the future
+
+### Providers
+
+| Provider | Protocol | SDK |
+|----------|----------|-----|
+| LM Studio | OpenAI-compatible HTTP | openai |
+| Ollama | OpenAI-compatible HTTP | openai |
+| Anthropic | Anthropic API (HTTPS) | anthropic |
+
+Active provider is selected by `"provider"` key in `/config/ai_service_config.json` (shared Docker volume), with env var overrides for dev.
+
+### Configuration (env var overrides)
+
+```
+AI_PROVIDER          lmstudio | ollama | anthropic
+LMSTUDIO_BASE_URL    http://host.docker.internal:1234/v1
+LMSTUDIO_API_KEY     sk-lm-…
+LMSTUDIO_MODEL       gemma-4-e4b-it          ← current
+OLLAMA_BASE_URL / OLLAMA_MODEL / OLLAMA_API_KEY
+ANTHROPIC_API_KEY / ANTHROPIC_MODEL
+```
+
+Credentials live in `features/ai-service/.env` (gitignored).
+
+### Error codes
+
+| Code | Meaning |
+|------|---------|
+| 422 | Bad request (empty messages, unknown priority) |
+| 502 | Provider connection / API error |
+| 503 | Provider not configured / unknown provider |
+| 504 | Provider timeout |
+
+---
+
+## Architecture
+
+```
+Callers (doc-service, future services)
+    │
+    └─▶ POST /chat (sync)         ─┐
+    └─▶ POST /queue/jobs (async)  ─┤
+                                   ▼
+                        asyncio.PriorityQueue
+                        (HIGH=1, NORMAL=3, LOW=5)
+                                   │
+                        QueueWorker (single task)
+                                   │
+                        execute_chat(request)
+                                   │
+                        Provider SDK (openai / anthropic)
+                                   │
+                        LM Studio / Ollama / Anthropic API
+```
+
+---
+
+## Known limitations / not implemented
+
+- **TLS to LM Studio** — communication is plain HTTP (`http://host.docker.internal:1234`). Deferred until LM Studio HTTPS configuration is confirmed. When ready: set `LMSTUDIO_BASE_URL=https://...` and optionally add `ssl_verify` + `ca_bundle` config keys to the OpenAI-compat provider.
+- **True preemption** — a HIGH job arriving while a LOW job is processing will be next in queue but will not interrupt the running inference.
+- **Queue persistence** — the in-memory queue is lost on container restart. Pending jobs are not persisted to disk.
+- **Authentication on queue endpoints** — `/queue/*` management endpoints have no auth guard. Should be protected before any public/multi-tenant deployment (internal network is the only current protection).
+- **Streaming responses** — `/chat` returns the full response after generation. Streaming (Server-Sent Events) not implemented.
+- **Metrics / observability** — no Prometheus metrics, no structured request logging per job.
+
+---
+
+## System prompts
+
+Each feature service (doc-service, future services) owns its own system prompt, stored in that service's config JSON on the shared volume. The backend settings API (`GET/PATCH /api/settings/system-prompts`) aggregates and edits them. The AI Service Settings UI exposes a **System Prompts** tab for editing all registered service prompts at runtime.
+
+---
+
+## Future work
+
+- [ ] TLS support for LM Studio / Ollama (`ssl_verify`, `ca_bundle` config)
+- [ ] Auth guard on queue management endpoints (admin token or internal-only route)
+- [ ] Streaming responses via SSE (`POST /chat/stream`)
+- [ ] Queue persistence (SQLite or Redis-backed) so jobs survive restarts
+- [ ] Job result TTL / cleanup (currently jobs accumulate in `_jobs` dict indefinitely)
+- [ ] Per-caller priority override (e.g. doc-service background jobs = LOW, user-triggered = NORMAL)
+- [ ] Metrics endpoint (`/metrics`) for queue depth, job latency, provider error rate

features/ai-service/app/core/config.py

@@ -0,0 +1,11 @@
+from pydantic_settings import BaseSettings
+
+
+class Settings(BaseSettings):
+    PROJECT_NAME: str = "ai-service"
+    STORAGE_SERVICE_URL: str = "http://storage-service:8020"
+
+    model_config = {"env_file": ".env", "extra": "ignore"}
+
+
+settings = Settings()

features/ai-service/app/main.py

@@ -0,0 +1,36 @@
+import logging
+from contextlib import asynccontextmanager
+
+from fastapi import FastAPI
+
+from app.core.config import settings
+from app.routers import chat, health, plugin
+from app.routers import queue as queue_router
+from app.services.config_reader import load_ai_config
+from app.services.queue import queue_service
+
+logger = logging.getLogger("ai-service")
+
+
+@asynccontextmanager
+async def lifespan(app: FastAPI):
+    config = await load_ai_config()
+    provider = config.get("provider", "lmstudio")
+    model = config.get(provider, {}).get("model", "unknown")
+    logger.info("[ai-service] active provider: %s  model: %s", provider, model)
+
+    queue_service.start()
+    logger.info("[ai-service] queue worker started")
+
+    yield
+
+    queue_service.stop()
+    logger.info("[ai-service] queue worker stopped")
+
+
+app = FastAPI(title=settings.PROJECT_NAME, lifespan=lifespan)
+
+app.include_router(chat.router, tags=["chat"])
+app.include_router(health.router, tags=["health"])
+app.include_router(queue_router.router)
+app.include_router(plugin.router, tags=["plugin"])

features/ai-service/app/providers/__init__.py

@@ -0,0 +1,20 @@
+from app.providers.base import AIProvider
+
+
+def get_provider(ai_config: dict) -> AIProvider:
+    """Return an AIProvider instance for the active provider in the config."""
+    provider_name = ai_config.get("provider", "lmstudio")
+    provider_cfg = ai_config.get(provider_name, {})
+
+    match provider_name:
+        case "anthropic":
+            from app.providers.anthropic_provider import AnthropicProvider
+            return AnthropicProvider(provider_cfg)
+        case "ollama" | "lmstudio":
+            from app.providers.openai_compat import OpenAICompatProvider
+            return OpenAICompatProvider(provider_cfg, provider_name=provider_name)
+        case _:
+            raise ValueError(f"Unknown AI provider: {provider_name!r}")
+
+
+__all__ = ["AIProvider", "get_provider"]

features/ai-service/app/providers/anthropic_provider.py

@@ -0,0 +1,54 @@
+import asyncio
+
+import anthropic
+
+from app.providers.base import AIProvider
+from app.schemas.chat import ChatMessage
+
+
+class AnthropicProvider(AIProvider):
+    def __init__(self, config: dict) -> None:
+        self._client = anthropic.AsyncAnthropic(api_key=config.get("api_key", ""))
+        self.model_name = config.get("model", "claude-haiku-4-5-20251001")
+        self.provider_name = "anthropic"
+
+    async def chat(
+        self,
+        messages: list[ChatMessage],
+        max_tokens: int,
+        temperature: float,
+    ) -> tuple[str, int | None, int | None]:
+        # Anthropic uses a top-level `system=` param, not a role in the messages array
+        system_content = ""
+        user_messages = []
+        for msg in messages:
+            if msg.role == "system":
+                system_content += msg.content + "\n"
+            else:
+                user_messages.append({"role": msg.role, "content": msg.content})
+
+        try:
+            response = await self._client.messages.create(
+                model=self.model_name,
+                max_tokens=max_tokens,
+                temperature=temperature,
+                system=system_content.strip() or anthropic.NOT_GIVEN,
+                messages=user_messages,
+            )
+        except anthropic.APIConnectionError as exc:
+            raise ProviderConnectionError(str(exc)) from exc
+        except anthropic.APITimeoutError as exc:
+            raise ProviderTimeoutError(str(exc)) from exc
+        except anthropic.APIStatusError as exc:
+            raise ProviderConnectionError(f"Anthropic API error {exc.status_code}: {exc.message}") from exc
+
+        content = response.content[0].text
+        return content, response.usage.input_tokens, response.usage.output_tokens
+
+
+class ProviderConnectionError(Exception):
+    pass
+
+
+class ProviderTimeoutError(Exception):
+    pass

features/ai-service/app/providers/base.py

@@ -0,0 +1,23 @@
+from abc import ABC, abstractmethod
+
+from app.schemas.chat import ChatMessage
+
+
+class AIProvider(ABC):
+    provider_name: str = "unknown"
+    model_name: str = "unknown"
+
+    @abstractmethod
+    async def chat(
+        self,
+        messages: list[ChatMessage],
+        max_tokens: int,
+        temperature: float,
+    ) -> tuple[str, int | None, int | None]:
+        """
+        Send messages to the provider and return (content, input_tokens, output_tokens).
+        Raises:
+            ProviderConnectionError: on network / auth failure
+            ProviderTimeoutError: on request timeout
+        """
+        ...

features/ai-service/app/providers/openai_compat.py

@@ -0,0 +1,52 @@
+"""OpenAI-compatible provider — handles both Ollama and LM Studio."""
+import asyncio
+
+import openai
+
+from app.providers.base import AIProvider
+from app.schemas.chat import ChatMessage
+
+
+class OpenAICompatProvider(AIProvider):
+    def __init__(self, config: dict, provider_name: str = "lmstudio") -> None:
+        self._client = openai.AsyncOpenAI(
+            base_url=config.get("base_url", "http://localhost:1234/v1"),
+            api_key=config.get("api_key") or "not-required",
+        )
+        self.model_name = config.get("model", "local-model")
+        self.provider_name = provider_name
+
+    async def chat(
+        self,
+        messages: list[ChatMessage],
+        max_tokens: int,
+        temperature: float,
+    ) -> tuple[str, int | None, int | None]:
+        raw_messages = [{"role": m.role, "content": m.content} for m in messages]
+        try:
+            response = await self._client.chat.completions.create(
+                model=self.model_name,
+                messages=raw_messages,
+                max_tokens=max_tokens,
+                temperature=temperature,
+            )
+        except openai.APIConnectionError as exc:
+            raise ProviderConnectionError(str(exc)) from exc
+        except openai.APITimeoutError as exc:
+            raise ProviderTimeoutError(str(exc)) from exc
+        except openai.APIStatusError as exc:
+            raise ProviderConnectionError(f"API error {exc.status_code}: {exc.message}") from exc
+
+        content = response.choices[0].message.content or ""
+        usage = response.usage
+        input_tokens = usage.prompt_tokens if usage else None
+        output_tokens = usage.completion_tokens if usage else None
+        return content, input_tokens, output_tokens
+
+
+class ProviderConnectionError(Exception):
+    pass
+
+
+class ProviderTimeoutError(Exception):
+    pass

features/ai-service/app/routers/chat.py

@@ -0,0 +1,101 @@
+"""
+POST /chat — synchronous chat endpoint.
+
+All requests are submitted to the priority queue at NORMAL priority and the caller
+waits for the result. This keeps the contract identical to the original endpoint
+while ensuring all AI traffic flows through one ordered queue.
+"""
+import asyncio
+import re
+
+from fastapi import APIRouter, HTTPException
+
+from app.providers import get_provider
+from app.providers.anthropic_provider import ProviderConnectionError as AnthropicConnError
+from app.providers.anthropic_provider import ProviderTimeoutError as AnthropicTimeoutError
+from app.providers.openai_compat import ProviderConnectionError as OpenAIConnError
+from app.providers.openai_compat import ProviderTimeoutError as OpenAITimeoutError
+from app.schemas.chat import ChatRequest, ChatResponse
+from app.services.config_reader import load_ai_config
+
+router = APIRouter()
+
+_FENCE_RE = re.compile(r"^```[a-z]*\n?(.*?)\n?```$", re.DOTALL)
+
+
+def _strip_fences(text: str) -> str:
+    m = _FENCE_RE.match(text.strip())
+    return m.group(1).strip() if m else text.strip()
+
+
+async def execute_chat(request: ChatRequest) -> ChatResponse:
+    """
+    Core provider call — invoked by the queue worker.
+    Raises HTTPException on provider errors so the queue worker stores the message.
+    """
+    config = await load_ai_config()
+
+    provider_name = config.get("provider", "lmstudio")
+    if provider_name not in ("anthropic", "ollama", "lmstudio"):
+        raise HTTPException(status_code=503, detail=f"Unknown provider configured: {provider_name!r}")
+
+    try:
+        provider = get_provider(config)
+    except ValueError as exc:
+        raise HTTPException(status_code=503, detail=str(exc))
+
+    timeout = config.get("timeout_seconds", 60)
+    max_retries = config.get("max_retries", 2)
+
+    for attempt in range(max_retries + 1):
+        try:
+            content, input_tokens, output_tokens = await asyncio.wait_for(
+                provider.chat(request.messages, request.max_tokens, request.temperature),
+                timeout=float(timeout),
+            )
+            break
+        except asyncio.TimeoutError as exc:
+            raise HTTPException(status_code=504, detail="AI provider timed out") from exc
+        except (AnthropicConnError, OpenAIConnError) as exc:
+            if attempt < max_retries:
+                await asyncio.sleep(0.5 * (attempt + 1))
+                continue
+            raise HTTPException(status_code=502, detail=f"AI provider error: {exc}") from exc
+        except (AnthropicTimeoutError, OpenAITimeoutError) as exc:
+            raise HTTPException(status_code=504, detail="AI provider timed out") from exc
+
+    if request.response_format == "json":
+        content = _strip_fences(content)
+
+    return ChatResponse(
+        content=content,
+        provider=provider.provider_name,
+        model=provider.model_name,
+        input_tokens=input_tokens,
+        output_tokens=output_tokens,
+    )
+
+
+@router.post("/chat", response_model=ChatResponse)
+async def chat(request: ChatRequest) -> ChatResponse:
+    """
+    Submit at NORMAL priority and block until the queue processes the job.
+    If the queue is paused or stopped, the call blocks until resumed (or times out).
+    """
+    from app.services.queue import Priority, queue_service  # deferred — avoids circular import
+
+    job = await queue_service.enqueue(request, Priority.NORMAL)
+    config = await load_ai_config()
+    timeout = float(config.get("timeout_seconds", 60)) + 5.0  # +5s buffer over provider timeout
+
+    try:
+        return await asyncio.wait_for(asyncio.shield(job.future), timeout=timeout)
+    except asyncio.TimeoutError:
+        queue_service.cancel_job(job.id)
+        raise HTTPException(status_code=504, detail="Timed out waiting for queue to process job")
+    except asyncio.CancelledError:
+        raise HTTPException(status_code=503, detail="Job was cancelled")
+    except Exception as exc:
+        if isinstance(exc, HTTPException):
+            raise
+        raise HTTPException(status_code=502, detail=str(exc)) from exc

features/ai-service/app/routers/health.py

@@ -0,0 +1,30 @@
+from fastapi import APIRouter
+
+from app.services.config_reader import load_ai_config
+
+router = APIRouter()
+
+
+@router.get("/health")
+async def health() -> dict:
+    return {"status": "ok"}
+
+
+@router.get("/health/provider")
+async def provider_status() -> dict:
+    config = await load_ai_config()
+    provider = config.get("provider", "lmstudio")
+    pcfg = config.get(provider, {})
+    model = pcfg.get("model", "")
+
+    # "configured" means we have the minimum required fields for the provider
+    if provider == "anthropic":
+        configured = bool(pcfg.get("api_key"))
+    else:
+        configured = bool(pcfg.get("base_url") and pcfg.get("model"))
+
+    return {
+        "provider": provider,
+        "model": model,
+        "configured": configured,
+    }

features/ai-service/app/routers/plugin.py

@@ -0,0 +1,31 @@
+"""
+Plugin manifest endpoint for the AI service.
+
+Exposes GET /plugin/manifest so the backend health-poller can discover the
+service's access rules and register it in the plugin system.
+
+No settings schema is exposed here — the AI service settings are complex
+(provider selection, conditional fields) and are rendered by a bespoke page
+rather than the generic PluginSchemaForm.
+"""
+from fastapi import APIRouter
+
+router = APIRouter()
+
+_MANIFEST = {
+    "id": "ai-service",
+    "name": "AI Service",
+    "icon": "cpu",
+    "version": "1.0",
+    "access": {
+        "allow_superuser": True,
+        "required_groups": ["ai-service-admin"],
+    },
+    # No settings_schema — the frontend uses a custom settings page
+    "settings_schema": None,
+}
+
+
+@router.get("/plugin/manifest")
+async def get_manifest() -> dict:
+    return _MANIFEST

features/ai-service/app/routers/queue.py

@@ -0,0 +1,104 @@
+"""
+Queue management router.
+
+POST  /queue/jobs          — enqueue a job, return immediately with job metadata
+GET   /queue/jobs/{id}     — poll job status / result
+DELETE /queue/jobs/{id}    — cancel a pending job
+
+GET   /queue/status        — worker state + queue depth
+POST  /queue/pause         — finish current job, stop picking new ones
+POST  /queue/resume        — resume from pause
+POST  /queue/start         — start (or restart) the worker
+POST  /queue/stop          — stop worker immediately (pending jobs stay queued)
+"""
+from fastapi import APIRouter, HTTPException
+
+from app.schemas.queue import JobStatus, QueueRequest, QueueStatus
+from app.services.queue import PRIORITY_MAP, Job, Priority, queue_service
+
+router = APIRouter(prefix="/queue", tags=["queue"])
+
+
+# ── Job endpoints ─────────────────────────────────────────────────────────────
+
+@router.post("/jobs", response_model=JobStatus, status_code=202)
+async def enqueue_job(request: QueueRequest) -> JobStatus:
+    priority = PRIORITY_MAP[request.priority]
+    job = await queue_service.enqueue(request, priority)
+    return _job_to_status(job)
+
+
+@router.get("/jobs/{job_id}", response_model=JobStatus)
+async def get_job(job_id: str) -> JobStatus:
+    job = queue_service.get_job(job_id)
+    if not job:
+        raise HTTPException(status_code=404, detail="Job not found")
+    return _job_to_status(job)
+
+
+@router.delete("/jobs/{job_id}", status_code=204)
+async def cancel_job(job_id: str) -> None:
+    if not queue_service.cancel_job(job_id):
+        raise HTTPException(status_code=404, detail="Job not found or already started")
+
+
+# ── Worker control endpoints ──────────────────────────────────────────────────
+
+@router.get("/status", response_model=QueueStatus)
+async def get_status() -> QueueStatus:
+    cur = queue_service.current_job
+    return QueueStatus(
+        running=queue_service._running,
+        paused=queue_service.is_paused,
+        queue_size=queue_service.queue_size,
+        current_job_id=cur.id if cur else None,
+    )
+
+
+@router.post("/pause", status_code=204)
+async def pause() -> None:
+    """Pause after the current job finishes."""
+    queue_service.pause()
+
+
+@router.post("/resume", status_code=204)
+async def resume() -> None:
+    """Resume from a paused state."""
+    queue_service.resume()
+
+
+@router.post("/start", status_code=204)
+async def start() -> None:
+    """Start (or restart) the worker task."""
+    queue_service.start()
+
+
+@router.post("/stop", status_code=204)
+async def stop() -> None:
+    """Stop the worker. Pending jobs remain in queue; POST /queue/start to resume."""
+    queue_service.stop()
+
+
+# ── Helper ────────────────────────────────────────────────────────────────────
+
+def _job_to_status(job: Job) -> JobStatus:
+    pos: int | None = None
+    if job.status == "pending":
+        # Count jobs that are ahead: same or higher priority AND earlier seq
+        pos = sum(
+            1
+            for j in queue_service._jobs.values()
+            if j.status == "pending"
+            and (int(j.priority), j.seq) < (int(job.priority), job.seq)
+        )
+    return JobStatus(
+        id=job.id,
+        status=job.status,
+        priority=Priority(job.priority).name.lower(),
+        position=pos,
+        created_at=job.created_at,
+        started_at=job.started_at,
+        finished_at=job.finished_at,
+        result=job.result,
+        error=job.error,
+    )

features/ai-service/app/schemas/chat.py

@@ -0,0 +1,30 @@
+from typing import Literal
+
+from pydantic import BaseModel, field_validator
+
+
+class ChatMessage(BaseModel):
+    role: Literal["system", "user", "assistant"]
+    content: str
+
+
+class ChatRequest(BaseModel):
+    messages: list[ChatMessage]
+    max_tokens: int = 2048
+    temperature: float = 0.0
+    response_format: Literal["json", "text"] = "text"
+
+    @field_validator("messages")
+    @classmethod
+    def messages_not_empty(cls, v: list) -> list:
+        if not v:
+            raise ValueError("messages must not be empty")
+        return v
+
+
+class ChatResponse(BaseModel):
+    content: str
+    provider: str
+    model: str
+    input_tokens: int | None = None
+    output_tokens: int | None = None

features/ai-service/app/schemas/queue.py

@@ -0,0 +1,40 @@
+from datetime import datetime
+from typing import Literal
+
+from pydantic import BaseModel, field_validator
+
+from app.schemas.chat import ChatMessage, ChatResponse
+
+
+class QueueRequest(BaseModel):
+    messages: list[ChatMessage]
+    max_tokens: int = 2048
+    temperature: float = 0.0
+    response_format: Literal["json", "text"] = "text"
+    priority: Literal["high", "normal", "low"] = "normal"
+
+    @field_validator("messages")
+    @classmethod
+    def messages_not_empty(cls, v: list) -> list:
+        if not v:
+            raise ValueError("messages must not be empty")
+        return v
+
+
+class JobStatus(BaseModel):
+    id: str
+    status: str
+    priority: str
+    position: int | None = None  # number of jobs ahead; None when not pending
+    created_at: datetime
+    started_at: datetime | None = None
+    finished_at: datetime | None = None
+    result: ChatResponse | None = None
+    error: str | None = None
+
+
+class QueueStatus(BaseModel):
+    running: bool
+    paused: bool
+    queue_size: int
+    current_job_id: str | None = None

features/ai-service/app/services/config_reader.py

@@ -0,0 +1,90 @@
+"""
+Reads ai_service_config.json from the storage-service config bucket.
+30-second TTL cache + env var overrides (dev credentials stay out of git).
+
+Env var overrides (all optional):
+  AI_PROVIDER          — "lmstudio" | "ollama" | "anthropic"
+  LMSTUDIO_BASE_URL, LMSTUDIO_API_KEY, LMSTUDIO_MODEL
+  OLLAMA_BASE_URL, OLLAMA_MODEL, OLLAMA_API_KEY
+  ANTHROPIC_API_KEY, ANTHROPIC_MODEL
+"""
+import json
+import os
+import time
+from copy import deepcopy
+
+import httpx
+
+from app.core.config import settings
+
+_CONFIG_KEY = "ai_service_config.json"
+
+_DEFAULT_CONFIG: dict = {
+    "provider": "lmstudio",
+    "timeout_seconds": 60,
+    "max_retries": 2,
+    "anthropic": {"api_key": "", "model": "claude-haiku-4-5-20251001"},
+    "ollama": {"base_url": "http://host.docker.internal:11434/v1", "model": "llama3.2", "api_key": "ollama"},
+    "lmstudio": {"base_url": "http://host.docker.internal:1234/v1", "model": "gemma-4-e4b-it", "api_key": "lm-studio"},
+}
+
+_cache: dict | None = None
+_cache_at: float = 0.0
+_CACHE_TTL = 30.0
+
+
+def _storage_url() -> str:
+    return f"{settings.STORAGE_SERVICE_URL}/objects/config/{_CONFIG_KEY}"
+
+
+async def _fetch_config() -> dict:
+    """Fetch config from storage-service. Returns defaults if not found."""
+    async with httpx.AsyncClient(timeout=10.0) as client:
+        resp = await client.get(_storage_url())
+        if resp.status_code == 404:
+            return deepcopy(_DEFAULT_CONFIG)
+        resp.raise_for_status()
+        return resp.json()
+
+
+def _apply_env_overrides(config: dict) -> dict:
+    cfg = deepcopy(config)
+
+    if v := os.environ.get("AI_PROVIDER"):
+        cfg["provider"] = v
+
+    lms = cfg.setdefault("lmstudio", {})
+    if v := os.environ.get("LMSTUDIO_BASE_URL"):
+        lms["base_url"] = v
+    if v := os.environ.get("LMSTUDIO_API_KEY"):
+        lms["api_key"] = v
+    if v := os.environ.get("LMSTUDIO_MODEL"):
+        lms["model"] = v
+
+    oll = cfg.setdefault("ollama", {})
+    if v := os.environ.get("OLLAMA_BASE_URL"):
+        oll["base_url"] = v
+    if v := os.environ.get("OLLAMA_MODEL"):
+        oll["model"] = v
+    if v := os.environ.get("OLLAMA_API_KEY"):
+        oll["api_key"] = v
+
+    ant = cfg.setdefault("anthropic", {})
+    if v := os.environ.get("ANTHROPIC_API_KEY"):
+        ant["api_key"] = v
+    if v := os.environ.get("ANTHROPIC_MODEL"):
+        ant["model"] = v
+
+    return cfg
+
+
+async def load_ai_config() -> dict:
+    global _cache, _cache_at
+    now = time.monotonic()
+    if _cache is not None and (now - _cache_at) < _CACHE_TTL:
+        return _cache
+    raw = await _fetch_config()
+    data = _apply_env_overrides(raw)
+    _cache = data
+    _cache_at = now
+    return data

features/ai-service/app/services/queue.py

@@ -0,0 +1,169 @@
+"""
+In-memory priority queue for AI requests.
+
+Jobs are ordered by (priority, sequence_number) so HIGH=1 jobs always run before
+NORMAL=3 and LOW=5 regardless of arrival order. Within the same priority level
+insertion order (FIFO) is preserved via the monotonically incrementing seq counter.
+
+The QueueService runs a single async worker task. It can be paused (current job
+finishes, no new jobs start), resumed, started, or stopped from outside.
+
+Module-level singleton `queue_service` is imported by routers and the app lifespan.
+"""
+import asyncio
+import uuid
+from dataclasses import dataclass, field
+from datetime import datetime, timezone
+from enum import IntEnum
+
+
+class Priority(IntEnum):
+    HIGH = 1
+    NORMAL = 3
+    LOW = 5
+
+
+PRIORITY_MAP: dict[str, Priority] = {
+    "high": Priority.HIGH,
+    "normal": Priority.NORMAL,
+    "low": Priority.LOW,
+}
+
+
+@dataclass
+class Job:
+    id: str
+    priority: Priority
+    seq: int
+    request: object  # ChatRequest — typed as object to avoid circular import
+    future: asyncio.Future
+    status: str = "pending"  # pending | processing | done | failed | cancelled
+    created_at: datetime = field(default_factory=lambda: datetime.now(timezone.utc))
+    started_at: datetime | None = None
+    finished_at: datetime | None = None
+    result: object = None
+    error: str | None = None
+
+    def __lt__(self, other: "Job") -> bool:
+        # asyncio.PriorityQueue requires items to be orderable
+        return (self.priority, self.seq) < (other.priority, other.seq)
+
+
+class QueueService:
+    def __init__(self) -> None:
+        self._queue: asyncio.PriorityQueue = asyncio.PriorityQueue()
+        self._jobs: dict[str, Job] = {}
+        self._seq: int = 0
+        self._worker_task: asyncio.Task | None = None
+        # Event: set = allowed to run; clear = paused
+        self._resume_event: asyncio.Event = asyncio.Event()
+        self._resume_event.set()
+        self._running: bool = False
+        self.current_job: Job | None = None
+
+    # ── Public API ────────────────────────────────────────────────────────────
+
+    async def enqueue(self, request: object, priority: Priority = Priority.NORMAL) -> Job:
+        self._seq += 1
+        job = Job(
+            id=str(uuid.uuid4()),
+            priority=priority,
+            seq=self._seq,
+            request=request,
+            future=asyncio.get_event_loop().create_future(),
+        )
+        self._jobs[job.id] = job
+        await self._queue.put((int(priority), self._seq, job))
+        return job
+
+    def get_job(self, job_id: str) -> Job | None:
+        return self._jobs.get(job_id)
+
+    def cancel_job(self, job_id: str) -> bool:
+        """Cancel a pending job. Returns False if not found or already started."""
+        job = self._jobs.get(job_id)
+        if job and job.status == "pending":
+            job.status = "cancelled"
+            if not job.future.done():
+                job.future.cancel()
+            return True
+        return False
+
+    def start(self) -> None:
+        """Start the worker. No-op if already running."""
+        if not self._running or (self._worker_task and self._worker_task.done()):
+            self._resume_event.set()
+            self._running = True
+            self._worker_task = asyncio.create_task(self._worker_loop())
+
+    def pause(self) -> None:
+        """Pause after the current job finishes. Does not cancel in-progress work."""
+        self._resume_event.clear()
+
+    def resume(self) -> None:
+        """Resume from a paused state."""
+        self._resume_event.set()
+
+    def stop(self) -> None:
+        """Stop the worker. Pending jobs remain in the queue; start() will resume them."""
+        self._running = False
+        self._resume_event.set()  # unblock the wait so the loop can exit
+        if self._worker_task and not self._worker_task.done():
+            self._worker_task.cancel()
+
+    @property
+    def is_paused(self) -> bool:
+        return not self._resume_event.is_set()
+
+    @property
+    def queue_size(self) -> int:
+        return self._queue.qsize()
+
+    # ── Internal ──────────────────────────────────────────────────────────────
+
+    async def _worker_loop(self) -> None:
+        while self._running:
+            # Block here while paused
+            await self._resume_event.wait()
+
+            try:
+                _, _, job = await asyncio.wait_for(self._queue.get(), timeout=1.0)
+            except asyncio.TimeoutError:
+                continue
+            except asyncio.CancelledError:
+                break
+
+            if job.status == "cancelled":
+                self._queue.task_done()
+                continue
+
+            try:
+                await self._process(job)
+            finally:
+                self._queue.task_done()
+
+    async def _process(self, job: Job) -> None:
+        # Deferred import — avoids circular dependency with chat router
+        from app.routers.chat import execute_chat  # noqa: PLC0415
+
+        job.status = "processing"
+        job.started_at = datetime.now(timezone.utc)
+        self.current_job = job
+        try:
+            result = await execute_chat(job.request)
+            job.status = "done"
+            job.result = result
+            if not job.future.done():
+                job.future.set_result(result)
+        except Exception as exc:
+            job.status = "failed"
+            job.error = str(exc)
+            if not job.future.done():
+                job.future.set_exception(exc)
+        finally:
+            job.finished_at = datetime.now(timezone.utc)
+            self.current_job = None
+
+
+# Singleton used throughout the app
+queue_service = QueueService()

features/ai-service/pyproject.toml

@@ -0,0 +1,30 @@
+[build-system]
+requires = ["setuptools>=45"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "ai-service"
+version = "0.1.0"
+requires-python = ">=3.11"
+dependencies = [
+    "fastapi>=0.111",
+    "uvicorn[standard]>=0.29",
+    "pydantic-settings>=2.2",
+    "anthropic>=0.28",
+    "openai>=1.0",
+    "httpx>=0.27",
+]
+
+[project.optional-dependencies]
+dev = [
+    "pytest>=8",
+    "pytest-asyncio>=0.23",
+    "httpx>=0.27",
+    "ruff>=0.4",
+]
+
+[tool.pytest.ini_options]
+asyncio_mode = "auto"
+
+[tool.ruff]
+line-length = 100

features/ai-service/scripts/start.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+set -e
+echo "[ai-service] starting uvicorn..."
+exec uvicorn app.main:app --host 0.0.0.0 --port 8010

features/ai-service/scripts/start_dev.sh

@@ -0,0 +1,4 @@
+#!/bin/sh
+set -e
+echo "[ai-service] starting uvicorn (dev)..."
+exec uvicorn app.main:app --host 0.0.0.0 --port 8010 --reload

features/ai-service/tests/conftest.py

@@ -0,0 +1,57 @@
+import pytest
+from httpx import ASGITransport, AsyncClient
+
+from app.main import app
+
+
+@pytest.fixture
+async def ai_client():
+    async with AsyncClient(transport=ASGITransport(app=app), base_url="http://test") as client:
+        yield client
+
+
+# ---------------------------------------------------------------------------
+# Config fixtures
+# ---------------------------------------------------------------------------
+
+LMSTUDIO_CONFIG = {
+    "provider": "lmstudio",
+    "timeout_seconds": 10,
+    "max_retries": 0,
+    "lmstudio": {
+        "base_url": "http://fake-lmstudio/v1",
+        "model": "test-model",
+        "api_key": "test-key",
+    },
+}
+
+OLLAMA_CONFIG = {
+    "provider": "ollama",
+    "timeout_seconds": 10,
+    "max_retries": 0,
+    "ollama": {
+        "base_url": "http://fake-ollama/v1",
+        "model": "llama3.2",
+        "api_key": "ollama",
+    },
+}
+
+ANTHROPIC_CONFIG = {
+    "provider": "anthropic",
+    "timeout_seconds": 10,
+    "max_retries": 0,
+    "anthropic": {
+        "api_key": "sk-ant-test",
+        "model": "claude-haiku-4-5-20251001",
+    },
+}
+
+MISSING_KEY_ANTHROPIC_CONFIG = {
+    "provider": "anthropic",
+    "timeout_seconds": 10,
+    "max_retries": 0,
+    "anthropic": {
+        "api_key": "",
+        "model": "claude-haiku-4-5-20251001",
+    },
+}

features/ai-service/tests/test_chat.py

@@ -0,0 +1,221 @@
+"""Tests for POST /chat."""
+import asyncio
+from unittest.mock import AsyncMock, MagicMock, patch
+
+import pytest
+
+from tests.conftest import ANTHROPIC_CONFIG, LMSTUDIO_CONFIG, OLLAMA_CONFIG
+
+# ---------------------------------------------------------------------------
+# Helpers
+# ---------------------------------------------------------------------------
+
+_LOAD_CONFIG = "app.routers.chat.load_ai_config"
+_PROVIDER_CHAT = "app.providers.openai_compat.OpenAICompatProvider.chat"
+_ANTHROPIC_CHAT = "app.providers.anthropic_provider.AnthropicProvider.chat"
+
+MESSAGES = [{"role": "user", "content": "Hello"}]
+SYSTEM_MESSAGES = [
+    {"role": "system", "content": "You are helpful."},
+    {"role": "user", "content": "Hello"},
+]
+
+
+def _mock_chat_response(content="ok", input_tokens=10, output_tokens=5):
+    return AsyncMock(return_value=(content, input_tokens, output_tokens))
+
+
+# ---------------------------------------------------------------------------
+# Success: each provider
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_chat_lmstudio_success(ai_client):
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG), patch(
+        _PROVIDER_CHAT, new=_mock_chat_response("lmstudio reply")
+    ):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 200
+    data = resp.json()
+    assert data["content"] == "lmstudio reply"
+    assert data["provider"] == "lmstudio"
+    assert data["model"] == "test-model"
+    assert data["input_tokens"] == 10
+    assert data["output_tokens"] == 5
+
+
+@pytest.mark.asyncio
+async def test_chat_ollama_success(ai_client):
+    with patch(_LOAD_CONFIG, return_value=OLLAMA_CONFIG), patch(
+        _PROVIDER_CHAT, new=_mock_chat_response("ollama reply")
+    ):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 200
+    data = resp.json()
+    assert data["content"] == "ollama reply"
+    assert data["provider"] == "ollama"
+
+
+@pytest.mark.asyncio
+async def test_chat_anthropic_success(ai_client):
+    with patch(_LOAD_CONFIG, return_value=ANTHROPIC_CONFIG), patch(
+        _ANTHROPIC_CHAT, new=_mock_chat_response("anthropic reply")
+    ):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 200
+    data = resp.json()
+    assert data["content"] == "anthropic reply"
+    assert data["provider"] == "anthropic"
+
+
+# ---------------------------------------------------------------------------
+# response_format
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_response_format_json_strips_fences(ai_client):
+    fenced = "```json\n{\"key\": \"value\"}\n```"
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG), patch(
+        _PROVIDER_CHAT, new=_mock_chat_response(fenced)
+    ):
+        resp = await ai_client.post(
+            "/chat",
+            json={"messages": MESSAGES, "response_format": "json"},
+        )
+
+    assert resp.status_code == 200
+    assert resp.json()["content"] == '{"key": "value"}'
+
+
+@pytest.mark.asyncio
+async def test_response_format_text_preserves_fences(ai_client):
+    fenced = "```python\nprint('hi')\n```"
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG), patch(
+        _PROVIDER_CHAT, new=_mock_chat_response(fenced)
+    ):
+        resp = await ai_client.post(
+            "/chat",
+            json={"messages": MESSAGES, "response_format": "text"},
+        )
+
+    assert resp.status_code == 200
+    assert "```" in resp.json()["content"]
+
+
+# ---------------------------------------------------------------------------
+# Validation errors
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_chat_missing_messages_returns_422(ai_client):
+    resp = await ai_client.post("/chat", json={})
+    assert resp.status_code == 422
+
+
+@pytest.mark.asyncio
+async def test_chat_empty_messages_returns_422(ai_client):
+    resp = await ai_client.post("/chat", json={"messages": []})
+    assert resp.status_code == 422
+
+
+# ---------------------------------------------------------------------------
+# Provider errors
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_chat_connection_error_returns_502(ai_client):
+    from app.providers.openai_compat import ProviderConnectionError
+
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG), patch(
+        _PROVIDER_CHAT, side_effect=ProviderConnectionError("refused")
+    ):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 502
+
+
+@pytest.mark.asyncio
+async def test_chat_timeout_returns_504(ai_client):
+    async def _slow(*_args, **_kwargs):
+        await asyncio.sleep(100)
+
+    with patch(_LOAD_CONFIG, return_value={**LMSTUDIO_CONFIG, "timeout_seconds": 0.01}), patch(
+        _PROVIDER_CHAT, new=_slow
+    ):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 504
+
+
+@pytest.mark.asyncio
+async def test_chat_unknown_provider_returns_503(ai_client):
+    bad_config = {**LMSTUDIO_CONFIG, "provider": "unknown-llm"}
+    with patch(_LOAD_CONFIG, return_value=bad_config):
+        resp = await ai_client.post("/chat", json={"messages": MESSAGES})
+
+    assert resp.status_code == 503
+
+
+# ---------------------------------------------------------------------------
+# Anthropic system message extraction
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_anthropic_system_message_extracted(ai_client):
+    """System-role messages must not appear in the user_messages list sent to Anthropic."""
+    captured_kwargs: dict = {}
+
+    async def _fake_create(**kwargs):
+        captured_kwargs.update(kwargs)
+        mock_resp = MagicMock()
+        mock_resp.content = [MagicMock(text="ok")]
+        mock_resp.usage = MagicMock(input_tokens=5, output_tokens=2)
+        return mock_resp
+
+    with patch(_LOAD_CONFIG, return_value=ANTHROPIC_CONFIG), patch(
+        "anthropic.AsyncAnthropic.messages",
+        new_callable=lambda: type(
+            "Messages",
+            (),
+            {"create": staticmethod(AsyncMock(side_effect=_fake_create))},
+        ),
+    ):
+        resp = await ai_client.post("/chat", json={"messages": SYSTEM_MESSAGES})
+
+    # Whether the call succeeded or not, no system role should reach the messages list
+    if "messages" in captured_kwargs:
+        roles = [m["role"] for m in captured_kwargs["messages"]]
+        assert "system" not in roles
+
+
+# ---------------------------------------------------------------------------
+# Parameter forwarding
+# ---------------------------------------------------------------------------
+
+
+@pytest.mark.asyncio
+async def test_max_tokens_and_temperature_forwarded(ai_client):
+    captured: dict = {}
+
+    async def _capture(messages, max_tokens, temperature):
+        captured["max_tokens"] = max_tokens
+        captured["temperature"] = temperature
+        return ("ok", 1, 1)
+
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG), patch(_PROVIDER_CHAT, new=_capture):
+        resp = await ai_client.post(
+            "/chat",
+            json={"messages": MESSAGES, "max_tokens": 512, "temperature": 0.7},
+        )
+
+    assert resp.status_code == 200
+    assert captured["max_tokens"] == 512
+    assert captured["temperature"] == pytest.approx(0.7)

features/ai-service/tests/test_health.py

@@ -0,0 +1,38 @@
+"""Tests for GET /health and GET /health/provider."""
+from unittest.mock import patch
+
+import pytest
+
+from tests.conftest import ANTHROPIC_CONFIG, LMSTUDIO_CONFIG, MISSING_KEY_ANTHROPIC_CONFIG
+
+_LOAD_CONFIG = "app.routers.health.load_ai_config"
+
+
+@pytest.mark.asyncio
+async def test_health_returns_ok(ai_client):
+    resp = await ai_client.get("/health")
+    assert resp.status_code == 200
+    assert resp.json() == {"status": "ok"}
+
+
+@pytest.mark.asyncio
+async def test_provider_status_configured(ai_client):
+    with patch(_LOAD_CONFIG, return_value=LMSTUDIO_CONFIG):
+        resp = await ai_client.get("/health/provider")
+
+    assert resp.status_code == 200
+    data = resp.json()
+    assert data["provider"] == "lmstudio"
+    assert data["model"] == "test-model"
+    assert data["configured"] is True
+
+
+@pytest.mark.asyncio
+async def test_provider_status_not_configured_when_api_key_missing(ai_client):
+    with patch(_LOAD_CONFIG, return_value=MISSING_KEY_ANTHROPIC_CONFIG):
+        resp = await ai_client.get("/health/provider")
+
+    assert resp.status_code == 200
+    data = resp.json()
+    assert data["provider"] == "anthropic"
+    assert data["configured"] is False

features/doc-service/.env.example

@@ -0,0 +1,21 @@
+DATABASE_URL=postgresql+asyncpg://postgres:password@db:5432/destroying_sap
+DATA_DIR=/data/documents
+CONFIG_PATH=/config/doc_service_config.json
+
+# Optional AI provider overrides — if set, these take precedence over
+# whatever is stored in doc_service_config.json.
+# Useful for pinning a dev environment to a specific local AI instance
+# without touching the shared config volume.
+#
+# AI_PROVIDER=lmstudio
+# LMSTUDIO_BASE_URL=http://host.docker.internal:1234/v1
+# LMSTUDIO_API_KEY=your-lm-studio-token
+# LMSTUDIO_MODEL=local-model
+#
+# AI_PROVIDER=ollama
+# OLLAMA_BASE_URL=http://host.docker.internal:11434/v1
+# OLLAMA_MODEL=llama3.2
+#
+# AI_PROVIDER=anthropic
+# ANTHROPIC_API_KEY=sk-ant-...
+# ANTHROPIC_MODEL=claude-haiku-4-5-20251001

features/doc-service/CLAUDE.md

@@ -0,0 +1,184 @@
+# doc-service — Claude context
+
+PDF extraction microservice, port 8001 (internal). Shares the same PostgreSQL instance as the backend. Receives proxied requests from `backend:8000`, which injects `x-user-id` and `x-user-groups` headers — doc-service trusts these headers directly. Calls `ai-service:8010` for document classification. All file/blob storage goes through `storage-service:8020` — no files are written directly to the filesystem. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
+
+---
+
+## Commands
+
+All commands run inside Docker — never on the host.
+
+```bash
+docker compose exec doc-service alembic revision --autogenerate -m "describe change"
+docker compose exec doc-service alembic upgrade head
+docker compose exec doc-service alembic downgrade -1
+```
+
+---
+
+## File & Folder Tree
+
+```
+features/doc-service/
+├── app/
+│   ├── main.py                    ← FastAPI, lifespan (file watcher start/stop)
+│   ├── database.py                ← Same PostgreSQL instance as backend
+│   ├── deps.py                    ← get_user_id, get_user_groups, get_user_is_admin, get_user_admin_groups (injected headers)
+│   ├── models/
+│   │   ├── document.py            ← Document model
+│   │   ├── category.py            ← DocumentCategory model
+│   │   ├── category_assignment.py ← CategoryAssignment (composite PK)
+│   │   └── document_share.py      ← DocumentShare model (group-based sharing)
+│   ├── schemas/
+│   │   ├── document.py            ← DocumentOut, DocumentPage, DocumentStatusOut, etc.
+│   │   ├── category.py            ← CategoryOut, CategoryCreate, CategoryUpdate
+│   │   └── share.py               ← DocumentShareOut, DocumentShareCreate, SharedDocumentOut
+│   ├── routers/
+│   │   ├── documents.py           ← Full CRUD + file serving + reprocess + suggestions + sharing
+│   │   ├── categories.py          ← Category CRUD (includes watch-owned categories)
+│   │   └── plugin.py              ← GET /plugin/manifest, GET+PATCH /plugin/settings
+│   └── services/
+│       ├── storage.py             ← Storage client: save_upload/download_file/delete_file → storage-service:8020 documents bucket
+│       ├── ai_client.py           ← classify_document() → ai-service:8010/chat
+│       ├── config_reader.py       ← Config load/save via storage-service config bucket (doc_service_config.json)
+│       └── file_watcher.py        ← watchdog-based PDF watcher + startup scan + ingestion
+├── alembic/versions/              ← Migration chain
+│   ├── 0003_add_watch_columns.py  ← source, watch_path, suggested_folder, suggested_filename
+│   ├── 0004_add_document_shares.py ← document_shares table (group-based sharing)
+│   └── 0008_rename_file_path_to_storage_key.py ← file_path → storage_key; strips /data/documents/ prefix from existing rows
+├── Dockerfile                     ← python:3.12-slim, non-root user 1001
+└── STATUS.md
+```
+
+---
+
+## Database Models
+
+### `documents`
+
+| Column | Type | Constraints | Notes |
+|--------|------|-------------|-------|
+| `id` | String | PK, UUID | |
+| `user_id` | String | indexed | not FK — trusts x-user-id header |
+| `filename` | String | NOT NULL | |
+| `storage_key` | String | NOT NULL | storage-service key: `{user_id}/{doc_id}.pdf` (documents bucket) |
+| `file_size` | Integer | NOT NULL | bytes |
+| `status` | String | default="pending" | pending / processing / done / failed |
+| `title` | String(500) | nullable | AI-extracted |
+| `document_type` | String | nullable | invoice / bill / receipt / order / expense / revenue / unknown |
+| `raw_text` | Text | nullable | first 500 k chars |
+| `extracted_data` | Text | nullable | JSON string |
+| `tags` | Text | nullable | JSON array string |
+| `error_message` | String(500) | nullable | |
+| `created_at` | DateTime(tz) | server_default=now() | |
+| `processed_at` | DateTime(tz) | nullable | |
+| `source` | String(16) | default="upload" | "upload" or "watch" |
+| `watch_path` | String | nullable | original absolute path in watch directory |
+| `suggested_folder` | String(128) | nullable | AI-suggested category (pending user confirm) |
+| `suggested_filename` | String(500) | nullable | AI-suggested title/rename (pending user confirm) |
+
+### `document_categories`
+
+| Column | Type | Constraints | Notes |
+|--------|------|-------------|-------|
+| `id` | String | PK, UUID | |
+| `user_id` | String | indexed | owner; "watch" for system categories |
+| `name` | String(128) | NOT NULL | PascalCase-with-dashes convention enforced on create/rename |
+| `scope` | String(16) | NOT NULL, default="personal" | "personal" / "group" / "system" |
+| `group_id` | String | nullable, indexed | set when scope="group" |
+| `created_at` | DateTime(tz) | server_default=now() | |
+
+### `document_category_assignments` (composite PK)
+
+| Column | Type | Constraints |
+|--------|------|-------------|
+| `document_id` | String | PK + FK→documents.id CASCADE |
+| `category_id` | String | PK + FK→document_categories.id CASCADE |
+
+### `document_shares`
+
+| Column | Type | Constraints | Notes |
+|--------|------|-------------|-------|
+| `id` | String | PK, UUID | |
+| `document_id` | String | indexed, NOT NULL | not FK — trusts proxy |
+| `group_id` | String | indexed, NOT NULL | group from backend |
+| `shared_by_user_id` | String | NOT NULL | owner who shared |
+| `can_delete` | Boolean | NOT NULL, default=false | allows group members to delete the doc |
+| `created_at` | DateTime(tz) | server_default=now() | |
+
+Unique constraint: `(document_id, group_id)`
+
+### Migration chain
+
+| Rev ID | Slug |
+|--------|------|
+| `0001` | `create_doc_tables` |
+| `0002` | `add_document_title` |
+| `0003` | `add_watch_columns` |
+| `0004` | `add_document_shares` |
+| `0005` | `add_share_can_delete` |
+| `0006` | `add_category_scope` |
+| `0007` | `capitalize_system_category_names` |
+| `0008` | `rename_file_path_to_storage_key` |
+
+---
+
+## API Endpoints (internal — reached via backend proxy)
+
+All these endpoints are proxied from `backend:8000`. The backend injects `x-user-id` and `x-user-groups` before forwarding.
+
+### Documents
+
+| Method | Path | Description |
+|--------|------|-------------|
+| POST | `/documents/upload` | Upload PDF (202, background processing) |
+| GET | `/documents` | Paginated list (filterable: search, status, type, category, sort) |
+| GET | `/documents/{id}` | Document detail |
+| GET | `/documents/{id}/status` | Processing status only |
+| PATCH | `/documents/{id}/type` | Update document type |
+| PATCH | `/documents/{id}/tags` | Update tags |
+| PATCH | `/documents/{id}/title` | Update title |
+| POST | `/documents/{id}/reprocess` | Re-run AI extraction |
+| DELETE | `/documents/{id}` | Delete document (204) |
+| GET | `/documents/{id}/file` | Download PDF (streaming) |
+| POST | `/documents/{id}/categories/{cat_id}` | Assign category |
+| DELETE | `/documents/{id}/categories/{cat_id}` | Remove category |
+| POST | `/documents/{id}/suggestions/folder/confirm` | Confirm AI folder suggestion |
+| POST | `/documents/{id}/suggestions/folder/reject` | Reject AI folder suggestion |
+| POST | `/documents/{id}/suggestions/filename/confirm` | Confirm AI filename suggestion |
+| POST | `/documents/{id}/suggestions/filename/reject` | Reject AI filename suggestion |
+| GET | `/documents/shared-with-me` | Documents shared with current user via their groups |
+| GET | `/documents/{id}/shares` | List groups the document is shared with (owner only) |
+| POST | `/documents/{id}/shares` | Share with a group (owner only; group must be in user's groups) |
+| DELETE | `/documents/{id}/shares/{group_id}` | Stop sharing with a group (owner only) |
+
+### Categories
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/categories` | List user's categories |
+| POST | `/categories` | Create category (triggers background AI reanalysis) |
+| PATCH | `/categories/{id}` | Rename |
+| DELETE | `/categories/{id}` | Delete (204) |
+
+### Plugin
+
+| Method | Path | Description |
+|--------|------|-------------|
+| GET | `/plugin/manifest` | Plugin manifest with settings JSON Schema |
+| GET | `/plugin/settings` | Current plugin settings |
+| PATCH | `/plugin/settings` | Update plugin settings |
+
+---
+
+## Default Values & Limits
+
+| Parameter | Value | Location |
+|-----------|-------|----------|
+| Document title max | 500 chars | `models/document.py` |
+| Category name max | 128 chars | `models/category.py` |
+| PDF max size (default) | 20 MB | admin settings (configurable) |
+| Raw text cap | 500 k chars | `services/ai_client.py` |
+| Documents per_page | 1–100, default 20 | `routers/documents.py` |
+| AI service timeout | 60 s | `services/ai_client.py` |
+| AI service max retries | 2 | `services/ai_client.py` |

features/doc-service/Dockerfile

@@ -0,0 +1,34 @@
+# ── Stage 1: dependency installation ─────────────────────────────────────────
+FROM python:3.12-slim AS builder
+
+WORKDIR /app
+
+RUN pip install --upgrade pip
+
+COPY pyproject.toml .
+RUN pip install --prefix=/install .
+
+# ── Stage 2: runtime ──────────────────────────────────────────────────────────
+FROM python:3.12-slim
+
+# Create non-root user (UID/GID 1001)
+RUN groupadd --gid 1001 appuser && \
+    useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
+
+# Pre-create watch dir with correct ownership.
+# /data/documents and /config are no longer used — all file/config storage goes through storage-service.
+RUN mkdir -p /data/watch && chown -R appuser:appuser /data
+
+WORKDIR /app
+
+COPY --from=builder /install /usr/local
+COPY --chown=appuser:appuser app ./app
+COPY --chown=appuser:appuser alembic ./alembic
+COPY --chown=appuser:appuser alembic.ini .
+COPY --chown=appuser:appuser scripts ./scripts
+
+USER appuser
+
+EXPOSE 8001
+
+CMD ["sh", "scripts/start.sh"]

features/doc-service/STATUS.md

@@ -0,0 +1,210 @@
+# Doc Service — Status
+
+## What it is
+
+PDF document management microservice. Handles upload, storage, async AI-powered extraction, tagging, categorisation, and retrieval of PDF documents on a per-user basis. Also supports automatic ingestion from a mounted watch directory (NAS, Nextcloud, Syncthing, etc.).
+
+Port: `8001` (internal only, not exposed to host). All traffic arrives via the backend proxy (`backend/app/routers/documents_proxy.py`), which injects the authenticated `x-user-id` header.
+
+Database: shared PostgreSQL instance, isolated via `alembic_version_doc_service` Alembic version table. Storage: `/data/documents/` (Docker named volume `doc_data`). Watch directory: `/data/watch` (named volume `watch_data` in prod; bind-mount in dev via `docker-compose.dev.yml`).
+
+---
+
+## Current functionality
+
+### Document lifecycle
+
+1. `POST /documents/upload` — validate PDF, persist file to `/data/documents/{user_id}/{doc_id}.pdf`, create DB row with `status=pending`, enqueue background extraction
+2. Background task: extract text with `pdfplumber` → POST to ai-service `/chat` → parse JSON result → update `status=done` (or `failed`)
+3. AI extracts: `title`, `document_type`, `tags`, `suggested_categories`, plus domain fields (vendor, customer, dates, amounts, etc.) into `extracted_data` (JSON string)
+
+### Endpoints
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `POST` | `/documents/upload` | Upload PDF; returns 202 with initial doc row |
+| `GET` | `/documents` | Paginated list with filters, sort, and optional `category_id` filter |
+| `GET` | `/documents/{id}` | Single document |
+| `GET` | `/documents/{id}/status` | Lightweight status poll |
+| `GET` | `/documents/{id}/download` | Stream file bytes |
+| `DELETE` | `/documents/{id}` | Delete document and file |
+| `PATCH` | `/documents/{id}/type` | Update document type |
+| `PATCH` | `/documents/{id}/tags` | Replace tag list (dedup, preserve order) |
+| `PATCH` | `/documents/{id}/title` | Update editable title |
+| `GET` | `/documents/categories` | List all categories (user + watch) |
+| `POST` | `/documents/categories` | Create a category; triggers re-analysis of documents in similar categories |
+| `POST` | `/documents/{id}/reprocess` | Reset status to pending and re-run AI extraction; 409 if already pending/processing |
+| `PATCH` | `/documents/categories/{id}` | Rename a category |
+| `DELETE` | `/documents/categories/{id}` | Delete a category |
+| `POST` | `/documents/{id}/categories/{cat_id}` | Assign category to document |
+| `DELETE` | `/documents/{id}/categories/{cat_id}` | Remove category from document |
+| `POST` | `/documents/{id}/suggestions/folder/confirm` | Apply AI folder suggestion → create/find category + assign |
+| `POST` | `/documents/{id}/suggestions/folder/reject` | Clear AI folder suggestion |
+| `POST` | `/documents/{id}/suggestions/filename/confirm` | Apply AI filename suggestion → set title |
+| `POST` | `/documents/{id}/suggestions/filename/reject` | Clear AI filename suggestion |
+
+### Plugin endpoints (internal — backend calls only)
+
+| Method | Path | Description |
+|--------|------|-------------|
+| `GET` | `/plugin/manifest` | Static manifest: metadata, JSON Schema for settings, access rules |
+| `GET` | `/plugin/settings` | Current watch/storage config values |
+| `PATCH` | `/plugin/settings` | Update watch/storage config (persisted to `/config/doc_service_config.json`) |
+
+### Pagination & filtering (`GET /documents`)
+
+Query params:
+
+| Param | Default | Notes |
+|-------|---------|-------|
+| `page` | 1 | ≥ 1 |
+| `per_page` | 20 | 1–100 |
+| `sort` | `created_at` | `created_at`, `processed_at`, `filename`, `title`, `file_size`, `status`, `document_type` |
+| `order` | `desc` | `asc` \| `desc` |
+| `status` | — | filter by status string |
+| `document_type` | — | filter by document type |
+| `search` | — | case-insensitive ILIKE on `title`, `filename`, `tags`, `document_type` |
+| `category_id` | — | filter to documents assigned to this category UUID |
+
+Response: `{ items: [...], total: N, page: N, pages: N }`
+
+### Document schema
+
+```
+id                UUID
+user_id           string (from x-user-id header; "watch" for watch-ingested docs)
+filename          original filename
+title             AI-suggested editable title (nullable)
+file_size         bytes
+status            pending | processing | done | failed
+document_type     AI-classified type (nullable)
+extracted_data    JSON string — all AI-extracted fields
+tags              JSON array string — editable tags
+error_message     set if status=failed
+created_at        upload timestamp
+processed_at      when extraction finished
+source            "upload" (default) or "watch"
+watch_path        original absolute path in watch directory (nullable)
+suggested_folder  AI-suggested category name, pending user confirm (nullable)
+suggested_filename AI-suggested title/rename, pending user confirm (nullable)
+categories        many-to-many via category_assignments
+```
+
+Watch-ingested documents (`user_id = "watch"`) are visible to all authenticated users.
+
+### AI extraction (via ai-service)
+
+System prompt and user prompt template are loaded at runtime from `doc_service_config.json` (`system_prompts` key). Defaults are built into the service and used as fallback if the config key is absent. Changes made via the AI Settings UI take effect within 30 seconds (config cache TTL).
+
+Prompt sends the first 50 000 chars of extracted text. Expected JSON response includes:
+- `title` — suggested human-readable title
+- `document_type` — invoice / bill / receipt / order / expense / revenue / unknown
+- `tags` — list of keyword tags
+- `suggested_categories` — list of category names to suggest in the UI
+- Domain fields: `vendor`, `customer`, `invoice_number`, `due_date`, `total_amount`, `currency`, etc.
+
+### Config (runtime, persisted to shared volume)
+
+`/config/doc_service_config.json`:
+```json
+{ "documents": { "max_pdf_bytes": 20971520 } }
+```
+Env override: `DOC_MAX_PDF_MB`
+
+### Watch directory feature
+
+Controlled via plugin settings (UI accessible to superusers and `doc-service-admin` group members):
+
+- `watch_enabled` — toggle file watching (default: false)
+- `watch_path` — mount point (read-only, `/data/watch`; override via Docker volume)
+- `ai_folder_suggestion` — AI suggests a category for each ingested doc (user confirms)
+- `ai_folder_default` — default category when AI suggestion is disabled
+- `ai_rename_suggestion` — AI suggests a title for each ingested doc (user confirms)
+
+On startup scan, the watcher walks the watch directory and ingests any PDFs not already in the database (idempotency check by `watch_path`). Subfolders are automatically mapped to categories (e.g. `watch/invoices/bill.pdf` → category "invoices"). No-remove policy: deleting a file from the watch directory does not delete the document record.
+
+### Document sharing (`document_shares`)
+
+Group-based sharing allows a document owner to share a document with all members of any group they belong to. Recipients can view and download the shared document; they cannot edit, re-analyse, delete, or re-share it.
+
+The gateway injects `X-User-Groups: <group_id1>,<group_id2>,...` alongside the existing `X-User-Id` header, so doc-service can evaluate group access without querying the backend DB.
+
+| Method | Path | Auth | Description |
+|--------|------|------|-------------|
+| `GET` | `/documents/shared-with-me` | Any user | Documents shared with the user via their groups; excludes own docs |
+| `GET` | `/documents/{id}/shares` | Owner only | List all groups the document is shared with |
+| `POST` | `/documents/{id}/shares` | Owner only | Share with a group (`{group_id}` in body); group must be in X-User-Groups |
+| `DELETE` | `/documents/{id}/shares/{group_id}` | Owner only | Stop sharing with that group |
+
+`DocumentOut` now includes `share_count: int` — the number of groups the document is shared with.
+
+`GET /documents/{id}/file` also allows access to shared documents (recipients can download).
+
+### Database migrations
+
+| Revision | Description |
+|----------|-------------|
+| 0001 | Initial schema (documents, categories, category_assignments) |
+| 0002 | Add `title` column to documents |
+| 0003 | Add `source`, `watch_path`, `suggested_folder`, `suggested_filename` columns |
+| 0004 | Add `document_shares` table (document_id, group_id, shared_by_user_id, created_at) |
+
+Run automatically on container start via `alembic upgrade head`.
+
+---
+
+## Architecture
+
+```
+backend (proxy)  →  doc-service:8001
+                        │
+           ┌────────────┼────────────────────┐
+      documents.py    categories.py        plugin.py
+           │               │             (internal only)
+  ┌────────┴────────┐
+upload           list/get/patch/suggest
+  │
+save_upload()        pdfplumber extraction
+  │                    │
+Document(status=pending)   ai_client.classify_document()
+  │                    │
+BackgroundTask         ai-service:8010/chat
+  │                    │
+process_document()   JSON result → update doc row
+
+file_watcher.py (watchdog Observer, daemon thread)
+  │
+  ├── _PdfEventHandler.on_created / on_moved
+  │       └── asyncio.run_coroutine_threadsafe(ingest_file, loop)
+  │
+  └── _scan_existing() on startup (catches offline gaps)
+```
+
+---
+
+## Known limitations / not implemented
+
+- **Re-process** — no endpoint to re-trigger AI extraction on an existing document (e.g. after changing the AI model or prompt)
+- **Advanced field-level search** — `search` param matches text fields via ILIKE but does not query into `extracted_data` JSON (e.g. filter by `vendor` or `due_date`)
+- **Bulk operations** — no bulk category assign/remove endpoint (frontend handles bulk delete/share individually)
+- **Advanced field-level search** — `search` matches text fields via ILIKE but does not query into `extracted_data` JSON
+- **Pagination in categories** — categories are returned as a full list (no pagination)
+- **File type** — only PDF supported
+- **Concurrent uploads** — no rate limiting per user
+
+---
+
+## Future work
+
+- [x] `POST /documents/{id}/reprocess` — re-run AI extraction
+- [x] Watch directory feature with file watcher, startup scan, folder-to-category mapping, AI suggestion toggles
+- [x] Plugin manifest endpoint (`/plugin/manifest`, `/plugin/settings`) for generic settings UI
+- [ ] Advanced filter: query `extracted_data` JSON fields (vendor, due_date, amount) — requires PostgreSQL `jsonb` column or indexed virtual columns
+- [ ] Bulk operations endpoint
+- [x] Document sharing via groups — `document_shares` table + share endpoints + shared-with-me view
+- [x] Frontend UI for suggestion badges (suggested_folder / suggested_filename confirm/reject buttons in slide-over)
+- [ ] Advanced filter: query `extracted_data` JSON fields (vendor, due_date, amount)
+- [ ] Support additional file types (images via OCR, DOCX)
+- [ ] Rate limiting on upload endpoint
+- [ ] Soft delete with restore
+- [ ] Edit rights for shared recipients (V2)

features/doc-service/alembic.ini

@@ -0,0 +1,45 @@
+[alembic]
+script_location = alembic
+prepend_sys_path = .
+version_path_separator = os
+sqlalchemy.url = postgresql+asyncpg://postgres:password@localhost:5432/destroying_sap
+
+# Use a separate version table so this service's migrations don't collide
+# with the main backend's alembic_version table in the shared postgres instance.
+version_table = alembic_version_doc_service
+
+[post_write_hooks]
+
+[loggers]
+keys = root,sqlalchemy,alembic
+
+[handlers]
+keys = console
+
+[formatters]
+keys = generic
+
+[logger_root]
+level = WARN
+handlers = console
+qualname =
+
+[logger_sqlalchemy]
+level = WARN
+handlers =
+qualname = sqlalchemy.engine
+
+[logger_alembic]
+level = INFO
+handlers =
+qualname = alembic
+
+[handler_console]
+class = StreamHandler
+args = (sys.stderr,)
+level = NOTSET
+formatter = generic
+
+[formatter_generic]
+format = %(levelname)-5.5s [%(name)s] %(message)s
+datefmt = %H:%M:%S

features/doc-service/alembic/env.py

@@ -0,0 +1,55 @@
+import asyncio
+from logging.config import fileConfig
+
+from alembic import context
+from sqlalchemy.ext.asyncio import create_async_engine
+
+from app.core.config import settings
+from app.database import Base
+import app.models  # noqa: F401 — registers Document, DocumentCategory, CategoryAssignment
+
+config = context.config
+config.set_main_option("sqlalchemy.url", settings.DATABASE_URL)
+
+if config.config_file_name:
+    fileConfig(config.config_file_name)
+
+target_metadata = Base.metadata
+
+# Separate version table — must not collide with the main backend's alembic_version table.
+VERSION_TABLE = "alembic_version_doc_service"
+
+
+def run_migrations_offline():
+    context.configure(
+        url=settings.DATABASE_URL,
+        target_metadata=target_metadata,
+        literal_binds=True,
+        dialect_opts={"paramstyle": "named"},
+        version_table=VERSION_TABLE,
+    )
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+def do_run_migrations(connection):
+    context.configure(
+        connection=connection,
+        target_metadata=target_metadata,
+        version_table=VERSION_TABLE,
+    )
+    with context.begin_transaction():
+        context.run_migrations()
+
+
+async def run_migrations_online():
+    engine = create_async_engine(settings.DATABASE_URL)
+    async with engine.connect() as conn:
+        await conn.run_sync(do_run_migrations)
+    await engine.dispose()
+
+
+if context.is_offline_mode():
+    run_migrations_offline()
+else:
+    asyncio.run(run_migrations_online())

features/doc-service/alembic/script.py.mako

@@ -0,0 +1,25 @@
+"""${message}
+
+Revision ID: ${up_revision}
+Revises: ${down_revision | comma,n}
+Create Date: ${create_date}
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+${imports if imports else ""}
+
+revision: str = ${repr(up_revision)}
+down_revision: Union[str, None] = ${repr(down_revision)}
+branch_labels: Union[str, Sequence[str], None] = ${repr(branch_labels)}
+depends_on: Union[str, Sequence[str], None] = ${repr(depends_on)}
+
+
+def upgrade() -> None:
+    ${upgrades if upgrades else "pass"}
+
+
+def downgrade() -> None:
+    ${downgrades if downgrades else "pass"}

features/doc-service/alembic/versions/0001_create_doc_tables.py

@@ -0,0 +1,79 @@
+"""create document tables
+
+Revision ID: 0001
+Revises:
+Create Date: 2026-04-14
+
+"""
+from typing import Sequence, Union
+
+from alembic import op
+import sqlalchemy as sa
+
+revision: str = "0001"
+down_revision: Union[str, None] = None
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "documents",
+        sa.Column("id", sa.String(), primary_key=True),
+        sa.Column("user_id", sa.String(), nullable=False),
+        sa.Column("filename", sa.String(), nullable=False),
+        sa.Column("file_path", sa.String(), nullable=False),
+        sa.Column("file_size", sa.Integer(), nullable=False),
+        sa.Column("status", sa.String(), nullable=False),
+        sa.Column("document_type", sa.String(), nullable=True),
+        sa.Column("raw_text", sa.Text(), nullable=True),
+        sa.Column("extracted_data", sa.Text(), nullable=True),
+        sa.Column("tags", sa.Text(), nullable=True),
+        sa.Column("error_message", sa.String(500), nullable=True),
+        sa.Column(
+            "created_at",
+            sa.DateTime(timezone=True),
+            server_default=sa.text("now()"),
+            nullable=False,
+        ),
+        sa.Column("processed_at", sa.DateTime(timezone=True), nullable=True),
+    )
+    op.create_index("ix_documents_user_id", "documents", ["user_id"])
+
+    op.create_table(
+        "document_categories",
+        sa.Column("id", sa.String(), primary_key=True),
+        sa.Column("user_id", sa.String(), nullable=False),
+        sa.Column("name", sa.String(128), nullable=False),
+        sa.Column(
+            "created_at",
+            sa.DateTime(timezone=True),
+            server_default=sa.text("now()"),
+            nullable=False,
+        ),
+    )
+    op.create_index("ix_document_categories_user_id", "document_categories", ["user_id"])
+
+    op.create_table(
+        "document_category_assignments",
+        sa.Column(
+            "document_id",
+            sa.String(),
+            sa.ForeignKey("documents.id", ondelete="CASCADE"),
+            primary_key=True,
+        ),
+        sa.Column(
+            "category_id",
+            sa.String(),
+            sa.ForeignKey("document_categories.id", ondelete="CASCADE"),
+            primary_key=True,
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_table("document_category_assignments")
+    op.drop_index("ix_document_categories_user_id", "document_categories")
+    op.drop_table("document_categories")
+    op.drop_index("ix_documents_user_id", "documents")
+    op.drop_table("documents")

features/doc-service/alembic/versions/0002_add_document_title.py

@@ -0,0 +1,24 @@
+"""add document title column
+
+Revision ID: 0002
+Revises: 0001
+Create Date: 2026-04-14
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0002"
+down_revision: Union[str, None] = "0001"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column("documents", sa.Column("title", sa.String(500), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column("documents", "title")

features/doc-service/alembic/versions/0003_add_watch_columns.py

@@ -0,0 +1,30 @@
+"""add watch directory columns to documents
+
+Revision ID: 0003
+Revises: 0002
+Create Date: 2026-04-18
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0003"
+down_revision: Union[str, None] = "0002"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column("documents", sa.Column("source", sa.String(16), nullable=False, server_default="upload"))
+    op.add_column("documents", sa.Column("watch_path", sa.String(), nullable=True))
+    op.add_column("documents", sa.Column("suggested_folder", sa.String(128), nullable=True))
+    op.add_column("documents", sa.Column("suggested_filename", sa.String(500), nullable=True))
+
+
+def downgrade() -> None:
+    op.drop_column("documents", "suggested_filename")
+    op.drop_column("documents", "suggested_folder")
+    op.drop_column("documents", "watch_path")
+    op.drop_column("documents", "source")

features/doc-service/alembic/versions/0004_add_document_shares.py

@@ -0,0 +1,42 @@
+"""add document_shares table
+
+Revision ID: 0004
+Revises: 0003
+Create Date: 2026-04-18
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0004"
+down_revision: Union[str, None] = "0003"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.create_table(
+        "document_shares",
+        sa.Column("id", sa.String(), nullable=False),
+        sa.Column("document_id", sa.String(), nullable=False),
+        sa.Column("group_id", sa.String(), nullable=False),
+        sa.Column("shared_by_user_id", sa.String(), nullable=False),
+        sa.Column(
+            "created_at",
+            sa.DateTime(timezone=True),
+            server_default=sa.text("now()"),
+            nullable=False,
+        ),
+        sa.PrimaryKeyConstraint("id"),
+        sa.UniqueConstraint("document_id", "group_id", name="uq_document_group_share"),
+    )
+    op.create_index("ix_document_shares_document_id", "document_shares", ["document_id"])
+    op.create_index("ix_document_shares_group_id", "document_shares", ["group_id"])
+
+
+def downgrade() -> None:
+    op.drop_index("ix_document_shares_group_id", table_name="document_shares")
+    op.drop_index("ix_document_shares_document_id", table_name="document_shares")
+    op.drop_table("document_shares")

features/doc-service/alembic/versions/0005_add_share_can_delete.py

@@ -0,0 +1,32 @@
+"""add can_delete to document_shares
+
+Revision ID: 0005
+Revises: 0004
+Create Date: 2026-04-18
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0005"
+down_revision: Union[str, None] = "0004"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "document_shares",
+        sa.Column(
+            "can_delete",
+            sa.Boolean(),
+            nullable=False,
+            server_default=sa.text("false"),
+        ),
+    )
+
+
+def downgrade() -> None:
+    op.drop_column("document_shares", "can_delete")

features/doc-service/alembic/versions/0006_add_category_scope.py

@@ -0,0 +1,42 @@
+"""add scope and group_id to document_categories
+
+Revision ID: 0006
+Revises: 0005
+Create Date: 2026-04-18
+
+"""
+from typing import Sequence, Union
+
+import sqlalchemy as sa
+from alembic import op
+
+revision: str = "0006"
+down_revision: Union[str, None] = "0005"
+branch_labels: Union[str, Sequence[str], None] = None
+depends_on: Union[str, Sequence[str], None] = None
+
+
+def upgrade() -> None:
+    op.add_column(
+        "document_categories",
+        sa.Column(
+            "scope",
+            sa.String(16),
+            nullable=False,
+            server_default="personal",
+        ),
+    )
+    op.add_column(
+        "document_categories",
+        sa.Column("group_id", sa.String(), nullable=True),
+    )
+    op.create_index("ix_document_categories_group_id", "document_categories", ["group_id"])
+
+    # Migrate existing watch-owned categories to system scope
+    op.execute("UPDATE document_categories SET scope = 'system' WHERE user_id = 'watch'")
+
+
+def downgrade() -> None:
+    op.drop_index("ix_document_categories_group_id", table_name="document_categories")
+    op.drop_column("document_categories", "group_id")
+    op.drop_column("document_categories", "scope")