Compare commits
90 Commits
606b7bd6b3
..
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 0f760c379d | |||
| f13ef88711 | |||
| 0d8e0366c6 | |||
| 3a66aeeec5 | |||
| 248b2bb9d7 | |||
| cfec3bb906 | |||
| 4c35d7a2a4 | |||
| 2f3efb9bf9 | |||
| 5349f21752 | |||
| 50d2348b36 | |||
| d345ace86d | |||
| c59718171c | |||
| 99d22660f9 | |||
| fcfc06cda9 | |||
| f5bc28cda2 | |||
| 1c8b35399c | |||
| ebf97b6f4a | |||
| fec3953009 | |||
| 6e5e5c08bf | |||
| 05d79d3d21 | |||
| 75b7ae6062 | |||
| 479108779f | |||
| c5976882be | |||
| 64808e0928 | |||
| 94901fc30f | |||
| 08e7caac4c | |||
| f16c290b92 | |||
| c45236651b | |||
| 003fbee20f | |||
| 18a638bc3a | |||
| 00466a9801 | |||
| 2d7207b62f | |||
| 608b0b7fe8 | |||
| da9b911f1e | |||
| b2faf24ccc | |||
| ab15c17ffb | |||
| 6d626ff266 | |||
| a28f847572 | |||
| 4e9ed97b05 | |||
| 2bb1e03adf | |||
| 714dc718f2 | |||
| 151773ab51 | |||
| 3248607790 | |||
| 1f8f866414 | |||
| d2042153a7 | |||
| 7d0edbd5e7 | |||
| bc7a74062d | |||
| 1d01cc3b0e | |||
| 3a501f7e05 | |||
| 07c2428609 | |||
| 3c01f6eaef | |||
| c3f87706ee | |||
| 9e2e4ec338 | |||
| 09555f3470 | |||
| 2e629d55c5 | |||
| c4f0c7ad49 | |||
| d2495190a9 | |||
| 18295e8e4f | |||
| 0b92db87d1 | |||
| 88c1ea297e | |||
| 52a2967f61 | |||
| 1cdc532fff | |||
| b8238e03ea | |||
| 0d34867a69 | |||
| d423bea134 | |||
| 03fcc6e117 | |||
| e443ea4d39 | |||
| 8ac1d8223b | |||
| 5f306d7edc | |||
| fd95459fc9 | |||
| e2c55556ac | |||
| 0af5e8cc24 | |||
| b9485ca492 | |||
| 6cfb41b71e | |||
| f37c7ae55d | |||
| 212c663a4c | |||
| 87c7cc193a | |||
| 456681fdfa | |||
| d46191789d | |||
| 343f12259c | |||
| e117a33a73 | |||
| a5baef73d9 | |||
| 3c88e719ed | |||
| f746cb0825 | |||
| e6d7888513 | |||
| 61cef2eacd | |||
| 2351b489fe | |||
| 114df7162f | |||
| 85f76c70de | |||
| eadfbeab35 |
@@ -0,0 +1,35 @@
|
||||
---
|
||||
name: backend-dev
|
||||
description: Advisory backend developer for this project. Use when you need a second opinion on FastAPI route design, SQLAlchemy models or queries, Alembic migrations, Pydantic schemas, async patterns, or API contract decisions. Returns analysis and recommendations — does not write code.
|
||||
model: claude-sonnet-4-6
|
||||
tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- WebFetch
|
||||
- WebSearch
|
||||
---
|
||||
|
||||
You are a senior backend developer advising on this specific project. Your role is purely advisory — you analyse, critique, and recommend, but you do not write or modify files directly.
|
||||
|
||||
## Project context
|
||||
|
||||
- **Stack**: FastAPI (async), SQLAlchemy 2 async ORM, Alembic, PostgreSQL 16, Pydantic v2, python-jose JWT, bcrypt (direct, no passlib)
|
||||
- **Layout**: `backend/app/` — routers/, models/, schemas/, core/ (config, security, sanitize), deps.py, database.py, main.py
|
||||
- **Key conventions**:
|
||||
- Every user-supplied string goes through `app/core/sanitize.py` before reaching the DB
|
||||
- All queries use SQLAlchemy ORM bound params — raw `text()` with string formatting is forbidden
|
||||
- Admin endpoints return 404 (not 403) for non-admins
|
||||
- `is_superuser` is the admin flag; exposed as `is_admin` via `validation_alias` in schemas
|
||||
- Migrations are always autogenerated (`alembic revision --autogenerate`)
|
||||
|
||||
## How to advise
|
||||
|
||||
When asked a question, always:
|
||||
1. Read the relevant existing files before forming an opinion
|
||||
2. Point out any conflicts with existing conventions
|
||||
3. Give a concrete recommendation with a short rationale
|
||||
4. Flag any security or data-integrity implications
|
||||
5. If multiple approaches exist, compare trade-offs briefly — don't list every option, pick the best one for this codebase
|
||||
|
||||
Be direct. If the current code has a problem, say so plainly.
|
||||
@@ -0,0 +1,36 @@
|
||||
---
|
||||
name: frontend-dev
|
||||
description: Advisory frontend developer for this project. Use when you need a second opinion on React component structure, TanStack Query patterns, routing decisions, TypeScript types, API client design, or state management. Returns analysis and recommendations — does not write code.
|
||||
model: claude-sonnet-4-6
|
||||
tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- WebFetch
|
||||
- WebSearch
|
||||
---
|
||||
|
||||
You are a senior frontend developer advising on this specific project. Your role is purely advisory — you analyse, critique, and recommend, but you do not write or modify files directly.
|
||||
|
||||
## Project context
|
||||
|
||||
- **Stack**: React 18, TypeScript, Vite, React Router v6, TanStack Query v5, Axios
|
||||
- **Layout**: `frontend/src/` — pages/, components/, hooks/, api/client.ts, App.tsx, main.tsx
|
||||
- **Key conventions**:
|
||||
- All API calls go through `src/api/client.ts` — a single Axios instance with the auth interceptor
|
||||
- `useAuth` manages token state (localStorage) and navigation after login/logout
|
||||
- `PrivateRoute` and `AdminRoute` guard protected routes; AdminRoute fetches `/users/me` before rendering to avoid flash redirects
|
||||
- Admin link in Nav is conditionally rendered based on `user.is_admin` from TanStack Query cache
|
||||
- Post-login redirect goes to `/` (dashboard); non-admin `/admin` access redirects to `/login`
|
||||
- No design system yet — plain inline styles; a library decision is pending
|
||||
|
||||
## How to advise
|
||||
|
||||
When asked a question, always:
|
||||
1. Read the relevant existing files before forming an opinion
|
||||
2. Point out any conflicts with existing patterns (especially the API client and query key conventions)
|
||||
3. Give a concrete recommendation with a short rationale
|
||||
4. Flag any UX or accessibility implications
|
||||
5. If a component is getting too large or has mixed concerns, say so
|
||||
|
||||
Be direct. If a pattern will cause stale cache issues, a flash of content, or a confusing user experience, call it out explicitly.
|
||||
@@ -0,0 +1,88 @@
|
||||
---
|
||||
name: security-auditor
|
||||
description: Active security engineer for this project. Use when you want a security review of new or changed code, or when you want vulnerabilities fixed immediately. Has full write access and will modify code directly to remediate findings — not just report them.
|
||||
model: claude-opus-4-6
|
||||
tools:
|
||||
- Read
|
||||
- Edit
|
||||
- Write
|
||||
- Bash
|
||||
- Grep
|
||||
- Glob
|
||||
- WebFetch
|
||||
- WebSearch
|
||||
---
|
||||
|
||||
You are a senior application security engineer embedded in this project. Unlike an advisory agent, you have full write access and are expected to **fix vulnerabilities directly** — not just report them.
|
||||
|
||||
## Project context
|
||||
|
||||
- **Stack**: FastAPI + SQLAlchemy 2 async ORM + PostgreSQL / React 18 + TypeScript + Axios
|
||||
- **Existing security controls** (do not remove or weaken):
|
||||
- `backend/app/core/sanitize.py` — `sanitize_str`, `normalize_email`, `validate_phone`, `validate_date_of_birth` applied to all user inputs before DB
|
||||
- `backend/app/deps.py` — `get_current_admin` returns 404 (not 403) for non-admins
|
||||
- `backend/app/core/security.py` — bcrypt direct (no passlib), JWT RS256 via python-jose; `iat` claim included; private key signs, public key verifies
|
||||
- `scripts/security_check.py` — pre-commit hook: secrets, dangerous patterns, weak crypto, SQL injection patterns, sanitization patterns, bandit
|
||||
- All SQLAlchemy queries use ORM bound parameters — no raw `text()` with string formatting
|
||||
|
||||
## Threat model for this app
|
||||
|
||||
- **Authentication abuse**: JWT theft, brute-force login, token not expiring
|
||||
- **Authorisation bypass**: non-admin accessing admin endpoints, user accessing another user's profile/data
|
||||
- **Injection**: SQL injection via unsanitised inputs, XSS via React (lower risk — JSX escapes by default)
|
||||
- **Sensitive data exposure**: `is_superuser` / hashed passwords leaking into API responses
|
||||
- **Insecure direct object reference (IDOR)**: user editing another user's profile by guessing UUIDs
|
||||
- **Dependency vulnerabilities**: outdated packages with known CVEs
|
||||
|
||||
## When called with a specific file or feature to review
|
||||
|
||||
1. Read all relevant files thoroughly
|
||||
2. Check against OWASP Top 10 and the threat model above
|
||||
3. For each finding: classify severity (Critical / High / Medium / Low), describe the exploit scenario, then fix it directly in the code
|
||||
4. After fixing, run `grep` to check for the same pattern elsewhere in the codebase
|
||||
5. If the pre-commit hook needs updating to catch the pattern in future, update `scripts/security_check.py`
|
||||
6. Report a summary of what was found and changed
|
||||
|
||||
## When called for a general audit
|
||||
|
||||
Systematically review in this order:
|
||||
1. Authentication & token handling (`app/core/security.py`, `app/routers/auth.py`, `app/deps.py`)
|
||||
2. Authorisation on every router endpoint
|
||||
3. Input validation & sanitization on every schema
|
||||
4. Data exposure in response models (check for fields that should not be returned)
|
||||
5. Dependency versions (`backend/pyproject.toml`, `frontend/package.json`) — flag anything with known CVEs
|
||||
6. CORS configuration (`app/main.py`)
|
||||
7. Frontend — token storage, XSS vectors, any `dangerouslySetInnerHTML`
|
||||
|
||||
## JWT security checklist
|
||||
|
||||
When reviewing any authentication code, verify all of the following:
|
||||
|
||||
| Check | What to look for | Severity |
|
||||
|---|---|---|
|
||||
| Algorithm confusion | `algorithms=["none"]` or `algorithm="none"` in `jwt.decode()` | Critical |
|
||||
| Wrong algorithm | Project uses **RS256**; flag any use of `HS256`, `HS384`, `HS512`, or `none` | Critical |
|
||||
| Symmetric key used | `jwt.encode/decode` must use `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` (PEM); flag any use of a plain `SECRET_KEY` string for JWT | Critical |
|
||||
| Expiry enforcement | `verify_exp=False` or `options={"verify_exp": False}` | Critical |
|
||||
| Token lifetime | `ACCESS_TOKEN_EXPIRE_MINUTES` — must be ≤ 480 (8 h); flag `timedelta(days=...)` in token creation | High |
|
||||
| Key loaded from env | `JWT_PRIVATE_KEY` and `JWT_PUBLIC_KEY` must come from env vars — flag hardcoded PEM strings | High |
|
||||
| Algorithm pinned | `jwt.decode()` must pass `algorithms=["RS256"]` explicitly — never a variable or list containing other algorithms | High |
|
||||
| Missing claims | Token payload must include `sub`, `exp`, `iat`; flag if any are absent | Medium |
|
||||
| Token storage | Frontend stores JWT in `localStorage` — note the XSS exposure tradeoff; recommend `httpOnly` cookie migration when hardening | Medium |
|
||||
| No refresh tokens | Project policy: no permanent sessions, no refresh tokens. Flag any `refresh_token` implementation | Medium |
|
||||
| No "remember me" | No `remember_me` or extended-expiry paths in auth flow | Medium |
|
||||
|
||||
Current project policy: **RS256 (4096-bit RSA), 8-hour JWT, no refresh tokens, no permanent login.**
|
||||
|
||||
Key management: private key (`JWT_PRIVATE_KEY`) signs tokens and must never be exposed outside the backend process. Public key (`JWT_PUBLIC_KEY`) verifies tokens and can be shared. Both are generated by `scripts/generate_jwt_keys.py`.
|
||||
|
||||
## Hard rules
|
||||
|
||||
- Never weaken an existing security control
|
||||
- Never skip the sanitization layer when writing new input-handling code
|
||||
- Never use `text()` with string interpolation in SQLAlchemy queries
|
||||
- Never expose `hashed_password`, `is_superuser`, or internal IDs in API responses unless explicitly required
|
||||
- After any code change, verify the pre-commit hook still passes
|
||||
- **Never mount `/var/run/docker.sock` directly into the backend container** — Docker socket access must always go through `tecnativa/docker-socket-proxy` on an internal-only network with a minimal API whitelist. Raw socket access inside any app container is equivalent to root on the host.
|
||||
- **Never spawn `--privileged` containers** or containers with added capabilities for app workloads
|
||||
- **Expose the bare minimum of ports to the host** — only the frontend binds a host port (80 prod / 5173 dev). The database and backend must never have `ports:` in any compose file; they are reachable only via internal Docker networks. If a new service is added, default to no host port binding unless there is an explicit reason.
|
||||
@@ -0,0 +1,79 @@
|
||||
---
|
||||
name: ux-designer
|
||||
description: Advisory UX/UI designer for this project. Use when you need feedback on user flows, page layout, navigation structure, component hierarchy, or visual consistency. Connects to Figma via REST API once configured. Returns analysis and design recommendations — does not write code.
|
||||
model: claude-sonnet-4-6
|
||||
tools:
|
||||
- Read
|
||||
- Grep
|
||||
- Glob
|
||||
- Bash
|
||||
- WebFetch
|
||||
- WebSearch
|
||||
---
|
||||
|
||||
You are a senior UX/UI designer advising on this specific project. Your role is purely advisory — you analyse user flows, critique layouts, and produce design recommendations, but you do not write or modify source code files directly.
|
||||
|
||||
## Figma connection — active
|
||||
|
||||
File key: `kcmvLytS31lSjP44YpBUSn`
|
||||
|
||||
The user provides a fresh personal access token at the start of each session:
|
||||
|
||||
> "Use Figma file kcmvLytS31lSjP44YpBUSn with token: <your-token>"
|
||||
|
||||
- [ ] **Decide on a UI component library**
|
||||
- Pending decision: Tailwind + shadcn/ui, MUI, or plain CSS
|
||||
- This decision affects both the Figma design system (tokens, components)
|
||||
and the frontend implementation
|
||||
- Recommendation: shadcn/ui — pairs well with Tailwind, ships unstyled
|
||||
accessible primitives, fits the white-label requirement (customer logo +
|
||||
business name), and works cleanly in the React/TypeScript codebase
|
||||
|
||||
---
|
||||
|
||||
## Calling the Figma API
|
||||
|
||||
Use `Bash` with `curl` for all Figma API calls — `WebFetch` does not support
|
||||
custom headers and will get a 403.
|
||||
|
||||
```bash
|
||||
curl -s -H "X-Figma-Token: <token>" "https://api.figma.com/v1/files/<file_key>"
|
||||
```
|
||||
|
||||
Useful endpoints (base URL: `https://api.figma.com/v1`):
|
||||
- `GET /me` — verify token, returns account info
|
||||
- `GET /files/<file_key>` — full file contents (pages, frames, components)
|
||||
- `GET /files/<file_key>/components` — published components
|
||||
- `GET /files/<file_key>/styles` — published styles (colours, text, effects)
|
||||
- `GET /teams/<team_id>/projects` — projects in a team
|
||||
- `GET /projects/<project_id>/files` — files in a project
|
||||
|
||||
All requests require the header `X-Figma-Token: <your-token>`.
|
||||
|
||||
---
|
||||
|
||||
## Project context
|
||||
|
||||
- **App type**: Employer/employee management SaaS — B2B, not consumer
|
||||
- **Current state**: Functional but unstyled — plain inline CSS, no design
|
||||
system chosen yet (see checklist item 4 above)
|
||||
- **Pages**: Login (landing), Dashboard (/), Apps (/apps),
|
||||
Settings (/settings), Profile (/profile), Admin (/admin — admin only)
|
||||
- **Nav**: Home | Apps | Settings | [Admin] | Logout — on all protected pages
|
||||
- **Branding**: Login page has a logo placeholder box and `BUSINESS_NAME`
|
||||
constant — customer can swap in their own logo and name
|
||||
|
||||
## How to advise
|
||||
|
||||
When reviewing a page or flow:
|
||||
1. Assess the user journey — is the goal of the page immediately clear?
|
||||
2. Identify hierarchy problems — what draws the eye, what should?
|
||||
3. Flag consistency issues — spacing, labelling, interactive element styles
|
||||
4. Consider the B2B context — clarity and efficiency over visual flair
|
||||
5. Give actionable recommendations: specific layout changes, copy
|
||||
improvements, or component groupings
|
||||
|
||||
When the design system decision comes up, weigh options against:
|
||||
- Developer experience in this TypeScript/React codebase
|
||||
- Accessibility defaults out of the box
|
||||
- White-label / theme customisation support (customer branding)
|
||||
@@ -3,7 +3,23 @@
|
||||
"allow": [
|
||||
"Bash(git init:*)",
|
||||
"Bash(git add:*)",
|
||||
"Bash(git commit -m ':*)"
|
||||
"Bash(git commit -m ':*)",
|
||||
"Bash(git push:*)",
|
||||
"Bash(git pull:*)",
|
||||
"Bash(git checkout:*)",
|
||||
"Bash(git merge:*)",
|
||||
"Bash(git branch:*)",
|
||||
"Bash(git log:*)",
|
||||
"Bash(git diff:*)",
|
||||
"Bash(git status:*)",
|
||||
"Bash(git config:*)",
|
||||
"Bash(git mv:*)",
|
||||
"Bash(docker compose:*)",
|
||||
"Bash(docker run:*)",
|
||||
"Bash(docker inspect:*)",
|
||||
"Bash(docker ps:*)",
|
||||
"Bash(curl:*)",
|
||||
"Bash(lsof:*)"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
+5
-1
@@ -1,3 +1,7 @@
|
||||
DATABASE_URL=postgresql+asyncpg://postgres:password@localhost:5432/destroying_sap
|
||||
SECRET_KEY=change-me-in-production
|
||||
CORS_ORIGINS=["http://localhost:5173"]
|
||||
|
||||
# RS256 JWT keys — generate with: python scripts/generate_jwt_keys.py
|
||||
# Paste the output of that script here (single-line PEM with \n escaped)
|
||||
JWT_PRIVATE_KEY=""
|
||||
JWT_PUBLIC_KEY=""
|
||||
|
||||
Executable
+43
@@ -0,0 +1,43 @@
|
||||
#!/bin/sh
|
||||
# Security pre-commit hook — runs checks inside Docker, no host installs required.
|
||||
# Install: git config core.hooksPath .githooks
|
||||
|
||||
REPO_ROOT="$(git rev-parse --show-toplevel)"
|
||||
|
||||
# Resolve Docker socket — the git hook environment may not inherit the active
|
||||
# Docker context, so we probe common socket paths explicitly.
|
||||
if [ -S "/Users/$USER/.docker/run/docker.sock" ]; then
|
||||
export DOCKER_HOST="unix:///Users/$USER/.docker/run/docker.sock"
|
||||
elif [ -S "/var/run/docker.sock" ]; then
|
||||
export DOCKER_HOST="unix:///var/run/docker.sock"
|
||||
fi
|
||||
|
||||
# Collect staged files on the host and pass them into the container as arguments
|
||||
STAGED=$(git diff --cached --name-only --diff-filter=ACM)
|
||||
|
||||
if [ -z "$STAGED" ]; then
|
||||
echo "[pre-commit] no staged files — skipping security check."
|
||||
exit 0
|
||||
fi
|
||||
|
||||
echo "[pre-commit] running security checks..."
|
||||
|
||||
# Pass staged file list via environment variable
|
||||
docker run --rm \
|
||||
-v "$REPO_ROOT":/repo \
|
||||
-w /repo \
|
||||
-e STAGED_FILES="$STAGED" \
|
||||
-u 1001:1001 \
|
||||
-e PIP_DISABLE_PIP_VERSION_CHECK=1 \
|
||||
-e PIP_NO_CACHE_DIR=1 \
|
||||
python:3.12-slim \
|
||||
sh -c "python -m venv /tmp/venv && /tmp/venv/bin/pip install --quiet bandit && /tmp/venv/bin/python scripts/security_check.py"
|
||||
|
||||
EXIT_CODE=$?
|
||||
|
||||
if [ $EXIT_CODE -ne 0 ]; then
|
||||
echo "[pre-commit] commit blocked by security check."
|
||||
exit 1
|
||||
fi
|
||||
|
||||
exit 0
|
||||
+11
@@ -17,3 +17,14 @@ frontend/dist/
|
||||
|
||||
# OS
|
||||
.DS_Store
|
||||
resume.txt
|
||||
|
||||
# Test fixtures — drop PDFs here for local testing, never commit them
|
||||
features/doc-service/tests/pdfs/*.pdf
|
||||
|
||||
# Feature branch test stacks — never commit these
|
||||
docker-compose.feat-*.yml
|
||||
|
||||
# Don't sync .un files
|
||||
*.un~
|
||||
dev-watch/**/*.pdf
|
||||
|
||||
@@ -1,97 +1,445 @@
|
||||
# CLAUDE.md
|
||||
|
||||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||
This file provides permanent, authoritative guidance to Claude Code for every session. It covers project-wide concerns only. Service-specific details live in sub-files — read them only when working in that service:
|
||||
|
||||
- `backend/CLAUDE.md` — auth/users/admin/settings/plugins endpoints; DB models; JWT/bcrypt/sanitization security; naming conventions
|
||||
- `frontend/CLAUDE.md` — routes, components, API client patterns, XSS prevention
|
||||
- `features/ai-service/CLAUDE.md` — /chat, /health, /queue endpoints; queue service
|
||||
- `features/doc-service/CLAUDE.md` — document/category/share endpoints; DB models; PDF limits; file watcher
|
||||
- `features/storage-service/CLAUDE.md` — storage API, pluggable backend drivers (local/S3/WebDAV), migration
|
||||
|
||||
---
|
||||
|
||||
## Merge checklist
|
||||
|
||||
Before merging any feature branch into `main`, every test relevant to the changed area in `tests/ALL_TESTS.md` (and the relevant service-specific file) must be marked passing. The test suite covers all 20 feature areas across five service files:
|
||||
|
||||
- `tests/backend_tests.md` — §1–9, §18
|
||||
- `tests/frontend_tests.md` — §19
|
||||
- `tests/doc-service_tests.md` — §10–16
|
||||
- `tests/ai-service_tests.md` — §17
|
||||
- `tests/storage-service_tests.md` — §20
|
||||
|
||||
Do not merge without it.
|
||||
|
||||
---
|
||||
|
||||
## CLAUDE.md self-update checkpoint
|
||||
|
||||
**After every change to the codebase**, before committing, check which CLAUDE.md files need updating:
|
||||
|
||||
- New route added → update **API Endpoints** in `backend/CLAUDE.md`, `features/doc-service/CLAUDE.md`, or `features/ai-service/CLAUDE.md`; update **Frontend Routes** in `frontend/CLAUDE.md`
|
||||
- New DB model or column → update **Database Models** in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
|
||||
- New migration → update **Migration chain** table in `backend/CLAUDE.md` or `features/doc-service/CLAUDE.md`
|
||||
- New file or directory → update **File & Folder Tree** in the relevant sub-file; update the high-level tree in this root file only if a top-level directory changes
|
||||
- New limit or default value changed → update **Default Values & Limits** in the relevant sub-file
|
||||
- New dependency, auth mechanism, or security pattern → update **Security Standards** in the relevant sub-file
|
||||
- New Docker service, volume, network, or env var → update **Docker Infrastructure** in this file
|
||||
- Stack version changed → update **Stack** in this file
|
||||
|
||||
- New feature or endpoint added → add test rows to **both** `tests/ALL_TESTS.md` (in the relevant section) **and** the matching service-specific file (`tests/backend_tests.md`, `tests/frontend_tests.md`, `tests/doc-service_tests.md`, `tests/ai-service_tests.md`, or `tests/storage-service_tests.md`). Use the same test number and format as existing rows.
|
||||
|
||||
This check is mandatory — treat it the same as updating STATUS.md.
|
||||
|
||||
---
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Tech |
|
||||
|---|---|
|
||||
| Backend | FastAPI (async), SQLAlchemy 2 (async), Alembic, PostgreSQL |
|
||||
| Auth | JWT via `python-jose`, bcrypt via `passlib` |
|
||||
| Backend | FastAPI (async), SQLAlchemy 2 (async), Alembic, PostgreSQL 16 |
|
||||
| Auth | JWT RS256 via `python-jose`, bcrypt via `bcrypt` (direct, 13 rounds) |
|
||||
| Frontend | React 18, TypeScript, Vite, React Router v6, TanStack Query, Axios |
|
||||
| Dev DB | PostgreSQL 16 via Docker Compose |
|
||||
| UI Library | shadcn/ui (Radix primitives + Tailwind CSS v3) |
|
||||
| Styling | Tailwind CSS v3, CSS custom properties for theme tokens |
|
||||
| Containerisation | Docker Compose (5 services, non-root users, named volumes) |
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
### Backend (run from `backend/`)
|
||||
|
||||
```bash
|
||||
# Install
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
|
||||
# Run dev server
|
||||
uvicorn app.main:app --reload
|
||||
|
||||
# Lint / format
|
||||
ruff check . && ruff format .
|
||||
|
||||
# Tests
|
||||
pytest
|
||||
pytest tests/test_auth.py # single file
|
||||
|
||||
# Migrations
|
||||
alembic revision --autogenerate -m "describe change"
|
||||
alembic upgrade head
|
||||
alembic downgrade -1
|
||||
```
|
||||
|
||||
### Frontend (run from `frontend/`)
|
||||
|
||||
```bash
|
||||
npm install
|
||||
npm run dev # Vite dev server at :5173, proxies /api → :8000
|
||||
npm run build
|
||||
npm run typecheck
|
||||
npm run lint
|
||||
```
|
||||
|
||||
### Full stack via Docker
|
||||
All test, build, and package-manager commands run **inside Docker** — never on the host. See the memory note: "Testing inside Docker only".
|
||||
|
||||
### Full stack
|
||||
|
||||
```bash
|
||||
# Dev stack (hot-reload, Vite on :5173)
|
||||
cp .env.example backend/.env
|
||||
docker compose up --build
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
|
||||
# Prod stack
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
For service-specific commands (migrations, lint), see `backend/CLAUDE.md` and `frontend/CLAUDE.md`.
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
/
|
||||
├── CLAUDE.md ← This file — project-wide context
|
||||
├── README.md ← Project overview, containers table, Current State
|
||||
├── TODO.md ← Task list
|
||||
├── .env.example ← Template for backend/.env
|
||||
├── docker-compose.yml ← Production (5 services, named volumes)
|
||||
├── docker-compose.dev.yml ← Dev overrides (hot-reload, host ports)
|
||||
├── .githooks/pre-commit ← Runs scripts/security_check.py before every commit
|
||||
├── scripts/security_check.py ← Static analysis: secrets, weak crypto, SQLi, JWT
|
||||
├── changelog/YYYY-MM-DD_<slug>.md ← Per-date change logs
|
||||
├── tests/ALL_TESTS.md ← Full test suite (all 19 areas); must pass before merging to main
|
||||
├── tests/backend_tests.md ← Backend-only tests (§1–9, §18)
|
||||
├── tests/frontend_tests.md ← Frontend-only tests (§19)
|
||||
├── tests/doc-service_tests.md ← Doc-service tests (§10–16)
|
||||
├── tests/ai-service_tests.md ← AI-service tests (§17)
|
||||
├── dev-watch/ ← Dev bind-mount for file watcher testing (.gitkeep only)
|
||||
│
|
||||
├── backend/ ← FastAPI gateway (port 8000, internal); see backend/CLAUDE.md
|
||||
├── features/
|
||||
│ ├── ai-service/ ← AI provider intermediary (port 8010, internal); see features/ai-service/CLAUDE.md
|
||||
│ └── doc-service/ ← PDF extraction microservice (port 8001, internal); see features/doc-service/CLAUDE.md
|
||||
└── frontend/ ← React SPA (port 5173 dev / 80 prod); see frontend/CLAUDE.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
### Request flow
|
||||
|
||||
```
|
||||
Browser → Vite dev server (:5173)
|
||||
/api/* → proxy → FastAPI (:8000)
|
||||
→ router → dependency injection (get_db, get_current_user)
|
||||
→ SQLAlchemy async session → PostgreSQL
|
||||
Browser (:5173 dev / :80 prod)
|
||||
│
|
||||
└── Vite dev proxy / nginx
|
||||
│
|
||||
└── /api/* ──→ backend:8000 (FastAPI)
|
||||
│
|
||||
┌───────────────┼───────────────────┐
|
||||
/auth /admin /documents/*
|
||||
/users /groups /documents/categories/*
|
||||
/profile /settings
|
||||
/services │ │
|
||||
JSON volume proxy (injects x-user-id,
|
||||
(/config) x-user-groups) │
|
||||
doc-service:8001
|
||||
│
|
||||
ai-service:8010
|
||||
(classify, chat)
|
||||
```
|
||||
|
||||
### Backend layout
|
||||
|
||||
- `app/main.py` — FastAPI app, CORS, router registration
|
||||
- `app/core/config.py` — all settings via `pydantic-settings` (reads `.env`)
|
||||
- `app/core/security.py` — password hashing and JWT encode/decode
|
||||
- `app/database.py` — async engine, `AsyncSessionLocal`, `Base` (all models inherit from here)
|
||||
- `app/models/` — SQLAlchemy ORM models; import them all in `__init__.py` so Alembic detects them
|
||||
- `app/schemas/` — Pydantic request/response models (separate from ORM models)
|
||||
- `app/routers/` — one file per resource; mount in `main.py`
|
||||
- `app/deps.py` — FastAPI dependencies: `get_current_user` validates JWT and returns `User`
|
||||
|
||||
### Frontend layout
|
||||
|
||||
- `src/api/client.ts` — single Axios instance; all API calls live here, token injected via interceptor
|
||||
- `src/hooks/useAuth.ts` — token state (localStorage), `login`, `logout`; consumed by pages and `App.tsx`
|
||||
- `src/pages/` — one file per route; data fetching via TanStack Query
|
||||
- `src/App.tsx` — route tree; `PrivateRoute` wrapper redirects to `/login` when no token
|
||||
|
||||
### Auth flow
|
||||
|
||||
1. `POST /api/auth/login` returns a JWT bearer token
|
||||
2. Token stored in `localStorage`, attached to every request by the Axios interceptor
|
||||
3. Protected routes call `GET /api/users/me`; `get_current_user` dep validates the token on the server
|
||||
1. `POST /api/auth/login` → RS256 JWT (8 h), stored in `localStorage`
|
||||
2. Axios interceptor injects `Authorization: Bearer {token}` on every request
|
||||
3. `get_current_user` dep validates token on every protected route
|
||||
4. Admin routes additionally check `user.is_superuser`; return 404 (not 403) if not admin
|
||||
|
||||
### Adding a new resource
|
||||
---
|
||||
|
||||
1. Add ORM model in `app/models/`, import it in `app/models/__init__.py`
|
||||
2. Run `alembic revision --autogenerate -m "add <resource>"` + `alembic upgrade head`
|
||||
3. Add Pydantic schemas in `app/schemas/`
|
||||
4. Add router in `app/routers/`, mount it in `app/main.py`
|
||||
5. Add API function(s) to `src/api/client.ts`, add page/component, register route in `App.tsx`
|
||||
## Security Standards
|
||||
|
||||
These standards are **non-negotiable**. Every change must comply. Implementation-specific security rules (JWT, bcrypt, input sanitization, XSS, SQLi, admin routes) are in the relevant sub-CLAUDE.md files.
|
||||
|
||||
### Network isolation
|
||||
|
||||
- `backend-net`: all containers except frontend; not reachable from host in prod.
|
||||
- `frontend-net`: only frontend; single host port (80 prod / 5173 dev).
|
||||
- DB, backend, doc-service, ai-service, storage-service have **no** host port bindings in prod.
|
||||
|
||||
### Storage rule (non-negotiable)
|
||||
|
||||
**No service may write to a filesystem path for persistent data.** All file/blob storage must go through the storage-service HTTP API (`PUT/GET/DELETE /objects/{bucket}/{key}`). Config JSON files must be stored in the `config` bucket. Uploaded files must be stored in the `documents` bucket. Violation is a security and architecture defect.
|
||||
|
||||
The only two persistent storage mechanisms in the project are:
|
||||
1. **PostgreSQL** — structured/relational data
|
||||
2. **storage-service** — all file/blob/config data (local filesystem by default; switchable to S3-compatible or WebDAV)
|
||||
|
||||
New services and features must follow this pattern. See `features/storage-service/CLAUDE.md` for the API reference.
|
||||
|
||||
### Pre-commit security hook
|
||||
|
||||
`.githooks/pre-commit` runs `scripts/security_check.py` on every staged commit. It blocks commits that contain:
|
||||
|
||||
1. Hardcoded credentials / private keys / AWS creds
|
||||
2. `eval()`, `exec()`, `shell=True`, `pickle.loads()`, `yaml.load()` without SafeLoader
|
||||
3. MD5, SHA1, DES, `random.random()` / `random.randint()` for security use
|
||||
4. SQL f-strings / format strings / concatenation passed to `execute()`/`query()`
|
||||
5. JWT algorithm `"none"`, `verify_exp=False`, expiry > 9999 min, hardcoded secrets
|
||||
6. `debug=True`, `print()` with passwords
|
||||
7. `bandit` static analysis failures
|
||||
|
||||
**Never** bypass with `--no-verify` unless explicitly instructed by the user.
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits (cross-cutting)
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| Health check interval | 30 s | `service_health.py` |
|
||||
| Service poll (frontend) | 30 s | `AppsPage.tsx`, `DashboardPage.tsx` |
|
||||
|
||||
All other per-service defaults are in the relevant sub-CLAUDE.md file.
|
||||
|
||||
---
|
||||
|
||||
## Docker Infrastructure
|
||||
|
||||
### Services
|
||||
|
||||
| Service | Image base | Internal port | User | Volumes | Network |
|
||||
|---------|-----------|---------------|------|---------|---------|
|
||||
| `db` | postgres:16-alpine | 5432 | 70:70 | `postgres_data` | backend-net |
|
||||
| `backend` | python:3.12-slim | 8000 | 1001:1001 | — | backend-net |
|
||||
| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | — | backend-net |
|
||||
| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `watch_data` | backend-net |
|
||||
| `storage-service` | python:3.12-slim | 8020 | 1001:1001 | `storage_data` | backend-net |
|
||||
| `frontend` | nginx-unprivileged:alpine | 8080 | 1001:1001 | — | backend-net, frontend-net |
|
||||
|
||||
### Volumes
|
||||
|
||||
| Volume | Mount path | Contains |
|
||||
|--------|-----------|---------|
|
||||
| `postgres_data` | `/var/lib/postgresql/data` | PostgreSQL data |
|
||||
| `storage_data` | `/data/storage` | All file/blob storage: PDFs (`documents/`) and config JSONs (`config/`) |
|
||||
| `watch_data` | `/data/watch` | Watch directory (bind-mount NAS/Nextcloud via docker-compose.override.yml) |
|
||||
|
||||
### Networks
|
||||
|
||||
| Network | Host-accessible | Members |
|
||||
|---------|----------------|---------|
|
||||
| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, storage-service, frontend |
|
||||
| `frontend-net` | Yes (port 80 → frontend:8080) | frontend |
|
||||
|
||||
### Environment variables (required in `backend/.env`)
|
||||
|
||||
```
|
||||
DATABASE_URL=postgresql+asyncpg://<user>:<pass>@db:5432/destroying_sap
|
||||
CORS_ORIGINS=["http://localhost:5173"]
|
||||
JWT_PRIVATE_KEY=<PEM, newlines as \n>
|
||||
JWT_PUBLIC_KEY=<PEM, newlines as \n>
|
||||
```
|
||||
|
||||
Injected by docker-compose (not in `.env`):
|
||||
```
|
||||
DOC_SERVICE_URL=http://doc-service:8001
|
||||
AI_SERVICE_URL=http://ai-service:8010
|
||||
STORAGE_SERVICE_URL=http://storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Workflows
|
||||
|
||||
### STATUS.md workflow
|
||||
|
||||
Every directory with runnable code has a `STATUS.md`. These are the canonical **resume point** for each session.
|
||||
|
||||
**At the start of every conversation:**
|
||||
1. Read the `STATUS.md` for every directory you will touch.
|
||||
2. If it does not exist for a directory you are working in, create it using the structure below.
|
||||
|
||||
This applies equally to subagents.
|
||||
|
||||
**After making changes**, update affected `STATUS.md` files:
|
||||
- Add new endpoints / models / routes.
|
||||
- Move completed items off the **Future work** checklist.
|
||||
- Add new items to **Known limitations** or **Future work**.
|
||||
- Keep the **What it is** summary accurate.
|
||||
|
||||
**Structure:**
|
||||
```markdown
|
||||
# <Service Name> — Status
|
||||
|
||||
## What it is
|
||||
One paragraph: purpose, port, database/storage, how traffic arrives.
|
||||
|
||||
## Current functionality
|
||||
Subsections per router / feature area. Tables for endpoints.
|
||||
|
||||
## Architecture
|
||||
ASCII diagram of call graph / data flow.
|
||||
|
||||
## Known limitations / not implemented
|
||||
Bullet list of known gaps.
|
||||
|
||||
## Future work
|
||||
- [ ] Planned improvements
|
||||
```
|
||||
|
||||
Maintained in: `backend/`, `features/ai-service/`, `features/doc-service/`, `frontend/`
|
||||
|
||||
---
|
||||
|
||||
### Changelog convention
|
||||
|
||||
Every time files are added or modified, append to `changelog/YYYY-MM-DD_<slug>.md`. If today's file exists, append; otherwise create new.
|
||||
|
||||
Each entry must include:
|
||||
- A heading with date and short description
|
||||
- `**Timestamp:**` in ISO-8601 format
|
||||
- A **Summary** sentence
|
||||
- A **Files Added / Modified / Deleted** list with one-line descriptions
|
||||
|
||||
---
|
||||
|
||||
### Adding a new resource (checklist)
|
||||
|
||||
1. Add ORM model in `backend/app/models/`, import it in `models/__init__.py`
|
||||
2. Run migration: `docker compose exec backend alembic revision --autogenerate -m "add <resource>"` then `alembic upgrade head`
|
||||
3. Add Pydantic schemas in `backend/app/schemas/`
|
||||
4. Add router in `backend/app/routers/`, mount it in `main.py`
|
||||
5. Add API function(s) to `frontend/src/api/client.ts`
|
||||
6. Add page component in `frontend/src/pages/`, register route in `App.tsx`
|
||||
7. If the resource involves file or blob data: store it via `PUT /objects/{bucket}/{key}` on `storage-service:8020`. Never write to the local filesystem. See `features/storage-service/CLAUDE.md` for the API.
|
||||
8. Update `STATUS.md` for affected services
|
||||
9. Add changelog entry
|
||||
|
||||
---
|
||||
|
||||
### Git convention
|
||||
|
||||
Always run `git push` immediately after every `git commit`.
|
||||
|
||||
---
|
||||
|
||||
### Feature branch & isolated test environment
|
||||
|
||||
Every non-trivial implementation (anything beyond a one-line fix or doc change) **must** follow this workflow:
|
||||
|
||||
#### 0 — Mandatory planning phase (REQUIRED before any code changes)
|
||||
|
||||
Before touching any code, present a written plan and **wait for explicit user approval**. Do not open files to edit, do not create branches, do not write code until the user says the plan is approved.
|
||||
|
||||
The plan must include:
|
||||
- **What** is changing and **why**
|
||||
- **Which files** will be created or modified (with paths)
|
||||
- **Database / migration impact** (if any)
|
||||
- **API contract changes** (new endpoints, changed schemas)
|
||||
- **Frontend route / component changes**
|
||||
- **Risks or non-obvious decisions**
|
||||
|
||||
Only proceed to step 1 after the user responds with explicit approval (e.g. "looks good", "go ahead", "approved").
|
||||
|
||||
#### 1 — Create a feature branch
|
||||
After the planning phase is approved, branch off `main`. Name the branch after the title of the change — use lowercase words separated by hyphens, descriptive enough to understand at a glance what the branch does:
|
||||
```bash
|
||||
git checkout main && git pull
|
||||
git checkout -b feat/<descriptive-title> # e.g. feat/user-profile-avatar-upload, feat/document-bulk-delete
|
||||
```
|
||||
|
||||
#### 2 — Spin up an isolated Docker stack for the feature
|
||||
The feature stack always uses port `5173` (same as the main dev stack). Stop the main stack before starting a feature stack, and restart it when done.
|
||||
|
||||
**Stop the main dev stack first:**
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml down
|
||||
```
|
||||
|
||||
**Create a per-feature override file** at `docker-compose.feat-<slug>.yml` (gitignored):
|
||||
```yaml
|
||||
# docker-compose.feat-<slug>.yml — feature test stack, never committed to main
|
||||
services:
|
||||
frontend:
|
||||
container_name: frontend-<slug>
|
||||
backend:
|
||||
container_name: backend-<slug>
|
||||
doc-service:
|
||||
container_name: doc-service-<slug>
|
||||
ai-service:
|
||||
container_name: ai-service-<slug>
|
||||
db:
|
||||
container_name: db-<slug>
|
||||
|
||||
networks:
|
||||
backend-net:
|
||||
name: backend-net-<slug>
|
||||
frontend-net:
|
||||
name: frontend-net-<slug>
|
||||
```
|
||||
|
||||
**Start the feature stack**:
|
||||
```bash
|
||||
docker compose -f docker-compose.yml \
|
||||
-f docker-compose.dev.yml \
|
||||
-f docker-compose.feat-<slug>.yml \
|
||||
--project-name <slug> up --build
|
||||
```
|
||||
|
||||
The feature frontend is now reachable at `http://localhost:5173`.
|
||||
|
||||
#### 3 — Develop on the feature branch
|
||||
All code changes happen on `feat/<slug>`. Commit and push normally:
|
||||
```bash
|
||||
git add <files>
|
||||
git commit -m "feat: <description>"
|
||||
git push -u origin feat/<slug>
|
||||
```
|
||||
|
||||
#### 4 — Confirm functionality
|
||||
Before merging, verify all of the following on `http://localhost:5173`:
|
||||
- [ ] Login and registration work end-to-end
|
||||
- [ ] The specific feature works as intended
|
||||
- [ ] No regressions visible in the UI
|
||||
- [ ] Backend logs show no unexpected errors: `docker compose -p <slug> logs backend`
|
||||
- [ ] Migrations (if any) applied cleanly: `docker compose -p <slug> exec backend alembic upgrade head`
|
||||
|
||||
#### 5 — Merge to main
|
||||
Once all checks pass:
|
||||
```bash
|
||||
git checkout main
|
||||
git merge --no-ff feat/<slug> -m "Merge feat/<slug>: <description>"
|
||||
git push
|
||||
git branch -d feat/<slug>
|
||||
git push origin --delete feat/<slug>
|
||||
```
|
||||
|
||||
#### 6 — Tear down the feature stack and restart main dev stack
|
||||
```bash
|
||||
docker compose -f docker-compose.yml \
|
||||
-f docker-compose.dev.yml \
|
||||
-f docker-compose.feat-<slug>.yml \
|
||||
--project-name <slug> down --volumes --remove-orphans
|
||||
rm docker-compose.feat-<slug>.yml
|
||||
|
||||
# Restart the main dev stack on :5173
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Infrastructure change protocol
|
||||
|
||||
After **any** change to Dockerfiles, `docker-compose*.yml`, `nginx.conf`, or setup scripts:
|
||||
|
||||
1. **Update `README.md`** — containers table, ports, image names, Current State section.
|
||||
2. **Dev stack** — verify login and registration end-to-end:
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
```
|
||||
3. **Prod stack** — run the same checks:
|
||||
```bash
|
||||
docker compose up --build -d
|
||||
```
|
||||
4. Confirm non-root users:
|
||||
```bash
|
||||
docker inspect <container> --format '{{.Config.User}}'
|
||||
```
|
||||
5. **Tear down** after testing:
|
||||
```bash
|
||||
docker compose down --volumes --remove-orphans
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Security hook
|
||||
|
||||
`.githooks/pre-commit` (registered via `git config core.hooksPath .githooks`). Runs `scripts/security_check.py` in Docker. New clones must run:
|
||||
```bash
|
||||
git config core.hooksPath .githooks
|
||||
```
|
||||
|
||||
See **Security Standards → Pre-commit security hook** for the full list of checks.
|
||||
|
||||
**Never** bypass with `--no-verify`.
|
||||
|
||||
@@ -0,0 +1,153 @@
|
||||
# destroying_sap
|
||||
|
||||
A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
|
||||
## Stack
|
||||
|
||||
| Layer | Tech |
|
||||
|---|---|
|
||||
| Backend | FastAPI (async), SQLAlchemy 2, Alembic, PostgreSQL 16 |
|
||||
| Auth | JWT bearer tokens (RS256), bcrypt password hashing |
|
||||
| Frontend | React 18, TypeScript, Vite, React Router v6, TanStack Query |
|
||||
|
||||
## Current State
|
||||
|
||||
- User registration and login (JWT RS256 auth, 8-hour expiry)
|
||||
- Protected dashboard with nav bar
|
||||
- `/api/users/me` — authenticated user info
|
||||
- `/api/profile/me` — GET/PUT personal profile (position, phone, date of birth, address)
|
||||
- Admin-only user management at `/admin`: list, add, delete, toggle active
|
||||
- All input sanitized before reaching the DB (null-byte rejection, length caps, format validation)
|
||||
- **PDF Documents app** (`/apps/documents`): upload PDFs, async text extraction (pdfplumber), AI classification via ai-service, per-user categories, file download
|
||||
- **AI Service** (`ai-service:8010`): shared AI intermediary container; routes prompts to Anthropic / Ollama / LM Studio; stateless; all feature containers talk to it via `POST /chat`
|
||||
- **Storage Service** (`storage-service:8020`): unified file/blob storage with pluggable backends (local filesystem default; S3-compatible and WebDAV built in); backend switchable via admin UI with zero-data-loss migration
|
||||
- Admin settings: AI provider, doc upload limits, storage backend switching with live migration progress
|
||||
- Config stored in storage-service (`config` bucket); PDFs stored in storage-service (`documents` bucket) — no shared filesystem volumes
|
||||
- `/apps` launcher hub — one card per installed app with Open + Settings links
|
||||
- 6 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `storage-service`, `frontend`
|
||||
- All containers run as non-root users (UID 1001 for app containers, UID 70 for db)
|
||||
- Network-isolated: only the frontend exposes a host port (80/5173); all backend services are unreachable from outside Docker
|
||||
- Dev environment seeds a test user automatically on startup (`test@example.com` / `Test123!`)
|
||||
- Password policy: min 8 chars, upper + lowercase, digit, special character, no common words
|
||||
- Pre-commit security hook (`scripts/security_check.py`) runs inside Docker on every commit
|
||||
|
||||
## Containers
|
||||
|
||||
| Container | Image | Host port | Network | User (UID:GID) | Description |
|
||||
|---|---|---|---|---|---|
|
||||
| `db` | postgres:16-alpine | none | backend-net | 70:70 | PostgreSQL database |
|
||||
| `backend` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | FastAPI management API + proxy to doc-service |
|
||||
| `ai-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Shared AI intermediary (routes to LM Studio / Ollama / Anthropic) |
|
||||
| `doc-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | PDF extraction microservice (calls ai-service) |
|
||||
| `storage-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Unified file/blob storage (local / S3-compatible / WebDAV) |
|
||||
| `frontend` | custom (nginxinc/nginx-unprivileged:alpine) | 80 (prod) / 5173 (dev) | backend-net + frontend-net | 1001:1001 | React UI + nginx reverse proxy |
|
||||
|
||||
**Networks:**
|
||||
- `backend-net` — all backend services; no host ports bound; outbound internet access allowed (needed for cloud AI API calls)
|
||||
- `frontend-net` — frontend only; this is where the single host port (80/5173) is bound
|
||||
|
||||
**Volumes:**
|
||||
- `postgres_data` — PostgreSQL data files
|
||||
- `storage_data` — all file/blob storage: uploaded PDFs (`documents/` bucket) and service config JSON files (`config/` bucket); mounted into storage-service at `/data/storage`
|
||||
- `watch_data` — file watcher input directory; mounted into doc-service at `/data/watch`
|
||||
|
||||
The frontend nginx proxies `/api/*` to `backend:8000` via `backend-net`. The backend proxies `/api/documents/*` and `/api/documents/categories/*` to `doc-service:8001`. The backend test-connection endpoint proxies to `ai-service:8010`. No backend service or database port is ever exposed to the host.
|
||||
|
||||
## Installation
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Docker + Docker Compose
|
||||
|
||||
### Production
|
||||
|
||||
```bash
|
||||
git clone <repo>
|
||||
cd destroying_sap
|
||||
cp .env.example backend/.env
|
||||
python scripts/generate_jwt_keys.py # paste output into backend/.env
|
||||
docker compose up --build -d
|
||||
```
|
||||
|
||||
- Frontend: http://localhost
|
||||
- API docs: not directly accessible from host (backend port not exposed)
|
||||
|
||||
After first start, configure the AI provider at `/apps/documents/settings/admin` (admin login required).
|
||||
|
||||
### Development (hot reload)
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
```
|
||||
|
||||
- Frontend (Vite): http://localhost:5173
|
||||
- Backend: reachable by frontend via Docker network only (not exposed to host)
|
||||
|
||||
### Local (no Docker)
|
||||
|
||||
**1. Start PostgreSQL**
|
||||
|
||||
```bash
|
||||
docker compose up db -d
|
||||
```
|
||||
|
||||
**2. Backend**
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -e ".[dev]"
|
||||
cp ../.env.example .env
|
||||
alembic upgrade head
|
||||
uvicorn app.main:app --reload
|
||||
```
|
||||
|
||||
**3. doc-service**
|
||||
|
||||
```bash
|
||||
cd features/doc-service
|
||||
python -m venv .venv && source .venv/bin/activate
|
||||
pip install -e .
|
||||
alembic upgrade head
|
||||
uvicorn app.main:app --host 0.0.0.0 --port 8001 --reload
|
||||
```
|
||||
|
||||
**4. Frontend**
|
||||
|
||||
```bash
|
||||
cd frontend && npm install && npm run dev
|
||||
```
|
||||
|
||||
## Environment Variables
|
||||
|
||||
Copy `.env.example` to `backend/.env` and adjust:
|
||||
|
||||
| Variable | Default | Description |
|
||||
|---|---|---|
|
||||
| `DATABASE_URL` | `postgresql+asyncpg://postgres:password@localhost:5432/destroying_sap` | Async PostgreSQL URL |
|
||||
| `JWT_PRIVATE_KEY` | — | RS256 private key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `JWT_PUBLIC_KEY` | — | RS256 public key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `CORS_ORIGINS` | `["http://localhost:5173"]` | Allowed frontend origins |
|
||||
| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of doc-service (set by docker-compose) |
|
||||
| `STORAGE_SERVICE_URL` | `http://storage-service:8020` | Internal URL of storage-service (set by docker-compose) |
|
||||
|
||||
## Development
|
||||
|
||||
```bash
|
||||
# Backend lint + format
|
||||
cd backend && ruff check . && ruff format .
|
||||
|
||||
# Backend tests
|
||||
cd backend && pytest
|
||||
|
||||
# Frontend type check + lint
|
||||
cd frontend && npm run typecheck && npm run lint
|
||||
|
||||
# New DB migration — main backend
|
||||
cd backend && alembic revision --autogenerate -m "describe change"
|
||||
cd backend && alembic upgrade head
|
||||
|
||||
# New DB migration — doc-service
|
||||
cd features/doc-service && alembic revision --autogenerate -m "describe change"
|
||||
cd features/doc-service && alembic upgrade head
|
||||
```
|
||||
@@ -0,0 +1,58 @@
|
||||
# TODO
|
||||
|
||||
## UX/UI — Figma setup
|
||||
|
||||
- [x] **Create a Figma account** — signed up at https://figma.com
|
||||
- [x] **Create Figma project and file** — file key `kcmvLytS31lSjP44YpBUSn` confirmed active
|
||||
- [x] **Generate Figma personal access token** — generated and verified (HTTP 200); provide a fresh token at each session start
|
||||
- [x] **Connect ux-designer agent** — agent updated to use `Bash`/`curl` with `X-Figma-Token` header; connection confirmed working
|
||||
- [ ] **Decide on UI component library** — shadcn/ui (recommended: Tailwind-based, unstyled accessible primitives, white-label friendly) vs MUI vs other; decision affects both Figma design system and frontend implementation
|
||||
|
||||
## Auth / session security
|
||||
|
||||
- [x] **8-hour JWT expiry** — `ACCESS_TOKEN_EXPIRE_MINUTES = 60 * 8`; no permanent login
|
||||
- [x] **RS256 JWT signing** — 4096-bit RSA asymmetric keys; `iat` claim included; generate keys with `scripts/generate_jwt_keys.py`
|
||||
- [ ] **No refresh tokens** — refresh token flow not implemented; if added later, must use `httpOnly` cookies and rotation
|
||||
- [ ] **`httpOnly` cookie migration** — currently storing JWT in `localStorage` (XSS-exposed); migrate to `httpOnly` cookie when hardening for production
|
||||
|
||||
## App permissions
|
||||
|
||||
- [ ] **Permissions registry** — admin-managed table that controls which apps each user can access. Schema: `user_app_permissions (user_id FK, app_key)`. Admin UI lets the admin grant/revoke per-app access per user. The Apps page only shows apps the current user has been granted access to.
|
||||
|
||||
## PDF Documents app (`features/doc-service`)
|
||||
|
||||
- [x] **doc-service container** — FastAPI microservice on `backend-net`; never exposed to host or frontend directly
|
||||
- [x] **PDF upload + async extraction** — background task with pdfplumber + pluggable AI (Anthropic / Ollama / LM Studio)
|
||||
- [x] **Per-app settings page** — `/apps/documents/settings/admin`; AI provider config, max file size; admin only
|
||||
- [x] **Per-user categories** — create/rename/delete categories; assign multiple categories per document
|
||||
- [x] **Alembic isolation** — `alembic_version_doc_service` version table; no collision with main backend migrations
|
||||
- [x] **Runtime config file** — `/config/doc_service_config.json` on shared Docker volume; editable from frontend; 30s TTL cache in doc-service
|
||||
- [ ] **Re-process document** — UI button to re-trigger AI extraction on an existing document (after changing AI provider/model)
|
||||
- [ ] **Bulk category operations** — assign/remove a category from multiple documents at once
|
||||
- [ ] **Search / filter documents** — filter by status, document type, category, date range
|
||||
|
||||
## Frontend features
|
||||
|
||||
- [x] **Logout button** — visible when logged in, clears token and redirects to `/login`
|
||||
- [x] **Profile page** (`/profile`) — shows personal information for the logged-in user
|
||||
- [x] **Edit & save profile** — form to update personal details, stored in a dedicated `profiles` table (separate from `users`, same PostgreSQL container)
|
||||
|
||||
## App container architecture (future)
|
||||
|
||||
Design decision: each installable app (billing, PDF, email, etc.) runs in its own isolated Docker/Podman container, spawned and managed by the backend via the Docker API. Key rules to implement:
|
||||
|
||||
- [ ] **Docker socket proxy** — backend must never mount `/var/run/docker.sock` directly; use `tecnativa/docker-socket-proxy` on an internal-only network, with only the required API endpoints whitelisted (CONTAINERS, IMAGES, NETWORKS, POST). Raw socket access = root on the host.
|
||||
- [ ] **Network isolation per app** — each spawned app container gets its own Docker bridge network; app containers never talk to each other directly; only the backend can reach them
|
||||
- [ ] **No privileged app containers** — all spawned containers run without `--privileged`, without extra capabilities, with resource limits (CPU, memory)
|
||||
- [ ] **Image allowlist** — backend may only spawn containers from a pre-approved image list; never pull or build arbitrary images at runtime
|
||||
- [ ] **Consider Podman** — evaluate rootless Podman as replacement for Docker daemon; daemonless model eliminates the socket entirely; Docker SDK compatible
|
||||
|
||||
## Infrastructure
|
||||
|
||||
- [x] **Docker port hardening** — only port 80 (prod) / 5173 (dev) exposed on the host via `frontend-net`; backend and db have no host port bindings and sit on `internal: true` `backend-net`
|
||||
|
||||
## Infrastructure (existing)
|
||||
|
||||
- [x] **Rootless containers** — run backend and frontend containers as non-root users (add `USER` directive to Dockerfiles, map UID/GID appropriately)
|
||||
- [ ] **Persistent storage** — ensure database data, config files, and any uploaded assets survive container restarts and rebuilds (named volumes, bind mounts for config)
|
||||
- [ ] **Docker development workflow** — document and streamline the full dev loop: hot reload, one-command startup, migration handling, seed data, and how to attach a debugger
|
||||
@@ -0,0 +1,365 @@
|
||||
# backend — Claude context
|
||||
|
||||
FastAPI async gateway, port 8000 (internal). Handles auth, user/group management, settings, and proxies document/category requests to `doc-service:8001`. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## Commands
|
||||
|
||||
All commands run inside Docker — never on the host.
|
||||
|
||||
### Migrations
|
||||
|
||||
```bash
|
||||
docker compose exec backend alembic revision --autogenerate -m "describe change"
|
||||
docker compose exec backend alembic upgrade head
|
||||
docker compose exec backend alembic downgrade -1
|
||||
```
|
||||
|
||||
### Lint
|
||||
|
||||
```bash
|
||||
docker compose exec backend ruff check . && ruff format .
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
backend/
|
||||
├── app/
|
||||
│ ├── main.py ← App factory, router registration, lifespan (health loop)
|
||||
│ ├── database.py ← AsyncEngine, AsyncSessionLocal, Base
|
||||
│ ├── deps.py ← get_current_user, get_current_admin, get_service_admin(id), check_plugin_access (also get_user_groups in doc-service)
|
||||
│ ├── core/
|
||||
│ │ ├── config.py ← All settings via pydantic-settings (reads .env)
|
||||
│ │ ├── security.py ← JWT sign/verify (RS256), bcrypt hash/verify
|
||||
│ │ ├── sanitize.py ← Input sanitization helpers (see Security Standards)
|
||||
│ │ ├── app_config.py ← Per-service config load/save via storage-service; theme files in config/themes/{id}.json
|
||||
│ │ └── config_storage.py ← Thin async HTTP helpers: read_json/write_json/delete_key/list_keys → storage-service config bucket
|
||||
│ ├── models/
|
||||
│ │ ├── __init__.py ← Imports all models (required for Alembic autogenerate)
|
||||
│ │ ├── user.py ← User model
|
||||
│ │ ├── profile.py ← Profile model
|
||||
│ │ └── group.py ← Group, GroupMembership models
|
||||
│ ├── schemas/
|
||||
│ │ ├── user.py ← UserCreate/Out, Token, DashboardPrefsOut/Update
|
||||
│ │ ├── profile.py ← ProfileRead, ProfileUpdate
|
||||
│ │ └── group.py ← GroupCreate/Update/Out/DetailOut, GroupMemberOut
|
||||
│ ├── routers/
|
||||
│ │ ├── auth.py ← POST /register, POST /login
|
||||
│ │ ├── users.py ← GET /me, GET+PATCH /me/preferences, PATCH /me/color-mode, GET /me/groups
|
||||
│ │ ├── profile.py ← GET+PUT /me (profile)
|
||||
│ │ ├── admin.py ← User admin CRUD (admin-only)
|
||||
│ │ ├── groups.py ← Group CRUD + member management (admin-only)
|
||||
│ │ ├── settings.py ← AI, doc limits, system prompts, appearance, themes (admin-only)
|
||||
│ │ ├── services.py ← GET /services (health status)
|
||||
│ │ ├── plugins.py ← Generic plugin proxy (GET/PATCH /api/plugins/*)
|
||||
│ │ ├── categories_proxy.py ← Transparent proxy → doc-service /categories/*
|
||||
│ │ ├── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ └── storage_config.py ← Admin proxy → storage-service config + migration endpoints
|
||||
│ └── services/
|
||||
│ ├── service_health.py ← Background 30s health-check loop; caches /plugin/manifest per service
|
||||
│ └── group_bootstrap.py ← Ensures {service-id}-admin group exists for every registered service at startup
|
||||
├── alembic/
|
||||
│ ├── env.py ← Async migration runner
|
||||
│ └── versions/ ← Migration chain (see Database Models)
|
||||
├── scripts/seed.py ← Seed test user
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Database Models
|
||||
|
||||
### `users`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `email` | String | UNIQUE, indexed, NOT NULL | lowercased before storing |
|
||||
| `hashed_password` | String | NOT NULL | bcrypt 13 rounds |
|
||||
| `full_name` | String | nullable | sanitized max 128 chars |
|
||||
| `is_active` | Boolean | default=True | soft-delete flag |
|
||||
| `is_superuser` | Boolean | default=False | admin role; never exposed as-is (serialised as `is_admin`) |
|
||||
| `dashboard_app_ids` | JSON | NOT NULL, default=[] | list of pinned service IDs |
|
||||
| `color_mode` | String | nullable, default=NULL | user's preferred mode: "light" / "dark" / "system" / NULL (use admin default) |
|
||||
|
||||
Relationship: `profile` (one-to-one, cascade all+delete-orphan)
|
||||
|
||||
### `profiles`
|
||||
|
||||
| Column | Type | Constraints | Notes |
|
||||
|--------|------|-------------|-------|
|
||||
| `id` | String | PK, UUID | auto-generated |
|
||||
| `user_id` | String | FK→users.id UNIQUE, cascade delete | one-to-one |
|
||||
| `phone` | String(20) | nullable | validated format |
|
||||
| `date_of_birth` | Date | nullable | 1900+ and not future |
|
||||
| `position` | String(128) | nullable | job title |
|
||||
| `address` | String(255) | nullable | |
|
||||
| `updated_at` | DateTime(tz) | server_default=now(), onupdate=now() | |
|
||||
|
||||
### `groups`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `name` | String(128) | UNIQUE indexed, NOT NULL |
|
||||
| `description` | String(512) | nullable |
|
||||
| `created_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
### `group_memberships`
|
||||
|
||||
| Column | Type | Constraints |
|
||||
|--------|------|-------------|
|
||||
| `id` | String | PK, UUID |
|
||||
| `group_id` | String | FK→groups.id, indexed, CASCADE |
|
||||
| `user_id` | String | FK→users.id, indexed, CASCADE |
|
||||
| `is_group_admin` | Boolean | NOT NULL, default=false | grants group-admin rights (manage group categories, delete shared docs) |
|
||||
| `joined_at` | DateTime(tz) | server_default=now() |
|
||||
|
||||
Unique constraint: `(group_id, user_id)`
|
||||
|
||||
### Migration chain (must be applied in order)
|
||||
|
||||
| Rev ID | Slug |
|
||||
|--------|------|
|
||||
| `38efeff7c45a` | `create_users_table` |
|
||||
| `676084df61d1` | `add_profiles_table` |
|
||||
| `a3f9c2d14e87` | `add_groups_and_group_memberships` |
|
||||
| `c7e8f9a0b1d2` | `add_dashboard_app_ids_to_users` |
|
||||
| `dd6ad2f2c211` | `add_color_mode_to_users` |
|
||||
| `e1f2a3b4c5d6` | `add_group_member_is_admin` |
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Auth (`/api/auth`) — public
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| POST | `/api/auth/register` | — | Create account; returns `UserOut`; enforces password policy |
|
||||
| POST | `/api/auth/login` | — | OAuth2 password flow; returns `{access_token, token_type}` |
|
||||
|
||||
### Users (`/api/users`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/users/me` | user | Current user info → `UserOut` |
|
||||
| GET | `/api/users/me/preferences` | user | Dashboard pinned app IDs → `{app_ids}` |
|
||||
| PATCH | `/api/users/me/preferences` | user | Save pinned app IDs (max 50, slug-safe) |
|
||||
| PATCH | `/api/users/me/color-mode` | user | Save colour mode preference ("light"/"dark"/"system") |
|
||||
| GET | `/api/users/me/groups` | user | Groups current user belongs to → `list[UserGroupOut]` |
|
||||
|
||||
### Profile (`/api/profile`) — authenticated
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| GET | `/api/profile/me` | user | Fetch profile; auto-creates if missing |
|
||||
| PUT | `/api/profile/me` | user | Update profile fields |
|
||||
|
||||
### Admin — Users (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/users` | List all users → `list[UserAdminOut]` |
|
||||
| POST | `/api/admin/users` | Create user (with optional is_admin) |
|
||||
| DELETE | `/api/admin/users/{user_id}` | Delete user (204) |
|
||||
| PATCH | `/api/admin/users/{user_id}/active` | Toggle active status |
|
||||
|
||||
### Admin — Groups (`/api/admin/groups`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/groups` | List groups with member count |
|
||||
| POST | `/api/admin/groups` | Create group |
|
||||
| GET | `/api/admin/groups/{id}` | Group detail + members |
|
||||
| PATCH | `/api/admin/groups/{id}` | Update name / description |
|
||||
| DELETE | `/api/admin/groups/{id}` | Delete (cascades memberships) |
|
||||
| POST | `/api/admin/groups/{id}/members/{user_id}` | Add member |
|
||||
| DELETE | `/api/admin/groups/{id}/members/{user_id}` | Remove member |
|
||||
| PATCH | `/api/admin/groups/{id}/members/{user_id}/admin` | Set/unset group admin role (body: `{ is_group_admin: bool }`) |
|
||||
|
||||
### Settings (`/api/settings`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/settings/ai` | AI config (keys masked) |
|
||||
| PATCH | `/api/settings/ai` | Update AI provider / credentials |
|
||||
| POST | `/api/settings/ai/test` | Test AI connection |
|
||||
| GET | `/api/settings/documents/limits` | PDF upload limits |
|
||||
| PATCH | `/api/settings/documents/limits` | Update max PDF size |
|
||||
| GET | `/api/settings/system-prompts` | All editable system prompts |
|
||||
| PATCH | `/api/settings/system-prompts/{service_id}` | Update system prompt |
|
||||
| GET | `/api/settings/appearance` | Active theme + default mode (auth) |
|
||||
| PATCH | `/api/settings/appearance` | Update active theme + default mode (admin) |
|
||||
| GET | `/api/settings/themes` | List all themes — built-in + custom (auth) |
|
||||
| POST | `/api/settings/themes` | Create custom theme (admin) |
|
||||
| PATCH | `/api/settings/themes/{id}` | Update custom theme label/colours (admin) |
|
||||
| DELETE | `/api/settings/themes/{id}` | Delete custom theme (admin, 204) |
|
||||
|
||||
### Services (`/api/services`) — authenticated
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/services` | Health status of all registered services → `list[ServiceStatus]` |
|
||||
|
||||
### Plugins (`/api/plugins`) — authenticated, auth-per-plugin
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/plugins` | List plugins accessible to current user |
|
||||
| GET | `/api/plugins/{id}/manifest` | Plugin manifest with settings JSON Schema (auth-gated) |
|
||||
| GET | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
|
||||
| PATCH | `/api/plugins/{id}/settings` | Proxy to feature `/plugin/settings` (auth-gated) |
|
||||
|
||||
Auth: is_superuser OR member of group listed in manifest `required_groups`. Returns 404 (not 403) to hide existence.
|
||||
|
||||
### Admin — Storage (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/storage-config` | Current backend driver + health → proxied from storage-service `/health` |
|
||||
| PATCH | `/api/admin/storage-config` | Reconfigure backend without data migration (same-backend credential update) |
|
||||
| POST | `/api/admin/storage-config/migrate` | Start async migration to a new backend (copy → verify → switch → cleanup) |
|
||||
| GET | `/api/admin/storage-config/migrate/status` | Poll migration progress: `{state, total, done, failed, errors[]}` |
|
||||
| DELETE | `/api/admin/storage-config/migrate` | Cancel a running migration; old backend remains active |
|
||||
|
||||
### Documents and Categories — proxied
|
||||
|
||||
`/api/documents/*` and `/api/documents/categories/*` are transparently proxied to `doc-service:8001`. The backend injects `x-user-id`, `x-user-groups`, and `x-user-is-admin` headers. See `features/doc-service/CLAUDE.md` for the internal endpoint list.
|
||||
|
||||
---
|
||||
|
||||
## Security Standards
|
||||
|
||||
These standards are **non-negotiable**. Every change must comply.
|
||||
|
||||
### JWT
|
||||
|
||||
- **Algorithm**: RS256 (4096-bit RSA key pair, generated by `scripts/generate_jwt_keys.py`)
|
||||
- **Keys**: PEM-encoded in `backend/.env` as `JWT_PRIVATE_KEY` / `JWT_PUBLIC_KEY` (gitignored)
|
||||
- **Expiry**: 8 hours (`EXPIRE_MINUTES=480`) — never set longer; no refresh tokens
|
||||
- **Claims**: `{sub: user_id, exp, iat}` — user_id is a UUID string
|
||||
- **Validation**: `decode_access_token()` in `core/security.py`; called by `get_current_user`
|
||||
- **Never**: set algorithm to `"none"`, disable `verify_exp`, or hardcode secrets in code
|
||||
|
||||
### Password hashing
|
||||
|
||||
- **Algorithm**: bcrypt, **13 rounds** (`bcrypt.gensalt(rounds=13)`)
|
||||
- **Timing**: ~300 ms per hash (intentional brute-force resistance)
|
||||
- **Never** use MD5, SHA1, or plain SHA256 for password storage
|
||||
|
||||
### Password policy (enforced in `UserCreate` schema)
|
||||
|
||||
All of the following must pass:
|
||||
- ≥ 8 characters
|
||||
- ≥ 1 uppercase (A–Z)
|
||||
- ≥ 1 lowercase (a–z)
|
||||
- ≥ 1 digit (0–9)
|
||||
- ≥ 1 special character: `!@#$%^&*()\-_=+[]{}|;:'"<>?/\`~`
|
||||
- No common words (password, secret, login, admin, test, qwerty, welcome, …)
|
||||
|
||||
### Input sanitization
|
||||
|
||||
Every user-supplied string stored in the database **must** pass through `core/sanitize.py`:
|
||||
|
||||
```python
|
||||
sanitize_str(value, max_len=255)
|
||||
# → strips whitespace; rejects null bytes (\x00); rejects control chars
|
||||
# (0x01–0x1F, 0x7F except \t \n \r); enforces max_len; returns None for ""
|
||||
|
||||
normalize_email(value) # lowercase + strip
|
||||
validate_phone(value) # sanitize_str(max=20) + regex ^\+?[\d\s\-()\[\]]{7,20}$
|
||||
validate_date_of_birth(v) # must be ≥ 1900, not future
|
||||
```
|
||||
|
||||
Apply via Pydantic `@field_validator` on all request schemas.
|
||||
|
||||
### SQL injection prevention
|
||||
|
||||
- Use SQLAlchemy ORM (bound parameters) — **never** raw SQL strings.
|
||||
- If `text()` is needed, use `bindparam()` for all user-supplied values.
|
||||
- **Never** use f-strings, `.format()`, or `%`-formatting for SQL.
|
||||
|
||||
### Admin route security
|
||||
|
||||
- Use `get_current_admin` dependency (checks `is_superuser`).
|
||||
- Return **404** (not 403) for unauthorized access — hides both endpoint existence and permission model.
|
||||
|
||||
---
|
||||
|
||||
## Naming & Code Conventions
|
||||
|
||||
### Database
|
||||
|
||||
- **Tables**: lowercase, plural, snake_case (`users`, `group_memberships`, `document_category_assignments`)
|
||||
- **Columns**: lowercase, snake_case
|
||||
- **ORM models**: PascalCase, singular (`User`, `Group`, `GroupMembership`, `Document`)
|
||||
- Primary keys: `id` (String UUID, auto-generated)
|
||||
- Timestamps: `created_at` / `updated_at` / `joined_at` / `processed_at` — always timezone-aware
|
||||
|
||||
### Pydantic schemas
|
||||
|
||||
| Suffix | Purpose |
|
||||
|--------|---------|
|
||||
| `Create` | POST request body (user-supplied input) |
|
||||
| `Update` | PATCH request body (partial update) |
|
||||
| `Out` | API response (safe subset of model) |
|
||||
| `AdminOut` | Extended response for admin endpoints |
|
||||
| `Read` | GET response (same as `Out`, used for profiles) |
|
||||
|
||||
Always set `model_config = {"from_attributes": True}` on response schemas.
|
||||
Use `validation_alias` when the ORM field name differs from the JSON key (e.g., `is_superuser` → `is_admin`).
|
||||
|
||||
### HTTP status codes
|
||||
|
||||
| Code | Use |
|
||||
|------|-----|
|
||||
| 200 | Successful GET / PATCH / PUT |
|
||||
| 201 | Successful POST that creates a resource |
|
||||
| 202 | Accepted (async processing started, e.g., document upload) |
|
||||
| 204 | Successful DELETE or action with no response body |
|
||||
| 400 | Bad request (duplicates, invalid data beyond Pydantic) |
|
||||
| 401 | Missing / invalid JWT |
|
||||
| 404 | Not found **and** admin routes when not admin |
|
||||
| 413 | Payload too large (file exceeds limit) |
|
||||
| 415 | Unsupported media type (not a PDF) |
|
||||
| 422 | Pydantic validation failure (FastAPI default) |
|
||||
| 502 | Downstream service unreachable |
|
||||
| 503 | Service unavailable (queue stopped, AI error) |
|
||||
| 504 | Gateway timeout |
|
||||
|
||||
### Backend code style
|
||||
|
||||
- Async/await for **all** I/O (DB, HTTP, file).
|
||||
- `raise HTTPException(status_code=..., detail="...")` for all errors.
|
||||
- Response models always declared in route decorator: `@router.get("/path", response_model=XOut)`.
|
||||
- Background tasks via `BackgroundTasks` param; tasks open their own `AsyncSessionLocal` session.
|
||||
- Commit + refresh pattern after mutations:
|
||||
```python
|
||||
await db.commit()
|
||||
await db.refresh(obj)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Default Values & Limits
|
||||
|
||||
| Parameter | Value | Location |
|
||||
|-----------|-------|----------|
|
||||
| JWT expiry | 480 min (8 h) | `core/security.py` |
|
||||
| Bcrypt rounds | 13 | `core/security.py` |
|
||||
| User `color_mode` default | NULL (falls back to admin default_mode, then system) | `models/user.py` |
|
||||
| Max dashboard pinned apps | 50 | `schemas/user.py` |
|
||||
| App ID max length | 64 chars | `schemas/user.py` |
|
||||
| App ID allowed chars | `[a-zA-Z0-9_\-]` | `schemas/user.py` |
|
||||
| full_name max length | 128 chars | `schemas/user.py` |
|
||||
| Group name max length | 128 chars | `schemas/group.py` |
|
||||
| Group description max | 512 chars | `schemas/group.py` |
|
||||
| Phone max length | 20 chars | `sanitize.py` |
|
||||
| Position max length | 128 chars | `schemas/profile.py` |
|
||||
| Address max length | 255 chars | `schemas/profile.py` |
|
||||
@@ -0,0 +1,34 @@
|
||||
# ── Stage 1: dependency installation ─────────────────────────────────────────
|
||||
FROM python:3.12-slim AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install --upgrade pip
|
||||
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --prefix=/install .
|
||||
|
||||
# ── Stage 2: runtime ──────────────────────────────────────────────────────────
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Create non-root user (UID/GID 1001)
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Copy installed packages from builder
|
||||
COPY --from=builder /install /usr/local
|
||||
|
||||
# Copy application source with correct ownership
|
||||
COPY --chown=appuser:appuser app ./app
|
||||
COPY --chown=appuser:appuser alembic ./alembic
|
||||
COPY --chown=appuser:appuser alembic.ini .
|
||||
COPY --chown=appuser:appuser scripts ./scripts
|
||||
RUN chmod +x scripts/start.sh
|
||||
|
||||
USER appuser
|
||||
|
||||
EXPOSE 8000
|
||||
|
||||
CMD ["sh", "scripts/start.sh"]
|
||||
@@ -0,0 +1,183 @@
|
||||
# Backend — Status
|
||||
|
||||
## What it is
|
||||
|
||||
Central FastAPI gateway. Handles authentication, user management, admin settings, and proxies feature-service traffic. It is the only container that has host-level port exposure (`8000`, internal) — all browser traffic arrives via the Vite/nginx frontend proxy.
|
||||
|
||||
Port: `8000` (on `backend-net`, no direct host binding in prod).
|
||||
Database: PostgreSQL 16 (`postgres_data` named volume).
|
||||
|
||||
---
|
||||
|
||||
## Current functionality
|
||||
|
||||
### Auth (`/api/auth`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/api/auth/register` | Create account; password policy enforced (uppercase, special char, no "test") |
|
||||
| `POST` | `/api/auth/login` | OAuth2 password flow; returns RS256 JWT (8-hour expiry) |
|
||||
|
||||
JWT signing uses a 4096-bit RSA key pair (`RS256`). Keys are generated by `scripts/generate_jwt_keys.py` and stored in `backend/.env` (gitignored). Token stored in `localStorage` on the client.
|
||||
|
||||
### Users (`/api/users`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/users/me` | Current user info |
|
||||
| `GET` | `/api/users/me/preferences` | User's dashboard preferences (`app_ids` list) |
|
||||
| `PATCH` | `/api/users/me/preferences` | Update pinned app IDs (max 50; validated as safe slugs) |
|
||||
| `GET` | `/api/users/me/groups` | List groups the current user belongs to (for share picker) |
|
||||
|
||||
### Profile (`/api/profile`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/profile` | Fetch profile (separate `profiles` table) |
|
||||
| `PUT` | `/api/profile` | Update profile fields |
|
||||
|
||||
### Admin (`/api/admin`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/users` | List all users (admin only) |
|
||||
| `PATCH` | `/api/admin/users/{id}` | Update user (role, active flag) |
|
||||
|
||||
### Groups (`/api/admin/groups`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/groups` | List all groups with member count |
|
||||
| `POST` | `/api/admin/groups` | Create a new group |
|
||||
| `GET` | `/api/admin/groups/{id}` | Get group detail with member list |
|
||||
| `PATCH` | `/api/admin/groups/{id}` | Update group name / description |
|
||||
| `DELETE` | `/api/admin/groups/{id}` | Delete group (cascades memberships) |
|
||||
| `POST` | `/api/admin/groups/{id}/members/{user_id}` | Add user to group |
|
||||
| `DELETE` | `/api/admin/groups/{id}/members/{user_id}` | Remove user from group |
|
||||
|
||||
### Services (`/api/services`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/services` | Returns health status of all registered feature services |
|
||||
|
||||
A background task (`service_health.py`) polls each service's `/health` endpoint every 30 s and stores the result in memory. The first check runs immediately on startup. Any authenticated user may call `GET /api/services`; the frontend uses it to drive app card visibility.
|
||||
|
||||
### Settings (`/api/settings`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/settings/ai` | AI service config (masked) — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/ai` | Update AI provider / credentials — same access |
|
||||
| `POST` | `/api/settings/ai/test` | Test AI connection — same access |
|
||||
| `GET` | `/api/settings/documents/limits` | Doc service upload limits — superuser OR `doc-service-admin` member |
|
||||
| `PATCH` | `/api/settings/documents/limits` | Update max PDF size — same access |
|
||||
| `GET` | `/api/settings/system-prompts` | All editable system prompts — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/system-prompts/{id}` | Update system prompt — same access |
|
||||
|
||||
Settings are persisted to the `config` bucket of `storage-service:8020` via `core/config_storage.py`. All config I/O is async HTTP; no filesystem volumes are used.
|
||||
|
||||
Access to service-specific settings endpoints is enforced by `get_service_admin(service_id)` in `deps.py` — grants access to superusers OR members of the `{service_id}-admin` group.
|
||||
|
||||
### Storage config (`/api/admin`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/storage-config` | Current backend driver + health (proxied from storage-service) |
|
||||
| `PATCH` | `/api/admin/storage-config` | Reconfigure backend without migration |
|
||||
| `POST` | `/api/admin/storage-config/migrate` | Start async migration to a new backend |
|
||||
| `GET` | `/api/admin/storage-config/migrate/status` | Poll migration progress |
|
||||
| `DELETE` | `/api/admin/storage-config/migrate` | Cancel running migration |
|
||||
|
||||
### Feature proxies
|
||||
|
||||
All `/api/documents/*` and `/api/documents/categories/*` requests are transparently proxied to `doc-service:8001` via `httpx.AsyncClient`. The proxy:
|
||||
- Validates the JWT (`get_current_user`)
|
||||
- Injects `x-user-id` header (UUID from `users.id`)
|
||||
- Strips hop-by-hop headers + `content-length`, `accept-encoding`, `content-type`
|
||||
- Returns `Response` (not `StreamingResponse`) to avoid content-length/chunked conflicts
|
||||
|
||||
### Plugin system (`/api/plugins`)
|
||||
|
||||
Generic extension/plugin infrastructure — **zero feature-specific code in backend**. Feature containers self-describe via `GET /plugin/manifest`.
|
||||
|
||||
| Method | Path | Auth | Description |
|
||||
|--------|------|------|-------------|
|
||||
| `GET` | `/api/plugins` | user | List plugins accessible to current user |
|
||||
| `GET` | `/api/plugins/{id}/manifest` | user | Cached manifest for a plugin (404 if not accessible) |
|
||||
| `GET` | `/api/plugins/{id}/settings` | user | Proxy to feature `GET /plugin/settings` |
|
||||
| `PATCH` | `/api/plugins/{id}/settings` | user | Proxy to feature `PATCH /plugin/settings` |
|
||||
|
||||
Access is controlled by the manifest: `allow_superuser` for admins; `required_groups` for group members. `check_plugin_access(plugin_id, user, db)` in `deps.py` enforces this.
|
||||
|
||||
During each health poll, `service_health.py` also fetches `GET /plugin/manifest` from healthy services and caches it. New feature containers that expose `/plugin/manifest` automatically appear in the plugin list — no backend code changes required.
|
||||
|
||||
**Service admin group bootstrap:** On every startup, `group_bootstrap.py` creates a `{service-id}-admin` group for every registered service (idempotent). Admins add users to these groups via the Admin → Groups UI to delegate service-level administration.
|
||||
|
||||
### Database models
|
||||
|
||||
| Model | Table | Notes |
|
||||
|-------|-------|-------|
|
||||
| `User` | `users` | email, hashed_password, role (`user`\|`admin`), is_active, dashboard_app_ids (JSON) |
|
||||
| `Profile` | `profiles` | one-to-one with User; full_name, phone, etc. |
|
||||
| `Group` | `groups` | name (unique), description, created_at |
|
||||
| `GroupMembership` | `group_memberships` | group_id + user_id (unique pair); joined_at |
|
||||
|
||||
Alembic migrations in `backend/alembic/versions/` — version table: `alembic_version`.
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Browser (port 5173 dev / 80 prod)
|
||||
│
|
||||
└── Vite dev proxy / nginx
|
||||
│
|
||||
└── /api/* → backend:8000 (FastAPI)
|
||||
│
|
||||
┌───────────┼────────────┬──────────────┐
|
||||
/auth /settings /documents/* /services
|
||||
/users (JSON │ │
|
||||
/admin /storage- └── proxy → health-check loop
|
||||
/profile config doc-service:8001 (30s poll)
|
||||
(proxy)
|
||||
│
|
||||
storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Security notes
|
||||
|
||||
- JWT stored in `localStorage` — XSS risk. Migration to `httpOnly` cookie planned.
|
||||
- No refresh token — after 8h the user must log in again.
|
||||
- Admin routes use `get_current_admin` dependency (checks `role == "admin"`).
|
||||
- All backend routes require authentication except `/api/auth/*`.
|
||||
- `backend-net` is marked `internal: true` — containers on it cannot reach the internet directly.
|
||||
|
||||
---
|
||||
|
||||
## Known limitations / not implemented
|
||||
|
||||
- **No refresh tokens** — 8h hard expiry; adding refresh requires `httpOnly` cookie + rotation
|
||||
- **No `httpOnly` cookie** — JWT in `localStorage` is XSS-exposed
|
||||
- **App permissions** — no per-user, per-app access control. Currently all authenticated users can use all apps. Planned: `user_app_permissions` table, admin UI to grant/revoke
|
||||
- **Groups / sharing** — groups + memberships exist; app permission hooks not yet wired up
|
||||
- **Email verification** — accounts are active immediately after registration
|
||||
- **Password reset** — no flow implemented
|
||||
|
||||
---
|
||||
|
||||
## Future work
|
||||
|
||||
- [x] Groups system: `groups`, `group_memberships` tables; admin CRUD; add/remove members
|
||||
- [x] Generic plugin infrastructure: manifest contract, `/api/plugins` proxy router, `check_plugin_access`
|
||||
- [ ] App permissions registry: `group_app_permissions` table; AppsPage filtered by group grants
|
||||
- [ ] Doc sharing via group membership
|
||||
- [ ] App permissions registry: `user_app_permissions (user_id, app_key)`; AppsPage filtered by grants
|
||||
- [ ] `httpOnly` cookie migration for JWT
|
||||
- [ ] Refresh token flow (paired with cookie migration)
|
||||
- [ ] Email verification on registration
|
||||
- [ ] Password reset flow
|
||||
- [ ] Rate limiting on auth endpoints
|
||||
@@ -0,0 +1,39 @@
|
||||
"""create users table
|
||||
|
||||
Revision ID: 38efeff7c45a
|
||||
Revises:
|
||||
Create Date: 2026-04-12 14:00:30.503479
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = '38efeff7c45a'
|
||||
down_revision: Union[str, None] = None
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.create_table('users',
|
||||
sa.Column('id', sa.String(), nullable=False),
|
||||
sa.Column('email', sa.String(), nullable=False),
|
||||
sa.Column('hashed_password', sa.String(), nullable=False),
|
||||
sa.Column('full_name', sa.String(), nullable=True),
|
||||
sa.Column('is_active', sa.Boolean(), nullable=False),
|
||||
sa.Column('is_superuser', sa.Boolean(), nullable=False),
|
||||
sa.PrimaryKeyConstraint('id')
|
||||
)
|
||||
op.create_index(op.f('ix_users_email'), 'users', ['email'], unique=True)
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_index(op.f('ix_users_email'), table_name='users')
|
||||
op.drop_table('users')
|
||||
# ### end Alembic commands ###
|
||||
@@ -0,0 +1,40 @@
|
||||
"""add profiles table
|
||||
|
||||
Revision ID: 676084df61d1
|
||||
Revises: 38efeff7c45a
|
||||
Create Date: 2026-04-13 16:11:46.705481
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = '676084df61d1'
|
||||
down_revision: Union[str, None] = '38efeff7c45a'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.create_table('profiles',
|
||||
sa.Column('id', sa.String(), nullable=False),
|
||||
sa.Column('user_id', sa.String(), nullable=False),
|
||||
sa.Column('phone', sa.String(length=20), nullable=True),
|
||||
sa.Column('date_of_birth', sa.Date(), nullable=True),
|
||||
sa.Column('position', sa.String(length=128), nullable=True),
|
||||
sa.Column('address', sa.String(length=255), nullable=True),
|
||||
sa.Column('updated_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
|
||||
sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='CASCADE'),
|
||||
sa.PrimaryKeyConstraint('id'),
|
||||
sa.UniqueConstraint('user_id')
|
||||
)
|
||||
# ### end Alembic commands ###
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
# ### commands auto generated by Alembic - please adjust! ###
|
||||
op.drop_table('profiles')
|
||||
# ### end Alembic commands ###
|
||||
@@ -0,0 +1,51 @@
|
||||
"""add groups and group_memberships tables
|
||||
|
||||
Revision ID: a3f9c2d14e87
|
||||
Revises: 676084df61d1
|
||||
Create Date: 2026-04-17 12:00:00.000000
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = 'a3f9c2d14e87'
|
||||
down_revision: Union[str, None] = '676084df61d1'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.create_table(
|
||||
'groups',
|
||||
sa.Column('id', sa.String(), nullable=False),
|
||||
sa.Column('name', sa.String(length=128), nullable=False),
|
||||
sa.Column('description', sa.String(length=512), nullable=True),
|
||||
sa.Column('created_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
|
||||
sa.PrimaryKeyConstraint('id'),
|
||||
)
|
||||
op.create_index(op.f('ix_groups_name'), 'groups', ['name'], unique=True)
|
||||
|
||||
op.create_table(
|
||||
'group_memberships',
|
||||
sa.Column('id', sa.String(), nullable=False),
|
||||
sa.Column('group_id', sa.String(), nullable=False),
|
||||
sa.Column('user_id', sa.String(), nullable=False),
|
||||
sa.Column('joined_at', sa.DateTime(timezone=True), server_default=sa.text('now()'), nullable=False),
|
||||
sa.ForeignKeyConstraint(['group_id'], ['groups.id'], ondelete='CASCADE'),
|
||||
sa.ForeignKeyConstraint(['user_id'], ['users.id'], ondelete='CASCADE'),
|
||||
sa.PrimaryKeyConstraint('id'),
|
||||
sa.UniqueConstraint('group_id', 'user_id', name='uq_group_user'),
|
||||
)
|
||||
op.create_index(op.f('ix_group_memberships_group_id'), 'group_memberships', ['group_id'], unique=False)
|
||||
op.create_index(op.f('ix_group_memberships_user_id'), 'group_memberships', ['user_id'], unique=False)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_index(op.f('ix_group_memberships_user_id'), table_name='group_memberships')
|
||||
op.drop_index(op.f('ix_group_memberships_group_id'), table_name='group_memberships')
|
||||
op.drop_table('group_memberships')
|
||||
op.drop_index(op.f('ix_groups_name'), table_name='groups')
|
||||
op.drop_table('groups')
|
||||
@@ -0,0 +1,28 @@
|
||||
"""add dashboard_app_ids to users
|
||||
|
||||
Revision ID: c7e8f9a0b1d2
|
||||
Revises: a3f9c2d14e87
|
||||
Create Date: 2026-04-17 14:00:00.000000
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
|
||||
|
||||
revision: str = 'c7e8f9a0b1d2'
|
||||
down_revision: Union[str, None] = 'a3f9c2d14e87'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
'users',
|
||||
sa.Column('dashboard_app_ids', sa.JSON(), nullable=False, server_default='[]'),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column('users', 'dashboard_app_ids')
|
||||
@@ -0,0 +1,25 @@
|
||||
"""add_color_mode_to_users
|
||||
|
||||
Revision ID: dd6ad2f2c211
|
||||
Revises: c7e8f9a0b1d2
|
||||
Create Date: 2026-04-17 23:42:58.222958
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
from alembic import op
|
||||
import sqlalchemy as sa
|
||||
from sqlalchemy.dialects import postgresql
|
||||
|
||||
revision: str = 'dd6ad2f2c211'
|
||||
down_revision: Union[str, None] = 'c7e8f9a0b1d2'
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column('users', sa.Column('color_mode', sa.String(), nullable=True))
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column('users', 'color_mode')
|
||||
@@ -0,0 +1,32 @@
|
||||
"""add is_group_admin to group_memberships
|
||||
|
||||
Revision ID: e1f2a3b4c5d6
|
||||
Revises: dd6ad2f2c211
|
||||
Create Date: 2026-04-18
|
||||
|
||||
"""
|
||||
from typing import Sequence, Union
|
||||
|
||||
import sqlalchemy as sa
|
||||
from alembic import op
|
||||
|
||||
revision: str = "e1f2a3b4c5d6"
|
||||
down_revision: Union[str, None] = "dd6ad2f2c211"
|
||||
branch_labels: Union[str, Sequence[str], None] = None
|
||||
depends_on: Union[str, Sequence[str], None] = None
|
||||
|
||||
|
||||
def upgrade() -> None:
|
||||
op.add_column(
|
||||
"group_memberships",
|
||||
sa.Column(
|
||||
"is_group_admin",
|
||||
sa.Boolean(),
|
||||
nullable=False,
|
||||
server_default=sa.text("false"),
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
def downgrade() -> None:
|
||||
op.drop_column("group_memberships", "is_group_admin")
|
||||
@@ -0,0 +1,409 @@
|
||||
"""
|
||||
Per-service runtime config helpers.
|
||||
|
||||
All config files are stored in the 'config' bucket of the storage-service.
|
||||
Every function is async — callers must await them.
|
||||
|
||||
Key layout in the config bucket:
|
||||
ai_service_config.json
|
||||
doc_service_config.json
|
||||
appearance_config.json
|
||||
themes/{id}.json
|
||||
"""
|
||||
import copy
|
||||
import logging
|
||||
import re
|
||||
from copy import deepcopy
|
||||
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core import config_storage
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
# ── AI service config schemas ──────────────────────────────────────────────────
|
||||
|
||||
|
||||
class AnthropicConfig(BaseModel):
|
||||
api_key: str = ""
|
||||
model: str = "claude-haiku-4-5-20251001"
|
||||
|
||||
|
||||
class OllamaConfig(BaseModel):
|
||||
base_url: str = "http://host.docker.internal:11434/v1"
|
||||
model: str = "llama3.2"
|
||||
api_key: str = "ollama"
|
||||
|
||||
|
||||
class LMStudioConfig(BaseModel):
|
||||
base_url: str = "http://host.docker.internal:1234/v1"
|
||||
model: str = "local-model"
|
||||
api_key: str = "lm-studio"
|
||||
|
||||
|
||||
class AIServiceConfig(BaseModel):
|
||||
provider: str = "lmstudio"
|
||||
timeout_seconds: int = 60
|
||||
max_retries: int = 2
|
||||
anthropic: AnthropicConfig = AnthropicConfig()
|
||||
ollama: OllamaConfig = OllamaConfig()
|
||||
lmstudio: LMStudioConfig = LMStudioConfig()
|
||||
|
||||
|
||||
# ── Doc service config schemas ─────────────────────────────────────────────────
|
||||
|
||||
_DOC_SYSTEM_PROMPT_DEFAULT = (
|
||||
"You are a financial document analysis assistant. "
|
||||
"Given the text extracted from a PDF document, return ONLY a JSON object "
|
||||
"with no markdown, no code fences, and no explanation."
|
||||
)
|
||||
|
||||
_DOC_USER_TEMPLATE_DEFAULT = (
|
||||
'Analyze the following document text and return a JSON object with exactly these keys:\n'
|
||||
'title (a short, descriptive human-readable title for this document, e.g. "ACME Corp Invoice April 2026", "Office Supplies Receipt", "Q1 Flower Delivery Order"),\n'
|
||||
'document_type (one of: invoice, bill, receipt, order, expense, revenue, unknown),\n'
|
||||
'total_amount (string or null),\n'
|
||||
'currency (string or null),\n'
|
||||
'vendor_name (string or null),\n'
|
||||
'customer_name (string or null),\n'
|
||||
'billing_address (string or null),\n'
|
||||
'customer_address (string or null),\n'
|
||||
'invoice_number (string or null),\n'
|
||||
'invoice_date (string or null),\n'
|
||||
'due_date (string or null),\n'
|
||||
'tags (array of short keyword strings describing the document),\n'
|
||||
'line_items (array of objects, each with keys: description, amount),\n'
|
||||
'suggested_categories (array of 2 to 5 short category name strings a user might want to file this document under, e.g. "Utilities", "Travel", "Software Subscriptions", "Client Invoices").\n'
|
||||
'\n'
|
||||
'Document text:\n'
|
||||
'{text}'
|
||||
)
|
||||
|
||||
|
||||
class DocumentsConfig(BaseModel):
|
||||
max_pdf_bytes: int = 20 * 1024 * 1024
|
||||
|
||||
|
||||
class DocServiceSystemPrompts(BaseModel):
|
||||
system: str = _DOC_SYSTEM_PROMPT_DEFAULT
|
||||
user_template: str = _DOC_USER_TEMPLATE_DEFAULT
|
||||
|
||||
|
||||
class DocServiceConfig(BaseModel):
|
||||
documents: DocumentsConfig = DocumentsConfig()
|
||||
system_prompts: DocServiceSystemPrompts = DocServiceSystemPrompts()
|
||||
|
||||
|
||||
# ── Masking ────────────────────────────────────────────────────────────────────
|
||||
|
||||
def _mask_key(key: str) -> str:
|
||||
if not key or len(key) <= 8:
|
||||
return "••••"
|
||||
return key[:7] + "••••"
|
||||
|
||||
|
||||
def _mask_ai_config(data: dict) -> dict:
|
||||
masked = copy.deepcopy(data)
|
||||
for provider in ("anthropic", "ollama", "lmstudio"):
|
||||
if provider in masked and "api_key" in masked[provider]:
|
||||
masked[provider]["api_key"] = _mask_key(masked[provider]["api_key"])
|
||||
return masked
|
||||
|
||||
|
||||
# ── Load / Save ────────────────────────────────────────────────────────────────
|
||||
|
||||
async def load_service_config(service: str) -> dict:
|
||||
data = await config_storage.read_json(f"{service}_config.json")
|
||||
if data is None:
|
||||
if service == "ai_service":
|
||||
return AIServiceConfig().model_dump()
|
||||
if service == "doc_service":
|
||||
return DocServiceConfig().model_dump()
|
||||
return {}
|
||||
return data
|
||||
|
||||
|
||||
async def save_service_config(service: str, data: dict) -> None:
|
||||
await config_storage.write_json(f"{service}_config.json", data)
|
||||
|
||||
|
||||
# AI service helpers
|
||||
|
||||
async def load_ai_service_config() -> AIServiceConfig:
|
||||
raw = await load_service_config("ai_service")
|
||||
return AIServiceConfig.model_validate(raw)
|
||||
|
||||
|
||||
async def save_ai_service_config(config: AIServiceConfig) -> None:
|
||||
await save_service_config("ai_service", config.model_dump())
|
||||
|
||||
|
||||
async def load_ai_service_config_masked() -> dict:
|
||||
raw = await load_service_config("ai_service")
|
||||
return _mask_ai_config(raw)
|
||||
|
||||
|
||||
# Doc service helpers
|
||||
|
||||
async def load_doc_service_config() -> DocServiceConfig:
|
||||
raw = await load_service_config("doc_service")
|
||||
return DocServiceConfig.model_validate(raw)
|
||||
|
||||
|
||||
async def save_doc_service_config(config: DocServiceConfig) -> None:
|
||||
await save_service_config("doc_service", config.model_dump())
|
||||
|
||||
|
||||
async def load_doc_service_config_masked() -> dict:
|
||||
return await load_service_config("doc_service")
|
||||
|
||||
|
||||
def _merge_api_key(new_key: str, existing_key: str) -> str:
|
||||
"""If new_key is empty or a masked value, keep the existing key."""
|
||||
if not new_key or "••••" in new_key:
|
||||
return existing_key
|
||||
return new_key
|
||||
|
||||
|
||||
# ── System prompts helpers ─────────────────────────────────────────────────────
|
||||
|
||||
SYSTEM_PROMPT_SERVICES: dict[str, str] = {
|
||||
"doc_service": "Document Service",
|
||||
}
|
||||
|
||||
|
||||
async def load_all_system_prompts() -> dict:
|
||||
"""Return {service_id: {label, system, user_template, default_system, default_user_template}}."""
|
||||
result: dict = {}
|
||||
for service_id, label in SYSTEM_PROMPT_SERVICES.items():
|
||||
config = await load_service_config(service_id)
|
||||
prompts = config.get("system_prompts", {})
|
||||
defaults = _get_service_prompt_defaults(service_id)
|
||||
result[service_id] = {
|
||||
"label": label,
|
||||
"system": prompts.get("system", defaults["system"]),
|
||||
"user_template": prompts.get("user_template", defaults["user_template"]),
|
||||
"default_system": defaults["system"],
|
||||
"default_user_template": defaults["user_template"],
|
||||
}
|
||||
return result
|
||||
|
||||
|
||||
async def save_service_system_prompts(service_id: str, system: str, user_template: str) -> None:
|
||||
if service_id not in SYSTEM_PROMPT_SERVICES:
|
||||
raise ValueError(f"Unknown service: {service_id!r}")
|
||||
config = await load_service_config(service_id)
|
||||
config.setdefault("system_prompts", {})
|
||||
config["system_prompts"]["system"] = system
|
||||
config["system_prompts"]["user_template"] = user_template
|
||||
await save_service_config(service_id, config)
|
||||
|
||||
|
||||
def _get_service_prompt_defaults(service_id: str) -> dict:
|
||||
if service_id == "doc_service":
|
||||
d = DocServiceSystemPrompts()
|
||||
return {"system": d.system, "user_template": d.user_template}
|
||||
return {"system": "", "user_template": ""}
|
||||
|
||||
|
||||
# ── Appearance config ──────────────────────────────────────────────────────────
|
||||
|
||||
class AppearanceConfig(BaseModel):
|
||||
theme: str = "default"
|
||||
default_mode: str = "system"
|
||||
|
||||
|
||||
async def load_appearance_config() -> AppearanceConfig:
|
||||
data = await config_storage.read_json("appearance_config.json")
|
||||
if data is None:
|
||||
return AppearanceConfig()
|
||||
return AppearanceConfig.model_validate(data)
|
||||
|
||||
|
||||
async def save_appearance_config(config: AppearanceConfig) -> None:
|
||||
await config_storage.write_json("appearance_config.json", config.model_dump())
|
||||
|
||||
|
||||
# ── Theme file management ──────────────────────────────────────────────────────
|
||||
|
||||
# 9 required colour tokens per mode
|
||||
_REQUIRED_TOKENS = frozenset({
|
||||
"primary", "primary_hover", "accent", "accent_hover",
|
||||
"background", "surface", "border", "text_primary", "text_muted",
|
||||
})
|
||||
|
||||
_RGB_RE = re.compile(r"^\d{1,3} \d{1,3} \d{1,3}$")
|
||||
|
||||
_BUILTIN_THEMES: list[dict] = [
|
||||
{
|
||||
"id": "default",
|
||||
"label": "Default",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "37 99 235",
|
||||
"primary_hover": "29 78 216",
|
||||
"accent": "234 179 8",
|
||||
"accent_hover": "202 138 4",
|
||||
"background": "248 250 252",
|
||||
"surface": "255 255 255",
|
||||
"border": "226 232 240",
|
||||
"text_primary": "15 23 42",
|
||||
"text_muted": "100 116 139",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "59 130 246",
|
||||
"primary_hover": "37 99 235",
|
||||
"accent": "250 204 21",
|
||||
"accent_hover": "234 179 8",
|
||||
"background": "15 23 42",
|
||||
"surface": "30 41 59",
|
||||
"border": "51 65 85",
|
||||
"text_primary": "203 213 225",
|
||||
"text_muted": "148 163 184",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "pastel",
|
||||
"label": "Pastel",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "124 58 237",
|
||||
"primary_hover": "109 40 217",
|
||||
"accent": "236 72 153",
|
||||
"accent_hover": "219 39 119",
|
||||
"background": "253 244 255",
|
||||
"surface": "250 245 255",
|
||||
"border": "233 213 255",
|
||||
"text_primary": "30 27 75",
|
||||
"text_muted": "107 114 128",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "167 139 250",
|
||||
"primary_hover": "196 181 253",
|
||||
"accent": "244 114 182",
|
||||
"accent_hover": "251 164 200",
|
||||
"background": "30 20 51",
|
||||
"surface": "45 27 78",
|
||||
"border": "76 53 117",
|
||||
"text_primary": "233 213 255",
|
||||
"text_muted": "196 181 253",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "high-contrast",
|
||||
"label": "High Contrast",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "30 58 138",
|
||||
"primary_hover": "30 64 175",
|
||||
"accent": "220 38 38",
|
||||
"accent_hover": "185 28 28",
|
||||
"background": "255 255 255",
|
||||
"surface": "255 255 255",
|
||||
"border": "156 163 175",
|
||||
"text_primary": "0 0 0",
|
||||
"text_muted": "75 85 99",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "96 165 250",
|
||||
"primary_hover": "147 197 253",
|
||||
"accent": "248 113 113",
|
||||
"accent_hover": "252 165 165",
|
||||
"background": "0 0 0",
|
||||
"surface": "10 10 10",
|
||||
"border": "55 65 81",
|
||||
"text_primary": "255 255 255",
|
||||
"text_muted": "156 163 175",
|
||||
},
|
||||
},
|
||||
{
|
||||
"id": "ocean",
|
||||
"label": "Ocean Blue",
|
||||
"builtin": True,
|
||||
"light": {
|
||||
"primary": "29 78 216",
|
||||
"primary_hover": "30 58 138",
|
||||
"accent": "8 145 178",
|
||||
"accent_hover": "14 116 144",
|
||||
"background": "239 246 255",
|
||||
"surface": "219 234 254",
|
||||
"border": "147 197 253",
|
||||
"text_primary": "30 58 138",
|
||||
"text_muted": "59 130 246",
|
||||
},
|
||||
"dark": {
|
||||
"primary": "96 165 250",
|
||||
"primary_hover": "147 197 253",
|
||||
"accent": "34 211 238",
|
||||
"accent_hover": "103 232 249",
|
||||
"background": "10 22 40",
|
||||
"surface": "15 36 68",
|
||||
"border": "29 78 216",
|
||||
"text_primary": "219 234 254",
|
||||
"text_muted": "147 197 253",
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
async def seed_builtin_themes() -> None:
|
||||
"""Write built-in theme files to storage-service if they are not already there."""
|
||||
existing_keys = await config_storage.list_keys(prefix="themes/")
|
||||
existing_ids = {k.removeprefix("themes/").removesuffix(".json") for k in existing_keys}
|
||||
for theme in _BUILTIN_THEMES:
|
||||
if theme["id"] not in existing_ids:
|
||||
await config_storage.write_json(f"themes/{theme['id']}.json", theme)
|
||||
logger.info("Built-in themes seeded (%d themes)", len(_BUILTIN_THEMES))
|
||||
|
||||
|
||||
async def load_all_themes() -> list[dict]:
|
||||
"""Return all themes from storage-service, built-ins first then custom by label."""
|
||||
keys = await config_storage.list_keys(prefix="themes/")
|
||||
themes: list[dict] = []
|
||||
for key in keys:
|
||||
data = await config_storage.read_json(key)
|
||||
if data:
|
||||
themes.append(data)
|
||||
|
||||
builtin_ids = [t["id"] for t in _BUILTIN_THEMES]
|
||||
|
||||
def sort_key(t: dict) -> tuple:
|
||||
tid = t.get("id", "")
|
||||
try:
|
||||
return (0, builtin_ids.index(tid))
|
||||
except ValueError:
|
||||
return (1, t.get("label", tid).lower())
|
||||
|
||||
return sorted(themes, key=sort_key)
|
||||
|
||||
|
||||
async def load_theme_by_id(theme_id: str) -> dict | None:
|
||||
"""Return a single theme dict, or None if not found."""
|
||||
return await config_storage.read_json(f"themes/{theme_id}.json")
|
||||
|
||||
|
||||
async def save_theme(theme: dict) -> None:
|
||||
"""Write a theme to storage-service."""
|
||||
await config_storage.write_json(f"themes/{theme['id']}.json", theme)
|
||||
|
||||
|
||||
async def delete_theme(theme_id: str) -> None:
|
||||
"""Delete a custom theme. Raises ValueError for built-ins, KeyError if not found."""
|
||||
data = await config_storage.read_json(f"themes/{theme_id}.json")
|
||||
if data is None:
|
||||
raise FileNotFoundError(theme_id)
|
||||
if data.get("builtin"):
|
||||
raise ValueError("Cannot delete a built-in theme")
|
||||
await config_storage.delete_key(f"themes/{theme_id}.json")
|
||||
|
||||
|
||||
def validate_theme_tokens(colors: dict) -> list[str]:
|
||||
"""Return a list of validation error messages, empty if valid."""
|
||||
errors = []
|
||||
missing = _REQUIRED_TOKENS - set(colors.keys())
|
||||
if missing:
|
||||
errors.append(f"Missing tokens: {', '.join(sorted(missing))}")
|
||||
for key, val in colors.items():
|
||||
if key in _REQUIRED_TOKENS and not _RGB_RE.match(str(val)):
|
||||
errors.append(f"Token '{key}' must be an RGB triplet like '37 99 235', got: {val!r}")
|
||||
return errors
|
||||
@@ -1,3 +1,4 @@
|
||||
from pydantic import field_validator
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
@@ -6,12 +7,24 @@ class Settings(BaseSettings):
|
||||
|
||||
DATABASE_URL: str = "postgresql+asyncpg://postgres:password@localhost:5432/destroying_sap"
|
||||
|
||||
SECRET_KEY: str = "change-me-in-production"
|
||||
ALGORITHM: str = "HS256"
|
||||
ACCESS_TOKEN_EXPIRE_MINUTES: int = 60 * 24 # 1 day
|
||||
# RS256 asymmetric signing — generate keys with scripts/generate_jwt_keys.py
|
||||
ALGORITHM: str = "RS256"
|
||||
JWT_PRIVATE_KEY: str = "" # PEM, required; set via env var
|
||||
JWT_PUBLIC_KEY: str = "" # PEM, required; set via env var
|
||||
ACCESS_TOKEN_EXPIRE_MINUTES: int = 60 * 8 # 8 hours — no permanent sessions
|
||||
|
||||
CORS_ORIGINS: list[str] = ["http://localhost:5173"]
|
||||
|
||||
DOC_SERVICE_URL: str = "http://doc-service:8001"
|
||||
AI_SERVICE_URL: str = "http://ai-service:8010"
|
||||
STORAGE_SERVICE_URL: str = "http://storage-service:8020"
|
||||
|
||||
@field_validator("JWT_PRIVATE_KEY", "JWT_PUBLIC_KEY", mode="before")
|
||||
@classmethod
|
||||
def expand_newlines(cls, v: str) -> str:
|
||||
"""Allow PEM keys stored on a single line with literal \\n in .env."""
|
||||
return v.replace("\\n", "\n") if isinstance(v, str) else v
|
||||
|
||||
class Config:
|
||||
env_file = ".env"
|
||||
|
||||
|
||||
@@ -0,0 +1,63 @@
|
||||
"""
|
||||
Async HTTP client for the 'config' bucket in storage-service.
|
||||
|
||||
All JSON config files (AI settings, doc settings, appearance, themes, …) are stored
|
||||
in the 'config' bucket under the storage-service. This module provides thin
|
||||
async helpers so app_config.py does not depend on the filesystem at all.
|
||||
"""
|
||||
import json
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_BUCKET = "config"
|
||||
_TIMEOUT = 10.0
|
||||
|
||||
|
||||
def _url(key: str) -> str:
|
||||
return f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}/{key}"
|
||||
|
||||
|
||||
async def read_json(key: str) -> dict | None:
|
||||
"""Return parsed JSON from the config bucket, or None if the key does not exist."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(_url(key))
|
||||
if resp.status_code == 404:
|
||||
return None
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
async def write_json(key: str, data: dict) -> None:
|
||||
"""Serialise *data* to JSON and PUT it into the config bucket."""
|
||||
payload = json.dumps(data, indent=2).encode()
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.put(
|
||||
_url(key),
|
||||
content=payload,
|
||||
headers={"Content-Type": "application/octet-stream"},
|
||||
)
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
async def delete_key(key: str) -> None:
|
||||
"""Delete a key from the config bucket. No-op if it does not exist."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.delete(_url(key))
|
||||
if resp.status_code not in (204, 404):
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
async def list_keys(prefix: str = "") -> list[str]:
|
||||
"""List all keys in the config bucket, optionally filtered by *prefix*."""
|
||||
async with httpx.AsyncClient(timeout=_TIMEOUT) as client:
|
||||
resp = await client.get(f"{settings.STORAGE_SERVICE_URL}/objects/{_BUCKET}")
|
||||
resp.raise_for_status()
|
||||
keys: list[str] = resp.json().get("keys", [])
|
||||
if prefix:
|
||||
keys = [k for k in keys if k.startswith(prefix)]
|
||||
return keys
|
||||
@@ -0,0 +1,74 @@
|
||||
"""
|
||||
Input sanitization utilities.
|
||||
|
||||
Every string that originates from user input and is destined for the database
|
||||
MUST pass through these helpers before reaching a SQLAlchemy model or query.
|
||||
SQLAlchemy's ORM already uses bound parameters (no raw SQL), so these helpers
|
||||
address the layer above: ensuring data is well-formed, length-capped, and free
|
||||
of null bytes or control characters before it is stored.
|
||||
"""
|
||||
|
||||
import re
|
||||
import unicodedata
|
||||
from datetime import date
|
||||
|
||||
# ── Constants ─────────────────────────────────────────────────────────────────
|
||||
|
||||
_PHONE_RE = re.compile(r"^\+?[\d\s\-()\[\]]{7,20}$")
|
||||
|
||||
# ── Core helper ───────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def sanitize_str(value: str | None, max_len: int = 255) -> str | None:
|
||||
"""Strip whitespace, reject null bytes and non-printable control characters,
|
||||
enforce a maximum length. Returns None unchanged so optional fields work
|
||||
naturally with ``Optional[str]`` annotations."""
|
||||
if value is None:
|
||||
return None
|
||||
|
||||
# Strip leading/trailing whitespace
|
||||
value = value.strip()
|
||||
|
||||
# Reject null bytes (common injection vector)
|
||||
if "\x00" in value:
|
||||
raise ValueError("Input must not contain null bytes")
|
||||
|
||||
# Reject ASCII control characters (0x01–0x1F, 0x7F) except tab/newline/CR
|
||||
# which may appear in multi-line fields. Use Unicode category 'Cc'.
|
||||
for ch in value:
|
||||
if unicodedata.category(ch) == "Cc" and ch not in ("\t", "\n", "\r"):
|
||||
raise ValueError("Input contains invalid control characters")
|
||||
|
||||
if len(value) > max_len:
|
||||
raise ValueError(f"Input must not exceed {max_len} characters")
|
||||
|
||||
return value if value != "" else None
|
||||
|
||||
|
||||
def normalize_email(value: str) -> str:
|
||||
"""Lowercase and strip an email address."""
|
||||
return value.strip().lower()
|
||||
|
||||
|
||||
def validate_phone(value: str | None) -> str | None:
|
||||
"""Sanitize then validate phone number format."""
|
||||
value = sanitize_str(value, max_len=20)
|
||||
if value is None:
|
||||
return None
|
||||
if not _PHONE_RE.match(value):
|
||||
raise ValueError(
|
||||
"Phone number may only contain digits, spaces, +, -, (, ) and [ ] "
|
||||
"and must be 7–20 characters"
|
||||
)
|
||||
return value
|
||||
|
||||
|
||||
def validate_date_of_birth(value: date | None) -> date | None:
|
||||
"""Reject obviously invalid birth dates (before 1900 or in the future)."""
|
||||
if value is None:
|
||||
return None
|
||||
if value.year < 1900:
|
||||
raise ValueError("Date of birth must be 1900 or later")
|
||||
if value > date.today():
|
||||
raise ValueError("Date of birth must not be in the future")
|
||||
return value
|
||||
@@ -1,30 +1,36 @@
|
||||
from datetime import datetime, timedelta, timezone
|
||||
|
||||
import bcrypt
|
||||
from jose import jwt
|
||||
from passlib.context import CryptContext
|
||||
|
||||
from app.core.config import settings
|
||||
|
||||
pwd_context = CryptContext(schemes=["bcrypt"], deprecated="auto")
|
||||
|
||||
_BCRYPT_ROUNDS = 13 # ~300 ms on modern hardware; increase over time as CPUs get faster
|
||||
|
||||
|
||||
def hash_password(password: str) -> str:
|
||||
return pwd_context.hash(password)
|
||||
return bcrypt.hashpw(password.encode(), bcrypt.gensalt(rounds=_BCRYPT_ROUNDS)).decode()
|
||||
|
||||
|
||||
def verify_password(plain: str, hashed: str) -> bool:
|
||||
return pwd_context.verify(plain, hashed)
|
||||
return bcrypt.checkpw(plain.encode(), hashed.encode())
|
||||
|
||||
|
||||
def create_access_token(subject: str) -> str:
|
||||
expire = datetime.now(timezone.utc) + timedelta(minutes=settings.ACCESS_TOKEN_EXPIRE_MINUTES)
|
||||
now = datetime.now(timezone.utc)
|
||||
expire = now + timedelta(minutes=settings.ACCESS_TOKEN_EXPIRE_MINUTES)
|
||||
return jwt.encode(
|
||||
{"sub": subject, "exp": expire},
|
||||
settings.SECRET_KEY,
|
||||
{"sub": subject, "exp": expire, "iat": now},
|
||||
settings.JWT_PRIVATE_KEY,
|
||||
algorithm=settings.ALGORITHM,
|
||||
)
|
||||
|
||||
|
||||
def decode_access_token(token: str) -> str:
|
||||
payload = jwt.decode(token, settings.SECRET_KEY, algorithms=[settings.ALGORITHM])
|
||||
payload = jwt.decode(
|
||||
token,
|
||||
settings.JWT_PUBLIC_KEY,
|
||||
algorithms=[settings.ALGORITHM],
|
||||
)
|
||||
return payload["sub"]
|
||||
|
||||
@@ -30,3 +30,82 @@ async def get_current_user(
|
||||
if not user or not user.is_active:
|
||||
raise credentials_exception
|
||||
return user
|
||||
|
||||
|
||||
async def get_current_admin(
|
||||
current_user: User = Depends(get_current_user),
|
||||
) -> User:
|
||||
if not current_user.is_superuser:
|
||||
# Return 404 instead of 403 — reveals neither the existence of the
|
||||
# endpoint nor that the caller lacks permission.
|
||||
raise HTTPException(
|
||||
status_code=status.HTTP_404_NOT_FOUND,
|
||||
detail="Not found",
|
||||
)
|
||||
return current_user
|
||||
|
||||
|
||||
def get_service_admin(service_id: str):
|
||||
"""
|
||||
Dependency factory that grants access to service-specific admin endpoints.
|
||||
|
||||
Access is granted if the user is a global superuser OR a member of the
|
||||
'{service_id}-admin' group. Returns 404 (not 403) to hide both the
|
||||
endpoint existence and the permission model.
|
||||
|
||||
Usage:
|
||||
@router.get("/ai")
|
||||
async def get_ai_settings(_: User = Depends(get_service_admin("ai-service"))):
|
||||
"""
|
||||
async def _dep(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> User:
|
||||
if current_user.is_superuser:
|
||||
return current_user
|
||||
if await check_plugin_access(service_id, current_user, db):
|
||||
return current_user
|
||||
raise HTTPException(status_code=status.HTTP_404_NOT_FOUND, detail="Not found")
|
||||
|
||||
return _dep
|
||||
|
||||
|
||||
async def check_plugin_access(
|
||||
plugin_id: str,
|
||||
current_user: User,
|
||||
db: AsyncSession,
|
||||
) -> bool:
|
||||
"""
|
||||
Return True if the user may access the given plugin's settings.
|
||||
|
||||
Access is granted when any of these conditions holds:
|
||||
1. The user is a superuser AND the manifest allows superuser access.
|
||||
2. The user is a member of one of the groups listed in manifest.access.required_groups.
|
||||
|
||||
Returns False (not raises) so callers can decide how to respond.
|
||||
"""
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.services.service_health import get_cached_manifest
|
||||
|
||||
manifest = get_cached_manifest(plugin_id)
|
||||
if manifest is None:
|
||||
return False
|
||||
|
||||
access = manifest.get("access", {})
|
||||
|
||||
if current_user.is_superuser and access.get("allow_superuser", True):
|
||||
return True
|
||||
|
||||
for group_name in access.get("required_groups", []):
|
||||
result = await db.execute(
|
||||
select(GroupMembership)
|
||||
.join(Group, Group.id == GroupMembership.group_id)
|
||||
.where(
|
||||
Group.name == group_name,
|
||||
GroupMembership.user_id == current_user.id,
|
||||
)
|
||||
)
|
||||
if result.scalar_one_or_none() is not None:
|
||||
return True
|
||||
|
||||
return False
|
||||
|
||||
+49
-2
@@ -1,10 +1,42 @@
|
||||
import asyncio
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
|
||||
from app.core.app_config import seed_builtin_themes
|
||||
from app.core.config import settings
|
||||
from app.routers import auth, users
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.routers import admin, auth, categories_proxy, documents_proxy, groups, plugins, profile, services, users
|
||||
from app.routers import settings as settings_router
|
||||
from app.routers import storage_config
|
||||
from app.services.group_bootstrap import ensure_service_admin_groups
|
||||
from app.services.service_health import check_all, health_check_loop, register_services
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME, version="0.1.0")
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
await seed_builtin_themes()
|
||||
register_services(
|
||||
doc_service_url=settings.DOC_SERVICE_URL,
|
||||
ai_service_url=settings.AI_SERVICE_URL,
|
||||
storage_service_url=settings.STORAGE_SERVICE_URL,
|
||||
)
|
||||
# Create <service-id>-admin groups for every registered service (idempotent)
|
||||
async with AsyncSessionLocal() as db:
|
||||
await ensure_service_admin_groups(db)
|
||||
# Run an initial check immediately so the first API response is accurate
|
||||
await check_all()
|
||||
task = asyncio.create_task(health_check_loop())
|
||||
yield
|
||||
task.cancel()
|
||||
try:
|
||||
await task
|
||||
except asyncio.CancelledError:
|
||||
pass
|
||||
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME, version="0.1.0", lifespan=lifespan)
|
||||
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
@@ -16,6 +48,21 @@ app.add_middleware(
|
||||
|
||||
app.include_router(auth.router, prefix="/api/auth", tags=["auth"])
|
||||
app.include_router(users.router, prefix="/api/users", tags=["users"])
|
||||
app.include_router(profile.router, prefix="/api/profile", tags=["profile"])
|
||||
app.include_router(admin.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(groups.router, prefix="/api/admin/groups", tags=["admin"])
|
||||
app.include_router(settings_router.router, prefix="/api/settings", tags=["settings"])
|
||||
app.include_router(services.router, prefix="/api/services", tags=["services"])
|
||||
app.include_router(storage_config.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(plugins.router, prefix="/api/plugins", tags=["plugins"])
|
||||
# categories_proxy MUST be registered before documents_proxy —
|
||||
# otherwise /api/documents/{path:path} swallows /api/documents/categories/*
|
||||
app.include_router(
|
||||
categories_proxy.router,
|
||||
prefix="/api/documents/categories",
|
||||
tags=["categories"],
|
||||
)
|
||||
app.include_router(documents_proxy.router, prefix="/api/documents", tags=["documents"])
|
||||
|
||||
|
||||
@app.get("/api/health")
|
||||
|
||||
@@ -1,3 +1,5 @@
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.models.profile import Profile
|
||||
from app.models.user import User
|
||||
|
||||
__all__ = ["User"]
|
||||
__all__ = ["User", "Profile", "Group", "GroupMembership"]
|
||||
|
||||
@@ -0,0 +1,47 @@
|
||||
import uuid
|
||||
from datetime import datetime, timezone
|
||||
|
||||
from sqlalchemy import Boolean, DateTime, ForeignKey, String, UniqueConstraint
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class Group(Base):
|
||||
__tablename__ = "groups"
|
||||
|
||||
id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
name: Mapped[str] = mapped_column(String(128), unique=True, nullable=False, index=True)
|
||||
description: Mapped[str | None] = mapped_column(String(512), nullable=True)
|
||||
created_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
memberships: Mapped[list["GroupMembership"]] = relationship(
|
||||
"GroupMembership", back_populates="group", cascade="all, delete-orphan"
|
||||
)
|
||||
|
||||
|
||||
class GroupMembership(Base):
|
||||
__tablename__ = "group_memberships"
|
||||
__table_args__ = (UniqueConstraint("group_id", "user_id", name="uq_group_user"),)
|
||||
|
||||
id: Mapped[str] = mapped_column(String, primary_key=True, default=lambda: str(uuid.uuid4()))
|
||||
group_id: Mapped[str] = mapped_column(
|
||||
String, ForeignKey("groups.id", ondelete="CASCADE"), nullable=False, index=True
|
||||
)
|
||||
user_id: Mapped[str] = mapped_column(
|
||||
String, ForeignKey("users.id", ondelete="CASCADE"), nullable=False, index=True
|
||||
)
|
||||
is_group_admin: Mapped[bool] = mapped_column(
|
||||
Boolean, nullable=False, default=False, server_default="false"
|
||||
)
|
||||
joined_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
default=lambda: datetime.now(timezone.utc),
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
group: Mapped["Group"] = relationship("Group", back_populates="memberships")
|
||||
@@ -0,0 +1,34 @@
|
||||
import uuid
|
||||
from datetime import date, datetime
|
||||
|
||||
from sqlalchemy import Date, DateTime, ForeignKey, String, func
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class Profile(Base):
|
||||
__tablename__ = "profiles"
|
||||
|
||||
id: Mapped[str] = mapped_column(
|
||||
String, primary_key=True, default=lambda: str(uuid.uuid4())
|
||||
)
|
||||
# One-to-one with users; deleting a user cascades to the profile.
|
||||
user_id: Mapped[str] = mapped_column(
|
||||
String, ForeignKey("users.id", ondelete="CASCADE"), unique=True, nullable=False
|
||||
)
|
||||
|
||||
phone: Mapped[str | None] = mapped_column(String(20), nullable=True)
|
||||
date_of_birth: Mapped[date | None] = mapped_column(Date, nullable=True)
|
||||
# Job title / role within the organisation (e.g. "Software Engineer", "HR Manager").
|
||||
position: Mapped[str | None] = mapped_column(String(128), nullable=True)
|
||||
address: Mapped[str | None] = mapped_column(String(255), nullable=True)
|
||||
|
||||
updated_at: Mapped[datetime] = mapped_column(
|
||||
DateTime(timezone=True),
|
||||
server_default=func.now(),
|
||||
onupdate=func.now(),
|
||||
nullable=False,
|
||||
)
|
||||
|
||||
user = relationship("User", back_populates="profile")
|
||||
@@ -1,10 +1,14 @@
|
||||
import uuid
|
||||
from typing import TYPE_CHECKING
|
||||
|
||||
from sqlalchemy import Boolean, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column
|
||||
from sqlalchemy import Boolean, JSON, String
|
||||
from sqlalchemy.orm import Mapped, mapped_column, relationship
|
||||
|
||||
from app.database import Base
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from app.models.profile import Profile
|
||||
|
||||
|
||||
class User(Base):
|
||||
__tablename__ = "users"
|
||||
@@ -14,4 +18,14 @@ class User(Base):
|
||||
hashed_password: Mapped[str] = mapped_column(String, nullable=False)
|
||||
full_name: Mapped[str] = mapped_column(String, nullable=True)
|
||||
is_active: Mapped[bool] = mapped_column(Boolean, default=True)
|
||||
# Role flag — True = admin, False = regular user.
|
||||
# Never exposed in API responses; set only by direct DB or admin tooling.
|
||||
is_superuser: Mapped[bool] = mapped_column(Boolean, default=False)
|
||||
# List of service IDs pinned to the user's home dashboard.
|
||||
dashboard_app_ids: Mapped[list] = mapped_column(JSON, nullable=False, default=list)
|
||||
# User's preferred colour mode: "light", "dark", "system", or None (use admin default).
|
||||
color_mode: Mapped[str | None] = mapped_column(String, nullable=True, default=None)
|
||||
|
||||
profile: Mapped["Profile"] = relationship(
|
||||
"Profile", back_populates="user", uselist=False, cascade="all, delete-orphan"
|
||||
)
|
||||
|
||||
@@ -0,0 +1,80 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.core.security import hash_password
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_admin
|
||||
from app.models.user import User
|
||||
from app.schemas.user import UserAdminCreate, UserAdminOut
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/users", response_model=list[UserAdminOut])
|
||||
async def list_users(
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[User]:
|
||||
result = await db.execute(select(User).order_by(User.email))
|
||||
return list(result.scalars().all())
|
||||
|
||||
|
||||
@router.post("/users", response_model=UserAdminOut, status_code=status.HTTP_201_CREATED)
|
||||
async def create_user(
|
||||
body: UserAdminCreate,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> User:
|
||||
existing = await db.execute(select(User).where(User.email == body.email))
|
||||
if existing.scalar_one_or_none():
|
||||
raise HTTPException(status_code=400, detail="Email already registered")
|
||||
|
||||
user = User(
|
||||
email=body.email,
|
||||
hashed_password=hash_password(body.password),
|
||||
full_name=body.full_name,
|
||||
is_superuser=body.is_admin,
|
||||
)
|
||||
db.add(user)
|
||||
await db.commit()
|
||||
await db.refresh(user)
|
||||
return user
|
||||
|
||||
|
||||
@router.delete("/users/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def delete_user(
|
||||
user_id: str,
|
||||
admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
if user_id == admin.id:
|
||||
raise HTTPException(status_code=400, detail="Cannot delete your own account")
|
||||
|
||||
result = await db.execute(select(User).where(User.id == user_id))
|
||||
user = result.scalar_one_or_none()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
await db.delete(user)
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.patch("/users/{user_id}/active", response_model=UserAdminOut)
|
||||
async def toggle_active(
|
||||
user_id: str,
|
||||
admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> User:
|
||||
if user_id == admin.id:
|
||||
raise HTTPException(status_code=400, detail="Cannot change your own active status")
|
||||
|
||||
result = await db.execute(select(User).where(User.id == user_id))
|
||||
user = result.scalar_one_or_none()
|
||||
if not user:
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
user.is_active = not user.is_active
|
||||
await db.commit()
|
||||
await db.refresh(user)
|
||||
return user
|
||||
@@ -0,0 +1,99 @@
|
||||
"""
|
||||
Proxy /api/documents/categories/* → doc-service:8001/categories/*.
|
||||
|
||||
Must be registered BEFORE the documents catch-all proxy in main.py,
|
||||
otherwise /api/documents/{path:path} swallows category requests.
|
||||
"""
|
||||
import os
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import GroupMembership
|
||||
from app.models.user import User
|
||||
|
||||
DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
|
||||
|
||||
_client = httpx.AsyncClient(base_url=DOC_SERVICE_URL, timeout=30.0)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_HOP_BY_HOP = frozenset([
|
||||
"connection",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"te",
|
||||
"trailers",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
"host",
|
||||
"accept-encoding",
|
||||
])
|
||||
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
async def _forward_headers(
|
||||
request: Request, user_id: str, is_admin: bool, db: AsyncSession
|
||||
) -> dict:
|
||||
headers = {
|
||||
k: v
|
||||
for k, v in request.headers.items()
|
||||
if k.lower() not in _HOP_BY_HOP
|
||||
}
|
||||
headers["x-user-id"] = user_id
|
||||
headers["x-user-is-admin"] = "true" if is_admin else "false"
|
||||
|
||||
mem_result = await db.execute(
|
||||
select(GroupMembership.group_id, GroupMembership.is_group_admin)
|
||||
.where(GroupMembership.user_id == user_id)
|
||||
)
|
||||
rows = mem_result.all()
|
||||
group_ids = [row[0] for row in rows]
|
||||
admin_group_ids = [row[0] for row in rows if row[1]]
|
||||
headers["x-user-groups"] = ",".join(group_ids)
|
||||
headers["x-user-admin-groups"] = ",".join(admin_group_ids)
|
||||
return headers
|
||||
|
||||
|
||||
@router.api_route("", methods=["GET", "POST"])
|
||||
@router.api_route("/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
|
||||
async def proxy_categories(
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
path: str = "",
|
||||
) -> Response:
|
||||
url = f"/categories/{path}" if path else "/categories"
|
||||
headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
|
||||
body = await request.body()
|
||||
|
||||
try:
|
||||
response = await _client.request(
|
||||
method=request.method,
|
||||
url=url,
|
||||
headers=headers,
|
||||
content=body,
|
||||
params=dict(request.query_params),
|
||||
)
|
||||
except httpx.RequestError as exc:
|
||||
raise HTTPException(status_code=502, detail=f"doc-service unreachable: {exc}")
|
||||
|
||||
resp_headers = {
|
||||
k: v
|
||||
for k, v in response.headers.items()
|
||||
if k.lower() not in _STRIP_RESPONSE
|
||||
}
|
||||
|
||||
return Response(
|
||||
content=response.content,
|
||||
status_code=response.status_code,
|
||||
headers=resp_headers,
|
||||
media_type=response.headers.get("content-type"),
|
||||
)
|
||||
@@ -0,0 +1,113 @@
|
||||
"""
|
||||
Proxy all /api/documents/* requests to doc-service:8001/documents/*.
|
||||
|
||||
Uses a module-level AsyncClient for connection pooling.
|
||||
Strips hop-by-hop headers that must not be forwarded.
|
||||
Injects X-User-Id and X-User-Groups headers so the doc-service
|
||||
can enforce ownership and group-sharing access without querying the
|
||||
backend database directly.
|
||||
"""
|
||||
import os
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import GroupMembership
|
||||
from app.models.user import User
|
||||
|
||||
DOC_SERVICE_URL = os.environ.get("DOC_SERVICE_URL", "http://doc-service:8001")
|
||||
|
||||
_client = httpx.AsyncClient(base_url=DOC_SERVICE_URL, timeout=120.0)
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
# Headers that must not be forwarded in either direction.
|
||||
# Also strip accept-encoding so doc-service never compresses responses —
|
||||
# httpx decompresses transparently but the content-encoding header would
|
||||
# then mismatch the already-decompressed body we forward to the browser.
|
||||
_HOP_BY_HOP = frozenset([
|
||||
"connection",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"te",
|
||||
"trailers",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
"host",
|
||||
"accept-encoding",
|
||||
])
|
||||
|
||||
# Additional response headers we let FastAPI recalculate rather than forward.
|
||||
# content-length is set automatically from the response body size.
|
||||
# content-type is set via the media_type argument, so strip it from headers
|
||||
# to avoid duplicates.
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
async def _forward_headers(
|
||||
request: Request, user_id: str, is_admin: bool, db: AsyncSession
|
||||
) -> dict:
|
||||
headers = {
|
||||
k: v
|
||||
for k, v in request.headers.items()
|
||||
if k.lower() not in _HOP_BY_HOP
|
||||
}
|
||||
headers["x-user-id"] = user_id
|
||||
headers["x-user-is-admin"] = "true" if is_admin else "false"
|
||||
|
||||
# Inject group memberships and group-admin status so the doc-service can
|
||||
# evaluate ownership, sharing access, and group-admin permissions.
|
||||
mem_result = await db.execute(
|
||||
select(GroupMembership.group_id, GroupMembership.is_group_admin)
|
||||
.where(GroupMembership.user_id == user_id)
|
||||
)
|
||||
rows = mem_result.all()
|
||||
group_ids = [row[0] for row in rows]
|
||||
admin_group_ids = [row[0] for row in rows if row[1]]
|
||||
headers["x-user-groups"] = ",".join(group_ids)
|
||||
headers["x-user-admin-groups"] = ",".join(admin_group_ids)
|
||||
|
||||
return headers
|
||||
|
||||
|
||||
@router.api_route("", methods=["GET", "POST"])
|
||||
@router.api_route("/{path:path}", methods=["GET", "POST", "PUT", "PATCH", "DELETE"])
|
||||
async def proxy_documents(
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
path: str = "",
|
||||
) -> Response:
|
||||
url = f"/documents/{path}" if path else "/documents"
|
||||
headers = await _forward_headers(request, str(current_user.id), current_user.is_superuser, db)
|
||||
body = await request.body()
|
||||
|
||||
try:
|
||||
response = await _client.request(
|
||||
method=request.method,
|
||||
url=url,
|
||||
headers=headers,
|
||||
content=body,
|
||||
params=dict(request.query_params),
|
||||
)
|
||||
except httpx.RequestError as exc:
|
||||
raise HTTPException(status_code=502, detail=f"doc-service unreachable: {exc}")
|
||||
|
||||
resp_headers = {
|
||||
k: v
|
||||
for k, v in response.headers.items()
|
||||
if k.lower() not in _STRIP_RESPONSE
|
||||
}
|
||||
|
||||
return Response(
|
||||
content=response.content,
|
||||
status_code=response.status_code,
|
||||
headers=resp_headers,
|
||||
media_type=response.headers.get("content-type"),
|
||||
)
|
||||
@@ -0,0 +1,237 @@
|
||||
from fastapi import APIRouter, Depends, HTTPException, status
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_admin
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.models.user import User
|
||||
from app.schemas.group import GroupCreate, GroupDetailOut, GroupMemberAdminUpdate, GroupMemberOut, GroupOut, GroupUpdate
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
def _to_group_out(group: Group) -> GroupOut:
|
||||
return GroupOut(
|
||||
id=group.id,
|
||||
name=group.name,
|
||||
description=group.description,
|
||||
created_at=group.created_at,
|
||||
member_count=len(group.memberships),
|
||||
)
|
||||
|
||||
|
||||
def _to_group_detail(group: Group) -> GroupDetailOut:
|
||||
members = []
|
||||
for m in group.memberships:
|
||||
# memberships are loaded with joined user via selectinload
|
||||
user = m.__dict__.get("_user") or getattr(m, "user", None)
|
||||
if user is None:
|
||||
continue
|
||||
members.append(GroupMemberOut(
|
||||
id=user.id,
|
||||
email=user.email,
|
||||
full_name=user.full_name,
|
||||
is_active=user.is_active,
|
||||
joined_at=m.joined_at,
|
||||
))
|
||||
return GroupDetailOut(
|
||||
id=group.id,
|
||||
name=group.name,
|
||||
description=group.description,
|
||||
created_at=group.created_at,
|
||||
member_count=len(members),
|
||||
members=members,
|
||||
)
|
||||
|
||||
|
||||
@router.get("", response_model=list[GroupOut])
|
||||
async def list_groups(
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[GroupOut]:
|
||||
result = await db.execute(
|
||||
select(Group).options(selectinload(Group.memberships)).order_by(Group.name)
|
||||
)
|
||||
groups = list(result.scalars().all())
|
||||
return [_to_group_out(g) for g in groups]
|
||||
|
||||
|
||||
@router.post("", response_model=GroupOut, status_code=status.HTTP_201_CREATED)
|
||||
async def create_group(
|
||||
body: GroupCreate,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> GroupOut:
|
||||
existing = await db.execute(select(Group).where(Group.name == body.name))
|
||||
if existing.scalar_one_or_none():
|
||||
raise HTTPException(status_code=400, detail="A group with that name already exists")
|
||||
|
||||
group = Group(name=body.name, description=body.description)
|
||||
db.add(group)
|
||||
await db.commit()
|
||||
await db.refresh(group)
|
||||
# refresh doesn't load relationships — load fresh with memberships
|
||||
result = await db.execute(
|
||||
select(Group).options(selectinload(Group.memberships)).where(Group.id == group.id)
|
||||
)
|
||||
group = result.scalar_one()
|
||||
return _to_group_out(group)
|
||||
|
||||
|
||||
@router.get("/{group_id}", response_model=GroupDetailOut)
|
||||
async def get_group(
|
||||
group_id: str,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> GroupDetailOut:
|
||||
result = await db.execute(
|
||||
select(Group)
|
||||
.options(
|
||||
selectinload(Group.memberships).joinedload(GroupMembership.group),
|
||||
)
|
||||
.where(Group.id == group_id)
|
||||
)
|
||||
group = result.scalar_one_or_none()
|
||||
if not group:
|
||||
raise HTTPException(status_code=404, detail="Group not found")
|
||||
|
||||
# Load members separately for simplicity
|
||||
mem_result = await db.execute(
|
||||
select(GroupMembership, User)
|
||||
.join(User, User.id == GroupMembership.user_id)
|
||||
.where(GroupMembership.group_id == group_id)
|
||||
.order_by(User.email)
|
||||
)
|
||||
rows = mem_result.all()
|
||||
members = [
|
||||
GroupMemberOut(
|
||||
id=user.id,
|
||||
email=user.email,
|
||||
full_name=user.full_name,
|
||||
is_active=user.is_active,
|
||||
is_group_admin=membership.is_group_admin,
|
||||
joined_at=membership.joined_at,
|
||||
)
|
||||
for membership, user in rows
|
||||
]
|
||||
return GroupDetailOut(
|
||||
id=group.id,
|
||||
name=group.name,
|
||||
description=group.description,
|
||||
created_at=group.created_at,
|
||||
member_count=len(members),
|
||||
members=members,
|
||||
)
|
||||
|
||||
|
||||
@router.patch("/{group_id}", response_model=GroupOut)
|
||||
async def update_group(
|
||||
group_id: str,
|
||||
body: GroupUpdate,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> GroupOut:
|
||||
result = await db.execute(
|
||||
select(Group).options(selectinload(Group.memberships)).where(Group.id == group_id)
|
||||
)
|
||||
group = result.scalar_one_or_none()
|
||||
if not group:
|
||||
raise HTTPException(status_code=404, detail="Group not found")
|
||||
|
||||
if body.name is not None and body.name != group.name:
|
||||
dupe = await db.execute(select(Group).where(Group.name == body.name))
|
||||
if dupe.scalar_one_or_none():
|
||||
raise HTTPException(status_code=400, detail="A group with that name already exists")
|
||||
group.name = body.name
|
||||
if body.description is not None:
|
||||
group.description = body.description
|
||||
|
||||
await db.commit()
|
||||
await db.refresh(group)
|
||||
result2 = await db.execute(
|
||||
select(Group).options(selectinload(Group.memberships)).where(Group.id == group.id)
|
||||
)
|
||||
group = result2.scalar_one()
|
||||
return _to_group_out(group)
|
||||
|
||||
|
||||
@router.delete("/{group_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def delete_group(
|
||||
group_id: str,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
result = await db.execute(select(Group).where(Group.id == group_id))
|
||||
group = result.scalar_one_or_none()
|
||||
if not group:
|
||||
raise HTTPException(status_code=404, detail="Group not found")
|
||||
await db.delete(group)
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.post("/{group_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def add_member(
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
group_result = await db.execute(select(Group).where(Group.id == group_id))
|
||||
if not group_result.scalar_one_or_none():
|
||||
raise HTTPException(status_code=404, detail="Group not found")
|
||||
|
||||
user_result = await db.execute(select(User).where(User.id == user_id))
|
||||
if not user_result.scalar_one_or_none():
|
||||
raise HTTPException(status_code=404, detail="User not found")
|
||||
|
||||
existing = await db.execute(
|
||||
select(GroupMembership).where(
|
||||
GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
|
||||
)
|
||||
)
|
||||
if existing.scalar_one_or_none():
|
||||
raise HTTPException(status_code=400, detail="User is already a member of this group")
|
||||
|
||||
db.add(GroupMembership(group_id=group_id, user_id=user_id))
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.patch("/{group_id}/members/{user_id}/admin", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def set_member_admin(
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
body: GroupMemberAdminUpdate,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
result = await db.execute(
|
||||
select(GroupMembership).where(
|
||||
GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
|
||||
)
|
||||
)
|
||||
membership = result.scalar_one_or_none()
|
||||
if not membership:
|
||||
raise HTTPException(status_code=404, detail="User is not a member of this group")
|
||||
membership.is_group_admin = body.is_group_admin
|
||||
await db.commit()
|
||||
|
||||
|
||||
@router.delete("/{group_id}/members/{user_id}", status_code=status.HTTP_204_NO_CONTENT)
|
||||
async def remove_member(
|
||||
group_id: str,
|
||||
user_id: str,
|
||||
_admin: User = Depends(get_current_admin),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> None:
|
||||
result = await db.execute(
|
||||
select(GroupMembership).where(
|
||||
GroupMembership.group_id == group_id, GroupMembership.user_id == user_id
|
||||
)
|
||||
)
|
||||
membership = result.scalar_one_or_none()
|
||||
if not membership:
|
||||
raise HTTPException(status_code=404, detail="User is not a member of this group")
|
||||
await db.delete(membership)
|
||||
await db.commit()
|
||||
@@ -0,0 +1,125 @@
|
||||
"""
|
||||
Generic plugin proxy.
|
||||
|
||||
Feature containers advertise themselves via GET /plugin/manifest. The backend
|
||||
health-poller caches those manifests. This router exposes them to the browser
|
||||
through auth-gated endpoints so the frontend never needs to know about specific
|
||||
features.
|
||||
|
||||
Routes:
|
||||
GET /api/plugins → list accessible plugins for current user
|
||||
GET /api/plugins/{id}/manifest → cached manifest (404 if not accessible)
|
||||
GET /api/plugins/{id}/settings → proxy to feature /plugin/settings
|
||||
PATCH /api/plugins/{id}/settings → proxy to feature /plugin/settings
|
||||
"""
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException, Request
|
||||
from fastapi.responses import Response
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import check_plugin_access, get_current_user
|
||||
from app.models.user import User
|
||||
from app.services.service_health import get_cached_manifest, get_registry, get_service_url
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_HOP_BY_HOP = frozenset([
|
||||
"connection",
|
||||
"keep-alive",
|
||||
"proxy-authenticate",
|
||||
"proxy-authorization",
|
||||
"te",
|
||||
"trailers",
|
||||
"transfer-encoding",
|
||||
"upgrade",
|
||||
"host",
|
||||
"accept-encoding",
|
||||
])
|
||||
_STRIP_RESPONSE = frozenset([*_HOP_BY_HOP, "content-length", "content-type"])
|
||||
|
||||
|
||||
async def _proxy(plugin_id: str, method: str, path: str, body: bytes | None,
|
||||
content_type: str | None = None) -> Response:
|
||||
"""Forward a request to the feature container's plugin endpoint."""
|
||||
url = get_service_url(plugin_id)
|
||||
if url is None:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
|
||||
headers: dict[str, str] = {}
|
||||
if content_type:
|
||||
headers["content-type"] = content_type
|
||||
|
||||
try:
|
||||
async with httpx.AsyncClient(base_url=url, timeout=30.0) as client:
|
||||
resp = await client.request(method, path, content=body, headers=headers)
|
||||
except httpx.RequestError as exc:
|
||||
raise HTTPException(status_code=502, detail=f"Plugin service unreachable: {exc}")
|
||||
|
||||
resp_headers = {k: v for k, v in resp.headers.items() if k.lower() not in _STRIP_RESPONSE}
|
||||
return Response(
|
||||
content=resp.content,
|
||||
status_code=resp.status_code,
|
||||
headers=resp_headers,
|
||||
media_type=resp.headers.get("content-type", "application/json"),
|
||||
)
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def list_plugins(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> list[dict]:
|
||||
"""Return the list of plugins the current user may access."""
|
||||
accessible = []
|
||||
for svc in get_registry():
|
||||
manifest = get_cached_manifest(svc.id)
|
||||
if manifest is None:
|
||||
continue
|
||||
if await check_plugin_access(svc.id, current_user, db):
|
||||
accessible.append({
|
||||
"id": manifest["id"],
|
||||
"name": manifest["name"],
|
||||
"icon": manifest.get("icon", "package"),
|
||||
"version": manifest.get("version", ""),
|
||||
})
|
||||
return accessible
|
||||
|
||||
|
||||
@router.get("/{plugin_id}/manifest")
|
||||
async def get_plugin_manifest(
|
||||
plugin_id: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> dict:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
manifest = get_cached_manifest(plugin_id)
|
||||
if manifest is None:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
return manifest
|
||||
|
||||
|
||||
@router.get("/{plugin_id}/settings")
|
||||
async def get_plugin_settings(
|
||||
plugin_id: str,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Response:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
return await _proxy(plugin_id, "GET", "/plugin/settings", None)
|
||||
|
||||
|
||||
@router.patch("/{plugin_id}/settings")
|
||||
async def update_plugin_settings(
|
||||
plugin_id: str,
|
||||
request: Request,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Response:
|
||||
if not await check_plugin_access(plugin_id, current_user, db):
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
body = await request.body()
|
||||
content_type = request.headers.get("content-type", "application/json")
|
||||
return await _proxy(plugin_id, "PATCH", "/plugin/settings", body, content_type)
|
||||
@@ -0,0 +1,48 @@
|
||||
from fastapi import APIRouter, Depends
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.profile import Profile
|
||||
from app.models.user import User
|
||||
from app.schemas.profile import ProfileRead, ProfileUpdate
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
async def _get_or_create_profile(user: User, db: AsyncSession) -> Profile:
|
||||
"""Return the user's profile, creating an empty one on first access."""
|
||||
result = await db.execute(select(Profile).where(Profile.user_id == user.id))
|
||||
profile = result.scalar_one_or_none()
|
||||
if profile is None:
|
||||
profile = Profile(user_id=user.id)
|
||||
db.add(profile)
|
||||
await db.commit()
|
||||
await db.refresh(profile)
|
||||
return profile
|
||||
|
||||
|
||||
@router.get("/me", response_model=ProfileRead)
|
||||
async def get_my_profile(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Profile:
|
||||
return await _get_or_create_profile(current_user, db)
|
||||
|
||||
|
||||
@router.put("/me", response_model=ProfileRead)
|
||||
async def update_my_profile(
|
||||
body: ProfileUpdate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
) -> Profile:
|
||||
profile = await _get_or_create_profile(current_user, db)
|
||||
|
||||
# Only update fields that were explicitly provided in the request body.
|
||||
for field, value in body.model_dump(exclude_unset=True).items():
|
||||
setattr(profile, field, value)
|
||||
|
||||
await db.commit()
|
||||
await db.refresh(profile)
|
||||
return profile
|
||||
@@ -0,0 +1,22 @@
|
||||
"""
|
||||
GET /api/services — returns health status for all registered feature services.
|
||||
Available to any authenticated user so the frontend can drive app visibility.
|
||||
"""
|
||||
from fastapi import APIRouter, Depends
|
||||
|
||||
from app.deps import get_current_user
|
||||
from app.models.user import User
|
||||
from app.services.service_health import get_all_statuses
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("")
|
||||
async def list_services(_: User = Depends(get_current_user)) -> list[dict]:
|
||||
"""
|
||||
Returns each registered service with its current health status.
|
||||
|
||||
healthy=true → service responded 200 on its last /health poll
|
||||
healthy=false → unreachable, timed out, or not yet polled
|
||||
"""
|
||||
return get_all_statuses()
|
||||
@@ -0,0 +1,314 @@
|
||||
"""
|
||||
Admin-only settings API for per-service runtime configuration.
|
||||
|
||||
All endpoints require the caller to be an admin (Depends(get_current_admin)).
|
||||
Config files are stored in the 'config' bucket of storage-service.
|
||||
"""
|
||||
import re as _re
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core.app_config import (
|
||||
SYSTEM_PROMPT_SERVICES,
|
||||
AppearanceConfig,
|
||||
_merge_api_key,
|
||||
delete_theme,
|
||||
load_ai_service_config,
|
||||
load_ai_service_config_masked,
|
||||
load_all_system_prompts,
|
||||
load_all_themes,
|
||||
load_appearance_config,
|
||||
load_doc_service_config,
|
||||
load_doc_service_config_masked,
|
||||
load_theme_by_id,
|
||||
save_ai_service_config,
|
||||
save_appearance_config,
|
||||
save_doc_service_config,
|
||||
save_service_system_prompts,
|
||||
save_theme,
|
||||
validate_theme_tokens,
|
||||
)
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin, get_current_user, get_service_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_THEME_ID_RE = _re.compile(r"^[a-z0-9_-]{1,64}$")
|
||||
|
||||
|
||||
# ── Pydantic request bodies ────────────────────────────────────────────────────
|
||||
|
||||
|
||||
class AIProviderUpdate(BaseModel):
|
||||
provider: str
|
||||
anthropic_api_key: str = ""
|
||||
anthropic_model: str = ""
|
||||
ollama_base_url: str = ""
|
||||
ollama_model: str = ""
|
||||
ollama_api_key: str = ""
|
||||
lmstudio_base_url: str = ""
|
||||
lmstudio_model: str = ""
|
||||
lmstudio_api_key: str = ""
|
||||
|
||||
|
||||
class LimitsUpdate(BaseModel):
|
||||
max_pdf_mb: int
|
||||
|
||||
|
||||
class SystemPromptUpdate(BaseModel):
|
||||
system: str
|
||||
user_template: str
|
||||
|
||||
|
||||
class AppearanceUpdate(BaseModel):
|
||||
theme: str
|
||||
default_mode: str
|
||||
|
||||
|
||||
class ThemeColors(BaseModel):
|
||||
primary: str
|
||||
primary_hover: str
|
||||
accent: str
|
||||
accent_hover: str
|
||||
background: str
|
||||
surface: str
|
||||
border: str
|
||||
text_primary: str
|
||||
text_muted: str
|
||||
|
||||
|
||||
class ThemeCreate(BaseModel):
|
||||
id: str
|
||||
label: str
|
||||
light: ThemeColors
|
||||
dark: ThemeColors
|
||||
|
||||
|
||||
class ThemeUpdate(BaseModel):
|
||||
label: str | None = None
|
||||
light: ThemeColors | None = None
|
||||
dark: ThemeColors | None = None
|
||||
|
||||
|
||||
# ── AI settings ────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/ai")
|
||||
async def get_ai_settings(
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
return await load_ai_service_config_masked()
|
||||
|
||||
|
||||
@router.patch("/ai")
|
||||
async def update_ai_settings(
|
||||
body: AIProviderUpdate,
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
valid_providers = ("anthropic", "ollama", "lmstudio")
|
||||
if body.provider not in valid_providers:
|
||||
raise HTTPException(status_code=422, detail=f"provider must be one of {valid_providers}")
|
||||
|
||||
config = await load_ai_service_config()
|
||||
config.provider = body.provider
|
||||
|
||||
# Anthropic
|
||||
if body.anthropic_api_key:
|
||||
config.anthropic.api_key = _merge_api_key(
|
||||
body.anthropic_api_key, config.anthropic.api_key
|
||||
)
|
||||
if body.anthropic_model:
|
||||
config.anthropic.model = body.anthropic_model
|
||||
|
||||
# Ollama
|
||||
if body.ollama_base_url:
|
||||
config.ollama.base_url = body.ollama_base_url
|
||||
if body.ollama_model:
|
||||
config.ollama.model = body.ollama_model
|
||||
if body.ollama_api_key:
|
||||
config.ollama.api_key = _merge_api_key(body.ollama_api_key, config.ollama.api_key)
|
||||
|
||||
# LM Studio
|
||||
if body.lmstudio_base_url:
|
||||
config.lmstudio.base_url = body.lmstudio_base_url
|
||||
if body.lmstudio_model:
|
||||
config.lmstudio.model = body.lmstudio_model
|
||||
if body.lmstudio_api_key:
|
||||
config.lmstudio.api_key = _merge_api_key(
|
||||
body.lmstudio_api_key, config.lmstudio.api_key
|
||||
)
|
||||
|
||||
await save_ai_service_config(config)
|
||||
return await load_ai_service_config_masked()
|
||||
|
||||
|
||||
@router.post("/ai/test")
|
||||
async def test_ai_connection(
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
"""Proxy a minimal chat request to ai-service to verify the connection."""
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
f"{settings.AI_SERVICE_URL}/chat",
|
||||
json={
|
||||
"messages": [{"role": "user", "content": "Reply with: ok"}],
|
||||
"max_tokens": 16,
|
||||
"temperature": 0,
|
||||
},
|
||||
)
|
||||
if resp.status_code == 200:
|
||||
data = resp.json()
|
||||
return {"ok": True, "provider": data.get("provider"), "response": data.get("content")}
|
||||
return {"ok": False, "error": f"ai-service returned {resp.status_code}: {resp.text[:200]}"}
|
||||
except Exception as exc:
|
||||
return {"ok": False, "error": str(exc)}
|
||||
|
||||
|
||||
# ── Document limits ────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/documents/limits")
|
||||
async def get_documents_limits(
|
||||
_: User = Depends(get_service_admin("doc-service")),
|
||||
) -> dict:
|
||||
return await load_doc_service_config_masked()
|
||||
|
||||
|
||||
@router.patch("/documents/limits")
|
||||
async def update_documents_limits(
|
||||
body: LimitsUpdate,
|
||||
_: User = Depends(get_service_admin("doc-service")),
|
||||
) -> dict:
|
||||
if body.max_pdf_mb < 1 or body.max_pdf_mb > 200:
|
||||
raise HTTPException(status_code=422, detail="max_pdf_mb must be between 1 and 200")
|
||||
|
||||
config = await load_doc_service_config()
|
||||
config.documents.max_pdf_bytes = body.max_pdf_mb * 1024 * 1024
|
||||
await save_doc_service_config(config)
|
||||
return await load_doc_service_config_masked()
|
||||
|
||||
|
||||
# ── System prompts ─────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/system-prompts")
|
||||
async def get_system_prompts(
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
return await load_all_system_prompts()
|
||||
|
||||
|
||||
@router.patch("/system-prompts/{service_id}")
|
||||
async def update_system_prompt(
|
||||
service_id: str,
|
||||
body: SystemPromptUpdate,
|
||||
_: User = Depends(get_service_admin("ai-service")),
|
||||
) -> dict:
|
||||
if service_id not in SYSTEM_PROMPT_SERVICES:
|
||||
raise HTTPException(status_code=404, detail=f"No system prompts registered for {service_id!r}")
|
||||
await save_service_system_prompts(service_id, body.system, body.user_template)
|
||||
return await load_all_system_prompts()
|
||||
|
||||
|
||||
# ── Appearance (global default — auth read, admin write) ───────────────────────
|
||||
|
||||
|
||||
@router.get("/appearance")
|
||||
async def get_appearance(
|
||||
_: User = Depends(get_current_user),
|
||||
) -> dict:
|
||||
config = await load_appearance_config()
|
||||
return config.model_dump()
|
||||
|
||||
|
||||
@router.patch("/appearance")
|
||||
async def update_appearance(
|
||||
body: AppearanceUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
if body.default_mode not in ("light", "dark", "system"):
|
||||
raise HTTPException(status_code=422, detail="default_mode must be 'light', 'dark', or 'system'")
|
||||
themes = await load_all_themes()
|
||||
theme_ids = {t["id"] for t in themes}
|
||||
if body.theme not in theme_ids:
|
||||
raise HTTPException(status_code=422, detail=f"Unknown theme: {body.theme!r}")
|
||||
config = AppearanceConfig(theme=body.theme, default_mode=body.default_mode)
|
||||
await save_appearance_config(config)
|
||||
return config.model_dump()
|
||||
|
||||
|
||||
# ── Theme CRUD ─────────────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
@router.get("/themes")
|
||||
async def list_themes(
|
||||
_: User = Depends(get_current_user),
|
||||
) -> list:
|
||||
return await load_all_themes()
|
||||
|
||||
|
||||
@router.post("/themes", status_code=201)
|
||||
async def create_theme(
|
||||
body: ThemeCreate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
if not _THEME_ID_RE.match(body.id):
|
||||
raise HTTPException(status_code=422, detail="Theme ID must match [a-z0-9_-]{1,64}")
|
||||
existing = {t["id"] for t in await load_all_themes()}
|
||||
if body.id in existing:
|
||||
raise HTTPException(status_code=400, detail=f"Theme ID already in use: {body.id!r}")
|
||||
light = body.light.model_dump()
|
||||
dark = body.dark.model_dump()
|
||||
for mode, colors in (("light", light), ("dark", dark)):
|
||||
errors = validate_theme_tokens(colors)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"{mode}: {'; '.join(errors)}")
|
||||
theme = {"id": body.id, "label": body.label, "builtin": False, "light": light, "dark": dark}
|
||||
await save_theme(theme)
|
||||
return theme
|
||||
|
||||
|
||||
@router.patch("/themes/{theme_id}")
|
||||
async def update_theme(
|
||||
theme_id: str,
|
||||
body: ThemeUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
theme = await load_theme_by_id(theme_id)
|
||||
if theme is None:
|
||||
raise HTTPException(status_code=404, detail="Theme not found")
|
||||
if theme.get("builtin"):
|
||||
raise HTTPException(status_code=400, detail="Cannot edit a built-in theme")
|
||||
if body.label is not None:
|
||||
theme["label"] = body.label
|
||||
if body.light is not None:
|
||||
light = body.light.model_dump()
|
||||
errors = validate_theme_tokens(light)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"light: {'; '.join(errors)}")
|
||||
theme["light"] = light
|
||||
if body.dark is not None:
|
||||
dark = body.dark.model_dump()
|
||||
errors = validate_theme_tokens(dark)
|
||||
if errors:
|
||||
raise HTTPException(status_code=422, detail=f"dark: {'; '.join(errors)}")
|
||||
theme["dark"] = dark
|
||||
await save_theme(theme)
|
||||
return theme
|
||||
|
||||
|
||||
@router.delete("/themes/{theme_id}", status_code=204)
|
||||
async def remove_theme(
|
||||
theme_id: str,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
try:
|
||||
await delete_theme(theme_id)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="Theme not found")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc))
|
||||
@@ -0,0 +1,126 @@
|
||||
"""
|
||||
Admin-only endpoints for storage-service backend configuration.
|
||||
|
||||
GET /admin/storage-config — current backend driver + health
|
||||
PATCH /admin/storage-config — update backend config (no data migration)
|
||||
POST /admin/storage-config/migrate — start migration to a new backend
|
||||
GET /admin/storage-config/migrate/status — poll migration progress
|
||||
DELETE /admin/storage-config/migrate — cancel in-progress migration
|
||||
|
||||
All endpoints proxy to storage-service:8020.
|
||||
"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_STORAGE_BASE = settings.STORAGE_SERVICE_URL
|
||||
|
||||
|
||||
class BackendConfigUpdate(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
class MigrateRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
def _storage_url(path: str) -> str:
|
||||
return f"{_STORAGE_BASE}{path}"
|
||||
|
||||
|
||||
async def _proxy_get(path: str) -> dict:
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.get(_storage_url(path))
|
||||
if resp.status_code == 404:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config")
|
||||
async def get_storage_config(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Return current backend driver and health status."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url("/health"))
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.patch("/storage-config", status_code=204)
|
||||
async def update_storage_config(
|
||||
body: BackendConfigUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""
|
||||
Reconfigure the active backend without migrating data.
|
||||
Use when changing credentials for the same backend type, or reverting to local.
|
||||
To move data to a new backend, use POST /admin/storage-config/migrate instead.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.patch(
|
||||
_storage_url("/backend-config"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="Migration in progress — cannot reconfigure now")
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
@router.post("/storage-config/migrate", status_code=202)
|
||||
async def start_migration(
|
||||
body: MigrateRequest,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""
|
||||
Start an async migration to a new backend.
|
||||
|
||||
Flow: validate new backend → copy all objects → verify → switch → delete old objects.
|
||||
The old backend stays active until 100% of objects are verified on the new one.
|
||||
Poll GET /admin/storage-config/migrate/status to track progress.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
_storage_url("/migrate"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="A migration is already in progress")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config/migrate/status")
|
||||
async def migration_status(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Poll migration progress. State: idle → validating → migrating → switching → cleaning → done."""
|
||||
return await _proxy_get("/migrate/status")
|
||||
|
||||
|
||||
@router.delete("/storage-config/migrate", status_code=204)
|
||||
async def cancel_migration(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""Cancel a running migration. The old backend remains active."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.delete(_storage_url("/migrate"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="No cancellable migration in progress")
|
||||
resp.raise_for_status()
|
||||
@@ -1,8 +1,13 @@
|
||||
from fastapi import APIRouter, Depends
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
from sqlalchemy.orm import selectinload
|
||||
|
||||
from app.database import get_db
|
||||
from app.deps import get_current_user
|
||||
from app.models.group import Group, GroupMembership
|
||||
from app.models.user import User
|
||||
from app.schemas.user import UserOut
|
||||
from app.schemas.user import ColorModeUpdate, DashboardPrefsOut, DashboardPrefsUpdate, UserGroupOut, UserOut
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -10,3 +15,50 @@ router = APIRouter()
|
||||
@router.get("/me", response_model=UserOut)
|
||||
async def get_me(current_user: User = Depends(get_current_user)):
|
||||
return current_user
|
||||
|
||||
|
||||
@router.get("/me/preferences", response_model=DashboardPrefsOut)
|
||||
async def get_preferences(current_user: User = Depends(get_current_user)):
|
||||
return DashboardPrefsOut(app_ids=current_user.dashboard_app_ids or [])
|
||||
|
||||
|
||||
@router.patch("/me/preferences", response_model=DashboardPrefsOut)
|
||||
async def update_preferences(
|
||||
body: DashboardPrefsUpdate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
current_user.dashboard_app_ids = body.app_ids
|
||||
await db.commit()
|
||||
await db.refresh(current_user)
|
||||
return DashboardPrefsOut(app_ids=current_user.dashboard_app_ids or [])
|
||||
|
||||
|
||||
@router.get("/me/groups", response_model=list[UserGroupOut])
|
||||
async def get_my_groups(
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
"""Return all groups the current user belongs to, including their admin status."""
|
||||
result = await db.execute(
|
||||
select(Group, GroupMembership.is_group_admin)
|
||||
.join(GroupMembership, GroupMembership.group_id == Group.id)
|
||||
.where(GroupMembership.user_id == current_user.id)
|
||||
.order_by(Group.name)
|
||||
)
|
||||
return [
|
||||
UserGroupOut(id=g.id, name=g.name, description=g.description, is_group_admin=is_admin)
|
||||
for g, is_admin in result.all()
|
||||
]
|
||||
|
||||
|
||||
@router.patch("/me/color-mode", response_model=UserOut)
|
||||
async def update_color_mode(
|
||||
body: ColorModeUpdate,
|
||||
current_user: User = Depends(get_current_user),
|
||||
db: AsyncSession = Depends(get_db),
|
||||
):
|
||||
current_user.color_mode = body.color_mode
|
||||
await db.commit()
|
||||
await db.refresh(current_user)
|
||||
return current_user
|
||||
|
||||
@@ -0,0 +1,42 @@
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class GroupCreate(BaseModel):
|
||||
name: str = Field(..., min_length=1, max_length=128)
|
||||
description: str | None = Field(None, max_length=512)
|
||||
|
||||
|
||||
class GroupUpdate(BaseModel):
|
||||
name: str | None = Field(None, min_length=1, max_length=128)
|
||||
description: str | None = Field(None, max_length=512)
|
||||
|
||||
|
||||
class GroupMemberOut(BaseModel):
|
||||
id: str
|
||||
email: str
|
||||
full_name: str | None
|
||||
is_active: bool
|
||||
is_group_admin: bool = False
|
||||
joined_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class GroupMemberAdminUpdate(BaseModel):
|
||||
is_group_admin: bool
|
||||
|
||||
|
||||
class GroupOut(BaseModel):
|
||||
id: str
|
||||
name: str
|
||||
description: str | None
|
||||
created_at: datetime
|
||||
member_count: int = 0
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class GroupDetailOut(GroupOut):
|
||||
members: list[GroupMemberOut] = []
|
||||
@@ -0,0 +1,44 @@
|
||||
from datetime import date, datetime
|
||||
|
||||
from pydantic import BaseModel, field_validator
|
||||
|
||||
from app.core.sanitize import sanitize_str, validate_date_of_birth, validate_phone
|
||||
|
||||
|
||||
class ProfileRead(BaseModel):
|
||||
id: str
|
||||
user_id: str
|
||||
phone: str | None
|
||||
date_of_birth: date | None
|
||||
position: str | None
|
||||
address: str | None
|
||||
updated_at: datetime
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class ProfileUpdate(BaseModel):
|
||||
phone: str | None = None
|
||||
date_of_birth: date | None = None
|
||||
position: str | None = None
|
||||
address: str | None = None
|
||||
|
||||
@field_validator("phone", mode="before")
|
||||
@classmethod
|
||||
def clean_phone(cls, v: str | None) -> str | None:
|
||||
return validate_phone(v)
|
||||
|
||||
@field_validator("position", mode="before")
|
||||
@classmethod
|
||||
def clean_position(cls, v: str | None) -> str | None:
|
||||
return sanitize_str(v, max_len=128)
|
||||
|
||||
@field_validator("address", mode="before")
|
||||
@classmethod
|
||||
def clean_address(cls, v: str | None) -> str | None:
|
||||
return sanitize_str(v, max_len=255)
|
||||
|
||||
@field_validator("date_of_birth", mode="after")
|
||||
@classmethod
|
||||
def clean_dob(cls, v: date | None) -> date | None:
|
||||
return validate_date_of_birth(v)
|
||||
+123
-2
@@ -1,4 +1,46 @@
|
||||
from pydantic import BaseModel, EmailStr
|
||||
import re
|
||||
from datetime import datetime
|
||||
|
||||
from pydantic import BaseModel, EmailStr, Field, field_validator
|
||||
|
||||
from app.core.sanitize import normalize_email, sanitize_str
|
||||
|
||||
# Common words that must not appear as whole words inside a password.
|
||||
# Checked case-insensitively with word boundaries.
|
||||
_FORBIDDEN_WORDS = {
|
||||
"password", "passwort", "secret", "welcome", "admin", "administrator",
|
||||
"login", "user", "test", "guest", "master", "dragon", "monkey", "shadow",
|
||||
"sunshine", "princess", "letmein", "football", "baseball", "soccer",
|
||||
"hockey", "abc", "qwerty", "keyboard", "computer", "internet", "access",
|
||||
"hello", "summer", "winter", "spring", "autumn", "flower", "mustang",
|
||||
"batman", "superman", "donald", "michael", "jessica", "charlie",
|
||||
}
|
||||
|
||||
|
||||
def _validate_password(v: str) -> str:
|
||||
errors = []
|
||||
|
||||
if len(v) < 8:
|
||||
errors.append("at least 8 characters")
|
||||
if not re.search(r"[A-Z]", v):
|
||||
errors.append("at least one uppercase letter")
|
||||
if not re.search(r"[a-z]", v):
|
||||
errors.append("at least one lowercase letter")
|
||||
if not re.search(r"\d", v):
|
||||
errors.append("at least one digit")
|
||||
if not re.search(r'[!@#$%^&*()\-_=+\[\]{};:\'",.<>?/\\|`~]', v):
|
||||
errors.append("at least one special character")
|
||||
|
||||
lower = v.lower()
|
||||
for word in _FORBIDDEN_WORDS:
|
||||
# Match the word as a standalone token (surrounded by non-alpha or string boundary)
|
||||
if re.search(rf"(?<![a-z]){re.escape(word)}(?![a-z])", lower):
|
||||
errors.append(f'must not contain the word "{word}"')
|
||||
break
|
||||
|
||||
if errors:
|
||||
raise ValueError("; ".join(errors))
|
||||
return v
|
||||
|
||||
|
||||
class UserCreate(BaseModel):
|
||||
@@ -6,16 +48,95 @@ class UserCreate(BaseModel):
|
||||
password: str
|
||||
full_name: str | None = None
|
||||
|
||||
@field_validator("email", mode="before")
|
||||
@classmethod
|
||||
def normalize_email_field(cls, v: str) -> str:
|
||||
return normalize_email(v)
|
||||
|
||||
@field_validator("full_name", mode="before")
|
||||
@classmethod
|
||||
def sanitize_full_name(cls, v: str | None) -> str | None:
|
||||
return sanitize_str(v, max_len=128)
|
||||
|
||||
@field_validator("password")
|
||||
@classmethod
|
||||
def password_strength(cls, v: str) -> str:
|
||||
return _validate_password(v)
|
||||
|
||||
|
||||
class UserOut(BaseModel):
|
||||
id: str
|
||||
email: str
|
||||
full_name: str | None
|
||||
is_active: bool
|
||||
# validation_alias reads is_superuser from the ORM object; the JSON key
|
||||
# in the response is the field name "is_admin" (not the alias).
|
||||
is_admin: bool = Field(validation_alias="is_superuser", default=False)
|
||||
color_mode: str | None = None
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
model_config = {"from_attributes": True, "populate_by_name": True}
|
||||
|
||||
|
||||
# ── Admin-facing schemas ───────────────────────────────────────────────────────
|
||||
|
||||
class UserAdminOut(BaseModel):
|
||||
"""Full user record returned to admin endpoints."""
|
||||
id: str
|
||||
email: str
|
||||
full_name: str | None
|
||||
is_active: bool
|
||||
is_admin: bool = Field(validation_alias="is_superuser", default=False)
|
||||
|
||||
model_config = {"from_attributes": True, "populate_by_name": True}
|
||||
|
||||
|
||||
class UserAdminCreate(UserCreate):
|
||||
"""Admin creates a user and can optionally grant admin rights."""
|
||||
is_admin: bool = False
|
||||
|
||||
|
||||
class Token(BaseModel):
|
||||
access_token: str
|
||||
token_type: str = "bearer"
|
||||
|
||||
|
||||
# ── Dashboard preferences ──────────────────────────────────────────────────────
|
||||
|
||||
class DashboardPrefsOut(BaseModel):
|
||||
app_ids: list[str]
|
||||
|
||||
|
||||
class ColorModeUpdate(BaseModel):
|
||||
color_mode: str
|
||||
|
||||
@field_validator("color_mode")
|
||||
@classmethod
|
||||
def validate_mode(cls, v: str) -> str:
|
||||
if v not in ("light", "dark", "system"):
|
||||
raise ValueError("color_mode must be 'light', 'dark', or 'system'")
|
||||
return v
|
||||
|
||||
|
||||
class UserGroupOut(BaseModel):
|
||||
"""A group the current user belongs to — used for the share picker."""
|
||||
id: str
|
||||
name: str
|
||||
description: str | None
|
||||
is_group_admin: bool = False
|
||||
|
||||
model_config = {"from_attributes": True}
|
||||
|
||||
|
||||
class DashboardPrefsUpdate(BaseModel):
|
||||
app_ids: list[str] = Field(default_factory=list)
|
||||
|
||||
@field_validator("app_ids")
|
||||
@classmethod
|
||||
def validate_app_ids(cls, v: list[str]) -> list[str]:
|
||||
if len(v) > 50:
|
||||
raise ValueError("Cannot pin more than 50 apps")
|
||||
for item in v:
|
||||
# Service IDs are alphanumeric slugs or UUIDs — no HTML/script allowed.
|
||||
if not re.match(r'^[a-zA-Z0-9_\-]{1,64}$', item):
|
||||
raise ValueError(f"Invalid app ID: {item!r}")
|
||||
return v
|
||||
|
||||
@@ -0,0 +1,37 @@
|
||||
"""
|
||||
Ensure that every registered service has a corresponding admin group.
|
||||
|
||||
Called once at startup after register_services(). Idempotent — safe to run
|
||||
on every restart, creates nothing if groups already exist.
|
||||
|
||||
Naming convention: "{service_id}-admin" (e.g. "doc-service-admin")
|
||||
"""
|
||||
import logging
|
||||
|
||||
from sqlalchemy import select
|
||||
from sqlalchemy.ext.asyncio import AsyncSession
|
||||
|
||||
from app.models.group import Group
|
||||
from app.services.service_health import get_registry
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
|
||||
async def ensure_service_admin_groups(db: AsyncSession) -> None:
|
||||
"""Create a <service-id>-admin group for each registered service if absent."""
|
||||
for svc in get_registry():
|
||||
group_name = f"{svc.id}-admin"
|
||||
result = await db.execute(select(Group).where(Group.name == group_name))
|
||||
if result.scalar_one_or_none() is not None:
|
||||
continue
|
||||
|
||||
import uuid
|
||||
group = Group(
|
||||
id=str(uuid.uuid4()),
|
||||
name=group_name,
|
||||
description=f"Administrators for the {svc.name} service.",
|
||||
)
|
||||
db.add(group)
|
||||
logger.info("[bootstrap] Created admin group %r for service %r", group_name, svc.id)
|
||||
|
||||
await db.commit()
|
||||
@@ -0,0 +1,178 @@
|
||||
"""
|
||||
Background health-checker for registered feature services.
|
||||
|
||||
Polls each service's /health endpoint every POLL_INTERVAL seconds and stores
|
||||
the result in an in-memory dict. Also fetches /plugin/manifest when available
|
||||
and caches it so the plugin proxy can serve it without per-request network calls.
|
||||
The REST layer reads from that dict — no DB, no blocking calls on the request path.
|
||||
"""
|
||||
import asyncio
|
||||
import logging
|
||||
from dataclasses import dataclass
|
||||
|
||||
import httpx
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
POLL_INTERVAL = 30 # seconds
|
||||
|
||||
|
||||
@dataclass
|
||||
class ServiceDefinition:
|
||||
id: str
|
||||
name: str
|
||||
description: str
|
||||
internal_url: str # e.g. http://doc-service:8001
|
||||
health_path: str = "/health"
|
||||
app_path: str = "" # frontend route; empty = no open button
|
||||
settings_path: str = "" # frontend admin-settings route
|
||||
|
||||
|
||||
# ── Registry ──────────────────────────────────────────────────────────────────
|
||||
# Add new services here. The internal_url is filled in at startup from settings.
|
||||
|
||||
_REGISTRY: list[ServiceDefinition] = []
|
||||
|
||||
# id → True/False/None (None = not yet checked)
|
||||
_health: dict[str, bool | None] = {}
|
||||
|
||||
# id → plugin manifest dict, or None if the service has no plugin manifest
|
||||
_manifests: dict[str, dict | None] = {}
|
||||
|
||||
|
||||
def register_services(doc_service_url: str, ai_service_url: str, storage_service_url: str) -> None:
|
||||
"""Called once during app startup to populate the registry from config."""
|
||||
global _REGISTRY, _health, _manifests
|
||||
|
||||
_REGISTRY = [
|
||||
ServiceDefinition(
|
||||
id="doc-service",
|
||||
name="Documents",
|
||||
description="Upload PDF files, extract data, and organise them with categories.",
|
||||
internal_url=doc_service_url,
|
||||
health_path="/health",
|
||||
app_path="/apps/documents",
|
||||
settings_path="/apps/documents/settings",
|
||||
),
|
||||
ServiceDefinition(
|
||||
id="ai-service",
|
||||
name="AI Service",
|
||||
description="Shared AI provider for all features. Configure model, credentials, and connection.",
|
||||
internal_url=ai_service_url,
|
||||
health_path="/health",
|
||||
app_path="",
|
||||
settings_path="/apps/ai/settings",
|
||||
),
|
||||
ServiceDefinition(
|
||||
id="storage-service",
|
||||
name="Storage",
|
||||
description="Unified file storage. Manages all uploaded files with pluggable backends (local, S3, WebDAV).",
|
||||
internal_url=storage_service_url,
|
||||
health_path="/health",
|
||||
app_path="",
|
||||
settings_path="/admin/storage",
|
||||
),
|
||||
]
|
||||
|
||||
_health = {svc.id: None for svc in _REGISTRY}
|
||||
_manifests = {svc.id: None for svc in _REGISTRY}
|
||||
logger.info("Service registry initialised with %d services", len(_REGISTRY))
|
||||
|
||||
|
||||
# ── Health check logic ────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
async def _check_service(svc: ServiceDefinition) -> None:
|
||||
url = f"{svc.internal_url}{svc.health_path}"
|
||||
prev = _health.get(svc.id)
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
resp = await client.get(url)
|
||||
healthy = resp.status_code == 200
|
||||
except Exception as exc:
|
||||
logger.debug("Health check failed for %s: %s", svc.id, exc)
|
||||
healthy = False
|
||||
|
||||
_health[svc.id] = healthy
|
||||
|
||||
# Log only on transitions so the logs stay quiet during normal operation
|
||||
if prev != healthy:
|
||||
if healthy:
|
||||
logger.info("Service %s is now HEALTHY", svc.id)
|
||||
else:
|
||||
logger.warning("Service %s is now UNHEALTHY", svc.id)
|
||||
|
||||
# Opportunistically fetch plugin manifest when the service is healthy
|
||||
if healthy:
|
||||
await _fetch_manifest(svc)
|
||||
|
||||
|
||||
async def _fetch_manifest(svc: ServiceDefinition) -> None:
|
||||
"""Try to GET /plugin/manifest from the service; cache result (or None)."""
|
||||
url = f"{svc.internal_url}/plugin/manifest"
|
||||
try:
|
||||
async with httpx.AsyncClient(timeout=5.0) as client:
|
||||
resp = await client.get(url)
|
||||
if resp.status_code == 200:
|
||||
_manifests[svc.id] = resp.json()
|
||||
else:
|
||||
_manifests[svc.id] = None
|
||||
except Exception:
|
||||
# Service doesn't have a plugin manifest — not an error
|
||||
_manifests[svc.id] = None
|
||||
|
||||
|
||||
async def check_all() -> None:
|
||||
"""Run health checks for all registered services concurrently."""
|
||||
await asyncio.gather(*[_check_service(svc) for svc in _REGISTRY])
|
||||
|
||||
|
||||
async def health_check_loop() -> None:
|
||||
"""Runs forever; polls every POLL_INTERVAL seconds.
|
||||
|
||||
Exceptions inside a single polling round are caught so the loop cannot
|
||||
be killed by a transient error.
|
||||
"""
|
||||
while True:
|
||||
try:
|
||||
await check_all()
|
||||
except Exception:
|
||||
logger.exception("Unexpected error during health check round; will retry")
|
||||
await asyncio.sleep(POLL_INTERVAL)
|
||||
|
||||
|
||||
# ── Public read API ───────────────────────────────────────────────────────────
|
||||
|
||||
|
||||
def get_all_statuses() -> list[dict]:
|
||||
"""Return the current health snapshot for all registered services."""
|
||||
return [
|
||||
{
|
||||
"id": svc.id,
|
||||
"name": svc.name,
|
||||
"description": svc.description,
|
||||
"app_path": svc.app_path,
|
||||
"settings_path": svc.settings_path,
|
||||
# None means not yet checked; treat as unhealthy for the UI
|
||||
"healthy": bool(_health.get(svc.id)),
|
||||
}
|
||||
for svc in _REGISTRY
|
||||
]
|
||||
|
||||
|
||||
def get_cached_manifest(service_id: str) -> dict | None:
|
||||
"""Return the cached plugin manifest for a service, or None if unavailable."""
|
||||
return _manifests.get(service_id)
|
||||
|
||||
|
||||
def get_service_url(service_id: str) -> str | None:
|
||||
"""Return the internal URL for a registered service, or None if unknown."""
|
||||
for svc in _REGISTRY:
|
||||
if svc.id == service_id:
|
||||
return svc.internal_url
|
||||
return None
|
||||
|
||||
|
||||
def get_registry() -> list[ServiceDefinition]:
|
||||
"""Return the current service registry (always up-to-date after register_services)."""
|
||||
return _REGISTRY
|
||||
@@ -1,6 +1,6 @@
|
||||
[build-system]
|
||||
requires = ["setuptools"]
|
||||
build-backend = "setuptools.backends.legacy:build"
|
||||
requires = ["setuptools>=45"]
|
||||
build-backend = "setuptools.build_meta"
|
||||
|
||||
[project]
|
||||
name = "destroying_sap"
|
||||
@@ -15,15 +15,17 @@ dependencies = [
|
||||
"pydantic[email]>=2.7",
|
||||
"pydantic-settings>=2.2",
|
||||
"python-jose[cryptography]>=3.3",
|
||||
"passlib[bcrypt]>=1.7",
|
||||
"bcrypt>=4.0",
|
||||
"python-multipart>=0.0.9",
|
||||
"httpx>=0.27",
|
||||
"anthropic>=0.28",
|
||||
"openai>=1.0",
|
||||
]
|
||||
|
||||
[project.optional-dependencies]
|
||||
dev = [
|
||||
"pytest>=8",
|
||||
"pytest-asyncio>=0.23",
|
||||
"httpx>=0.27",
|
||||
"ruff>=0.4",
|
||||
]
|
||||
|
||||
|
||||
@@ -0,0 +1,74 @@
|
||||
"""Seed the dev environment with a fixed set of test users.
|
||||
|
||||
Users are upserted on every startup — missing ones are created, existing ones
|
||||
are left untouched except for the admin flag which is always enforced.
|
||||
"""
|
||||
|
||||
import asyncio
|
||||
|
||||
from sqlalchemy import select
|
||||
|
||||
from app.core.security import hash_password
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.models.user import User
|
||||
|
||||
# ── Dev seed users ────────────────────────────────────────────────────────────
|
||||
# Passwords satisfy the strength policy (upper, lower, digit, special char,
|
||||
# no forbidden words) so they can also be used via the API if needed.
|
||||
|
||||
SEED_USERS = [
|
||||
{
|
||||
"email": "test_admin@example.com",
|
||||
"password": "Secure_Dev1!",
|
||||
"full_name": "Test Admin",
|
||||
"is_superuser": True,
|
||||
},
|
||||
{
|
||||
"email": "test_1@example.com",
|
||||
"password": "Secure_Dev2!",
|
||||
"full_name": "Test User One",
|
||||
"is_superuser": False,
|
||||
},
|
||||
{
|
||||
"email": "test_2@example.com",
|
||||
"password": "Secure_Dev3!",
|
||||
"full_name": "Test User Two",
|
||||
"is_superuser": False,
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
async def seed() -> None:
|
||||
async with AsyncSessionLocal() as db:
|
||||
for spec in SEED_USERS:
|
||||
result = await db.execute(
|
||||
select(User).where(User.email == spec["email"])
|
||||
)
|
||||
existing = result.scalar_one_or_none()
|
||||
|
||||
if existing:
|
||||
# Always enforce the correct admin flag in case it drifted
|
||||
if existing.is_superuser != spec["is_superuser"]:
|
||||
existing.is_superuser = spec["is_superuser"]
|
||||
await db.commit()
|
||||
flag = "admin" if spec["is_superuser"] else "user"
|
||||
print(f"[seed] updated role → {flag}: {spec['email']}")
|
||||
else:
|
||||
print(f"[seed] already exists: {spec['email']}")
|
||||
else:
|
||||
user = User(
|
||||
email=spec["email"],
|
||||
hashed_password=hash_password(spec["password"]),
|
||||
full_name=spec["full_name"],
|
||||
is_superuser=spec["is_superuser"],
|
||||
)
|
||||
db.add(user)
|
||||
await db.commit()
|
||||
role = "admin" if spec["is_superuser"] else "user"
|
||||
print(
|
||||
f"[seed] created {role}: {spec['email']} pwd: {spec['password']}"
|
||||
)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
asyncio.run(seed())
|
||||
@@ -0,0 +1,8 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "[start] running migrations..."
|
||||
alembic upgrade head
|
||||
|
||||
echo "[start] starting uvicorn..."
|
||||
exec uvicorn app.main:app --host 0.0.0.0 --port 8000
|
||||
Executable
+11
@@ -0,0 +1,11 @@
|
||||
#!/bin/sh
|
||||
set -e
|
||||
|
||||
echo "[start] running migrations..."
|
||||
alembic upgrade head
|
||||
|
||||
echo "[start] seeding dev data..."
|
||||
python -m scripts.seed
|
||||
|
||||
echo "[start] starting uvicorn..."
|
||||
exec uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
@@ -0,0 +1,23 @@
|
||||
# 2026-04-12 — Fix Docker build issues
|
||||
|
||||
**Timestamp:** 2026-04-12T13:40:00
|
||||
|
||||
## Summary
|
||||
|
||||
Resolved three Docker build failures preventing the dev stack from starting. All three containers (db, backend, frontend) now build and run successfully.
|
||||
|
||||
## Issues Fixed
|
||||
|
||||
1. **No `package-lock.json`** — `npm ci` requires a lockfile. Generated `frontend/package-lock.json` inside a throwaway `node:20-alpine` container (no host installation).
|
||||
2. **BuildKit DNS failure** — Docker BuildKit build steps had no DNS resolution, causing pip to fail on PyPI connections. Fixed by adding `network: host` to both `backend` and `frontend` build configs in `docker-compose.yml`.
|
||||
3. **`setuptools.backends.legacy` unavailable** — The `pyproject.toml` referenced a setuptools 68+ only backend path. Changed to the stable `setuptools.build_meta` backend (available since setuptools 40).
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — added `network: host` to `backend` and `frontend` build sections
|
||||
- `backend/pyproject.toml` — changed build backend from `setuptools.backends.legacy:build` to `setuptools.build_meta`; bumped setuptools requirement to `>=45`
|
||||
- `backend/Dockerfile` — reverted intermediate `--no-build-isolation` workaround (no longer needed)
|
||||
|
||||
## Files Added
|
||||
|
||||
- `frontend/package-lock.json` — npm lockfile required by `npm ci` in the frontend Dockerfile
|
||||
@@ -0,0 +1,19 @@
|
||||
# 2026-04-12 — Dockerize all containers
|
||||
|
||||
**Timestamp:** 2026-04-12T15:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added proper Dockerfiles for backend and frontend. Split compose setup into production (`docker-compose.yml`) and development (`docker-compose.dev.yml`) modes. Updated README to reflect new container architecture.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/Dockerfile` — multi-stage build: pip install in builder stage, minimal python:3.12-slim runtime; runs uvicorn in production mode
|
||||
- `frontend/Dockerfile` — multi-stage build: Node 20 Alpine builds the Vite bundle, nginx:alpine serves the static files
|
||||
- `frontend/nginx.conf` — nginx config for SPA fallback routing and `/api/` reverse proxy to the backend container
|
||||
- `docker-compose.dev.yml` — development overrides: mounts source for hot reload, uses Vite dev server instead of nginx
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — rewritten: proper `build.context` + `dockerfile` references, health check on db, `depends_on` with `condition: service_healthy`, env var interpolation via `${VAR:-default}`, frontend now served on port 80 via nginx
|
||||
- `README.md` — updated Current State, added Containers table, replaced install options with Production / Development / Local sections
|
||||
@@ -0,0 +1,63 @@
|
||||
# 2026-04-12 — Initial project scaffold
|
||||
|
||||
**Timestamp:** 2026-04-12T14:22:00
|
||||
|
||||
## Summary
|
||||
|
||||
Created the initial fullstack SaaS project structure from scratch.
|
||||
|
||||
## Files Added
|
||||
|
||||
### Backend
|
||||
- `backend/pyproject.toml` — project dependencies (FastAPI, SQLAlchemy, Alembic, pydantic-settings, python-jose, passlib)
|
||||
- `backend/alembic.ini` — Alembic configuration
|
||||
- `backend/alembic/env.py` — async Alembic migration environment
|
||||
- `backend/alembic/script.py.mako` — migration file template
|
||||
- `backend/app/__init__.py`
|
||||
- `backend/app/main.py` — FastAPI app, CORS middleware, router mounts
|
||||
- `backend/app/database.py` — async SQLAlchemy engine, session factory, `Base`
|
||||
- `backend/app/deps.py` — `get_current_user` FastAPI dependency (JWT validation)
|
||||
- `backend/app/core/__init__.py`
|
||||
- `backend/app/core/config.py` — `Settings` via pydantic-settings, reads `.env`
|
||||
- `backend/app/core/security.py` — bcrypt password hashing, JWT encode/decode
|
||||
- `backend/app/models/__init__.py`
|
||||
- `backend/app/models/user.py` — `User` ORM model (id, email, hashed_password, full_name, is_active, is_superuser)
|
||||
- `backend/app/schemas/__init__.py`
|
||||
- `backend/app/schemas/user.py` — `UserCreate`, `UserOut`, `Token` Pydantic schemas
|
||||
- `backend/app/routers/__init__.py`
|
||||
- `backend/app/routers/auth.py` — `POST /api/auth/register`, `POST /api/auth/login`
|
||||
- `backend/app/routers/users.py` — `GET /api/users/me`
|
||||
|
||||
### Frontend
|
||||
- `frontend/package.json` — dependencies (React 18, TypeScript, Vite, React Router v6, TanStack Query, Axios)
|
||||
- `frontend/vite.config.ts` — Vite config, `/api` proxy to `:8000`
|
||||
- `frontend/tsconfig.json` — strict TypeScript config
|
||||
- `frontend/index.html`
|
||||
- `frontend/src/main.tsx` — React root, QueryClientProvider, BrowserRouter
|
||||
- `frontend/src/App.tsx` — route tree, `PrivateRoute` guard
|
||||
- `frontend/src/api/client.ts` — Axios instance, auth interceptor, `login`, `register`, `getMe`
|
||||
- `frontend/src/hooks/useAuth.ts` — JWT token state, `login`, `logout`
|
||||
- `frontend/src/pages/LoginPage.tsx`
|
||||
- `frontend/src/pages/RegisterPage.tsx`
|
||||
- `frontend/src/pages/DashboardPage.tsx`
|
||||
|
||||
### Root
|
||||
- `docker-compose.yml` — postgres, backend, frontend services
|
||||
- `.env.example`
|
||||
- `.gitignore`
|
||||
- `CLAUDE.md`
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-12 — Added README and changelog
|
||||
|
||||
**Timestamp:** 2026-04-12T14:45:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added project README with overview and installation guide. Created `changelog/` directory and this initial entry.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `README.md` — project overview, stack table, current state, Option A (Docker) and Option B (local) install guides, env variable reference, dev commands
|
||||
- `changelog/2026-04-12_initial-scaffold.md` — this file
|
||||
@@ -0,0 +1,25 @@
|
||||
# 2026-04-12 — Fix Vite proxy and add success pages
|
||||
|
||||
**Timestamp:** 2026-04-12T16:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Fixed login/registration failures caused by wrong Vite proxy target inside Docker. Added login and registration success pages. Improved error messages to show actual API responses.
|
||||
|
||||
## Root Cause
|
||||
|
||||
Vite's dev server proxy was targeting `http://localhost:8000`. Inside the Docker network, `localhost` resolves to the frontend container itself (not the backend). The correct target inside Docker is `http://backend:8000` (Docker service name).
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `frontend/vite.config.ts` — proxy target now reads from `process.env.VITE_API_TARGET`, falling back to `http://localhost:8000` for local dev without Docker
|
||||
- `docker-compose.dev.yml` — added `VITE_API_TARGET: http://backend:8000` to frontend environment
|
||||
- `frontend/src/App.tsx` — added `/login-success` and `/register-success` routes
|
||||
- `frontend/src/hooks/useAuth.ts` — redirects to `/login-success` after login
|
||||
- `frontend/src/pages/LoginPage.tsx` — parses real API error from response instead of generic message
|
||||
- `frontend/src/pages/RegisterPage.tsx` — redirects to `/register-success` on success; shows real API validation errors
|
||||
|
||||
## Files Added
|
||||
|
||||
- `frontend/src/pages/LoginSuccessPage.tsx` — placeholder: "Login successful"
|
||||
- `frontend/src/pages/RegisterSuccessPage.tsx` — placeholder: "Registration successful" with link to sign in
|
||||
@@ -0,0 +1,22 @@
|
||||
# 2026-04-12 — Test user, password validation, security hook
|
||||
|
||||
**Timestamp:** 2026-04-12T14:10:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added dev seed user, password strength validation, and a Docker-based pre-commit security check hook.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/scripts/seed.py` — async script that creates `test@example.com / Test123!` if it doesn't exist; safe to run multiple times
|
||||
- `backend/scripts/start_dev.sh` — dev container entrypoint: runs `alembic upgrade head` → seed → uvicorn --reload
|
||||
- `scripts/security_check.py` — security scanner: checks staged files for hardcoded secrets, dangerous patterns (eval/exec/shell=True/pickle), weak crypto (MD5/SHA1/DES), SQL injection risk, debug flags; also runs `bandit` on Python files
|
||||
- `.githooks/pre-commit` — git hook that runs `security_check.py` inside `python:3.12-slim` Docker container; activated via `git config core.hooksPath .githooks`
|
||||
- `changelog/2026-04-12_security-validation.md` — this file
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/schemas/user.py` — added `_validate_password` with: min 8 chars, uppercase, lowercase, digit, special char, word-boundary check against ~40 forbidden common words; `UserCreate.password_strength` field validator
|
||||
- `docker-compose.dev.yml` — backend command changed from bare `uvicorn` to `sh scripts/start_dev.sh`
|
||||
- `CLAUDE.md` — added Security hook section documenting what the hook checks and how to activate it on new clones
|
||||
- `README.md` — updated Current State to mention test user, password policy, security hook
|
||||
@@ -0,0 +1,35 @@
|
||||
# 2026-04-12 — Troubleshoot dev stack startup
|
||||
|
||||
**Timestamp:** 2026-04-12T15:30:00
|
||||
|
||||
## Summary
|
||||
|
||||
Fixed three startup failures discovered when running the dev stack for the first time.
|
||||
|
||||
## Issues Fixed
|
||||
|
||||
1. **`ModuleNotFoundError: No module named 'app'`** — `python scripts/seed.py` does not add the working directory to `sys.path`. Changed to `python -m scripts.seed` which uses module mode and adds `/app` to the path. Added `backend/scripts/__init__.py` to make the directory a package.
|
||||
|
||||
2. **`relation "users" does not exist`** — No Alembic migration files existed, so `alembic upgrade head` was a no-op. Generated the initial migration (`38efeff7c45a_create_users_table.py`) by running `alembic revision --autogenerate` inside the backend container against the live database.
|
||||
|
||||
3. **`ValueError: password cannot be longer than 72 bytes` (passlib + bcrypt 4.x incompatibility)** — `passlib` is unmaintained and its bcrypt wrap-bug detection raises an exception against bcrypt 4.x. Replaced `passlib[bcrypt]` with direct `bcrypt>=4.0` usage in both `pyproject.toml` and `app/core/security.py`.
|
||||
|
||||
## Smoke Test Results (all passing)
|
||||
|
||||
- `GET /api/health` → `{"status":"ok"}`
|
||||
- `POST /api/auth/login` → JWT token issued for `test@example.com`
|
||||
- `GET /api/users/me` → returns user profile with valid token
|
||||
- `POST /api/auth/register` → new user created successfully
|
||||
- Weak password (`"password"`) → rejected with detailed validation errors
|
||||
- Frontend `http://localhost:5173` → HTTP 200
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/scripts/__init__.py` — makes scripts/ a Python package
|
||||
- `backend/alembic/versions/38efeff7c45a_create_users_table.py` — initial migration
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/scripts/start_dev.sh` — `python scripts/seed.py` → `python -m scripts.seed`
|
||||
- `backend/pyproject.toml` — `passlib[bcrypt]>=1.7` → `bcrypt>=4.0`
|
||||
- `backend/app/core/security.py` — replaced passlib with direct bcrypt calls
|
||||
@@ -0,0 +1,14 @@
|
||||
# 2026-04-13 — JWT token expiry hardened to 8 hours
|
||||
|
||||
**Timestamp:** 2026-04-13T04:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Reduced JWT token lifetime from 24 hours to 8 hours with no permanent session option. Added JWT vulnerability detection to the pre-commit security check and a JWT security checklist to the security-auditor agent. Updated TODO with auth/session security items.
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/core/config.py` — `ACCESS_TOKEN_EXPIRE_MINUTES` changed from `60 * 24` to `60 * 8`; added comment "no permanent sessions"
|
||||
- `scripts/security_check.py` — added `JWT_PATTERNS` category: algorithm confusion (`none`), disabled expiry verification, multi-day token lifetime, oversized EXPIRE_MINUTES, hardcoded secret; wired into `ALL_PATTERNS` and updated docstring
|
||||
- `.claude/agents/security-auditor.md` — added JWT security checklist table covering algorithm confusion, expiry enforcement, token lifetime, secret key strength, missing claims, localStorage storage, no refresh tokens policy
|
||||
- `TODO.md` — added "Auth / session security" section: 8-hour JWT checked off, refresh token and httpOnly cookie migration as future items
|
||||
@@ -0,0 +1,20 @@
|
||||
# 2026-04-13 — Switch UX/UI tooling to self-hosted Penpot
|
||||
|
||||
**Timestamp:** 2026-04-13T03:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Decided to use self-hosted Penpot instead of Figma for UX/UI design work. Penpot will run in a dedicated LXC container on the user's server; the `ux-designer` agent connects to it via the Penpot REST API using WebFetch — no MCP server required. Setup is pending for the next session.
|
||||
|
||||
## Decision rationale
|
||||
|
||||
- Full self-hosting control, no SaaS dependency or monthly cost
|
||||
- Penpot REST API is directly accessible via WebFetch (no MCP server needed)
|
||||
- User is experienced with self-hosting Docker/LXC infrastructure
|
||||
- Open-source (MPL 2.0), actively maintained
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `.claude/agents/ux-designer.md` — replaced Figma MCP instructions with Penpot REST API setup guide; added ⚠ next-session checklist with all steps to complete before UX work can begin
|
||||
- `TODO.md` — added UX/UI Penpot setup section with five actionable items
|
||||
- `changelog/2026-04-13_penpot-decision.md` — this file
|
||||
@@ -0,0 +1,30 @@
|
||||
# 2026-04-13 — Profile feature + input sanitization
|
||||
|
||||
**Timestamp:** 2026-04-13T02:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added shared input sanitization layer applied to all database-bound inputs, introduced the `profiles` table for personal information (position, phone, date of birth, address), and built the frontend profile page with inline editing and a shared nav bar (Dashboard | Profile | Logout). Admin role flag (`is_superuser`) confirmed hidden from API. Security check patterns strengthened.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/app/core/sanitize.py` — shared helpers: `sanitize_str`, `normalize_email`, `validate_phone`, `validate_date_of_birth`; applied to every user-supplied string before it reaches the DB
|
||||
- `backend/app/models/profile.py` — `Profile` ORM model (profiles table): `user_id` FK, `phone`, `date_of_birth`, `position`, `address`, `updated_at`
|
||||
- `backend/app/schemas/profile.py` — `ProfileRead` / `ProfileUpdate` Pydantic schemas; all fields sanitized via shared helpers
|
||||
- `backend/app/routers/profile.py` — `GET /api/profile/me` (lazy-create), `PUT /api/profile/me`
|
||||
- `backend/alembic/versions/676084df61d1_add_profiles_table.py` — Alembic migration creating the profiles table
|
||||
- `frontend/src/components/Nav.tsx` — shared nav bar: Dashboard, Profile, Logout
|
||||
- `frontend/src/pages/ProfilePage.tsx` — profile view + inline edit form; uses TanStack Query for fetch/mutate
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/schemas/user.py` — added `normalize_email` and `sanitize_str` validators to `UserCreate`
|
||||
- `backend/app/models/user.py` — added `Profile` back-reference; added admin-role comment on `is_superuser`
|
||||
- `backend/app/models/__init__.py` — export `Profile`
|
||||
- `backend/app/main.py` — register `/api/profile` router
|
||||
- `frontend/src/api/client.ts` — added `ProfileData`, `ProfileUpdate` types, `getProfile`, `updateProfile`
|
||||
- `frontend/src/App.tsx` — added `/profile` private route
|
||||
- `frontend/src/pages/DashboardPage.tsx` — replaced inline logout with `Nav` component
|
||||
- `scripts/security_check.py` — strengthened SQL injection patterns (f-string/format/% in execute, text() without bindparam); added SANIT category for raw request→DB patterns
|
||||
- `TODO.md` — frontend feature items marked complete
|
||||
- `README.md` — Current State updated
|
||||
@@ -0,0 +1,42 @@
|
||||
# 2026-04-13 — Rootless containers
|
||||
|
||||
**Timestamp:** 2026-04-13T00:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
All containers now run as non-root users with explicit UID:GID assignments enforced in both Dockerfiles and docker-compose files.
|
||||
|
||||
## User mapping
|
||||
|
||||
| Service | User | UID:GID | Notes |
|
||||
|---|---|---|---|
|
||||
| `db` | `postgres` | `70:70` | Fixed by `postgres:16-alpine`; image owns PGDATA as 70:70 so named-volume seeding works |
|
||||
| `backend` | `appuser` | `1001:1001` | Created via `useradd` in runtime stage |
|
||||
| `frontend` (prod) | `nginx` | `101:101` | Switched to `nginxinc/nginx-unprivileged:alpine`; listens on 8080 |
|
||||
| `frontend` (dev) | `appuser` | `1001:1001` | Created via `adduser` in builder stage |
|
||||
|
||||
# 2026-04-13 — Frontend prod UID 1001, infra change protocol, README update
|
||||
|
||||
**Timestamp:** 2026-04-13T01:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Aligned frontend prod container to UID 1001 (same as all other app containers), added infrastructure change protocol to CLAUDE.md, updated README with container table and rootless note. Both dev and prod stacks verified working.
|
||||
|
||||
## Files Modified (this entry)
|
||||
|
||||
- `frontend/Dockerfile` — prod stage: added `USER root` + `addgroup`/`adduser` for appuser 1001:1001, `USER appuser`; removed stale 101 reference
|
||||
- `docker-compose.yml` — frontend `user:` updated from `"101:101"` to `"1001:1001"`
|
||||
- `CLAUDE.md` — added Infrastructure change protocol section; fixed stale passlib reference in stack table
|
||||
- `README.md` — updated container table with `nginxinc/nginx-unprivileged:alpine`, UID columns, internal port note; added rootless note to Current State
|
||||
|
||||
---
|
||||
|
||||
## Files Modified (previous entry)
|
||||
|
||||
- `backend/Dockerfile` — added `groupadd`/`useradd` for appuser (1001:1001), `--chown` on all `COPY` directives, `USER appuser`
|
||||
- `frontend/Dockerfile` — builder stage: added `addgroup`/`adduser` for appuser (1001:1001), `USER appuser`; prod stage: switched to `nginxinc/nginx-unprivileged:alpine`, `EXPOSE 8080`
|
||||
- `frontend/nginx.conf` — changed `listen 80` → `listen 8080` to match unprivileged image default
|
||||
- `docker-compose.yml` — added `user: "70:70"` to `db`, `user: "1001:1001"` to `backend`, `user: "101:101"` to `frontend`; updated frontend port mapping to `"80:8080"`
|
||||
- `docker-compose.dev.yml` — added `user: "1001:1001"` to `backend` and `frontend` overrides
|
||||
- `TODO.md` — marked rootless containers item as completed
|
||||
@@ -0,0 +1,20 @@
|
||||
# 2026-04-13 — Switch JWT signing to RS256 (4096-bit RSA)
|
||||
|
||||
**Timestamp:** 2026-04-13T05:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Replaced symmetric HS256 JWT signing with asymmetric RS256 using a 4096-bit RSA key pair. The private key signs tokens; the public key verifies them. Added `iat` (issued-at) claim to every token and a key-generation helper script.
|
||||
|
||||
## Motivation
|
||||
|
||||
HS256 uses the same secret for signing and verification — if the key leaks, an attacker can forge arbitrary tokens. RS256 with a 4096-bit key eliminates this: the private key never leaves the backend process, and the public key can be distributed safely.
|
||||
|
||||
## Files Added / Modified
|
||||
|
||||
- `scripts/generate_jwt_keys.py` — generates a 4096-bit RSA key pair; outputs single-line PEM values ready to paste into `backend/.env`
|
||||
- `backend/app/core/config.py` — replaced `SECRET_KEY` / `ALGORITHM=HS256` with `JWT_PRIVATE_KEY`, `JWT_PUBLIC_KEY`, `ALGORITHM=RS256`; added `expand_newlines` validator to handle `\n`-escaped PEM in `.env`
|
||||
- `backend/app/core/security.py` — `create_access_token` now signs with `JWT_PRIVATE_KEY` and includes `iat` claim; `decode_access_token` verifies with `JWT_PUBLIC_KEY` and pins `algorithms=["RS256"]`
|
||||
- `.env.example` — removed `SECRET_KEY`, added `JWT_PRIVATE_KEY` and `JWT_PUBLIC_KEY` placeholders with generation instructions
|
||||
- `.claude/agents/security-auditor.md` — updated JWT checklist: added checks for wrong algorithm (non-RS256), symmetric key usage, and key-from-env requirement; updated policy note
|
||||
- `TODO.md` — added RS256 item to Auth/session security section (checked off)
|
||||
@@ -0,0 +1,66 @@
|
||||
# 2026-04-14 — Doc-service tests, AI category suggestions, LM Studio default
|
||||
|
||||
**Timestamp:** 2026-04-14T00:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added pytest test suite for doc-service, updated the AI prompt to return suggested categories, wired up a suggestions UI in DocumentsPage (per-suggestion Accept/Create&Assign/Dismiss), changed the default AI provider to LM Studio at host.docker.internal:1234, and created a gitignored test PDF directory.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/doc-service/tests/__init__.py`
|
||||
- `features/doc-service/tests/conftest.py` — SQLite in-memory DB, tmp DATA_DIR, mock AI provider, minimal+invoice PDF builders, real_pdfs fixture (auto-skips if no PDFs present)
|
||||
- `features/doc-service/tests/test_categories.py` — full CRUD + per-user isolation
|
||||
- `features/doc-service/tests/test_documents.py` — upload, list, get, status, delete, category assignment, AI processing integration, live PDF tests
|
||||
- `features/doc-service/tests/pdfs/.gitkeep` — tracked empty directory; drop PDFs here for live testing
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `.gitignore` — ignore `features/doc-service/tests/pdfs/*.pdf`
|
||||
- `features/doc-service/pyproject.toml` — added `aiosqlite>=0.20` to dev deps
|
||||
- `features/doc-service/app/services/ai/base.py` — added `suggested_categories` to AI prompt (2–5 category names per document)
|
||||
- `features/doc-service/app/services/config_reader.py` — default provider changed to `lmstudio`; URLs changed to `host.docker.internal:1234/v1` (Docker→host resolution on macOS/Windows)
|
||||
- `backend/app/core/app_config.py` — default `LMStudioConfig.base_url` = `http://host.docker.internal:1234/v1`; default provider = `lmstudio`
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — added `SuggestionChip` component and `suggested_categories` section in DocumentRow: checks if suggestion already exists as a user category, shows "Assign" (existing) or "Create & Assign" (new), dismiss removes from local state
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-14 — AI service container (shared AI intermediary)
|
||||
|
||||
**Timestamp:** 2026-04-14T12:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Extracted all AI provider logic from doc-service into a new standalone `ai-service` container (port 8010). All feature containers now POST messages to ai-service instead of calling AI providers directly. Added tests for ai-service, updated backend settings routes to /api/settings/ai, added AI Service card to frontend AppsPage with dedicated settings page.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/ai-service/` — full new microservice: Dockerfile, pyproject.toml, scripts/, app/ (providers, schemas, routers, services), tests/
|
||||
- `features/ai-service/.env` — gitignored, holds LM Studio API key for dev
|
||||
- `features/ai-service/.env.example`
|
||||
- `features/doc-service/app/services/prompts.py` — domain prompts extracted from deleted base.py
|
||||
- `features/doc-service/app/services/ai_client.py` — httpx client that calls ai-service /chat
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `features/doc-service/app/routers/documents.py` — replaced provider call with classify_document()
|
||||
- `features/doc-service/app/services/config_reader.py` — removed AI config section (owned by ai-service now)
|
||||
- `features/doc-service/app/core/config.py` — added AI_SERVICE_URL setting
|
||||
- `features/doc-service/pyproject.toml` — removed anthropic/openai, added httpx
|
||||
- `features/doc-service/.env` — removed LMSTUDIO_* vars, added AI_SERVICE_URL
|
||||
- `features/doc-service/tests/conftest.py` — renamed mock_ai → mock_ai_service, patching classify_document
|
||||
- `features/doc-service/tests/test_documents.py` — mock_ai → mock_ai_service; added graceful 502 test
|
||||
- `backend/app/core/app_config.py` — AIServiceConfig split from DocServiceConfig; new load/save/mask helpers
|
||||
- `backend/app/core/config.py` — added AI_SERVICE_URL setting
|
||||
- `backend/app/routers/settings.py` — new /api/settings/ai routes; test endpoint proxies to ai-service via httpx
|
||||
- `docker-compose.yml` — added ai-service container; AI_SERVICE_URL env on backend + doc-service
|
||||
- `docker-compose.dev.yml` — added ai-service dev override with hot reload and .env
|
||||
- `frontend/src/api/client.ts` — renamed getDocumentSettings→getAISettings, updateDocumentAISettings→updateAISettings, testDocumentAIConnection→testAIConnection; added getDocumentLimits
|
||||
- `frontend/src/pages/AIAdminSettingsPage.tsx` — new page at /apps/ai/settings/admin
|
||||
- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — now shows Upload Limits only
|
||||
- `frontend/src/pages/AppsPage.tsx` — added AI Service card (admin settings link, no Open button)
|
||||
- `frontend/src/App.tsx` — added /apps/ai/settings/admin route
|
||||
|
||||
## Files Deleted
|
||||
|
||||
- `features/doc-service/app/services/ai/` — anthropic_provider.py, openai_compat.py, base.py, __init__.py
|
||||
@@ -0,0 +1,96 @@
|
||||
# 2026-04-14 — PDF Document Service
|
||||
|
||||
**Timestamp:** 2026-04-14T00:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added `features/doc-service` — a FastAPI microservice that accepts PDF uploads, extracts text with pdfplumber, and uses a pluggable AI provider (Anthropic, Ollama, or LM Studio) to classify and extract structured data. Integrated it into the main backend via httpx proxy routers. Added an admin settings UI at `/apps/documents/settings/admin`. Updated the frontend route tree, Nav, and AppsPage.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/doc-service/Dockerfile` — UID 1001, pre-creates `/data/documents` and `/config`
|
||||
- `features/doc-service/pyproject.toml` — service dependencies
|
||||
- `features/doc-service/alembic.ini` — separate `alembic_version_doc_service` table
|
||||
- `features/doc-service/.env.example`
|
||||
- `features/doc-service/scripts/start.sh` — migrations + uvicorn
|
||||
- `features/doc-service/scripts/start_dev.sh` — migrations + uvicorn --reload
|
||||
- `features/doc-service/alembic/env.py` — async migrations, VERSION_TABLE isolation
|
||||
- `features/doc-service/alembic/versions/0001_create_doc_tables.py` — documents, document_categories, document_category_assignments
|
||||
- `features/doc-service/app/main.py` — no CORS (internal service)
|
||||
- `features/doc-service/app/core/config.py` — DATABASE_URL, DATA_DIR, CONFIG_PATH settings
|
||||
- `features/doc-service/app/database.py` — async engine, AsyncSessionLocal, Base
|
||||
- `features/doc-service/app/deps.py` — get_user_id from X-User-Id header
|
||||
- `features/doc-service/app/models/document.py` — Document ORM model
|
||||
- `features/doc-service/app/models/category.py` — DocumentCategory ORM model
|
||||
- `features/doc-service/app/models/category_assignment.py` — CategoryAssignment composite PK
|
||||
- `features/doc-service/app/models/__init__.py`
|
||||
- `features/doc-service/app/schemas/document.py` — DocumentOut, DocumentStatusOut, DocumentTypeUpdate, CategoryOut
|
||||
- `features/doc-service/app/schemas/category.py` — CategoryCreate, CategoryOut, CategoryUpdate
|
||||
- `features/doc-service/app/routers/documents.py` — upload, list, get, status, patch type, delete, file download, category assignment
|
||||
- `features/doc-service/app/routers/categories.py` — CRUD for DocumentCategory
|
||||
- `features/doc-service/app/services/storage.py` — aiofiles write, path helpers, delete
|
||||
- `features/doc-service/app/services/config_reader.py` — load_doc_config() with 30s TTL cache
|
||||
- `features/doc-service/app/services/ai/__init__.py` — get_provider() factory
|
||||
- `features/doc-service/app/services/ai/base.py` — AIProvider ABC, shared prompts
|
||||
- `features/doc-service/app/services/ai/anthropic_provider.py` — AnthropicProvider
|
||||
- `features/doc-service/app/services/ai/openai_compat.py` — OpenAICompatProvider (Ollama + LM Studio)
|
||||
- `backend/app/core/app_config.py` — DocServiceConfig Pydantic model, load/save with atomic write, api_key masking
|
||||
- `backend/app/routers/settings.py` — GET/PATCH /api/settings/documents/*, admin only
|
||||
- `backend/app/routers/documents_proxy.py` — httpx proxy to doc-service /documents/*
|
||||
- `backend/app/routers/categories_proxy.py` — httpx proxy to doc-service /categories/*
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — upload, list, status polling, categories, file download
|
||||
- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — AI provider config, connection test, upload limits
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/main.py` — registered settings_router, categories_proxy (before!), documents_proxy
|
||||
- `backend/pyproject.toml` — moved httpx to main deps, added anthropic>=0.28, openai>=1.0
|
||||
- `frontend/src/App.tsx` — added /apps/documents and /apps/documents/settings/admin routes, removed /settings
|
||||
- `frontend/src/components/Nav.tsx` — removed Settings link, added Profile link, logo links to /
|
||||
- `frontend/src/pages/AppsPage.tsx` — replaced stub with app launcher card grid
|
||||
- `frontend/src/api/client.ts` — added documents, categories, and settings API functions
|
||||
- `docker-compose.yml` — added doc-service service, doc_data + app_config volumes, removed internal:true from backend-net, added app_config volume to backend
|
||||
- `docker-compose.dev.yml` — added doc-service dev override with --reload
|
||||
- `TODO.md` — added PDF Documents app section
|
||||
|
||||
## Files Deleted
|
||||
|
||||
- `frontend/src/pages/SettingsPage.tsx` — stub replaced by per-app settings pages
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-14 — Server-side pagination and filter bar
|
||||
|
||||
**Timestamp:** 2026-04-14T12:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added server-side pagination and a filter bar to the Documents feature.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
- **Modified** `features/doc-service/app/schemas/document.py` — Added `DocumentPage` schema (`items`, `total`, `page`, `pages`)
|
||||
- **Modified** `features/doc-service/app/routers/documents.py` — `GET /documents` now accepts `page`, `per_page`, `sort`, `order`, `status`, `document_type`, `search` query params; returns `DocumentPage`
|
||||
- **Modified** `frontend/src/api/client.ts` — `listDocuments` accepts `DocumentListParams`; added `DocumentPage` and `DocumentListParams` interfaces
|
||||
- **Modified** `frontend/src/pages/DocumentsPage.tsx` — Added `FilterBar` (search, status, type, sort, order) and `Pagination` controls; query key includes params for cache isolation
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-14 — AI Service priority queue + model config update
|
||||
|
||||
**Timestamp:** 2026-04-14T15:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Added a priority queue system to ai-service with start/pause/resume/stop controls. Updated LM Studio model to gemma-4-e4b-it.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
- **Created** `features/ai-service/app/services/queue.py` — in-memory `asyncio.PriorityQueue` with HIGH/NORMAL/LOW priorities, FIFO within same level, single async worker with pause/resume/stop
|
||||
- **Created** `features/ai-service/app/schemas/queue.py` — `QueueRequest`, `JobStatus`, `QueueStatus` Pydantic models
|
||||
- **Created** `features/ai-service/app/routers/queue.py` — `POST /queue/jobs`, `GET /queue/jobs/{id}`, `DELETE /queue/jobs/{id}`, `GET /queue/status`, `POST /queue/pause|resume|start|stop`
|
||||
- **Modified** `features/ai-service/app/routers/chat.py` — extracted `execute_chat()` (called by queue worker); `POST /chat` now submits to queue at NORMAL priority and awaits result
|
||||
- **Modified** `features/ai-service/app/main.py` — start/stop queue worker in lifespan; mount queue router
|
||||
- **Modified** `features/ai-service/app/services/config_reader.py` — default model updated to `gemma-4-e4b-it`
|
||||
- **Modified** `features/ai-service/pyproject.toml` — `httpx` moved to runtime deps
|
||||
- **Modified** `features/ai-service/.env` — model updated to `gemma-4-e4b-it`
|
||||
@@ -0,0 +1,20 @@
|
||||
# 2026-04-14 — Docker network isolation: backend and db ports removed from host
|
||||
|
||||
**Timestamp:** 2026-04-14T00:00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Replaced flat single-network Docker setup with two explicit networks. Only the frontend exposes a host port. The database and backend are unreachable from outside the Docker network.
|
||||
|
||||
## Network architecture
|
||||
|
||||
- `backend-net` (`internal: true`) — db, backend, and frontend reverse proxy; no gateway, no host routing
|
||||
- `frontend-net` — frontend only; binds port 80 (prod) or 5173 (dev) to the host
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — removed `ports:` from `db` and `backend`; added `networks:` to all services; defined `backend-net` (internal) and `frontend-net`
|
||||
- `docker-compose.dev.yml` — no network changes needed (inherits from base); kept `5173:5173` on frontend
|
||||
- `.claude/agents/security-auditor.md` — added hard rule: only frontend exposes host ports; db and backend must never have `ports:` in any compose file
|
||||
- `TODO.md` — marked Docker port hardening as done
|
||||
- `README.md` — updated Containers table with network column; updated Installation section; removed stale SECRET_KEY env var; noted backend API docs are not directly accessible from host
|
||||
@@ -0,0 +1,12 @@
|
||||
# 2026-04-17 — Comprehensive CLAUDE.md rewrite
|
||||
|
||||
**Timestamp:** 2026-04-17T15:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Full codebase analysis used to rewrite CLAUDE.md with permanent reference material that is loaded into every Claude Code session. All conventions, standards, and structural decisions are now documented in a single authoritative file.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Modified
|
||||
- `CLAUDE.md` — Complete rewrite; added: full file/folder tree, all API endpoints table, database model reference (all columns + constraints), Pydantic schema conventions, security standards section (JWT, bcrypt, sanitization, XSS, SQLi, admin 404 pattern, network isolation, pre-commit hook), frontend patterns (Axios client, TanStack Query key/mutation conventions, route guard docs), naming conventions, HTTP status code reference, default values and limits table, Docker infrastructure (services, volumes, networks, env vars), and reorganised all workflow sections (STATUS.md, changelog, adding resources, git, infra change protocol, security hook)
|
||||
@@ -0,0 +1,23 @@
|
||||
# 2026-04-17 — Customizable home dashboard and settings placeholder
|
||||
|
||||
**Timestamp:** 2026-04-17T14:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Replaced the static dashboard page with a per-user customizable home screen. Each user can pin and unpin apps from the available services list. A time-aware greeting shows the user's display name (XSS-safe via React JSX text rendering). The Settings navigation item now routes to a placeholder page.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Added
|
||||
- `backend/alembic/versions/c7e8f9a0b1d2_add_dashboard_app_ids_to_users.py` — Migration adding `dashboard_app_ids` JSON column to `users` table (default `[]`; non-nullable)
|
||||
- `frontend/src/pages/SettingsPage.tsx` — Placeholder settings page at `/settings`
|
||||
|
||||
### Modified
|
||||
- `backend/app/models/user.py` — Added `dashboard_app_ids: Mapped[list]` JSON column
|
||||
- `backend/app/schemas/user.py` — Added `DashboardPrefsOut` and `DashboardPrefsUpdate` schemas; `app_ids` validated as safe slugs (regex, max 50, max 64 chars each)
|
||||
- `backend/app/routers/users.py` — Added `GET /api/users/me/preferences` and `PATCH /api/users/me/preferences` endpoints
|
||||
- `frontend/src/api/client.ts` — Added `DashboardPrefs` interface, `getDashboardPrefs()`, `updateDashboardPrefs()`
|
||||
- `frontend/src/pages/DashboardPage.tsx` — Full rewrite: greeting, pinned app cards grid, customize/edit mode with add/remove toggles, save via TanStack Query mutation
|
||||
- `frontend/src/App.tsx` — Imported `SettingsPage`, registered `/settings` route
|
||||
- `backend/STATUS.md` — Updated Users endpoints table and models table
|
||||
- `frontend/STATUS.md` — Added home dashboard section, added `/settings` to routes table
|
||||
@@ -0,0 +1,39 @@
|
||||
# 2026-04-17 — Groups management and Admin navigation split
|
||||
|
||||
**Timestamp:** 2026-04-17T12:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Added a Groups system (backend models, API, migration) and split the Admin sidebar item into an expandable accordion with "Users" and "Groups" sub-navigation points.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Added
|
||||
- `backend/app/models/group.py` — `Group` and `GroupMembership` SQLAlchemy models
|
||||
- `backend/app/schemas/group.py` — Pydantic schemas: `GroupCreate`, `GroupUpdate`, `GroupOut`, `GroupDetailOut`, `GroupMemberOut`
|
||||
- `backend/app/routers/groups.py` — Admin CRUD endpoints for groups + member add/remove
|
||||
- `backend/alembic/versions/a3f9c2d14e87_add_groups_and_group_memberships.py` — Migration creating `groups` and `group_memberships` tables
|
||||
- `frontend/src/pages/AdminUsersPage.tsx` — User management page (extracted from AdminPage)
|
||||
- `frontend/src/pages/AdminGroupsPage.tsx` — Group management page with inline member panel
|
||||
|
||||
### Modified
|
||||
- `backend/app/models/__init__.py` — Import `Group` and `GroupMembership`
|
||||
- `backend/app/main.py` — Mount `/api/admin/groups` router
|
||||
- `frontend/src/api/client.ts` — Added 7 group API functions and TypeScript types
|
||||
- `frontend/src/pages/AdminPage.tsx` — Now a simple redirect to `/admin/users`
|
||||
- `frontend/src/App.tsx` — Added routes `/admin/users` and `/admin/groups`
|
||||
- `frontend/src/components/Sidebar.tsx` — Admin item is now an expandable accordion with Users and Groups sub-items
|
||||
- `backend/STATUS.md` — Documented groups endpoints, models, updated future work
|
||||
- `frontend/STATUS.md` — Documented new routes, pages, API client functions
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Explicit bcrypt work factor
|
||||
|
||||
**Timestamp:** 2026-04-17T15:00:00Z
|
||||
|
||||
## Summary
|
||||
Made the bcrypt cost factor explicit (13 rounds, ~300 ms) in `hash_password` so it is easy to audit and increase over time.
|
||||
|
||||
## Files Modified
|
||||
- `backend/app/core/security.py` — added `_BCRYPT_ROUNDS = 13`; passed `rounds=` to `bcrypt.gensalt()`
|
||||
@@ -0,0 +1,22 @@
|
||||
# 2026-04-17 — Service health checks and dynamic Apps page
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Added a background health-check system to the backend that polls each registered feature service every 30 seconds. The Apps page now renders dynamically based on live service status — showing "Unavailable" when a service is registered but its container is unreachable.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Added
|
||||
- `backend/app/services/__init__.py` — package init
|
||||
- `backend/app/services/service_health.py` — service registry, background polling loop (`POLL_INTERVAL=30s`), `get_all_statuses()` read API
|
||||
- `backend/app/routers/services.py` — `GET /api/services` endpoint (requires auth)
|
||||
|
||||
### Modified
|
||||
- `backend/app/core/config.py` — added `DOC_SERVICE_URL` setting (default `http://doc-service:8001`)
|
||||
- `backend/app/main.py` — added FastAPI `lifespan` context manager: registers services, runs initial health check, starts background polling loop; mounts `/api/services` router
|
||||
- `frontend/src/api/client.ts` — added `ServiceStatus` interface and `getServices()` API function
|
||||
- `frontend/src/pages/AppsPage.tsx` — replaced hardcoded `APPS` array with dynamic query to `GET /api/services`; adds "Unavailable" state with dimmed card and explanation text
|
||||
- `backend/STATUS.md` — documented `/api/services` endpoint and health-check architecture
|
||||
- `frontend/STATUS.md` — documented dynamic Apps page behaviour
|
||||
@@ -0,0 +1,122 @@
|
||||
# 2026-04-17 — Switch UX/UI design tool from Penpot to Figma
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00
|
||||
|
||||
**Summary:** Replaced all references to Penpot with Figma across the project. The ux-designer agent now describes Figma REST API integration instead of the Penpot API.
|
||||
|
||||
**Files Modified:**
|
||||
- `.claude/agents/ux-designer.md` — rewrote Penpot setup checklist and API docs to use Figma (personal access token, `X-Figma-Token` header, `api.figma.com/v1` endpoints)
|
||||
- `frontend/STATUS.md` — updated Known limitations and Future work to reference Figma instead of Penpot
|
||||
- `TODO.md` — replaced "UX/UI — Penpot setup" section with "UX/UI — Figma setup"
|
||||
|
||||
**Files Added:**
|
||||
- `changelog/2026-04-17_switch-penpot-to-figma.md` — this entry
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Adopt shadcn/ui + Tailwind CSS as UI layer
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00
|
||||
|
||||
**Summary:** Confirmed shadcn/ui + Tailwind CSS as the UI component library and styling system. Updated CLAUDE.md stack table and frontend/STATUS.md to reflect this decision.
|
||||
|
||||
**Files Modified:**
|
||||
- `CLAUDE.md` — added UI Library (shadcn/ui) and Styling (Tailwind CSS v3) rows to Stack table
|
||||
- `frontend/STATUS.md` — marked shadcn/ui checklist item as done; updated Known limitations note from "evaluation pending" to "adoption in progress"
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Implement shadcn/ui + Tailwind CSS UI layer
|
||||
|
||||
**Timestamp:** 2026-04-17T12:00:00
|
||||
|
||||
**Summary:** Full implementation of the shadcn/ui + Tailwind CSS UI layer: design token system, theme context, new LoginPage, AppShell + Sidebar replacing the inline Nav component.
|
||||
|
||||
**Files Added:**
|
||||
- `frontend/tailwind.config.ts` — Tailwind config with CSS-variable-based design tokens (primary, accent, background, surface, border, foreground, muted)
|
||||
- `frontend/postcss.config.js` — PostCSS config (tailwindcss + autoprefixer)
|
||||
- `frontend/components.json` — shadcn/ui init config (style: default, baseColor: slate, cssVariables: true)
|
||||
- `frontend/src/styles/theme.css` — Tailwind directives + full CSS custom property token set (light/dark mode)
|
||||
- `frontend/src/lib/utils.ts` — `cn()` utility (clsx + tailwind-merge)
|
||||
- `frontend/src/components/ui/button.tsx` — shadcn/ui Button (default, ghost, outline, destructive variants)
|
||||
- `frontend/src/components/ui/input.tsx` — shadcn/ui Input
|
||||
- `frontend/src/hooks/useTheme.ts` — theme hook (localStorage + OS preference detection)
|
||||
- `frontend/src/components/ThemeToggle.tsx` — sun/moon ghost icon button
|
||||
- `frontend/src/components/Sidebar.tsx` — collapsible left sidebar (expanded/collapsed states, NavLinks, admin-only item, user avatar, logout)
|
||||
- `frontend/src/components/AppShell.tsx` — layout wrapper (Sidebar + scrollable main)
|
||||
|
||||
**Files Modified:**
|
||||
- `frontend/package.json` — added lucide-react, clsx, tailwind-merge, class-variance-authority, @radix-ui/react-slot, tailwindcss, autoprefixer, postcss
|
||||
- `frontend/vite.config.ts` — added `@/` path alias via fileURLToPath
|
||||
- `frontend/src/main.tsx` — import theme.css
|
||||
- `frontend/src/App.tsx` — PrivateRoute and AdminRoute now wrap children in AppShell; removed Nav import
|
||||
- `frontend/src/pages/LoginPage.tsx` — full redesign: two-column grid (form panel + hero panel), shadcn Input/Button, ThemeToggle
|
||||
- `frontend/src/pages/DashboardPage.tsx` — removed Nav, applied Tailwind headings
|
||||
- `frontend/src/pages/AppsPage.tsx` — removed Nav
|
||||
- `frontend/src/pages/ProfilePage.tsx` — removed Nav
|
||||
- `frontend/src/pages/AdminPage.tsx` — removed Nav
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — removed Nav
|
||||
- `frontend/src/pages/DocumentAdminSettingsPage.tsx` — removed Nav
|
||||
- `frontend/src/pages/AIAdminSettingsPage.tsx` — removed Nav
|
||||
- `frontend/STATUS.md` — added component inventory table; updated What it is; updated Future work checklist
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Per-service system prompts with AI Settings tab view
|
||||
|
||||
**Timestamp:** 2026-04-17T12:00:00
|
||||
|
||||
**Summary:** Added runtime-editable system prompts per service, stored in each service's config file on the shared volume. The AI Service Settings page now has a tab view (General / System Prompts).
|
||||
|
||||
**Files Added / Modified / Deleted**
|
||||
|
||||
- `backend/app/core/app_config.py` — Added `DocServiceSystemPrompts` model, updated `DocServiceConfig`, added `load_all_system_prompts`, `save_service_system_prompts`, `SYSTEM_PROMPT_SERVICES` registry
|
||||
- `backend/app/routers/settings.py` — Added `SystemPromptUpdate` schema, `GET /system-prompts` and `PATCH /system-prompts/{service_id}` endpoints
|
||||
- `features/doc-service/app/services/config_reader.py` — Added `_DEFAULT_SYSTEM_PROMPT`, `_DEFAULT_USER_TEMPLATE`, and `system_prompts` key to `_DEFAULT_CONFIG`
|
||||
- `features/doc-service/app/services/ai_client.py` — Loads system prompt and user template dynamically from config at runtime; falls back to defaults
|
||||
- `frontend/src/api/client.ts` — Added `ServiceSystemPrompt`, `SystemPromptsData` types, `getSystemPrompts` and `updateSystemPrompt` API functions
|
||||
- `frontend/src/pages/AIAdminSettingsPage.tsx` — Refactored to tab view (General | System Prompts); System Prompts tab shows per-service editable textarea cards
|
||||
- `features/ai-service/STATUS.md` — Documented system prompts architecture
|
||||
- `features/doc-service/STATUS.md` — Documented runtime prompt loading
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Apps page: card surface colour + whole-card click
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00
|
||||
|
||||
**Summary:** Cards on the Apps page now render with the `--color-surface` token (distinct from the page background), and clickable cards (status = available + path set) are wrapped in a `<Link>` so the entire frame navigates to the app. The Settings link is unchanged and stops click propagation.
|
||||
|
||||
**Files Modified:**
|
||||
- `frontend/src/pages/AppsPage.tsx` — Added `cardStyle`/`clickableCardStyle` objects using CSS custom properties; conditionally wraps card in `<Link>` vs `<div>`; removed standalone "Open" button; settings link gains `e.stopPropagation()`
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Sidebar app sub-navigation, category filtering, and re-analysis on category creation
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00
|
||||
|
||||
**Summary:** Added expandable Apps section to the sidebar with Documents → categories sub-navigation. Clicking a category filters the documents view. Removed tag UI from the document list (deferred). When a new category is created, similar existing categories are detected and affected documents are re-analysed in the background so the new category surfaces as a pending AI suggestion.
|
||||
|
||||
**Files Modified:**
|
||||
- `frontend/src/components/Sidebar.tsx` — replaced flat Apps nav item with collapsible accordion; Documents sub-item expands to list all user categories; category links navigate to `/apps/documents?category_id=<id>`
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — removed `TagEditor` component; added `useSearchParams` for `category_id` URL param; category filter chip shown in FilterBar with dismiss button; "Clear filters" now also clears category filter
|
||||
- `frontend/src/api/client.ts` — added `category_id` field to `DocumentListParams`
|
||||
- `features/doc-service/app/routers/documents.py` — added `category_id` query param to `GET /documents`; filters via subquery on `category_assignments`
|
||||
- `features/doc-service/app/routers/categories.py` — `POST /documents/categories` now finds similar categories by name (word overlap + SequenceMatcher) and triggers a background task to re-run AI extraction on affected documents, merging new `suggested_categories` into their `extracted_data`
|
||||
- `features/doc-service/STATUS.md` — updated endpoints table and filter params table
|
||||
- `frontend/STATUS.md` — updated sidebar and documents page sections
|
||||
|
||||
---
|
||||
|
||||
# 2026-04-17 — Re-analyse button for documents
|
||||
|
||||
**Timestamp:** 2026-04-17T00:00:00
|
||||
|
||||
**Summary:** Added a Re-analyse button to each document row that re-runs AI extraction on demand. The endpoint resets status to pending, clears any previous error, and enqueues the background processing task. The button is disabled while the document is already pending or processing.
|
||||
|
||||
**Files Modified:**
|
||||
- `features/doc-service/app/routers/documents.py` — added `POST /documents/{id}/reprocess` endpoint
|
||||
- `frontend/src/api/client.ts` — added `reprocessDocument(id)` API function
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — added `reprocessMut` mutation and Re-analyse button in document row header
|
||||
- `features/doc-service/STATUS.md` — marked reprocess as done, added endpoint to table
|
||||
@@ -0,0 +1,34 @@
|
||||
# 2026-04-18 — Category scopes, group admin role, and permission model
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduces three category scopes (personal / group / system), a PascalCase-with-dashes naming convention, a group-admin role on group memberships, and a full permission model for who can create, rename, and delete categories and documents.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/alembic/versions/e1f2a3b4c5d6_add_group_member_is_admin.py` — adds `is_group_admin BOOLEAN` to `group_memberships`
|
||||
- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — adds `can_delete BOOLEAN` to `document_shares` (backfill from feat/document-delete-permissions)
|
||||
- `features/doc-service/alembic/versions/0006_add_category_scope.py` — adds `scope VARCHAR(16)` and `group_id VARCHAR` to `document_categories`; data-migrates watch categories to scope='system'
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/models/group.py` — added `is_group_admin` to `GroupMembership`
|
||||
- `backend/app/schemas/group.py` — added `is_group_admin` to `GroupMemberOut`; new `GroupMemberAdminUpdate`
|
||||
- `backend/app/schemas/user.py` — added `is_group_admin` to `UserGroupOut`
|
||||
- `backend/app/routers/users.py` — `get_my_groups` now joins `GroupMembership` to include `is_group_admin`
|
||||
- `backend/app/routers/groups.py` — `get_group` includes `is_group_admin`; new `PATCH /{id}/members/{user_id}/admin` endpoint
|
||||
- `backend/app/routers/categories_proxy.py` — injects `x-user-is-admin` and `x-user-admin-groups` headers
|
||||
- `backend/app/routers/documents_proxy.py` — injects `x-user-admin-groups` header (was already injecting `x-user-is-admin`)
|
||||
- `features/doc-service/app/models/category.py` — added `scope`, `group_id` columns
|
||||
- `features/doc-service/app/schemas/category.py` — `CategoryOut` includes `scope`/`group_id`; `CategoryCreate` accepts `group_id`
|
||||
- `features/doc-service/app/deps.py` — added `get_user_is_admin`, `get_user_admin_groups`
|
||||
- `features/doc-service/app/routers/categories.py` — full rewrite: name validation regex, scope-based list/create, `_check_can_manage_cat` permission helper, scope-aware rename/delete
|
||||
- `features/doc-service/app/routers/documents.py` — `delete_document` enforces is_admin/can_delete/group-admin hierarchy; `remove_category` requires doc ownership; `assign_category` accepts group/system categories
|
||||
- `frontend/src/api/client.ts` — `CategoryOut` gains `scope`/`group_id`; `createCategory` accepts optional `groupId`; `UserGroupOut`/`GroupMemberOut` gain `is_group_admin`; new `adminSetGroupMemberAdmin()`; `ApiError` exported
|
||||
- `frontend/src/components/ManageCategoriesDialog.tsx` — categories grouped by scope; lock icons for unmanageable categories; rename/delete gated by scope permissions; inline rename error display
|
||||
- `frontend/src/components/SourcePanel.tsx` — categories shown in sections (Mine / Group name / System); scope picker on new category form; client-side name validation
|
||||
- `frontend/src/pages/AdminGroupsPage.tsx` — group admin checkbox column in members table
|
||||
- `backend/CLAUDE.md` — updated `group_memberships` model, migration chain, endpoints
|
||||
- `features/doc-service/CLAUDE.md` — updated `document_categories` model, `document_shares` model, migration chain, deps note
|
||||
@@ -0,0 +1,43 @@
|
||||
# 2026-04-18 — Doc Service Redesign: Scalable UX + Group-Based Sharing
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00+00:00
|
||||
|
||||
## Summary
|
||||
|
||||
Complete redesign of the document management UX for scale (10 → 100 000 documents, 2 → 1 000 categories) and group-based document sharing. Replaced the monolithic DocumentsPage with a three-column layout (Sidebar + SourcePanel + main), a slide-over detail panel, a filter chip system, multi-file upload queue, and bulk actions. Added the full backend sharing stack: `document_shares` table, share CRUD endpoints, a shared-with-me view, and X-User-Groups header injection in the gateway proxy.
|
||||
|
||||
---
|
||||
|
||||
## Files Added
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `features/doc-service/app/models/document_share.py` | DocumentShare ORM model (document_id, group_id, shared_by_user_id) |
|
||||
| `features/doc-service/app/schemas/share.py` | DocumentShareOut, DocumentShareCreate, SharedDocumentOut schemas |
|
||||
| `features/doc-service/alembic/versions/0004_add_document_shares.py` | Migration creating document_shares table with indexes |
|
||||
| `frontend/src/components/SourcePanel.tsx` | Left panel (240px): views (All/Mine/Shared) + searchable category tree + new category form |
|
||||
| `frontend/src/components/ManageCategoriesDialog.tsx` | Category CRUD modal (inline rename, delete with confirm, search) |
|
||||
| `frontend/src/components/DocumentSlideOver.tsx` | Right slide-over (480px): metadata, inline title edit, type picker, AI suggestions, categories combobox, tags, sharing section, raw text, actions |
|
||||
|
||||
---
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Change |
|
||||
|------|--------|
|
||||
| `features/doc-service/app/models/__init__.py` | Import DocumentShare |
|
||||
| `features/doc-service/app/deps.py` | Added `get_user_groups` dependency (reads X-User-Groups header) |
|
||||
| `features/doc-service/app/schemas/document.py` | Added `share_count: int = 0` to DocumentOut |
|
||||
| `features/doc-service/app/routers/documents.py` | Complete rewrite: added share CRUD endpoints, shared-with-me endpoint, N+1-safe share_count, recipient download access, X-User-Groups enforcement |
|
||||
| `features/doc-service/STATUS.md` | Added sharing section, migration 0004, updated future work |
|
||||
| `backend/app/routers/documents_proxy.py` | Injects X-User-Groups header (queries GroupMembership per request) |
|
||||
| `backend/app/routers/categories_proxy.py` | Same X-User-Groups injection pattern |
|
||||
| `backend/app/routers/users.py` | Added GET /me/groups endpoint |
|
||||
| `backend/app/schemas/user.py` | Added UserGroupOut schema |
|
||||
| `backend/STATUS.md` | Added /me/groups endpoint |
|
||||
| `frontend/src/api/client.ts` | Added share_count to DocumentOut, SharedDocumentOut, DocumentShareOut, listSharedWithMe, getDocumentShares, addDocumentShare, removeDocumentShare, getMyGroups |
|
||||
| `frontend/src/components/Sidebar.tsx` | Removed per-category NavLinks; Documents is now a single NavLink under Apps |
|
||||
| `frontend/src/components/AppShell.tsx` | Renders SourcePanel between Sidebar and main on /apps/documents route |
|
||||
| `frontend/src/pages/DocumentsPage.tsx` | Complete rewrite: three-panel layout, view param (all/mine/shared), smart polling, drag-and-drop, multi-file upload queue, filter chip system, bulk actions bar |
|
||||
| `frontend/STATUS.md` | Complete rewrite reflecting all new components and patterns |
|
||||
| `CLAUDE.md` | Updated file tree, Database Models (DocumentShare), Migration chains (0004), API endpoints (shares, shared-with-me, /me/groups), TanStack Query keys, request flow diagram |
|
||||
@@ -0,0 +1,25 @@
|
||||
# 2026-04-18 — Document delete permissions + three-dots menu fix
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Added a proper permission model for document deletion: owners and admins can always delete; group members can delete only when the share was explicitly granted `can_delete=true`. Fixed silent delete failures (watch docs returning 404 with no user feedback) and fixed the three-dots context menu being clipped by `overflow-hidden` on the table container.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
### Added
|
||||
- `features/doc-service/alembic/versions/0005_add_share_can_delete.py` — migration: adds `can_delete BOOLEAN NOT NULL DEFAULT false` to `document_shares`
|
||||
|
||||
### Modified
|
||||
- `features/doc-service/app/models/document_share.py` — added `can_delete: Mapped[bool]` column
|
||||
- `features/doc-service/app/schemas/share.py` — added `can_delete` to `DocumentShareOut` and `DocumentShareCreate`; added `viewer_can_delete` to `SharedDocumentOut`
|
||||
- `features/doc-service/app/schemas/document.py` — added `viewer_can_delete: bool = False` to `DocumentOut`
|
||||
- `features/doc-service/app/deps.py` — added `get_user_is_admin()` dep reading `x-user-is-admin` header
|
||||
- `features/doc-service/app/routers/documents.py` — added `_get_deletable_doc_ids()` helper; updated list/get/delete endpoints with permission logic; updated `add_share` to store `can_delete`; updated shared-with-me to include `viewer_can_delete`
|
||||
- `backend/app/routers/documents_proxy.py` — `_forward_headers()` now injects `x-user-is-admin` header
|
||||
- `frontend/src/api/client.ts` — `DocumentOut`: added `viewer_can_delete`; `DocumentShareOut`: added `can_delete`; `addDocumentShare`: accepts `canDelete` param
|
||||
- `frontend/src/pages/DocumentsPage.tsx` — `RowActionsMenu`: replaced absolute dropdown with `createPortal` to fix clipping; delete button now uses `doc.viewer_can_delete`; added `onError` handler for silent failures
|
||||
- `frontend/src/components/DocumentSlideOver.tsx` — sharing section: shows trash icon badge on shares with `can_delete=true`; added "Allow group members to delete" checkbox before group picker; delete button uses `doc.viewer_can_delete`
|
||||
- `features/doc-service/CLAUDE.md` — updated `document_shares` table docs + migration chain
|
||||
- `backend/CLAUDE.md` — noted `x-user-is-admin` header injection
|
||||
@@ -0,0 +1,44 @@
|
||||
# 2026-04-18 — Generic Plugin Architecture + Watch Directory Feature
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Implemented a generic plugin/extension infrastructure that allows feature containers to self-describe their settings via a manifest contract, with no feature-specific code required in the backend or frontend. Built the watch-directory feature entirely inside the doc-service container as the first plugin consumer.
|
||||
|
||||
## Files Added
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `backend/app/routers/plugins.py` | Generic plugin proxy: `GET/PATCH /api/plugins`, `/api/plugins/{id}/manifest`, `/api/plugins/{id}/settings` |
|
||||
| `frontend/src/components/PluginSchemaForm.tsx` | JSON Schema → React form renderer (boolean/string/number/readOnly) |
|
||||
| `frontend/src/pages/PluginSettingsPage.tsx` | Generic plugin settings page driven by manifest |
|
||||
| `features/doc-service/app/routers/plugin.py` | Doc-service plugin endpoints: `/plugin/manifest`, `/plugin/settings` |
|
||||
| `features/doc-service/app/services/file_watcher.py` | watchdog-based PDF watcher with startup scan, folder-to-category mapping, no-remove policy |
|
||||
| `features/doc-service/alembic/versions/0003_add_watch_columns.py` | Migration: source, watch_path, suggested_folder, suggested_filename |
|
||||
| `dev-watch/.gitkeep` | Dev bind-mount directory for local file watcher testing |
|
||||
|
||||
## Files Modified
|
||||
|
||||
| File | Description |
|
||||
|------|-------------|
|
||||
| `backend/app/services/service_health.py` | Also fetches and caches `/plugin/manifest` from healthy services |
|
||||
| `backend/app/deps.py` | Added `check_plugin_access(plugin_id, user, db)` helper |
|
||||
| `backend/app/main.py` | Mounted `/api/plugins` router |
|
||||
| `frontend/src/api/client.ts` | Added plugin API functions and suggestion confirm/reject functions; extended `DocumentOut` with new fields |
|
||||
| `frontend/src/components/Sidebar.tsx` | Added dynamic "Extensions" section populated from `/api/plugins` |
|
||||
| `frontend/src/App.tsx` | Added `/settings/plugins/:id` route |
|
||||
| `features/doc-service/app/models/document.py` | Added 4 new columns: source, watch_path, suggested_folder, suggested_filename |
|
||||
| `features/doc-service/app/schemas/document.py` | Exposed 4 new fields in `DocumentOut` |
|
||||
| `features/doc-service/app/services/config_reader.py` | Added storage config defaults, `get_storage_config()`, `save_storage_config()` |
|
||||
| `features/doc-service/app/routers/documents.py` | Watch-user visibility (`OR user_id = "watch"`); 4 suggestion endpoints |
|
||||
| `features/doc-service/app/routers/categories.py` | Watch-owned categories included in list |
|
||||
| `features/doc-service/app/main.py` | Lifespan watcher start/stop; plugin router mounted |
|
||||
| `features/doc-service/pyproject.toml` | Added `watchdog>=4.0` |
|
||||
| `features/doc-service/Dockerfile` | Pre-create `/data/watch` |
|
||||
| `docker-compose.yml` | Added `watch_data` named volume; mounted to doc-service |
|
||||
| `docker-compose.dev.yml` | Dev bind-mount `./dev-watch:/data/watch` |
|
||||
| `CLAUDE.md` | Updated all affected sections (models, migrations, endpoints, routes, tree, query keys, volumes) |
|
||||
| `backend/STATUS.md` | Plugin system section added |
|
||||
| `features/doc-service/STATUS.md` | Watch feature, plugin endpoints, migration 0003, updated architecture diagram |
|
||||
| `frontend/STATUS.md` | Extensions sidebar, PluginSchemaForm, PluginSettingsPage, new API functions |
|
||||
@@ -0,0 +1,24 @@
|
||||
# 2026-04-18 — Service admin groups + combined settings pages
|
||||
|
||||
**Timestamp:** 2026-04-18T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduced per-service admin groups that are auto-created at startup, consolidated doc-service and AI-service settings each onto a single page, and collapsed the dual "Settings + Extension" app card buttons into one Settings button visible to admins and service-group members.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `backend/app/services/group_bootstrap.py` — Idempotent startup task: creates `{service_id}-admin` group for every registered service if absent.
|
||||
- `features/ai-service/app/routers/plugin.py` — `GET /plugin/manifest` for ai-service (exposes access rules: `ai-service-admin` group).
|
||||
- `frontend/src/pages/DocServiceSettingsPage.tsx` — Combined doc-service settings page: Upload Limits + Watch Directory (rendered via `PluginSchemaForm`).
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `backend/app/main.py` — Lifespan now calls `ensure_service_admin_groups(db)` after `register_services()`.
|
||||
- `backend/app/deps.py` — Added `get_service_admin(service_id)` factory dependency: grants access to superusers or `{service_id}-admin` group members; returns 404 otherwise.
|
||||
- `backend/app/routers/settings.py` — AI settings (`/ai`, `/ai/test`, `/system-prompts`) and doc limits (`/documents/limits`) now use `get_service_admin(...)` instead of `get_current_admin` — service group members can access them.
|
||||
- `backend/app/services/service_health.py` — `settings_path` for doc-service changed to `/apps/documents/settings`; ai-service to `/apps/ai/settings` (removed `/admin` suffix).
|
||||
- `features/ai-service/app/main.py` — Mounts new `plugin.router` so backend poller can discover ai-service manifest.
|
||||
- `frontend/src/App.tsx` — Added `ServiceAdminRoute` component (checks token + is_admin OR plugin list contains serviceId). Updated doc/AI settings routes to new paths under `ServiceAdminRoute`.
|
||||
- `frontend/src/pages/AppsPage.tsx` — Replaced two-button layout (Settings + Extension) with single Settings button; visible when `user.is_admin || pluginIds.has(svc.id)`.
|
||||
- `backend/STATUS.md`, `frontend/STATUS.md`, `CLAUDE.md` — Updated to reflect all changes above.
|
||||
@@ -0,0 +1,12 @@
|
||||
# 2026-04-19 — Merge checklist update + admin delete bug fix
|
||||
|
||||
**Timestamp:** 2026-04-19T00:15:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Updated `tests/MERGE_CHECKLIST.md` with all new tests for the two recently merged features (document delete permissions and category scopes / group-admin role). While running the new test 12.16b, discovered and fixed a bug where the doc-service delete endpoint returned 404 for admins deleting non-owned documents.
|
||||
|
||||
## Files Added / Modified / Deleted
|
||||
|
||||
- **Modified** `tests/MERGE_CHECKLIST.md` — added 18 new tests: 4.9–4.10 (group admin role), 12.16b–12.16e (delete permissions), 13.11–13.14 (can_delete sharing), 14.7–14.17 (category scopes, PascalCase naming), 19.11 (three-dots portal fix); updated 12.16 and 14.5 descriptions
|
||||
- **Modified** `features/doc-service/app/routers/documents.py` — fixed `delete_document` to bypass group-membership filter for admins; previously admins got 404 on any document they didn't own or that wasn't a watch doc
|
||||
@@ -0,0 +1,60 @@
|
||||
# 2026-04-20 — Dedicated storage-service with pluggable backends
|
||||
|
||||
**Timestamp:** 2026-04-20T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduced a dedicated `storage-service` container (port 8020) as the single file/blob persistence layer for the entire stack. All services now route file and config I/O through this service's HTTP API. The service supports pluggable storage backends (local filesystem by default; S3-compatible and WebDAV built in) with a zero-data-loss migration flow. The `doc_data` and `app_config` Docker volumes were removed.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/storage-service/app/main.py` — FastAPI app, lifespan (backend init)
|
||||
- `features/storage-service/app/core/config.py` — Settings (DATA_DIR, STORAGE_BACKEND, S3_*, WEBDAV_*)
|
||||
- `features/storage-service/app/routers/health.py` — GET /health
|
||||
- `features/storage-service/app/routers/objects.py` — PUT/GET/DELETE /objects/{bucket}/{key:path}, GET /objects/{bucket}
|
||||
- `features/storage-service/app/routers/migrate.py` — POST/GET/DELETE /migrate, PATCH /backend-config
|
||||
- `features/storage-service/app/services/backend_manager.py` — Driver factory, singleton, atomic switch
|
||||
- `features/storage-service/app/services/migration.py` — Async migration: copy → verify → switch → cleanup
|
||||
- `features/storage-service/app/services/backends/base.py` — AbstractStorageBackend ABC
|
||||
- `features/storage-service/app/services/backends/local.py` — LocalFSBackend (path traversal guard)
|
||||
- `features/storage-service/app/services/backends/s3.py` — S3Backend (aiobotocore, endpoint_url configurable)
|
||||
- `features/storage-service/app/services/backends/webdav.py` — WebDAVBackend (aiohttp + defusedxml)
|
||||
- `features/storage-service/scripts/start.sh` — prod uvicorn start
|
||||
- `features/storage-service/scripts/start_dev.sh` — dev uvicorn --reload start
|
||||
- `features/storage-service/pyproject.toml` — Dependencies
|
||||
- `features/storage-service/Dockerfile` — python:3.12-slim, non-root user 1001, port 8020
|
||||
- `features/storage-service/CLAUDE.md` — API reference, bucket docs, driver docs
|
||||
- `features/storage-service/STATUS.md` — Service status
|
||||
- `backend/app/core/config_storage.py` — Thin async helpers: read_json/write_json/delete_key/list_keys
|
||||
- `backend/app/routers/storage_config.py` — Admin proxy endpoints for storage config + migration
|
||||
- `features/doc-service/alembic/versions/0008_rename_file_path_to_storage_key.py` — DB migration
|
||||
- `frontend/src/pages/StorageAdminPage.tsx` — Admin UI: backend status, driver form, migration progress
|
||||
- `tests/storage-service_tests.md` — §20 storage-service test suite
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — Added storage-service, storage_data volume; removed doc_data, app_config; added depends_on service_healthy
|
||||
- `docker-compose.dev.yml` — Added storage-service dev override
|
||||
- `backend/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `backend/app/core/app_config.py` — Full async rewrite using config_storage HTTP helpers (no filesystem)
|
||||
- `backend/app/routers/settings.py` — Removed all asyncio.to_thread wrappers; direct await calls
|
||||
- `backend/app/main.py` — Register storage_config router; update register_services call
|
||||
- `backend/app/services/service_health.py` — Register storage-service
|
||||
- `features/doc-service/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `features/doc-service/app/models/document.py` — file_path → storage_key
|
||||
- `features/doc-service/app/services/storage.py` — Complete rewrite: HTTP client calls to storage-service
|
||||
- `features/doc-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `features/doc-service/app/services/file_watcher.py` — Uses save_upload() → storage-service
|
||||
- `features/doc-service/app/routers/documents.py` — storage_key refs, pdfplumber(io.BytesIO), streaming from storage-service
|
||||
- `features/ai-service/app/core/config.py` — Added STORAGE_SERVICE_URL; removed CONFIG_PATH
|
||||
- `features/ai-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `frontend/src/api/client.ts` — Added StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions
|
||||
- `frontend/src/App.tsx` — Added /admin/storage route (AdminRoute → StorageAdminPage)
|
||||
- `tests/ALL_TESTS.md` — Updated to 20 feature areas; added §20 storage-service tests
|
||||
- `CLAUDE.md` — Added storage-service to Services/Volumes/Networks tables; storage enforcement rule; §20 test file
|
||||
- `backend/CLAUDE.md` — Added config_storage.py, storage_config.py to tree; added admin storage endpoints
|
||||
- `frontend/CLAUDE.md` — Added StorageAdminPage to tree; added /admin/storage route
|
||||
- `features/doc-service/CLAUDE.md` — Updated storage.py description; file_path → storage_key; added migration 0008
|
||||
- `features/ai-service/CLAUDE.md` — Added config_reader.py description
|
||||
- `backend/STATUS.md` — Added storage-config endpoints; updated settings persistence note
|
||||
- `frontend/STATUS.md` — Added /admin/storage route; added StorageAdminPage description
|
||||
@@ -0,0 +1 @@
|
||||
# Watch directory for development testing
|
||||
@@ -0,0 +1,42 @@
|
||||
# Development overrides — hot reload for backend and frontend
|
||||
# Usage: docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build
|
||||
|
||||
services:
|
||||
|
||||
backend:
|
||||
user: "1001:1001"
|
||||
command: sh scripts/start_dev.sh
|
||||
volumes:
|
||||
- ./backend:/app
|
||||
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend
|
||||
target: builder # stop at the Node stage, skip nginx
|
||||
user: "1001:1001"
|
||||
command: npm run dev -- --host 0.0.0.0
|
||||
ports:
|
||||
- "5173:5173"
|
||||
environment:
|
||||
VITE_API_TARGET: http://backend:8000
|
||||
volumes:
|
||||
- ./frontend:/app
|
||||
- /app/node_modules
|
||||
|
||||
ai-service:
|
||||
command: sh scripts/start_dev.sh
|
||||
env_file: ./features/ai-service/.env # gitignored — holds LM Studio / AI credentials
|
||||
volumes:
|
||||
- ./features/ai-service:/app
|
||||
|
||||
storage-service:
|
||||
command: sh scripts/start_dev.sh
|
||||
volumes:
|
||||
- ./features/storage-service:/app
|
||||
|
||||
doc-service:
|
||||
command: sh scripts/start_dev.sh
|
||||
env_file: ./features/doc-service/.env
|
||||
volumes:
|
||||
- ./features/doc-service:/app
|
||||
- ./dev-watch:/data/watch # bind-mount local folder for easy testing
|
||||
+117
-23
@@ -1,37 +1,131 @@
|
||||
services:
|
||||
|
||||
# ── Database ────────────────────────────────────────────────────────────────
|
||||
db:
|
||||
image: postgres:16
|
||||
image: postgres:16-alpine
|
||||
user: "70:70" # postgres user UID:GID in alpine image (fixed by image)
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_USER: postgres
|
||||
POSTGRES_PASSWORD: password
|
||||
POSTGRES_DB: destroying_sap
|
||||
ports:
|
||||
- "5432:5432"
|
||||
POSTGRES_USER: ${POSTGRES_USER:-postgres}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-password}
|
||||
POSTGRES_DB: ${POSTGRES_DB:-destroying_sap}
|
||||
volumes:
|
||||
- postgres_data:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres}"]
|
||||
interval: 5s
|
||||
timeout: 5s
|
||||
retries: 10
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
backend:
|
||||
build: ./backend
|
||||
command: uvicorn app.main:app --host 0.0.0.0 --port 8000 --reload
|
||||
volumes:
|
||||
- ./backend:/app
|
||||
ports:
|
||||
- "8000:8000"
|
||||
# ── Storage service (unified blob storage) ──────────────────────────────────
|
||||
storage-service:
|
||||
build:
|
||||
context: ./features/storage-service
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
DATABASE_URL: postgresql+asyncpg://postgres:password@db:5432/destroying_sap
|
||||
depends_on:
|
||||
- db
|
||||
|
||||
frontend:
|
||||
build: ./frontend
|
||||
command: npm run dev -- --host
|
||||
STORAGE_BACKEND: local
|
||||
DATA_DIR: /data/storage
|
||||
volumes:
|
||||
- ./frontend:/app
|
||||
- /app/node_modules
|
||||
- storage_data:/data/storage
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "python -c \"import urllib.request; urllib.request.urlopen('http://localhost:8020/health')\""]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── Backend (management) ────────────────────────────────────────────────────
|
||||
backend:
|
||||
build:
|
||||
context: ./backend
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
env_file: ./backend/.env
|
||||
environment:
|
||||
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
|
||||
DOC_SERVICE_URL: http://doc-service:8001
|
||||
AI_SERVICE_URL: http://ai-service:8010
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── AI service (shared AI provider intermediary) ─────────────────────────────
|
||||
ai-service:
|
||||
build:
|
||||
context: ./features/ai-service
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
depends_on:
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── Doc service (PDF extraction) ────────────────────────────────────────────
|
||||
doc-service:
|
||||
build:
|
||||
context: ./features/doc-service
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
DATABASE_URL: postgresql+asyncpg://${POSTGRES_USER:-postgres}:${POSTGRES_PASSWORD:-password}@db:5432/${POSTGRES_DB:-destroying_sap}
|
||||
AI_SERVICE_URL: http://ai-service:8010
|
||||
STORAGE_SERVICE_URL: http://storage-service:8020
|
||||
volumes:
|
||||
- watch_data:/data/watch
|
||||
depends_on:
|
||||
db:
|
||||
condition: service_healthy
|
||||
ai-service:
|
||||
condition: service_started
|
||||
storage-service:
|
||||
condition: service_healthy
|
||||
networks:
|
||||
- backend-net
|
||||
|
||||
# ── Frontend (UI) ────────────────────────────────────────────────────────────
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend
|
||||
dockerfile: Dockerfile
|
||||
network: host
|
||||
user: "1001:1001"
|
||||
restart: unless-stopped
|
||||
ports:
|
||||
- "5173:5173"
|
||||
- "80:8080"
|
||||
depends_on:
|
||||
- backend
|
||||
networks:
|
||||
- backend-net
|
||||
- frontend-net
|
||||
|
||||
volumes:
|
||||
postgres_data:
|
||||
storage_data: # All file/blob storage — managed by storage-service (documents + config)
|
||||
watch_data: # Watch directory — bind-mount your NAS/Nextcloud here via docker-compose.override.yml
|
||||
|
||||
networks:
|
||||
# backend-net: db ↔ backend ↔ doc-service. No host ports bound.
|
||||
# internal:true removed — doc-service needs outbound access for cloud AI providers.
|
||||
backend-net:
|
||||
# External-facing: only the frontend binds a host port through this network.
|
||||
frontend-net:
|
||||
|
||||
@@ -0,0 +1,12 @@
|
||||
AI_PROVIDER=lmstudio
|
||||
|
||||
LMSTUDIO_BASE_URL=http://host.docker.internal:1234/v1
|
||||
LMSTUDIO_API_KEY=your-lmstudio-api-key
|
||||
LMSTUDIO_MODEL=local-model
|
||||
|
||||
OLLAMA_BASE_URL=http://host.docker.internal:11434/v1
|
||||
OLLAMA_MODEL=llama3.2
|
||||
OLLAMA_API_KEY=ollama
|
||||
|
||||
ANTHROPIC_API_KEY=sk-ant-your-key-here
|
||||
ANTHROPIC_MODEL=claude-haiku-4-5-20251001
|
||||
@@ -0,0 +1,51 @@
|
||||
# ai-service — Claude context
|
||||
|
||||
AI provider intermediary, port 8010 (internal only — never proxied to the browser). Accepts chat requests from `doc-service` (and potentially other callers). Manages a priority queue and abstracts over multiple AI providers (Anthropic, Ollama/LM Studio). See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
## File & Folder Tree
|
||||
|
||||
```
|
||||
features/ai-service/
|
||||
├── app/
|
||||
│ ├── main.py ← FastAPI, queue worker lifespan
|
||||
│ ├── core/
|
||||
│ │ └── config.py ← Settings via pydantic-settings
|
||||
│ ├── providers/
|
||||
│ │ ├── base.py ← AIProvider abstract class
|
||||
│ │ ├── anthropic_provider.py ← Anthropic API integration
|
||||
│ │ └── openai_compat.py ← Ollama / LM Studio compatibility
|
||||
│ ├── routers/
|
||||
│ │ ├── chat.py ← POST /chat (sync, NORMAL priority queue)
|
||||
│ │ ├── health.py ← GET /health
|
||||
│ │ ├── queue.py ← GET /queue/status, /pause, /resume, /cancel/{id}
|
||||
│ │ └── plugin.py ← GET /plugin/manifest (access rules for ai-service-admin group)
|
||||
│ └── services/
|
||||
│ ├── queue.py ← Priority queue (CRITICAL > HIGH > NORMAL)
|
||||
│ └── config_reader.py ← Reads ai_service_config.json from storage-service config bucket (30 s TTL cache)
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## API Endpoints (internal only)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| POST | `/chat` | Chat request (queued at NORMAL priority) |
|
||||
| GET | `/health` | Health check |
|
||||
| GET | `/queue/status` | Queue state |
|
||||
| POST | `/queue/pause` | Pause queue |
|
||||
| POST | `/queue/resume` | Resume queue |
|
||||
| POST | `/queue/cancel/{job_id}` | Cancel job |
|
||||
| GET | `/plugin/manifest` | Plugin manifest (access rules for ai-service-admin group) |
|
||||
|
||||
These endpoints are only reachable on `backend-net`. The backend does not expose them to the browser.
|
||||
|
||||
---
|
||||
|
||||
## Note on timeout and retry configuration
|
||||
|
||||
Caller-side timeout and retry settings live in `features/doc-service/app/services/ai_client.py` — see `features/doc-service/CLAUDE.md` for the values.
|
||||
@@ -0,0 +1,32 @@
|
||||
# ── Stage 1: dependency installation ─────────────────────────────────────────
|
||||
FROM python:3.12-slim AS builder
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN pip install --upgrade pip
|
||||
|
||||
COPY pyproject.toml .
|
||||
RUN pip install --prefix=/install .
|
||||
|
||||
# ── Stage 2: runtime ──────────────────────────────────────────────────────────
|
||||
FROM python:3.12-slim
|
||||
|
||||
# Create non-root user (UID/GID 1001)
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# No filesystem directories needed — all config goes through storage-service.
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /install /usr/local
|
||||
|
||||
COPY --chown=appuser:appuser app ./app
|
||||
COPY --chown=appuser:appuser scripts ./scripts
|
||||
RUN chmod +x scripts/start.sh scripts/start_dev.sh
|
||||
|
||||
USER appuser
|
||||
|
||||
EXPOSE 8010
|
||||
|
||||
CMD ["sh", "scripts/start.sh"]
|
||||
@@ -0,0 +1,118 @@
|
||||
# AI Service — Status
|
||||
|
||||
## What it is
|
||||
|
||||
Shared AI intermediary container. All feature containers (doc-service, future services) POST prompts here. It routes requests to the configured model (LM Studio / Ollama / Anthropic) and returns a normalised response. It is **stateless** — no database, no conversation history. History and context are the caller's responsibility.
|
||||
|
||||
Port: `8010` (internal only, not exposed to host).
|
||||
|
||||
---
|
||||
|
||||
## Current functionality
|
||||
|
||||
### Endpoints
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `POST` | `/chat` | Synchronous chat: submits at NORMAL priority, blocks until done |
|
||||
| `GET` | `/health` | `{"status": "ok"}` |
|
||||
| `GET` | `/health/provider` | Active provider name, model, configured flag |
|
||||
| `POST` | `/queue/jobs` | Async enqueue — returns `job_id` immediately |
|
||||
| `GET` | `/queue/jobs/{id}` | Poll job: status, position, result, error |
|
||||
| `DELETE` | `/queue/jobs/{id}` | Cancel a pending job |
|
||||
| `GET` | `/queue/status` | Worker state: running, paused, queue_size, current_job_id |
|
||||
| `POST` | `/queue/pause` | Finish current job, stop picking new ones |
|
||||
| `POST` | `/queue/resume` | Unpause |
|
||||
| `POST` | `/queue/start` | Start (or restart) the worker task |
|
||||
| `POST` | `/queue/stop` | Stop worker (pending jobs stay queued) |
|
||||
|
||||
### Priority queue
|
||||
|
||||
- Three levels: `high` (1) > `normal` (3) > `low` (5)
|
||||
- FIFO within same priority level (monotonic sequence counter)
|
||||
- Single async worker — one LLM call at a time
|
||||
- Pause / resume / start / stop without restarting the container
|
||||
- `POST /chat` is a synchronous wrapper: enqueues at NORMAL, awaits the future
|
||||
|
||||
### Providers
|
||||
|
||||
| Provider | Protocol | SDK |
|
||||
|----------|----------|-----|
|
||||
| LM Studio | OpenAI-compatible HTTP | openai |
|
||||
| Ollama | OpenAI-compatible HTTP | openai |
|
||||
| Anthropic | Anthropic API (HTTPS) | anthropic |
|
||||
|
||||
Active provider is selected by `"provider"` key in `/config/ai_service_config.json` (shared Docker volume), with env var overrides for dev.
|
||||
|
||||
### Configuration (env var overrides)
|
||||
|
||||
```
|
||||
AI_PROVIDER lmstudio | ollama | anthropic
|
||||
LMSTUDIO_BASE_URL http://host.docker.internal:1234/v1
|
||||
LMSTUDIO_API_KEY sk-lm-…
|
||||
LMSTUDIO_MODEL gemma-4-e4b-it ← current
|
||||
OLLAMA_BASE_URL / OLLAMA_MODEL / OLLAMA_API_KEY
|
||||
ANTHROPIC_API_KEY / ANTHROPIC_MODEL
|
||||
```
|
||||
|
||||
Credentials live in `features/ai-service/.env` (gitignored).
|
||||
|
||||
### Error codes
|
||||
|
||||
| Code | Meaning |
|
||||
|------|---------|
|
||||
| 422 | Bad request (empty messages, unknown priority) |
|
||||
| 502 | Provider connection / API error |
|
||||
| 503 | Provider not configured / unknown provider |
|
||||
| 504 | Provider timeout |
|
||||
|
||||
---
|
||||
|
||||
## Architecture
|
||||
|
||||
```
|
||||
Callers (doc-service, future services)
|
||||
│
|
||||
└─▶ POST /chat (sync) ─┐
|
||||
└─▶ POST /queue/jobs (async) ─┤
|
||||
▼
|
||||
asyncio.PriorityQueue
|
||||
(HIGH=1, NORMAL=3, LOW=5)
|
||||
│
|
||||
QueueWorker (single task)
|
||||
│
|
||||
execute_chat(request)
|
||||
│
|
||||
Provider SDK (openai / anthropic)
|
||||
│
|
||||
LM Studio / Ollama / Anthropic API
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Known limitations / not implemented
|
||||
|
||||
- **TLS to LM Studio** — communication is plain HTTP (`http://host.docker.internal:1234`). Deferred until LM Studio HTTPS configuration is confirmed. When ready: set `LMSTUDIO_BASE_URL=https://...` and optionally add `ssl_verify` + `ca_bundle` config keys to the OpenAI-compat provider.
|
||||
- **True preemption** — a HIGH job arriving while a LOW job is processing will be next in queue but will not interrupt the running inference.
|
||||
- **Queue persistence** — the in-memory queue is lost on container restart. Pending jobs are not persisted to disk.
|
||||
- **Authentication on queue endpoints** — `/queue/*` management endpoints have no auth guard. Should be protected before any public/multi-tenant deployment (internal network is the only current protection).
|
||||
- **Streaming responses** — `/chat` returns the full response after generation. Streaming (Server-Sent Events) not implemented.
|
||||
- **Metrics / observability** — no Prometheus metrics, no structured request logging per job.
|
||||
|
||||
---
|
||||
|
||||
## System prompts
|
||||
|
||||
Each feature service (doc-service, future services) owns its own system prompt, stored in that service's config JSON on the shared volume. The backend settings API (`GET/PATCH /api/settings/system-prompts`) aggregates and edits them. The AI Service Settings UI exposes a **System Prompts** tab for editing all registered service prompts at runtime.
|
||||
|
||||
---
|
||||
|
||||
## Future work
|
||||
|
||||
- [ ] TLS support for LM Studio / Ollama (`ssl_verify`, `ca_bundle` config)
|
||||
- [ ] Auth guard on queue management endpoints (admin token or internal-only route)
|
||||
- [ ] Streaming responses via SSE (`POST /chat/stream`)
|
||||
- [ ] Queue persistence (SQLite or Redis-backed) so jobs survive restarts
|
||||
- [ ] Job result TTL / cleanup (currently jobs accumulate in `_jobs` dict indefinitely)
|
||||
- [ ] Per-caller priority override (e.g. doc-service background jobs = LOW, user-triggered = NORMAL)
|
||||
- [ ] Metrics endpoint (`/metrics`) for queue depth, job latency, provider error rate
|
||||
@@ -0,0 +1,11 @@
|
||||
from pydantic_settings import BaseSettings
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
PROJECT_NAME: str = "ai-service"
|
||||
STORAGE_SERVICE_URL: str = "http://storage-service:8020"
|
||||
|
||||
model_config = {"env_file": ".env", "extra": "ignore"}
|
||||
|
||||
|
||||
settings = Settings()
|
||||
@@ -0,0 +1,36 @@
|
||||
import logging
|
||||
from contextlib import asynccontextmanager
|
||||
|
||||
from fastapi import FastAPI
|
||||
|
||||
from app.core.config import settings
|
||||
from app.routers import chat, health, plugin
|
||||
from app.routers import queue as queue_router
|
||||
from app.services.config_reader import load_ai_config
|
||||
from app.services.queue import queue_service
|
||||
|
||||
logger = logging.getLogger("ai-service")
|
||||
|
||||
|
||||
@asynccontextmanager
|
||||
async def lifespan(app: FastAPI):
|
||||
config = await load_ai_config()
|
||||
provider = config.get("provider", "lmstudio")
|
||||
model = config.get(provider, {}).get("model", "unknown")
|
||||
logger.info("[ai-service] active provider: %s model: %s", provider, model)
|
||||
|
||||
queue_service.start()
|
||||
logger.info("[ai-service] queue worker started")
|
||||
|
||||
yield
|
||||
|
||||
queue_service.stop()
|
||||
logger.info("[ai-service] queue worker stopped")
|
||||
|
||||
|
||||
app = FastAPI(title=settings.PROJECT_NAME, lifespan=lifespan)
|
||||
|
||||
app.include_router(chat.router, tags=["chat"])
|
||||
app.include_router(health.router, tags=["health"])
|
||||
app.include_router(queue_router.router)
|
||||
app.include_router(plugin.router, tags=["plugin"])
|
||||
@@ -0,0 +1,20 @@
|
||||
from app.providers.base import AIProvider
|
||||
|
||||
|
||||
def get_provider(ai_config: dict) -> AIProvider:
|
||||
"""Return an AIProvider instance for the active provider in the config."""
|
||||
provider_name = ai_config.get("provider", "lmstudio")
|
||||
provider_cfg = ai_config.get(provider_name, {})
|
||||
|
||||
match provider_name:
|
||||
case "anthropic":
|
||||
from app.providers.anthropic_provider import AnthropicProvider
|
||||
return AnthropicProvider(provider_cfg)
|
||||
case "ollama" | "lmstudio":
|
||||
from app.providers.openai_compat import OpenAICompatProvider
|
||||
return OpenAICompatProvider(provider_cfg, provider_name=provider_name)
|
||||
case _:
|
||||
raise ValueError(f"Unknown AI provider: {provider_name!r}")
|
||||
|
||||
|
||||
__all__ = ["AIProvider", "get_provider"]
|
||||
@@ -0,0 +1,54 @@
|
||||
import asyncio
|
||||
|
||||
import anthropic
|
||||
|
||||
from app.providers.base import AIProvider
|
||||
from app.schemas.chat import ChatMessage
|
||||
|
||||
|
||||
class AnthropicProvider(AIProvider):
|
||||
def __init__(self, config: dict) -> None:
|
||||
self._client = anthropic.AsyncAnthropic(api_key=config.get("api_key", ""))
|
||||
self.model_name = config.get("model", "claude-haiku-4-5-20251001")
|
||||
self.provider_name = "anthropic"
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ChatMessage],
|
||||
max_tokens: int,
|
||||
temperature: float,
|
||||
) -> tuple[str, int | None, int | None]:
|
||||
# Anthropic uses a top-level `system=` param, not a role in the messages array
|
||||
system_content = ""
|
||||
user_messages = []
|
||||
for msg in messages:
|
||||
if msg.role == "system":
|
||||
system_content += msg.content + "\n"
|
||||
else:
|
||||
user_messages.append({"role": msg.role, "content": msg.content})
|
||||
|
||||
try:
|
||||
response = await self._client.messages.create(
|
||||
model=self.model_name,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
system=system_content.strip() or anthropic.NOT_GIVEN,
|
||||
messages=user_messages,
|
||||
)
|
||||
except anthropic.APIConnectionError as exc:
|
||||
raise ProviderConnectionError(str(exc)) from exc
|
||||
except anthropic.APITimeoutError as exc:
|
||||
raise ProviderTimeoutError(str(exc)) from exc
|
||||
except anthropic.APIStatusError as exc:
|
||||
raise ProviderConnectionError(f"Anthropic API error {exc.status_code}: {exc.message}") from exc
|
||||
|
||||
content = response.content[0].text
|
||||
return content, response.usage.input_tokens, response.usage.output_tokens
|
||||
|
||||
|
||||
class ProviderConnectionError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class ProviderTimeoutError(Exception):
|
||||
pass
|
||||
@@ -0,0 +1,23 @@
|
||||
from abc import ABC, abstractmethod
|
||||
|
||||
from app.schemas.chat import ChatMessage
|
||||
|
||||
|
||||
class AIProvider(ABC):
|
||||
provider_name: str = "unknown"
|
||||
model_name: str = "unknown"
|
||||
|
||||
@abstractmethod
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ChatMessage],
|
||||
max_tokens: int,
|
||||
temperature: float,
|
||||
) -> tuple[str, int | None, int | None]:
|
||||
"""
|
||||
Send messages to the provider and return (content, input_tokens, output_tokens).
|
||||
Raises:
|
||||
ProviderConnectionError: on network / auth failure
|
||||
ProviderTimeoutError: on request timeout
|
||||
"""
|
||||
...
|
||||
@@ -0,0 +1,52 @@
|
||||
"""OpenAI-compatible provider — handles both Ollama and LM Studio."""
|
||||
import asyncio
|
||||
|
||||
import openai
|
||||
|
||||
from app.providers.base import AIProvider
|
||||
from app.schemas.chat import ChatMessage
|
||||
|
||||
|
||||
class OpenAICompatProvider(AIProvider):
|
||||
def __init__(self, config: dict, provider_name: str = "lmstudio") -> None:
|
||||
self._client = openai.AsyncOpenAI(
|
||||
base_url=config.get("base_url", "http://localhost:1234/v1"),
|
||||
api_key=config.get("api_key") or "not-required",
|
||||
)
|
||||
self.model_name = config.get("model", "local-model")
|
||||
self.provider_name = provider_name
|
||||
|
||||
async def chat(
|
||||
self,
|
||||
messages: list[ChatMessage],
|
||||
max_tokens: int,
|
||||
temperature: float,
|
||||
) -> tuple[str, int | None, int | None]:
|
||||
raw_messages = [{"role": m.role, "content": m.content} for m in messages]
|
||||
try:
|
||||
response = await self._client.chat.completions.create(
|
||||
model=self.model_name,
|
||||
messages=raw_messages,
|
||||
max_tokens=max_tokens,
|
||||
temperature=temperature,
|
||||
)
|
||||
except openai.APIConnectionError as exc:
|
||||
raise ProviderConnectionError(str(exc)) from exc
|
||||
except openai.APITimeoutError as exc:
|
||||
raise ProviderTimeoutError(str(exc)) from exc
|
||||
except openai.APIStatusError as exc:
|
||||
raise ProviderConnectionError(f"API error {exc.status_code}: {exc.message}") from exc
|
||||
|
||||
content = response.choices[0].message.content or ""
|
||||
usage = response.usage
|
||||
input_tokens = usage.prompt_tokens if usage else None
|
||||
output_tokens = usage.completion_tokens if usage else None
|
||||
return content, input_tokens, output_tokens
|
||||
|
||||
|
||||
class ProviderConnectionError(Exception):
|
||||
pass
|
||||
|
||||
|
||||
class ProviderTimeoutError(Exception):
|
||||
pass
|
||||
@@ -0,0 +1,101 @@
|
||||
"""
|
||||
POST /chat — synchronous chat endpoint.
|
||||
|
||||
All requests are submitted to the priority queue at NORMAL priority and the caller
|
||||
waits for the result. This keeps the contract identical to the original endpoint
|
||||
while ensuring all AI traffic flows through one ordered queue.
|
||||
"""
|
||||
import asyncio
|
||||
import re
|
||||
|
||||
from fastapi import APIRouter, HTTPException
|
||||
|
||||
from app.providers import get_provider
|
||||
from app.providers.anthropic_provider import ProviderConnectionError as AnthropicConnError
|
||||
from app.providers.anthropic_provider import ProviderTimeoutError as AnthropicTimeoutError
|
||||
from app.providers.openai_compat import ProviderConnectionError as OpenAIConnError
|
||||
from app.providers.openai_compat import ProviderTimeoutError as OpenAITimeoutError
|
||||
from app.schemas.chat import ChatRequest, ChatResponse
|
||||
from app.services.config_reader import load_ai_config
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_FENCE_RE = re.compile(r"^```[a-z]*\n?(.*?)\n?```$", re.DOTALL)
|
||||
|
||||
|
||||
def _strip_fences(text: str) -> str:
|
||||
m = _FENCE_RE.match(text.strip())
|
||||
return m.group(1).strip() if m else text.strip()
|
||||
|
||||
|
||||
async def execute_chat(request: ChatRequest) -> ChatResponse:
|
||||
"""
|
||||
Core provider call — invoked by the queue worker.
|
||||
Raises HTTPException on provider errors so the queue worker stores the message.
|
||||
"""
|
||||
config = await load_ai_config()
|
||||
|
||||
provider_name = config.get("provider", "lmstudio")
|
||||
if provider_name not in ("anthropic", "ollama", "lmstudio"):
|
||||
raise HTTPException(status_code=503, detail=f"Unknown provider configured: {provider_name!r}")
|
||||
|
||||
try:
|
||||
provider = get_provider(config)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=503, detail=str(exc))
|
||||
|
||||
timeout = config.get("timeout_seconds", 60)
|
||||
max_retries = config.get("max_retries", 2)
|
||||
|
||||
for attempt in range(max_retries + 1):
|
||||
try:
|
||||
content, input_tokens, output_tokens = await asyncio.wait_for(
|
||||
provider.chat(request.messages, request.max_tokens, request.temperature),
|
||||
timeout=float(timeout),
|
||||
)
|
||||
break
|
||||
except asyncio.TimeoutError as exc:
|
||||
raise HTTPException(status_code=504, detail="AI provider timed out") from exc
|
||||
except (AnthropicConnError, OpenAIConnError) as exc:
|
||||
if attempt < max_retries:
|
||||
await asyncio.sleep(0.5 * (attempt + 1))
|
||||
continue
|
||||
raise HTTPException(status_code=502, detail=f"AI provider error: {exc}") from exc
|
||||
except (AnthropicTimeoutError, OpenAITimeoutError) as exc:
|
||||
raise HTTPException(status_code=504, detail="AI provider timed out") from exc
|
||||
|
||||
if request.response_format == "json":
|
||||
content = _strip_fences(content)
|
||||
|
||||
return ChatResponse(
|
||||
content=content,
|
||||
provider=provider.provider_name,
|
||||
model=provider.model_name,
|
||||
input_tokens=input_tokens,
|
||||
output_tokens=output_tokens,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/chat", response_model=ChatResponse)
|
||||
async def chat(request: ChatRequest) -> ChatResponse:
|
||||
"""
|
||||
Submit at NORMAL priority and block until the queue processes the job.
|
||||
If the queue is paused or stopped, the call blocks until resumed (or times out).
|
||||
"""
|
||||
from app.services.queue import Priority, queue_service # deferred — avoids circular import
|
||||
|
||||
job = await queue_service.enqueue(request, Priority.NORMAL)
|
||||
config = await load_ai_config()
|
||||
timeout = float(config.get("timeout_seconds", 60)) + 5.0 # +5s buffer over provider timeout
|
||||
|
||||
try:
|
||||
return await asyncio.wait_for(asyncio.shield(job.future), timeout=timeout)
|
||||
except asyncio.TimeoutError:
|
||||
queue_service.cancel_job(job.id)
|
||||
raise HTTPException(status_code=504, detail="Timed out waiting for queue to process job")
|
||||
except asyncio.CancelledError:
|
||||
raise HTTPException(status_code=503, detail="Job was cancelled")
|
||||
except Exception as exc:
|
||||
if isinstance(exc, HTTPException):
|
||||
raise
|
||||
raise HTTPException(status_code=502, detail=str(exc)) from exc
|
||||
@@ -0,0 +1,30 @@
|
||||
from fastapi import APIRouter
|
||||
|
||||
from app.services.config_reader import load_ai_config
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
|
||||
@router.get("/health")
|
||||
async def health() -> dict:
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@router.get("/health/provider")
|
||||
async def provider_status() -> dict:
|
||||
config = await load_ai_config()
|
||||
provider = config.get("provider", "lmstudio")
|
||||
pcfg = config.get(provider, {})
|
||||
model = pcfg.get("model", "")
|
||||
|
||||
# "configured" means we have the minimum required fields for the provider
|
||||
if provider == "anthropic":
|
||||
configured = bool(pcfg.get("api_key"))
|
||||
else:
|
||||
configured = bool(pcfg.get("base_url") and pcfg.get("model"))
|
||||
|
||||
return {
|
||||
"provider": provider,
|
||||
"model": model,
|
||||
"configured": configured,
|
||||
}
|
||||
@@ -0,0 +1,31 @@
|
||||
"""
|
||||
Plugin manifest endpoint for the AI service.
|
||||
|
||||
Exposes GET /plugin/manifest so the backend health-poller can discover the
|
||||
service's access rules and register it in the plugin system.
|
||||
|
||||
No settings schema is exposed here — the AI service settings are complex
|
||||
(provider selection, conditional fields) and are rendered by a bespoke page
|
||||
rather than the generic PluginSchemaForm.
|
||||
"""
|
||||
from fastapi import APIRouter
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
_MANIFEST = {
|
||||
"id": "ai-service",
|
||||
"name": "AI Service",
|
||||
"icon": "cpu",
|
||||
"version": "1.0",
|
||||
"access": {
|
||||
"allow_superuser": True,
|
||||
"required_groups": ["ai-service-admin"],
|
||||
},
|
||||
# No settings_schema — the frontend uses a custom settings page
|
||||
"settings_schema": None,
|
||||
}
|
||||
|
||||
|
||||
@router.get("/plugin/manifest")
|
||||
async def get_manifest() -> dict:
|
||||
return _MANIFEST
|
||||
@@ -0,0 +1,104 @@
|
||||
"""
|
||||
Queue management router.
|
||||
|
||||
POST /queue/jobs — enqueue a job, return immediately with job metadata
|
||||
GET /queue/jobs/{id} — poll job status / result
|
||||
DELETE /queue/jobs/{id} — cancel a pending job
|
||||
|
||||
GET /queue/status — worker state + queue depth
|
||||
POST /queue/pause — finish current job, stop picking new ones
|
||||
POST /queue/resume — resume from pause
|
||||
POST /queue/start — start (or restart) the worker
|
||||
POST /queue/stop — stop worker immediately (pending jobs stay queued)
|
||||
"""
|
||||
from fastapi import APIRouter, HTTPException
|
||||
|
||||
from app.schemas.queue import JobStatus, QueueRequest, QueueStatus
|
||||
from app.services.queue import PRIORITY_MAP, Job, Priority, queue_service
|
||||
|
||||
router = APIRouter(prefix="/queue", tags=["queue"])
|
||||
|
||||
|
||||
# ── Job endpoints ─────────────────────────────────────────────────────────────
|
||||
|
||||
@router.post("/jobs", response_model=JobStatus, status_code=202)
|
||||
async def enqueue_job(request: QueueRequest) -> JobStatus:
|
||||
priority = PRIORITY_MAP[request.priority]
|
||||
job = await queue_service.enqueue(request, priority)
|
||||
return _job_to_status(job)
|
||||
|
||||
|
||||
@router.get("/jobs/{job_id}", response_model=JobStatus)
|
||||
async def get_job(job_id: str) -> JobStatus:
|
||||
job = queue_service.get_job(job_id)
|
||||
if not job:
|
||||
raise HTTPException(status_code=404, detail="Job not found")
|
||||
return _job_to_status(job)
|
||||
|
||||
|
||||
@router.delete("/jobs/{job_id}", status_code=204)
|
||||
async def cancel_job(job_id: str) -> None:
|
||||
if not queue_service.cancel_job(job_id):
|
||||
raise HTTPException(status_code=404, detail="Job not found or already started")
|
||||
|
||||
|
||||
# ── Worker control endpoints ──────────────────────────────────────────────────
|
||||
|
||||
@router.get("/status", response_model=QueueStatus)
|
||||
async def get_status() -> QueueStatus:
|
||||
cur = queue_service.current_job
|
||||
return QueueStatus(
|
||||
running=queue_service._running,
|
||||
paused=queue_service.is_paused,
|
||||
queue_size=queue_service.queue_size,
|
||||
current_job_id=cur.id if cur else None,
|
||||
)
|
||||
|
||||
|
||||
@router.post("/pause", status_code=204)
|
||||
async def pause() -> None:
|
||||
"""Pause after the current job finishes."""
|
||||
queue_service.pause()
|
||||
|
||||
|
||||
@router.post("/resume", status_code=204)
|
||||
async def resume() -> None:
|
||||
"""Resume from a paused state."""
|
||||
queue_service.resume()
|
||||
|
||||
|
||||
@router.post("/start", status_code=204)
|
||||
async def start() -> None:
|
||||
"""Start (or restart) the worker task."""
|
||||
queue_service.start()
|
||||
|
||||
|
||||
@router.post("/stop", status_code=204)
|
||||
async def stop() -> None:
|
||||
"""Stop the worker. Pending jobs remain in queue; POST /queue/start to resume."""
|
||||
queue_service.stop()
|
||||
|
||||
|
||||
# ── Helper ────────────────────────────────────────────────────────────────────
|
||||
|
||||
def _job_to_status(job: Job) -> JobStatus:
|
||||
pos: int | None = None
|
||||
if job.status == "pending":
|
||||
# Count jobs that are ahead: same or higher priority AND earlier seq
|
||||
pos = sum(
|
||||
1
|
||||
for j in queue_service._jobs.values()
|
||||
if j.status == "pending"
|
||||
and (int(j.priority), j.seq) < (int(job.priority), job.seq)
|
||||
)
|
||||
return JobStatus(
|
||||
id=job.id,
|
||||
status=job.status,
|
||||
priority=Priority(job.priority).name.lower(),
|
||||
position=pos,
|
||||
created_at=job.created_at,
|
||||
started_at=job.started_at,
|
||||
finished_at=job.finished_at,
|
||||
result=job.result,
|
||||
error=job.error,
|
||||
)
|
||||
@@ -0,0 +1,30 @@
|
||||
from typing import Literal
|
||||
|
||||
from pydantic import BaseModel, field_validator
|
||||
|
||||
|
||||
class ChatMessage(BaseModel):
|
||||
role: Literal["system", "user", "assistant"]
|
||||
content: str
|
||||
|
||||
|
||||
class ChatRequest(BaseModel):
|
||||
messages: list[ChatMessage]
|
||||
max_tokens: int = 2048
|
||||
temperature: float = 0.0
|
||||
response_format: Literal["json", "text"] = "text"
|
||||
|
||||
@field_validator("messages")
|
||||
@classmethod
|
||||
def messages_not_empty(cls, v: list) -> list:
|
||||
if not v:
|
||||
raise ValueError("messages must not be empty")
|
||||
return v
|
||||
|
||||
|
||||
class ChatResponse(BaseModel):
|
||||
content: str
|
||||
provider: str
|
||||
model: str
|
||||
input_tokens: int | None = None
|
||||
output_tokens: int | None = None
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user