Compare commits
5 Commits
cfec3bb906
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 0f760c379d | |||
| f13ef88711 | |||
| 0d8e0366c6 | |||
| 3a66aeeec5 | |||
| 248b2bb9d7 |
@@ -292,8 +292,9 @@ Each entry must include:
|
||||
4. Add router in `backend/app/routers/`, mount it in `main.py`
|
||||
5. Add API function(s) to `frontend/src/api/client.ts`
|
||||
6. Add page component in `frontend/src/pages/`, register route in `App.tsx`
|
||||
7. Update `STATUS.md` for affected services
|
||||
8. Add changelog entry
|
||||
7. If the resource involves file or blob data: store it via `PUT /objects/{bucket}/{key}` on `storage-service:8020`. Never write to the local filesystem. See `features/storage-service/CLAUDE.md` for the API.
|
||||
8. Update `STATUS.md` for affected services
|
||||
9. Add changelog entry
|
||||
|
||||
---
|
||||
|
||||
@@ -329,23 +330,18 @@ git checkout -b feat/<descriptive-title> # e.g. feat/user-profile-avatar-uploa
|
||||
```
|
||||
|
||||
#### 2 — Spin up an isolated Docker stack for the feature
|
||||
A dedicated compose stack runs alongside the main dev stack so both can be tested independently.
|
||||
The feature stack always uses port `5173` (same as the main dev stack). Stop the main stack before starting a feature stack, and restart it when done.
|
||||
|
||||
**Find the next free port** (main dev stack owns 5173):
|
||||
**Stop the main dev stack first:**
|
||||
```bash
|
||||
for port in $(seq 5174 5200); do
|
||||
lsof -iTCP:$port -sTCP:LISTEN -t &>/dev/null || { echo "$port"; break; }
|
||||
done
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml down
|
||||
```
|
||||
Use the first free port returned (call it `$PORT`).
|
||||
|
||||
**Create a per-feature override file** at `docker-compose.feat-<slug>.yml` (gitignored):
|
||||
```yaml
|
||||
# docker-compose.feat-<slug>.yml — feature test stack, never committed to main
|
||||
services:
|
||||
frontend:
|
||||
ports:
|
||||
- "$PORT:8080" # e.g. 5174:8080
|
||||
container_name: frontend-<slug>
|
||||
backend:
|
||||
container_name: backend-<slug>
|
||||
@@ -371,8 +367,7 @@ docker compose -f docker-compose.yml \
|
||||
--project-name <slug> up --build
|
||||
```
|
||||
|
||||
The feature frontend is now reachable at `http://localhost:$PORT`.
|
||||
The main dev stack continues running unaffected on `:5173`.
|
||||
The feature frontend is now reachable at `http://localhost:5173`.
|
||||
|
||||
#### 3 — Develop on the feature branch
|
||||
All code changes happen on `feat/<slug>`. Commit and push normally:
|
||||
@@ -383,7 +378,7 @@ git push -u origin feat/<slug>
|
||||
```
|
||||
|
||||
#### 4 — Confirm functionality
|
||||
Before merging, verify all of the following on `http://localhost:$PORT`:
|
||||
Before merging, verify all of the following on `http://localhost:5173`:
|
||||
- [ ] Login and registration work end-to-end
|
||||
- [ ] The specific feature works as intended
|
||||
- [ ] No regressions visible in the UI
|
||||
@@ -400,13 +395,16 @@ git branch -d feat/<slug>
|
||||
git push origin --delete feat/<slug>
|
||||
```
|
||||
|
||||
#### 6 — Tear down the feature stack
|
||||
#### 6 — Tear down the feature stack and restart main dev stack
|
||||
```bash
|
||||
docker compose -f docker-compose.yml \
|
||||
-f docker-compose.dev.yml \
|
||||
-f docker-compose.feat-<slug>.yml \
|
||||
--project-name <slug> down --volumes --remove-orphans
|
||||
rm docker-compose.feat-<slug>.yml
|
||||
|
||||
# Restart the main dev stack on :5173
|
||||
docker compose -f docker-compose.yml -f docker-compose.dev.yml up --build -d
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -20,10 +20,11 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
- All input sanitized before reaching the DB (null-byte rejection, length caps, format validation)
|
||||
- **PDF Documents app** (`/apps/documents`): upload PDFs, async text extraction (pdfplumber), AI classification via ai-service, per-user categories, file download
|
||||
- **AI Service** (`ai-service:8010`): shared AI intermediary container; routes prompts to Anthropic / Ollama / LM Studio; stateless; all feature containers talk to it via `POST /chat`
|
||||
- Admin settings: `/apps/ai/settings/admin` (provider, credentials, test connection); `/apps/documents/settings/admin` (upload limits only)
|
||||
- Config stored in shared Docker volume: `/config/ai_service_config.json` and `/config/doc_service_config.json`
|
||||
- **Storage Service** (`storage-service:8020`): unified file/blob storage with pluggable backends (local filesystem default; S3-compatible and WebDAV built in); backend switchable via admin UI with zero-data-loss migration
|
||||
- Admin settings: AI provider, doc upload limits, storage backend switching with live migration progress
|
||||
- Config stored in storage-service (`config` bucket); PDFs stored in storage-service (`documents` bucket) — no shared filesystem volumes
|
||||
- `/apps` launcher hub — one card per installed app with Open + Settings links
|
||||
- 5 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `frontend`
|
||||
- 6 separate Docker containers: `db`, `backend`, `ai-service`, `doc-service`, `storage-service`, `frontend`
|
||||
- All containers run as non-root users (UID 1001 for app containers, UID 70 for db)
|
||||
- Network-isolated: only the frontend exposes a host port (80/5173); all backend services are unreachable from outside Docker
|
||||
- Dev environment seeds a test user automatically on startup (`test@example.com` / `Test123!`)
|
||||
@@ -38,6 +39,7 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
| `backend` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | FastAPI management API + proxy to doc-service |
|
||||
| `ai-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Shared AI intermediary (routes to LM Studio / Ollama / Anthropic) |
|
||||
| `doc-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | PDF extraction microservice (calls ai-service) |
|
||||
| `storage-service` | custom (python:3.12-slim) | none | backend-net | 1001:1001 | Unified file/blob storage (local / S3-compatible / WebDAV) |
|
||||
| `frontend` | custom (nginxinc/nginx-unprivileged:alpine) | 80 (prod) / 5173 (dev) | backend-net + frontend-net | 1001:1001 | React UI + nginx reverse proxy |
|
||||
|
||||
**Networks:**
|
||||
@@ -46,8 +48,8 @@ A fullstack SaaS web application built with FastAPI, React, and PostgreSQL.
|
||||
|
||||
**Volumes:**
|
||||
- `postgres_data` — PostgreSQL data files
|
||||
- `doc_data` — uploaded PDF files (mounted into doc-service at `/data/documents`)
|
||||
- `app_config` — per-service runtime config JSON files (mounted into backend, ai-service, and doc-service at `/config`)
|
||||
- `storage_data` — all file/blob storage: uploaded PDFs (`documents/` bucket) and service config JSON files (`config/` bucket); mounted into storage-service at `/data/storage`
|
||||
- `watch_data` — file watcher input directory; mounted into doc-service at `/data/watch`
|
||||
|
||||
The frontend nginx proxies `/api/*` to `backend:8000` via `backend-net`. The backend proxies `/api/documents/*` and `/api/documents/categories/*` to `doc-service:8001`. The backend test-connection endpoint proxies to `ai-service:8010`. No backend service or database port is ever exposed to the host.
|
||||
|
||||
@@ -126,10 +128,8 @@ Copy `.env.example` to `backend/.env` and adjust:
|
||||
| `JWT_PRIVATE_KEY` | — | RS256 private key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `JWT_PUBLIC_KEY` | — | RS256 public key PEM (generate with `scripts/generate_jwt_keys.py`) |
|
||||
| `CORS_ORIGINS` | `["http://localhost:5173"]` | Allowed frontend origins |
|
||||
| `APP_CONFIG_DIR` | `/config` | Directory for per-service runtime config JSON files |
|
||||
| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of the doc-service (set by docker-compose) |
|
||||
|
||||
`doc-service` reads `DATABASE_URL`, `DATA_DIR`, and `CONFIG_PATH` from its own environment (set in `docker-compose.yml`).
|
||||
| `DOC_SERVICE_URL` | `http://doc-service:8001` | Internal URL of doc-service (set by docker-compose) |
|
||||
| `STORAGE_SERVICE_URL` | `http://storage-service:8020` | Internal URL of storage-service (set by docker-compose) |
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
@@ -15,8 +15,7 @@ FROM python:3.12-slim
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# Pre-create the config directory with correct ownership
|
||||
RUN mkdir -p /config && chown -R appuser:appuser /config
|
||||
# No filesystem directories needed — all config goes through storage-service.
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -15,9 +15,9 @@ FROM python:3.12-slim
|
||||
RUN groupadd --gid 1001 appuser && \
|
||||
useradd --uid 1001 --gid 1001 --no-create-home --shell /bin/sh appuser
|
||||
|
||||
# Pre-create data and config dirs with correct ownership.
|
||||
# Named volumes mounted over these paths will inherit ownership on first creation.
|
||||
RUN mkdir -p /data/documents /data/watch /config && chown -R appuser:appuser /data /config
|
||||
# Pre-create watch dir with correct ownership.
|
||||
# /data/documents and /config are no longer used — all file/config storage goes through storage-service.
|
||||
RUN mkdir -p /data/watch && chown -R appuser:appuser /data
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
|
||||
@@ -29,7 +29,7 @@ from app.schemas.document import (
|
||||
from app.schemas.share import DocumentShareCreate, DocumentShareOut, SharedDocumentOut
|
||||
from app.services.ai_client import AIServiceError, classify_document
|
||||
from app.services.config_reader import load_doc_config
|
||||
from app.services.storage import delete_file, download_file, save_upload
|
||||
from app.services.storage import delete_file, download_file as storage_download, save_upload
|
||||
|
||||
router = APIRouter()
|
||||
|
||||
@@ -146,7 +146,7 @@ async def process_document(doc_id: str) -> None:
|
||||
await db.commit()
|
||||
|
||||
try:
|
||||
pdf_bytes = await download_file(doc.storage_key)
|
||||
pdf_bytes = await storage_download(doc.storage_key)
|
||||
text = await asyncio.to_thread(_extract_pdf_text, pdf_bytes)
|
||||
result = await classify_document(text)
|
||||
|
||||
@@ -611,7 +611,7 @@ async def download_file(
|
||||
raise HTTPException(status_code=404, detail="Document not found")
|
||||
|
||||
try:
|
||||
pdf_bytes = await download_file(doc.storage_key)
|
||||
pdf_bytes = await storage_download(doc.storage_key)
|
||||
except FileNotFoundError:
|
||||
raise HTTPException(status_code=404, detail="File not found in storage")
|
||||
|
||||
|
||||
@@ -1,11 +1,10 @@
|
||||
import { useState, useEffect, useRef } from "react";
|
||||
import { useState } from "react";
|
||||
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
|
||||
import {
|
||||
getStorageConfig,
|
||||
getMigrationStatus,
|
||||
startStorageMigration,
|
||||
cancelMigration,
|
||||
updateStorageConfig,
|
||||
type StorageBackendConfig,
|
||||
type MigrationStatus,
|
||||
} from "../api/client";
|
||||
|
||||
Reference in New Issue
Block a user