feat: Phase 4+5 — admin storage UI, backend proxy, CLAUDE.md enforcement
- backend/app/routers/storage_config.py: 5 admin-only endpoints proxying storage-service config + migration API (GET/PATCH/POST/DELETE) - backend/app/main.py: register storage_config router - frontend/src/api/client.ts: StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions - frontend/src/pages/StorageAdminPage.tsx: full admin UI — backend health dot, driver selector (local/S3/WebDAV), conditional credential fields, Test & Migrate button, live 2s-poll migration progress bar, Cancel - frontend/src/App.tsx: /admin/storage route (AdminRoute guard) - CLAUDE.md: storage enforcement rule, updated Docker tables (6 services, 3 volumes), §20 in merge checklist - backend/CLAUDE.md, frontend/CLAUDE.md, doc-service/CLAUDE.md, ai-service/CLAUDE.md: updated to reflect storage-service integration - tests/ALL_TESTS.md + tests/storage-service_tests.md: §20 (20 tests) - backend/STATUS.md, frontend/STATUS.md: updated with new endpoints/routes - changelog/2026-04-20_storage-service.md: full change log Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
curo1305
@@ -6,17 +6,19 @@ This file provides permanent, authoritative guidance to Claude Code for every se
|
||||
- `frontend/CLAUDE.md` — routes, components, API client patterns, XSS prevention
|
||||
- `features/ai-service/CLAUDE.md` — /chat, /health, /queue endpoints; queue service
|
||||
- `features/doc-service/CLAUDE.md` — document/category/share endpoints; DB models; PDF limits; file watcher
|
||||
- `features/storage-service/CLAUDE.md` — storage API, pluggable backend drivers (local/S3/WebDAV), migration
|
||||
|
||||
---
|
||||
|
||||
## Merge checklist
|
||||
|
||||
Before merging any feature branch into `main`, every test relevant to the changed area in `tests/ALL_TESTS.md` (and the relevant service-specific file) must be marked passing. The test suite covers all 19 feature areas across four service files:
|
||||
Before merging any feature branch into `main`, every test relevant to the changed area in `tests/ALL_TESTS.md` (and the relevant service-specific file) must be marked passing. The test suite covers all 20 feature areas across five service files:
|
||||
|
||||
- `tests/backend_tests.md` — §1–9, §18
|
||||
- `tests/frontend_tests.md` — §19
|
||||
- `tests/doc-service_tests.md` — §10–16
|
||||
- `tests/ai-service_tests.md` — §17
|
||||
- `tests/storage-service_tests.md` — §20
|
||||
|
||||
Do not merge without it.
|
||||
|
||||
@@ -35,7 +37,7 @@ Do not merge without it.
|
||||
- New Docker service, volume, network, or env var → update **Docker Infrastructure** in this file
|
||||
- Stack version changed → update **Stack** in this file
|
||||
|
||||
- New feature or endpoint added → add test rows to **both** `tests/ALL_TESTS.md` (in the relevant section) **and** the matching service-specific file (`tests/backend_tests.md`, `tests/frontend_tests.md`, `tests/doc-service_tests.md`, or `tests/ai-service_tests.md`). Use the same test number and format as existing rows.
|
||||
- New feature or endpoint added → add test rows to **both** `tests/ALL_TESTS.md` (in the relevant section) **and** the matching service-specific file (`tests/backend_tests.md`, `tests/frontend_tests.md`, `tests/doc-service_tests.md`, `tests/ai-service_tests.md`, or `tests/storage-service_tests.md`). Use the same test number and format as existing rows.
|
||||
|
||||
This check is mandatory — treat it the same as updating STATUS.md.
|
||||
|
||||
@@ -143,7 +145,17 @@ These standards are **non-negotiable**. Every change must comply. Implementation
|
||||
|
||||
- `backend-net`: all containers except frontend; not reachable from host in prod.
|
||||
- `frontend-net`: only frontend; single host port (80 prod / 5173 dev).
|
||||
- DB, backend, doc-service, ai-service have **no** host port bindings in prod.
|
||||
- DB, backend, doc-service, ai-service, storage-service have **no** host port bindings in prod.
|
||||
|
||||
### Storage rule (non-negotiable)
|
||||
|
||||
**No service may write to a filesystem path for persistent data.** All file/blob storage must go through the storage-service HTTP API (`PUT/GET/DELETE /objects/{bucket}/{key}`). Config JSON files must be stored in the `config` bucket. Uploaded files must be stored in the `documents` bucket. Violation is a security and architecture defect.
|
||||
|
||||
The only two persistent storage mechanisms in the project are:
|
||||
1. **PostgreSQL** — structured/relational data
|
||||
2. **storage-service** — all file/blob/config data (local filesystem by default; switchable to S3-compatible or WebDAV)
|
||||
|
||||
New services and features must follow this pattern. See `features/storage-service/CLAUDE.md` for the API reference.
|
||||
|
||||
### Pre-commit security hook
|
||||
|
||||
@@ -179,9 +191,10 @@ All other per-service defaults are in the relevant sub-CLAUDE.md file.
|
||||
| Service | Image base | Internal port | User | Volumes | Network |
|
||||
|---------|-----------|---------------|------|---------|---------|
|
||||
| `db` | postgres:16-alpine | 5432 | 70:70 | `postgres_data` | backend-net |
|
||||
| `backend` | python:3.12-slim | 8000 | 1001:1001 | `app_config` | backend-net |
|
||||
| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | `app_config` | backend-net |
|
||||
| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `doc_data`, `watch_data`, `app_config` | backend-net |
|
||||
| `backend` | python:3.12-slim | 8000 | 1001:1001 | — | backend-net |
|
||||
| `ai-service` | python:3.12-slim | 8010 | 1001:1001 | — | backend-net |
|
||||
| `doc-service` | python:3.12-slim | 8001 | 1001:1001 | `watch_data` | backend-net |
|
||||
| `storage-service` | python:3.12-slim | 8020 | 1001:1001 | `storage_data` | backend-net |
|
||||
| `frontend` | nginx-unprivileged:alpine | 8080 | 1001:1001 | — | backend-net, frontend-net |
|
||||
|
||||
### Volumes
|
||||
@@ -189,15 +202,14 @@ All other per-service defaults are in the relevant sub-CLAUDE.md file.
|
||||
| Volume | Mount path | Contains |
|
||||
|--------|-----------|---------|
|
||||
| `postgres_data` | `/var/lib/postgresql/data` | PostgreSQL data |
|
||||
| `doc_data` | `/data/documents` | Uploaded PDF files |
|
||||
| `storage_data` | `/data/storage` | All file/blob storage: PDFs (`documents/`) and config JSONs (`config/`) |
|
||||
| `watch_data` | `/data/watch` | Watch directory (bind-mount NAS/Nextcloud via docker-compose.override.yml) |
|
||||
| `app_config` | `/config` | Per-service runtime config JSON files |
|
||||
|
||||
### Networks
|
||||
|
||||
| Network | Host-accessible | Members |
|
||||
|---------|----------------|---------|
|
||||
| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, frontend |
|
||||
| `backend-net` | No (no host ports in prod) | db, backend, ai-service, doc-service, storage-service, frontend |
|
||||
| `frontend-net` | Yes (port 80 → frontend:8080) | frontend |
|
||||
|
||||
### Environment variables (required in `backend/.env`)
|
||||
@@ -213,6 +225,7 @@ Injected by docker-compose (not in `.env`):
|
||||
```
|
||||
DOC_SERVICE_URL=http://doc-service:8001
|
||||
AI_SERVICE_URL=http://ai-service:8010
|
||||
STORAGE_SERVICE_URL=http://storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
+14
-2
@@ -36,7 +36,8 @@ backend/
|
||||
│ │ ├── config.py ← All settings via pydantic-settings (reads .env)
|
||||
│ │ ├── security.py ← JWT sign/verify (RS256), bcrypt hash/verify
|
||||
│ │ ├── sanitize.py ← Input sanitization helpers (see Security Standards)
|
||||
│ │ └── app_config.py ← Per-service config load/save to /config volume; theme files in /config/themes/
|
||||
│ │ ├── app_config.py ← Per-service config load/save via storage-service; theme files in config/themes/{id}.json
|
||||
│ │ └── config_storage.py ← Thin async HTTP helpers: read_json/write_json/delete_key/list_keys → storage-service config bucket
|
||||
│ ├── models/
|
||||
│ │ ├── __init__.py ← Imports all models (required for Alembic autogenerate)
|
||||
│ │ ├── user.py ← User model
|
||||
@@ -56,7 +57,8 @@ backend/
|
||||
│ │ ├── services.py ← GET /services (health status)
|
||||
│ │ ├── plugins.py ← Generic plugin proxy (GET/PATCH /api/plugins/*)
|
||||
│ │ ├── categories_proxy.py ← Transparent proxy → doc-service /categories/*
|
||||
│ │ └── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ ├── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ └── storage_config.py ← Admin proxy → storage-service config + migration endpoints
|
||||
│ └── services/
|
||||
│ ├── service_health.py ← Background 30s health-check loop; caches /plugin/manifest per service
|
||||
│ └── group_bootstrap.py ← Ensures {service-id}-admin group exists for every registered service at startup
|
||||
@@ -216,6 +218,16 @@ Unique constraint: `(group_id, user_id)`
|
||||
|
||||
Auth: is_superuser OR member of group listed in manifest `required_groups`. Returns 404 (not 403) to hide existence.
|
||||
|
||||
### Admin — Storage (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/storage-config` | Current backend driver + health → proxied from storage-service `/health` |
|
||||
| PATCH | `/api/admin/storage-config` | Reconfigure backend without data migration (same-backend credential update) |
|
||||
| POST | `/api/admin/storage-config/migrate` | Start async migration to a new backend (copy → verify → switch → cleanup) |
|
||||
| GET | `/api/admin/storage-config/migrate/status` | Poll migration progress: `{state, total, done, failed, errors[]}` |
|
||||
| DELETE | `/api/admin/storage-config/migrate` | Cancel a running migration; old backend remains active |
|
||||
|
||||
### Documents and Categories — proxied
|
||||
|
||||
`/api/documents/*` and `/api/documents/categories/*` are transparently proxied to `doc-service:8001`. The backend injects `x-user-id`, `x-user-groups`, and `x-user-is-admin` headers. See `features/doc-service/CLAUDE.md` for the internal endpoint list.
|
||||
|
||||
+16
-3
@@ -75,10 +75,20 @@ A background task (`service_health.py`) polls each service's `/health` endpoint
|
||||
| `GET` | `/api/settings/system-prompts` | All editable system prompts — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/system-prompts/{id}` | Update system prompt — same access |
|
||||
|
||||
Settings are persisted to JSON files on the `app_config` Docker named volume and read by the respective feature services.
|
||||
Settings are persisted to the `config` bucket of `storage-service:8020` via `core/config_storage.py`. All config I/O is async HTTP; no filesystem volumes are used.
|
||||
|
||||
Access to service-specific settings endpoints is enforced by `get_service_admin(service_id)` in `deps.py` — grants access to superusers OR members of the `{service_id}-admin` group.
|
||||
|
||||
### Storage config (`/api/admin`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/storage-config` | Current backend driver + health (proxied from storage-service) |
|
||||
| `PATCH` | `/api/admin/storage-config` | Reconfigure backend without migration |
|
||||
| `POST` | `/api/admin/storage-config/migrate` | Start async migration to a new backend |
|
||||
| `GET` | `/api/admin/storage-config/migrate/status` | Poll migration progress |
|
||||
| `DELETE` | `/api/admin/storage-config/migrate` | Cancel running migration |
|
||||
|
||||
### Feature proxies
|
||||
|
||||
All `/api/documents/*` and `/api/documents/categories/*` requests are transparently proxied to `doc-service:8001` via `httpx.AsyncClient`. The proxy:
|
||||
@@ -129,8 +139,11 @@ Browser (port 5173 dev / 80 prod)
|
||||
┌───────────┼────────────┬──────────────┐
|
||||
/auth /settings /documents/* /services
|
||||
/users (JSON │ │
|
||||
/admin volume) └── proxy → health-check loop
|
||||
/profile doc-service:8001 (30s poll)
|
||||
/admin /storage- └── proxy → health-check loop
|
||||
/profile config doc-service:8001 (30s poll)
|
||||
(proxy)
|
||||
│
|
||||
storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -9,6 +9,7 @@ from app.core.config import settings
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.routers import admin, auth, categories_proxy, documents_proxy, groups, plugins, profile, services, users
|
||||
from app.routers import settings as settings_router
|
||||
from app.routers import storage_config
|
||||
from app.services.group_bootstrap import ensure_service_admin_groups
|
||||
from app.services.service_health import check_all, health_check_loop, register_services
|
||||
|
||||
@@ -52,6 +53,7 @@ app.include_router(admin.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(groups.router, prefix="/api/admin/groups", tags=["admin"])
|
||||
app.include_router(settings_router.router, prefix="/api/settings", tags=["settings"])
|
||||
app.include_router(services.router, prefix="/api/services", tags=["services"])
|
||||
app.include_router(storage_config.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(plugins.router, prefix="/api/plugins", tags=["plugins"])
|
||||
# categories_proxy MUST be registered before documents_proxy —
|
||||
# otherwise /api/documents/{path:path} swallows /api/documents/categories/*
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""
|
||||
Admin-only endpoints for storage-service backend configuration.
|
||||
|
||||
GET /admin/storage-config — current backend driver + health
|
||||
PATCH /admin/storage-config — update backend config (no data migration)
|
||||
POST /admin/storage-config/migrate — start migration to a new backend
|
||||
GET /admin/storage-config/migrate/status — poll migration progress
|
||||
DELETE /admin/storage-config/migrate — cancel in-progress migration
|
||||
|
||||
All endpoints proxy to storage-service:8020.
|
||||
"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_STORAGE_BASE = settings.STORAGE_SERVICE_URL
|
||||
|
||||
|
||||
class BackendConfigUpdate(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
class MigrateRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
def _storage_url(path: str) -> str:
|
||||
return f"{_STORAGE_BASE}{path}"
|
||||
|
||||
|
||||
async def _proxy_get(path: str) -> dict:
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.get(_storage_url(path))
|
||||
if resp.status_code == 404:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config")
|
||||
async def get_storage_config(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Return current backend driver and health status."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url("/health"))
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.patch("/storage-config", status_code=204)
|
||||
async def update_storage_config(
|
||||
body: BackendConfigUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""
|
||||
Reconfigure the active backend without migrating data.
|
||||
Use when changing credentials for the same backend type, or reverting to local.
|
||||
To move data to a new backend, use POST /admin/storage-config/migrate instead.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.patch(
|
||||
_storage_url("/backend-config"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="Migration in progress — cannot reconfigure now")
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
@router.post("/storage-config/migrate", status_code=202)
|
||||
async def start_migration(
|
||||
body: MigrateRequest,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""
|
||||
Start an async migration to a new backend.
|
||||
|
||||
Flow: validate new backend → copy all objects → verify → switch → delete old objects.
|
||||
The old backend stays active until 100% of objects are verified on the new one.
|
||||
Poll GET /admin/storage-config/migrate/status to track progress.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
_storage_url("/migrate"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="A migration is already in progress")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config/migrate/status")
|
||||
async def migration_status(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Poll migration progress. State: idle → validating → migrating → switching → cleaning → done."""
|
||||
return await _proxy_get("/migrate/status")
|
||||
|
||||
|
||||
@router.delete("/storage-config/migrate", status_code=204)
|
||||
async def cancel_migration(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""Cancel a running migration. The old backend remains active."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.delete(_storage_url("/migrate"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="No cancellable migration in progress")
|
||||
resp.raise_for_status()
|
||||
@@ -0,0 +1,60 @@
|
||||
# 2026-04-20 — Dedicated storage-service with pluggable backends
|
||||
|
||||
**Timestamp:** 2026-04-20T00:00:00Z
|
||||
|
||||
## Summary
|
||||
|
||||
Introduced a dedicated `storage-service` container (port 8020) as the single file/blob persistence layer for the entire stack. All services now route file and config I/O through this service's HTTP API. The service supports pluggable storage backends (local filesystem by default; S3-compatible and WebDAV built in) with a zero-data-loss migration flow. The `doc_data` and `app_config` Docker volumes were removed.
|
||||
|
||||
## Files Added
|
||||
|
||||
- `features/storage-service/app/main.py` — FastAPI app, lifespan (backend init)
|
||||
- `features/storage-service/app/core/config.py` — Settings (DATA_DIR, STORAGE_BACKEND, S3_*, WEBDAV_*)
|
||||
- `features/storage-service/app/routers/health.py` — GET /health
|
||||
- `features/storage-service/app/routers/objects.py` — PUT/GET/DELETE /objects/{bucket}/{key:path}, GET /objects/{bucket}
|
||||
- `features/storage-service/app/routers/migrate.py` — POST/GET/DELETE /migrate, PATCH /backend-config
|
||||
- `features/storage-service/app/services/backend_manager.py` — Driver factory, singleton, atomic switch
|
||||
- `features/storage-service/app/services/migration.py` — Async migration: copy → verify → switch → cleanup
|
||||
- `features/storage-service/app/services/backends/base.py` — AbstractStorageBackend ABC
|
||||
- `features/storage-service/app/services/backends/local.py` — LocalFSBackend (path traversal guard)
|
||||
- `features/storage-service/app/services/backends/s3.py` — S3Backend (aiobotocore, endpoint_url configurable)
|
||||
- `features/storage-service/app/services/backends/webdav.py` — WebDAVBackend (aiohttp + defusedxml)
|
||||
- `features/storage-service/scripts/start.sh` — prod uvicorn start
|
||||
- `features/storage-service/scripts/start_dev.sh` — dev uvicorn --reload start
|
||||
- `features/storage-service/pyproject.toml` — Dependencies
|
||||
- `features/storage-service/Dockerfile` — python:3.12-slim, non-root user 1001, port 8020
|
||||
- `features/storage-service/CLAUDE.md` — API reference, bucket docs, driver docs
|
||||
- `features/storage-service/STATUS.md` — Service status
|
||||
- `backend/app/core/config_storage.py` — Thin async helpers: read_json/write_json/delete_key/list_keys
|
||||
- `backend/app/routers/storage_config.py` — Admin proxy endpoints for storage config + migration
|
||||
- `features/doc-service/alembic/versions/0008_rename_file_path_to_storage_key.py` — DB migration
|
||||
- `frontend/src/pages/StorageAdminPage.tsx` — Admin UI: backend status, driver form, migration progress
|
||||
- `tests/storage-service_tests.md` — §20 storage-service test suite
|
||||
|
||||
## Files Modified
|
||||
|
||||
- `docker-compose.yml` — Added storage-service, storage_data volume; removed doc_data, app_config; added depends_on service_healthy
|
||||
- `docker-compose.dev.yml` — Added storage-service dev override
|
||||
- `backend/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `backend/app/core/app_config.py` — Full async rewrite using config_storage HTTP helpers (no filesystem)
|
||||
- `backend/app/routers/settings.py` — Removed all asyncio.to_thread wrappers; direct await calls
|
||||
- `backend/app/main.py` — Register storage_config router; update register_services call
|
||||
- `backend/app/services/service_health.py` — Register storage-service
|
||||
- `features/doc-service/app/core/config.py` — Added STORAGE_SERVICE_URL
|
||||
- `features/doc-service/app/models/document.py` — file_path → storage_key
|
||||
- `features/doc-service/app/services/storage.py` — Complete rewrite: HTTP client calls to storage-service
|
||||
- `features/doc-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `features/doc-service/app/services/file_watcher.py` — Uses save_upload() → storage-service
|
||||
- `features/doc-service/app/routers/documents.py` — storage_key refs, pdfplumber(io.BytesIO), streaming from storage-service
|
||||
- `features/ai-service/app/core/config.py` — Added STORAGE_SERVICE_URL; removed CONFIG_PATH
|
||||
- `features/ai-service/app/services/config_reader.py` — Complete rewrite: reads/writes via storage-service config bucket
|
||||
- `frontend/src/api/client.ts` — Added StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions
|
||||
- `frontend/src/App.tsx` — Added /admin/storage route (AdminRoute → StorageAdminPage)
|
||||
- `tests/ALL_TESTS.md` — Updated to 20 feature areas; added §20 storage-service tests
|
||||
- `CLAUDE.md` — Added storage-service to Services/Volumes/Networks tables; storage enforcement rule; §20 test file
|
||||
- `backend/CLAUDE.md` — Added config_storage.py, storage_config.py to tree; added admin storage endpoints
|
||||
- `frontend/CLAUDE.md` — Added StorageAdminPage to tree; added /admin/storage route
|
||||
- `features/doc-service/CLAUDE.md` — Updated storage.py description; file_path → storage_key; added migration 0008
|
||||
- `features/ai-service/CLAUDE.md` — Added config_reader.py description
|
||||
- `backend/STATUS.md` — Added storage-config endpoints; updated settings persistence note
|
||||
- `frontend/STATUS.md` — Added /admin/storage route; added StorageAdminPage description
|
||||
@@ -22,7 +22,8 @@ features/ai-service/
|
||||
│ │ ├── queue.py ← GET /queue/status, /pause, /resume, /cancel/{id}
|
||||
│ │ └── plugin.py ← GET /plugin/manifest (access rules for ai-service-admin group)
|
||||
│ └── services/
|
||||
│ └── queue.py ← Priority queue (CRITICAL > HIGH > NORMAL)
|
||||
│ ├── queue.py ← Priority queue (CRITICAL > HIGH > NORMAL)
|
||||
│ └── config_reader.py ← Reads ai_service_config.json from storage-service config bucket (30 s TTL cache)
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# doc-service — Claude context
|
||||
|
||||
PDF extraction microservice, port 8001 (internal). Shares the same PostgreSQL instance as the backend. Receives proxied requests from `backend:8000`, which injects `x-user-id` and `x-user-groups` headers — doc-service trusts these headers directly. Calls `ai-service:8010` for document classification. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
PDF extraction microservice, port 8001 (internal). Shares the same PostgreSQL instance as the backend. Receives proxied requests from `backend:8000`, which injects `x-user-id` and `x-user-groups` headers — doc-service trusts these headers directly. Calls `ai-service:8010` for document classification. All file/blob storage goes through `storage-service:8020` — no files are written directly to the filesystem. See root `CLAUDE.md` for architecture, Docker, and project-wide workflows.
|
||||
|
||||
---
|
||||
|
||||
@@ -38,13 +38,14 @@ features/doc-service/
|
||||
│ │ ├── categories.py ← Category CRUD (includes watch-owned categories)
|
||||
│ │ └── plugin.py ← GET /plugin/manifest, GET+PATCH /plugin/settings
|
||||
│ └── services/
|
||||
│ ├── storage.py ← File I/O
|
||||
│ ├── storage.py ← Storage client: save_upload/download_file/delete_file → storage-service:8020 documents bucket
|
||||
│ ├── ai_client.py ← classify_document() → ai-service:8010/chat
|
||||
│ ├── config_reader.py ← Config load/save including storage/watch settings
|
||||
│ ├── config_reader.py ← Config load/save via storage-service config bucket (doc_service_config.json)
|
||||
│ └── file_watcher.py ← watchdog-based PDF watcher + startup scan + ingestion
|
||||
├── alembic/versions/ ← Migration chain
|
||||
│ ├── 0003_add_watch_columns.py ← source, watch_path, suggested_folder, suggested_filename
|
||||
│ └── 0004_add_document_shares.py ← document_shares table (group-based sharing)
|
||||
│ ├── 0004_add_document_shares.py ← document_shares table (group-based sharing)
|
||||
│ └── 0008_rename_file_path_to_storage_key.py ← file_path → storage_key; strips /data/documents/ prefix from existing rows
|
||||
├── Dockerfile ← python:3.12-slim, non-root user 1001
|
||||
└── STATUS.md
|
||||
```
|
||||
@@ -60,7 +61,7 @@ features/doc-service/
|
||||
| `id` | String | PK, UUID | |
|
||||
| `user_id` | String | indexed | not FK — trusts x-user-id header |
|
||||
| `filename` | String | NOT NULL | |
|
||||
| `file_path` | String | NOT NULL | absolute path under /data/documents |
|
||||
| `storage_key` | String | NOT NULL | storage-service key: `{user_id}/{doc_id}.pdf` (documents bucket) |
|
||||
| `file_size` | Integer | NOT NULL | bytes |
|
||||
| `status` | String | default="pending" | pending / processing / done / failed |
|
||||
| `title` | String(500) | nullable | AI-extracted |
|
||||
@@ -118,6 +119,7 @@ Unique constraint: `(document_id, group_id)`
|
||||
| `0005` | `add_share_can_delete` |
|
||||
| `0006` | `add_category_scope` |
|
||||
| `0007` | `capitalize_system_category_names` |
|
||||
| `0008` | `rename_file_path_to_storage_key` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
+3
-1
@@ -37,7 +37,8 @@ frontend/
|
||||
│ │ └── ui/ ← shadcn/ui components (Button, Input, …)
|
||||
│ ├── pages/ ← One file per route
|
||||
│ │ ├── DocServiceSettingsPage.tsx ← Combined doc-service settings: upload limits + watch directory
|
||||
│ │ └── PluginSettingsPage.tsx ← Generic plugin settings page driven by manifest
|
||||
│ │ ├── PluginSettingsPage.tsx ← Generic plugin settings page driven by manifest
|
||||
│ │ └── StorageAdminPage.tsx ← Admin storage backend config + live migration progress
|
||||
│ ├── lib/utils.ts ← cn() = clsx + tailwind-merge
|
||||
│ └── styles/theme.css ← CSS custom properties, Tailwind setup
|
||||
├── vite.config.ts ← /api/* proxied to backend:8000
|
||||
@@ -66,6 +67,7 @@ frontend/
|
||||
| `/admin/users` | `AdminUsersPage` | AdminRoute |
|
||||
| `/admin/groups` | `AdminGroupsPage` | AdminRoute |
|
||||
| `/admin/appearance` | `AdminAppearancePage` | AdminRoute |
|
||||
| `/admin/storage` | `StorageAdminPage` | AdminRoute |
|
||||
| `*` | redirect to `/` | — |
|
||||
|
||||
`PrivateRoute` — checks `token` from `useAuth`, redirects to `/login` if absent.
|
||||
|
||||
@@ -21,6 +21,7 @@ All API calls go through `src/api/client.ts` (single Axios instance, JWT injecte
|
||||
| `/admin` | `AdminPage` (redirects to `/admin/users`) | Admin only |
|
||||
| `/admin/users` | `AdminUsersPage` | Admin only |
|
||||
| `/admin/groups` | `AdminGroupsPage` | Admin only |
|
||||
| `/admin/storage` | `StorageAdminPage` | Admin only |
|
||||
| `/profile` | `ProfilePage` | Required |
|
||||
| `/settings` | `SettingsPage` (placeholder) | Required |
|
||||
| `/settings/plugins/:id` | `PluginSettingsPage` | Required (per-plugin access control) |
|
||||
@@ -114,6 +115,10 @@ Provider selector, per-provider fields, Test Connection, Save.
|
||||
|
||||
Upload limits + watch directory config.
|
||||
|
||||
### Admin — Storage page (`/admin/storage`)
|
||||
|
||||
Current backend status (green/red health dot). Driver selector (local/S3/WebDAV) with conditional credential fields. "Test & Migrate" button triggers an async migration that copies all objects to the new backend, verifies, then switches atomically. Live progress bar with 2s polling (states: validating → migrating → switching → cleaning → done). Cancel button during in-progress migrations.
|
||||
|
||||
### Admin — Users page (`/admin/users`)
|
||||
|
||||
User list, toggle active, create user, delete user.
|
||||
@@ -202,6 +207,7 @@ Key document-related functions:
|
||||
- [x] AI suggestion confirm/reject UI (folder + filename)
|
||||
- [x] Groups admin UI
|
||||
- [x] Replace Axios with native fetch; add global 401 → `/login` redirect for expired sessions
|
||||
- [x] Admin storage page with live migration progress bar
|
||||
- [ ] Toast notification system
|
||||
- [ ] Loading skeletons
|
||||
- [ ] Cmd+K global search (`CommandDialog`)
|
||||
|
||||
@@ -16,6 +16,7 @@ import DocServiceSettingsPage from "./pages/DocServiceSettingsPage";
|
||||
import AIAdminSettingsPage from "./pages/AIAdminSettingsPage";
|
||||
import SettingsPage from "./pages/SettingsPage";
|
||||
import PluginSettingsPage from "./pages/PluginSettingsPage";
|
||||
import StorageAdminPage from "./pages/StorageAdminPage";
|
||||
|
||||
function PrivateRoute({ children }: { children: React.ReactNode }) {
|
||||
const { token } = useAuth();
|
||||
@@ -102,6 +103,7 @@ export default function App() {
|
||||
<Route path="/admin/users" element={<AdminRoute><AdminUsersPage /></AdminRoute>} />
|
||||
<Route path="/admin/groups" element={<AdminRoute><AdminGroupsPage /></AdminRoute>} />
|
||||
<Route path="/admin/appearance" element={<AdminRoute><AdminAppearancePage /></AdminRoute>} />
|
||||
<Route path="/admin/storage" element={<AdminRoute><StorageAdminPage /></AdminRoute>} />
|
||||
|
||||
{/* Catch-all */}
|
||||
<Route path="*" element={<Navigate to="/" replace />} />
|
||||
|
||||
@@ -585,6 +585,49 @@ export interface PluginManifest {
|
||||
};
|
||||
}
|
||||
|
||||
// ── Storage admin ──────────────────────────────────────────────────────────────
|
||||
|
||||
export interface StorageStatus {
|
||||
status: string;
|
||||
backend: string;
|
||||
}
|
||||
|
||||
export interface MigrationStatus {
|
||||
state:
|
||||
| "idle"
|
||||
| "validating"
|
||||
| "migrating"
|
||||
| "switching"
|
||||
| "cleaning"
|
||||
| "done"
|
||||
| "failed"
|
||||
| "cancelled";
|
||||
total: number;
|
||||
done: number;
|
||||
failed: number;
|
||||
errors: string[];
|
||||
}
|
||||
|
||||
export interface StorageBackendConfig {
|
||||
driver: string;
|
||||
config: Record<string, string>;
|
||||
}
|
||||
|
||||
export const getStorageConfig = () => api.get<StorageStatus>("/admin/storage-config");
|
||||
|
||||
export const updateStorageConfig = (body: StorageBackendConfig) =>
|
||||
api.patch<void>("/admin/storage-config", body);
|
||||
|
||||
export const startStorageMigration = (body: StorageBackendConfig) =>
|
||||
api.post<{ status: string; driver: string }>("/admin/storage-config/migrate", body);
|
||||
|
||||
export const getMigrationStatus = () =>
|
||||
api.get<MigrationStatus>("/admin/storage-config/migrate/status");
|
||||
|
||||
export const cancelMigration = () => api.delete<void>("/admin/storage-config/migrate");
|
||||
|
||||
// ── Plugins ────────────────────────────────────────────────────────────────────
|
||||
|
||||
export const getPlugins = () => api.get<PluginOut[]>("/plugins");
|
||||
|
||||
export const getPluginManifest = (id: string) =>
|
||||
|
||||
@@ -0,0 +1,382 @@
|
||||
import { useState, useEffect, useRef } from "react";
|
||||
import { useQuery, useMutation, useQueryClient } from "@tanstack/react-query";
|
||||
import {
|
||||
getStorageConfig,
|
||||
getMigrationStatus,
|
||||
startStorageMigration,
|
||||
cancelMigration,
|
||||
updateStorageConfig,
|
||||
type StorageBackendConfig,
|
||||
type MigrationStatus,
|
||||
} from "../api/client";
|
||||
|
||||
type Driver = "local" | "s3" | "webdav";
|
||||
|
||||
function Section({ title, children }: { title: string; children: React.ReactNode }) {
|
||||
return (
|
||||
<section style={{ marginBottom: 36 }}>
|
||||
<h2 style={{ fontSize: 18, marginBottom: 16 }}>{title}</h2>
|
||||
{children}
|
||||
</section>
|
||||
);
|
||||
}
|
||||
|
||||
function Field({ label, children }: { label: string; children: React.ReactNode }) {
|
||||
return (
|
||||
<div style={{ marginBottom: 12 }}>
|
||||
<label style={{ display: "block", fontSize: 13, marginBottom: 4, color: "var(--color-text-muted)" }}>
|
||||
{label}
|
||||
</label>
|
||||
{children}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
function inputStyle(disabled = false): React.CSSProperties {
|
||||
return {
|
||||
width: "100%",
|
||||
padding: "6px 10px",
|
||||
border: "1px solid var(--color-border)",
|
||||
borderRadius: 6,
|
||||
fontSize: 14,
|
||||
background: disabled ? "var(--color-surface)" : "var(--color-background)",
|
||||
color: "var(--color-text-primary)",
|
||||
opacity: disabled ? 0.7 : 1,
|
||||
};
|
||||
}
|
||||
|
||||
function MigrationProgressBar({ status }: { status: MigrationStatus }) {
|
||||
const pct = status.total > 0 ? Math.round((status.done / status.total) * 100) : 0;
|
||||
const isBusy = ["validating", "migrating", "switching", "cleaning"].includes(status.state);
|
||||
|
||||
const stateLabel: Record<string, string> = {
|
||||
idle: "Idle",
|
||||
validating: "Validating new backend…",
|
||||
migrating: `Migrating — ${status.done} / ${status.total} objects`,
|
||||
switching: "Switching active backend…",
|
||||
cleaning: "Cleaning up old backend…",
|
||||
done: "Migration complete",
|
||||
failed: "Migration failed",
|
||||
cancelled: "Migration cancelled",
|
||||
};
|
||||
|
||||
return (
|
||||
<div style={{ marginTop: 12 }}>
|
||||
<div style={{ display: "flex", justifyContent: "space-between", marginBottom: 4, fontSize: 13 }}>
|
||||
<span style={{ color: status.state === "failed" ? "#dc2626" : "var(--color-text-primary)" }}>
|
||||
{stateLabel[status.state] ?? status.state}
|
||||
</span>
|
||||
{isBusy && <span style={{ color: "var(--color-text-muted)" }}>{pct}%</span>}
|
||||
</div>
|
||||
{(isBusy || status.state === "done") && (
|
||||
<div
|
||||
style={{
|
||||
height: 8,
|
||||
background: "var(--color-border)",
|
||||
borderRadius: 4,
|
||||
overflow: "hidden",
|
||||
}}
|
||||
>
|
||||
<div
|
||||
style={{
|
||||
height: "100%",
|
||||
width: status.state === "done" ? "100%" : `${pct}%`,
|
||||
background: status.state === "done" ? "#16a34a" : "var(--color-primary)",
|
||||
borderRadius: 4,
|
||||
transition: "width 0.3s",
|
||||
}}
|
||||
/>
|
||||
</div>
|
||||
)}
|
||||
{status.errors.length > 0 && (
|
||||
<div
|
||||
style={{
|
||||
marginTop: 8,
|
||||
padding: "8px 10px",
|
||||
background: "#fef2f2",
|
||||
borderRadius: 6,
|
||||
fontSize: 12,
|
||||
color: "#991b1b",
|
||||
maxHeight: 120,
|
||||
overflowY: "auto",
|
||||
}}
|
||||
>
|
||||
{status.errors.slice(0, 10).map((e, i) => (
|
||||
<div key={i} style={{ marginBottom: 2 }}>
|
||||
{e}
|
||||
</div>
|
||||
))}
|
||||
{status.errors.length > 10 && (
|
||||
<div style={{ opacity: 0.7 }}>…and {status.errors.length - 10} more</div>
|
||||
)}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
export default function StorageAdminPage() {
|
||||
const queryClient = useQueryClient();
|
||||
|
||||
const { data: storageStatus } = useQuery({
|
||||
queryKey: ["storage-config"],
|
||||
queryFn: getStorageConfig,
|
||||
refetchInterval: 10_000,
|
||||
});
|
||||
|
||||
const { data: migStatus, refetch: refetchMig } = useQuery({
|
||||
queryKey: ["migration-status"],
|
||||
queryFn: getMigrationStatus,
|
||||
refetchInterval: (query) => {
|
||||
const state = query.state.data?.state;
|
||||
return state && ["validating", "migrating", "switching", "cleaning"].includes(state)
|
||||
? 2_000
|
||||
: false;
|
||||
},
|
||||
});
|
||||
|
||||
const isMigrating =
|
||||
migStatus &&
|
||||
["validating", "migrating", "switching", "cleaning"].includes(migStatus.state);
|
||||
|
||||
// ── New backend form ─────────────────────────────────────────────────────────
|
||||
const [driver, setDriver] = useState<Driver>("local");
|
||||
const [s3EndpointUrl, setS3EndpointUrl] = useState("");
|
||||
const [s3AccessKey, setS3AccessKey] = useState("");
|
||||
const [s3SecretKey, setS3SecretKey] = useState("");
|
||||
const [s3Region, setS3Region] = useState("us-east-1");
|
||||
const [webdavUrl, setWebdavUrl] = useState("");
|
||||
const [webdavUsername, setWebdavUsername] = useState("");
|
||||
const [webdavPassword, setWebdavPassword] = useState("");
|
||||
const [webdavRootPath, setWebdavRootPath] = useState("/");
|
||||
const [error, setError] = useState("");
|
||||
|
||||
function buildConfig(): StorageBackendConfig {
|
||||
if (driver === "s3") {
|
||||
return {
|
||||
driver,
|
||||
config: {
|
||||
endpoint_url: s3EndpointUrl,
|
||||
access_key: s3AccessKey,
|
||||
secret_key: s3SecretKey,
|
||||
region: s3Region,
|
||||
},
|
||||
};
|
||||
}
|
||||
if (driver === "webdav") {
|
||||
return {
|
||||
driver,
|
||||
config: {
|
||||
url: webdavUrl,
|
||||
username: webdavUsername,
|
||||
password: webdavPassword,
|
||||
root_path: webdavRootPath,
|
||||
},
|
||||
};
|
||||
}
|
||||
return { driver: "local", config: {} };
|
||||
}
|
||||
|
||||
const migrateMutation = useMutation({
|
||||
mutationFn: startStorageMigration,
|
||||
onSuccess: () => {
|
||||
setError("");
|
||||
refetchMig();
|
||||
},
|
||||
onError: (e: Error) => setError(e.message),
|
||||
});
|
||||
|
||||
const cancelMutation = useMutation({
|
||||
mutationFn: cancelMigration,
|
||||
onSuccess: () => {
|
||||
queryClient.invalidateQueries({ queryKey: ["migration-status"] });
|
||||
queryClient.invalidateQueries({ queryKey: ["storage-config"] });
|
||||
},
|
||||
onError: (e: Error) => setError(e.message),
|
||||
});
|
||||
|
||||
const currentDriver = storageStatus?.backend ?? "—";
|
||||
|
||||
return (
|
||||
<div style={{ maxWidth: 680, margin: "0 auto", padding: "32px 16px" }}>
|
||||
<h1 style={{ fontSize: 24, marginBottom: 4 }}>Storage</h1>
|
||||
<p style={{ color: "var(--color-text-muted)", marginBottom: 32, fontSize: 14 }}>
|
||||
All uploaded files are stored through the storage-service. Switch between local filesystem,
|
||||
S3-compatible cloud storage, or WebDAV (Nextcloud).
|
||||
</p>
|
||||
|
||||
<Section title="Current backend">
|
||||
<div
|
||||
style={{
|
||||
display: "inline-flex",
|
||||
alignItems: "center",
|
||||
gap: 8,
|
||||
padding: "6px 12px",
|
||||
background: "var(--color-surface)",
|
||||
border: "1px solid var(--color-border)",
|
||||
borderRadius: 6,
|
||||
fontSize: 14,
|
||||
}}
|
||||
>
|
||||
<span
|
||||
style={{
|
||||
width: 8,
|
||||
height: 8,
|
||||
borderRadius: "50%",
|
||||
background: storageStatus?.status === "ok" ? "#16a34a" : "#dc2626",
|
||||
flexShrink: 0,
|
||||
}}
|
||||
/>
|
||||
<strong>{currentDriver}</strong>
|
||||
{storageStatus?.status === "ok" ? " — healthy" : " — unreachable"}
|
||||
</div>
|
||||
</Section>
|
||||
|
||||
<Section title="Switch backend">
|
||||
<p style={{ fontSize: 13, color: "var(--color-text-muted)", marginBottom: 16 }}>
|
||||
When you click <strong>Test & Migrate</strong>, all existing files will be copied to the
|
||||
new backend, verified, and the switch will happen only after every file is confirmed. The
|
||||
old backend is cleaned up automatically.
|
||||
</p>
|
||||
|
||||
<Field label="Backend driver">
|
||||
<select
|
||||
value={driver}
|
||||
onChange={(e) => setDriver(e.target.value as Driver)}
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
>
|
||||
<option value="local">Local filesystem (default)</option>
|
||||
<option value="s3">S3-compatible (MinIO / AWS S3 / Backblaze / Cloudflare R2)</option>
|
||||
<option value="webdav">WebDAV (Nextcloud / …)</option>
|
||||
</select>
|
||||
</Field>
|
||||
|
||||
{driver === "s3" && (
|
||||
<>
|
||||
<Field label="Endpoint URL (leave empty for real AWS S3)">
|
||||
<input
|
||||
value={s3EndpointUrl}
|
||||
onChange={(e) => setS3EndpointUrl(e.target.value)}
|
||||
placeholder="http://minio:9000"
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="Access key">
|
||||
<input
|
||||
value={s3AccessKey}
|
||||
onChange={(e) => setS3AccessKey(e.target.value)}
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="Secret key">
|
||||
<input
|
||||
type="password"
|
||||
value={s3SecretKey}
|
||||
onChange={(e) => setS3SecretKey(e.target.value)}
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="Region">
|
||||
<input
|
||||
value={s3Region}
|
||||
onChange={(e) => setS3Region(e.target.value)}
|
||||
placeholder="us-east-1"
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
</>
|
||||
)}
|
||||
|
||||
{driver === "webdav" && (
|
||||
<>
|
||||
<Field label="Server URL">
|
||||
<input
|
||||
value={webdavUrl}
|
||||
onChange={(e) => setWebdavUrl(e.target.value)}
|
||||
placeholder="https://nextcloud.example.com"
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="Username">
|
||||
<input
|
||||
value={webdavUsername}
|
||||
onChange={(e) => setWebdavUsername(e.target.value)}
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="Password">
|
||||
<input
|
||||
type="password"
|
||||
value={webdavPassword}
|
||||
onChange={(e) => setWebdavPassword(e.target.value)}
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
<Field label="WebDAV root path">
|
||||
<input
|
||||
value={webdavRootPath}
|
||||
onChange={(e) => setWebdavRootPath(e.target.value)}
|
||||
placeholder="/remote.php/dav/files/username"
|
||||
disabled={!!isMigrating}
|
||||
style={inputStyle(!!isMigrating)}
|
||||
/>
|
||||
</Field>
|
||||
</>
|
||||
)}
|
||||
|
||||
{error && (
|
||||
<p style={{ color: "#dc2626", fontSize: 13, marginBottom: 8 }}>{error}</p>
|
||||
)}
|
||||
|
||||
<div style={{ display: "flex", gap: 10, marginTop: 16 }}>
|
||||
<button
|
||||
onClick={() => migrateMutation.mutate(buildConfig())}
|
||||
disabled={!!isMigrating || migrateMutation.isPending}
|
||||
style={{
|
||||
padding: "8px 18px",
|
||||
background: "var(--color-primary)",
|
||||
color: "#fff",
|
||||
border: "none",
|
||||
borderRadius: 6,
|
||||
fontSize: 14,
|
||||
cursor: isMigrating || migrateMutation.isPending ? "not-allowed" : "pointer",
|
||||
opacity: isMigrating || migrateMutation.isPending ? 0.6 : 1,
|
||||
}}
|
||||
>
|
||||
{migrateMutation.isPending ? "Starting…" : "Test & Migrate"}
|
||||
</button>
|
||||
|
||||
{isMigrating && (
|
||||
<button
|
||||
onClick={() => cancelMutation.mutate()}
|
||||
disabled={cancelMutation.isPending}
|
||||
style={{
|
||||
padding: "8px 18px",
|
||||
background: "transparent",
|
||||
color: "#dc2626",
|
||||
border: "1px solid #dc2626",
|
||||
borderRadius: 6,
|
||||
fontSize: 14,
|
||||
cursor: "pointer",
|
||||
}}
|
||||
>
|
||||
Cancel
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{migStatus && migStatus.state !== "idle" && (
|
||||
<MigrationProgressBar status={migStatus} />
|
||||
)}
|
||||
</Section>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
+29
-1
@@ -1,11 +1,12 @@
|
||||
# ALL_TESTS — Full Test Suite
|
||||
|
||||
Complete test suite covering all 19 feature areas. Run tests relevant to the changed area before merging any feature branch into `main`. Service-specific subsets live in separate files:
|
||||
Complete test suite covering all 20 feature areas. Run tests relevant to the changed area before merging any feature branch into `main`. Service-specific subsets live in separate files:
|
||||
|
||||
- `tests/backend_tests.md` — §1–9, §18 (auth, users, admin, groups, appearance, service health, plugins, AI/doc settings, infra/security)
|
||||
- `tests/frontend_tests.md` — §19 (UI & routing)
|
||||
- `tests/doc-service_tests.md` — §10–16 (upload/processing, list/filtering, slide-over, sharing, categories, bulk actions, watch directory)
|
||||
- `tests/ai-service_tests.md` — §17 (AI queue & providers)
|
||||
- `tests/storage-service_tests.md` — §20 (storage-service: objects, backend switching, migration)
|
||||
|
||||
Every test describes the exact UI action or API call to perform and the expected outcome.
|
||||
|
||||
@@ -351,3 +352,30 @@ Mark each row before opening the PR.
|
||||
| 19.9 | TanStack Query cache | Navigate away from docs → back | List loads from cache instantly; background refetch runs |
|
||||
| 19.10 | 30s service poll | Leave `/apps` open for 30s | `GET /api/services` fires again in network tab |
|
||||
| 19.11 | Three-dots menu not clipped | Scroll document table → open three-dot actions on any row | Dropdown renders above the table's overflow-hidden container; not cut off |
|
||||
|
||||
---
|
||||
|
||||
## 20. Storage Service
|
||||
|
||||
| # | Test | Steps | Expected |
|
||||
|---|------|-------|----------|
|
||||
| 20.1 | Upload object | `PUT /objects/documents/test/file.pdf` with binary body | 204; object stored |
|
||||
| 20.2 | Download object | `GET /objects/documents/test/file.pdf` after 20.1 | 200; binary content matches upload |
|
||||
| 20.3 | Delete object | `DELETE /objects/documents/test/file.pdf` | 204; subsequent GET returns 404 |
|
||||
| 20.4 | List bucket | `GET /objects/documents` | 200; JSON array of keys includes `test/file.pdf` |
|
||||
| 20.5 | Health endpoint | `GET /health` | `{"status":"ok","backend":"local"}` |
|
||||
| 20.6 | Path traversal rejected | `PUT /objects/documents/../etc/passwd` | 400 |
|
||||
| 20.7 | PDF upload via UI | Upload a PDF document | File stored in storage-service under `documents/{user_id}/{doc_id}.pdf`; `doc_data` volume absent |
|
||||
| 20.8 | PDF download via UI | Download a previously uploaded PDF | File streams correctly from storage-service |
|
||||
| 20.9 | Document delete via UI | Delete a document | `DELETE /objects/documents/{key}` called; storage-service key gone |
|
||||
| 20.10 | Config persistence | Restart all containers | `doc_service_config.json` and AI config survive restart in storage-service config bucket |
|
||||
| 20.11 | Admin storage page | Navigate to `/admin/storage` as admin | Page loads; current backend shows "local — healthy" |
|
||||
| 20.12 | Non-admin storage page blocked | Navigate to `/admin/storage` as non-admin | Redirected to `/login` |
|
||||
| 20.13 | Start migration — local to local | Select "Local filesystem" and click "Test & Migrate" | 400 or migration completes instantly; no data loss |
|
||||
| 20.14 | Migration progress poll | Start a migration | Status badge updates every ~2 s: validating → migrating → done |
|
||||
| 20.15 | Cancel migration | Start migration; immediately click Cancel | Migration state becomes "cancelled"; old backend remains active |
|
||||
| 20.16 | Migration conflict | Start a migration while one is running | 409 "A migration is already in progress" |
|
||||
| 20.17 | Migration — switch to S3 | Configure MinIO credentials; click "Test & Migrate" | All objects copied to S3 bucket; `GET /health` reports `backend: s3`; old local files gone |
|
||||
| 20.18 | No doc_data volume | `docker volume ls` after full stack up | `doc_data` volume absent |
|
||||
| 20.19 | No app_config volume | `docker volume ls` after full stack up | `app_config` volume absent |
|
||||
| 20.20 | Only storage_data volume | Verify `storage_data` volume exists | `docker volume ls` shows `storage_data`; all config and documents in it |
|
||||
|
||||
@@ -0,0 +1,32 @@
|
||||
# Storage Service Tests — §20
|
||||
|
||||
Storage-service tests. Run these before merging any change that touches `features/storage-service/`, `docker-compose.yml` storage volumes, or storage-related backend/doc-service code.
|
||||
|
||||
See `tests/ALL_TESTS.md` for the full suite and legend.
|
||||
|
||||
---
|
||||
|
||||
## 20. Storage Service
|
||||
|
||||
| # | Test | Steps | Expected |
|
||||
|---|------|-------|----------|
|
||||
| 20.1 | Upload object | `PUT /objects/documents/test/file.pdf` with binary body | 204; object stored |
|
||||
| 20.2 | Download object | `GET /objects/documents/test/file.pdf` after 20.1 | 200; binary content matches upload |
|
||||
| 20.3 | Delete object | `DELETE /objects/documents/test/file.pdf` | 204; subsequent GET returns 404 |
|
||||
| 20.4 | List bucket | `GET /objects/documents` | 200; JSON array of keys includes `test/file.pdf` |
|
||||
| 20.5 | Health endpoint | `GET /health` | `{"status":"ok","backend":"local"}` |
|
||||
| 20.6 | Path traversal rejected | `PUT /objects/documents/../etc/passwd` | 400 |
|
||||
| 20.7 | PDF upload via UI | Upload a PDF document | File stored in storage-service under `documents/{user_id}/{doc_id}.pdf`; `doc_data` volume absent |
|
||||
| 20.8 | PDF download via UI | Download a previously uploaded PDF | File streams correctly from storage-service |
|
||||
| 20.9 | Document delete via UI | Delete a document | `DELETE /objects/documents/{key}` called; storage-service key gone |
|
||||
| 20.10 | Config persistence | Restart all containers | `doc_service_config.json` and AI config survive restart in storage-service config bucket |
|
||||
| 20.11 | Admin storage page | Navigate to `/admin/storage` as admin | Page loads; current backend shows "local — healthy" |
|
||||
| 20.12 | Non-admin storage page blocked | Navigate to `/admin/storage` as non-admin | Redirected to `/login` |
|
||||
| 20.13 | Start migration — local to local | Select "Local filesystem" and click "Test & Migrate" | 400 or migration completes instantly; no data loss |
|
||||
| 20.14 | Migration progress poll | Start a migration | Status badge updates every ~2 s: validating → migrating → done |
|
||||
| 20.15 | Cancel migration | Start migration; immediately click Cancel | Migration state becomes "cancelled"; old backend remains active |
|
||||
| 20.16 | Migration conflict | Start a migration while one is running | 409 "A migration is already in progress" |
|
||||
| 20.17 | Migration — switch to S3 | Configure MinIO credentials; click "Test & Migrate" | All objects copied to S3 bucket; `GET /health` reports `backend: s3`; old local files gone |
|
||||
| 20.18 | No doc_data volume | `docker volume ls` after full stack up | `doc_data` volume absent |
|
||||
| 20.19 | No app_config volume | `docker volume ls` after full stack up | `app_config` volume absent |
|
||||
| 20.20 | Only storage_data volume | Verify `storage_data` volume exists | `docker volume ls` shows `storage_data`; all config and documents in it |
|
||||
Reference in New Issue
Block a user