feat: Phase 4+5 — admin storage UI, backend proxy, CLAUDE.md enforcement
- backend/app/routers/storage_config.py: 5 admin-only endpoints proxying storage-service config + migration API (GET/PATCH/POST/DELETE) - backend/app/main.py: register storage_config router - frontend/src/api/client.ts: StorageStatus, MigrationStatus, StorageBackendConfig interfaces + 5 API functions - frontend/src/pages/StorageAdminPage.tsx: full admin UI — backend health dot, driver selector (local/S3/WebDAV), conditional credential fields, Test & Migrate button, live 2s-poll migration progress bar, Cancel - frontend/src/App.tsx: /admin/storage route (AdminRoute guard) - CLAUDE.md: storage enforcement rule, updated Docker tables (6 services, 3 volumes), §20 in merge checklist - backend/CLAUDE.md, frontend/CLAUDE.md, doc-service/CLAUDE.md, ai-service/CLAUDE.md: updated to reflect storage-service integration - tests/ALL_TESTS.md + tests/storage-service_tests.md: §20 (20 tests) - backend/STATUS.md, frontend/STATUS.md: updated with new endpoints/routes - changelog/2026-04-20_storage-service.md: full change log Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
curo1305
+14
-2
@@ -36,7 +36,8 @@ backend/
|
||||
│ │ ├── config.py ← All settings via pydantic-settings (reads .env)
|
||||
│ │ ├── security.py ← JWT sign/verify (RS256), bcrypt hash/verify
|
||||
│ │ ├── sanitize.py ← Input sanitization helpers (see Security Standards)
|
||||
│ │ └── app_config.py ← Per-service config load/save to /config volume; theme files in /config/themes/
|
||||
│ │ ├── app_config.py ← Per-service config load/save via storage-service; theme files in config/themes/{id}.json
|
||||
│ │ └── config_storage.py ← Thin async HTTP helpers: read_json/write_json/delete_key/list_keys → storage-service config bucket
|
||||
│ ├── models/
|
||||
│ │ ├── __init__.py ← Imports all models (required for Alembic autogenerate)
|
||||
│ │ ├── user.py ← User model
|
||||
@@ -56,7 +57,8 @@ backend/
|
||||
│ │ ├── services.py ← GET /services (health status)
|
||||
│ │ ├── plugins.py ← Generic plugin proxy (GET/PATCH /api/plugins/*)
|
||||
│ │ ├── categories_proxy.py ← Transparent proxy → doc-service /categories/*
|
||||
│ │ └── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ ├── documents_proxy.py ← Transparent proxy → doc-service /documents/*
|
||||
│ │ └── storage_config.py ← Admin proxy → storage-service config + migration endpoints
|
||||
│ └── services/
|
||||
│ ├── service_health.py ← Background 30s health-check loop; caches /plugin/manifest per service
|
||||
│ └── group_bootstrap.py ← Ensures {service-id}-admin group exists for every registered service at startup
|
||||
@@ -216,6 +218,16 @@ Unique constraint: `(group_id, user_id)`
|
||||
|
||||
Auth: is_superuser OR member of group listed in manifest `required_groups`. Returns 404 (not 403) to hide existence.
|
||||
|
||||
### Admin — Storage (`/api/admin`) — admin-only
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| GET | `/api/admin/storage-config` | Current backend driver + health → proxied from storage-service `/health` |
|
||||
| PATCH | `/api/admin/storage-config` | Reconfigure backend without data migration (same-backend credential update) |
|
||||
| POST | `/api/admin/storage-config/migrate` | Start async migration to a new backend (copy → verify → switch → cleanup) |
|
||||
| GET | `/api/admin/storage-config/migrate/status` | Poll migration progress: `{state, total, done, failed, errors[]}` |
|
||||
| DELETE | `/api/admin/storage-config/migrate` | Cancel a running migration; old backend remains active |
|
||||
|
||||
### Documents and Categories — proxied
|
||||
|
||||
`/api/documents/*` and `/api/documents/categories/*` are transparently proxied to `doc-service:8001`. The backend injects `x-user-id`, `x-user-groups`, and `x-user-is-admin` headers. See `features/doc-service/CLAUDE.md` for the internal endpoint list.
|
||||
|
||||
+16
-3
@@ -75,10 +75,20 @@ A background task (`service_health.py`) polls each service's `/health` endpoint
|
||||
| `GET` | `/api/settings/system-prompts` | All editable system prompts — superuser OR `ai-service-admin` member |
|
||||
| `PATCH` | `/api/settings/system-prompts/{id}` | Update system prompt — same access |
|
||||
|
||||
Settings are persisted to JSON files on the `app_config` Docker named volume and read by the respective feature services.
|
||||
Settings are persisted to the `config` bucket of `storage-service:8020` via `core/config_storage.py`. All config I/O is async HTTP; no filesystem volumes are used.
|
||||
|
||||
Access to service-specific settings endpoints is enforced by `get_service_admin(service_id)` in `deps.py` — grants access to superusers OR members of the `{service_id}-admin` group.
|
||||
|
||||
### Storage config (`/api/admin`)
|
||||
|
||||
| Method | Path | Description |
|
||||
|--------|------|-------------|
|
||||
| `GET` | `/api/admin/storage-config` | Current backend driver + health (proxied from storage-service) |
|
||||
| `PATCH` | `/api/admin/storage-config` | Reconfigure backend without migration |
|
||||
| `POST` | `/api/admin/storage-config/migrate` | Start async migration to a new backend |
|
||||
| `GET` | `/api/admin/storage-config/migrate/status` | Poll migration progress |
|
||||
| `DELETE` | `/api/admin/storage-config/migrate` | Cancel running migration |
|
||||
|
||||
### Feature proxies
|
||||
|
||||
All `/api/documents/*` and `/api/documents/categories/*` requests are transparently proxied to `doc-service:8001` via `httpx.AsyncClient`. The proxy:
|
||||
@@ -129,8 +139,11 @@ Browser (port 5173 dev / 80 prod)
|
||||
┌───────────┼────────────┬──────────────┐
|
||||
/auth /settings /documents/* /services
|
||||
/users (JSON │ │
|
||||
/admin volume) └── proxy → health-check loop
|
||||
/profile doc-service:8001 (30s poll)
|
||||
/admin /storage- └── proxy → health-check loop
|
||||
/profile config doc-service:8001 (30s poll)
|
||||
(proxy)
|
||||
│
|
||||
storage-service:8020
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
@@ -9,6 +9,7 @@ from app.core.config import settings
|
||||
from app.database import AsyncSessionLocal
|
||||
from app.routers import admin, auth, categories_proxy, documents_proxy, groups, plugins, profile, services, users
|
||||
from app.routers import settings as settings_router
|
||||
from app.routers import storage_config
|
||||
from app.services.group_bootstrap import ensure_service_admin_groups
|
||||
from app.services.service_health import check_all, health_check_loop, register_services
|
||||
|
||||
@@ -52,6 +53,7 @@ app.include_router(admin.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(groups.router, prefix="/api/admin/groups", tags=["admin"])
|
||||
app.include_router(settings_router.router, prefix="/api/settings", tags=["settings"])
|
||||
app.include_router(services.router, prefix="/api/services", tags=["services"])
|
||||
app.include_router(storage_config.router, prefix="/api/admin", tags=["admin"])
|
||||
app.include_router(plugins.router, prefix="/api/plugins", tags=["plugins"])
|
||||
# categories_proxy MUST be registered before documents_proxy —
|
||||
# otherwise /api/documents/{path:path} swallows /api/documents/categories/*
|
||||
|
||||
@@ -0,0 +1,126 @@
|
||||
"""
|
||||
Admin-only endpoints for storage-service backend configuration.
|
||||
|
||||
GET /admin/storage-config — current backend driver + health
|
||||
PATCH /admin/storage-config — update backend config (no data migration)
|
||||
POST /admin/storage-config/migrate — start migration to a new backend
|
||||
GET /admin/storage-config/migrate/status — poll migration progress
|
||||
DELETE /admin/storage-config/migrate — cancel in-progress migration
|
||||
|
||||
All endpoints proxy to storage-service:8020.
|
||||
"""
|
||||
import logging
|
||||
|
||||
import httpx
|
||||
from fastapi import APIRouter, Depends, HTTPException
|
||||
from pydantic import BaseModel
|
||||
|
||||
from app.core.config import settings
|
||||
from app.deps import get_current_admin
|
||||
from app.models.user import User
|
||||
|
||||
router = APIRouter()
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
_STORAGE_BASE = settings.STORAGE_SERVICE_URL
|
||||
|
||||
|
||||
class BackendConfigUpdate(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
class MigrateRequest(BaseModel):
|
||||
driver: str
|
||||
config: dict = {}
|
||||
|
||||
|
||||
def _storage_url(path: str) -> str:
|
||||
return f"{_STORAGE_BASE}{path}"
|
||||
|
||||
|
||||
async def _proxy_get(path: str) -> dict:
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.get(_storage_url(path))
|
||||
if resp.status_code == 404:
|
||||
raise HTTPException(status_code=404, detail="Not found")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config")
|
||||
async def get_storage_config(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Return current backend driver and health status."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.get(_storage_url("/health"))
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.patch("/storage-config", status_code=204)
|
||||
async def update_storage_config(
|
||||
body: BackendConfigUpdate,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""
|
||||
Reconfigure the active backend without migrating data.
|
||||
Use when changing credentials for the same backend type, or reverting to local.
|
||||
To move data to a new backend, use POST /admin/storage-config/migrate instead.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=15.0) as client:
|
||||
resp = await client.patch(
|
||||
_storage_url("/backend-config"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="Migration in progress — cannot reconfigure now")
|
||||
resp.raise_for_status()
|
||||
|
||||
|
||||
@router.post("/storage-config/migrate", status_code=202)
|
||||
async def start_migration(
|
||||
body: MigrateRequest,
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""
|
||||
Start an async migration to a new backend.
|
||||
|
||||
Flow: validate new backend → copy all objects → verify → switch → delete old objects.
|
||||
The old backend stays active until 100% of objects are verified on the new one.
|
||||
Poll GET /admin/storage-config/migrate/status to track progress.
|
||||
"""
|
||||
async with httpx.AsyncClient(timeout=30.0) as client:
|
||||
resp = await client.post(
|
||||
_storage_url("/migrate"),
|
||||
json={"driver": body.driver, "config": body.config},
|
||||
)
|
||||
if resp.status_code == 400:
|
||||
raise HTTPException(status_code=400, detail=resp.json().get("detail", "Validation failed"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="A migration is already in progress")
|
||||
resp.raise_for_status()
|
||||
return resp.json()
|
||||
|
||||
|
||||
@router.get("/storage-config/migrate/status")
|
||||
async def migration_status(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> dict:
|
||||
"""Poll migration progress. State: idle → validating → migrating → switching → cleaning → done."""
|
||||
return await _proxy_get("/migrate/status")
|
||||
|
||||
|
||||
@router.delete("/storage-config/migrate", status_code=204)
|
||||
async def cancel_migration(
|
||||
_: User = Depends(get_current_admin),
|
||||
) -> None:
|
||||
"""Cancel a running migration. The old backend remains active."""
|
||||
async with httpx.AsyncClient(timeout=10.0) as client:
|
||||
resp = await client.delete(_storage_url("/migrate"))
|
||||
if resp.status_code == 409:
|
||||
raise HTTPException(status_code=409, detail="No cancellable migration in progress")
|
||||
resp.raise_for_status()
|
||||
Reference in New Issue
Block a user