docs(05-01): complete Wave 0 Nyquist scaffold plan

- Create 05-01-SUMMARY.md documenting all 3 tasks and 5 files modified
- Update STATE.md: session record, progress 78% (25/32 plans), resume file → 05-02
- Update ROADMAP.md: Phase 5 progress (1/8 summaries, In Progress)
- Update REQUIREMENTS.md: mark CLOUD-01..07 complete (Wave 0 scaffold)
This commit is contained in:
curo1305
2026-05-28 20:54:51 +02:00
parent b53ea863dd
commit 664451b8e6
4 changed files with 218 additions and 16 deletions
+41 -2
View File
@@ -25,18 +25,21 @@ Before any phase is marked complete, all three gates must pass:
## Phase Details
### Phase 1: Infrastructure Foundation
**Goal**: PostgreSQL + MinIO are wired into Docker Compose with a complete Alembic-managed schema; all services boot cleanly and the existing single-user document scanner continues to work exactly as before — no user-facing behavior change.
**Mode:** mvp
**Depends on**: Nothing (first phase)
**Requirements**: STORE-01, STORE-02, STORE-07
**Success Criteria** (what must be TRUE):
1. `docker compose up` starts PostgreSQL, MinIO, and the FastAPI backend with no errors; health checks pass for all three services
2. Running `alembic upgrade head` applies the initial migration cleanly against the fresh PostgreSQL instance with no errors
3. The full existing document upload, text extraction, and AI classification workflow completes successfully — no regression in single-user behavior
4. MinIO object key schema `{user_id}/{document_id}/{uuid4()}{ext}` is enforced in the model layer; human-readable filenames are stored in the DB column, not in the MinIO key
**Plans**: 5 plans
- [x] 01-01-PLAN.md — Docker Compose service topology + Postgres init + Pydantic Settings + requirements
- [x] 01-02-PLAN.md — Wave 0 test scaffolds (xfail/skip stubs) + async pytest fixtures
- [x] 01-03-PLAN.md — SQLAlchemy ORM models + async engine + Alembic async migration (incl. alembic upgrade head)
@@ -46,12 +49,14 @@ Before any phase is marked complete, all three gates must pass:
---
### Phase 2: Users & Authentication
**Goal**: Users can register, log in (with optional TOTP 2FA), reset their password, and sign out all active sessions; admins can manage user accounts and assign AI providers — all enforced by a complete FastAPI dependency chain.
**Mode:** mvp
**Depends on**: Phase 1
**Requirements**: AUTH-01, AUTH-02, AUTH-03, AUTH-04, AUTH-05, AUTH-06, AUTH-07, AUTH-08, SEC-01, SEC-02, SEC-03, SEC-05, SEC-06, SEC-07, ADMIN-01, ADMIN-02, ADMIN-03, ADMIN-04, ADMIN-05, ADMIN-07
**Success Criteria** (what must be TRUE):
1. A new user can register with an email and password that passes strength validation; a password from the HaveIBeenPwned list is rejected with a clear error
2. A logged-in user can enroll a TOTP authenticator app, receive 810 backup codes, explicitly acknowledge them, and thereafter be required to supply a TOTP code (or backup code) on every login — a backup code is invalidated on first use
3. A user who forgets their password can receive a reset email, follow the link within 1 hour, set a new password, and is then returned to the TOTP login gate (not auto-logged in)
@@ -61,21 +66,27 @@ Before any phase is marked complete, all three gates must pass:
**Plans**: 5 plans
**Wave 1** — Foundation
- [x] 02-01-PLAN.md — Auth service layer (Argon2, JWT, refresh tokens, TOTP, backup codes, HIBP, security alert), FastAPI deps, BackupCode model + password_must_change migration
**Wave 2** *(blocked on Wave 1 completion)*
- [x] 02-02-PLAN.md — Register/login (TOTP + backup code paths) + refresh/logout/change-password endpoints + CSP/Origin validation/rate-limit (IP + per-account) + Vue auth store + router guard + Login/Register views
**Wave 3** *(blocked on Wave 2 completion)*
- [x] 02-03-PLAN.md — TOTP enrollment + backup codes + password reset + sign-out-all endpoints + AccountView + TotpEnrollment + BackupCodesDisplay + PasswordReset views
**Wave 4** *(blocked on Wave 3 completion)*
- [x] 02-04-PLAN.md — Admin backend: user CRUD, quota, AI config endpoints with get_current_admin enforced + tests
**Wave 5** *(blocked on Wave 4 completion)*
- [x] 02-05-PLAN.md — Admin panel frontend: AdminView + three tab components + AppSidebar admin link and user identity footer
**Cross-cutting constraints:**
- JWT access token in Pinia memory only — never localStorage (Plans 02, 03, 05)
- Refresh token httpOnly SameSite=Strict cookie on all token issuance (Plans 02, 03)
- Admin endpoints never return document content or credentials_enc (Plans 04, 05)
@@ -86,12 +97,14 @@ Before any phase is marked complete, all three gates must pass:
---
### Phase 3: Document Migration & Multi-User Isolation
**Goal**: All existing documents have been migrated from flat-file JSON + filesystem into PostgreSQL + MinIO; all new uploads use the presigned URL flow; per-user isolation is enforced at the DB level; the existing document UI works without regression; the backend is stateless and ready for horizontal scaling.
**Mode:** mvp
**Depends on**: Phase 2
**Requirements**: STORE-03, STORE-04, STORE-05, STORE-06, STORE-08, SEC-04, DOC-03, DOC-04, DOC-05
**Success Criteria** (what must be TRUE):
1. Every document present before migration is accessible after migration with the same metadata and extracted text; a count reconciliation check confirms zero document loss
2. Two concurrent uploads that would together exceed a user's 100 MB quota result in exactly one success and one 413 rejection — the quota never goes over limit
3. A document delete atomically decrements the user's recorded quota usage; after deletion the quota reflects the freed bytes
@@ -101,27 +114,34 @@ Before any phase is marked complete, all three gates must pass:
**Plans**: 5 plans
**Wave 1** — Migration + test scaffolds
- [x] 03-01-PLAN.md — Wave 0 test scaffolds (auth_user/admin_user/MinIO mock fixtures + 19 xfail stubs) + Alembic migration 0003 (null-user cleanup, NOT NULL constraint, topic cleanup, quota reconciliation, ix_topics_user_id) — Complete 2026-05-23
**Wave 2** *(blocked on Wave 1)*
- [ ] 03-02-PLAN.md — Presigned upload backend: StorageBackend ABC + MinIOBackend dual client + generate_presigned_put_url/stat_object + /api/documents/upload-url + /api/documents/{id}/confirm with atomic quota UPDATE + GET /api/auth/me/quota + delete-with-quota + abandoned-upload Celery beat + docker-compose CORS/celery-beat
**Wave 3** *(blocked on Wave 2)*
- [x] 03-03-PLAN.md — Auth guards: get_regular_user dep + ownership assertions on every /api/documents/* handler (404 not 403) + admin 403 + real user_id in object_key + namespace-scoped /api/topics/* + POST /api/admin/topics + classifier topic-namespace plumbing
**Wave 4** *(blocked on Wave 3)*
- [x] 03-04-PLAN.md — Settings retirement + per-user AI: delete /api/settings + remove load_settings/save_settings + classifier accepts ai_provider/ai_model kwargs + Celery task resolves user.ai_provider via DB + frontend SettingsView placeholder + remove settings store/API — Complete 2026-05-23
**Wave 5** *(blocked on Wave 4)*
- [ ] 03-05-PLAN.md — Frontend upload flow + quota bar: 3-step upload action with XHR progress + UploadProgress.vue progress bar and quota rejection error block + QuotaBar.vue + AppSidebar embed + quota state in auth store + human checkpoint
**Cross-cutting constraints:**
- Atomic quota UPDATE pattern only lives in Plan 02; never duplicate (CLAUDE.md)
- Every /api/documents/* handler injects get_regular_user (Plan 03)
- AI provider/model resolved only via Celery task DB lookup (Plan 04)
- Browser XHR PUT to MinIO sends NO Authorization header (Plan 05)
**Phase gates (must pass before Phase 3 is complete):**
- [ ] `pytest -v` — zero failures; presigned URL, quota enforcement, ownership isolation, and admin-403 all covered
- [ ] Security agent: path traversal check on object key construction; cross-user IDOR tests; quota race condition test
- [ ] Bandit + pip audit + npm audit all clean
@@ -131,12 +151,14 @@ Before any phase is marked complete, all three gates must pass:
---
### Phase 4: Folders, Sharing, Quotas & Document UX
**Goal**: Users have a complete document management experience — organized with folders, shared by handle, warned before they hit quota, able to preview PDFs in-browser, and served by a searchable document list; admins can view the append-only audit log.
**Mode:** mvp
**Depends on**: Phase 3
**Requirements**: FOLD-01, FOLD-02, FOLD-03, FOLD-04, FOLD-05, SHARE-01, SHARE-02, SHARE-03, SHARE-04, SHARE-05, SEC-08, SEC-09, ADMIN-06, DOC-01, DOC-02
**Success Criteria** (what must be TRUE):
1. A user can create, rename, and delete folders; moving a document between folders preserves its metadata and AI classification; deleting a non-empty folder prompts with the content count before proceeding
2. A user can share a document with another user by handle; the recipient sees it appear in a "Shared with me" virtual folder with no storage quota charged against them; the owner can revoke access and the shared entry disappears immediately for the recipient
3. The sidebar quota bar displays current usage in MB; it turns amber at 80% and red at 95%; an upload that would exceed the limit is rejected with an error showing current usage, the rejected file size, and a link to storage settings
@@ -146,27 +168,34 @@ Before any phase is marked complete, all three gates must pass:
**Plans**: 9 plans
**Wave 1** — Test scaffolds + DB migration (parallel)
- [x] 04-01-PLAN.md — Wave 0 test stubs: test_folders.py + test_shares.py + test_audit.py + proxy stubs in test_documents.py + SEC-08/SEC-09 stubs in test_security.py
- [x] 04-02-PLAN.md — Alembic migration 0004 (users.pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw()
**Wave 2** *(blocked on Wave 1)*
- [x] 04-03-PLAN.md — Audit service (write_audit_log) + Folders API (FOLD-01..05): POST/GET/PATCH/DELETE /api/folders + PATCH /api/documents/{id}/folder + document list sort/search/is_shared extension
- [ ] 04-04-PLAN.md — Shares API (SHARE-01..05): POST/GET /api/shares + GET /api/shares/received + DELETE /api/shares/{id} with IDOR protection
**Wave 3** *(blocked on Wave 2)*
- [ ] 04-05-PLAN.md — PDF streaming proxy GET /api/documents/{id}/content with Range header support + PATCH /api/auth/me/preferences (pdf_open_mode)
- [ ] 04-06-PLAN.md — Admin audit log API (GET /api/admin/audit-log, CSV export) + Celery beat daily audit export task + celery_app.py beat schedule
**Wave 4** *(blocked on Wave 3)*
- [ ] 04-07-PLAN.md — SEC-08/SEC-09 hardening + audit log backfill into auth.py/admin.py/documents.py + CloudConnectionOut Pydantic model + delete-user file cleanup
**Wave 5** *(blocked on Wave 4)*
- [ ] 04-08-PLAN.md — Frontend data layer: API client functions + useFoldersStore + documents store extension + Vue Router routes (/folders/:folderId, /shared)
**Wave 6** *(blocked on Wave 5)*
- [ ] 04-09-PLAN.md — Frontend UI: all new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + view wiring (AppSidebar, DocumentCard, HomeView, FolderView, SharedView, SettingsView, AdminView) + human checkpoint
**Phase gates (must pass before Phase 4 is complete):**
- [ ] `pytest -v` — zero failures; folder ownership, share revocation, quota bar, PDF proxy (no presigned URL exposure) all covered
- [ ] Security agent: audit log verified to contain zero document content; sharing IDOR tests; PDF proxy verified to not leak presigned URLs or object keys
- [ ] Bandit + pip audit + npm audit all clean
@@ -176,12 +205,14 @@ Before any phase is marked complete, all three gates must pass:
---
### Phase 5: Cloud Storage Backends
**Goal**: Users can connect OneDrive, Google Drive, Nextcloud, or a generic WebDAV server as a personal storage backend; credentials are encrypted with a per-user HKDF-derived key; connection status is visible; local and cloud storage coexist; the `StorageBackend` ABC makes adding further backends straightforward.
**Mode:** mvp
**Depends on**: Phase 4
**Requirements**: CLOUD-01, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07
**Success Criteria** (what must be TRUE):
1. A user can connect OneDrive, Google Drive, Nextcloud, or a WebDAV endpoint through an OAuth or credential flow; the connection status is displayed as `ACTIVE`, `REQUIRES_REAUTH`, or `ERROR` — never shows raw credentials
2. When an OAuth token is revoked externally (simulated `invalid_grant` response), the connection status transitions to `REQUIRES_REAUTH` without a 500 error; the user is shown a re-authentication prompt
3. A user can select their connected cloud backend as the default storage destination for new uploads; local MinIO storage remains available as an alternative; existing local documents are unaffected
@@ -191,28 +222,36 @@ Before any phase is marked complete, all three gates must pass:
**Plans**: 8 plans
**Wave 1** — Test scaffold + dependencies
- [ ] 05-01-PLAN.md — Wave 0 xfail stubs, conftest cloud fixtures, requirements.txt packages, config.py settings
- [x] 05-01-PLAN.md — Wave 0 xfail stubs, conftest cloud fixtures, requirements.txt packages, config.py settings
**Wave 2** — Shared utilities
- [ ] 05-02-PLAN.md — cloud_utils.py (SSRF + HKDF), cloud_cache.py (TTLCache), storage factory extension
**Wave 3** — Cloud backends (parallel, both blocked on Wave 2 / Plan 05-02)
- [ ] 05-03-PLAN.md — GoogleDriveBackend + OneDriveBackend (all 7 StorageBackend methods)
- [ ] 05-04-PLAN.md — NextcloudBackend + WebDAVBackend (all 7 StorageBackend methods)
**Wave 4** — Cloud API
- [ ] 05-05-PLAN.md — All /api/cloud/* endpoints + /api/users/me/default-storage + main.py router registration
**Wave 5** — Document routing + full test suite
- [ ] 05-06-PLAN.md — Upload/content proxy cloud routing + all 15 tests promoted to passing
**Wave 6** — Frontend settings UI
- [ ] 05-07-PLAN.md — cloudConnections store + API client + SettingsView 3-tab + SettingsCloudTab + CloudCredentialModal
**Wave 7** — Frontend sidebar (human checkpoint)
- [ ] 05-08-PLAN.md — AppSidebar cloud section + CloudProviderTreeItem + CloudFolderTreeItem + human checkpoint
**Phase gates (must pass before Phase 5 is complete):**
- [ ] `pytest -v` — zero failures; SSRF prevention on WebDAV/Nextcloud user-supplied URLs; credential encryption/decryption round-trip; admin response never exposes `credentials_enc`; OAuth invalid_grant handling
- [ ] Security agent: SSRF allowlist verification; credential key derivation correctness; connection status never leaks raw credential values
- [ ] Bandit + pip audit + npm audit all clean
@@ -229,4 +268,4 @@ Before any phase is marked complete, all three gates must pass:
| 2. Users & Authentication | 5/5 | Complete | 2026-05-22 |
| 3. Document Migration & Multi-User Isolation | 5/5 | Complete | 2026-05-25 |
| 4. Folders, Sharing, Quotas & Document UX | 9/9 | Complete | 2026-05-28 |
| 5. Cloud Storage Backends | 0/8 | In Progress | - |
| 5. Cloud Storage Backends | 1/8 | In Progress| |