docs: create milestone v0.3 roadmap (5 phases)
This commit is contained in:
@@ -86,16 +86,50 @@
|
||||
|
||||
## Traceability
|
||||
|
||||
Populated during roadmap creation. Every v0.3 requirement must map to exactly one phase.
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| CONN-01 | Phase 13 | Pending |
|
||||
| CONN-02 | Phase 13 | Pending |
|
||||
| CONN-03 | Phase 13 | Pending |
|
||||
| CONN-04 | Phase 12 | Pending |
|
||||
| CLOUD-01 | Phase 12 | Pending |
|
||||
| CLOUD-02 | Phase 13 | Pending |
|
||||
| CLOUD-03 | Phase 13 | Pending |
|
||||
| CLOUD-04 | Phase 13 | Pending |
|
||||
| CLOUD-05 | Phase 13 | Pending |
|
||||
| CLOUD-06 | Phase 13 | Pending |
|
||||
| CLOUD-07 | Phase 13 | Pending |
|
||||
| CLOUD-08 | Phase 12 | Pending |
|
||||
| CLOUD-09 | Phase 13 | Pending |
|
||||
| ANALYZE-01 | Phase 14 | Pending |
|
||||
| ANALYZE-02 | Phase 14 | Pending |
|
||||
| ANALYZE-03 | Phase 14 | Pending |
|
||||
| ANALYZE-04 | Phase 14 | Pending |
|
||||
| ANALYZE-05 | Phase 14 | Pending |
|
||||
| ANALYZE-06 | Phase 14 | Pending |
|
||||
| ANALYZE-07 | Phase 14 | Pending |
|
||||
| SEARCH-01 | Phase 15 | Pending |
|
||||
| SEARCH-02 | Phase 15 | Pending |
|
||||
| SEARCH-03 | Phase 15 | Pending |
|
||||
| SEARCH-04 | Phase 15 | Pending |
|
||||
| SEARCH-05 | Phase 15 | Pending |
|
||||
| SEARCH-06 | Phase 15 | Pending |
|
||||
| SEARCH-07 | Phase 15 | Pending |
|
||||
| CACHE-01 | Phase 12 | Pending |
|
||||
| CACHE-02 | Phase 12 | Pending |
|
||||
| CACHE-03 | Phase 14 | Pending |
|
||||
| CACHE-04 | Phase 14 | Pending |
|
||||
| CACHE-05 | Phase 14 | Pending |
|
||||
| SYNC-01 | Phase 12 | Pending |
|
||||
| SYNC-02 | Phase 16 | Pending |
|
||||
| SYNC-03 | Phase 16 | Pending |
|
||||
| SYNC-04 | Phase 16 | Pending |
|
||||
|
||||
**Coverage:**
|
||||
- v0.3 requirements: 36 total
|
||||
- Mapped to phases: 0
|
||||
- Unmapped: 36
|
||||
- Mapped to phases: 36
|
||||
- Unmapped: 0
|
||||
|
||||
---
|
||||
*Requirements defined: 2026-06-17*
|
||||
*Last updated: 2026-06-17 after milestone requirements approval*
|
||||
*Last updated: 2026-06-17 after roadmap creation*
|
||||
|
||||
+99
-543
@@ -1,561 +1,117 @@
|
||||
# DocuVault — v1 Roadmap
|
||||
# DocuVault Roadmap: v0.3 Reimagining Cloud Storage integration
|
||||
|
||||
_Last updated: 2026-06-02_
|
||||
**Status:** Proposed
|
||||
**Phases:** 12-16
|
||||
**Requirements:** 36
|
||||
**Defined:** 2026-06-17
|
||||
|
||||
## Mandatory Cross-Cutting Gates (every phase)
|
||||
## Milestone Goal
|
||||
|
||||
Before any phase is marked complete, all three gates must pass:
|
||||
Make connected cloud storage behave like first-class DocuVault storage: users can navigate and manage provider-owned files as if they were local, selectively analyze them, and find them through unified keyword and semantic search while DocuVault minimizes locally cached file bytes.
|
||||
|
||||
1. **Test gate** — `pytest -v` passes with zero failures; every new function/endpoint has at least one test; all security invariant tests pass (wrong owner, admin block, token replay)
|
||||
2. **Security gate** — Security agent runs `bandit -r backend/` (zero HIGH), `pip audit` (zero critical/high), `npm audit --audit-level=high` (zero high/critical); admin endpoints verified to never return `password_hash`, `credentials_enc`, or document content; no hardcoded secrets
|
||||
3. **Bug fix rule** — Any bug fix during execution must: (a) target the root cause, (b) change ≤50 lines, (c) include a regression test — no workarounds permitted
|
||||
## Mandatory Cross-Cutting Gates
|
||||
|
||||
---
|
||||
Before any phase is marked complete:
|
||||
|
||||
## Phases
|
||||
1. Backend and frontend test suites pass with zero failures; every new function, endpoint, store, and component has focused coverage.
|
||||
2. Security gate verifies user/connection/item ownership, credential secrecy, SSRF defenses, metadata-only audit logs, cache isolation, and no admin document-content access.
|
||||
3. Dependency audits report no high/critical vulnerabilities.
|
||||
4. Provider integration tests cover supported capabilities and explicit unsupported behavior.
|
||||
5. User-facing changes update `AGENTS.md`, relevant README sections, and application patch/minor versions according to project protocol.
|
||||
|
||||
- [x] **Phase 1: Infrastructure Foundation** — PostgreSQL + MinIO wired into Docker Compose; Alembic migrations running; existing app still works
|
||||
- [x] **Phase 2: Users & Authentication** — Full auth flow end-to-end (register, login, TOTP, backup codes, password reset, sign-out-all) with admin panel for user management
|
||||
- [x] **Phase 3: Document Migration & Multi-User Isolation** — All documents in PostgreSQL + MinIO; per-user isolation enforced; existing UI still works
|
||||
- [x] **Phase 4: Folders, Sharing, Quotas & Document UX** — Full document management UX (folders, sharing, quota bar, PDF preview, search, audit log)
|
||||
- [x] **Phase 5: Cloud Storage Backends** — Users can connect OneDrive, Google Drive, Nextcloud, or WebDAV as a personal storage backend
|
||||
## Phase Summary
|
||||
|
||||
---
|
||||
| Phase | Name | Goal | Requirements |
|
||||
|------:|------|------|--------------|
|
||||
| 12 | Cloud Resource Foundation | Establish provider capabilities and durable virtual-document metadata without mirroring provider bytes | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 |
|
||||
| 13 | Virtual-Local Cloud Operations | Make cloud browsing and core file actions feel local while preserving provider semantics and security | CONN-01..03, CLOUD-02..07, CLOUD-09 |
|
||||
| 14 | Selective Analysis and Byte Cache | Analyze selected cloud scopes through cancellable jobs backed by bounded, private, on-demand byte caching | ANALYZE-01..07, CACHE-03..05 |
|
||||
| 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
|
||||
| 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
|
||||
|
||||
## Phase Details
|
||||
|
||||
### Phase 1: Infrastructure Foundation
|
||||
### Phase 12: Cloud Resource Foundation
|
||||
|
||||
**Goal**: PostgreSQL + MinIO are wired into Docker Compose with a complete Alembic-managed schema; all services boot cleanly and the existing single-user document scanner continues to work exactly as before — no user-facing behavior change.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Nothing (first phase)
|
||||
**Requirements**: STORE-01, STORE-02, STORE-07
|
||||
**Goal:** DocuVault has a provider-neutral cloud file capability contract and durable per-user cloud item index, and the shared browser can render cloud resources and supported actions without copying full provider files into local storage.
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
**Depends on:** Phase 11
|
||||
**Requirements:** CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01
|
||||
|
||||
1. `docker compose up` starts PostgreSQL, MinIO, and the FastAPI backend with no errors; health checks pass for all three services
|
||||
2. Running `alembic upgrade head` applies the initial migration cleanly against the fresh PostgreSQL instance with no errors
|
||||
3. The full existing document upload, text extraction, and AI classification workflow completes successfully — no regression in single-user behavior
|
||||
4. MinIO object key schema `{user_id}/{document_id}/{uuid4()}{ext}` is enforced in the model layer; human-readable filenames are stored in the DB column, not in the MinIO key
|
||||
**Success Criteria:**
|
||||
|
||||
**Plans**: 5 plans
|
||||
1. Every supported provider reports normalized capabilities for browse, open, preview, upload, folder creation, rename, move, delete, and change tracking.
|
||||
2. Browsing any connected provider persists normalized, owner-scoped cloud item metadata including provider ID/path, parent, type, size, modification time, and version/etag.
|
||||
3. `StorageBrowser.vue` remains the single file browser and renders cloud actions from capabilities, including disabled actions with provider-specific explanations.
|
||||
4. Provider-owned bytes remain outside DocuVault while metadata, extracted text placeholders, topic links, and semantic-index state can be persisted independently.
|
||||
5. Ownership and admin-negative tests prove cloud connection and cloud item metadata cannot cross user boundaries.
|
||||
|
||||
- [x] 01-01-PLAN.md — Docker Compose service topology + Postgres init + Pydantic Settings + requirements
|
||||
- [x] 01-02-PLAN.md — Wave 0 test scaffolds (xfail/skip stubs) + async pytest fixtures
|
||||
- [x] 01-03-PLAN.md — SQLAlchemy ORM models + async engine + Alembic async migration (incl. alembic upgrade head)
|
||||
- [x] 01-04-PLAN.md — StorageBackend ABC + MinIO backend + rewritten async services/storage.py
|
||||
- [x] 01-05-PLAN.md — Lifespan + /health + API cutover + Celery worker + walking-skeleton e2e verify
|
||||
### Phase 13: Virtual-Local Cloud Operations
|
||||
|
||||
**Goal:** Users can maintain healthy cloud connections and perform the main file-management workflows in connected storage with the same shared browser interactions used for local files.
|
||||
|
||||
**Depends on:** Phase 12
|
||||
**Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Users can connect, test, reconnect, and disconnect providers while seeing current health and actionable reauthentication/error states.
|
||||
2. Users can open or preview supported cloud documents and upload files into the current cloud folder through authorized DocuVault endpoints.
|
||||
3. Users can create folders, rename items, move items within one connection, and delete items after confirmation wherever the provider supports each action.
|
||||
4. Successful mutations immediately invalidate affected listings/index entries, refresh the shared browser, and write metadata-only audit events.
|
||||
5. Provider-specific tests verify operation semantics, conflict/error responses, token refresh persistence, SSRF protection, and explicit unsupported capabilities.
|
||||
|
||||
### Phase 14: Selective Analysis and Byte Cache
|
||||
|
||||
**Goal:** Users can analyze individual files, selected scopes, or whole connections through observable and controllable background jobs, while file bytes are hydrated only on demand and evicted safely.
|
||||
|
||||
**Depends on:** Phase 13
|
||||
**Requirements:** ANALYZE-01, ANALYZE-02, ANALYZE-03, ANALYZE-04, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-04, CACHE-05
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Users can analyze one file, a multi-selection, a folder tree, or a whole connection after reviewing recursive file-count and byte-size estimates.
|
||||
2. The UI exposes aggregate and per-item queued, downloading, extracting, classifying, indexed, cancelled, and failed states with retry and queued-work cancellation.
|
||||
3. Analysis jobs are idempotent by cloud item and provider version/etag, and they never mutate the provider-owned file.
|
||||
4. Cloud bytes are cached only for active opening, preview, or analysis work and are evicted by configurable limits without interrupting pinned active jobs.
|
||||
5. Cache ownership, cache-key versioning, cancellation, duplicate-job, failure-retry, and eviction behavior have dedicated tests.
|
||||
|
||||
### Phase 15: Unified Smart Search
|
||||
|
||||
**Goal:** One search experience finds local and analyzed cloud documents by keywords, sentences, or semantic ideas and opens cloud results through on-demand hydration.
|
||||
|
||||
**Depends on:** Phase 14
|
||||
**Requirements:** SEARCH-01, SEARCH-02, SEARCH-03, SEARCH-04, SEARCH-05, SEARCH-06, SEARCH-07
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. A single query returns authorized local and analyzed cloud results ranked through full-text and semantic relevance.
|
||||
2. Keyword and sentence searches match persisted extracted text, while idea searches use persisted embeddings or equivalent semantic indexes.
|
||||
3. Results clearly show source, provider, cloud location, topics, and current analysis/stale status and can be filtered by those fields.
|
||||
4. Search execution performs no provider file download; only opening or previewing a result may hydrate bytes.
|
||||
5. Search isolation, ranking, filtering, stale-result handling, and no-download-on-query invariants have automated coverage.
|
||||
|
||||
### Phase 16: Change Tracking and Reliability
|
||||
|
||||
**Goal:** DocuVault remains accurate when users or other applications modify connected cloud storage outside DocuVault.
|
||||
|
||||
**Depends on:** Phase 15
|
||||
**Requirements:** SYNC-02, SYNC-03, SYNC-04
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Provider delta feeds or bounded metadata refreshes detect changed versions and mark previously analyzed items stale until reanalysis.
|
||||
2. Items deleted outside DocuVault disappear from navigation and are excluded from search without deleting unrelated user data.
|
||||
3. Refresh jobs are idempotent, cursor-aware where supported, rate-limited, and retry transient failures with bounded exponential backoff.
|
||||
4. Users can see refresh/stale/error state and deliberately reanalyze changed items.
|
||||
5. End-to-end UAT verifies cloud-as-local operations, selective analysis, cache offload, unified smart search, and external-change recovery across supported provider classes.
|
||||
|
||||
## Coverage
|
||||
|
||||
- v0.3 requirements: 36
|
||||
- Mapped to phases: 36
|
||||
- Unmapped: 0
|
||||
- Duplicate mappings: 0
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Users & Authentication
|
||||
|
||||
**Goal**: Users can register, log in (with optional TOTP 2FA), reset their password, and sign out all active sessions; admins can manage user accounts and assign AI providers — all enforced by a complete FastAPI dependency chain.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 1
|
||||
**Requirements**: AUTH-01, AUTH-02, AUTH-03, AUTH-04, AUTH-05, AUTH-06, AUTH-07, AUTH-08, SEC-01, SEC-02, SEC-03, SEC-05, SEC-06, SEC-07, ADMIN-01, ADMIN-02, ADMIN-03, ADMIN-04, ADMIN-05, ADMIN-07
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A new user can register with an email and password that passes strength validation; a password from the HaveIBeenPwned list is rejected with a clear error
|
||||
2. A logged-in user can enroll a TOTP authenticator app, receive 8–10 backup codes, explicitly acknowledge them, and thereafter be required to supply a TOTP code (or backup code) on every login — a backup code is invalidated on first use
|
||||
3. A user who forgets their password can receive a reset email, follow the link within 1 hour, set a new password, and is then returned to the TOTP login gate (not auto-logged in)
|
||||
4. A user can trigger "sign out all devices" from account settings; all other active sessions are immediately invalidated and any reuse of a rotated refresh token revokes the entire token family
|
||||
5. An admin user can create, deactivate, and reset a user account, and assign an AI provider and model to that user; admin API endpoints never return document content or credentials_enc (per-user document auth enforcement deferred to Phase 3 per D-07)
|
||||
|
||||
**Plans**: 5 plans
|
||||
|
||||
**Wave 1** — Foundation
|
||||
|
||||
- [x] 02-01-PLAN.md — Auth service layer (Argon2, JWT, refresh tokens, TOTP, backup codes, HIBP, security alert), FastAPI deps, BackupCode model + password_must_change migration
|
||||
|
||||
**Wave 2** *(blocked on Wave 1 completion)*
|
||||
|
||||
- [x] 02-02-PLAN.md — Register/login (TOTP + backup code paths) + refresh/logout/change-password endpoints + CSP/Origin validation/rate-limit (IP + per-account) + Vue auth store + router guard + Login/Register views
|
||||
|
||||
**Wave 3** *(blocked on Wave 2 completion)*
|
||||
|
||||
- [x] 02-03-PLAN.md — TOTP enrollment + backup codes + password reset + sign-out-all endpoints + AccountView + TotpEnrollment + BackupCodesDisplay + PasswordReset views
|
||||
|
||||
**Wave 4** *(blocked on Wave 3 completion)*
|
||||
|
||||
- [x] 02-04-PLAN.md — Admin backend: user CRUD, quota, AI config endpoints with get_current_admin enforced + tests
|
||||
|
||||
**Wave 5** *(blocked on Wave 4 completion)*
|
||||
|
||||
- [x] 02-05-PLAN.md — Admin panel frontend: AdminView + three tab components + AppSidebar admin link and user identity footer
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- JWT access token in Pinia memory only — never localStorage (Plans 02, 03, 05)
|
||||
- Refresh token httpOnly SameSite=Strict cookie on all token issuance (Plans 02, 03)
|
||||
- Admin endpoints never return document content or credentials_enc (Plans 04, 05)
|
||||
- All auth endpoints rate-limited per-IP and per-account (Plans 02, 03)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Document Migration & Multi-User Isolation
|
||||
|
||||
**Goal**: All existing documents have been migrated from flat-file JSON + filesystem into PostgreSQL + MinIO; all new uploads use the presigned URL flow; per-user isolation is enforced at the DB level; the existing document UI works without regression; the backend is stateless and ready for horizontal scaling.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 2
|
||||
**Requirements**: STORE-03, STORE-04, STORE-05, STORE-06, STORE-08, SEC-04, DOC-03, DOC-04, DOC-05
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Every document present before migration is accessible after migration with the same metadata and extracted text; a count reconciliation check confirms zero document loss
|
||||
2. Two concurrent uploads that would together exceed a user's 100 MB quota result in exactly one success and one 413 rejection — the quota never goes over limit
|
||||
3. A document delete atomically decrements the user's recorded quota usage; after deletion the quota reflects the freed bytes
|
||||
4. Requesting a document object key or presigned URL for a document owned by a different user returns 403 — no cross-user object access is possible through any request parameter manipulation; all /api/documents/* endpoints enforce get_current_user and return 403 when the requesting user's role is admin (completing SC5 from Phase 2)
|
||||
5. AI classification for each document uses the provider and model assigned to that user by the admin, not any user-supplied or default value
|
||||
|
||||
**Plans**: 5 plans
|
||||
|
||||
**Wave 1** — Migration + test scaffolds
|
||||
|
||||
- [x] 03-01-PLAN.md — Wave 0 test scaffolds (auth_user/admin_user/MinIO mock fixtures + 19 xfail stubs) + Alembic migration 0003 (null-user cleanup, NOT NULL constraint, topic cleanup, quota reconciliation, ix_topics_user_id) — Complete 2026-05-23
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [ ] 03-02-PLAN.md — Presigned upload backend: StorageBackend ABC + MinIOBackend dual client + generate_presigned_put_url/stat_object + /api/documents/upload-url + /api/documents/{id}/confirm with atomic quota UPDATE + GET /api/auth/me/quota + delete-with-quota + abandoned-upload Celery beat + docker-compose CORS/celery-beat
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)*
|
||||
|
||||
- [x] 03-03-PLAN.md — Auth guards: get_regular_user dep + ownership assertions on every /api/documents/* handler (404 not 403) + admin 403 + real user_id in object_key + namespace-scoped /api/topics/* + POST /api/admin/topics + classifier topic-namespace plumbing
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)*
|
||||
|
||||
- [x] 03-04-PLAN.md — Settings retirement + per-user AI: delete /api/settings + remove load_settings/save_settings + classifier accepts ai_provider/ai_model kwargs + Celery task resolves user.ai_provider via DB + frontend SettingsView placeholder + remove settings store/API — Complete 2026-05-23
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)*
|
||||
|
||||
- [ ] 03-05-PLAN.md — Frontend upload flow + quota bar: 3-step upload action with XHR progress + UploadProgress.vue progress bar and quota rejection error block + QuotaBar.vue + AppSidebar embed + quota state in auth store + human checkpoint
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- Atomic quota UPDATE pattern only lives in Plan 02; never duplicate (CLAUDE.md)
|
||||
- Every /api/documents/* handler injects get_regular_user (Plan 03)
|
||||
- AI provider/model resolved only via Celery task DB lookup (Plan 04)
|
||||
- Browser XHR PUT to MinIO sends NO Authorization header (Plan 05)
|
||||
|
||||
**Phase gates (must pass before Phase 3 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; presigned URL, quota enforcement, ownership isolation, and admin-403 all covered
|
||||
- [ ] Security agent: path traversal check on object key construction; cross-user IDOR tests; quota race condition test
|
||||
- [ ] Bandit + pip audit + npm audit all clean
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Folders, Sharing, Quotas & Document UX
|
||||
|
||||
**Goal**: Users have a complete document management experience — organized with folders, shared by handle, warned before they hit quota, able to preview PDFs in-browser, and served by a searchable document list; admins can view the append-only audit log.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 3
|
||||
**Requirements**: FOLD-01, FOLD-02, FOLD-03, FOLD-04, FOLD-05, SHARE-01, SHARE-02, SHARE-03, SHARE-04, SHARE-05, SEC-08, SEC-09, ADMIN-06, DOC-01, DOC-02
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A user can create, rename, and delete folders; moving a document between folders preserves its metadata and AI classification; deleting a non-empty folder prompts with the content count before proceeding
|
||||
2. A user can share a document with another user by handle; the recipient sees it appear in a "Shared with me" virtual folder with no storage quota charged against them; the owner can revoke access and the shared entry disappears immediately for the recipient
|
||||
3. The sidebar quota bar displays current usage in MB; it turns amber at 80% and red at 95%; an upload that would exceed the limit is rejected with an error showing current usage, the rejected file size, and a link to storage settings
|
||||
4. Any document in the user's library can be previewed in-browser as a PDF; document bytes are proxied through the app and no presigned URLs are exposed to the browser (native browser PDF rendering via Content-Type header)
|
||||
5. An admin can view the audit log filtered by date range, user, and action type; the log contains no document content, filenames, or extracted text; account deletion triggers cleanup of all user files before DB records are removed
|
||||
|
||||
**Plans**: 9 plans
|
||||
|
||||
**Wave 1** — Test scaffolds + DB migration (parallel)
|
||||
|
||||
- [x] 04-01-PLAN.md — Wave 0 test stubs: test_folders.py + test_shares.py + test_audit.py + proxy stubs in test_documents.py + SEC-08/SEC-09 stubs in test_security.py
|
||||
- [x] 04-02-PLAN.md — Alembic migration 0004 (users.pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw()
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [x] 04-03-PLAN.md — Audit service (write_audit_log) + Folders API (FOLD-01..05): POST/GET/PATCH/DELETE /api/folders + PATCH /api/documents/{id}/folder + document list sort/search/is_shared extension
|
||||
- [ ] 04-04-PLAN.md — Shares API (SHARE-01..05): POST/GET /api/shares + GET /api/shares/received + DELETE /api/shares/{id} with IDOR protection
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)*
|
||||
|
||||
- [ ] 04-05-PLAN.md — PDF streaming proxy GET /api/documents/{id}/content with Range header support + PATCH /api/auth/me/preferences (pdf_open_mode)
|
||||
- [ ] 04-06-PLAN.md — Admin audit log API (GET /api/admin/audit-log, CSV export) + Celery beat daily audit export task + celery_app.py beat schedule
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)*
|
||||
|
||||
- [ ] 04-07-PLAN.md — SEC-08/SEC-09 hardening + audit log backfill into auth.py/admin.py/documents.py + CloudConnectionOut Pydantic model + delete-user file cleanup
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)*
|
||||
|
||||
- [ ] 04-08-PLAN.md — Frontend data layer: API client functions + useFoldersStore + documents store extension + Vue Router routes (/folders/:folderId, /shared)
|
||||
|
||||
**Wave 6** *(blocked on Wave 5)*
|
||||
|
||||
- [ ] 04-09-PLAN.md — Frontend UI: all new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + view wiring (AppSidebar, DocumentCard, HomeView, FolderView, SharedView, SettingsView, AdminView) + human checkpoint
|
||||
|
||||
**Phase gates (must pass before Phase 4 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; folder ownership, share revocation, quota bar, PDF proxy (no presigned URL exposure) all covered
|
||||
- [ ] Security agent: audit log verified to contain zero document content; sharing IDOR tests; PDF proxy verified to not leak presigned URLs or object keys
|
||||
- [ ] Bandit + pip audit + npm audit all clean
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Cloud Storage Backends
|
||||
|
||||
**Goal**: Users can connect OneDrive, Google Drive, Nextcloud, or a generic WebDAV server as a personal storage backend; credentials are encrypted with a per-user HKDF-derived key; connection status is visible; local and cloud storage coexist; the `StorageBackend` ABC makes adding further backends straightforward.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 4
|
||||
**Requirements**: CLOUD-01, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A user can connect OneDrive, Google Drive, Nextcloud, or a WebDAV endpoint through an OAuth or credential flow; the connection status is displayed as `ACTIVE`, `REQUIRES_REAUTH`, or `ERROR` — never shows raw credentials
|
||||
2. When an OAuth token is revoked externally (simulated `invalid_grant` response), the connection status transitions to `REQUIRES_REAUTH` without a 500 error; the user is shown a re-authentication prompt
|
||||
3. A user can select their connected cloud backend as the default storage destination for new uploads; local MinIO storage remains available as an alternative; existing local documents are unaffected
|
||||
4. A user can disconnect a cloud backend; credentials are permanently deleted from the DB and a subsequent attempt to use that backend returns an appropriate error — no orphaned data remains
|
||||
5. An admin API response for a user's cloud connections returns only `provider, display_name, connected_at, status` — the `credentials_enc` column is never present in any serialized response
|
||||
|
||||
**Plans**: 12 plans (8 original + 3 UAT gap closure + 1 gap closure wave)
|
||||
|
||||
**Wave 1** — Test scaffold + dependencies
|
||||
|
||||
- [x] 05-01-PLAN.md — Wave 0 xfail stubs, conftest cloud fixtures, requirements.txt packages, config.py settings
|
||||
|
||||
**Wave 2** — Shared utilities
|
||||
|
||||
- [x] 05-02-PLAN.md — cloud_utils.py (SSRF + HKDF), cloud_cache.py (TTLCache), storage factory extension
|
||||
|
||||
**Wave 3** — Cloud backends (parallel, both blocked on Wave 2 / Plan 05-02)
|
||||
|
||||
- [x] 05-03-PLAN.md — GoogleDriveBackend + OneDriveBackend (all 7 StorageBackend methods)
|
||||
- [x] 05-04-PLAN.md — NextcloudBackend + WebDAVBackend (all 7 StorageBackend methods)
|
||||
|
||||
**Wave 4** — Cloud API
|
||||
|
||||
- [x] 05-05-PLAN.md — All /api/cloud/* endpoints + /api/users/me/default-storage + main.py router registration
|
||||
|
||||
**Wave 5** — Document routing + full test suite
|
||||
|
||||
- [x] 05-06-PLAN.md — Upload/content proxy cloud routing + all 15 tests promoted to passing
|
||||
|
||||
**Wave 6** — Frontend settings UI
|
||||
|
||||
- [x] 05-07-PLAN.md — cloudConnections store + API client + SettingsView 3-tab + SettingsCloudTab + CloudCredentialModal
|
||||
|
||||
**Wave 7** — Frontend sidebar (human checkpoint)
|
||||
|
||||
- [x] 05-08-PLAN.md — AppSidebar cloud section + CloudProviderTreeItem + CloudFolderTreeItem + human checkpoint
|
||||
|
||||
**Wave 8** — UAT gap closure (parallel, all independent)
|
||||
|
||||
- [x] 05-09-PLAN.md — Cloud document open/re-analyze/edit: authenticated fetch+Blob URL, cloud-aware Celery task, PATCH /api/documents/{id}
|
||||
- [x] 05-10-PLAN.md — OAuth initiate fix (JSON response), Nextcloud custom endpoint edit round-trip, Edit button on ERROR rows, confirmation text overflow
|
||||
- [x] 05-11-PLAN.md — Admin hard-delete with password confirmation: UserDeleteConfirm backend model + inline frontend panel
|
||||
|
||||
**Wave 9** — Post-UAT gap closure
|
||||
|
||||
- [x] 05-12-PLAN.md — OAuth 400 preflight (unconfigured creds), 502 cloud fallback, upload hint in CloudStorageView, celery-worker volume mount
|
||||
|
||||
**Phase gates (must pass before Phase 5 is complete):**
|
||||
|
||||
- [x] `pytest -v` — zero failures; SSRF prevention on WebDAV/Nextcloud user-supplied URLs; credential encryption/decryption round-trip; admin response never exposes `credentials_enc`; OAuth invalid_grant handling
|
||||
- [x] Security agent: SSRF allowlist verification; credential key derivation correctness; connection status never leaks raw credential values
|
||||
- [x] Bandit + pip audit + npm audit all clean
|
||||
- [x] UAT gaps resolved and re-tested (05-09, 05-10, 05-11, 05-12)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Performance & Production Hardening
|
||||
|
||||
**Goal**: The application is ready for production deployment — observable, load-tested, and hardened; response times meet SLA targets under concurrent load; all auth and document endpoints are rate-limited; structured logging and distributed tracing are in place; the Docker image runs as a non-root user with a read-only filesystem.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 5
|
||||
**Requirements**: TBD
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. All API endpoints respond within defined latency targets (p50/p95/p99) under a realistic load test (e.g., 50 concurrent users, 5-minute soak)
|
||||
2. Structured JSON logging (correlation IDs, user ID, request latency) is emitted to stdout; a local log aggregation stack (Loki or similar) captures and queries them
|
||||
3. All auth endpoints (login, register, password reset, TOTP) enforce per-IP and per-account rate limits that cannot be bypassed by header manipulation
|
||||
4. Container hardening is complete: non-root user, read-only root filesystem, dropped Linux capabilities; `docker scout` or equivalent reports zero critical CVEs
|
||||
5. A runbook documents all environment variables, startup/shutdown procedures, backup strategy, and on-call escalation path; the app can be stood up from scratch using only the runbook
|
||||
|
||||
**Plans**: 6 plans (4 waves)
|
||||
|
||||
**Wave 0** — Test scaffolds + package verification
|
||||
|
||||
- [x] 06-01-PLAN.md — Nyquist Wave 0: xfail stubs (test_logging.py, test_rate_limiting.py), Locust skeleton, package legitimacy checkpoint (D-01, D-04, D-11, D-12)
|
||||
|
||||
**Wave 1** *(blocked on Wave 0 completion)*
|
||||
|
||||
- [x] 06-02-PLAN.md — structlog JSON logging + CorrelationIDMiddleware + Loki/Promtail/Grafana Docker Compose stack (D-01, D-02, D-03)
|
||||
- [x] 06-03-PLAN.md — Locust locustfile.py with JWT auth + SLA gate listener (D-04, D-05, D-06)
|
||||
|
||||
**Wave 2** *(blocked on Wave 1 completion)*
|
||||
|
||||
- [x] 06-04-PLAN.md — Multi-stage Dockerfile + read_only/tmpfs/cap_drop on backend + celery-worker (D-07, D-08, D-09)
|
||||
- [x] 06-05-PLAN.md — Trusted-proxy get_client_ip body + per-account rate limiter on document/cloud endpoints (D-11, D-12, D-13)
|
||||
|
||||
**Wave 3** *(blocked on Wave 2 completion)*
|
||||
|
||||
- [x] 06-06-PLAN.md — docker scout CVE gate + RUNBOOK.md (D-10, D-14)
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- get_client_ip lives ONLY in backend/deps/utils.py — replace body in-place, no new function (Plans 05)
|
||||
- celery-beat intentionally excluded from read_only: true (Plan 04)
|
||||
- locust must NOT be in requirements.txt — use requirements-dev.txt (Plan 03)
|
||||
- CorrelationIDMiddleware registered LAST in main.py — Starlette reverse order (Plan 02)
|
||||
|
||||
---
|
||||
|
||||
### Phase 6.1: Close v1.0 audit gaps: SHARE-02/STORE-06/ADMIN-06
|
||||
|
||||
**Goal**: Close three v1.0 requirements that remain unimplemented — atomic quota decrement on document delete (STORE-06), "Shared with me" virtual folder without recipient quota charge (SHARE-02), and admin audit log viewer with date/user/action type filters (ADMIN-06).
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 6
|
||||
**Requirements**: STORE-06, SHARE-02, ADMIN-06
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Deleting a document atomically decrements the owning user's quota; after deletion the quota reflects the freed bytes with no race condition under concurrent deletes
|
||||
2. A user who receives a shared document sees it appear in a "Shared with me" virtual folder; the recipient's quota usage is not charged for the shared document's storage
|
||||
3. An admin can view the audit log filtered independently by date range, user, and action type; filtered results contain no document content, filenames, or extracted text
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 1** — Test promotion (parallel)
|
||||
|
||||
- [x] 06.1-01-PLAN.md — Promote test_shares.py stubs to real tests + second_auth_user fixture (SHARE-01..05)
|
||||
- [x] 06.1-02-PLAN.md — Promote test_audit.py stubs to real tests (ADMIN-06)
|
||||
|
||||
**Phase gates (must pass before Phase 6.1 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; all 7 share tests + 4 audit log tests passing
|
||||
- [ ] Security agent: bandit + pip audit + npm audit all clean
|
||||
- [ ] STORE-06 confirmed: `test_delete_decrements_quota` passes under `INTEGRATION=1`
|
||||
|
||||
---
|
||||
|
||||
### Phase 6.2: Close v1 sharing + cloud-delete + CSV export gaps
|
||||
|
||||
**Goal**: Close remaining v1 gaps — sharing edge cases (SHARE-03/SHARE-05), cloud document deletion propagation to the remote backend, and CSV export + daily export UI for the admin audit log (ADMIN-06).
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 6.1
|
||||
**Requirements**: SHARE-03, SHARE-05, ADMIN-06
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Documents shared with others display a "Shared" badge in the owner's list view (reads doc.is_shared, not doc.share_count)
|
||||
2. Owner can set permission to "view" or "edit" when creating a share and toggle it per-recipient afterward; PATCH /api/shares/{id} enforces IDOR protection (404 on wrong owner)
|
||||
3. Deleting a cloud document propagates the delete to the cloud provider; failure shows a warning modal with "Remove from app" fallback; ?remove_only=true removes only the DB record; cloud docs never affect quota on delete
|
||||
4. Admin can download filtered audit log CSV via fetch+Blob (not window.location.href); audit log entries show user handles instead of raw UUIDs; user filter accepts handles (not UUIDs)
|
||||
5. Admin can list and download Celery-generated daily audit export files from a new section in the Audit Log tab
|
||||
|
||||
**Plans**: 4 plans
|
||||
|
||||
**Wave 0** — Test stubs
|
||||
|
||||
- [x] 06.2-01-PLAN.md — 11 xfail stubs across test_shares.py, test_documents.py, test_audit.py
|
||||
|
||||
**Wave 1** — Feature slices (parallel)
|
||||
|
||||
- [x] 06.2-02-PLAN.md — SHARE-05 badge fix + SHARE-03 permission control (backend PATCH + frontend dropdown + toggle)
|
||||
- [x] 06.2-03-PLAN.md — Cloud-delete propagation + structured error response + remove_only path + DocumentView warning modal
|
||||
|
||||
**Wave 2** — Audit log enrichment
|
||||
|
||||
- [x] 06.2-04-PLAN.md — Audit handle JOIN + user_handle filter + CSV fetch+Blob fix + daily-export list + download endpoints + AuditLogTab UI
|
||||
|
||||
**Phase gates (must pass before Phase 6.2 is complete):**
|
||||
|
||||
- [x] `pytest -v` — 344 passed, 1 pre-existing unrelated failure (test_extract_docx missing module)
|
||||
- [x] Security agent: bandit + pip audit + npm audit all clean (SECURITY.md threats_open: 0)
|
||||
- [x] IDOR on PATCH /api/shares/{id}: test_share_patch_idor passes
|
||||
- [x] Date regex validation confirmed: GET /api/admin/audit-log/daily-exports/invalid-date returns 404
|
||||
- [x] window.location.href removed from AuditLogTab.vue confirmed by grep
|
||||
|
||||
**Status: ✓ Complete (2026-06-01)**
|
||||
|
||||
### Phase 7: Redo and optimize LLM integration
|
||||
|
||||
**Goal:** A fully refactored, production-reliable AI provider layer: AI provider settings live in a new `system_settings` DB table (replacing env-only config), API keys encrypted at rest via HKDF/AES-GCM (same pattern as cloud credentials), all OpenAI-compatible providers (Groq, xAI, DeepSeek, OpenRouter, Gemini-compat, Mistral-compat, Ollama, LMStudio) handled by a single `GenericOpenAIProvider` class, Anthropic uses native `output_config.format.type="json_schema"` structured output, JSON-mode enforced everywhere with `parse_classification()` as fallback, Celery exponential-backoff retry (30s/90s/270s) replaces silent failure on classification errors, per-provider context window size with smart 60/40 truncation replaces the global `MAX_AI_CHARS`, the singleton `_client` pattern restores httpx connection-pool reuse, an admin AI Providers panel allows interactive configuration plus test-connection, and a "Re-analyze" button on the document card re-queues failed classifications.
|
||||
**Mode:** standard
|
||||
**Depends on:** Phase 6.2
|
||||
**Requirements**: D-01..D-18 (CONTEXT.md decisions; no REQ-IDs yet mapped — phase introduces new infrastructure)
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. AI provider settings live in `system_settings`; admin can view, edit, and atomically activate any provider via `PUT /api/admin/ai-config`; `GET /api/admin/ai-config` never returns `api_key_enc` or any decrypted key value
|
||||
2. A document whose classification raises an exception is automatically retried by Celery at 30 s, 90 s, and 270 s; after the third failure the document's status remains `classification_failed` and no further retries occur
|
||||
3. All OpenAI-compatible providers route through `GenericOpenAIProvider`; classify/suggest pass `response_format={"type":"json_object"}` when `supports_json_mode` is True (Gemini preset is False and falls back to `parse_classification()`); Anthropic uses `output_config.format.type="json_schema"` with a constrained schema
|
||||
4. `MAX_AI_CHARS` no longer exists in `openai_provider.py`, `anthropic_provider.py`, or `classifier.py`; each provider truncates input text using its own `context_chars` value via a 60% head + 40% tail strategy
|
||||
5. A failed document shows a red "Classification failed" badge and a "Re-analyze" button on the document card; clicking the button calls `POST /api/documents/{id}/classify`, which sets the document to `processing` and re-queues the Celery task
|
||||
|
||||
**Plans**: 5 plans (5 waves)
|
||||
|
||||
**Wave 1** — Foundation: migration, ORM model, encryption helpers, Wave 0 test stubs
|
||||
|
||||
- [x] 07-01-PLAN.md — Alembic migration 0005 (system_settings table) + SystemSettings ORM model + services/ai_config.py (HKDF helpers + load_provider_config + env seed on startup) + Wave 0 xfail stubs for D-01..D-16 (test_ai_providers.py, test_ai_config.py, test_admin_ai_config.py, test_document_tasks.py)
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)* — Provider layer: ProviderConfig, GenericOpenAIProvider, singleton client fix, registry
|
||||
|
||||
- [x] 07-02-PLAN.md — ProviderConfig Pydantic model + PROVIDER_DEFAULTS + SUPPORTS_JSON_MODE + GenericOpenAIProvider(OpenAIProvider) + OpenAIProvider singleton refactor + ollama/lmstudio context_chars + MAX_AI_CHARS removal from openai_provider.py + classifier.py + registry-based ai/__init__.py get_provider(config: ProviderConfig) + anthropic>=0.95.0 pin
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)* — Anthropic + Classifier wiring
|
||||
|
||||
- [x] 07-03-PLAN.md — AnthropicProvider singleton + output_config json_schema + smart truncation + MAX_AI_CHARS removal + classifier.classify_document driven by load_provider_config (no inline _settings dict) + ai_config stub removed in favor of real ProviderConfig + load_provider_config_by_id helper
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)* — Celery retry harness + Re-queue endpoint
|
||||
|
||||
- [x] 07-04-PLAN.md — Celery extract_and_classify decorator gains bind=True + max_retries=3 + _ClassificationError sentinel + 30/90/270 backoff + _mark_classification_failed on exhaustion + POST /api/documents/{id}/classify changes to re-queue Celery (sets status=processing, calls .delay()) — promotes test_retry_backoff, test_exhaustion_sets_failed_status, test_reclassify_requeues_celery
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)* — Admin UI + Frontend badge
|
||||
|
||||
- [x] 07-05-PLAN.md — Admin AI-config backend (GET/PUT/test-connection with whitelist + audit logging + atomic is_active flip) + frontend API client helpers (getAiConfig/saveAiConfig/testAiConnection) + AdminAiConfigTab.vue global System AI Providers section ABOVE existing per-user table + DocumentCard.vue classification_failed badge + Re-analyze button + human checkpoint UAT
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- `api_key_enc` is never returned by any admin endpoint — enforced by `_ai_config_to_dict()` whitelist (Plan 05)
|
||||
- HKDF domain separation: AI settings use `info=b"ai-provider-settings"`, cloud credentials use `info=b"cloud-credentials"` (Plan 01)
|
||||
- `is_active` flip is atomic: single `UPDATE SET is_active = (provider_id = $target)` — never read-then-write (Plan 05)
|
||||
- Provider clients are stored as `self._client` in `__init__` — never recreated per call (Plans 02, 03)
|
||||
- `MAX_AI_CHARS` removal must cover all three locations: openai_provider.py L5, anthropic_provider.py L5, classifier.py L28 (Plans 02 + 03)
|
||||
- Celery `self.retry()` must be raised from the outer sync task body, never inside `asyncio.run()` (Plan 04 — Pitfall 3)
|
||||
- `extra_hosts: ["host.docker.internal:host-gateway"]` already present in docker-compose.yml — D-14 is already done; no docker-compose changes required (RESEARCH.md)
|
||||
- AdminAiConfigTab.vue per-user assignment table is preserved untouched; the new global system section is added ABOVE it (Plan 05 — Pitfall 6)
|
||||
|
||||
**Phase gates (must pass before Phase 7 is complete):**
|
||||
|
||||
- [x] `pytest -v` — zero failures; D-01/D-03/D-05/D-06/D-07/D-09/D-10/D-11/D-12/D-13/D-16 covered by promoted tests (347 passed, 1 pre-existing failure test_extract_docx missing module)
|
||||
- [x] Security agent: bandit -r backend/ (zero HIGH), npm audit --audit-level=high (2 moderate esbuild/vite dev-only — no high/critical); pip-audit not runnable locally (Python 3.9 vs 3.12 requirements), inherited clean gate from Phase 6.2
|
||||
- [x] `_ai_config_to_dict()` confirmed to never include `api_key_enc` (test_get_never_returns_key passes; whitelist at admin.py:62)
|
||||
- [x] HKDF domain separation verified: test_encrypt_api_key_domain_isolation passes (test_ai_config.py:21)
|
||||
- [x] Atomic `is_active` flip verified: test_set_active_provider_atomic passes (test_admin_ai_config.py:77)
|
||||
- [x] Human checkpoint UAT approves admin panel + DocumentCard badge end-to-end (07-UAT.md: 11/11 passed 2026-06-05)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.1: Security: session revocation on privilege change (CR-01..03) (INSERTED)
|
||||
|
||||
**Goal**: Fix the three missing session-revocation calls in `backend/api/auth.py`: `change_password`, `enable_totp`, and `disable_totp` must all call `revoke_all_refresh_tokens()` (excluding the current session). Add `sessions_revoked` to their response shapes and a frontend toast when other sessions are terminated.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7
|
||||
**Requirements**: CR-01, CR-02, CR-03
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 1** — Backend: service + API changes
|
||||
|
||||
- [ ] 07.1-01-PLAN.md — Add skip_token_hash param to revoke_all_refresh_tokens + wire revoke into change_password, enable_totp, disable_totp with sessions_revoked response + audit log
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [ ] 07.1-02-PLAN.md — Tests (3 new test_*_revokes_other_sessions) + frontend toast in SettingsAccountTab.vue + TotpEnrollment.vue
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.2: Security — JTI Claim + Redis Access-Token Revocation (INSERTED)
|
||||
|
||||
**Goal**: Add a `jti` (JWT ID) claim to every issued access token. In `get_current_user`, check `redis.get("jti_revoked:{jti}")` and raise 401 if set. Add `revoke_access_token(jti, ttl)` helper called from `change_password`, `enable_totp`, `disable_totp`, and admin account deactivation. Closes the 15-minute window where a revoked session's live access token remains valid.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.1
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"No JTI Claim and No JTI Revocation in Redis"
|
||||
|
||||
**Status:** Not planned yet
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.3: Security — ES256 Algorithm Upgrade (INSERTED)
|
||||
|
||||
**Goal**: Replace HS256 with ES256 (ECDSA P-256) for JWT signing. Generate a P-256 key pair; store private key in `JWT_PRIVATE_KEY` env var, public key in `JWT_PUBLIC_KEY`. Update `create_access_token` and all `decode_*` functions. Rotate all active refresh tokens on first boot after deploy. A leaked public key cannot forge tokens.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.2
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"JWT Algorithm Downgrade: HS256 Instead of ES256"
|
||||
|
||||
**Plans**: 3 plans
|
||||
|
||||
**Wave 0** — Test scaffolds (no production code)
|
||||
|
||||
- [x] 07.3-01-PLAN.md — Wave 0 xfail stubs: test_auth_es256.py (9 stubs covering ES256-01..05 + RM-01..03 + CFG-01 satellite) + extend test_settings_has_jwt_config for refresh_token_expire_hours
|
||||
|
||||
**Wave 1** *(blocked on Wave 0)* — ES256 core + startup rotation
|
||||
|
||||
- [x] 07.3-02-PLAN.md — config.py jwt_private_key/jwt_public_key/refresh_token_expire_hours + services/auth.py 4 sites to ES256 + main.py _rotate_tokens_on_algorithm_change lifespan hook + docker-compose JWT_PRIVATE_KEY/JWT_PUBLIC_KEY + README key-gen snippet + .env.example + version bump 0.1.2
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)* — "Remember me" 16h/30d TTL split
|
||||
|
||||
- [x] 07.3-03-PLAN.md — create_refresh_token remember_me param + LoginRequest.remember_me + _set_refresh_cookie max_age conditional + LoginView.vue "Stay signed in for 30 days" checkbox + stores/auth.js + api/client.js forwarding + human checkpoint
|
||||
|
||||
**Status:** Complete (2026-06-06)
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.4: Security — Token Fingerprinting / Token Binding (INSERTED)
|
||||
|
||||
**Goal**: Add a `fgp` (fingerprint) claim = `hmac(key, User-Agent + Accept-Language)[:16]` to every issued access token. In `get_current_user`, recompute the fingerprint from the request headers and compare with `hmac.compare_digest`. Limits replay of stolen access tokens to the original device/browser context.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.3
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"No Token Fingerprint / Token Binding"
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 0** — Test scaffolds (no production code)
|
||||
|
||||
- [x] 07.4-01-PLAN.md — Wave 0 xfail stubs: test_auth_fgp.py (4 stubs covering FGP-01..04)
|
||||
|
||||
**Wave 1** *(blocked on Wave 0)* — Production implementation + test promotion
|
||||
|
||||
- [x] 07.4-02-PLAN.md — _compute_fgp helper + create_access_token fgp claim + get_current_user fgp validation + login/refresh call site updates + promote all 4 FGP tests
|
||||
|
||||
**Status:** Complete (2026-06-06)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
- ✅ **v0.2 — UI Overhaul and Optimization** — Phases 8–11 (shipped 2026-06-17)
|
||||
|
||||
<details>
|
||||
<summary>✅ v0.2 — UI Overhaul and Optimization (Phases 8–11) — SHIPPED 2026-06-17</summary>
|
||||
|
||||
- [x] Phase 8: Stack Upgrade & Backend Decomposition (8/8 plans) — completed 2026-06-12
|
||||
- [x] Phase 9: Admin Panel Rearchitecture (5/5 plans) — completed 2026-06-13
|
||||
- [x] Phase 10: UX & Interaction (13/13 plans) — completed 2026-06-16
|
||||
- [x] Phase 11: Visual Design, Responsive Layout & Cleanup (7/7 plans) — completed 2026-06-17
|
||||
|
||||
Full archive: `.planning/milestones/v0.2-ROADMAP.md`
|
||||
|
||||
</details>
|
||||
|
||||
## v0.2 + v0.1 Progress Table
|
||||
|
||||
| Phase | Milestone | Plans Complete | Status | Completed |
|
||||
|-------|-----------|----------------|--------|-----------|
|
||||
| 8. Stack Upgrade & Backend Decomposition | v0.2 | 8/8 | Complete | 2026-06-12 |
|
||||
| 9. Admin Panel Rearchitecture | v0.2 | 5/5 | Complete | 2026-06-13 |
|
||||
| 10. UX & Interaction | v0.2 | 13/13 | Complete | 2026-06-16 |
|
||||
| 11. Visual Design, Responsive Layout & Cleanup | v0.2 | 7/7 | Complete | 2026-06-17 |
|
||||
| 1. Infrastructure Foundation | v0.1 | 5/5 | Complete | 2026-05-22 |
|
||||
| 2. Users & Authentication | v0.1 | 6/6 | Complete | 2026-06-01 |
|
||||
| 3. Document Migration & Multi-User Isolation | v0.1 | 5/5 | Complete | 2026-05-25 |
|
||||
| 4. Folders, Sharing, Quotas & Document UX | v0.1 | 9/9 | Complete | 2026-05-28 |
|
||||
| 5. Cloud Storage Backends | v0.1 | 12/12 | Complete | 2026-05-30 |
|
||||
| 6. Performance & Production Hardening | v0.1 | 6/6 | Complete | 2026-05-30 |
|
||||
| 6.1. Close v1.0 audit gaps | v0.1 | 2/2 | Complete | 2026-05-30 |
|
||||
| 6.2. Close v1 sharing + cloud-delete + CSV export gaps | v0.1 | 5/5 | Complete | 2026-05-31 |
|
||||
| 7. Redo and optimize LLM integration | v0.1 | 5/5 | Complete | 2026-06-05 |
|
||||
| 7.1. Security: session revocation on privilege change | v0.1 | 2/2 | Complete | 2026-06-08 |
|
||||
| 7.2. Security: JTI claim + Redis access-token revocation | v0.1 | 3/3 | Complete | 2026-06-05 |
|
||||
| 7.3. Security: ES256 algorithm upgrade | v0.1 | 3/3 | Complete | 2026-06-06 |
|
||||
| 7.4. Security: token fingerprinting / token binding | v0.1 | 2/2 | Complete | 2026-06-06 |
|
||||
*Roadmap proposed: 2026-06-17*
|
||||
|
||||
+20
-17
@@ -2,11 +2,12 @@
|
||||
gsd_state_version: 1.0
|
||||
milestone: v0.3
|
||||
milestone_name: Reimagining Cloud Storage integration
|
||||
current_phase: 12
|
||||
status: planning
|
||||
last_updated: "2026-06-17T21:15:24.719Z"
|
||||
last_activity: 2026-06-17
|
||||
last_updated: "2026-06-17"
|
||||
last_activity: 2026-06-17 -- v0.3 roadmap proposed
|
||||
progress:
|
||||
total_phases: 0
|
||||
total_phases: 5
|
||||
completed_phases: 0
|
||||
total_plans: 0
|
||||
completed_plans: 0
|
||||
@@ -16,33 +17,34 @@ progress:
|
||||
# Project State
|
||||
|
||||
**Project:** DocuVault
|
||||
**Status:** v0.3 milestone planning — defining requirements
|
||||
**Status:** v0.3 milestone planning — roadmap awaiting approval
|
||||
**Last Updated:** 2026-06-17
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: Not started (defining requirements)
|
||||
Phase: 12 of 16 — Cloud Resource Foundation
|
||||
Plan: —
|
||||
Status: Defining requirements
|
||||
Last activity: 2026-06-17 — Milestone v0.3 started
|
||||
Status: Roadmap proposed; phase not yet planned
|
||||
Last activity: 2026-06-17 — v0.3 roadmap proposed
|
||||
|
||||
## Phase Status
|
||||
|
||||
| Phase | Requirements | Status |
|
||||
|-------|-------------|--------|
|
||||
| 8. Stack Upgrade & Backend Decomposition | PERF-01, CODE-01, CODE-02, CODE-03, CODE-04, CODE-08 | **Complete (8/8 plans)** |
|
||||
| 9. Admin Panel Rearchitecture | ADMIN-08..12, CODE-06, CODE-09 | **Complete (5/5 plans)** |
|
||||
| 10. UX & Interaction | UX-01..14, CODE-05 | **Complete (13/13 plans)** |
|
||||
| 11. Visual Design, Responsive Layout & Cleanup | VISUAL-01..04, RESP-01..05, CODE-07, PERF-02, PERF-03 | **Complete (7/7 plans)** |
|
||||
| 12. Cloud Resource Foundation | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 | **Not started** |
|
||||
| 13. Virtual-Local Cloud Operations | CONN-01..03, CLOUD-02..07, CLOUD-09 | **Not started** |
|
||||
| 14. Selective Analysis and Byte Cache | ANALYZE-01..07, CACHE-03..05 | **Not started** |
|
||||
| 15. Unified Smart Search | SEARCH-01..07 | **Not started** |
|
||||
| 16. Change Tracking and Reliability | SYNC-02..04 | **Not started** |
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Phases complete | 4 / 4 |
|
||||
| Requirements satisfied | 40 / 40 |
|
||||
| Plans complete | 33 / 33 |
|
||||
| Tests at close | 277 |
|
||||
| Phases complete | 0 / 5 |
|
||||
| Requirements satisfied | 0 / 36 |
|
||||
| Plans complete | 0 / 0 |
|
||||
| Tests at milestone start | 277 |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -67,6 +69,7 @@ Last activity: 2026-06-17 — Milestone v0.3 started
|
||||
- v0.1 completed: all 7 foundation phases + security hardening (2026-06-06)
|
||||
- v0.2 completed: UI overhaul, admin panel rearchitecture, responsive layout, codebase quality (2026-06-17)
|
||||
- v0.2 archived to `.planning/milestones/v0.2-ROADMAP.md`
|
||||
- v0.3 proposed: virtual-local cloud operations, selective cloud analysis, bounded byte caching, unified smart search, and provider change tracking
|
||||
|
||||
### Open Questions
|
||||
|
||||
@@ -82,7 +85,7 @@ _Updated at each phase transition._
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Last session | 2026-06-17 — v0.3 milestone started |
|
||||
| Next action | Define v0.3 requirements and roadmap |
|
||||
| Last session | 2026-06-17 — v0.3 roadmap proposed |
|
||||
| Next action | Approve roadmap, then discuss or plan Phase 12 |
|
||||
| Pending decisions | None |
|
||||
| Resume file | None |
|
||||
|
||||
Reference in New Issue
Block a user