kite/.planning/ROADMAP.md

# DocuVault — v1 Roadmap

_Last updated: 2026-05-25_

## Mandatory Cross-Cutting Gates (every phase)

Before any phase is marked complete, all three gates must pass:

1. **Test gate** — `pytest -v` passes with zero failures; every new function/endpoint has at least one test; all security invariant tests pass (wrong owner, admin block, token replay)
2. **Security gate** — Security agent runs `bandit -r backend/` (zero HIGH), `pip audit` (zero critical/high), `npm audit --audit-level=high` (zero high/critical); admin endpoints verified to never return `password_hash`, `credentials_enc`, or document content; no hardcoded secrets
3. **Bug fix rule** — Any bug fix during execution must: (a) target the root cause, (b) change ≤50 lines, (c) include a regression test — no workarounds permitted

---

## Phases

- [x] **Phase 1: Infrastructure Foundation** — PostgreSQL + MinIO wired into Docker Compose; Alembic migrations running; existing app still works
- [x] **Phase 2: Users & Authentication** — Full auth flow end-to-end (register, login, TOTP, backup codes, password reset, sign-out-all) with admin panel for user management
- [x] **Phase 3: Document Migration & Multi-User Isolation** — All documents in PostgreSQL + MinIO; per-user isolation enforced; existing UI still works
- [x] **Phase 4: Folders, Sharing, Quotas & Document UX** — Full document management UX (folders, sharing, quota bar, PDF preview, search, audit log)
- [x] **Phase 5: Cloud Storage Backends** — Users can connect OneDrive, Google Drive, Nextcloud, or WebDAV as a personal storage backend

---

## Phase Details

### Phase 1: Infrastructure Foundation

**Goal**: PostgreSQL + MinIO are wired into Docker Compose with a complete Alembic-managed schema; all services boot cleanly and the existing single-user document scanner continues to work exactly as before — no user-facing behavior change.
**Mode:** mvp
**Depends on**: Nothing (first phase)
**Requirements**: STORE-01, STORE-02, STORE-07

**Success Criteria** (what must be TRUE):

1. `docker compose up` starts PostgreSQL, MinIO, and the FastAPI backend with no errors; health checks pass for all three services
2. Running `alembic upgrade head` applies the initial migration cleanly against the fresh PostgreSQL instance with no errors
3. The full existing document upload, text extraction, and AI classification workflow completes successfully — no regression in single-user behavior
4. MinIO object key schema `{user_id}/{document_id}/{uuid4()}{ext}` is enforced in the model layer; human-readable filenames are stored in the DB column, not in the MinIO key

**Plans**: 5 plans

- [x] 01-01-PLAN.md — Docker Compose service topology + Postgres init + Pydantic Settings + requirements
- [x] 01-02-PLAN.md — Wave 0 test scaffolds (xfail/skip stubs) + async pytest fixtures
- [x] 01-03-PLAN.md — SQLAlchemy ORM models + async engine + Alembic async migration (incl. alembic upgrade head)
- [x] 01-04-PLAN.md — StorageBackend ABC + MinIO backend + rewritten async services/storage.py
- [x] 01-05-PLAN.md — Lifespan + /health + API cutover + Celery worker + walking-skeleton e2e verify

---

### Phase 2: Users & Authentication

**Goal**: Users can register, log in (with optional TOTP 2FA), reset their password, and sign out all active sessions; admins can manage user accounts and assign AI providers — all enforced by a complete FastAPI dependency chain.
**Mode:** mvp
**Depends on**: Phase 1
**Requirements**: AUTH-01, AUTH-02, AUTH-03, AUTH-04, AUTH-05, AUTH-06, AUTH-07, AUTH-08, SEC-01, SEC-02, SEC-03, SEC-05, SEC-06, SEC-07, ADMIN-01, ADMIN-02, ADMIN-03, ADMIN-04, ADMIN-05, ADMIN-07

**Success Criteria** (what must be TRUE):

1. A new user can register with an email and password that passes strength validation; a password from the HaveIBeenPwned list is rejected with a clear error
2. A logged-in user can enroll a TOTP authenticator app, receive 8–10 backup codes, explicitly acknowledge them, and thereafter be required to supply a TOTP code (or backup code) on every login — a backup code is invalidated on first use
3. A user who forgets their password can receive a reset email, follow the link within 1 hour, set a new password, and is then returned to the TOTP login gate (not auto-logged in)
4. A user can trigger "sign out all devices" from account settings; all other active sessions are immediately invalidated and any reuse of a rotated refresh token revokes the entire token family
5. An admin user can create, deactivate, and reset a user account, and assign an AI provider and model to that user; admin API endpoints never return document content or credentials_enc (per-user document auth enforcement deferred to Phase 3 per D-07)

**Plans**: 5 plans

**Wave 1** — Foundation

- [x] 02-01-PLAN.md — Auth service layer (Argon2, JWT, refresh tokens, TOTP, backup codes, HIBP, security alert), FastAPI deps, BackupCode model + password_must_change migration

**Wave 2** *(blocked on Wave 1 completion)*

- [x] 02-02-PLAN.md — Register/login (TOTP + backup code paths) + refresh/logout/change-password endpoints + CSP/Origin validation/rate-limit (IP + per-account) + Vue auth store + router guard + Login/Register views

**Wave 3** *(blocked on Wave 2 completion)*

- [x] 02-03-PLAN.md — TOTP enrollment + backup codes + password reset + sign-out-all endpoints + AccountView + TotpEnrollment + BackupCodesDisplay + PasswordReset views

**Wave 4** *(blocked on Wave 3 completion)*

- [x] 02-04-PLAN.md — Admin backend: user CRUD, quota, AI config endpoints with get_current_admin enforced + tests

**Wave 5** *(blocked on Wave 4 completion)*

- [x] 02-05-PLAN.md — Admin panel frontend: AdminView + three tab components + AppSidebar admin link and user identity footer

**Cross-cutting constraints:**

- JWT access token in Pinia memory only — never localStorage (Plans 02, 03, 05)
- Refresh token httpOnly SameSite=Strict cookie on all token issuance (Plans 02, 03)
- Admin endpoints never return document content or credentials_enc (Plans 04, 05)
- All auth endpoints rate-limited per-IP and per-account (Plans 02, 03)

**UI hint**: yes

---

### Phase 3: Document Migration & Multi-User Isolation

**Goal**: All existing documents have been migrated from flat-file JSON + filesystem into PostgreSQL + MinIO; all new uploads use the presigned URL flow; per-user isolation is enforced at the DB level; the existing document UI works without regression; the backend is stateless and ready for horizontal scaling.
**Mode:** mvp
**Depends on**: Phase 2
**Requirements**: STORE-03, STORE-04, STORE-05, STORE-06, STORE-08, SEC-04, DOC-03, DOC-04, DOC-05

**Success Criteria** (what must be TRUE):

1. Every document present before migration is accessible after migration with the same metadata and extracted text; a count reconciliation check confirms zero document loss
2. Two concurrent uploads that would together exceed a user's 100 MB quota result in exactly one success and one 413 rejection — the quota never goes over limit
3. A document delete atomically decrements the user's recorded quota usage; after deletion the quota reflects the freed bytes
4. Requesting a document object key or presigned URL for a document owned by a different user returns 403 — no cross-user object access is possible through any request parameter manipulation; all /api/documents/* endpoints enforce get_current_user and return 403 when the requesting user's role is admin (completing SC5 from Phase 2)
5. AI classification for each document uses the provider and model assigned to that user by the admin, not any user-supplied or default value

**Plans**: 5 plans

**Wave 1** — Migration + test scaffolds

- [x] 03-01-PLAN.md — Wave 0 test scaffolds (auth_user/admin_user/MinIO mock fixtures + 19 xfail stubs) + Alembic migration 0003 (null-user cleanup, NOT NULL constraint, topic cleanup, quota reconciliation, ix_topics_user_id) — Complete 2026-05-23

**Wave 2** *(blocked on Wave 1)*

- [ ] 03-02-PLAN.md — Presigned upload backend: StorageBackend ABC + MinIOBackend dual client + generate_presigned_put_url/stat_object + /api/documents/upload-url + /api/documents/{id}/confirm with atomic quota UPDATE + GET /api/auth/me/quota + delete-with-quota + abandoned-upload Celery beat + docker-compose CORS/celery-beat

**Wave 3** *(blocked on Wave 2)*

- [x] 03-03-PLAN.md — Auth guards: get_regular_user dep + ownership assertions on every /api/documents/* handler (404 not 403) + admin 403 + real user_id in object_key + namespace-scoped /api/topics/* + POST /api/admin/topics + classifier topic-namespace plumbing

**Wave 4** *(blocked on Wave 3)*

- [x] 03-04-PLAN.md — Settings retirement + per-user AI: delete /api/settings + remove load_settings/save_settings + classifier accepts ai_provider/ai_model kwargs + Celery task resolves user.ai_provider via DB + frontend SettingsView placeholder + remove settings store/API — Complete 2026-05-23

**Wave 5** *(blocked on Wave 4)*

- [ ] 03-05-PLAN.md — Frontend upload flow + quota bar: 3-step upload action with XHR progress + UploadProgress.vue progress bar and quota rejection error block + QuotaBar.vue + AppSidebar embed + quota state in auth store + human checkpoint

**Cross-cutting constraints:**

- Atomic quota UPDATE pattern only lives in Plan 02; never duplicate (CLAUDE.md)
- Every /api/documents/* handler injects get_regular_user (Plan 03)
- AI provider/model resolved only via Celery task DB lookup (Plan 04)
- Browser XHR PUT to MinIO sends NO Authorization header (Plan 05)

**Phase gates (must pass before Phase 3 is complete):**

- [ ] `pytest -v` — zero failures; presigned URL, quota enforcement, ownership isolation, and admin-403 all covered
- [ ] Security agent: path traversal check on object key construction; cross-user IDOR tests; quota race condition test
- [ ] Bandit + pip audit + npm audit all clean

**UI hint**: yes

---

### Phase 4: Folders, Sharing, Quotas & Document UX

**Goal**: Users have a complete document management experience — organized with folders, shared by handle, warned before they hit quota, able to preview PDFs in-browser, and served by a searchable document list; admins can view the append-only audit log.
**Mode:** mvp
**Depends on**: Phase 3
**Requirements**: FOLD-01, FOLD-02, FOLD-03, FOLD-04, FOLD-05, SHARE-01, SHARE-02, SHARE-03, SHARE-04, SHARE-05, SEC-08, SEC-09, ADMIN-06, DOC-01, DOC-02

**Success Criteria** (what must be TRUE):

1. A user can create, rename, and delete folders; moving a document between folders preserves its metadata and AI classification; deleting a non-empty folder prompts with the content count before proceeding
2. A user can share a document with another user by handle; the recipient sees it appear in a "Shared with me" virtual folder with no storage quota charged against them; the owner can revoke access and the shared entry disappears immediately for the recipient
3. The sidebar quota bar displays current usage in MB; it turns amber at 80% and red at 95%; an upload that would exceed the limit is rejected with an error showing current usage, the rejected file size, and a link to storage settings
4. Any document in the user's library can be previewed in-browser as a PDF; document bytes are proxied through the app and no presigned URLs are exposed to the browser (native browser PDF rendering via Content-Type header)
5. An admin can view the audit log filtered by date range, user, and action type; the log contains no document content, filenames, or extracted text; account deletion triggers cleanup of all user files before DB records are removed

**Plans**: 9 plans

**Wave 1** — Test scaffolds + DB migration (parallel)

- [x] 04-01-PLAN.md — Wave 0 test stubs: test_folders.py + test_shares.py + test_audit.py + proxy stubs in test_documents.py + SEC-08/SEC-09 stubs in test_security.py
- [x] 04-02-PLAN.md — Alembic migration 0004 (users.pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw()

**Wave 2** *(blocked on Wave 1)*

- [x] 04-03-PLAN.md — Audit service (write_audit_log) + Folders API (FOLD-01..05): POST/GET/PATCH/DELETE /api/folders + PATCH /api/documents/{id}/folder + document list sort/search/is_shared extension
- [ ] 04-04-PLAN.md — Shares API (SHARE-01..05): POST/GET /api/shares + GET /api/shares/received + DELETE /api/shares/{id} with IDOR protection

**Wave 3** *(blocked on Wave 2)*

- [ ] 04-05-PLAN.md — PDF streaming proxy GET /api/documents/{id}/content with Range header support + PATCH /api/auth/me/preferences (pdf_open_mode)
- [ ] 04-06-PLAN.md — Admin audit log API (GET /api/admin/audit-log, CSV export) + Celery beat daily audit export task + celery_app.py beat schedule

**Wave 4** *(blocked on Wave 3)*

- [ ] 04-07-PLAN.md — SEC-08/SEC-09 hardening + audit log backfill into auth.py/admin.py/documents.py + CloudConnectionOut Pydantic model + delete-user file cleanup

**Wave 5** *(blocked on Wave 4)*

- [ ] 04-08-PLAN.md — Frontend data layer: API client functions + useFoldersStore + documents store extension + Vue Router routes (/folders/:folderId, /shared)

**Wave 6** *(blocked on Wave 5)*

- [ ] 04-09-PLAN.md — Frontend UI: all new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + view wiring (AppSidebar, DocumentCard, HomeView, FolderView, SharedView, SettingsView, AdminView) + human checkpoint

**Phase gates (must pass before Phase 4 is complete):**

- [ ] `pytest -v` — zero failures; folder ownership, share revocation, quota bar, PDF proxy (no presigned URL exposure) all covered
- [ ] Security agent: audit log verified to contain zero document content; sharing IDOR tests; PDF proxy verified to not leak presigned URLs or object keys
- [ ] Bandit + pip audit + npm audit all clean

**UI hint**: yes

---

### Phase 5: Cloud Storage Backends

**Goal**: Users can connect OneDrive, Google Drive, Nextcloud, or a generic WebDAV server as a personal storage backend; credentials are encrypted with a per-user HKDF-derived key; connection status is visible; local and cloud storage coexist; the `StorageBackend` ABC makes adding further backends straightforward.
**Mode:** mvp
**Depends on**: Phase 4
**Requirements**: CLOUD-01, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07

**Success Criteria** (what must be TRUE):

1. A user can connect OneDrive, Google Drive, Nextcloud, or a WebDAV endpoint through an OAuth or credential flow; the connection status is displayed as `ACTIVE`, `REQUIRES_REAUTH`, or `ERROR` — never shows raw credentials
2. When an OAuth token is revoked externally (simulated `invalid_grant` response), the connection status transitions to `REQUIRES_REAUTH` without a 500 error; the user is shown a re-authentication prompt
3. A user can select their connected cloud backend as the default storage destination for new uploads; local MinIO storage remains available as an alternative; existing local documents are unaffected
4. A user can disconnect a cloud backend; credentials are permanently deleted from the DB and a subsequent attempt to use that backend returns an appropriate error — no orphaned data remains
5. An admin API response for a user's cloud connections returns only `provider, display_name, connected_at, status` — the `credentials_enc` column is never present in any serialized response

**Plans**: 11 plans (8 original + 3 UAT gap closure)

**Wave 1** — Test scaffold + dependencies

- [x] 05-01-PLAN.md — Wave 0 xfail stubs, conftest cloud fixtures, requirements.txt packages, config.py settings

**Wave 2** — Shared utilities

- [x] 05-02-PLAN.md — cloud_utils.py (SSRF + HKDF), cloud_cache.py (TTLCache), storage factory extension

**Wave 3** — Cloud backends (parallel, both blocked on Wave 2 / Plan 05-02)

- [x] 05-03-PLAN.md — GoogleDriveBackend + OneDriveBackend (all 7 StorageBackend methods)
- [x] 05-04-PLAN.md — NextcloudBackend + WebDAVBackend (all 7 StorageBackend methods)

**Wave 4** — Cloud API

- [x] 05-05-PLAN.md — All /api/cloud/* endpoints + /api/users/me/default-storage + main.py router registration

**Wave 5** — Document routing + full test suite

- [x] 05-06-PLAN.md — Upload/content proxy cloud routing + all 15 tests promoted to passing

**Wave 6** — Frontend settings UI

- [x] 05-07-PLAN.md — cloudConnections store + API client + SettingsView 3-tab + SettingsCloudTab + CloudCredentialModal

**Wave 7** — Frontend sidebar (human checkpoint)

- [x] 05-08-PLAN.md — AppSidebar cloud section + CloudProviderTreeItem + CloudFolderTreeItem + human checkpoint

**Wave 8** — UAT gap closure (parallel, all independent)

- [ ] 05-09-PLAN.md — Cloud document open/re-analyze/edit: authenticated fetch+Blob URL, cloud-aware Celery task, PATCH /api/documents/{id}
- [ ] 05-10-PLAN.md — OAuth initiate fix (JSON response), Nextcloud custom endpoint edit round-trip, Edit button on ERROR rows, confirmation text overflow
- [ ] 05-11-PLAN.md — Admin hard-delete with password confirmation: UserDeleteConfirm backend model + inline frontend panel

**Phase gates (must pass before Phase 5 is complete):**

- [x] `pytest -v` — zero failures; SSRF prevention on WebDAV/Nextcloud user-supplied URLs; credential encryption/decryption round-trip; admin response never exposes `credentials_enc`; OAuth invalid_grant handling
- [x] Security agent: SSRF allowlist verification; credential key derivation correctness; connection status never leaks raw credential values
- [x] Bandit + pip audit + npm audit all clean
- [ ] UAT gaps 05-09, 05-10, 05-11 resolved and re-tested

**UI hint**: yes

---

## Progress Table

| Phase | Plans Complete | Status | Completed |
|-------|----------------|--------|-----------|
| 1. Infrastructure Foundation | 5/5 | Complete | 2026-05-22 |
| 2. Users & Authentication | 5/5 | Complete | 2026-05-22 |
| 3. Document Migration & Multi-User Isolation | 5/5 | Complete | 2026-05-25 |
| 4. Folders, Sharing, Quotas & Document UX | 9/9 | Complete | 2026-05-28 |
| 5. Cloud Storage Backends | 8/11 | UAT gap closure in progress | — |