diff --git a/.planning/PROJECT.md b/.planning/PROJECT.md index 42bc449..a4c332c 100644 --- a/.planning/PROJECT.md +++ b/.planning/PROJECT.md @@ -8,103 +8,93 @@ DocuVault is a self-hosted, multi-user SaaS document management platform. Users Every user's documents — and the credentials they use to store them — are inaccessible to everyone except that user, while the platform scales horizontally and supports pluggable storage backends. +## Current Milestone: v0.2 — UI Overhaul and Optimization + +**Goal:** Redesign the frontend into a polished, performant, and responsive interface, and overhaul both the backend and frontend codebase to senior-dev quality — minimal, DRY, readable code with no unnecessary comments. + +**Target features:** +- Visual redesign — refined Tailwind component system, consistent palette, typography, spacing +- UX & interaction improvements — drag-and-drop upload, keyboard shortcuts, empty states, loading skeletons +- Frontend performance — lazy loading, virtual scrolling, bundle size reduction, API response caching +- Responsive / mobile layout — adaptive layouts for phones/tablets, touch-friendly controls +- Proper admin panel — standalone admin interface with its own routes, layout, and nav (not tabs bolted onto the user app) +- Codebase quality overhaul — eliminate duplication, extract shared utilities, delete dead code, flatten unnecessary abstractions; comments only where code alone is insufficient + ## Requirements -### Validated +### Validated (v0.1 — shipped 2026-06-06) -*Capabilities already shipping in the codebase:* +- ✓ Document upload and text extraction (PDF, DOCX, image, plain text) +- ✓ AI-based topic classification via configurable provider +- ✓ Multiple AI provider support (Anthropic, OpenAI, GenericOpenAI-compat, Ollama, LMStudio) +- ✓ Topic CRUD management (per-user namespace) +- ✓ Docker containerization (Compose) with PostgreSQL + MinIO +- ✓ User registration with email/password (Argon2id, strength enforcement, HaveIBeenPwned check) +- ✓ JWT session management (ES256 asymmetric, 15 min access token, 16h/30d refresh, token fingerprinting, JTI revocation) +- ✓ TOTP 2FA with backup codes; session revocation on privilege change +- ✓ Admin: create, deactivate, reset password, assign AI provider per user +- ✓ Admin cannot access user documents or cloud credentials +- ✓ Per-user isolated storage with 100 MB free-tier quota (atomic enforcement) +- ✓ Folder creation, rename, delete, move documents between folders +- ✓ Document sharing by handle (view/edit permission, revocable, no recipient quota charge) +- ✓ "Shared with me" virtual folder +- ✓ Cloud storage backends: OneDrive, Google Drive, Nextcloud, WebDAV (HKDF-encrypted credentials) +- ✓ PDF in-browser preview (proxied, no presigned URL exposure) +- ✓ Full-text search and sorting on document list +- ✓ Audit log (metadata only): logins, uploads, deletes, shares, quota changes +- ✓ Admin audit log viewer with date/user/action filters and CSV export +- ✓ Structured JSON logging (structlog + correlation IDs) +- ✓ Container hardening (non-root user, read-only filesystem, dropped capabilities) +- ✓ AI provider settings in `system_settings` DB table (HKDF-encrypted API keys) +- ✓ Celery retry backoff (30 s / 90 s / 270 s) on classification failure +- ✓ Backend stateless — all state in PostgreSQL and MinIO -- ✓ Document upload and text extraction (PDF, DOCX, image, plain text) — existing -- ✓ AI-based topic classification via configurable provider — existing -- ✓ Multiple AI provider support (Anthropic, OpenAI, Ollama, LMStudio) — existing -- ✓ Topic CRUD management — existing -- ✓ System prompt configuration — existing -- ✓ Docker containerization (Compose) — existing +### Active (v0.2 — this milestone) -### Active - -**Users & Auth** -- [ ] User can register with email and password (enforced strength: length, complexity, breach check) -- [ ] User can log in and maintain session via JWT -- [ ] User can enable TOTP authenticator app for 2FA -- [ ] Admin can create, deactivate, and reset passwords for user accounts -- [ ] Admin cannot access any user's documents or cloud storage credentials - -**Storage & Quotas** -- [ ] Each user has an isolated storage area with a 100 MB free-tier quota -- [ ] Quota usage is tracked and enforced; uploads exceeding quota are rejected with a clear error -- [ ] Admin can adjust individual user storage quotas -- [ ] Platform migrates from flat-file JSON + filesystem to PostgreSQL + MinIO (S3-compatible) - -**Folder Structure** -- [ ] User can create, rename, and delete folders to organize documents -- [ ] Document organization is preserved on move/rename (no auto-rearrangement by AI) -- [ ] A "Shared with me" folder appears automatically when another user shares a document - -**Document Sharing** -- [ ] User can share a document (or folder) with another user by their unique handle -- [ ] Shared access is view-only by default; owner controls permission level -- [ ] Revoking share removes access immediately; shared copy is not duplicated in recipient's quota - -**Cloud Storage Integration** -- [ ] User can connect an external cloud storage backend (OneDrive, Google Drive, Nextcloud; extensible) -- [ ] Local storage and cloud storage coexist; user selects their default storage destination -- [ ] Cloud storage credentials are encrypted at rest and never readable by admins -- [ ] Documents stored in cloud backend are accessed via the app without being re-copied to local storage - -**AI Configuration (Admin-controlled)** -- [ ] Admin can assign an AI provider and model per user or per group -- [ ] System-wide default AI provider and model set by admin -- [ ] Users cannot change their own AI provider or model -- [ ] Per-user topic overrides on top of system default topics - -**Audit Logging** -- [ ] Audit log captures: logins, failed logins, uploads, deletes, sharing events, quota changes -- [ ] Audit log records metadata only — no document content -- [ ] Admin can view and filter audit logs - -**Scalability** -- [ ] Backend stateless — multiple instances can run behind a load balancer -- [ ] All state in PostgreSQL and MinIO (no local file locks, no per-instance JSON) +*Defined in REQUIREMENTS.md* ### Out of Scope -- Subscription billing / payment processing — future milestone (quotas designed to plug in) +- Subscription billing / payment processing — future milestone (quota table designed for it) - SSO (Microsoft, Google, Apple) — future; auth layer designed for extension - Keycloak / SAML / OAuth enterprise federation — future -- Group admin roles — future; groups table will be seeded in schema +- Group admin roles — future; groups table seeded in schema - Document annotation or in-app editing — not planned -- Mobile app — not planned -- Public document sharing (unauthenticated link) — not planned for v1 +- Mobile native app — not planned (responsive web only in v0.2) +- Public document sharing (unauthenticated link) — not planned for v0.x ## Context -- **Existing codebase**: Functional single-user document scanner (FastAPI + Vue 3, Docker Compose). AI provider abstraction already in place — cloud storage will follow the same adapter pattern. -- **Brownfield migration**: Flat-file JSON persistence and per-process file locks must be replaced with PostgreSQL + MinIO before multi-user isolation is safe. -- **Privacy constraint**: SaaS model with strict admin/user data separation. Admin role is a platform operator, not a content viewer. Cloud credentials must be encrypted server-side; the encryption key must not be readable by admin queries. -- **Free tier baseline**: 100 MB per user. Quota model should be designed so future subscription tiers can expand it without schema changes. -- **Cloud storage**: Follows same provider/adapter pattern as existing AI providers. Each cloud integration is an adapter implementing a common StorageBackend interface. +- **Current state**: v0.1 complete (2026-06-06). All foundation features shipped and security-hardened. App is functional for local/self-hosted use but not hardened for public internet deployment. +- **Tech stack**: FastAPI 0.136+ (Python 3.12), SQLAlchemy 2.0 async, Alembic, MinIO SDK; Vue 3 (Options API), Pinia, Vue Router 4, Vite, Tailwind CSS. +- **Code quality**: v0.1 was built feature-first under time pressure. Both backend and frontend contain duplication, inconsistent patterns, and components that grew beyond their original scope. v0.2 addresses this systematically. +- **Admin panel**: AdminView.vue currently renders admin functionality as tabs appended to the main user layout. This is architecturally wrong — the admin interface needs its own route subtree, layout component, and nav. +- **Privacy constraint**: Admin role is a platform operator, not a content viewer. Cloud credentials encrypted with per-user HKDF key; API keys encrypted with separate HKDF domain. Neither is ever in an API response. ## Constraints -- **Tech stack**: FastAPI (Python) + Vue 3 — keep existing stack, extend it -- **Database**: PostgreSQL (replaces flat-file JSON) -- **Object storage**: MinIO (S3-compatible, Docker-native) — replaces local filesystem for documents -- **Auth**: bcrypt passwords, JWT sessions, TOTP 2FA (PyOTP / similar) -- **Cloud credentials**: Encrypted at rest (Fernet symmetric encryption or PostgreSQL pgcrypto) — key in env var, never in DB -- **Scalability target**: Horizontal (multiple backend containers) — no file-system-level coordination -- **Deployment**: Docker Compose (must remain the primary deployment target) +- **Tech stack**: FastAPI (Python) + Vue 3 — keep existing stack, no framework switch +- **Database**: PostgreSQL (Alembic-managed schema, 5 migrations completed) +- **Object storage**: MinIO (S3-compatible, Docker-native) +- **Auth**: Argon2id passwords, ES256 JWT, TOTP 2FA (pyotp), HKDF key derivation +- **Deployment**: Docker Compose (primary deployment target, must remain so) +- **No rewrites**: Refactor, don't rewrite. Preserve all existing functionality and test coverage. ## Key Decisions | Decision | Rationale | Outcome | |---|---|---| -| PostgreSQL + MinIO over flat files | Multi-user quotas + horizontal scaling require shared, consistent state | Replacing JSON + filesystem | -| Cloud storage adapter pattern | Mirrors existing AI provider pattern — consistent, extensible | New `storage/` module analogous to `ai/` | -| Privacy-first admin model | SaaS legal/trust requirement — admins must not be able to access user data | Admin queries exclude document content; cloud creds encrypted with user-scoped key | -| Admin controls AI config, not users | Prevents cost overruns and model misuse; future group-admin delegation designed in | AI provider assignment stored per-user in DB, configurable by admin | -| 100 MB free tier | Baseline for subscription model; quota table has a `limit_bytes` column admin can override | Quota enforced at upload time | -| TOTP 2FA before SSO | State-of-the-art security without third-party dependency; SSO added when subscription model lands | TOTP via authenticator app (RFC 6238) | +| PostgreSQL + MinIO over flat files | Multi-user quotas + horizontal scaling require shared, consistent state | Shipped in v0.1 Phase 1 | +| Cloud storage adapter pattern | Mirrors AI provider pattern — consistent, extensible | Shipped in v0.1 Phase 5 | +| Privacy-first admin model | SaaS legal/trust requirement | Admin queries exclude document content; cloud creds encrypted with user-scoped key | +| Admin controls AI config, not users | Prevents cost overruns and model misuse | AI provider assignment stored per-user in DB, configurable by admin only | +| 100 MB free tier | Baseline for future subscription tiers | Quota table has `limit_bytes` column admin can override | +| TOTP 2FA before SSO | State-of-the-art security without third-party dependency | Via pyotp (RFC 6238), shipped v0.1 Phase 2 | +| ES256 over HS256 for JWT | Leaked public key cannot forge tokens; asymmetric signing | Shipped v0.1 Phase 7.3 | +| Token fingerprinting (fgp claim) | Limits stolen access token replay to original device context | HMAC of User-Agent + Accept-Language, shipped v0.1 Phase 7.4 | +| JTI claim + Redis revocation | Closes 15-min window where revoked session's access token stays valid | Shipped v0.1 Phase 7.2 | +| Options API preserved in v0.2 refactor | Composition API migration is scope-creep for a UX milestone; refactor within Options API | Decision logged to prevent scope drift | +| Admin panel as standalone route subtree | AdminView.vue as tabs-on-user-layout is an architectural mistake | v0.2 introduces /admin/* routes with own layout | ## Evolution @@ -124,4 +114,4 @@ This document evolves at phase transitions and milestone boundaries. 4. Update Context with current state --- -*Last updated: 2026-05-21 after initialization* +*Last updated: 2026-06-07 — milestone v0.2 started* diff --git a/.planning/REQUIREMENTS.md b/.planning/REQUIREMENTS.md index 963044c..8149cd2 100644 --- a/.planning/REQUIREMENTS.md +++ b/.planning/REQUIREMENTS.md @@ -1,173 +1,113 @@ -# DocuVault — v1 Requirements +# DocuVault v0.2 — Requirements -_Last updated: 2026-05-21_ - -## v1 Requirements - -### Authentication (AUTH) - -- [x] **AUTH-01**: User can register with email and password (Argon2 hashing; strength enforced: ≥12 chars, uppercase, lowercase, number, special char; HaveIBeenPwned breach check) -- [x] **AUTH-02**: User can log in and maintain a session (JWT access token in Pinia memory only — never localStorage; refresh token in `httpOnly; Secure; SameSite=Strict` cookie; 15-min access / 30-day refresh) -- [ ] **AUTH-03**: User can enroll a TOTP authenticator app (RFC 6238; 8–10 single-use backup codes issued and explicitly acknowledged before TOTP is marked active) -- [x] **AUTH-04**: User can complete login using TOTP code or a one-time backup code (backup code invalidated on use) -- [ ] **AUTH-05**: User can reset password via email (signed token, 1-hour expiry; reset does not auto-login — user must pass TOTP gate on next login) -- [ ] **AUTH-06**: User can sign out all active sessions (revokes all refresh tokens in DB; "sign out all devices" control in account settings) -- [ ] **AUTH-07**: Refresh token rotation with family revocation — reuse of a rotated token revokes the entire family and emits a security alert to the user -- [ ] **AUTH-08**: TOTP codes are single-use (mark used in DB within the validity window; prevent replay attacks) - -### Security (SEC) — Cross-Cutting - -- [x] **SEC-01**: All state-changing endpoints are protected against CSRF (SameSite=Strict cookie + origin validation) -- [x] **SEC-02**: Auth endpoints (login, register, password reset, TOTP verify) are rate-limited (per-IP and per-account) -- [x] **SEC-03**: All DB queries use parameterized statements / ORM (zero raw string interpolation into queries) -- [x] **SEC-04**: All file/document access resolved through DB lookup — object keys are never reconstructed from request parameters (prevents path traversal and cross-user access) -- [x] **SEC-05**: Content-Security-Policy, X-Frame-Options, and X-Content-Type-Options headers set on all responses -- [ ] **SEC-06**: Constant-time comparison used for all token and code verification (prevents timing attacks) -- [ ] **SEC-07**: Admin role verified on every admin endpoint request; admin cannot access document content, extracted text, or cloud credentials in any response -- [ ] **SEC-08**: Cloud credential ciphertext (`credentials_enc`) excluded from all API serializers by default — admin and user responses return only `provider, display_name, connected_at, status` -- [x] **SEC-09**: Account deletion triggers `delete_user_files()` on every active cloud connection before removing DB records (prevents orphaned cloud data and satisfies GDPR Article 17) - -### Users & Admin (ADMIN) - -- [x] **ADMIN-01**: Admin can create user accounts (email, temporary password that must be changed on first login) -- [x] **ADMIN-02**: Admin can deactivate a user account (blocks all logins and API access; data preserved) -- [x] **ADMIN-03**: Admin can initiate password reset for a user (sends reset email; does not grant admin access to the account) -- [x] **ADMIN-04**: Admin can view and adjust individual user storage quotas (warns if new limit is below current usage) -- [x] **ADMIN-05**: Admin can assign AI provider and model per user (users cannot modify their own AI configuration) -- [ ] **ADMIN-06**: Admin can view audit log filtered by date range, user, and action type (metadata only — no document content, filenames, or extracted text) -- [x] **ADMIN-07**: Admin impersonation ("log in as user") is explicitly excluded by architecture — no endpoint or UI pathway exists - -### Storage & Infrastructure (STORE) - -- [ ] **STORE-01**: Platform storage layer migrated from flat-file JSON + local filesystem to PostgreSQL (metadata) + MinIO (objects); existing documents preserved via dual-write migration script -- [ ] **STORE-02**: Each user's MinIO objects use `{user_id}/{document_id}/{uuid4()}{ext}` keys — human-readable filenames stored in DB only -- [ ] **STORE-03**: Each user has a 100 MB storage quota enforced atomically at upload using `UPDATE quotas SET used_bytes = used_bytes + $delta WHERE (used_bytes + $delta) <= limit_bytes RETURNING used_bytes` -- [ ] **STORE-04**: User sees quota usage bar in sidebar (X MB of Y MB) with amber warning at 80% and red warning at 95% -- [ ] **STORE-05**: Upload rejected at quota limit with a specific error showing current usage, rejected file size, and a link to storage settings -- [ ] **STORE-06**: Document delete atomically decrements quota usage -- [ ] **STORE-07**: Backend is stateless — no per-instance file locks; multiple instances can run behind a load balancer -- [ ] **STORE-08**: FastAPI `BackgroundTasks` replaced with Celery + Redis or pgqueuer before horizontal scaling is enabled - -### Folders & Organization (FOLD) - -- [x] **FOLD-01**: User can create, rename, and delete folders (delete confirms content count before proceeding) -- [x] **FOLD-02**: User can move documents between folders -- [x] **FOLD-03**: Breadcrumb navigation renders current folder path; each segment is clickable to navigate up -- [x] **FOLD-04**: Document list supports sort by name, date uploaded, and file size -- [x] **FOLD-05**: Full-text search across user's documents (PostgreSQL `tsvector` index on extracted text) - -### Document Sharing (SHARE) - -- [ ] **SHARE-01**: User can share a document with another user by their unique handle (at-handle or user ID) -- [ ] **SHARE-02**: Shared documents appear in a "Shared with me" virtual folder for the recipient (no storage quota counted against recipient) -- [ ] **SHARE-03**: Shared access is view-only by default; owner controls permission level -- [ ] **SHARE-04**: Owner can revoke share access; revocation is immediate -- [ ] **SHARE-05**: Documents shared with others display a "shared" indicator in the owner's list view - -### Cloud Storage (CLOUD) - -- [x] **CLOUD-01**: User can connect OneDrive (Microsoft Graph), Google Drive (v3 API), Nextcloud, or generic WebDAV as a personal storage backend -- [x] **CLOUD-02**: Cloud OAuth credentials encrypted using HKDF per-user key derivation (`HKDF(master_key, salt=user_id_bytes, info=b"cloud-credentials")`); master key in `CLOUD_CREDS_KEY` env var; never stored in DB -- [x] **CLOUD-03**: Local MinIO storage and connected cloud backends coexist; user can select their default storage destination -- [x] **CLOUD-04**: Each cloud connection displays status: `ACTIVE | REQUIRES_REAUTH | ERROR` -- [x] **CLOUD-05**: On OAuth revocation (`invalid_grant`), connection status transitions to `REQUIRES_REAUTH` — the error is surfaced to the user, not retried silently -- [x] **CLOUD-06**: User can disconnect a cloud backend; credentials are permanently deleted from the DB -- [x] **CLOUD-07**: Storage backend abstracted via `StorageBackend` ABC + factory in `storage/` module (mirrors existing `ai/` provider pattern) - -### Documents & AI (DOC) - -- [ ] **DOC-01**: User can view document metadata and extracted text for any document in their library -- [ ] **DOC-02**: In-browser PDF preview (PDF.js); document bytes proxied through the app — no presigned URLs exposed to the browser (privacy model) -- [x] **DOC-03**: AI provider and model assigned by admin per user; user cannot change AI configuration -- [x] **DOC-04**: System default topics + per-user topic overrides preserved from existing implementation -- [x] **DOC-05**: AI classification uses the user's assigned provider and model (from DB, not from user-supplied settings) +**Milestone:** v0.2 — UI Overhaul and Optimization +**Created:** 2026-06-07 +**Total requirements:** 36 --- -## v2 Requirements (Deferred) +## Quality Mandate (applies to all phases) -- Subscription billing and payment processing (quota model designed to plug in) -- SSO: Microsoft, Google, Apple (auth layer designed for extension) -- Keycloak / SAML / OAuth2 enterprise federation -- Group admin roles (groups table seeded in schema, unpopulated) -- Share permission levels beyond view-only (edit, comment) -- Document version history -- Share expiry dates -- Real-time collaboration or comments -- Mobile app -- GDPR data export (Article 20) — async background job, deferred to v2 -- Email notifications for sharing events -- Public link sharing (unauthenticated) +Every line of code written or modified in v0.2 must be: + +- **Minimal:** smallest amount of code that correctly solves the problem. No speculative abstractions. +- **DRY:** if the same logic exists twice, extract it. Duplication is a defect. +- **Readable:** names things accurately. A reader should understand intent without comments. +- **No redundant comments:** comments exist only where the *why* is non-obvious. Never explain what the code does. +- **No dead code:** unreferenced files, components, or functions are deleted — not commented out. --- +## CODE — Codebase Quality + +- [ ] **CODE-01**: Backend `api/admin.py` (934L) decomposed into `api/admin/` package with `users.py`, `quotas.py`, `ai.py`; `__init__.py` aggregates sub-routers under the existing `/api/admin` prefix. No URL changes, no behavior changes, all tests stay green. +- [ ] **CODE-02**: `api/documents.py` (852L) decomposed into focused sub-modules (upload flow, content proxy, document CRUD, search/listing) within `api/documents/` package. Prefix and behavior unchanged. +- [ ] **CODE-03**: `api/auth.py` (825L) decomposed into focused sub-modules (login/tokens, TOTP, password management, session management) within `api/auth/` package. Prefix and behavior unchanged. +- [ ] **CODE-04**: Frontend `api/client.js` (635L) decomposed into domain modules (`documents.js`, `auth.js`, `admin.js`, `folders.js`, `shares.js`, `cloud.js`, `topics.js`); `client.js` becomes the HTTP transport layer and re-export barrel. Zero changes to any of the 35+ consumer files. +- [ ] **CODE-05**: All inline SVG blocks (~66 instances) replaced with ``; all icon path data centralized in `components/ui/AppIcon.vue`. No duplicated path strings. +- [ ] **CODE-06**: Tailwind `safelist` configured for all dynamic class name patterns in `formatters.js` (provider colors, backgrounds, badge text). Production builds render topic and provider colors correctly. +- [ ] **CODE-07**: All unreferenced files, components, stores, and unused imports deleted. No dead code retained. +- [ ] **CODE-08**: No duplicated Pydantic model definitions or shared validators across router files. Shared schemas extracted to dedicated modules. +- [ ] **CODE-09**: No comment in any file describes what the code does. Comments exist only where intent or constraint would not be obvious to a competent reader. + +--- + +## ADMIN — Admin Panel + +- [ ] **ADMIN-08**: Admin panel moved to `/admin/*` route subtree with `AdminLayout.vue` as the route component. `AdminLayout` has its own sidebar with admin-specific nav only — no user quota bar, no folder tree, no topic list. `AdminView.vue` is deleted. +- [ ] **ADMIN-09**: Admin sidebar nav links (in order): Overview, Users, Quotas, AI Config, Audit Log. "Back to app" link at the bottom returns to `/`. +- [ ] **ADMIN-10**: Each admin section is its own deep-linkable URL (`/admin/users`, `/admin/quotas`, `/admin/ai`, `/admin/audit`). Browser back button works within the admin section. +- [ ] **ADMIN-11**: Admin overview page (`/admin`) shows: total registered user count, total platform storage in use, document status breakdown (processing/ready/failed), last 10 audit log entries. Requires new backend aggregate query endpoints. +- [ ] **ADMIN-12**: `requiresAdmin` guard enforced for all `/admin/*` child routes via `to.matched.some(r => r.meta.requiresAdmin)`. No admin child route is accessible to non-admin users. + +--- + +## UX — UX and Interaction + +- [ ] **UX-01**: Every zero-content context shows a contextually distinct empty state using a shared `EmptyState.vue` component (headline, subtext, optional CTA). Applied to: root file list, individual folder, search with no results, shared-with-me, topics list, audit log, cloud connections. No plain "No items" text remains. +- [ ] **UX-02**: Document list (`StorageBrowser`) displays 5-col skeleton grid rows during loading (matching actual grid layout, `animate-pulse`). "Loading..." text removed. +- [ ] **UX-03**: Sidebar folder tree and topics list display skeleton placeholder items during loading. "Loading..." text removed. +- [ ] **UX-04**: Admin user table and audit log table display skeleton table rows during loading. +- [ ] **UX-05**: Pressing `/` when no input is focused moves keyboard focus to the search bar. +- [ ] **UX-06**: Pressing `Escape` closes any open modal (`ShareModal`, `FolderDeleteModal`, `DocumentPreviewModal`) and clears active search. Handled at the overlay level, not only inside inputs. +- [ ] **UX-07**: Pressing `U` when no input is focused triggers the file upload picker. +- [ ] **UX-08**: Pressing `N` when no input is focused starts the new folder inline input in the file manager. +- [ ] **UX-09**: Dragging files from the OS anywhere onto the browser window shows a full-screen drop overlay. Releasing uploads the files via the existing upload flow. Only activates when `dataTransfer.types` includes `'Files'`. +- [ ] **UX-10**: Toast notification system shows transient messages (auto-dismiss 4 seconds, manual dismiss on click) for: upload complete, upload error, document deleted, share revoked, rename successful. Toasts stack vertically and do not block interaction. +- [ ] **UX-11**: Drag-to-move a document onto a folder row in `StorageBrowser` works correctly end-to-end. Drop highlight (`ring-2 ring-inset ring-amber-300`) renders on valid folder targets. "Move to folder" button remains as non-drag alternative. +- [ ] **UX-12**: All views display a breadcrumb reflecting the full current navigation path. Examples: `Folders › Reports › Q1`, `Cloud › Google Drive › Projects`, `Admin › Users`, `Settings › Account`, `Folders › Reports › document.pdf`. Breadcrumb updates on navigation. A single shared breadcrumb component is used across all view types — no per-view implementations. +- [ ] **UX-13**: All dropdown menus and modal dialogs render their full content within the viewport. No text or interactive element is clipped by a viewport edge or sibling element. Dropdowns that risk edge-clipping use fixed positioning computed from `getBoundingClientRect()` or ``. +- [ ] **UX-14**: The inline "New" button next to the Folders section header in `AppSidebar.vue` is removed. Folder creation accessible only from within the file manager view. + +--- + +## VISUAL — Visual Design + +- [ ] **VISUAL-01**: Consistent spacing scale across all components using Tailwind utilities only. No arbitrary `px-[13px]` values or inline `style` margins. +- [ ] **VISUAL-02**: All form elements (inputs, selects, textareas, checkboxes, radio buttons) have consistent cross-browser baseline styling via `@tailwindcss/forms` plugin. +- [ ] **VISUAL-03**: All interactive elements (buttons, links, card rows) have consistent hover states, `focus-visible:` rings, and active states. +- [ ] **VISUAL-04**: Typography scale is consistent: one heading size per level, one body size, one label/caption size. No per-component font-size overrides. + +--- + +## RESP — Responsive Layout + +- [ ] **RESP-01**: Sidebar hidden below `lg` (1024px). Hamburger button in mobile-only header toggles a slide-in overlay drawer. Overlay closes on nav tap or backdrop tap. +- [ ] **RESP-02**: Document list hides Size column below `md` (768px) and Modified column below `sm` (640px). Icon, name, and actions always visible. +- [ ] **RESP-03**: Inline icon action buttons have minimum 36×36px touch target on viewports below `md`. +- [ ] **RESP-04**: All modal dialogs are scrollable on viewports below 640px. No modal content overflows the screen. +- [ ] **RESP-05**: Admin layout has the same responsive behavior as the user layout: admin sidebar hidden on mobile, hamburger shows admin nav drawer. + +--- + +## PERF — Performance and Stack + +- [ ] **PERF-01**: Frontend dependencies bumped/added: `vue@^3.5.0`, `vite@^6.4.3`, `@vueuse/core@^14.3.0`, `@vueuse/integrations@^14.3.0`, `sortablejs@^1.15.7`, `@tailwindcss/forms@^0.5.11`. Dev: `rollup-plugin-visualizer@^7.0.1`, `@types/sortablejs`. +- [ ] **PERF-02**: Bundle baseline measured with `rollup-plugin-visualizer` before optimizations begin. Second measurement taken after all refactoring is complete. Both reports committed to `.planning/`. +- [ ] **PERF-03**: All non-initial-render route components are lazy-loaded. Admin views explicitly lazy-loaded. No synchronous import for routes not on the critical render path. + +--- + +## Future (deferred) + +- Virtual scrolling — quota cap (100 MB/user) limits lists to hundreds of items; `v-for` is sufficient. Revisit with server-side pagination. +- Dark mode — coherent color token system must exist first. +- Folder reordering by drag — requires persistent `position` column in DB. +- Multi-select with batch operations — low frequency in a personal vault. + ## Out of Scope -- Admin impersonation / "log in as user" — violates privacy-first core value; explicit architectural exclusion -- Document editing or annotation — not planned -- Document viewer for non-PDF types beyond metadata (DOCX, image renders) — v2 -- AI-generated document summaries beyond topic classification — v2 -- Webhooks or API access for third parties — not planned for v1 +- Composition API migration — refactor within Options API; Composition API is a separate milestone. +- Component library switch (shadcn-vue, PrimeVue) — stay with Tailwind. +- SSO, billing, group roles — v0.x exclusions unchanged from v0.1. +- Document annotation or in-app editing — not planned. --- ## Traceability -_Filled by roadmapper — 2026-05-21._ +*Filled by roadmapper after phases are defined.* -| REQ-ID | Phase | Notes | -|---|---|---| -| STORE-01 | 1 | Dual-write migration script; schema and Alembic wiring | -| STORE-02 | 1 | Object key schema enforced in model layer | -| STORE-07 | 1 | Stateless backend; no per-instance file locks | -| AUTH-01 | 2 | Registration with Argon2 + HaveIBeenPwned check | -| AUTH-02 | 2 | JWT session; httpOnly refresh cookie; Pinia memory access token | -| AUTH-03 | 2 | TOTP enrollment with backup code acknowledgement flow | -| AUTH-04 | 2 | Login via TOTP code or single-use backup code | -| AUTH-05 | 2 | Password reset email; routes back to TOTP gate | -| AUTH-06 | 2 | Sign out all devices; revokes all refresh tokens | -| AUTH-07 | 2 | Refresh token family revocation on reuse; security alert | -| AUTH-08 | 2 | TOTP single-use enforcement within validity window | -| SEC-01 | 2 | CSRF protection on all state-changing endpoints | -| SEC-02 | 2 | Rate limiting on auth endpoints (per-IP and per-account) | -| SEC-03 | 2 | Parameterized queries / ORM enforced from first migration | -| SEC-05 | 2 | Security response headers on all responses | -| SEC-06 | 2 | Constant-time comparison for token/code verification | -| SEC-07 | 2 | Admin role dependency; admin blocked from document content | -| ADMIN-01 | 2 | Admin creates user with temporary password | -| ADMIN-02 | 2 | Admin deactivates user account | -| ADMIN-03 | 2 | Admin initiates password reset for user | -| ADMIN-04 | 2 | Admin views and adjusts user storage quotas | -| ADMIN-05 | 2 | Admin assigns AI provider and model per user | -| ADMIN-07 | 2 | Explicit architectural exclusion of admin impersonation | -| STORE-03 | 3 | Atomic quota enforcement at upload | -| STORE-04 | 3 | Quota usage bar with 80%/95% warnings | -| STORE-05 | 3 | Upload rejection at quota limit with detailed error | -| STORE-06 | 3 | Atomic quota decrement on document delete | -| STORE-08 | 3 | BackgroundTasks replaced with Celery+Redis or pgqueuer | -| SEC-04 | 3 | DB-lookup-only file access; no key reconstruction from params | -| DOC-03 | 3 | AI provider/model from DB per user; not user-supplied | -| DOC-04 | 3 | System default topics + per-user topic overrides preserved | -| DOC-05 | 3 | Classification uses user's assigned provider and model | -| FOLD-01 | 4 | Folder CRUD with content-count confirmation on delete | -| FOLD-02 | 4 | Document move between folders | -| FOLD-03 | 4 | Breadcrumb navigation with clickable path segments | -| FOLD-04 | 4 | Document list sort by name, date, and file size | -| FOLD-05 | 4 | Full-text search via PostgreSQL tsvector index | -| SHARE-01 | 4 | Share document by user handle | -| SHARE-02 | 4 | "Shared with me" virtual folder; no quota charged to recipient | -| SHARE-03 | 4 | View-only default sharing; owner controls permission level | -| SHARE-04 | 4 | Immediate share revocation | -| SHARE-05 | 4 | Shared indicator on documents in owner's list view | -| SEC-08 | 4 | credentials_enc excluded from all serializers | -| SEC-09 | 4 | Account deletion triggers delete_user_files() per cloud connection | -| ADMIN-06 | 4 | Admin audit log viewer filtered by date, user, action | -| DOC-01 | 4 | View document metadata and extracted text | -| DOC-02 | 4 | In-browser PDF preview via PDF.js; bytes proxied through app | -| CLOUD-01 | 5 | Connect OneDrive, Google Drive, Nextcloud, WebDAV | -| CLOUD-02 | 5 | HKDF per-user key derivation for credential encryption | -| CLOUD-03 | 5 | Local and cloud storage coexist; user selects default | -| CLOUD-04 | 5 | Connection status display: ACTIVE / REQUIRES_REAUTH / ERROR | -| CLOUD-05 | 5 | invalid_grant transitions to REQUIRES_REAUTH; surfaced to user | -| CLOUD-06 | 5 | Disconnect cloud backend; credentials permanently deleted | -| CLOUD-07 | 5 | StorageBackend ABC + factory in storage/ module | +| REQ-ID | Phase | Plan | +|--------|-------|------| +| — | — | — | diff --git a/.planning/STATE.md b/.planning/STATE.md index 02be971..73384a6 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -1,176 +1,65 @@ --- gsd_state_version: 1.0 -milestone: v1.0 -milestone_name: "audit gaps: SHARE-02/STORE-06/ADMIN-06" -current_phase: 07.4 -status: completed -last_updated: "2026-06-06T17:42:13.466Z" +milestone: v0.2 +milestone_name: "UI Overhaul and Optimization" +current_phase: none +status: planning +last_updated: "2026-06-07T00:00:00.000Z" progress: - total_phases: 7 - completed_phases: 6 - total_plans: 20 - completed_plans: 20 - percent: 86 + total_phases: 0 + completed_phases: 0 + total_plans: 0 + completed_plans: 0 + percent: 0 --- # Project State **Project:** DocuVault -**Status:** Phase 07.3 Complete — Ready for Phase 07.4 -**Current Phase:** 07.4 -**Last Updated:** 2026-06-06 - -## Phase Status - -| Phase | Name | Status | -|---|---|---| -| 1 | Infrastructure Foundation | ✓ Complete | -| 2 | Users & Authentication | ✓ Complete (5/5 plans) | -| 3 | Document Migration & Multi-User Isolation | ✓ Complete (5/5 plans, UAT passed, security gate passed) | -| 4 | Folders, Sharing, Quotas & Document UX | ✓ Complete (9/9 plans, UAT 14/15 passed, 1 bug fixed) | -| 5 | Cloud Storage Backends | ✓ Complete (12/12 plans, UAT 5/6 passed, 3 gaps closed by 05-12) | -| 6 | Performance & Production Hardening | ✓ Complete (6/6 plans, UAT passed, CVE gate passed) | -| 6.1 | Close v1.0 audit gaps: SHARE-02/STORE-06/ADMIN-06 | ✓ Complete (2/2 plans) | -| 6.2 | Close v1 sharing + cloud-delete + CSV export gaps | ✓ Complete (5/5 plans, UAT passed, security gate passed) | -| 7 | Redo and Optimize LLM Integration | ✓ Complete (5/5 plans, UAT 11/11 passed, security gate passed) | -| 7.1 | Security: session revocation on privilege change (CR-01..03) | ✓ Complete (2/2 plans, 3 new tests, frontend toasts) | -| 7.2 | Security: JTI claim + Redis access-token revocation | ✓ Complete (3/3 plans, 9/9 verified, 84/84 tests) | -| 7.3 | Security: ES256 algorithm upgrade | ✓ Complete (3/3 plans, all 9 tests passing, remember_me shipped) | -| 7.4 | Security: Token fingerprinting / token binding | ✓ Complete (2/2 plans, 4 FGP tests passing, v0.1.3) | +**Status:** Milestone v0.2 started — defining requirements +**Current Phase:** None (requirements phase) +**Last Updated:** 2026-06-07 ## Current Position -Phase: 07.3 (security-es256-algorithm-upgrade-inserted) — COMPLETE (3/3 plans, 9/9 tests, 2026-06-06) -Phase: 07.2 (security-jti-claim-redis-access-token-revocation-inserted) — COMPLETE (3/3 plans, 9/9 verified) -**Progress:** [██████████] 100% (v1.0 base complete; Phase 7.1 inserted as urgent follow-up) +Phase: Not started (defining requirements) +Plan: — +Status: Defining requirements +Last activity: 2026-06-07 — Milestone v0.2 started + +## Phase Status + +*Phases will be added after roadmap is approved.* ## Performance Metrics | Metric | Value | |---|---| -| Phases complete | 7 / 7 | -| Requirements mapped | 54 / 54 | -| Plans written | 5 (Phase 7) | -| Plans complete | 5 (Phase 7, all phases done) | +| Phases complete | — | +| Requirements mapped | — | +| Plans written | — | +| Plans complete | — | ## Accumulated Context ### Key Decisions +*(Carries forward from v0.1 — see PROJECT.md for full decision log.)* + | Decision | Rationale | |---|---| -| PostgreSQL + MinIO | Multi-user quotas and horizontal scaling require shared, consistent state | -| HKDF per-user key derivation | Single Fernet key would be catastrophic on leak — must be derived before first credential is stored | -| Presigned MinIO URL flow | FastAPI handles metadata only; bytes never pass through the API layer | -| Atomic PostgreSQL quota UPDATE | Never perform quota arithmetic in Python between two DB statements | -| JWT in httpOnly cookie | Refresh token in httpOnly cookie; access token in Pinia memory only — never localStorage | -| Refresh token family revocation | RFC 9700 — reuse of a rotated token revokes entire family and alerts user | -| BackgroundTasks replacement | FastAPI BackgroundTasks is per-instance; replace with Celery+Redis or pgqueuer before horizontal scale | -| AuditLog metadata_ ORM attribute | `metadata` is reserved on DeclarativeBase; ORM attribute is `metadata_` with `name="metadata"` kwarg to avoid silent collision | -| documents.user_id nullable Phase 1 | D-03 — no auth in Phase 1; Phase 2 migration adds NOT NULL after auth lands | -| groups stub table Phase 1 | D-02 — groups is a v2 feature; table created now for schema completeness, no rows until Phase 2+ | -| SEQUENCES grants in migration | GRANT USAGE/SELECT on sequences required for audit_log.id autoincrement nextval() by docuvault_app | -| Admin impersonation excluded | Explicit architectural exclusion — no endpoint or UI pathway; violates privacy-first core value | -| user_id as refresh token family proxy | No separate family_id column; user_id serves as family per RFC 9700 — simpler schema | -| pwdlib over passlib | pwdlib actively maintained with clean Argon2Hasher API; passlib unmaintained | -| TOTP replay TTL=90s | valid_window=1 covers ±30s (90s total) — TTL matches window | -| HIBP fail-open | Network errors return False + log warning; auth never blocked by external service | -| Two-DSN PostgreSQL strategy | DATABASE_URL (docuvault_app, DML only) + DATABASE_MIGRATE_URL (docuvault_migrate, DDL only); celery-worker gets only DATABASE_URL | -| MinIO healthcheck via mc ready local | curl removed from MinIO Docker image since Oct 2023; mc is the correct in-container healthcheck tool | -| pydantic-settings v2 SettingsConfigDict | SettingsConfigDict API used (not deprecated class Config form) for env var config | -| async_client fixture name | Distinct from legacy sync `client` fixture to avoid collision; both coexist until Plan 05 | -| xfail(strict=False) for Wave 0 | All pre-implementation scaffolds use strict=False so unexpected passes don't break CI | -| StorageBackend ABC + factory mirrors ai/ pattern | 5 abstract methods; get_storage_backend() factory; MinIOBackend wraps all sync Minio SDK calls in asyncio.to_thread() | -| Explicit localhost string block in validate_cloud_url | hostname == "localhost" blocked before DNS resolution — OS-agnostic (getaddrinfo("localhost") behaviour varies by OS) | -| Fresh HKDF instance per _derive_fernet_key call | cryptography library raises AlreadyFinalized on 2nd .derive() call; always create new HKDF(...) instance — never cache | -| Lazy import of cloud backends in get_storage_backend_for_document | Avoids circular imports at module load time; backends imported inside function body with type: ignore[import] until Plans 05-03..05-05 create them | -| Fetch-outside-lock async cache pattern | get_cloud_folders_cached acquires lock to check cache, releases lock, awaits fetch_fn, re-acquires lock to write — prevents event loop blocking on cache miss | -| STORE-02 key enforced in code | MinIOBackend.put_object constructs {user_id}/{document_id}/{uuid4()}{ext}; no filename parameter — only extension passes through | -| null-user D-03 sentinel | services/storage.save_upload uses user_id="null-user" in Phase 1 (no auth); Phase 2 replaces with str(current_user.id) | -| load_settings flat-file Phase 1 | users.ai_provider/ai_model columns cannot be populated until Phase 2; settings remain flat-file JSON for Phase 1 | -| Deferred Celery import in /password-reset | send_reset_email.delay called via from tasks.email_tasks import send_reset_email inside handler body — same circular-import fix as document_tasks | -| TOTP QR code as otpauth:// link | No QR library installed; plan permits manual secret display for MVP; functional flow complete without rendered QR image | -| ConfirmBlock no acknowledgment checkbox | ConfirmBlock handles message + button pair; BackupCodesDisplay owns its separate acknowledgment checkbox — no overlap | -| ADMIN-07 enforced by omission | No impersonation endpoint exists; AST check + test_admin_impersonation_not_found verify absence; violates privacy-first core value | -| _user_to_dict() whitelist for admin responses | Explicit field whitelist prevents accidental password_hash/credentials_enc leakage from admin endpoints | -| Quota warning is 200 not 4xx | Below-usage limit change is applied; warning=True advisory field returned — not a rejection | -| AdminQuotasTab fetches quotas per-user via Promise.allSettled | adminListUsers() does not include quota fields; per-user endpoint parallelized; failed quotas filtered silently | -| Temp password via crypto.getRandomValues | Browser-native CSPRNG; no external library; always satisfies AUTH-01 strength rules | -| batch_alter_table for NOT NULL in migration 0003 | SQLite requires batch_alter_table for ALTER COLUMN; transparent passthrough on PostgreSQL — enables SQLite CI test runs | -| MinIO step in migration 0003 gated on MINIO_ENDPOINT | Migration skips MinIO deletions when env var absent; enables safe SQLite test runs per T-03-02 | -| raising=False for Phase 3 MinIO mock fixtures | mock_minio_presigned + mock_minio_stat patch methods that don't exist until Plan 03-02; raising=False pre-installs them | -| Dual MinIO client (internal + public) | Presigned URL HMAC signature must be computed with browser-visible hostname (localhost:9000); using internal Docker client (minio:9000) causes browser signature mismatch | -| Wave 2 user_id=None guard | upload-url sets user_id=None + object_key "null-user/" prefix; confirm skips quota when user_id is None; Plan 03-03 removes both guards | -| SQLite quota xfail(strict=False) | SQLite stores UUID as CHAR(32) without dashes; raw SQL WHERE user_id = :uid never matches str(uuid) dashed format — test-env limitation, not code defect | -| Celery mock required in /confirm tests | extract_and_classify.delay() connects to Redis; monkeypatch blocks it in unit tests; MagicMock pattern established for all confirm endpoint tests | -| get_regular_user raises 403 for admin | Admin is authenticated but must not access document content; 401 would falsely imply unauthenticated — 403 is correct for role rejection | -| Cross-user doc access returns 404 not 403 | Combining "not found" and "wrong owner" into 404 prevents attacker from learning which doc IDs exist for other users (D-16, T-03-11) | -| CASE WHEN replaces GREATEST in quota decrement | SQLite lacks GREATEST scalar function; CASE WHEN used_bytes > :delta THEN used_bytes - :delta ELSE 0 END is semantically equivalent and SQLite-compatible | -| load_topics_for_user uses or_(user_id == x, user_id.is_(None)) | SQLAlchemy is_(None) not == None; or_() combines system topics and user's own topics for namespace-scoped query (D-17, DOC-04) | -| AI-suggested topics go in user namespace | classifier passes user_id=doc.user_id to create_topic; AI-suggested topics are per-user not system-wide (D-11) | -| Celery task signature unchanged for ai_provider | Task receives only document_id; ai_provider/ai_model resolved inside _run via session.get(User, doc.user_id) — prevents broker injection (T-03-19) | -| _DEFAULT_SYSTEM_PROMPT in classifier.py | System prompt env var is optional; hardcoded fallback kept in classifier module not config.py (D-13) | -| Default AI provider is ollama/llama3.2 | Code defaults; overridable via DEFAULT_AI_PROVIDER / DEFAULT_AI_MODEL env vars (D-15) | -| /settings route kept as static placeholder | SettingsView shows admin-managed card; route not removed to avoid UX regression (Risk 6) | -| Plain anchor in quota rejection block | used instead of to avoid import dependency in upload component | -| uploadProgress entries owned by parent | Store does not clear uploadProgress map entries after upload; DropZone/parent clears on row dismiss | -| fetchQuota silent catch in auth store | Silent catch keeps last-known values; QuotaBar owns loadFailed state and hides on error (UI-SPEC) | -| XHR PUT progress range 5–90 | 5 + Math.round(pct * 0.85) maps XHR 0-100 → visual 5-90; remaining 10% covers confirm + enqueue | -| FTS stubs carry both xfail and skipif(INTEGRATION) | skipif fires first in non-INTEGRATION runs (tests appear SKIPPED); xfail catches failures when INTEGRATION=1 — both decorators required | -| Wave 0 stubs: single-line body only | All Phase 4 stubs: body is only pytest.xfail("not implemented yet") — no assertion code; strict=False so xpass never breaks CI | -| GIN index via op.execute() raw SQL | Alembic autogenerate cannot round-trip expression indexes; raw SQL with comment prevents re-creation on every --autogenerate run (issue #1390) | -| put_object_raw not in StorageBackend ABC | audit-logs bucket is MinIO-only; local/WebDAV backends have no audit concept; MinIOBackend-only method | -| write_audit_log uses session.flush() | D-14: caller owns the transaction; flush queues the audit entry without committing — commit remains caller's responsibility | -| Breadcrumb uses iterative Python parent-walk | Not WITH RECURSIVE — ensures SQLite unit tests pass; cycle guard (visited set) prevents infinite loop on malformed data | -| document_move_router is a separate APIRouter | PATCH /api/documents/{id}/folder placed in folders.py not documents.py; separate router with /api/documents prefix avoids circular import | -| FTS plainto_tsquery wrapped in try/except | SQLite silently degrades to unfiltered results when plainto_tsquery unavailable; PostgreSQL works fully — no unit test breakage | -| Share IDOR: DELETE returns 404 not 403 | Prevents share ID enumeration; attacker cannot learn which share IDs exist for other users (T-04-04-02) | -| /received before /{share_id} in router | Path parameter conflict: FastAPI routes /received as /{share_id}="received" if DELETE is defined first — ordering enforced by comment | -| No quota touch in shares.py | Recipient's quota is never modified by share operations (T-04-04-04); sharing is metadata-only from quota's perspective | -| login_failed audit metadata_=None | No email, no hash, no PII in login failure audit events — T-04-07-01 threat mitigation | -| document audit metadata whitelist | document.uploaded contains only size_bytes and storage_backend; document.deleted contains only size_bytes — no filename, no extracted_text | -| CloudConnectionOut whitelist pattern | Pydantic model with exactly the safe fields; credentials_enc absent by omission — SEC-08 safe-by-default | -| admin.user_deleted flush before delete | audit write flushed (session.flush()) while user FK still valid; session.delete(user) follows — preserves audit FK integrity | -| test_admin_impersonation 405 acceptable | DELETE /users/{id} causes GET to return 405 not 422; both mean no GET impersonation endpoint; test updated to accept {404, 405, 422} | -| CloudConnectionError shared exception type | Defined once in google_drive_backend.py; imported by onedrive_backend.py — single exception type across all cloud backends | -| cache_discovery=False on Drive build() | Prevents /tmp discovery cache writes — directory traversal vector (T-05-03-05) | -| createUploadSession for all OneDrive uploads | No 4 MB size gate; resumable sessions handle small and large files through same code path (Pitfall 6) | -| MSAL invalid_grant via result.get('error') | MSAL returns dict (never raises); field-level check is correct — Assumption A3 confirmed | -| WebDAVBackend SSRF double guard pattern | validate_cloud_url in __init__ (construct-time) AND before every asyncio.to_thread() call — mirrors D-17 requirement for DNS-rebinding mitigation | -| nextcloud/webdav dispatch to distinct classes | NextcloudBackend for 'nextcloud' provider (has list_folder); WebDAVBackend for 'webdav' — identical constructor signatures | -| webdavclient3 upload_to/download_from confirmed | A1 assumption in RESEARCH.md was correct; verified via runtime dir(Client) inspection before use | -| OAuth callback not authenticated via JWT | OAuth redirect flow cannot carry Bearer header; state token (256 bits, TTL 1800s, single-use) provides equivalent security | -| Cloud cleanup added to admin delete_user only | auth.py has no DELETE /api/users/me; admin-initiated deletion is the only account deletion code path | -| Cloud cleanup runs before MinIO cleanup | credentials still in DB when get_storage_backend_for_document is called; sessions.flush() after conn deletes | +| Options API preserved in v0.2 refactor | Composition API migration is scope-creep; refactor within Options API | +| Admin panel as standalone route subtree | AdminView as tabs-on-user-layout is architecturally wrong; v0.2 fixes this | +| No rewrites — refactor only | Preserve existing tests and behavior; change internal structure, not contracts | ### Roadmap Evolution -- Phase 6 added: Performance & Production Hardening (2026-05-30) -- Phase 6.1 inserted: Close v1.0 audit gaps — SHARE-02/STORE-06/ADMIN-06 (2026-05-30) -- Phase 6.2 inserted: Close v1 sharing + cloud-delete + CSV export gaps (2026-05-31) -- Phase 7 added: Redo and optimize LLM integration -- Phase 7.1 inserted (URGENT): Security: session revocation on privilege change (CR-01..03) — inserted after Phase 7 (2026-06-05) +- v0.1 completed: all 7 foundation phases + security hardening (2026-06-06) +- v0.2 started: UI overhaul, code quality, admin panel rearchitecture (2026-06-07) ### Open Questions -- Verify cloud SDK minor versions on PyPI before Phase 5 pinning - -### Workflow Changes (2026-05-25) - -Two mandatory cross-cutting gates added to all phases going forward: - -**1. Test gate** — every plan must leave `pytest -v` passing with zero failures. Every new function/endpoint/component requires at least one test. All security-invariant negative tests (wrong owner, admin block, token replay) must exist and pass. - -**2. Security gate** — a security agent runs after every plan execution and is a blocking requirement before phase advancement. It: - -- Runs `bandit -r backend/`, `pip audit`, `npm audit --audit-level=high` -- Checks for path traversal, IDOR, SSRF, timing attacks, mass assignment, token replay -- Verifies admin endpoints never return `password_hash`, `credentials_enc`, or document content -- Fixes issues directly (full edit access) rather than deferring - -**3. Bug fix rule** — all fixes: root cause only, ≤50 lines, regression test required, no workarounds. - -See CLAUDE.md "Testing Protocol" and "Security Protocol" sections for full detail. +None. ### Blockers @@ -182,34 +71,7 @@ _Updated at each phase transition._ | Field | Value | |---|---| -| Last session | 2026-05-25 — Phase 3 UAT complete (10/10); security gate passed (3 fixes: bandit B324, Referrer-Policy, IDOR on /topics/suggest); test fix for test_lmstudio.py import | -| Last session | 2026-05-25 — Phase 4 context gathered (4 areas: folder nav, sharing, PDF proxy, audit log) | -| Last session | 2026-05-25 — Phase 4 UI-SPEC approved (6 dimensions: 2 PASS clean, 3 FLAG non-blocking, 0 BLOCK) | -| Last session | 2026-05-25 — Phase 4 plans created (9 plans, 7 waves) + verification passed (0 blockers, 2 warnings) | -| Last session | 2026-05-25 — Plan 04-01 executed: 30 Wave 0 xfail stubs across 5 test files; 39 xfailed total, zero new failures | -| Last session | 2026-05-25 — Plan 04-02 executed: migration 0004 (pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw(); 122 tests pass | -| Last session | 2026-05-25 — Plan 04-03 executed: write_audit_log() helper (flush-not-commit, never-raises) + FOLD-01..05 folder API + document sort/FTS/move; 122 pass, 0 new failures | -| Last session | 2026-05-25 — Plan 04-04 executed: Sharing API (SHARE-01..05) — grant/list/received/revoke with IDOR protection; 7 xfailed, zero new failures | -| Last session | 2026-05-28 — Phase 4 UAT complete (14/15 passed, 1 bug found + fixed: duplicate folder on creation); sidebar collapsible folder tree added; Phase 4 marked complete | -| Last session | 2026-05-28 — Phase 5 UI-SPEC approved (6/6 dimensions passed; 2 revision rounds: Cancel label → context-specific, text-lg → text-xl) | -| Last session | 2026-05-28 — Phase 5 planned (8 plans, 7 waves); verification passed (4 blockers → resolved: D-05 API-layer refresh path, SEC-09 cloud cleanup, frontend_url config, RESEARCH resolved markers) | -| Last session | 2026-05-28 — Plan 05-01 executed: Wave 0 Nyquist scaffold — 19 xfail stubs in test_cloud.py, 4 cloud fixtures in conftest.py, 6 package pins, 8 config settings; 172 passed / 43 xfailed | -| Last session | 2026-05-28 — Plan 05-02 executed: cloud_utils.py (SSRF+HKDF), cloud_cache.py (TTLCache), storage factory extended; 199 passed / 43 xfailed / 1 pre-existing failure | -| Last session | 2026-05-28 — Plan 05-03 executed: GoogleDriveBackend (Drive v3, cache_discovery=False, asyncio.to_thread) + OneDriveBackend (MSAL, resumable upload, CHUNK_SIZE=10MB); 262 passed / 43 xfailed / 1 pre-existing failure | -| Last session | 2026-05-28 — Plan 05-04 executed: WebDAVBackend + NextcloudBackend (SSRF double-guard, asyncio.to_thread, list_folder); 262 passed / 43 xfailed / 1 pre-existing failure | -| Last session | 2026-05-29 — Plan 05-05 executed: cloud.py (7 endpoints), main.py (routers registered), admin.py (SEC-09 cloud cleanup); 262 passed / 43 xfailed / 1 pre-existing failure | -| Last session | 2026-05-29 — Plan 05-06 executed: documents.py cloud upload+content-proxy extension; all 15 xfail stubs promoted to 20 passing tests (CLOUD-03, CLOUD-05, CLOUD-07); 282 passed / 24 xfailed / 1 pre-existing failure | -| Last session | 2026-05-29 — Plan 05-07 executed: useCloudConnectionsStore, 3-tab SettingsView, SettingsCloudTab (4 providers, status badges, OAuth callback), CloudCredentialModal; 61 tests passing, build exits 0 | -| Last session | 2026-05-29 — Phase 5 complete: 4 cloud backends (Google Drive, OneDrive, Nextcloud, WebDAV), HKDF credential encryption, SSRF prevention, OAuth flows, cloud API (7 endpoints), frontend Settings 3-tab + CloudCredentialModal, AppSidebar cloud section, all 20 Phase 5 tests passing, security gates passed | -| Last session | 2026-05-30 — Phase 5 UAT: 5/6 tests passed; 3 gaps diagnosed (OneDrive unconfigured 500, cloud doc stream opaque 500, DropZone disappeared); gap-closure plan 05-12 created (3 tasks, wave 1) | -| Last session | 2026-05-30 — Plan 05-12 executed: OAuth 400 preflight (unconfigured creds), 502 cloud fallback, celery-worker volume mount, upload hint in CloudStorageView; 293 passed / 24 xfailed / 1 pre-existing failure | -| Last session | 2026-05-30 — Phase 6.1 executed: 7 share tests + 4 audit tests promoted from xfail stubs; second_auth_user fixture added; 309 passed / 0 failed | -| Last session | 2026-05-31 — Phase 6.2 planned: 4 plans (3 waves); SHARE-03/SHARE-05 (Plan 02), cloud-delete (Plan 03), ADMIN-06 audit enrichment + CSV + daily exports (Plan 04); verification passed (0 blockers, 2 cosmetic warnings fixed) | -| Last session | 2026-06-03 — Phase 7 planned: 5 plans (5 waves); system_settings DB + HKDF (Plan 01), ProviderConfig + GenericOpenAIProvider + singleton fix (Plan 02), Anthropic output_config + classifier wiring (Plan 03), Celery retry harness + re-queue endpoint (Plan 04), admin AI panel + DocumentCard badge (Plan 05); verification passed (4 blockers fixed in revision round, 0 remaining) | -| Last session | 2026-06-05 — Phase 7 complete: all 5 plans executed; UAT 11/11 passed; security gate passed (bandit zero HIGH, npm audit zero high/critical); _doc_to_dict status field fix + regression test committed; v1.0 milestone DONE | -| Last session | 2026-06-05 — Phase 7.1 complete: CR-01/CR-02/CR-03 implemented; skip_token_hash added to revoke_all_refresh_tokens; change_password, enable_totp, disable_totp now revoke other sessions and return sessions_revoked; 3 new tests passing; frontend toasts in SettingsAccountTab + TotpEnrollment; 373 passed 0 failed; v0.1.1 | -| Last session | 2026-06-05 — Phase 7.2 planned: 3 plans (Wave 0 test scaffolding, Wave 1 jti+NBF check, Wave 2 user_nbf writes in 4 handlers); verification passed (0 blockers, 1 warning fixed); ready to execute | -| Last session | 2026-06-06 — Phase 7.3 complete: ES256 asymmetric JWT signing live; default session 16h; remember_me opt-in 30d; startup bulk-revocation hook; all 9 ES256/remember-me tests green; v0.1.2 | -| Next action | Phase 7.4 complete — security review then ship | +| Last session | 2026-06-07 — Milestone v0.2 started; requirements definition in progress | +| Next action | Approve requirements, then roadmap | | Pending decisions | None | | Resume file | None | diff --git a/.planning/research/ARCHITECTURE.md b/.planning/research/ARCHITECTURE.md index 30ca059..f5f44e0 100644 --- a/.planning/research/ARCHITECTURE.md +++ b/.planning/research/ARCHITECTURE.md @@ -1,704 +1,508 @@ -# Architecture Research +# Architecture Patterns -**Domain:** Multi-user SaaS document management platform (FastAPI + Vue 3 brownfield migration) -**Researched:** 2026-05-21 -**Confidence:** HIGH (auth DI pattern confirmed via official FastAPI docs; storage/DB patterns are well-established S3/PostgreSQL engineering standards cross-verified against official MinIO and SQLAlchemy docs) +**Domain:** v0.2 UI Overhaul and Optimization — integration analysis for existing DocuVault codebase +**Researched:** 2026-06-07 --- -## Standard Architecture +## How the Layout System Currently Works -### System Overview +App.vue is the layout switch. It checks `route.meta.layout` and renders one of two branches: ``` -┌──────────────────────────────────────────────────────────────────────┐ -│ Browser (Vue 3 SPA) │ -│ ┌───────────┐ ┌──────────────┐ ┌───────────┐ ┌──────────────┐ │ -│ │ auth store│ │ docs store │ │quota store│ │settings store│ │ -│ └─────┬─────┘ └──────┬───────┘ └─────┬─────┘ └──────┬───────┘ │ -│ └───────────────┴────────────────┴────────────────┘ │ -│ api/client.js (Bearer token injected) │ -└───────────────────────────────────┬──────────────────────────────────┘ - │ HTTPS/JSON + multipart - ┌─────────▼─────────┐ - │ Load Balancer │ (future; optional now) - └────────┬──────────┘ - ┌───────────────────┼───────────────────┐ - │ │ │ - ┌──────────▼──────┐ ┌─────────▼──────┐ ┌────────▼───────┐ - │ FastAPI inst 1 │ │ FastAPI inst 2 │ │ FastAPI inst N │ - │ (stateless) │ │ (stateless) │ │ (stateless) │ - └──────────┬───────┘ └────────┬────────┘ └────────┬───────┘ - └───────────────────┼───────────────────┘ - ┌─────────▼──────────┐ - │ Shared Services │ - ┌──────────┴──────────────────────┴─────────┐ - │ │ - ┌──────────▼──────────┐ ┌────────────────▼──────┐ - │ PostgreSQL │ │ MinIO │ - │ (users, docs, meta, │ │ (object storage, │ - │ quotas, audit) │ │ one bucket per user │ - └──────────────────────┘ │ OR prefix-per-user) │ - └───────────────────────┘ - │ - ┌──────────────────────────┼─────────────────┐ - │ │ │ - ┌──────────▼────────┐ ┌────────────▼───────┐ ┌────▼──────┐ - │ Cloud Storage │ │ OneDrive Adapter │ │ WebDAV │ - │ Adapter (base) │ │ Google Drive │ │ Adapter │ - └───────────────────┘ └────────────────────┘ └───────────┘ +route.meta.layout === 'auth' → (renders inside a centered card) +(anything else) → AppSidebar +
``` +There are currently only two layouts: +- **AuthLayout.vue** — centered card, no sidebar, used for login/register/password-reset +- **Implicit main layout** — App.vue's `v-else` branch: `` + scrollable main + +The admin panel (`/admin`) currently falls into the `v-else` branch. AdminView.vue renders inside the main user sidebar, inheriting AppSidebar's folder tree, quota bar, and user nav — none of which are relevant to an admin. + --- -## Component Boundaries +## Integration Point 1: FastAPI Admin Router Decomposition -| Component | Responsibility | Communicates With | -|-----------|---------------|-------------------| -| `api/auth.py` | Registration, login, token refresh, TOTP enroll/verify | `services/user_service.py`, DB | -| `api/documents.py` | Upload, list, get, delete, reclassify, share | `services/document_service.py`, quota dep | -| `api/folders.py` | Folder CRUD, move | `services/folder_service.py` | -| `api/storage_backends.py` | Connect/disconnect cloud accounts, list/browse | `services/cloud_service.py` | -| `api/admin.py` | User CRUD, quota adjustments, audit log, AI config | `services/admin_service.py` | -| `deps/auth.py` | `get_current_user` — verifies JWT, returns `User` model | DB, `jose`/`PyJWT` | -| `deps/quota.py` | `check_quota` — reads user's usage, raises 413 if exceeded | DB | -| `deps/db.py` | `get_db` — yields async SQLAlchemy session | PostgreSQL | -| `services/document_service.py` | Orchestrates extract → classify → store flow | `extractor`, `classifier`, `storage_service` | -| `services/storage_service.py` | Routes to MinIO or cloud adapter; enforces object key namespacing | MinIO, cloud adapters | -| `services/user_service.py` | Password hashing, TOTP provisioning, breach check | DB, `bcrypt`, `pyotp` | -| `services/quota_service.py` | Compute used bytes from DB, update after upload/delete | DB | -| `services/audit_service.py` | Append-only audit log writes | DB | -| `services/cloud_service.py` | Manage encrypted cloud credentials, proxy operations | Cloud adapters, DB | -| `storage/base.py` | `StorageBackend` ABC (mirrors `ai/base.py` pattern) | — | -| `storage/minio_backend.py` | MinIO S3 implementation | MinIO | -| `storage/onedrive_backend.py` | OneDrive Graph API implementation | Microsoft Graph | -| `storage/gdrive_backend.py` | Google Drive API implementation | Google Drive API | -| `storage/nextcloud_backend.py` | Nextcloud WebDAV implementation | WebDAV | -| `db/models.py` | SQLAlchemy ORM models | PostgreSQL | -| `db/migrations/` | Alembic migration history | — | +### Current state ---- +`backend/api/admin.py` is a single file with `router = APIRouter(prefix="/api/admin")`. It is registered in `main.py` as: -## Recommended Project Structure - -``` -backend/ -├── main.py # FastAPI app factory, middleware, router registration -├── config.py # pydantic-settings: DB URL, MinIO creds, secret keys -├── deps/ -│ ├── auth.py # get_current_user, get_current_admin -│ ├── db.py # get_db (async session dependency) -│ └── quota.py # check_upload_quota (raises 413 if exceeded) -├── api/ -│ ├── auth.py # /auth/register, /auth/login, /auth/refresh, /auth/totp/* -│ ├── documents.py # /documents/* (existing routes, now user-scoped) -│ ├── folders.py # /folders/* -│ ├── storage_backends.py # /storage-backends/* (cloud account management) -│ └── admin.py # /admin/* (users, quotas, audit, AI config) -├── services/ -│ ├── document_service.py # upload orchestration (extract → classify → store → quota) -│ ├── storage_service.py # routes uploads to correct StorageBackend -│ ├── quota_service.py # read/write quota usage -│ ├── user_service.py # user creation, password, TOTP -│ ├── audit_service.py # audit log writes -│ └── cloud_service.py # cloud backend credential management -├── storage/ # cloud storage adapter layer (mirrors ai/) -│ ├── base.py # StorageBackend ABC -│ ├── __init__.py # get_storage_backend() factory -│ ├── minio_backend.py # default local-S3 backend -│ ├── onedrive_backend.py -│ ├── gdrive_backend.py -│ └── nextcloud_backend.py # WebDAV-based -├── ai/ # unchanged — existing provider abstraction -│ └── ... -├── db/ -│ ├── models.py # all SQLAlchemy ORM models -│ ├── session.py # async engine + sessionmaker -│ └── migrations/ # Alembic env + version scripts -└── tests/ -``` - -### Structure Rationale - -- **`deps/`:** FastAPI dependency functions isolated from service logic. Auth, DB session, and quota are injected independently — routes compose them without coupling. -- **`storage/`:** Direct mirror of `ai/` module. Same ABC + factory pattern. Existing team mental model applies immediately. -- **`db/`:** ORM models and session config separated from services, ensuring migrations can be run independently of app startup. - ---- - -## Architectural Patterns - -### Pattern 1: JWT Verification via Dependency Injection (not middleware) - -**What:** JWT parsing and user lookup happens in `deps/auth.py::get_current_user`, injected via `Depends()` per route. - -**When to use:** All authenticated routes. Admin routes additionally inject `get_current_admin` which calls `get_current_user` then checks `user.role == "admin"`. - -**Trade-offs:** Unauthenticated routes (health check, login, register) require no special exclusion logic. Middleware-based auth forces you to maintain an allowlist of public routes — that list inevitably drifts. DI is opt-in per route, which is safer. - -**Example:** ```python -# deps/auth.py -oauth2_scheme = OAuth2PasswordBearer(tokenUrl="/auth/login") - -async def get_current_user( - token: Annotated[str, Depends(oauth2_scheme)], - db: Annotated[AsyncSession, Depends(get_db)], -) -> User: - try: - payload = jwt.decode(token, settings.jwt_secret, algorithms=["HS256"]) - user_id: str = payload.get("sub") - if user_id is None: - raise credentials_exception - except JWTError: - raise credentials_exception - user = await db.get(User, user_id) - if user is None or not user.is_active: - raise credentials_exception - return user - -# api/documents.py -@router.get("/documents") -async def list_documents( - current_user: Annotated[User, Depends(get_current_user)], - db: Annotated[AsyncSession, Depends(get_db)], -): - ... +from api.admin import router as admin_router +app.include_router(admin_router) ``` -**Confirmed:** HIGH confidence — FastAPI official documentation explicitly recommends this pattern over middleware for auth. +The file mixes four concern areas: +- User management (CRUD on users, status, password reset) +- Quota management (get/patch quota per user) +- AI provider configuration (system-level: get, put, test-connection, get-models) +- Audit log was already extracted to `api/audit.py` in v0.1 ---- +### Target structure -### Pattern 2: Refresh Token Rotation - -**What:** Short-lived access tokens (15 min) + long-lived refresh tokens (30 days) stored in `refresh_tokens` table. On every `/auth/refresh` call, the old token is invalidated and a new pair is issued. - -**When to use:** Always, for multi-user SaaS. Prevents stolen tokens having indefinite access. - -**Trade-offs:** Requires `refresh_tokens` DB table and one extra DB write per refresh. The alternative (long-lived JWTs) cannot be revoked without a blocklist, which has the same cost. - -**Implementation notes:** -- Refresh token = opaque random UUID (not JWT) — store hashed in DB alongside `user_id`, `expires_at`, `revoked` -- Access token = JWT with `sub=user_id`, `exp=now+15m`, `jti=uuid` (for optional blocklist future) -- On logout or password change: set `revoked=true` on all user's refresh tokens -- On TOTP failure after password success: do not issue any token; log failed_mfa audit event - ---- - -### Pattern 3: MinIO Presigned URL Flow (preferred over streaming proxy) - -**What:** FastAPI generates a short-lived presigned PUT URL from MinIO; the browser uploads directly to MinIO. For downloads, FastAPI generates a presigned GET URL and redirects. - -**When to use:** All document uploads and downloads where the client is a browser on the same network as MinIO (typical Docker Compose deployment). Use streaming proxy only when MinIO is not reachable from the browser (e.g., MinIO is behind an internal network). - -**Trade-offs:** -- Presigned URL avoids buffering the file through FastAPI — reduces memory pressure and latency significantly for large files. -- The FastAPI instance must be able to reach MinIO to generate the URL, but does not need to handle the byte stream. -- For Docker Compose: MinIO is on the internal Docker network; expose only the presigned-URL-generating endpoint externally. The presigned URL itself points to the MinIO public port. - -**Flow:** ``` -1. POST /documents/upload-url (FastAPI, authenticated) - → quota check - → generate presigned PUT URL (expires 5 min) - → return { upload_url, object_key, document_id } - -2. PUT (browser → MinIO directly) - → no FastAPI involvement - -3. POST /documents/confirm { document_id } (FastAPI, authenticated) - → verify object exists in MinIO - → trigger text extraction + classification (background task) - → update document status to "processing" - → return document record +backend/api/admin/ + __init__.py + users.py — /api/admin/users endpoints + quotas.py — /api/admin/users/{id}/quota endpoints + ai.py — /api/admin/ai-config endpoints ``` -**Object key namespace:** `{user_id}/{document_id}/{filename}` — ensures per-user isolation without separate buckets. One bucket (`docuvault-documents`) is sufficient; IAM policies or object key prefix checks enforce isolation in code. +### How to register without changing URL prefix + +FastAPI supports two-level include: a parent router with the shared prefix, sub-routers without prefix. This is the correct pattern: -**Presigned GET for downloads:** ```python -url = minio_client.presigned_get_object( - bucket_name="docuvault-documents", - object_name=f"{user_id}/{document_id}/{filename}", - expires=timedelta(minutes=30), -) -return RedirectResponse(url) +# backend/api/admin/__init__.py +from fastapi import APIRouter +from .users import router as users_router +from .quotas import router as quotas_router +from .ai import router as ai_router + +router = APIRouter(prefix="/api/admin", tags=["admin"]) +router.include_router(users_router) +router.include_router(quotas_router) +router.include_router(ai_router) ``` -**Confidence:** HIGH for the S3 presigned URL pattern (standard across all S3-compatible stores). MinIO Python SDK `presigned_put_object` and `presigned_get_object` methods confirmed as stable API. +Each sub-router declares its routes WITHOUT a prefix (no `prefix=` arg): ---- - -### Pattern 4: Cloud Storage Adapter (StorageBackend ABC) - -**What:** A `StorageBackend` ABC in `storage/base.py` defines the interface. Each cloud integration implements it. `storage_service.py` routes to the correct backend based on the user's `default_storage_backend` setting. - -**When to use:** Any operation that reads or writes document bytes. The service layer never calls MinIO or Google Drive directly — always via the adapter. - -**Interface:** ```python -# storage/base.py -from abc import ABC, abstractmethod -from typing import AsyncIterator +# backend/api/admin/users.py +router = APIRouter() -class StorageBackend(ABC): - @abstractmethod - async def put_object(self, key: str, data: bytes, content_type: str) -> str: - """Store object, return canonical reference (URL or key).""" - - @abstractmethod - async def get_object(self, key: str) -> bytes: - """Retrieve object bytes.""" - - @abstractmethod - async def delete_object(self, key: str) -> None: - """Delete object.""" - - @abstractmethod - async def get_presigned_url(self, key: str, expires_seconds: int = 3600) -> str | None: - """Return a time-limited direct URL, or None if backend doesn't support it.""" - - @abstractmethod - async def list_objects(self, prefix: str) -> list[str]: - """List keys under prefix.""" - - @abstractmethod - async def health_check(self) -> bool: - """Verify connectivity.""" +@router.get("/users") # resolves to /api/admin/users +@router.post("/users") # resolves to /api/admin/users +@router.patch("/users/{id}/status") +... ``` -**Factory:** +`main.py` changes from: + ```python -# storage/__init__.py -def get_storage_backend(user: User, credentials: dict | None) -> StorageBackend: - backend_type = user.default_storage_backend # "minio" | "onedrive" | "gdrive" | ... - if backend_type == "minio": - return MinIOBackend(settings.minio_endpoint, ...) - elif backend_type == "onedrive": - return OneDriveBackend(credentials) # decrypted before passing in - ... +from api.admin import router as admin_router +app.include_router(admin_router) ``` -**Credentials encryption:** Cloud OAuth tokens and refresh tokens are stored encrypted with Fernet symmetric encryption. The key is in `CLOUD_CREDS_KEY` env var. Encryption/decryption happens in `cloud_service.py` before the credentials are passed to the backend constructor — the adapter itself always receives plaintext credentials and never touches the DB. +to the identical line — because `api/admin/__init__.py` exports `router` with the same name. **No URL changes. No consumer changes.** + +### What moves where + +| Endpoints | Target file | Pydantic models that move with it | +|---|---|---| +| GET/POST /users, PATCH /users/{id}/status, POST /users/{id}/password-reset, DELETE /users/{id} | `admin/users.py` | `UserCreate`, `UserStatusUpdate`, `UserDeleteConfirm`, `_user_to_dict` | +| GET/PATCH /users/{id}/quota, PATCH /users/{id}/ai-config | `admin/quotas.py` | `QuotaUpdate`, `UserAiConfigUpdate` | +| GET/PUT /ai-config, POST /ai-config/test-connection, GET /ai-config/models | `admin/ai.py` | `SystemAiConfigUpdate`, `TestConnectionRequest`, `_ai_config_to_dict` | + +`POST /topics` (create system topic) stays in `admin/users.py` — it is short and has no unique models. Alternatively it can go in its own `admin/topics.py`; a decision either way is fine. + +### Shared helpers + +`_DEFAULT_QUOTA_BYTES`, `CloudConnectionOut`, and any imports shared across sub-routers can live in `backend/api/admin/shared.py` and be imported by each sub-router. Do not duplicate them. + +### Files changed + +| File | Change type | +|---|---| +| `backend/api/admin.py` | DELETED — replaced by package | +| `backend/api/admin/__init__.py` | CREATED — aggregating router | +| `backend/api/admin/users.py` | CREATED | +| `backend/api/admin/quotas.py` | CREATED | +| `backend/api/admin/ai.py` | CREATED | +| `backend/api/admin/shared.py` | CREATED (optional, holds shared constants/helpers) | +| `backend/main.py` | NO CHANGE — import path `api.admin.router` remains valid | --- -### Pattern 5: Storage Quota Enforcement via Service Layer (not middleware, not DB constraint) +## Integration Point 2: Vue Admin Layout -**What:** Quota is checked in `deps/quota.py::check_upload_quota` — a FastAPI dependency injected on upload routes. After successful upload, `quota_service.increment_usage(user_id, bytes)` is called. +### Current layout switch -**Where NOT to enforce:** -- **Not in middleware:** Middleware cannot easily read the `Content-Length` before the body is buffered, and cannot know user identity without re-implementing auth. -- **Not as a DB constraint:** `CHECK (used_bytes <= limit_bytes)` would require the DB to reject the commit, creating a race between the object already uploaded to MinIO and the metadata not committed. Inconsistency. +`App.vue`: -**Correct sequence:** -``` -1. Pre-upload: deps/quota.py reads user.quota_used_bytes + Content-Length header - → if (used + incoming) > limit_bytes: raise HTTP 413 with quota detail - -2. Upload proceeds to MinIO (presigned URL or proxy) - -3. Post-upload: quota_service.increment_usage atomically: - UPDATE quotas SET used_bytes = used_bytes + $delta - WHERE user_id = $uid AND (used_bytes + $delta) <= limit_bytes - RETURNING used_bytes - → if no rows returned: another concurrent upload exceeded quota; delete from MinIO + 413 +```html + +
+ +
+ +
+
``` -**Why atomic update with check:** Two simultaneous uploads can both pass the pre-check. The atomic UPDATE with WHERE guard prevents double-spend. This is the correct pattern for optimistic quota enforcement under concurrency. +### How to add AdminLayout without breaking anything + +Extend the `v-if` chain to a three-way switch using `route.meta.layout`: + +```html + + +
+ +
+ +
+
+``` + +`AdminLayout.vue` renders its own sidebar and a ``. It is structurally identical in shape to the main layout but with admin-specific nav items. + +### Router changes + +Current single admin route: + +```js +{ path: '/admin', component: () => import('../views/AdminView.vue'), meta: { requiresAdmin: true } } +``` + +Replace with a route subtree: + +```js +{ + path: '/admin', + component: AdminLayout, + meta: { requiresAdmin: true, layout: 'admin' }, + children: [ + { path: '', redirect: '/admin/users' }, + { path: 'users', component: () => import('../views/admin/AdminUsersView.vue') }, + { path: 'quotas', component: () => import('../views/admin/AdminQuotasView.vue') }, + { path: 'ai', component: () => import('../views/admin/AdminAiView.vue') }, + { path: 'audit', component: () => import('../views/admin/AdminAuditView.vue') }, + ], +} +``` + +### Meta inheritance caveat + +Vue Router 4 does NOT automatically merge `meta` from parent to children — `route.meta` on a child route only contains what is declared on that child. `App.vue` currently reads `route.meta.layout` directly. For the child routes to inherit `layout: 'admin'`, the check in App.vue must walk `route.matched`: + +```js +// App.vue — layout resolution +const layout = computed(() => route.matched.find(r => r.meta.layout)?.meta.layout) +``` + +Likewise, the navigation guard in `router/index.js` must change from `to.meta.requiresAdmin` to `to.matched.some(r => r.meta.requiresAdmin)`. The current guard already handles `.meta.public` the same way — so this is a one-line change pattern that's already established in the router. + +### AdminLayout.vue structure + +``` +AdminLayout.vue + └── AdminSidebar.vue (nav links: Users, Quotas, AI Config, Audit Log) + └── +``` + +`AdminSidebar.vue` is NOT `AppSidebar.vue`. Do not reuse or extend `AppSidebar`. The admin sidebar is a new, independent component with admin-specific nav. + +### Views created + +Each current tab component becomes a thin view wrapper (following the View -> Smart -> Presentational hierarchy): + +| New file | Wraps existing component | +|---|---| +| `views/admin/AdminUsersView.vue` | `AdminUsersTab.vue` (kept unchanged as smart component) | +| `views/admin/AdminQuotasView.vue` | `AdminQuotasTab.vue` | +| `views/admin/AdminAiView.vue` | `AdminAiConfigTab.vue` | +| `views/admin/AdminAuditView.vue` | `AuditLogTab.vue` | + +The existing tab components (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, etc.) are NOT renamed or restructured in this step. They become the smart components rendered inside the new thin view wrappers. + +### Files changed + +| File | Change type | +|---|---| +| `frontend/src/App.vue` | MODIFIED — add `v-else-if` for admin layout; meta layout resolution uses `route.matched` | +| `frontend/src/router/index.js` | MODIFIED — /admin replaced with children subtree; guard uses `to.matched` | +| `frontend/src/layouts/AdminLayout.vue` | CREATED | +| `frontend/src/components/layout/AdminSidebar.vue` | CREATED | +| `frontend/src/views/AdminView.vue` | DELETED — replaced by subtree | +| `frontend/src/views/admin/AdminUsersView.vue` | CREATED (thin wrapper) | +| `frontend/src/views/admin/AdminQuotasView.vue` | CREATED (thin wrapper) | +| `frontend/src/views/admin/AdminAiView.vue` | CREATED (thin wrapper) | +| `frontend/src/views/admin/AdminAuditView.vue` | CREATED (thin wrapper) | +| `frontend/src/components/admin/AdminUsersTab.vue` | KEPT (smart component, unchanged) | +| `frontend/src/components/admin/AdminQuotasTab.vue` | KEPT | +| `frontend/src/components/admin/AdminAiConfigTab.vue` | KEPT | +| `frontend/src/components/admin/AuditLogTab.vue` | KEPT | --- -## PostgreSQL Schema Design +## Integration Point 3: client.js Decomposition -### Core Tables +### Current import pattern -```sql --- Users -CREATE TABLE users ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - handle TEXT UNIQUE NOT NULL, -- @username for sharing - email TEXT UNIQUE NOT NULL, - password_hash TEXT NOT NULL, -- bcrypt - totp_secret TEXT, -- NULL = TOTP not enabled - totp_enabled BOOLEAN NOT NULL DEFAULT FALSE, - role TEXT NOT NULL DEFAULT 'user', -- 'user' | 'admin' - is_active BOOLEAN NOT NULL DEFAULT TRUE, - ai_provider TEXT, -- NULL = use system default - ai_model TEXT, - created_at TIMESTAMPTZ NOT NULL DEFAULT now() -); +Every consumer does one of: --- Quotas (1:1 with users; separate for clean admin queries) -CREATE TABLE quotas ( - user_id UUID PRIMARY KEY REFERENCES users(id) ON DELETE CASCADE, - limit_bytes BIGINT NOT NULL DEFAULT 104857600, -- 100 MB - used_bytes BIGINT NOT NULL DEFAULT 0, - CONSTRAINT no_negative_usage CHECK (used_bytes >= 0) -); - --- Refresh tokens -CREATE TABLE refresh_tokens ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - token_hash TEXT NOT NULL UNIQUE, -- SHA-256 of the opaque token - expires_at TIMESTAMPTZ NOT NULL, - revoked BOOLEAN NOT NULL DEFAULT FALSE, - created_at TIMESTAMPTZ NOT NULL DEFAULT now() -); -CREATE INDEX ON refresh_tokens(user_id, revoked); - --- Folders -CREATE TABLE folders ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - parent_id UUID REFERENCES folders(id) ON DELETE CASCADE, -- NULL = root - name TEXT NOT NULL, - created_at TIMESTAMPTZ NOT NULL DEFAULT now(), - UNIQUE (user_id, parent_id, name) -); - --- Documents -CREATE TABLE documents ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - folder_id UUID REFERENCES folders(id) ON DELETE SET NULL, - filename TEXT NOT NULL, - content_type TEXT NOT NULL, - size_bytes BIGINT NOT NULL DEFAULT 0, - storage_backend TEXT NOT NULL DEFAULT 'minio', -- 'minio' | 'onedrive' | ... - object_key TEXT NOT NULL, -- backend-specific reference - extracted_text TEXT, -- NULL until extraction complete - status TEXT NOT NULL DEFAULT 'pending', -- pending | processing | ready | error - created_at TIMESTAMPTZ NOT NULL DEFAULT now(), - updated_at TIMESTAMPTZ NOT NULL DEFAULT now() -); -CREATE INDEX ON documents(user_id, folder_id); -CREATE INDEX ON documents(user_id, created_at DESC); - --- Document topics (M:N) -CREATE TABLE document_topics ( - document_id UUID NOT NULL REFERENCES documents(id) ON DELETE CASCADE, - topic_id UUID NOT NULL REFERENCES topics(id) ON DELETE CASCADE, - PRIMARY KEY (document_id, topic_id) -); - --- Topics (per-user; admin sets defaults) -CREATE TABLE topics ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID REFERENCES users(id) ON DELETE CASCADE, -- NULL = system default - name TEXT NOT NULL, - UNIQUE (user_id, name) -); - --- Document shares -CREATE TABLE document_shares ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - document_id UUID NOT NULL REFERENCES documents(id) ON DELETE CASCADE, - owner_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - recipient_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - permission TEXT NOT NULL DEFAULT 'view', -- 'view' | 'download' (future) - created_at TIMESTAMPTZ NOT NULL DEFAULT now(), - UNIQUE (document_id, recipient_id) -); -CREATE INDEX ON document_shares(recipient_id); - --- Cloud storage backends per user -CREATE TABLE cloud_backends ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE, - backend_type TEXT NOT NULL, -- 'onedrive' | 'gdrive' | 'nextcloud' | 'webdav' - display_name TEXT NOT NULL, - credentials_enc TEXT NOT NULL, -- Fernet-encrypted JSON blob - is_default BOOLEAN NOT NULL DEFAULT FALSE, - created_at TIMESTAMPTZ NOT NULL DEFAULT now() -); -CREATE INDEX ON cloud_backends(user_id); - --- Audit log (append-only) -CREATE TABLE audit_log ( - id BIGSERIAL PRIMARY KEY, - user_id UUID REFERENCES users(id) ON DELETE SET NULL, - actor_id UUID REFERENCES users(id) ON DELETE SET NULL, -- admin acting on behalf - event_type TEXT NOT NULL, -- login | login_failed | upload | delete | share | quota_change | ... - resource_id UUID, -- document_id / folder_id / user_id depending on context - ip_address INET, - metadata JSONB, -- event-specific extra fields (no document content) - created_at TIMESTAMPTZ NOT NULL DEFAULT now() -); -CREATE INDEX ON audit_log(user_id, created_at DESC); -CREATE INDEX ON audit_log(event_type, created_at DESC); --- NOTE: no UPDATE or DELETE grants on audit_log for app user; only INSERT + SELECT +```js +import * as api from '../api/client.js' // namespace import — stores, views, most components +import { specificFn } from '../api/client.js' // named import — a few components ``` -**Schema design notes:** -- `topics.user_id IS NULL` = system-wide default topics visible to all users; per-user topics shadow them. -- `documents.object_key` stores the backend-relative reference — for MinIO it is `{user_id}/{document_id}/{filename}`; for OneDrive it is the Drive item ID. The `storage_backend` column tells the service which adapter to use. -- `cloud_backends.credentials_enc` is never returned in any API response; only the adapter factory decrypts it server-side. -- Audit log uses `BIGSERIAL` (not UUID) for append-ordered natural scan and to discourage random access patterns. +35+ files import from `client.js`. A decomposition that changes the import path in all 35 files is a large, purely mechanical refactor with meaningful test/review cost for zero user-visible benefit. + +### Minimal, non-breaking approach: barrel re-export + +Create domain modules alongside `client.js`, then have `client.js` re-export everything so no consumer changes: + +``` +frontend/src/api/ + client.js — request() transport function + re-exports from domain files + documents.js — document API calls (import request from './client.js') + topics.js + auth.js + admin.js + folders.js + shares.js + cloud.js +``` + +`client.js` becomes the HTTP transport layer and a re-export barrel: + +```js +// client.js (after decomposition) — transport only + re-exports +async function request(path, options = {}) { ... } +export async function fetchDocumentContent(docId, options = {}) { ... } + +export * from './documents.js' +export * from './topics.js' +export * from './auth.js' +export * from './admin.js' +export * from './folders.js' +export * from './shares.js' +export * from './cloud.js' +``` + +Each domain file imports `request` from `client.js`: + +```js +// documents.js +import { request } from './client.js' +export function listDocuments(...) { return request('/api/documents?...') } +``` + +This means `import * as api from '../api/client.js'` continues to work for all 35 existing consumers. The decomposition is fully internal. + +### Test mock compatibility + +Existing Vitest tests mock `api/client.js` at the module level (`vi.mock('../../api/client.js')`). Because the re-exports are in `client.js`, mocking `client.js` still intercepts all domain functions. No test changes needed. + +### Files changed + +| File | Change type | +|---|---| +| `frontend/src/api/client.js` | MODIFIED — split out domain functions, keep re-exports | +| `frontend/src/api/documents.js` | CREATED | +| `frontend/src/api/topics.js` | CREATED | +| `frontend/src/api/auth.js` | CREATED (name does not conflict with `stores/auth.js` — different path) | +| `frontend/src/api/admin.js` | CREATED | +| `frontend/src/api/folders.js` | CREATED | +| `frontend/src/api/shares.js` | CREATED | +| `frontend/src/api/cloud.js` | CREATED | +| All 35 consumer files | NO CHANGE | --- -## Data Flow +## Integration Point 4: Icon Strategy -### Document Upload Flow (MinIO presigned URL path) +### Current state + +There are 58 inline `` blocks in `frontend/src/components/` and 8 more in `frontend/src/views/`. Each is a 3-6 line block with `stroke`, `viewBox`, `class`, and a ``. The same path data appears multiple times for the same icon (e.g., the document icon in the sidebar, document card, and document view are separate copies of the same path string). + +### Options assessed + +**Option A: SVG sprite sheet** +A ``-based sprite in `index.html` with `` in components. Minimal bundle impact, no component overhead. Drawback: requires a build step or manual maintenance of the sprite file; poor DX for adding new icons during the visual redesign. + +**Option B: `AppIcon.vue` component with a `name` prop** +A Vue component that maps `name` to an inline `` block. All SVG paths live in one file. Zero external dependencies. Easy to add icons. + +**Option C: lucide-vue-next** +Purpose-built icon library for Tailwind projects. Tree-shakeable individual icon components: ``. Maintained, typed, comprehensive. Avoids maintaining path data entirely. + +**Recommendation: Option B (`AppIcon.vue`) unless a dependency is explicitly approved.** + +The project already has 66 SVG paths hand-chosen to match the visual style. Switching to lucide means auditing that every lucide icon matches the expected visual. During a visual redesign milestone that may be desirable, but it is a separate decision. The minimal refactor that eliminates duplication without changing visual output is Option B. + +**Named slots are wrong for this problem.** Named slots solve "inject different markup into different zones of a layout". This problem is "render the correct path data for a given icon name". A `name` prop is the correct API. + +**Option B implementation:** ``` -Browser - │ - ├─[1] POST /documents/upload-url {filename, size, content_type, folder_id?} - │ → get_current_user dep (JWT verify → load User from DB) - │ → check_upload_quota dep (reads quotas table, compares size) - │ → document_service.prepare_upload() - │ → INSERT documents row (status='pending') - │ → minio_backend.generate_presigned_put(object_key, expires=300s) - │ ← {upload_url, object_key, document_id} - │ - ├─[2] PUT (browser → MinIO, no FastAPI) - │ - ├─[3] POST /documents/{id}/confirm - │ → get_current_user dep - │ → document_service.confirm_upload() - │ → verify object exists in MinIO (HEAD request) - │ → quota_service.increment_usage(user_id, size_bytes) [atomic] - │ → UPDATE documents SET status='processing' - │ → enqueue background task: extract_and_classify(document_id) - │ ← {document_id, status: "processing"} - │ - └─[4] Background: extract_and_classify(document_id) - → extractor.extract_text(object bytes from MinIO) - → classifier.classify(text, user_topics) - → UPDATE documents SET extracted_text=..., status='ready' - → UPDATE document_topics - → audit_service.log(event='upload', ...) +frontend/src/components/ui/AppIcon.vue ``` -### Authentication Flow - -``` -Browser - │ - ├─[1] POST /auth/login {email, password, totp_code?} - │ → user_service.verify_password(email, password) - │ → if totp_enabled: pyotp.TOTP(secret).verify(totp_code) - │ → issue access_token (JWT, 15 min) + refresh_token (opaque UUID) - │ → store hash(refresh_token) in refresh_tokens table - │ ← {access_token, refresh_token, expires_in} - │ - ├─[2] Any authenticated request - │ Authorization: Bearer - │ → get_current_user dep decodes JWT locally (no DB round-trip for valid tokens) - │ - └─[3] POST /auth/refresh {refresh_token} - → look up hash(refresh_token) in refresh_tokens table - → verify not revoked, not expired - → set revoked=true on old token - → issue new access_token + new refresh_token (rotation) - ← {access_token, refresh_token, expires_in} +```html + + ``` -### Shared Document Access Flow +Usage: `` + +### Files changed (Option B) + +| File | Change type | +|---|---| +| `frontend/src/components/ui/AppIcon.vue` | CREATED | +| ~35 components/views with inline SVGs | MODIFIED — replace `...` with `` | + +--- + +## Suggested Build Order + +Dependencies between the four changes determine order. A change that other changes depend on must land first. ``` -Recipient accesses "Shared with me" - │ - ├─ GET /documents/shared-with-me - │ → SELECT d.* FROM documents d - │ JOIN document_shares s ON s.document_id = d.id - │ WHERE s.recipient_id = :current_user_id - │ ← list of document records (owner's documents, recipient has view access) - │ - └─ GET /documents/{id}/download (recipient, shared document) - → verify document_shares row exists for (document_id, current_user_id) - → generate presigned GET URL using owner's object_key - ← 302 redirect to presigned URL - (file bytes flow from MinIO → browser, never through FastAPI) +Step 1 — Backend admin router decomposition +Step 2 — API client decomposition (client.js -> domain files) +Step 3 — Admin layout + route subtree (frontend) +Step 4 — Icon refactor +``` + +### Rationale + +**Step 1 (backend) first:** +- No frontend dependencies. All existing tests pass with zero behavior change (same URLs, same auth, same import path in `main.py`). +- Risk: lowest. Pure internal restructuring. + +**Step 2 (client.js) second:** +- No dependency on the admin layout. +- New admin view files created in Step 3 can import from `api/admin.js` specifically rather than the monolith. +- Internal-only change. All 35 consumer files unchanged. All existing mock-based tests unchanged. +- Risk: low. + +**Step 3 (admin layout + route subtree) third:** +- Highest risk: modifies `App.vue` (central layout switch), `router/index.js` (all navigation), creates new layout files. +- The existing smart components (`AdminUsersTab.vue` etc.) are not touched — they are simply wrapped by new thin view files. +- Navigation guard logic for `requiresAdmin` and `layout` meta must be re-tested after this step. +- Risk: medium. + +**Step 4 (icon refactor) last:** +- Pure cosmetic. Zero behavior change. +- Can be done incrementally (file by file) without blocking anything. +- Risk: lowest. + +### Parallelism + +Steps 1 and 2 are independent and can be executed simultaneously — they touch no shared files. + +Steps 3 and 4 overlap on the admin component files (icons are replaced in Step 4, structure is set in Step 3). Complete Step 3 first to avoid merge conflicts. + +--- + +## Component Architecture: Existing vs. Target + +### Current (wrong) + +``` +App.vue + └── AppSidebar (always present) + └── AdminView.vue (tabs rendered in user sidebar context) + └── AdminUsersTab.vue + └── AdminQuotasTab.vue + └── AdminAiConfigTab.vue + └── AuditLogTab.vue +``` + +### Target (correct) + +``` +App.vue (layout switch) + └── AuthLayout.vue (for meta.layout === 'auth') + └── AdminLayout.vue (for meta.layout === 'admin') + └── AdminSidebar.vue + └── + → AdminUsersView.vue (thin — passes props to AdminUsersTab) + → AdminQuotasView.vue + → AdminAiView.vue + → AdminAuditView.vue + └── Implicit main layout (v-else) + └── AppSidebar.vue + └── + → FileManagerView, DocumentView, TopicsView, ... +``` + +The smart components (`AdminUsersTab.vue` etc.) are unchanged. The view layer above them is what changes. + +--- + +## What Is New vs. Modified: Complete Inventory + +### New files + +``` +backend/api/admin/__init__.py +backend/api/admin/users.py +backend/api/admin/quotas.py +backend/api/admin/ai.py +backend/api/admin/shared.py (optional — shared constants/helpers) + +frontend/src/layouts/AdminLayout.vue +frontend/src/components/layout/AdminSidebar.vue +frontend/src/views/admin/AdminUsersView.vue +frontend/src/views/admin/AdminQuotasView.vue +frontend/src/views/admin/AdminAiView.vue +frontend/src/views/admin/AdminAuditView.vue +frontend/src/api/documents.js +frontend/src/api/topics.js +frontend/src/api/auth.js +frontend/src/api/admin.js +frontend/src/api/folders.js +frontend/src/api/shares.js +frontend/src/api/cloud.js +frontend/src/components/ui/AppIcon.vue (Option B only) +``` + +### Modified files + +``` +backend/main.py (no functional change — import stays identical) + +frontend/src/App.vue (add v-else-if; use route.matched for layout) +frontend/src/router/index.js (admin route becomes children subtree; guard uses to.matched) +frontend/src/api/client.js (gutted to transport + re-exports) +All ~35 components with inline SVGs (icons replaced with AppIcon or lucide) +``` + +### Deleted files + +``` +backend/api/admin.py (replaced by package) +frontend/src/views/AdminView.vue (replaced by route subtree) ``` --- -## Migration Path: Flat-File → PostgreSQL + MinIO +## Minimal vs. Proper Assessment -### Principle: parallel-run, not flag-day cutover - -The safest approach is to keep the existing flat-file code running and introduce the new stack incrementally, in a sequence that never breaks the existing API contract from the Vue frontend's perspective. - -### Phase 1 — Infrastructure, no behavior change - -1. Add PostgreSQL and MinIO services to `docker-compose.yml` -2. Create `db/models.py` with initial schema (users, documents, quotas — no auth yet) -3. Add Alembic, run initial migration -4. Add `deps/db.py` with async session dependency -5. No API changes. Existing flat-file code still runs. - -**Validation:** `docker-compose up` boots all services without errors. Alembic migrations apply cleanly. - -### Phase 2 — Auth layer (new endpoints, existing endpoints temporarily open) - -1. Add `users` table, `refresh_tokens` table -2. Implement `/auth/register`, `/auth/login`, `/auth/refresh` -3. Add `get_current_user` dependency to `deps/auth.py` -4. Add a single test authenticated route (`GET /auth/me`) -5. Existing document endpoints remain unauthenticated (guarded by feature flag or separate router prefix) - -**Validation:** Auth endpoints work independently. Existing UI still calls existing routes without tokens. - -### Phase 3 — Document storage migration (dual-write period) - -1. Add MinIO `minio_backend.py`, integrate into `storage_service.py` -2. Create a one-time migration script: - - Reads each `data/metadata/.json` - - Inserts a `documents` row with `user_id = SYSTEM_USER_ID` (a single placeholder user) - - Uploads `data/uploads/.` to MinIO -3. New uploads go to MinIO + PostgreSQL; old flat-file data has been migrated -4. Update document API routes to read from PostgreSQL + MinIO, guarded behind `get_current_user` - -**Critical:** Run migration script in a transaction; if any file fails, roll back DB inserts (MinIO objects can be cleaned up separately). Do not delete flat-file data until validation is complete. - -### Phase 4 — Multi-user isolation - -1. Add per-user `quotas` rows, `folders` table -2. Enforce user-scoped queries: all document queries include `WHERE user_id = :current_user_id` -3. Add quota enforcement dependency on upload routes -4. Add `document_shares`, `cloud_backends` tables - -### Phase 5 — Cloud storage backends - -1. Implement `StorageBackend` ABC and `MinIOBackend` -2. Implement first cloud adapter (OneDrive or Google Drive) -3. Add `/storage-backends/*` API endpoints -4. Add frontend UI for connecting cloud accounts - -### Frontend changes during migration - -- Add `Authorization: Bearer ` header injection in `src/api/client.js` (single change point — all API calls go through this module already) -- Add login/register views and auth Pinia store -- Redirect to `/login` on 401 responses -- No other frontend changes required until Phase 4 (user-scoped UI) - ---- - -## Horizontal Scaling Concerns - -| Concern | What to Share | What Can Be Instance-Local | -|---------|--------------|---------------------------| -| DB connections | PostgreSQL (shared) — use connection pooling (asyncpg pool size 10-20 per instance) | None | -| Object storage | MinIO (shared) — all instances use same endpoint | None | -| Refresh token state | PostgreSQL `refresh_tokens` table (shared) | JWT validation (CPU-only, no shared state needed) | -| Quota state | PostgreSQL `quotas` table with atomic UPDATE (shared) | Pre-flight Content-Length check (instance-local read, final write shared) | -| Background tasks | Cannot use `BackgroundTasks` across instances — use Celery + Redis OR Postgres-backed queue (pg_boss / pgqueuer) | Single-instance: `BackgroundTasks` is fine for Phase 1 | -| File upload temp buffers | If streaming proxy pattern used: RAM per instance | Use presigned URLs to avoid this entirely | -| AI provider instances | Re-instantiated per request already — no shared state | Per-instance re-instantiation is fine | -| CORS / session | Stateless JWT — no sticky sessions needed | — | - -**First bottleneck:** Background task queue. `FastAPI BackgroundTasks` runs in the same process. When classification is slow or multiple uploads arrive simultaneously, workers block. Introduce a task queue (Celery + Redis, or pgqueuer) before scaling to N instances — otherwise each instance has its own queue and tasks are not distributed. - -**Second bottleneck:** DB connection count. With N instances × 20 connections = N×20 PostgreSQL connections. Add PgBouncer in transaction mode in front of PostgreSQL before N gets large. - ---- - -## Anti-Patterns - -### Anti-Pattern 1: Per-Instance File Locks for Quota - -**What people do:** Carry forward the `filelock` pattern into the multi-instance world, using a lock file on a shared volume. - -**Why it's wrong:** Shared NFS/volume file locking has undefined behavior under Docker Compose networking, requires a shared filesystem mount (kills stateless instances), and is slower than a DB atomic update. - -**Do this instead:** Atomic `UPDATE quotas SET used_bytes = used_bytes + $delta WHERE user_id = $uid AND (used_bytes + $delta) <= limit_bytes` in PostgreSQL. Single round-trip, correct under concurrency, no shared filesystem required. - ---- - -### Anti-Pattern 2: Streaming All File Traffic Through FastAPI - -**What people do:** `POST /upload` receives the multipart body into memory, then POSTs it to MinIO from FastAPI. - -**Why it's wrong:** Doubles memory usage (once in FastAPI, once in MinIO client buffer). Saturates FastAPI worker threads during large uploads. Introduces FastAPI as a bottleneck for byte transfer. - -**Do this instead:** Two-step presigned URL flow (Pattern 3 above). FastAPI only handles metadata; bytes flow browser → MinIO directly. - ---- - -### Anti-Pattern 3: Auth in Middleware Instead of Dependencies - -**What people do:** Write a custom ASGI middleware that reads the `Authorization` header and either passes or rejects requests. - -**Why it's wrong:** Middleware runs before FastAPI routing. To allow public routes (login, register, health), you must maintain an exclusion list in the middleware. This list inevitably goes stale when new public routes are added. Middleware cannot easily populate `request.state.user` in a way that's type-safe for path operations. - -**Do this instead:** `Depends(get_current_user)` on each protected router. Optional auth uses `Depends(get_optional_user)` returning `User | None`. Explicit, type-safe, co-located with the route it protects. Confirmed as FastAPI's recommended pattern. - ---- - -### Anti-Pattern 4: Storing Cloud Credentials Unencrypted (or Relying on DB-Level Encryption Alone) - -**What people do:** Store OAuth tokens in plaintext DB columns, assuming DB-level TLS or disk encryption is sufficient. - -**Why it's wrong:** Any user with DB read access (admin, compromised migration, backup leak) can extract all users' cloud tokens. Violates the privacy-first admin model requirement. - -**Do this instead:** Fernet-encrypt the credential JSON blob in `cloud_service.py` before writing to `cloud_backends.credentials_enc`. The Fernet key lives in `CLOUD_CREDS_KEY` env var only — never in the DB. Admin queries on `cloud_backends` return only `id`, `backend_type`, `display_name`, `is_default` — the `credentials_enc` column is excluded from all admin-facing serializers. - ---- - -### Anti-Pattern 5: One MinIO Bucket Per User - -**What people do:** Create a new MinIO bucket for each registered user to enforce isolation. - -**Why it's wrong:** MinIO is not designed for millions of buckets. Bucket creation is a management operation. IAM policies per bucket become complex to manage at scale. - -**Do this instead:** Single bucket, key prefix isolation: `{user_id}/{document_id}/{filename}`. Enforce prefix scoping in `storage_service.py` — never let a user-supplied key escape their `{user_id}/` prefix. Verify in every `get_object` and `delete_object` call that the resolved key starts with the authenticated user's ID. - ---- - -## Integration Points - -### External Services - -| Service | Integration Pattern | Notes | -|---------|---------------------|-------| -| PostgreSQL | SQLAlchemy 2.0 async (`asyncpg` driver), sessions via `Depends(get_db)` | Use `asyncpg` pool, not per-request connections | -| MinIO | `minio` Python SDK (sync) wrapped in `asyncio.to_thread()`, or `aiobotocore` for async S3 | Presigned URL generation is CPU-bound, not I/O-bound — `to_thread` is fine | -| OneDrive | Microsoft Graph API via `httpx` async client + OAuth2 PKCE flow | Refresh tokens stored encrypted in `cloud_backends` | -| Google Drive | Google Drive API v3 via `httpx` or `google-auth` library | Same credential model as OneDrive | -| Nextcloud | WebDAV via `httpx` (PUT/GET/DELETE) or `webdavclient3` library | Basic auth or app password — simpler than OAuth | -| PyOTP | TOTP generation/verification (`pyotp.TOTP(secret).verify(code)`) | Time-window tolerance: default ±1 period (±30 sec) is sufficient | -| `python-jose` or `PyJWT` | JWT encode/decode | Use `HS256` with a 256-bit secret. `python-jose` has broader algorithm support; `PyJWT` is simpler and more actively maintained | -| `cryptography` (Fernet) | Cloud credential encryption/decryption | `Fernet.generate_key()` at setup; store in `CLOUD_CREDS_KEY` env var | -| `passlib[bcrypt]` | Password hashing | bcrypt work factor 12 minimum | - -### Internal Boundaries - -| Boundary | Communication | Notes | -|----------|---------------|-------| -| `api/` ↔ `services/` | Direct async function calls | Services never import from `api/`; dependency is one-directional | -| `services/` ↔ `storage/` | `StorageBackend` ABC interface | Services import from `storage/__init__.py` factory only | -| `services/` ↔ `ai/` | Existing `get_provider()` factory — unchanged | AI provider is still re-instantiated per call | -| `deps/` ↔ `services/` | Services can be called from deps (e.g., quota_service from quota dep) | Keep deps thin — prefer passing a DB session to the dep and calling service functions | -| `db/models.py` ↔ everywhere | Import models directly | No repository pattern needed at this scale; SQLAlchemy session + models is sufficient | - ---- - -## Scaling Considerations - -| Scale | Architecture | Notes | -|-------|-------------|-------| -| 1–100 users | Single FastAPI instance, `BackgroundTasks`, no queue | This milestone's target; simplest path | -| 100–10k users | Add Celery + Redis for background tasks; add PgBouncer; scale FastAPI to 2–4 instances | Background task queue is the first change needed | -| 10k–100k users | Read replica for PostgreSQL (document listing queries), MinIO multi-node cluster | Document metadata reads dominate; separate read/write paths | -| 100k+ users | Consider separate microservice for classification (GPU workers); CDN in front of MinIO presigned URLs | Classification latency becomes user-facing bottleneck | +| Change | Is the minimal solution also the proper one? | +|---|---| +| Backend decomposition | Yes. Sub-router via `__init__.py` aggregator is both minimal and the standard FastAPI pattern. | +| client.js decomposition | Yes. Re-export barrel means zero consumer churn while achieving internal separation. | +| Admin layout | Yes. Three-way `v-if` in App.vue + new layout file is 20 lines of change in existing files. | +| Icon refactor | The minimal solution (AppIcon.vue) is simpler than lucide but adds maintenance burden for the path map. lucide is more proper but adds a dependency. Call must be made by project owner. | --- ## Sources -- FastAPI official docs — Security / OAuth2 with JWT: https://fastapi.tiangolo.com/tutorial/security/oauth2-jwt/ (HIGH confidence — directly confirmed pattern) -- FastAPI official docs — Advanced Middleware: https://fastapi.tiangolo.com/advanced/middleware/ (HIGH confidence — confirms DI > middleware for auth) -- FastAPI official docs — SQL Databases: https://fastapi.tiangolo.com/tutorial/sql-databases/ (HIGH confidence — session-per-request via Depends confirmed) -- MinIO S3 presigned URL pattern: S3-compatible standard, documented in AWS S3 and MinIO docs (HIGH confidence — industry-standard pattern) -- PostgreSQL atomic UPDATE for quota enforcement: standard optimistic concurrency pattern (HIGH confidence) -- Fernet symmetric encryption (`cryptography` library): well-documented Python standard for symmetric key encryption (HIGH confidence) -- Refresh token rotation pattern: IETF OAuth 2.0 Security BCP (RFC 9700 / draft-ietf-oauth-security-topics) (HIGH confidence) - ---- -*Architecture research for: DocuVault multi-user SaaS document management (FastAPI + Vue 3 brownfield)* -*Researched: 2026-05-21* +- Inspected source: `backend/main.py`, `backend/api/admin.py`, `frontend/src/App.vue`, `frontend/src/router/index.js`, `frontend/src/layouts/AuthLayout.vue`, `frontend/src/api/client.js`, `frontend/src/views/AdminView.vue`, `frontend/src/components/layout/AppSidebar.vue` +- FastAPI sub-router `include_router` nesting: standard FastAPI router composition (HIGH confidence) +- Vue Router 4 `route.matched` for meta inheritance: Vue Router 4 documented behavior (HIGH confidence) +- lucide-vue-next: tree-shakeable Vue 3 icon library (MEDIUM confidence — verified as active, Tailwind-compatible) diff --git a/.planning/research/FEATURES.md b/.planning/research/FEATURES.md index 07792c6..ca3fd57 100644 --- a/.planning/research/FEATURES.md +++ b/.planning/research/FEATURES.md @@ -1,449 +1,257 @@ -# Feature Research +# Feature Landscape — v0.2 UI Overhaul -**Domain:** SaaS Document Management Platform (multi-user, quota-enforced, privacy-first) -**Researched:** 2026-05-21 -**Confidence:** MEDIUM (web fetch/search unavailable; based on training knowledge of Google Drive, OneDrive, Dropbox, Box, Notion, Paperless-ngx, DocuWare through Aug 2025 — all mature platforms with stable, well-documented feature sets) +**Domain:** Self-hosted SaaS document management (DocuVault) +**Researched:** 2026-06-07 +**Scope:** UX patterns and UI feature decisions for v0.2. Existing functional features (upload, extraction, sharing, cloud backends, auth) are not re-researched — only their UX surface. --- -## Research Scope +## Table Stakes -Platforms surveyed: Google Drive, Microsoft OneDrive, Dropbox, Notion, Box, Paperless-ngx (self-hosted), DocuWare (enterprise DMS). +Features users expect. Missing = product feels incomplete or broken. -Eight areas analyzed per the brief: -1. Auth & access control -2. Storage & quota UX -3. Folder/organization UX -4. Sharing model -5. Cloud storage integration UX -6. Admin panel -7. Audit / compliance -8. Document viewer features +| Feature | Why Expected | Complexity | Depends On | +|---------|--------------|------------|------------| +| Standalone admin layout (`/admin/*` routes with own sidebar) | AdminView.vue is currently a tab sheet bolted to the user layout. Admin and user contexts share no chrome. Nextcloud, Mayan EDMS, and every self-hosted SaaS treat admin as a distinct area. | Medium | Router nested routes, new `AdminLayout.vue` | +| Admin sidebar with discrete nav items (Users, Quotas, AI Config, Audit Log) | Each admin tab is already a full standalone component. Route-based nav replaces in-view tabs — enables deep linking, browser back button, and future per-section permissions. | Low | Vue Router `/admin/users`, `/admin/quotas`, `/admin/ai`, `/admin/audit` child routes | +| Empty states for every zero-content context | File list empty, folder empty, search no-results, shared-with-me empty, topics empty — each needs a distinct, actionable state. Current implementation is plain gray text. Zero-content states are the first thing new users see. | Medium | `StorageBrowser.vue`, `SharedView.vue`, `TopicsView.vue` | +| Loading skeletons for document list and sidebar tree | The document list (`StorageBrowser`) and sidebar folder tree currently render a plain "Loading…" text string. Skeletons reduce perceived load time by 30–50% on equivalent wait times per usability studies. The `QuotaBar` already uses `animate-pulse` — extend that pattern. | Medium | `StorageBrowser.vue`, `AppSidebar.vue`, new `SkeletonRow.vue` | +| Global search focus shortcut (`/` key) | Table stakes for document management tools. Paperless-ngx, Notion, GitHub, Linear all use `/` to focus search. Users who type `/` and nothing happens feel the app is unpolished. | Low | `SearchBar.vue`, `FileManagerView.vue` global `keydown` listener | +| Escape key closes modals and clears search | Universal browser convention. Escape should: close `ShareModal`, close `FolderDeleteModal`, close `DocumentPreviewModal`, clear the inline folder rename input, and clear search field. Several of these already handle `keydown.escape` in inputs but not at the modal overlay level. | Low | `ShareModal.vue`, `FolderDeleteModal.vue`, `DocumentPreviewModal.vue` | +| Sidebar collapses on mobile (hamburger toggle, overlay drawer) | On viewports below 1024px the fixed 256px sidebar eats 25–40% of the viewport. The current layout has no mobile handling — sidebar overlaps content on small screens. Minimum viable mobile: hamburger button + slide-in overlay drawer. | Medium | `AppSidebar.vue`, `App.vue`, new mobile toggle state | +| Responsive document list column hiding | `StorageBrowser` grid already uses `hidden md:block` for Size and `hidden sm:block` for Modified. This is correct — verify it works and document the pattern as intentional. Ensure the icon + name + actions columns always render. | Low | `StorageBrowser.vue` (verify existing classes) | +| Upload trigger keyboard shortcut (`U` key when no input is focused) | Secondary table stakes. Windows Explorer, Nextcloud, and Dropbox support pressing a key to trigger upload when focus is on the file list. Makes keyboard-first workflows viable. | Low | `DropZone.vue` or `FileManagerView.vue` global `keydown` listener | +| Visible drag-and-drop affordance on the drop zone at all times | `DropZone.vue` already exists and handles dragover/drop. The gap: the drop zone is a full-width dashed box inside the content area, which is correct, but there is no whole-page drag overlay when a user drags a file from the OS file manager. Users miss the target and the drop is ignored. | Medium | `FileManagerView.vue` document-level dragenter/dragleave listener | +| Contextual breadcrumb with admin section label | Admin subroutes need breadcrumbs: `Admin > Users`, `Admin > Audit Log`. Without them, admin pages feel unmoored. The user-facing `FolderBreadcrumb` is already built — admin can use the same component with static segments. | Low | `FolderBreadcrumb.vue`, `AdminLayout.vue` | --- -## Feature Landscape +## Differentiators -### Table Stakes (Users Expect These) +Features that add genuine value beyond the baseline. Not expected by most users, but valued when present. -Features users assume exist. Missing these = product feels incomplete or broken. - -#### 1. Auth & Access Control - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| Email + password registration with validation | Every SaaS has this | LOW | Strength rules (length, complexity) expected; breach-check (HaveIBeenPwned API) is a notable addition | -| Persistent sessions with "remember me" | Users hate logging in every visit | LOW | JWT with refresh token; sliding expiry. Pure stateless JWT without refresh feels cheap. | -| Password reset via email | Users forget passwords constantly | LOW | Time-limited signed token; mandatory | -| TOTP 2FA (authenticator app) | Expected at any security-conscious SaaS | MEDIUM | PyOTP / HOTP RFC 6238. Users expect QR code setup + backup codes. Missing backup codes = major UX gap. | -| Forced logout / session revocation | Power users and security-conscious users expect this | MEDIUM | "Sign out all devices" is table stakes at Box and Google. Requires server-side session tracking (defeats pure JWT — use a token revocation list or short-lived JWTs + refresh token table). | -| Per-user isolated data space | Every cloud storage product does this | LOW | Violated = catastrophic trust failure. MinIO prefix isolation per user ID. | -| Account deletion with data wipe | GDPR + user trust | MEDIUM | Must cascade: documents, quotas, shares, cloud credentials, audit entries scoped to user. | - -#### 2. Storage & Quota UX - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| Visible quota indicator (used / total) | Google Drive, Dropbox, OneDrive all show this prominently | LOW | Progress bar + "X MB of Y MB used" in sidebar or settings. Missing = users feel blind. | -| Quota check at upload time with clear error | Every platform rejects over-quota uploads | LOW | Error must say "You've used X of Y MB — free up space or upgrade." Not a generic 500. | -| Per-file size shown in file list | Dropbox, Drive show sizes in list view | LOW | Users make quota decisions based on file size visibility. | -| Sort/filter by size | Users cleaning up quota expect to find large files | LOW | "Largest files first" sort is a common quota-management action. | -| Quota warning at 80% and 95% | Drive warns at 80%; Dropbox emails at 90% | LOW | In-app banner + optionally email. Users are surprised when they hit the wall without warning. | -| Storage usage breakdown by folder/category | Drive shows breakdown by file type | MEDIUM | "X MB in Documents, Y MB in Images" helps users understand what to delete. Folder-level usage is lower priority but expected by power users. | -| Delete confirmation that shows freed space | Minor but Dropbox/Drive do this | LOW | "Deleting this file will free 24 MB." Reinforces value of the action. | - -#### 3. Folder / Organization UX - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| Create folder | Universal | LOW | Standard. | -| Rename folder | Universal | LOW | Inline rename (click-to-edit) is expected; modal is acceptable. | -| Delete folder (with contents confirmation) | Universal | LOW | Must warn if non-empty: "This will delete N documents." Soft-delete preferred; hard-delete on confirm. | -| Move document to folder (drag-and-drop or context menu) | Drive, Dropbox have both | MEDIUM | Drag-and-drop is a strong UX expectation in desktop-class web apps. Context menu "Move to..." is the fallback minimum. | -| Move folder into another folder (nesting) | Drive, OneDrive support arbitrary depth | MEDIUM | Users expect arbitrary depth. Cap at 5–8 levels deep to avoid pathological nesting; unlimited is fine for v1. | -| Breadcrumb navigation | Universal in file managers | LOW | Clickable breadcrumb showing full path: Root > Invoices > 2025. Without this users get lost in nested folders. | -| Sort documents within folder (name, date, size, type) | All platforms have this | LOW | Default: date descending (newest first). | -| "Recent" / "Last accessed" virtual folder | Drive, OneDrive surface this prominently | LOW | Not a real folder — a filtered view. Users navigating back to recent work expect it. | -| Search across all documents | Every platform has global search | MEDIUM | Full-text or metadata search. For DocuVault, text is already extracted — expose it in search. Missing = product feels like a filing cabinet with no index. | -| Multi-select for batch operations | Drive, Dropbox support shift-click + checkbox | MEDIUM | Batch delete, batch move. Without this, managing many files is tedious. | -| "Shared with me" virtual folder | Drive has this; Box has it; Dropbox has it | LOW | Auto-populated when another user shares a document. Already in PROJECT.md. | - -#### 4. Sharing Model - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| Share with named user (by handle or email) | Drive, Box, Dropbox all do this | MEDIUM | Core sharing primitive. Must show: who has access, their permission level, revoke button. | -| View-only vs. edit permission distinction | Box, Drive, Dropbox all distinguish view/edit | LOW | For DocuVault, edit means "re-upload / annotate" — since no in-app editing, view-only is the primary mode. Still must distinguish the concept for future extensibility. | -| Revoke share immediately | All platforms do this | LOW | No delay. If the user is currently viewing the document, their next request should 403. | -| Share notification to recipient | Drive/Dropbox send email; Box shows in-app | LOW | At minimum: in-app notification. Email is table stakes for platforms where users may not be logged in. | -| See list of "what I've shared" | Drive, Box surface this | LOW | Users need to audit their own shares. A "Shared by me" list per document + a global shares list in settings. | -| Accept / decline incoming share | Box requires acceptance; Drive shows silently | MEDIUM | DocuVault's "Shared with me" folder appearing is implicit acceptance. An explicit accept/decline adds trust. Drive skips it; Box requires it for external shares. Given privacy-first positioning, explicit accept is the better default. | -| Share expiry date | Box supports this; Drive (with Workspace) does; Dropbox Business does | MEDIUM | Not universal at consumer tier but expected in security-conscious / business contexts. | - -#### 5. Cloud Storage Integration UX - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| OAuth2 connect flow for Google Drive / OneDrive | Standard for these integrations | HIGH | Redirect-based OAuth2 PKCE. Users expect "Connect" → browser popup → granted → back to app. No credential entry. | -| Connection status indicator (connected / disconnected / error) | Any integration page shows this | LOW | Green dot = connected, red = error with message. Dropbox Business sync indicators are the model. | -| Disconnect / re-authenticate option | Users rotate tokens; credentials expire | LOW | "Disconnect" button. Re-auth if token expires (silent token refresh where possible). | -| Default storage selector | Users need to know where new uploads go | LOW | Clear radio/dropdown: "Local storage" vs "Google Drive" vs "OneDrive." Show current default prominently. | -| Upload routing confirmation | When cloud is default, users want to see it confirmed | LOW | "Stored in Google Drive" in upload success message. | -| Error state when cloud storage unreachable | Cloud services go down; credentials expire | MEDIUM | Graceful: queue locally, show warning, retry. Hard fail (lose document) = catastrophic. Minimum: "Storage unavailable — document saved locally." | -| Basic usage from cloud backend | Show how much cloud storage is used vs available | MEDIUM | Google Drive has 15 GB free; OneDrive 5 GB. Surfacing "You've used 3 GB of your 15 GB Google Drive" alongside local quota gives users a full picture. | - -#### 6. Admin Panel - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| User list with search and pagination | Every admin panel has this | LOW | Show: username, email, registration date, last login, quota used/total, active/inactive status. | -| Create user account | Admin needs to onboard users manually (invite flow) | LOW | Form: email, temp password (force reset on first login), quota assignment. | -| Deactivate / reactivate user | Admin must be able to remove access without deleting data | LOW | Soft disable: user cannot log in; data preserved. Separate from delete. | -| Force password reset | Standard admin action | LOW | Invalidates current sessions; user gets reset email. | -| Quota adjustment per user | Already in PROJECT.md | LOW | Input: new quota in MB. Show current usage. Prevent setting quota below current usage (warn, not block). | -| System-wide default quota setting | Admins want to change the free tier baseline | LOW | Global default that new users inherit. Existing users keep their individually-set quota. | -| AI provider/model assignment per user | Already in PROJECT.md | MEDIUM | Dropdown of configured providers + models. Show "using system default" until overridden. | -| System-wide AI provider default | Already in PROJECT.md | LOW | Global fallback when no user override exists. | -| Audit log viewer | All enterprise DMS products have this | MEDIUM | See Audit section below. | -| Platform health indicators | Admins need to know if things are broken | MEDIUM | Storage backend connectivity, database connection, queue depth. A simple status page at /admin/health. | - -#### 7. Audit / Compliance - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| Audit log of security events | GDPR, SOC2 baseline, enterprise buyers expect this | MEDIUM | Mandatory events: login, failed login, password reset, 2FA enable/disable, session revoke. | -| Audit log of data events | Same compliance baseline | MEDIUM | Mandatory events: document upload, document delete, document move/rename, share create, share revoke, quota change. | -| Metadata-only audit (no content) | Privacy requirement — DocuVault's stated constraint | LOW | Log: actor_user_id, action, target_resource_id, timestamp, IP address. Never log document content or filenames in the log (filename is sensitive metadata). | -| Admin-viewable audit log with filters | Already in PROJECT.md | MEDIUM | Filter by: user, action type, date range. Export to CSV is expected by compliance-oriented admins. | -| Immutable audit log | Audit logs lose their value if they can be edited | MEDIUM | Append-only table. No UPDATE/DELETE on audit rows. Admin UI shows only read operations. | -| IP address logging | Standard for login events; expected by security teams | LOW | Log IP on all auth events. GDPR note: IP is PII — include in retention policy. | -| Log retention policy | GDPR requires defined retention | LOW | Configurable in admin settings. Default: 1 year. Automated purge of older entries. | -| GDPR data export (user's own data) | GDPR Article 20 right to portability | HIGH | User can request export of all their data: document list + metadata (not necessarily content), account info, audit log of their own actions. Full content export is optional but the metadata export is required. | - -#### 8. Document Viewer Features - -| Feature | Why Expected | Complexity | Notes | -|---------|--------------|------------|-------| -| In-browser PDF preview | Drive, Dropbox, Box all have this | MEDIUM | Use browser's native PDF viewer or PDF.js. Without preview, users must download to read — major friction. | -| Document metadata panel | All DMS platforms show this | LOW | Show: upload date, file size, MIME type, AI-assigned topics, uploader, storage backend. | -| AI topics display | Core DocuVault feature | LOW | Show assigned topics with confidence if available. Allow manual topic override from document detail. | -| Download original file | Universal | LOW | Always available. Even if preview fails, download must work. | -| Document rename | Drive, Dropbox allow rename from detail view | LOW | Inline rename in detail view or list view. | -| Delete from detail view | Universal | LOW | With confirmation. Returns to parent folder. | -| "Shared with" indicator on document | Drive shows a "people" icon on shared items | LOW | Visual indicator in list view that document is shared. Click to see who. | -| Basic image preview (JPG, PNG) | All platforms preview images natively | LOW | Browser tag is sufficient. | -| Text file preview | Drive, Dropbox show .txt inline | LOW | Simple
 render of extracted text is adequate. |
+| Feature | Value Proposition | Complexity | Depends On |
+|---------|-------------------|------------|------------|
+| Whole-page drop zone (drag files anywhere on the screen to upload) | Removes the requirement to aim at the DropZone rectangle. Paperless-ngx supports this. Implementation: listen for `dragenter` at the document level, show a full-screen overlay with the drop target, then pipe the `drop` event into the existing upload emit chain. | Medium | `FileManagerView.vue`, new `PageDropOverlay.vue`, existing `DropZone` emit chain |
+| Skeleton placeholders shaped like the real content (5-column grid rows) | The QuotaBar has a simple pulse bar. The document list skeleton should be 5-column grid rows matching the actual layout — icon placeholder, name placeholder, size placeholder, date placeholder. This makes the transition from loading to content feel instantaneous rather than a layout jump. | Medium | New `SkeletonRow.vue`, `StorageBrowser.vue` |
+| Admin dashboard overview page (`/admin` root — stats at a glance) | User count, storage total across all users, recent audit log entries. Gives admins a reason to visit the admin panel rather than jumping straight to sub-sections. Self-hosted tools consistently provide this at the admin root. | Medium | New `AdminOverviewView.vue`, read-only aggregate API endpoints (may require backend work) |
+| Inline toast/notification system for upload completion and errors | Currently errors appear in the UploadProgress inline list and are never dismissed. A toast system enables transient success/error messages for any action (rename, delete, share) — not just upload. Constraint: keep it simple, not a full notification center. | Medium | New `ToastManager.vue` in `App.vue`, composable `useToast()` |
+| Document count badge polish on topic sidebar links | Already has `{{ topic.doc_count }}` in the sidebar. What's missing: show 0 counts as dimmed rather than hidden (users want to know the topic exists even if empty), and truncate to "9+" at double digits. | Low | `AppSidebar.vue` (minor change) |
+| Upload from keyboard shortcut `U` with accessible file picker trigger | The keyboard shortcut fires `triggerInput()` on DropZone. Accessibility requirement: this is equivalent to clicking the hidden file input — screen readers announce the file dialog opening. Already partially supported by `DropZone.vue`'s `triggerInput()` — wire up the global shortcut. | Low | `DropZone.vue`, `FileManagerView.vue` |
+| Tablet landscape layout: sidebar + content, no collapse needed | At 1024px (lg breakpoint), the sidebar is fully visible alongside content. This is the natural Tailwind `lg:flex` pattern. Verify it and make the breakpoint logic explicit in the layout wrapper. | Low | `App.vue` layout wrapper |
 
 ---
 
-### Differentiators (Competitive Advantage)
+## Anti-Features
 
-Features that align with DocuVault's core value (privacy-first, self-hosted, AI-classified).
+Features to explicitly NOT build in v0.2.
 
-| Feature | Value Proposition | Complexity | Notes |
-|---------|-------------------|------------|-------|
-| Privacy-first admin model (admin cannot read documents) | Unique trust proposition vs Google Drive/OneDrive where the operator can read everything | HIGH | Encryption-at-rest with user-scoped keys; admin queries explicitly exclude content. No other mainstream cloud platform offers this by default. Strong differentiator for privacy-conscious users. |
-| Bring-your-own cloud storage backend | Users keep their existing cloud storage; DocuVault is the intelligent layer on top | HIGH | Google Drive/OneDrive are the storage, DocuVault is the classifier/organizer. Removes the "I have to trust another cloud with my docs" objection. |
-| AI topic classification on upload (automatic) | No other mainstream DMS auto-classifies by topic on upload | MEDIUM | Paperless-ngx has auto-tagging; DocuWare has rules-based classification. AI-driven flexible topics is differentiated for non-enterprise users. |
-| Multiple AI provider support (local/private inference) | Users who want fully on-premises AI (Ollama) get privacy guarantee no cloud DMS offers | MEDIUM | Ollama / LMStudio means documents never leave the user's infrastructure. Strong appeal to privacy-first and regulated-industry users. |
-| Per-user topic customization on top of system defaults | Users get personalized classification without admin overhead | MEDIUM | System topics + per-user overrides. No mainstream cloud DMS supports per-user AI taxonomy customization. |
-| Share expiry with automatic revocation | Goes beyond basic sharing; prevents forgotten shares | MEDIUM | Dropbox Business and Box have this at paid tiers. Including it at v1 is differentiating for a free-tier self-hosted platform. |
-| Explicit share accept/decline | Recipient has control; aligns with privacy-first positioning | LOW | Box has this; Drive doesn't require it. Gives recipients agency. |
-| Storage backend per-document routing | Some documents go to Drive, others to local — user decides per-upload | HIGH | No mainstream platform does this. Users with mixed sensitivity needs can route sensitive docs to local and bulk docs to Drive. Complex to implement but uniquely valuable. |
-| In-app audit log visible to the user (not just admin) | Users can see their own activity history | LOW | GDPR-aligned; builds trust. Google Activity Dashboard is the model. Most self-hosted DMS don't surface this to users. |
+| Anti-Feature | Why Avoid | What to Do Instead |
+|--------------|-----------|-------------------|
+| Folder reorder by drag-and-drop (drag folder A above folder B to change sidebar order) | Requires a persistent `position` column in the DB, a backend migration, and drag state tracking across the sidebar tree. The sidebar is already collapsible. Payoff is marginal. | Keep alphabetical or creation-time ordering. Users can rename folders to impose their own order (e.g., "1. Contracts"). |
+| Reorderable sidebar sections (drag "Folders" above "Cloud Storage") | Same DB persistence problem. Section order is fixed and meaningful — user folders first, then external cloud. No self-hosted document manager reorders sections. | Fixed section order in `AppSidebar.vue`. |
+| Virtual scrolling for the document list | DocuVault is quota-capped at 100 MB per user. At typical document sizes (10–500 KB each), a user has at most a few hundred documents before hitting quota. Virtual scrolling adds significant complexity (dynamic row height, scroll restoration) for an edge case that the quota cap prevents. | Standard `v-for`. Add server-side pagination in a later milestone if needed. |
+| Rich text document annotations or in-app editing | Already scoped out in `PROJECT.md`. Mentioned here because UX redesigns often attract "what if we added..." scope creep. The preview modal shows PDF; that is the limit. | Stay read-only. Document content is immutable after upload. |
+| Dark mode toggle | Requires a systematic color token system across all components plus full testing of the inverted palette. The current light-mode palette is inconsistent — fix that first. Tailwind makes dark mode mechanically possible but the v0.2 payoff is low relative to scope. | Design tokens and CSS variables in v0.2 that would support dark mode in a future milestone. |
+| Drag-and-drop to reorder documents within a folder | Requires a persistent sort order column per user per folder. Documents are naturally sorted by creation date or name — both useful orderings. Custom ordering has high complexity and low frequency of use in a document vault. | Keep the existing sort controls (created_at, name, size). Server-side sort is already implemented. |
+| Full-screen modal for file upload | The DropZone is embedded in the file browser content area, which is the correct placement. A modal upload workflow adds navigation overhead with no benefit for a simple file picker. | The whole-page drag overlay (differentiator above) is the right scope. |
+| Notification center / inbox for async events | Building a polling-based notification center (classification done, quota warning) is a full feature, not UX polish. Celery classification runs in the background and its result is visible when the user next opens the document. | Toast on upload completion. Badge count on the document list is sufficient for v0.2. |
+| Multi-select with batch operations (select 10 files, delete all) | Requires a selection state layer across the file list, a selection-aware toolbar, and confirmed batch-delete UX. Medium-to-high complexity for a feature that is rarely needed in a personal document vault. | Address in a future milestone if user feedback identifies it as a gap. |
+| Drag documents from file list to sidebar folder entries | Cross-component drag state requires either a Pinia drag store or provide/inject. The sidebar `FolderTreeItem` and `StorageBrowser` have no shared parent that can coordinate state without significant refactoring. The "Move to folder" picker button already covers this use case. | The existing folder picker button in the file row actions menu. |
 
 ---
 
-### Anti-Features (Deliberately Excluded)
+## Admin Panel: Specific Pattern Decisions
 
-Features that seem good but would create problems for this project.
-
-| Feature | Why Requested | Why Problematic | Alternative |
-|---------|---------------|-----------------|-------------|
-| Public link sharing (unauthenticated) | Users want to share with people not on the platform | Creates a public attack surface; quota abuse; legal exposure if abused; hard to revoke at scale | Out of scope for v1 per PROJECT.md. Named-user sharing with accept/decline serves legitimate use cases. |
-| In-app document editing | Users want a full office suite | Massive scope expansion (collaborative editing = CRDT, OT, conflict resolution); not core to document management; every vendor locks you into their editor | "View and organize" is the value; editing stays in the originating app. |
-| Real-time collaborative editing (Google Docs model) | Feels modern | Requires WebSocket infrastructure, OT/CRDT algorithms, presence indicators — easily 3x the codebase complexity of everything else combined | Explicit non-goal; DocuVault stores files, not live documents. |
-| Mobile app (iOS/Android) | Users want mobile access | React Native or Flutter doubles the implementation surface; mobile OAuth2 and background sync are non-trivial | Responsive web app is the minimum. PWA capabilities (offline-capable via service worker) are a future v2 differentiator. |
-| SSO / SAML / OAuth enterprise federation | Enterprise buyers ask for it | Premature: adds Keycloak or similar dependency, requires session model changes, needs testing against multiple IdPs | TOTP 2FA first; SSO when subscription billing lands. Schema already designed for extension. |
-| Subscription billing in-platform | Users want self-service upgrades | Payment processing is a separate product (Stripe integration, dunning, invoicing, tax). Doing it half-way creates billing bugs that destroy trust | Quota model is billing-ready (limit_bytes column); add Stripe when subscription tier is validated. |
-| Soft-delete / Trash with restore | Users accidentally delete things | Reasonable feature, but complicates quota accounting (does trash count against quota?), storage lifecycle, and purge timing. Drive's 30-day trash has caused user confusion about "why is my quota still full?" | Clear confirmation dialog before hard delete is the v1 approach. Trash can be added in v1.x. |
-| Document version history | Users want to see old versions | Correct for collaborative tools; for a personal document store where users own their files, versioning explodes storage use. Paperless-ngx doesn't have it. DocuWare does but it's enterprise | Hash-based deduplication (don't store the same file twice) is a better v1 primitive. Version history in v2 for users who need it. |
-| AI-generated document summaries | Seems valuable given AI integration | Every summary requires an AI call → cost scales with uploads; summaries become stale when topics change; storing summaries consumes quota | AI is used for classification only. Summary generation can be a user-triggered on-demand action later. |
-| Admin impersonation / "log in as user" | Admins want to debug user issues | Directly contradicts the privacy-first core value. If admin can impersonate, admin can access user documents. Trust collapses. | Structured audit logs give admins enough to diagnose issues without impersonation. Document this as an explicit architectural decision. |
-
----
-
-## Feature Dependencies
+### Route structure
 
 ```
-[User Registration + Auth]
-    └──requires──> [Email verification] (optional at v1, needed for password reset)
-    └──requires──> [JWT session management]
-                       └──requires──> [Token revocation list] (for forced logout)
-
-[TOTP 2FA]
-    └──requires──> [Backup codes] (users get locked out without these)
-    └──requires──> [User Auth base]
-
-[Document Upload]
-    └──requires──> [User Auth]
-    └──requires──> [Quota enforcement]
-                       └──requires──> [Quota tracking (DB)]
-
-[Folder Structure]
-    └──requires──> [Document Upload]
-    └──enhances──> [Search] (search within folder scope)
-
-[Document Sharing]
-    └──requires──> [User accounts] (both sender and recipient)
-    └──requires──> [Folder Structure] ("Shared with me" virtual folder)
-    └──requires──> [Permission model in DB]
-
-[Share expiry]
-    └──requires──> [Document Sharing base]
-    └──requires──> [Background job / cron] (to revoke expired shares)
-
-[Cloud Storage Integration]
-    └──requires──> [User Auth] (OAuth2 tokens stored per user)
-    └──requires──> [Credential encryption] (Fernet / pgcrypto)
-    └──requires──> [StorageBackend adapter interface] (already planned)
-    └──enhances──> [Quota tracking] (cloud quota shown separately)
-
-[Per-document storage routing]
-    └──requires──> [Cloud Storage Integration]
-    └──requires──> [Multiple backends connected]
-
-[Admin Panel]
-    └──requires──> [User accounts]
-    └──requires──> [Quota DB model]
-    └──requires──> [Audit log]
-
-[Audit Log]
-    └──requires──> [User Auth] (actor_user_id)
-    └──requires──> [All logged operations exist]
-    └──conflicts──> [Admin impersonation] (impersonation breaks log integrity)
-
-[GDPR data export]
-    └──requires──> [Audit log]
-    └──requires──> [Document metadata model]
-    └──requires──> [Background export job] (large exports are async)
-
-[AI classification]
-    └──requires──> [Document upload + text extraction] (already exists)
-    └──requires──> [Topic model] (already exists)
-    └──enhances──> [Search] (topic-filtered search)
-
-[In-browser PDF preview]
-    └──requires──> [Document stored accessibly] (MinIO presigned URL or proxy endpoint)
-    └──independent of──> [Cloud storage] (preview proxied through app, not direct cloud URL)
+/admin                    → AdminLayout.vue wrapper (persistent sidebar + router-view)
+  /admin/                 → AdminOverviewView.vue (stats at a glance)
+  /admin/users            → AdminUsersView.vue (wraps AdminUsersTab.vue)
+  /admin/quotas           → AdminQuotasView.vue (wraps AdminQuotasTab.vue)
+  /admin/ai               → AdminAiConfigView.vue (wraps AdminAiConfigTab.vue)
+  /admin/audit            → AdminAuditView.vue (wraps AuditLogTab.vue)
 ```
 
-### Dependency Notes
+The existing tab components (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, `AdminAiConfigTab.vue`, `AuditLogTab.vue`) are promoted to route-level views. The `AdminLayout.vue` replaces `AdminView.vue`. The tab strip becomes a sidebar with router-link active styling. No logic changes inside the tab components — only the container changes. `AdminView.vue` is deleted after the migration.
 
-- **TOTP requires backup codes:** Without backup codes, users who lose their phone lose their account permanently. This is a support nightmare and a documented UX failure in many 2FA implementations.
-- **Forced logout requires server-side token tracking:** Pure stateless JWT cannot support "sign out all devices." A revocation list (Redis or DB table) or short-lived JWTs (15 min) + refresh token table is required. This is a non-trivial architecture decision that must be resolved before auth is implemented.
-- **Share expiry requires a background job:** Expiry cannot be purely DB-enforced. A cron job or Celery task must scan and revoke expired shares, or each share-check must evaluate expiry at access time (lazy revocation — simpler, acceptable for v1).
-- **Per-document storage routing is independent of the default storage selector:** Default storage is "which backend do new uploads go to by default?" Routing is "for this specific upload, override the default." Routing is a differentiator; default selector is table stakes.
-- **Admin impersonation conflicts with privacy-first model:** These two features are architecturally incompatible. Document this as an explicit decision in PROJECT.md.
-- **GDPR export is async:** Exporting all user data (documents + metadata) can take minutes for large accounts. Must be a background job with "email me when ready" — not a synchronous HTTP download.
+Vue Router 4 nested routes pattern (HIGH confidence — verified against official docs):
+
+```js
+{
+  path: '/admin',
+  component: AdminLayout,   // contains 
+  meta: { requiresAdmin: true },
+  children: [
+    { path: '', component: AdminOverviewView },
+    { path: 'users', component: AdminUsersView },
+    { path: 'quotas', component: AdminQuotasView },
+    { path: 'ai', component: AdminAiConfigView },
+    { path: 'audit', component: AdminAuditView },
+  ],
+}
+```
+
+The `requiresAdmin` guard moves from the child route to the parent. All children inherit it automatically.
+
+### Admin sidebar nav items (in order)
+1. Overview
+2. Users
+3. Quotas
+4. AI Config
+5. Audit Log
+
+A "Back to app" link at the bottom of the admin sidebar returns to `/` — admin should not feel like a dead end.
+
+### Admin sidebar does NOT share code with `AppSidebar.vue`
+
+`AdminSidebar.vue` is a separate component. It has no folder tree, no cloud tree, no topics list, no quota bar. Those belong to the user interface. Sharing the sidebar component would create coupling between two logically separate contexts and conflict with the "thin view + smart component" architecture described in CLAUDE.md.
+
+### Breadcrumb
+
+Use the existing `FolderBreadcrumb.vue` or a simpler static variant: `Admin > [Section Name]`. No dynamic nesting required. The admin sidebar's active link already signals location; the breadcrumb adds skimmability for users who navigate via URL.
 
 ---
 
-## MVP Definition (This Milestone)
+## Empty State Patterns: Specific Decisions
 
-This is a subsequent milestone — the core document + AI system exists. MVP for this milestone is everything in PROJECT.md Active section, implemented in a way that doesn't create rework.
+Each zero-content context gets a distinct implementation. Generic "No items" text is the anti-pattern.
 
-### Must Ship (v1 — this milestone)
+| Context | Headline | Subtext | CTA |
+|---------|----------|---------|-----|
+| Root folder (no folders yet) | "Start organizing your documents" | "Create a folder to group related files together." | "New folder" button |
+| Folder is empty (exists but has no contents) | "This folder is empty" | "Upload files here or move documents into this folder." | "Upload files" link (triggers DropZone click) |
+| Search returns no results | "No matches for '{query}'" | "Try a different search term or clear your search." | "Clear search" button |
+| Shared with me (no shares yet) | "Nothing shared with you yet" | "When someone shares a document with your handle, it appears here." | None (passive state) |
+| Topics list empty | "No topics yet" | "Topics are created automatically when you upload and classify a document." | None |
+| Audit log (admin, no entries yet) | "No audit events recorded" | "Events are recorded when users log in, upload documents, or make changes." | None |
+| Cloud storage — no connections | "No cloud storage connected" | "Connect OneDrive, Google Drive, Nextcloud, or WebDAV in Settings." | "Open Settings" link |
 
-- [ ] Email + password registration with strength validation — table stakes, nothing works without it
-- [ ] JWT sessions with refresh tokens + forced logout capability — security baseline
-- [ ] TOTP 2FA with backup codes — PROJECT.md requires it; backup codes prevent lockouts
-- [ ] Password reset via email — users will forget passwords
-- [ ] Per-user isolated storage (MinIO prefix per user ID) — without this, multi-user is unsafe
-- [ ] Quota tracking (DB) + enforcement at upload + visible indicator — table stakes for any quota system
-- [ ] Quota warnings at 80% and 95% — prevents users hitting the wall blindly
-- [ ] Create / rename / delete folders with breadcrumb navigation — table stakes file management
-- [ ] Move document to folder (context menu minimum; drag-drop in v1.x) — without this folder structure is useless
-- [ ] Sort by name/date/size — basic list management
-- [ ] Global search (metadata + extracted text) — without this the product is a dumb file cabinet
-- [ ] Share document with named user by handle (view-only default) — core sharing primitive
-- [ ] Revoke share with immediate effect — security; users must be able to undo shares
-- [ ] "Shared with me" virtual folder — PROJECT.md requires it
-- [ ] Cloud storage connect flow (Google Drive + OneDrive OAuth2) — PROJECT.md requires it
-- [ ] Connection status indicator + error state with local fallback — users need to know if cloud is broken
-- [ ] Default storage backend selector — users need to control where uploads go
-- [ ] Credential encryption (Fernet + env key) — PROJECT.md hard requirement
-- [ ] Admin: user list, create, deactivate, quota adjustment — PROJECT.md requires it
-- [ ] Admin: AI provider/model assignment per user + system default — PROJECT.md requires it
-- [ ] Audit log (append-only): auth events + data events — PROJECT.md requires it
-- [ ] Admin audit log viewer with date/user/action filters — PROJECT.md requires it
-- [ ] In-browser PDF preview (PDF.js or native) — without preview, users must download to read
-- [ ] Document metadata panel (size, date, topics, storage backend) — contextual information users expect
+Rule: every empty state that the user can resolve with an action gets exactly one CTA. Empty states that are passive (shared-with-me before anyone shares, topics before first upload, audit before any events) get explanatory subtext only — no CTA that leads nowhere useful.
 
-### Add After Core Validated (v1.x)
-
-- [ ] Share expiry date — valuable for security-conscious users; requires background job
-- [ ] Explicit share accept/decline — adds trust; low complexity
-- [ ] "What I've shared" global list in settings — users need to audit their shares
-- [ ] In-app share notification — without email infrastructure, in-app is the fallback
-- [ ] Storage usage breakdown by folder — power user feature; quota bar is sufficient at v1
-- [ ] CSV export of audit log — compliance teams want this; not day-one critical
-- [ ] Log retention policy configuration — needed for GDPR compliance; default 1yr is fine for v1
-- [ ] Drag-and-drop document move — UX polish; context menu is the functional minimum
-- [ ] "Recent documents" virtual view — convenience; not blocking
-
-### Future (v2+)
-
-- [ ] GDPR data export (Article 20) — required eventually; complex (async job); defer until user base exists
-- [ ] Per-document storage routing (override default per upload) — strong differentiator but complex; v2
-- [ ] User-facing activity log (own audit trail) — GDPR-aligned trust feature; v2
-- [ ] Soft-delete / Trash with restore — adds quota accounting complexity; solve properly in v2
-- [ ] Document version history — storage-intensive; needs a clear retention policy first
-- [ ] SSO / SAML — per PROJECT.md, after subscription billing
-- [ ] Public link sharing — per PROJECT.md, explicitly out of scope for v1
-- [ ] Platform health dashboard at /admin/health — operational convenience; Docker healthchecks handle v1
+Implementation: extract a shared `EmptyState.vue` presentational component that accepts `headline`, `subtext`, and an optional `action` slot. Use it in all seven contexts above instead of duplicating the layout.
 
 ---
 
-## Feature Prioritization Matrix
+## Loading State Patterns: Specific Decisions
 
-| Feature | User Value | Implementation Cost | Priority |
-|---------|------------|---------------------|----------|
-| Per-user auth (registration, login, JWT) | HIGH | LOW | P1 |
-| TOTP 2FA + backup codes | HIGH | MEDIUM | P1 |
-| Password reset | HIGH | LOW | P1 |
-| Quota tracking + enforcement + indicator | HIGH | LOW | P1 |
-| Quota warnings (80%, 95%) | HIGH | LOW | P1 |
-| Folder CRUD + breadcrumb | HIGH | LOW | P1 |
-| Move document to folder | HIGH | LOW | P1 |
-| Global search | HIGH | MEDIUM | P1 |
-| Share with named user (view-only) | HIGH | MEDIUM | P1 |
-| Revoke share | HIGH | LOW | P1 |
-| "Shared with me" folder | HIGH | LOW | P1 |
-| Cloud connect flow (OAuth2) | HIGH | HIGH | P1 |
-| Connection status + error state + local fallback | HIGH | MEDIUM | P1 |
-| Default storage selector | HIGH | LOW | P1 |
-| Credential encryption | HIGH | MEDIUM | P1 |
-| Admin user management (list/create/deactivate/quota) | HIGH | LOW | P1 |
-| Admin AI config | MEDIUM | LOW | P1 |
-| Audit log (append-only) | HIGH | MEDIUM | P1 |
-| Admin audit log viewer | MEDIUM | MEDIUM | P1 |
-| In-browser PDF preview | HIGH | MEDIUM | P1 |
-| Document metadata panel | MEDIUM | LOW | P1 |
-| Forced logout / session revocation | MEDIUM | MEDIUM | P2 |
-| Share expiry date | MEDIUM | MEDIUM | P2 |
-| Explicit share accept/decline | MEDIUM | LOW | P2 |
-| "What I've shared" list | MEDIUM | LOW | P2 |
-| Storage usage breakdown by folder | MEDIUM | MEDIUM | P2 |
-| CSV export of audit log | MEDIUM | LOW | P2 |
-| Log retention policy | MEDIUM | LOW | P2 |
-| Drag-and-drop move | MEDIUM | MEDIUM | P2 |
-| Recent documents view | LOW | LOW | P2 |
-| Per-document storage routing | HIGH | HIGH | P3 |
-| GDPR data export | MEDIUM | HIGH | P3 |
-| Soft-delete / Trash | MEDIUM | HIGH | P3 |
-| User-facing activity log | MEDIUM | LOW | P3 |
-| Document version history | LOW | HIGH | P3 |
-| Platform health dashboard | LOW | MEDIUM | P3 |
+| Component | Current | Target | Rationale |
+|-----------|---------|--------|-----------|
+| Document list (`StorageBrowser`) | `
Loading…
` text | 6–8 skeleton rows (5-col grid, `animate-pulse`) | List loads on every folder navigation. Skeletons match the grid layout exactly, preventing layout jump. | +| Sidebar folder tree | `
Loading…
` text | 3 skeleton nav items (indent + placeholder bar) | Sidebar loads once on mount. Short skeleton is sufficient. | +| Sidebar topics list | `
Loading…
` text | 4 skeleton topic pills (color dot + bar) | Same rationale as folder tree. | +| Admin user table | No loading state | Skeleton table rows (3–5 rows, matching column count) | Admin tables are data-heavy; skeleton preserves table chrome during load. | +| Audit log table | No loading state | Skeleton rows matching timestamp/user/action columns | Same. | +| Upload progress item | Spinner + progress bar (correct) | Keep as-is | Upload is a blocking action, not a content load. Spinners are correct for actions. | +| QuotaBar | `animate-pulse` bar (correct) | No change | Already implemented correctly. Document as the canonical skeleton pattern. | +| Document preview modal | None | Spinner centered in modal body | PDF proxy fetch is a single-request action. Spinner is correct. | + +Rule: if the loading UI represents content the user will read (list, table, tree), use a skeleton. If the loading UI represents a system action (save, upload, submit, process), use a spinner. Never show a loading state that will be visible for less than 300ms — add a minimum display delay. --- -## Competitor Feature Analysis +## Drag-and-Drop: Scope Boundaries -| Feature | Google Drive | Dropbox | Box | Paperless-ngx | DocuVault Plan | -|---------|--------------|---------|-----|----------------|----------------| -| Share by named user | Yes (email) | Yes (email) | Yes (email + internal) | No (single-user) | Yes (by handle) | -| Share permission levels | View / Comment / Edit | View / Edit | View / Edit / Co-owner | N/A | View-only v1; edit later | -| Share expiry | Google Workspace only | Business+ only | Business+ free tier | N/A | Include in v1.x — differentiator at free tier | -| Public link sharing | Yes | Yes | Yes | No | Explicitly excluded v1 | -| Quota indicator | Yes (prominent) | Yes | Yes | N/A (local disk) | Yes — progress bar + text | -| Quota warnings | Yes (email at 80%) | Yes (email at 90%) | Yes | N/A | In-app banner at 80%, 95% | -| Folder organization | Yes (arbitrary depth) | Yes | Yes | Yes | Yes (arbitrary depth) | -| Drag-and-drop move | Yes | Yes | Yes | Yes | v1.x (context menu at v1) | -| Global search | Yes (full-text) | Yes (full-text) | Yes (full-text) | Yes (extracted text) | Yes (extracted text — already extracted) | -| AI classification | No (manual labels) | No | No | Basic rules-based | Yes — core differentiator | -| Audit log (admin) | Google Workspace | Business+ | All tiers | No | Yes — all tiers | -| TOTP 2FA | Yes | Yes | Yes | Yes | Yes | -| Backup codes | Yes | Yes | Yes | Yes | Yes — required with TOTP | -| Bring-your-own storage | No | No | No | Partially (local FS) | Yes — core differentiator | -| Privacy-first admin model | No (Google can read) | No | No | N/A (self-hosted) | Yes — core differentiator | -| In-browser PDF preview | Yes | Yes | Yes | Yes | Yes (PDF.js) | -| Document version history | Yes | Yes (30 days free) | Yes (30 days free) | No | v2 | -| GDPR data export | Yes (Google Takeout) | Yes | Yes | N/A | v2 | +### Build in v0.2 + +1. **Whole-page drop overlay for file upload** — listen for `dragenter` at the document level, show a full-screen overlay, delegate the `drop` event to the existing DropZone emit chain. The drop must only activate when files are being dragged (check `e.dataTransfer.types.includes('Files')`). + +2. **Move a document to a folder by dragging the file row onto a folder row** — already implemented in `StorageBrowser.vue`. Verify it functions correctly and polish the drop highlight: `ring-2 ring-inset ring-amber-300` on the folder row is correct. Ensure the file row shows reduced opacity during drag (`opacity-50` already applied). + +### Do Not Build in v0.2 + +- Dragging files from the file list onto sidebar folder entries (requires cross-component drag state via Pinia or provide/inject — the "Move to folder" picker already covers this) +- Touch drag-and-drop on mobile (HTML5 drag events do not fire on iOS Safari without a polyfill library; defer to the move picker button) +- Drag folders to reorder them + +### Required for every drag interaction + +- Visual affordance on dragover (already done on DropZone; add to whole-page overlay) +- Non-drag alternative for every drag action: the "Move to folder" picker button covers the drag-to-move case +- Drop must be clearly confirmed: file row disappears from source location and appears in target on success --- -## Critical UX Patterns to Follow +## Keyboard Shortcuts: Final List -### Quota UX (from Drive / Dropbox study) +| Shortcut | Action | Condition | +|----------|--------|-----------| +| `/` | Focus search bar | Not in an input, textarea, or contenteditable | +| `Escape` | Close modal or clear active inline input | Modal open, or search focused with content | +| `U` | Trigger file picker (upload) | Not in any input, on a folder/file manager view | +| `N` | Start new folder inline input | Not in any input, on a folder/file manager view | -**Sidebar quota bar pattern** (Drive): Always-visible storage indicator at the bottom of the left sidebar. Shows "X.X MB of Y MB used" with a color-coded bar (green → yellow → red as quota fills). This is the pattern users are conditioned to expect. +### Do Not Build -**Upload rejection UX**: Never show a generic error. Show: current usage, quota limit, how much the rejected file would have added, and a direct link to storage settings. Dropbox does this well; many self-hosted tools fail here. - -**Quota warning banner**: Non-modal, dismissable, but persistent. Appears at 80% — amber. At 95% — red. At 100% — blocks upload with inline error (not just a banner). - -### Folder UX (from Drive / Explorer pattern) - -**Breadcrumb is mandatory**: Any folder deeper than one level without a breadcrumb creates navigation confusion. Users instinctively hit "back" in their browser and are surprised to leave the app. - -**Empty folder state**: Show a clear empty state ("This folder has no documents yet. Upload one to get started.") — not a blank white space. - -**Delete folder confirmation must list contents**: "Are you sure? This will permanently delete 47 documents." — not just a generic "are you sure?" - -### Sharing UX (from Box / Drive) - -**Share dialog anti-pattern to avoid**: Auto-sending share notifications without preview. Drive's "A notification will be sent" with an optional message is the correct pattern — user controls whether to notify. - -**Shared document visual indicator**: An icon overlay (people icon) in list view on shared items. Without this, users lose track of what they've shared. Drive and Box both do this. - -**Revoke is immediate and feedback is given**: "Access revoked for user@handle" toast. Not silent. - -### Cloud Integration UX (from Dropbox Business / Google Drive) - -**Connection health must be persistent**: Don't only show errors at connect time. Show ongoing status in storage settings. A disconnected cloud backend that silently fails uploads is a data-loss scenario. - -**Token expiry is silent without handling**: OAuth2 tokens expire. If the app doesn't handle refresh silently (or at least alert the user clearly), users will think their uploads succeeded when they didn't. **This is a critical pitfall.** +- `J`/`K` for row-by-row navigation — requires focus management, ARIA selected state, and keyboard event bubbling control. Over-engineered for a vault where opening a document is secondary to browsing and uploading. +- `Ctrl+A` / `Cmd+A` for select all — requires multi-select, which is an anti-feature. +- Arrow key tree navigation in the sidebar — screen readers handle this natively; custom arrow-key navigation would conflict with browser defaults. +- `?` shortcut for a shortcut reference modal — too few shortcuts to warrant a help overlay. --- -## What We Might Miss (Gap Analysis) +## Responsive Layout: Breakpoint Decisions -Items that competitors have which aren't in PROJECT.md Active requirements and are easy to overlook: +| Viewport | Tailwind breakpoint | Sidebar | Document list | Admin | +|----------|---------------------|---------|--------------|-------| +| Mobile portrait | `< 640px` (below `sm`) | Hidden; hamburger button + overlay drawer | Icon + name only; Size (`hidden md:block`) and Modified (`hidden sm:block`) hidden | Same drawer | +| Mobile landscape / small tablet | `640–767px` (`sm`) | Hidden; hamburger still active | Icon + name + Modified visible; Size hidden | Same | +| Tablet portrait | `768–1023px` (`md`) | Hidden; hamburger still active | Full 5-col grid | Full columns | +| Tablet landscape / small desktop | `1024px+` (`lg`) | Always visible (256px fixed) | Full 5-col grid | Full admin sidebar + content | +| Desktop | `1280px+` (`xl`) | Always visible | Full grid + comfortable padding | Full layout | -1. **Backup codes for TOTP** — PROJECT.md mentions TOTP but not backup codes. Without backup codes, a lost phone = permanently locked-out account. Every 2FA implementation must include these. HIGH severity omission. +Key decision: sidebar collapses at the `lg` breakpoint (1024px), using `hidden lg:flex` on the sidebar and `lg:hidden` on the hamburger button. At 1024px the sidebar (256px) leaves 768px for content — sufficient for the 5-column grid with room for padding. -2. **Quota warning thresholds** — PROJECT.md says "quota enforced; uploads rejected." It doesn't mention pre-emptive warnings at 80%/95%. Users who hit the wall without warning give negative reviews. Easy to implement; easy to forget. +Touch targets: all inline action icon buttons are currently `p-1.5` (approximately 30px). Increase to `p-2` on `sm:` and below using responsive padding classes. 44px is the WCAG target — not achievable for inline buttons without breaking the grid layout, but 36px is a reasonable minimum. -3. **Session revocation / forced logout** — Not in PROJECT.md. JWT-based auth has no built-in revocation. If a user believes their account is compromised, they need "sign out everywhere." Requires either short-lived JWTs + refresh token table, or a server-side revocation list. +--- -4. **Breadcrumb navigation** — Easy to forget in folder implementation. The folder CRUD is in PROJECT.md but the navigation UX isn't. Without breadcrumbs, nested folders become unusable. +## Feature Dependencies (summary) -5. **"What I've shared" list** — PROJECT.md covers sharing mechanics but doesn't cover share auditability from the sharer's perspective. Users who've shared many documents need a way to manage them all. +``` +Admin route subtree + → AdminLayout.vue (new) contains AdminSidebar.vue (new) + + → Child views wrap existing AdminUsersTab, AdminQuotasTab, AdminAiConfigTab, AuditLogTab + → AdminView.vue deleted after migration -6. **Upload error when cloud backend is unreachable** — PROJECT.md says "documents stored in cloud backend accessed via app." What happens if the cloud backend is down at upload time? Needs explicit handling: local fallback with a flag, or queue with retry, or reject with explanation. Silence = data loss. +Empty states + → New EmptyState.vue (presentational, accepts headline/subtext/action slot) + → Used in StorageBrowser.vue, SharedView.vue, TopicsView.vue, CloudStorageView.vue, AuditLogTab.vue -7. **MinIO presigned URL vs proxy for document access** — Not a feature gap but an architecture gap that affects features: if documents in cloud backends are accessed via presigned URLs, the URL leaks the storage path. If proxied through the app, privacy is preserved but adds load to the backend. For a privacy-first platform, proxy is the correct choice — but it must be a conscious decision before the cloud integration is built. +Skeleton rows + → New SkeletonRow.vue (5-col grid, animate-pulse) + → Used in StorageBrowser.vue (document list), AppSidebar.vue (folder tree, topics list) -8. **Empty folder state and confirmation dialogs** — Micro-UX that competitors all get right. Easy to skip in implementation. Users notice. +Keyboard shortcuts + → Global keydown listener in FileManagerView.vue + → SearchBar.vue exposes focus() method + → DropZone.vue exposes triggerInput() (already exists as triggerInput()) + → StorageBrowser.vue exposes startNewFolder() (already defineExposed) -9. **Shared document icon in list view** — Small visual indicator but prevents users from losing track of what they've shared. Three lines of CSS; easy to miss in sprint planning. +Mobile sidebar + → App.vue holds sidebarOpen boolean state + → AppSidebar.vue accepts isOpen prop + emits close + → Hamburger button in App.vue header (mobile only, hidden lg:hidden) -10. **Admin cannot set quota below current usage without warning** — Obvious in hindsight but easy to leave as a silent truncation. Must warn and require explicit confirmation. +Whole-page drop overlay + → New PageDropOverlay.vue (full-screen overlay, accepts active prop) + → FileManagerView.vue listens for document dragenter/dragleave + → On drop, delegates to existing upload flow via onFilesSelected +``` --- ## Sources -- Google Drive feature set: training knowledge through Aug 2025 (stable, well-documented product) -- Microsoft OneDrive feature set: training knowledge through Aug 2025 -- Dropbox feature set: training knowledge through Aug 2025 -- Box feature set: training knowledge through Aug 2025 (enterprise DMS reference) -- Notion: training knowledge through Aug 2025 (organization patterns) -- Paperless-ngx: training knowledge through Aug 2025 (self-hosted DMS reference — open source, well-documented) -- DocuWare: training knowledge through Aug 2025 (enterprise DMS reference) -- Web fetch and web search unavailable during this research session; confidence reduced to MEDIUM for specific UI detail claims; HIGH for feature existence claims (these are all mature, stable platforms) - ---- -*Feature research for: DocuVault — SaaS Document Management Platform* -*Researched: 2026-05-21* +- Vue Router 4 nested routes: https://router.vuejs.org/guide/essentials/nested-routes +- Skeleton screens vs. spinners UX: https://www.onething.design/post/skeleton-screens-vs-loading-spinners +- Skeleton loading screen design: https://blog.logrocket.com/ux-design/skeleton-loading-screen-design/ +- SaaS empty state patterns: https://pixxen.com/saas-empty-state-design/ +- IBM Carbon Design System empty states: https://carbondesignsystem.com/patterns/empty-states-pattern/ +- Drag-and-drop SaaS UX: https://www.eleken.co/blog-posts/drag-and-drop-ui +- Drag-and-drop NN/g guidelines: https://www.nngroup.com/articles/drag-drop/ +- Responsive SaaS breakpoints: https://tryhoverify.com/blog/responsive-design-breakpoints-2024-guide/ +- Keyboard shortcut design for web apps: https://sashika.medium.com/j-k-or-how-to-choose-keyboard-shortcuts-for-web-applications-a7c3b7b408ee +- Nextcloud layout patterns: https://docs.nextcloud.com/server/latest/developer_manual/design/layout.html +- Paperless-ngx frontend architecture: https://deepwiki.com/paperless-ngx/paperless-ngx/2-frontend-architecture diff --git a/.planning/research/PITFALLS.md b/.planning/research/PITFALLS.md index 3ad6e18..85f10e7 100644 --- a/.planning/research/PITFALLS.md +++ b/.planning/research/PITFALLS.md @@ -1,550 +1,290 @@ -# Pitfalls Research +# Domain Pitfalls — v0.2 UI Overhaul and Optimization -**Domain:** Multi-user SaaS document management platform (FastAPI + Vue 3, PostgreSQL + MinIO, cloud storage integrations) -**Researched:** 2026-05-21 -**Confidence:** HIGH (auth/storage/multi-tenancy patterns are well-established; specific FastAPI + MinIO combination is MEDIUM — no web search available) +**Project:** DocuVault +**Milestone:** v0.2 — UI Overhaul and Optimization on existing FastAPI + Vue 3 app +**Researched:** 2026-06-07 +**Confidence:** HIGH (all pitfalls grounded in actual source files: router/index.js, App.vue, main.py, client.js, StorageBrowser.vue, DocumentCard.vue, DropZone.vue, AppSidebar.vue, and all api/*.py router files) --- ## Critical Pitfalls -### Pitfall 1: JWT in localStorage — XSS Gives Full Account Takeover +Mistakes that cause regressions, security gaps, or rewrites. + +--- + +### Pitfall 1: FastAPI Router Splitting — Prefix Already Embedded in APIRouter, Not in include_router + +**Phase:** Backend refactoring (split large router files) + +**What goes wrong:** Every existing router in this codebase declares its prefix on the `APIRouter` instance itself — `router = APIRouter(prefix="/api/documents", tags=["documents"])` — NOT on the `include_router(router, prefix=...)` call in `main.py`. When splitting a large file (e.g. `documents.py` at 852 lines, `admin.py` at 934 lines) into sub-modules, the new sub-routers pick up their endpoints via an `include_router` call inside the original module, or they are wired directly in `main.py`. Either way, if you add a prefix on `include_router` in addition to the prefix on the inner `APIRouter`, all URLs double-prefix to `/api/documents/api/documents/...`. Likewise, if you strip the prefix from the inner router and forget to add it to `include_router`, all URLs return 404. + +**Why it happens:** The original pattern (`router = APIRouter(prefix=...)` in the module + `app.include_router(router)` without a prefix in `main.py`) is unusual but consistent in this project. Developers splitting a file expect the standard pattern where prefixes go on `include_router`, not on the router definition. + +**Consequences:** 404 on all endpoints in the split module; existing tests and the frontend `client.js` (which hard-codes `/api/documents`, `/api/admin`, etc.) silently break. + +**Prevention:** When splitting, create sub-routers with NO prefix: `sub = APIRouter(tags=["documents-upload"])`. Register them inside the parent module with `router.include_router(sub)` so the parent's prefix propagates. The parent module keeps its existing prefix and its existing `app.include_router(parent)` call in `main.py` is unchanged. + +**Detection warning signs:** Any test that calls `/api/documents/*` returns 404 immediately after the split; Swagger UI shows doubled path segments. + +--- + +### Pitfall 2: FastAPI Router Splitting — Circular Import via Shared Dependencies + +**Phase:** Backend refactoring + +**What goes wrong:** `folders.py` already exports a `document_move_router = APIRouter(prefix="/api/documents")` — a cross-cutting concern documented in the file (line 439-442). When splitting `documents.py`, if the new sub-module imports anything from `api/folders.py` (e.g. a shared validator or Pydantic schema), Python raises `ImportError: cannot import name X` at startup because both modules are in partial initialization state. + +**Why it happens:** FastAPI modules that share models and validators tend to grow cross-references. The `document_move_router` in `folders.py` under the `/api/documents` prefix is the existing example — any new import in the opposite direction closes the import cycle. + +**Consequences:** Application fails to start. Docker container exits immediately. + +**Prevention:** Shared Pydantic models and validators must live in `db/models.py` or a dedicated `schemas/` module — not in router files. Before splitting, grep for any `from api.X import` inside the router files being split and move the shared symbols to a neutral module first. + +**Detection warning signs:** `ImportError` or `cannot import name` in the uvicorn startup log. + +--- + +### Pitfall 3: AdminLayout Vue Router Integration — beforeEach Guard Misses /admin/* Children + +**Phase:** Admin panel rearchitecture + +**What goes wrong:** The current guard in `router/index.js` (lines 81-93) checks `to.meta.requiresAdmin` on the matched route object. The current `/admin` route is a flat route with that meta set directly. When `/admin` becomes a parent route with nested children (`/admin/users`, `/admin/audit`, etc.), the children do NOT automatically inherit `meta: { requiresAdmin: true }` from the parent in Vue Router 4 unless the guard iterates `to.matched`. If the children are added without repeating the meta, the guard's `to.meta.requiresAdmin` check evaluates to `undefined` (falsy) for every child route — non-admin users can navigate to `/admin/users` directly. + +**Why it happens:** Vue Router 4 does NOT merge parent `meta` into child `meta` by default. `to.meta` reflects only the deepest matched route's own meta, not the parent's. + +**Consequences:** Security regression — the admin guard that currently works correctly stops protecting all admin child routes. Non-admin users can access `/admin/users`, `/admin/audit`, etc. + +**Prevention:** Update the guard to check `to.matched.some(r => r.meta.requiresAdmin)` instead of `to.meta.requiresAdmin`. This is the correct Vue Router 4 idiom for inherited meta and requires one change to the guard. Apply the same fix to the `!to.meta.public` check if any auth routes gain children. + +```javascript +// router/index.js — change: +if (to.meta.requiresAdmin && authStore.user?.role !== 'admin') { +// to: +if (to.matched.some(r => r.meta.requiresAdmin) && authStore.user?.role !== 'admin') { +``` + +--- + +### Pitfall 4: AdminLayout Integration — App.vue Layout Switch Breaks for /admin/* Routes + +**Phase:** Admin panel rearchitecture + +**What goes wrong:** `App.vue` currently switches between `AuthLayout` (when `route.meta.layout === 'auth'`) and the main app shell (sidebar + ``) for everything else. The admin panel needs a third layout (`AdminLayout`) with its own nav. The naive approach — `v-else-if="route.meta.layout === 'admin'"` in `App.vue` rendering `AdminLayout` which then renders `` — creates double rendering: `App.vue`'s `` already resolved the full route. The inner `` inside `AdminLayout` gets nothing because the route is already consumed. + +**Consequences:** Admin layout renders with an empty content area, or the admin page renders without the admin chrome. + +**Prevention:** The correct structure for a standalone admin layout in Vue Router 4: + +```javascript +// router/index.js +{ + path: '/admin', + component: AdminLayout, // resolves to the layout shell, which contains + meta: { requiresAdmin: true }, + children: [ + { path: '', redirect: '/admin/users' }, + { path: 'users', component: AdminUsersView }, + { path: 'quotas', component: AdminQuotasView }, + { path: 'ai', component: AdminAiView }, + { path: 'audit', component: AdminAuditView }, + ] +} +``` + +`AdminLayout.vue` contains `` where page content renders. `App.vue` remains unchanged — it renders the component that the route resolves to, which is now `AdminLayout`. No third branch in `App.vue` is needed. + +--- + +### Pitfall 5: client.js Splitting — noRefreshPaths Breaks When Paths Move to Sub-modules + +**Phase:** Frontend refactoring (client.js split) + +**What goes wrong:** `client.js` has a hardcoded `noRefreshPaths` list inside `request()` (line 26) that guards against infinite refresh loops. If `request()` is moved to a shared `api/core.js` module and the path list stays there, it works. But the three blob-download functions (`adminExportAuditLogCsv`, `adminDownloadDailyExport`, `fetchDocumentContent`) each contain a duplicated copy of the 401-refresh-retry logic. When splitting `client.js` into domain files, those duplicates will be copied into new sub-files. This is the correct moment to eliminate them — but if missed, the bug surface multiplies. + +**Additionally:** If components are changed to import from sub-files directly (e.g. `import { listDocuments } from '../api/documents.js'`) instead of the barrel, every other component that imports from `client.js` still works — but a future refactor that renames a function in the sub-file will miss the barrel re-export and break components that still use the barrel. Both import paths exist simultaneously and diverge. + +**Prevention:** Split in this order: +1. Extract `request()` + `noRefreshPaths` into `api/core.js`. Export `request` from there. +2. Extract the 401-retry pattern from the three blob-download functions into a single `fetchWithRetry(url, headers)` helper also in `api/core.js`. +3. Create domain files (`api/documents.js`, `api/admin.js`, `api/auth.js`, etc.) that import from `api/core.js`. +4. Keep `api/client.js` as a re-export barrel: `export * from './documents.js'; export * from './admin.js'` etc. This preserves all existing `import { listDocuments } from '../api/client.js'` calls without touching any component. Never change component imports to point at sub-files directly. + +--- + +### Pitfall 6: Drag-and-Drop on DocumentCard — dragstart vs @click Conflict + +**Phase:** UX improvements (drag-and-drop) + +**What goes wrong:** `DocumentCard.vue` navigates on `@click="$router.push('/document/' + doc.id)"` (line 4). Browsers fire `click` after `dragend` when the drag distance is below the browser's drag-initiation threshold — typically ~4px. A user who clicks and releases without moving far enough triggers both `dragstart` and then `click`: the card navigates to the document even though the user intended to start a drag. + +**Also:** The action buttons inside `DocumentCard` use `@click.stop`. If a drag handle overlaps an action button area, `@click.stop` prevents the click from propagating but does not prevent `dragstart` from bubbling — behavior becomes inconsistent depending on which pixel the user starts from. + +**Note:** `StorageBrowser.vue` already implements drag-and-drop on list rows correctly (lines 141-145: `draggable`, `@dragstart`, `@dragend`) and does not have this problem because the row's `@click` emits `file-open` rather than navigating imperatively. The risk is specific to adding drag to `DocumentCard.vue` (card grid used in `TopicsView`). + +**Prevention:** +1. Track drag state with a component-level `dragging` ref. Set `dragging = true` in `@dragstart`, reset to `false` in `@dragend`. Guard `@click`: `if (this.dragging) return`. +2. Preferred: use a dedicated drag handle element (grip icon, `cursor-grab`) and set `draggable="true"` only on that element. Add `@click.stop` on the handle. The rest of the card retains click-to-navigate. + +--- + +### Pitfall 7: Virtual Scrolling — StorageBrowser DOM Assumptions Break + +**Phase:** Frontend performance (virtual scrolling) **What goes wrong:** -The Vue 3 SPA stores the JWT access token in `localStorage`. Any JavaScript injected via XSS (in file names, document content previewed in the UI, a compromised dependency) can call `localStorage.getItem('token')` and exfiltrate a long-lived credential. The attacker then impersonates the user from any origin, bypasses TOTP entirely (the token is post-authentication), and can access all documents, including cloud storage credentials. -**Why it happens:** -`localStorage` is the path of least resistance in SPAs. It survives page reloads, works with Axios interceptors trivially, and requires no server-side session state. FastAPI tutorials almost universally use `Authorization: Bearer` headers set from localStorage. +`StorageBrowser.vue` renders its item list with `divide-y divide-gray-100` (line 76). The Tailwind `divide-y` utility inserts borders using adjacent-sibling CSS (`& > * + * { border-top-width: 1px }`). This only works when all children are present in the DOM simultaneously. Virtual scroll libraries keep only visible rows in the DOM — `divide-y` produces borders only between the visible subset, leaving gaps at virtual boundaries. -**How to avoid:** -- Issue the JWT as an **httpOnly, SameSite=Strict, Secure cookie** — JavaScript cannot read it. -- Use a short-lived access token (15 minutes) in the httpOnly cookie. -- Issue a separate refresh token (httpOnly cookie, longer TTL, `/auth/refresh` path-scoped) to rotate access tokens silently. -- The Vue frontend never holds the raw token string. Axios is configured with `withCredentials: true`; the browser attaches cookies automatically. -- CSRF protection: because `SameSite=Strict` blocks cross-site cookie submission, CSRF tokens are not strictly required for same-origin SPAs, but add a CSRF header check (`X-Requested-With: XMLHttpRequest`) as defence-in-depth. +The folder picker dropdown inside file rows (lines 185-213) is `absolute right-0 top-full` — positioned relative to its parent row. Virtual scroll repositions rows via `transform: translateY()`. Absolutely positioned children escape their transformed parent and appear at wrong viewport positions. -**Warning signs:** -- `localStorage.getItem` anywhere in auth-related frontend code -- JWT decode functions in frontend code (the frontend should not need to decode a token it can't read) -- `Authorization: Bearer ${token}` header set manually in Axios interceptor +CSS transitions on rows (`transition-all`, `transition-colors`) work with `v-for` because Vue animates DOM insertion/removal. Virtual scroll does not insert/remove — it repositions. Vue's `` cannot wrap a virtually scrolled list and will throw warnings. -**Phase to address:** Auth phase (Phase 1 — Users & Auth) +**Prevention:** +1. Replace `divide-y` with `border-b border-gray-100` on each individual row element. Rows become self-contained and do not depend on DOM adjacency. +2. The folder picker dropdown must use `` with `position: fixed` coordinates computed from `getBoundingClientRect()`. This is a prerequisite for virtual scroll adoption, not optional. +3. Remove any `` wrapping a virtually scrolled list. Use per-row fade-in via a CSS class added on first mount instead. --- -### Pitfall 2: TOTP Bypass via Password Reset Flow - -**What goes wrong:** -The password reset flow issues a one-time token by email. After the user resets their password, the system logs them in and redirects to the dashboard — skipping the TOTP prompt because the session was created through the reset path, not the login path. An attacker who compromises a user's email account can therefore completely bypass TOTP. - -**Why it happens:** -Password reset and login are treated as separate code paths. The TOTP check lives in the login handler; the reset handler creates a session directly after credential update without going through the TOTP gate. - -**How to avoid:** -- After a successful password reset, issue a partial session or a `password_reset_pending` state, not a full authenticated session. -- Force the user to complete the full login flow (including TOTP if enabled) from the new credentials. -- Alternatively, on password reset completion, invalidate all existing sessions and send the user to the login page (no auto-login). -- Log password resets in the audit trail with IP and user-agent. - -**Warning signs:** -- Password reset handler calls the same session-creation function as login but omits 2FA state checks -- No `mfa_verified` flag on sessions (only `authenticated`) -- Users can reach protected endpoints via a token created from the reset path - -**Phase to address:** Auth phase (Phase 1 — Users & Auth) +## Moderate Pitfalls --- -### Pitfall 3: Refresh Token Rotation — Stolen Refresh Token Not Detected +### Pitfall 8: Responsive Layout — w-64 Sidebar vs Flex-1 Content on Mobile -**What goes wrong:** -Refresh tokens are issued as single-use rotating credentials. If an attacker steals a refresh token and uses it before the legitimate user does, the server rotates the token and the attacker has a valid new one. The legitimate user's next refresh request fails — but without a detection mechanism the failure just looks like a session expiry, no alert is raised, and the attacker continues with the stolen session indefinitely. +**Phase:** Responsive / mobile layout -**Why it happens:** -Teams implement the "rotate on use" mechanic without implementing the "revoke family on reuse" detection. The `refresh_tokens` table lacks a `family_id` column linking reissued tokens. +**What goes wrong:** `App.vue` uses `flex h-screen overflow-hidden` with `AppSidebar` fixed at `w-64 shrink-0`. On screens narrower than ~320px the content area collapses below usable width. `StorageBrowser`'s 5-column grid `grid-cols-[2rem_1fr_6rem_8rem_6rem]` has `hidden sm:block` and `hidden md:block` to hide size and date columns — but the fixed `2rem` icon and `6rem` action columns still consume 128px on either side of the filename, leaving minimal space on a 375px screen. -**How to avoid:** -- Store refresh tokens in the database with a `family_id` (UUID for the original issuance chain) and a `revoked` flag. -- When a refresh token is presented: if it is already marked `revoked` (i.e., a previously rotated token), revoke the **entire family** — force logout of all sessions for that user. -- Emit a security alert (audit log + optionally email) when a reuse attempt is detected. -- Refresh tokens should be hashed before storage (bcrypt or SHA-256 with a per-row salt), same as passwords. - -**Warning signs:** -- Refresh tokens stored as plain values in DB with no `revoked` column -- No `family_id` linking related rotation chains -- Stolen refresh token detection treated as "v2 feature" - -**Phase to address:** Auth phase (Phase 1 — Users & Auth) +**Prevention:** +1. On mobile (`< lg`), hide the sidebar. Use a hamburger button in a mobile-only header bar. Toggle with `translate-x-0 / -translate-x-full` transition on the sidebar element. +2. The sidebar open/closed state needs to be in a Pinia store ref (or provided from `App.vue`) so the hamburger button in the header, the sidebar overlay, and the main content area all react to it. Do not put it in a component's `data()`. +3. On mobile, simplify the grid to `grid-cols-[2rem_1fr_4rem]` — filename and a single action column. Test at 375px explicitly. --- -### Pitfall 4: TOTP — Timing Attack on Code Verification +### Pitfall 9: Admin Panel + Responsive — Double Padding After Extracting Tab Content -**What goes wrong:** -TOTP verification uses Python `==` string comparison. Python's `==` on strings is not constant-time — it short-circuits on the first differing character. A sufficiently sophisticated timing oracle (millions of requests from a local network) can distinguish valid from invalid codes, reducing the 6-digit brute-force space. More practically: without rate limiting, an attacker can brute-force all 1,000,000 possible 6-digit codes during a 30-second window. +**Phase:** Admin panel rearchitecture + responsive layout (interaction pitfall) -**Why it happens:** -TOTP libraries (PyOTP) return a string; developers do `if provided == expected`. Rate limiting is added "later" and often never lands. +**What goes wrong:** `AdminView.vue` wraps all content in `p-8 max-w-5xl mx-auto`. Each tab component (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, etc.) has its own internal padding. When tab components are extracted into standalone page views (`AdminUsersView.vue`, etc.), the `p-8 max-w-5xl mx-auto` wrapper moves to `AdminLayout.vue`. If the extracted page views retain their own inner padding, content is double-padded. If `AdminLayout` does not include the constraint, the new admin pages render full-bleed against the viewport. -**How to avoid:** -- Use `hmac.compare_digest(provided, expected)` instead of `==` for TOTP comparison. -- Rate-limit TOTP attempts: 5 attempts per 30-second window, 15-minute lockout on excess. -- The lockout must be stored server-side (Redis or DB), not client-side. -- Accept only the current window and optionally ±1 window for clock drift — do not accept wider ranges. -- Log every failed TOTP attempt with IP. - -**Warning signs:** -- `if totp.verify(code):` without inspecting PyOTP's internal comparison method -- No rate limit on `POST /auth/totp/verify` -- TOTP window set to 2+ (default is ±1 window = 90 seconds valid — wider than needed) - -**Phase to address:** Auth phase (Phase 1 — Users & Auth) +**Prevention:** When extracting tab components into page views, strip all top-level padding and `max-w` from the extracted component — those belong to `AdminLayout`'s content area. The page components should fill 100% of the space provided by the layout slot. --- -### Pitfall 5: Path Traversal in User File Access (MinIO Object Keys) +### Pitfall 10: Options API Refactoring — Prop Mutation via v-model -**What goes wrong:** -MinIO object keys are constructed from user input: `f"users/{user_id}/{filename}"`. If `filename` contains `../`, `../../`, or URL-encoded equivalents (`%2e%2e%2f`), the resulting key may escape the user's prefix and land in another user's namespace. A request for `../../other_user_id/secret.pdf` resolves to `users/other_user_id/secret.pdf`. +**Phase:** Frontend refactoring (Vue component decomposition) -**Why it happens:** -Developers trust that MinIO will sanitize paths. It does not — S3-compatible APIs treat object keys as arbitrary strings. The prefix-based isolation is only as safe as the key construction code. +**What goes wrong:** If a component receives a prop and binds it directly to ``, Vue emits a prop mutation warning in development. In production (no warnings), the mutation silently fails to propagate: the component's displayed value updates (local vdom), but the parent never receives the change because no `update:propName` event was emitted. -**How to avoid:** -- Never use raw filenames as object key components. Generate a UUID (or UUID + original extension) as the stored key: `f"users/{user_id}/{uuid4()}{ext}"`. -- Store the human-readable filename in the database metadata row, completely decoupled from the storage key. -- If filenames must appear in keys for any reason, strip or reject any `/`, `\`, `..`, `%` characters before key construction. -- On retrieval, look up the object key from the database row (which is owned by `user_id`) rather than constructing it from user input. +**Specific risk in this codebase:** `AdminUsersTab`, `AdminQuotasTab`, and `AuditLogTab` currently own their filter/search state internally. When extracted into standalone views, these may receive initial state from route query params and accidentally bind those props directly to inputs. -**Warning signs:** -- `object_key = f"users/{user_id}/{request.filename}"` -- File download endpoint accepts a `path` or `filename` query parameter and constructs the key from it -- No database lookup intermediating between "user requests file" and "MinIO key used" - -**Phase to address:** Storage migration phase (Phase 2 — DB + MinIO migration) +**Prevention:** Any input the component controls must bind to a local `data()` copy initialized from the prop: `data() { return { localFilter: this.initialFilter } }`. Emit `update:initialFilter` on change. Never write `v-model="$props.X"`. --- -### Pitfall 6: Quota Race Condition — Concurrent Uploads Bypass the Limit +### Pitfall 11: Options API Refactoring — data() vs computed() in Decomposed Components -**What goes wrong:** -Two upload requests arrive simultaneously for a user at 99 MB of a 100 MB quota. Both read quota usage as 99 MB, both pass the `99 + 1 < 100` check, both proceed to upload — the user ends at 101 MB. At larger scale (many simultaneous large uploads) the overage can be significant. +**Phase:** Frontend refactoring -**Why it happens:** -Quota enforcement is implemented as: read current usage → check → write file → update usage. This is a classic check-then-act race when the check and the write are not atomic. +**What goes wrong:** When extracting a large component, logic that was a `computed()` in the parent often gets copy-pasted into a child's `data()`. A `data()` value initializes once at mount — it does not react to prop changes. If the parent later passes a different prop value (e.g. the document list changes after a search), the child's stale `data()` copy never updates. -**How to avoid:** -- Enforce quota atomically using a `SELECT ... FOR UPDATE` on the quota row before uploading, or use a PostgreSQL advisory lock keyed on `user_id`. -- Better: use an optimistic update: `UPDATE quotas SET used_bytes = used_bytes + $new WHERE user_id = $uid AND used_bytes + $new <= limit_bytes RETURNING used_bytes`. If 0 rows are updated, the quota was exceeded — reject before touching MinIO. -- Only update `used_bytes` after the MinIO upload succeeds, but hold the lock/reservation through the upload, or use a two-phase: reserve bytes → upload → confirm, with a cleanup job for stuck reservations. -- Never read quota, do arithmetic in Python, then write back as two separate statements. - -**Warning signs:** -- `current = get_quota(user_id); if current + size <= limit: upload()` -- No database transaction wrapping quota check and update -- Quota table updated with `SET used_bytes = $computed_value` (full overwrite) rather than `used_bytes + delta` - -**Phase to address:** Storage migration phase (Phase 2 — DB + MinIO migration) and Quotas phase +**Prevention:** For every value in a newly created component, ask: "If the parent passes a different value for this prop, should the child update?" Yes → `computed()`. No → `data()`. Initialized from prop but locally modified afterward → `data()` with a `watch` on the prop to reset when it changes externally. --- -### Pitfall 7: Admin Privilege Escalation via Missing Ownership Checks +### Pitfall 12: Keyboard Shortcuts — Conflict with Text Inputs and Browser Defaults -**What goes wrong:** -An admin API endpoint is gated on `is_admin=True` but document-access endpoints only check `is_authenticated`. An admin user calling `GET /api/documents/{id}` with any document ID can read any user's document because the handler checks authentication but not `document.owner_id == current_user.id`. The privacy-first model is violated without any special exploit — just a correctly authenticated request. +**Phase:** UX improvements (keyboard shortcuts) -**Why it happens:** -Authorization is implemented as authentication + role checks ("is this user an admin?") without resource-level ownership verification ("does this user own this resource?"). FastAPI dependency injection makes it easy to write `current_user = Depends(get_current_user)` and forget to check `resource.user_id == current_user.id`. +**What goes wrong:** A global `keydown` listener on `window` fires regardless of focused element. Shortcuts like `/` (focus search), `n` (new folder), `Delete` (delete selected) are also typed characters. They must not fire when any ``, `