Compare commits
4
Commits
123ae5b29b
...
a323276e37
| Author | SHA1 | Date | |
|---|---|---|---|
 curo1305
|
a323276e37 | ||
|
curo1305
|
608acedaf6 | ||
|
curo1305
|
9150d28fe1 | ||
|
curo1305
|
0e56c85349 |
+23
-3
@@ -14,6 +14,18 @@ Every user's documents — and the credentials they use to store them — are in
|
||||
|
||||
See `.planning/MILESTONES.md` and `.planning/milestones/v0.2-ROADMAP.md` for full archive.
|
||||
|
||||
## Current Milestone: v0.3 Reimagining Cloud Storage integration
|
||||
|
||||
**Goal:** Make connected cloud storage behave like first-class DocuVault storage: users can browse and manage cloud files as if they were local, analyze all or selected cloud documents, and search them by keywords, sentences, or ideas while minimizing DocuVault-owned local storage.
|
||||
|
||||
**Target features:**
|
||||
- Native-feeling cloud file navigation and actions: open, preview, upload, move, delete, rename, and organize files from connected providers.
|
||||
- Provider setup, health, reconnect, credential lifecycle, and error handling that make cloud connections feel reliable and understandable.
|
||||
- Analyze all cloud documents or a selected subset without requiring manual local upload.
|
||||
- Classify and index cloud documents so they become smart-searchable like locally uploaded documents.
|
||||
- Cache and offload cloud file bytes on demand, similar to a desktop sync client, preserving local-like navigation while minimizing local storage use.
|
||||
- Rework storage architecture where needed: provider capabilities, background jobs, sync/import/index state, conflict handling, and test coverage.
|
||||
|
||||
## Requirements
|
||||
|
||||
### Validated (v0.1 — shipped 2026-06-06)
|
||||
@@ -65,7 +77,14 @@ See `.planning/MILESTONES.md` and `.planning/milestones/v0.2-ROADMAP.md` for ful
|
||||
- ✓ Dead code deleted (FolderRow.vue, AccountView.vue, stale test files) — v0.2
|
||||
- ✓ WHY-only comment policy enforced (CODE-09) — v0.2
|
||||
|
||||
### Active (next milestone — not yet defined)
|
||||
### Active (v0.3 — Reimagining Cloud Storage integration)
|
||||
|
||||
- [ ] Cloud files can be browsed and managed with the same core actions as local files: open, preview, upload, move, delete, and rename.
|
||||
- [ ] Cloud provider setup and maintenance includes health status, reconnect flows, credential lifecycle handling, and useful error states.
|
||||
- [ ] Users can analyze all cloud documents or a selected subset from connected storage.
|
||||
- [ ] Analyzed cloud documents are classified and indexed for smart search by keywords, sentences, and semantic ideas.
|
||||
- [ ] DocuVault minimizes local storage by caching cloud file bytes only when needed and offloading them when safe.
|
||||
- [ ] Storage architecture models provider capabilities, sync/index state, background jobs, and conflict/error handling explicitly.
|
||||
|
||||
### Out of Scope
|
||||
|
||||
@@ -79,7 +98,7 @@ See `.planning/MILESTONES.md` and `.planning/milestones/v0.2-ROADMAP.md` for ful
|
||||
|
||||
## Context
|
||||
|
||||
- **Current state**: v0.2 shipped 2026-06-17. All 4 v0.2 phases complete: stack upgrade + decomposition (Phase 8), admin panel rearchitecture (Phase 9), UX interaction layer (Phase 10), visual design + responsive layout (Phase 11). 277 tests pass. Bundle −81 kB from baseline. App is mobile-responsive, keyboard-navigable, and fully polished. Ready for next milestone definition.
|
||||
- **Current state**: v0.3 started 2026-06-17 after v0.2 shipped. The next focus is reimagining cloud storage as virtual-local document storage: connected cloud files should feel local to navigate and manage, while DocuVault adds classification and smart search without unnecessarily storing full cloud file contents.
|
||||
- **Tech stack**: FastAPI 0.136+ (Python 3.12), SQLAlchemy 2.0 async, Alembic, MinIO SDK; Vue 3 (Options API), Pinia, Vue Router 4, Vite, Tailwind CSS.
|
||||
- **Code quality**: v0.1 was built feature-first under time pressure. Both backend and frontend contain duplication, inconsistent patterns, and components that grew beyond their original scope. v0.2 addresses this systematically.
|
||||
- **Admin panel**: Standalone /admin/* route subtree with AdminLayout as the route component, AdminSidebar with 5 nav links, and 5 dedicated view components. AdminView.vue and legacy tab components deleted. Admin users redirect to /admin on login; non-admin blocked by to.matched.some() guard.
|
||||
@@ -120,6 +139,7 @@ See `.planning/MILESTONES.md` and `.planning/milestones/v0.2-ROADMAP.md` for ful
|
||||
| Teleport + getBoundingClientRect for dropdowns | Viewport-edge clipping eliminated without complex position logic; prerequisite for virtual scrolling | ✓ Shipped Phase 10 (UX-13) |
|
||||
| Admin routes lazy-loaded | All 5 admin views excluded from initial bundle; −81 kB (−30.6%) improvement | ✓ Shipped Phase 11 (PERF-03) |
|
||||
| Vite 8 upgrade for npm audit | Closed high-severity esbuild CVE present in Vite 5/6; npm audit now clean | ✓ Shipped Phase 11 (11-07) |
|
||||
| Virtual-local cloud storage model | Users already organize documents in their cloud provider; DocuVault should add local-like actions, analysis, and smart search without forcing full import | — Pending v0.3 |
|
||||
|
||||
## Evolution
|
||||
|
||||
@@ -141,4 +161,4 @@ Last updated: 2026-06-16
|
||||
4. Update Context with current state
|
||||
|
||||
---
|
||||
*Last updated: 2026-06-17 — after v0.2 milestone*
|
||||
*Last updated: 2026-06-17 — after starting v0.3 milestone*
|
||||
|
||||
@@ -0,0 +1,135 @@
|
||||
# Requirements: DocuVault v0.3
|
||||
|
||||
**Defined:** 2026-06-17
|
||||
**Core Value:** Every user's documents — and the credentials they use to store them — are inaccessible to everyone except that user, while the platform scales horizontally and supports pluggable storage backends.
|
||||
|
||||
## v0.3 Requirements
|
||||
|
||||
### Connections
|
||||
|
||||
- [ ] **CONN-01**: User can connect, reconnect, test, and disconnect each supported cloud provider.
|
||||
- [ ] **CONN-02**: User can see connection health and actionable errors for expired, revoked, or invalid credentials.
|
||||
- [ ] **CONN-03**: Reconnecting or refreshing credentials invalidates stale provider caches without exposing credentials.
|
||||
- [ ] **CONN-04**: User interface reflects the file operations supported by each connected provider.
|
||||
|
||||
### Cloud File Management
|
||||
|
||||
- [ ] **CLOUD-01**: User can browse connected cloud files and folders through the shared `StorageBrowser`.
|
||||
- [ ] **CLOUD-02**: User can open and preview supported cloud documents through DocuVault authorization.
|
||||
- [ ] **CLOUD-03**: User can upload files into the currently viewed cloud folder.
|
||||
- [ ] **CLOUD-04**: User can create folders in connected cloud storage where the provider supports it.
|
||||
- [ ] **CLOUD-05**: User can rename cloud files and folders where the provider supports it.
|
||||
- [ ] **CLOUD-06**: User can move files and folders within the same cloud connection where the provider supports it.
|
||||
- [ ] **CLOUD-07**: User can delete cloud files and folders after explicit confirmation.
|
||||
- [ ] **CLOUD-08**: User sees unsupported cloud actions disabled with a provider-specific explanation.
|
||||
- [ ] **CLOUD-09**: Successful cloud mutations update navigation promptly and produce metadata-only audit events.
|
||||
|
||||
### Cloud Analysis
|
||||
|
||||
- [ ] **ANALYZE-01**: User can request analysis of an individual cloud document.
|
||||
- [ ] **ANALYZE-02**: User can request analysis of selected cloud files or folders.
|
||||
- [ ] **ANALYZE-03**: User can request analysis of an entire cloud connection after reviewing a file-count and byte-size estimate.
|
||||
- [ ] **ANALYZE-04**: User can monitor aggregate and per-item analysis progress through queued, downloading, extracting, classifying, indexed, and failed states.
|
||||
- [ ] **ANALYZE-05**: User can cancel queued analysis work and retry failed cloud documents.
|
||||
- [ ] **ANALYZE-06**: DocuVault does not repeat analysis when the provider item version or etag has not changed.
|
||||
- [ ] **ANALYZE-07**: Analyzing a cloud document never moves, renames, or rewrites the provider-owned file.
|
||||
|
||||
### Smart Search
|
||||
|
||||
- [ ] **SEARCH-01**: User can search local documents and analyzed cloud documents through one search experience.
|
||||
- [ ] **SEARCH-02**: User can find documents using keywords or sentences matched against extracted text.
|
||||
- [ ] **SEARCH-03**: User can find documents using semantic concepts or ideas rather than exact wording.
|
||||
- [ ] **SEARCH-04**: Search results identify local/cloud source, provider, location, topics, and analysis status.
|
||||
- [ ] **SEARCH-05**: User can filter search results by source, provider, folder, topic, and analysis status.
|
||||
- [ ] **SEARCH-06**: Opening a cloud search result hydrates provider bytes only when preview or download requires them.
|
||||
- [ ] **SEARCH-07**: Executing a search query never downloads cloud file bytes.
|
||||
|
||||
### Cache and Storage Minimization
|
||||
|
||||
- [ ] **CACHE-01**: Provider-owned file bytes remain the source of truth unless the user explicitly imports a file into local DocuVault storage.
|
||||
- [ ] **CACHE-02**: DocuVault persists cloud metadata, extracted text, topics, and semantic search data independently of cached file bytes.
|
||||
- [ ] **CACHE-03**: DocuVault caches cloud file bytes only when required for opening, preview, or analysis.
|
||||
- [ ] **CACHE-04**: Configurable cache limits and safe eviction minimize local storage use without interrupting active work.
|
||||
- [ ] **CACHE-05**: Cached cloud content is isolated by user and connection and is never served without an ownership check.
|
||||
|
||||
### Change Tracking
|
||||
|
||||
- [ ] **SYNC-01**: DocuVault records provider item identifiers, parent/location, version or etag, size, and modification time for indexed cloud items.
|
||||
- [ ] **SYNC-02**: DocuVault marks analyzed cloud items stale when their provider version changes.
|
||||
- [ ] **SYNC-03**: Files deleted outside DocuVault are removed from cloud navigation and excluded from search results after refresh.
|
||||
- [ ] **SYNC-04**: Provider refresh jobs are idempotent and retry transient failures with bounded exponential backoff.
|
||||
|
||||
## Future Requirements
|
||||
|
||||
### Extended Cloud Mobility
|
||||
|
||||
- **XFER-01**: User can copy or move files between different cloud providers.
|
||||
- **IMPORT-01**: User can permanently pin or import a cloud file into DocuVault-owned local storage.
|
||||
- **OFFLINE-01**: User can mark folders for persistent offline availability.
|
||||
|
||||
### Sync Clients
|
||||
|
||||
- **CLIENT-01**: User can install a native desktop client for operating-system file integration.
|
||||
- **CLIENT-02**: Native clients can synchronize selected DocuVault and cloud folders.
|
||||
|
||||
## Out of Scope
|
||||
|
||||
| Feature | Reason |
|
||||
|---------|--------|
|
||||
| Cross-provider move/copy | Requires transfer orchestration and conflict semantics beyond the virtual-local single-provider milestone |
|
||||
| Native desktop sync client | v0.3 remains a responsive web application |
|
||||
| Whole-drive offline mirror | Conflicts with the milestone's local-storage minimization goal |
|
||||
| Real-time collaborative editing | Document management and discovery are the current focus |
|
||||
| Automatic whole-cloud analysis without consent | Could create unexpected AI cost, provider load, and privacy impact |
|
||||
| Permanent local pinning/import | Temporary bounded caching is sufficient for v0.3; explicit import remains future scope |
|
||||
| Public unauthenticated links | Existing v0.x privacy boundary remains unchanged |
|
||||
|
||||
## Traceability
|
||||
|
||||
| Requirement | Phase | Status |
|
||||
|-------------|-------|--------|
|
||||
| CONN-01 | Phase 13 | Pending |
|
||||
| CONN-02 | Phase 13 | Pending |
|
||||
| CONN-03 | Phase 13 | Pending |
|
||||
| CONN-04 | Phase 12 | Pending |
|
||||
| CLOUD-01 | Phase 12 | Pending |
|
||||
| CLOUD-02 | Phase 13 | Pending |
|
||||
| CLOUD-03 | Phase 13 | Pending |
|
||||
| CLOUD-04 | Phase 13 | Pending |
|
||||
| CLOUD-05 | Phase 13 | Pending |
|
||||
| CLOUD-06 | Phase 13 | Pending |
|
||||
| CLOUD-07 | Phase 13 | Pending |
|
||||
| CLOUD-08 | Phase 12 | Pending |
|
||||
| CLOUD-09 | Phase 13 | Pending |
|
||||
| ANALYZE-01 | Phase 14 | Pending |
|
||||
| ANALYZE-02 | Phase 14 | Pending |
|
||||
| ANALYZE-03 | Phase 14 | Pending |
|
||||
| ANALYZE-04 | Phase 14 | Pending |
|
||||
| ANALYZE-05 | Phase 14 | Pending |
|
||||
| ANALYZE-06 | Phase 14 | Pending |
|
||||
| ANALYZE-07 | Phase 14 | Pending |
|
||||
| SEARCH-01 | Phase 15 | Pending |
|
||||
| SEARCH-02 | Phase 15 | Pending |
|
||||
| SEARCH-03 | Phase 15 | Pending |
|
||||
| SEARCH-04 | Phase 15 | Pending |
|
||||
| SEARCH-05 | Phase 15 | Pending |
|
||||
| SEARCH-06 | Phase 15 | Pending |
|
||||
| SEARCH-07 | Phase 15 | Pending |
|
||||
| CACHE-01 | Phase 12 | Pending |
|
||||
| CACHE-02 | Phase 12 | Pending |
|
||||
| CACHE-03 | Phase 14 | Pending |
|
||||
| CACHE-04 | Phase 14 | Pending |
|
||||
| CACHE-05 | Phase 14 | Pending |
|
||||
| SYNC-01 | Phase 12 | Pending |
|
||||
| SYNC-02 | Phase 16 | Pending |
|
||||
| SYNC-03 | Phase 16 | Pending |
|
||||
| SYNC-04 | Phase 16 | Pending |
|
||||
|
||||
**Coverage:**
|
||||
- v0.3 requirements: 36 total
|
||||
- Mapped to phases: 36
|
||||
- Unmapped: 0
|
||||
|
||||
---
|
||||
*Requirements defined: 2026-06-17*
|
||||
*Last updated: 2026-06-17 after roadmap creation*
|
||||
+99
-543
@@ -1,561 +1,117 @@
|
||||
# DocuVault — v1 Roadmap
|
||||
# DocuVault Roadmap: v0.3 Reimagining Cloud Storage integration
|
||||
|
||||
_Last updated: 2026-06-02_
|
||||
**Status:** Proposed
|
||||
**Phases:** 12-16
|
||||
**Requirements:** 36
|
||||
**Defined:** 2026-06-17
|
||||
|
||||
## Mandatory Cross-Cutting Gates (every phase)
|
||||
## Milestone Goal
|
||||
|
||||
Before any phase is marked complete, all three gates must pass:
|
||||
Make connected cloud storage behave like first-class DocuVault storage: users can navigate and manage provider-owned files as if they were local, selectively analyze them, and find them through unified keyword and semantic search while DocuVault minimizes locally cached file bytes.
|
||||
|
||||
1. **Test gate** — `pytest -v` passes with zero failures; every new function/endpoint has at least one test; all security invariant tests pass (wrong owner, admin block, token replay)
|
||||
2. **Security gate** — Security agent runs `bandit -r backend/` (zero HIGH), `pip audit` (zero critical/high), `npm audit --audit-level=high` (zero high/critical); admin endpoints verified to never return `password_hash`, `credentials_enc`, or document content; no hardcoded secrets
|
||||
3. **Bug fix rule** — Any bug fix during execution must: (a) target the root cause, (b) change ≤50 lines, (c) include a regression test — no workarounds permitted
|
||||
## Mandatory Cross-Cutting Gates
|
||||
|
||||
---
|
||||
Before any phase is marked complete:
|
||||
|
||||
## Phases
|
||||
1. Backend and frontend test suites pass with zero failures; every new function, endpoint, store, and component has focused coverage.
|
||||
2. Security gate verifies user/connection/item ownership, credential secrecy, SSRF defenses, metadata-only audit logs, cache isolation, and no admin document-content access.
|
||||
3. Dependency audits report no high/critical vulnerabilities.
|
||||
4. Provider integration tests cover supported capabilities and explicit unsupported behavior.
|
||||
5. User-facing changes update `AGENTS.md`, relevant README sections, and application patch/minor versions according to project protocol.
|
||||
|
||||
- [x] **Phase 1: Infrastructure Foundation** — PostgreSQL + MinIO wired into Docker Compose; Alembic migrations running; existing app still works
|
||||
- [x] **Phase 2: Users & Authentication** — Full auth flow end-to-end (register, login, TOTP, backup codes, password reset, sign-out-all) with admin panel for user management
|
||||
- [x] **Phase 3: Document Migration & Multi-User Isolation** — All documents in PostgreSQL + MinIO; per-user isolation enforced; existing UI still works
|
||||
- [x] **Phase 4: Folders, Sharing, Quotas & Document UX** — Full document management UX (folders, sharing, quota bar, PDF preview, search, audit log)
|
||||
- [x] **Phase 5: Cloud Storage Backends** — Users can connect OneDrive, Google Drive, Nextcloud, or WebDAV as a personal storage backend
|
||||
## Phase Summary
|
||||
|
||||
---
|
||||
| Phase | Name | Goal | Requirements |
|
||||
|------:|------|------|--------------|
|
||||
| 12 | Cloud Resource Foundation | Establish provider capabilities and durable virtual-document metadata without mirroring provider bytes | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 |
|
||||
| 13 | Virtual-Local Cloud Operations | Make cloud browsing and core file actions feel local while preserving provider semantics and security | CONN-01..03, CLOUD-02..07, CLOUD-09 |
|
||||
| 14 | Selective Analysis and Byte Cache | Analyze selected cloud scopes through cancellable jobs backed by bounded, private, on-demand byte caching | ANALYZE-01..07, CACHE-03..05 |
|
||||
| 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
|
||||
| 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
|
||||
|
||||
## Phase Details
|
||||
|
||||
### Phase 1: Infrastructure Foundation
|
||||
### Phase 12: Cloud Resource Foundation
|
||||
|
||||
**Goal**: PostgreSQL + MinIO are wired into Docker Compose with a complete Alembic-managed schema; all services boot cleanly and the existing single-user document scanner continues to work exactly as before — no user-facing behavior change.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Nothing (first phase)
|
||||
**Requirements**: STORE-01, STORE-02, STORE-07
|
||||
**Goal:** DocuVault has a provider-neutral cloud file capability contract and durable per-user cloud item index, and the shared browser can render cloud resources and supported actions without copying full provider files into local storage.
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
**Depends on:** Phase 11
|
||||
**Requirements:** CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01
|
||||
|
||||
1. `docker compose up` starts PostgreSQL, MinIO, and the FastAPI backend with no errors; health checks pass for all three services
|
||||
2. Running `alembic upgrade head` applies the initial migration cleanly against the fresh PostgreSQL instance with no errors
|
||||
3. The full existing document upload, text extraction, and AI classification workflow completes successfully — no regression in single-user behavior
|
||||
4. MinIO object key schema `{user_id}/{document_id}/{uuid4()}{ext}` is enforced in the model layer; human-readable filenames are stored in the DB column, not in the MinIO key
|
||||
**Success Criteria:**
|
||||
|
||||
**Plans**: 5 plans
|
||||
1. Every supported provider reports normalized capabilities for browse, open, preview, upload, folder creation, rename, move, delete, and change tracking.
|
||||
2. Browsing any connected provider persists normalized, owner-scoped cloud item metadata including provider ID/path, parent, type, size, modification time, and version/etag.
|
||||
3. `StorageBrowser.vue` remains the single file browser and renders cloud actions from capabilities, including disabled actions with provider-specific explanations.
|
||||
4. Provider-owned bytes remain outside DocuVault while metadata, extracted text placeholders, topic links, and semantic-index state can be persisted independently.
|
||||
5. Ownership and admin-negative tests prove cloud connection and cloud item metadata cannot cross user boundaries.
|
||||
|
||||
- [x] 01-01-PLAN.md — Docker Compose service topology + Postgres init + Pydantic Settings + requirements
|
||||
- [x] 01-02-PLAN.md — Wave 0 test scaffolds (xfail/skip stubs) + async pytest fixtures
|
||||
- [x] 01-03-PLAN.md — SQLAlchemy ORM models + async engine + Alembic async migration (incl. alembic upgrade head)
|
||||
- [x] 01-04-PLAN.md — StorageBackend ABC + MinIO backend + rewritten async services/storage.py
|
||||
- [x] 01-05-PLAN.md — Lifespan + /health + API cutover + Celery worker + walking-skeleton e2e verify
|
||||
### Phase 13: Virtual-Local Cloud Operations
|
||||
|
||||
**Goal:** Users can maintain healthy cloud connections and perform the main file-management workflows in connected storage with the same shared browser interactions used for local files.
|
||||
|
||||
**Depends on:** Phase 12
|
||||
**Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Users can connect, test, reconnect, and disconnect providers while seeing current health and actionable reauthentication/error states.
|
||||
2. Users can open or preview supported cloud documents and upload files into the current cloud folder through authorized DocuVault endpoints.
|
||||
3. Users can create folders, rename items, move items within one connection, and delete items after confirmation wherever the provider supports each action.
|
||||
4. Successful mutations immediately invalidate affected listings/index entries, refresh the shared browser, and write metadata-only audit events.
|
||||
5. Provider-specific tests verify operation semantics, conflict/error responses, token refresh persistence, SSRF protection, and explicit unsupported capabilities.
|
||||
|
||||
### Phase 14: Selective Analysis and Byte Cache
|
||||
|
||||
**Goal:** Users can analyze individual files, selected scopes, or whole connections through observable and controllable background jobs, while file bytes are hydrated only on demand and evicted safely.
|
||||
|
||||
**Depends on:** Phase 13
|
||||
**Requirements:** ANALYZE-01, ANALYZE-02, ANALYZE-03, ANALYZE-04, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-04, CACHE-05
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Users can analyze one file, a multi-selection, a folder tree, or a whole connection after reviewing recursive file-count and byte-size estimates.
|
||||
2. The UI exposes aggregate and per-item queued, downloading, extracting, classifying, indexed, cancelled, and failed states with retry and queued-work cancellation.
|
||||
3. Analysis jobs are idempotent by cloud item and provider version/etag, and they never mutate the provider-owned file.
|
||||
4. Cloud bytes are cached only for active opening, preview, or analysis work and are evicted by configurable limits without interrupting pinned active jobs.
|
||||
5. Cache ownership, cache-key versioning, cancellation, duplicate-job, failure-retry, and eviction behavior have dedicated tests.
|
||||
|
||||
### Phase 15: Unified Smart Search
|
||||
|
||||
**Goal:** One search experience finds local and analyzed cloud documents by keywords, sentences, or semantic ideas and opens cloud results through on-demand hydration.
|
||||
|
||||
**Depends on:** Phase 14
|
||||
**Requirements:** SEARCH-01, SEARCH-02, SEARCH-03, SEARCH-04, SEARCH-05, SEARCH-06, SEARCH-07
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. A single query returns authorized local and analyzed cloud results ranked through full-text and semantic relevance.
|
||||
2. Keyword and sentence searches match persisted extracted text, while idea searches use persisted embeddings or equivalent semantic indexes.
|
||||
3. Results clearly show source, provider, cloud location, topics, and current analysis/stale status and can be filtered by those fields.
|
||||
4. Search execution performs no provider file download; only opening or previewing a result may hydrate bytes.
|
||||
5. Search isolation, ranking, filtering, stale-result handling, and no-download-on-query invariants have automated coverage.
|
||||
|
||||
### Phase 16: Change Tracking and Reliability
|
||||
|
||||
**Goal:** DocuVault remains accurate when users or other applications modify connected cloud storage outside DocuVault.
|
||||
|
||||
**Depends on:** Phase 15
|
||||
**Requirements:** SYNC-02, SYNC-03, SYNC-04
|
||||
|
||||
**Success Criteria:**
|
||||
|
||||
1. Provider delta feeds or bounded metadata refreshes detect changed versions and mark previously analyzed items stale until reanalysis.
|
||||
2. Items deleted outside DocuVault disappear from navigation and are excluded from search without deleting unrelated user data.
|
||||
3. Refresh jobs are idempotent, cursor-aware where supported, rate-limited, and retry transient failures with bounded exponential backoff.
|
||||
4. Users can see refresh/stale/error state and deliberately reanalyze changed items.
|
||||
5. End-to-end UAT verifies cloud-as-local operations, selective analysis, cache offload, unified smart search, and external-change recovery across supported provider classes.
|
||||
|
||||
## Coverage
|
||||
|
||||
- v0.3 requirements: 36
|
||||
- Mapped to phases: 36
|
||||
- Unmapped: 0
|
||||
- Duplicate mappings: 0
|
||||
|
||||
---
|
||||
|
||||
### Phase 2: Users & Authentication
|
||||
|
||||
**Goal**: Users can register, log in (with optional TOTP 2FA), reset their password, and sign out all active sessions; admins can manage user accounts and assign AI providers — all enforced by a complete FastAPI dependency chain.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 1
|
||||
**Requirements**: AUTH-01, AUTH-02, AUTH-03, AUTH-04, AUTH-05, AUTH-06, AUTH-07, AUTH-08, SEC-01, SEC-02, SEC-03, SEC-05, SEC-06, SEC-07, ADMIN-01, ADMIN-02, ADMIN-03, ADMIN-04, ADMIN-05, ADMIN-07
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A new user can register with an email and password that passes strength validation; a password from the HaveIBeenPwned list is rejected with a clear error
|
||||
2. A logged-in user can enroll a TOTP authenticator app, receive 8–10 backup codes, explicitly acknowledge them, and thereafter be required to supply a TOTP code (or backup code) on every login — a backup code is invalidated on first use
|
||||
3. A user who forgets their password can receive a reset email, follow the link within 1 hour, set a new password, and is then returned to the TOTP login gate (not auto-logged in)
|
||||
4. A user can trigger "sign out all devices" from account settings; all other active sessions are immediately invalidated and any reuse of a rotated refresh token revokes the entire token family
|
||||
5. An admin user can create, deactivate, and reset a user account, and assign an AI provider and model to that user; admin API endpoints never return document content or credentials_enc (per-user document auth enforcement deferred to Phase 3 per D-07)
|
||||
|
||||
**Plans**: 5 plans
|
||||
|
||||
**Wave 1** — Foundation
|
||||
|
||||
- [x] 02-01-PLAN.md — Auth service layer (Argon2, JWT, refresh tokens, TOTP, backup codes, HIBP, security alert), FastAPI deps, BackupCode model + password_must_change migration
|
||||
|
||||
**Wave 2** *(blocked on Wave 1 completion)*
|
||||
|
||||
- [x] 02-02-PLAN.md — Register/login (TOTP + backup code paths) + refresh/logout/change-password endpoints + CSP/Origin validation/rate-limit (IP + per-account) + Vue auth store + router guard + Login/Register views
|
||||
|
||||
**Wave 3** *(blocked on Wave 2 completion)*
|
||||
|
||||
- [x] 02-03-PLAN.md — TOTP enrollment + backup codes + password reset + sign-out-all endpoints + AccountView + TotpEnrollment + BackupCodesDisplay + PasswordReset views
|
||||
|
||||
**Wave 4** *(blocked on Wave 3 completion)*
|
||||
|
||||
- [x] 02-04-PLAN.md — Admin backend: user CRUD, quota, AI config endpoints with get_current_admin enforced + tests
|
||||
|
||||
**Wave 5** *(blocked on Wave 4 completion)*
|
||||
|
||||
- [x] 02-05-PLAN.md — Admin panel frontend: AdminView + three tab components + AppSidebar admin link and user identity footer
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- JWT access token in Pinia memory only — never localStorage (Plans 02, 03, 05)
|
||||
- Refresh token httpOnly SameSite=Strict cookie on all token issuance (Plans 02, 03)
|
||||
- Admin endpoints never return document content or credentials_enc (Plans 04, 05)
|
||||
- All auth endpoints rate-limited per-IP and per-account (Plans 02, 03)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 3: Document Migration & Multi-User Isolation
|
||||
|
||||
**Goal**: All existing documents have been migrated from flat-file JSON + filesystem into PostgreSQL + MinIO; all new uploads use the presigned URL flow; per-user isolation is enforced at the DB level; the existing document UI works without regression; the backend is stateless and ready for horizontal scaling.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 2
|
||||
**Requirements**: STORE-03, STORE-04, STORE-05, STORE-06, STORE-08, SEC-04, DOC-03, DOC-04, DOC-05
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Every document present before migration is accessible after migration with the same metadata and extracted text; a count reconciliation check confirms zero document loss
|
||||
2. Two concurrent uploads that would together exceed a user's 100 MB quota result in exactly one success and one 413 rejection — the quota never goes over limit
|
||||
3. A document delete atomically decrements the user's recorded quota usage; after deletion the quota reflects the freed bytes
|
||||
4. Requesting a document object key or presigned URL for a document owned by a different user returns 403 — no cross-user object access is possible through any request parameter manipulation; all /api/documents/* endpoints enforce get_current_user and return 403 when the requesting user's role is admin (completing SC5 from Phase 2)
|
||||
5. AI classification for each document uses the provider and model assigned to that user by the admin, not any user-supplied or default value
|
||||
|
||||
**Plans**: 5 plans
|
||||
|
||||
**Wave 1** — Migration + test scaffolds
|
||||
|
||||
- [x] 03-01-PLAN.md — Wave 0 test scaffolds (auth_user/admin_user/MinIO mock fixtures + 19 xfail stubs) + Alembic migration 0003 (null-user cleanup, NOT NULL constraint, topic cleanup, quota reconciliation, ix_topics_user_id) — Complete 2026-05-23
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [ ] 03-02-PLAN.md — Presigned upload backend: StorageBackend ABC + MinIOBackend dual client + generate_presigned_put_url/stat_object + /api/documents/upload-url + /api/documents/{id}/confirm with atomic quota UPDATE + GET /api/auth/me/quota + delete-with-quota + abandoned-upload Celery beat + docker-compose CORS/celery-beat
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)*
|
||||
|
||||
- [x] 03-03-PLAN.md — Auth guards: get_regular_user dep + ownership assertions on every /api/documents/* handler (404 not 403) + admin 403 + real user_id in object_key + namespace-scoped /api/topics/* + POST /api/admin/topics + classifier topic-namespace plumbing
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)*
|
||||
|
||||
- [x] 03-04-PLAN.md — Settings retirement + per-user AI: delete /api/settings + remove load_settings/save_settings + classifier accepts ai_provider/ai_model kwargs + Celery task resolves user.ai_provider via DB + frontend SettingsView placeholder + remove settings store/API — Complete 2026-05-23
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)*
|
||||
|
||||
- [ ] 03-05-PLAN.md — Frontend upload flow + quota bar: 3-step upload action with XHR progress + UploadProgress.vue progress bar and quota rejection error block + QuotaBar.vue + AppSidebar embed + quota state in auth store + human checkpoint
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- Atomic quota UPDATE pattern only lives in Plan 02; never duplicate (CLAUDE.md)
|
||||
- Every /api/documents/* handler injects get_regular_user (Plan 03)
|
||||
- AI provider/model resolved only via Celery task DB lookup (Plan 04)
|
||||
- Browser XHR PUT to MinIO sends NO Authorization header (Plan 05)
|
||||
|
||||
**Phase gates (must pass before Phase 3 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; presigned URL, quota enforcement, ownership isolation, and admin-403 all covered
|
||||
- [ ] Security agent: path traversal check on object key construction; cross-user IDOR tests; quota race condition test
|
||||
- [ ] Bandit + pip audit + npm audit all clean
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 4: Folders, Sharing, Quotas & Document UX
|
||||
|
||||
**Goal**: Users have a complete document management experience — organized with folders, shared by handle, warned before they hit quota, able to preview PDFs in-browser, and served by a searchable document list; admins can view the append-only audit log.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 3
|
||||
**Requirements**: FOLD-01, FOLD-02, FOLD-03, FOLD-04, FOLD-05, SHARE-01, SHARE-02, SHARE-03, SHARE-04, SHARE-05, SEC-08, SEC-09, ADMIN-06, DOC-01, DOC-02
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A user can create, rename, and delete folders; moving a document between folders preserves its metadata and AI classification; deleting a non-empty folder prompts with the content count before proceeding
|
||||
2. A user can share a document with another user by handle; the recipient sees it appear in a "Shared with me" virtual folder with no storage quota charged against them; the owner can revoke access and the shared entry disappears immediately for the recipient
|
||||
3. The sidebar quota bar displays current usage in MB; it turns amber at 80% and red at 95%; an upload that would exceed the limit is rejected with an error showing current usage, the rejected file size, and a link to storage settings
|
||||
4. Any document in the user's library can be previewed in-browser as a PDF; document bytes are proxied through the app and no presigned URLs are exposed to the browser (native browser PDF rendering via Content-Type header)
|
||||
5. An admin can view the audit log filtered by date range, user, and action type; the log contains no document content, filenames, or extracted text; account deletion triggers cleanup of all user files before DB records are removed
|
||||
|
||||
**Plans**: 9 plans
|
||||
|
||||
**Wave 1** — Test scaffolds + DB migration (parallel)
|
||||
|
||||
- [x] 04-01-PLAN.md — Wave 0 test stubs: test_folders.py + test_shares.py + test_audit.py + proxy stubs in test_documents.py + SEC-08/SEC-09 stubs in test_security.py
|
||||
- [x] 04-02-PLAN.md — Alembic migration 0004 (users.pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw()
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [x] 04-03-PLAN.md — Audit service (write_audit_log) + Folders API (FOLD-01..05): POST/GET/PATCH/DELETE /api/folders + PATCH /api/documents/{id}/folder + document list sort/search/is_shared extension
|
||||
- [ ] 04-04-PLAN.md — Shares API (SHARE-01..05): POST/GET /api/shares + GET /api/shares/received + DELETE /api/shares/{id} with IDOR protection
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)*
|
||||
|
||||
- [ ] 04-05-PLAN.md — PDF streaming proxy GET /api/documents/{id}/content with Range header support + PATCH /api/auth/me/preferences (pdf_open_mode)
|
||||
- [ ] 04-06-PLAN.md — Admin audit log API (GET /api/admin/audit-log, CSV export) + Celery beat daily audit export task + celery_app.py beat schedule
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)*
|
||||
|
||||
- [ ] 04-07-PLAN.md — SEC-08/SEC-09 hardening + audit log backfill into auth.py/admin.py/documents.py + CloudConnectionOut Pydantic model + delete-user file cleanup
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)*
|
||||
|
||||
- [ ] 04-08-PLAN.md — Frontend data layer: API client functions + useFoldersStore + documents store extension + Vue Router routes (/folders/:folderId, /shared)
|
||||
|
||||
**Wave 6** *(blocked on Wave 5)*
|
||||
|
||||
- [ ] 04-09-PLAN.md — Frontend UI: all new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + view wiring (AppSidebar, DocumentCard, HomeView, FolderView, SharedView, SettingsView, AdminView) + human checkpoint
|
||||
|
||||
**Phase gates (must pass before Phase 4 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; folder ownership, share revocation, quota bar, PDF proxy (no presigned URL exposure) all covered
|
||||
- [ ] Security agent: audit log verified to contain zero document content; sharing IDOR tests; PDF proxy verified to not leak presigned URLs or object keys
|
||||
- [ ] Bandit + pip audit + npm audit all clean
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 5: Cloud Storage Backends
|
||||
|
||||
**Goal**: Users can connect OneDrive, Google Drive, Nextcloud, or a generic WebDAV server as a personal storage backend; credentials are encrypted with a per-user HKDF-derived key; connection status is visible; local and cloud storage coexist; the `StorageBackend` ABC makes adding further backends straightforward.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 4
|
||||
**Requirements**: CLOUD-01, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. A user can connect OneDrive, Google Drive, Nextcloud, or a WebDAV endpoint through an OAuth or credential flow; the connection status is displayed as `ACTIVE`, `REQUIRES_REAUTH`, or `ERROR` — never shows raw credentials
|
||||
2. When an OAuth token is revoked externally (simulated `invalid_grant` response), the connection status transitions to `REQUIRES_REAUTH` without a 500 error; the user is shown a re-authentication prompt
|
||||
3. A user can select their connected cloud backend as the default storage destination for new uploads; local MinIO storage remains available as an alternative; existing local documents are unaffected
|
||||
4. A user can disconnect a cloud backend; credentials are permanently deleted from the DB and a subsequent attempt to use that backend returns an appropriate error — no orphaned data remains
|
||||
5. An admin API response for a user's cloud connections returns only `provider, display_name, connected_at, status` — the `credentials_enc` column is never present in any serialized response
|
||||
|
||||
**Plans**: 12 plans (8 original + 3 UAT gap closure + 1 gap closure wave)
|
||||
|
||||
**Wave 1** — Test scaffold + dependencies
|
||||
|
||||
- [x] 05-01-PLAN.md — Wave 0 xfail stubs, conftest cloud fixtures, requirements.txt packages, config.py settings
|
||||
|
||||
**Wave 2** — Shared utilities
|
||||
|
||||
- [x] 05-02-PLAN.md — cloud_utils.py (SSRF + HKDF), cloud_cache.py (TTLCache), storage factory extension
|
||||
|
||||
**Wave 3** — Cloud backends (parallel, both blocked on Wave 2 / Plan 05-02)
|
||||
|
||||
- [x] 05-03-PLAN.md — GoogleDriveBackend + OneDriveBackend (all 7 StorageBackend methods)
|
||||
- [x] 05-04-PLAN.md — NextcloudBackend + WebDAVBackend (all 7 StorageBackend methods)
|
||||
|
||||
**Wave 4** — Cloud API
|
||||
|
||||
- [x] 05-05-PLAN.md — All /api/cloud/* endpoints + /api/users/me/default-storage + main.py router registration
|
||||
|
||||
**Wave 5** — Document routing + full test suite
|
||||
|
||||
- [x] 05-06-PLAN.md — Upload/content proxy cloud routing + all 15 tests promoted to passing
|
||||
|
||||
**Wave 6** — Frontend settings UI
|
||||
|
||||
- [x] 05-07-PLAN.md — cloudConnections store + API client + SettingsView 3-tab + SettingsCloudTab + CloudCredentialModal
|
||||
|
||||
**Wave 7** — Frontend sidebar (human checkpoint)
|
||||
|
||||
- [x] 05-08-PLAN.md — AppSidebar cloud section + CloudProviderTreeItem + CloudFolderTreeItem + human checkpoint
|
||||
|
||||
**Wave 8** — UAT gap closure (parallel, all independent)
|
||||
|
||||
- [x] 05-09-PLAN.md — Cloud document open/re-analyze/edit: authenticated fetch+Blob URL, cloud-aware Celery task, PATCH /api/documents/{id}
|
||||
- [x] 05-10-PLAN.md — OAuth initiate fix (JSON response), Nextcloud custom endpoint edit round-trip, Edit button on ERROR rows, confirmation text overflow
|
||||
- [x] 05-11-PLAN.md — Admin hard-delete with password confirmation: UserDeleteConfirm backend model + inline frontend panel
|
||||
|
||||
**Wave 9** — Post-UAT gap closure
|
||||
|
||||
- [x] 05-12-PLAN.md — OAuth 400 preflight (unconfigured creds), 502 cloud fallback, upload hint in CloudStorageView, celery-worker volume mount
|
||||
|
||||
**Phase gates (must pass before Phase 5 is complete):**
|
||||
|
||||
- [x] `pytest -v` — zero failures; SSRF prevention on WebDAV/Nextcloud user-supplied URLs; credential encryption/decryption round-trip; admin response never exposes `credentials_enc`; OAuth invalid_grant handling
|
||||
- [x] Security agent: SSRF allowlist verification; credential key derivation correctness; connection status never leaks raw credential values
|
||||
- [x] Bandit + pip audit + npm audit all clean
|
||||
- [x] UAT gaps resolved and re-tested (05-09, 05-10, 05-11, 05-12)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 6: Performance & Production Hardening
|
||||
|
||||
**Goal**: The application is ready for production deployment — observable, load-tested, and hardened; response times meet SLA targets under concurrent load; all auth and document endpoints are rate-limited; structured logging and distributed tracing are in place; the Docker image runs as a non-root user with a read-only filesystem.
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 5
|
||||
**Requirements**: TBD
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. All API endpoints respond within defined latency targets (p50/p95/p99) under a realistic load test (e.g., 50 concurrent users, 5-minute soak)
|
||||
2. Structured JSON logging (correlation IDs, user ID, request latency) is emitted to stdout; a local log aggregation stack (Loki or similar) captures and queries them
|
||||
3. All auth endpoints (login, register, password reset, TOTP) enforce per-IP and per-account rate limits that cannot be bypassed by header manipulation
|
||||
4. Container hardening is complete: non-root user, read-only root filesystem, dropped Linux capabilities; `docker scout` or equivalent reports zero critical CVEs
|
||||
5. A runbook documents all environment variables, startup/shutdown procedures, backup strategy, and on-call escalation path; the app can be stood up from scratch using only the runbook
|
||||
|
||||
**Plans**: 6 plans (4 waves)
|
||||
|
||||
**Wave 0** — Test scaffolds + package verification
|
||||
|
||||
- [x] 06-01-PLAN.md — Nyquist Wave 0: xfail stubs (test_logging.py, test_rate_limiting.py), Locust skeleton, package legitimacy checkpoint (D-01, D-04, D-11, D-12)
|
||||
|
||||
**Wave 1** *(blocked on Wave 0 completion)*
|
||||
|
||||
- [x] 06-02-PLAN.md — structlog JSON logging + CorrelationIDMiddleware + Loki/Promtail/Grafana Docker Compose stack (D-01, D-02, D-03)
|
||||
- [x] 06-03-PLAN.md — Locust locustfile.py with JWT auth + SLA gate listener (D-04, D-05, D-06)
|
||||
|
||||
**Wave 2** *(blocked on Wave 1 completion)*
|
||||
|
||||
- [x] 06-04-PLAN.md — Multi-stage Dockerfile + read_only/tmpfs/cap_drop on backend + celery-worker (D-07, D-08, D-09)
|
||||
- [x] 06-05-PLAN.md — Trusted-proxy get_client_ip body + per-account rate limiter on document/cloud endpoints (D-11, D-12, D-13)
|
||||
|
||||
**Wave 3** *(blocked on Wave 2 completion)*
|
||||
|
||||
- [x] 06-06-PLAN.md — docker scout CVE gate + RUNBOOK.md (D-10, D-14)
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- get_client_ip lives ONLY in backend/deps/utils.py — replace body in-place, no new function (Plans 05)
|
||||
- celery-beat intentionally excluded from read_only: true (Plan 04)
|
||||
- locust must NOT be in requirements.txt — use requirements-dev.txt (Plan 03)
|
||||
- CorrelationIDMiddleware registered LAST in main.py — Starlette reverse order (Plan 02)
|
||||
|
||||
---
|
||||
|
||||
### Phase 6.1: Close v1.0 audit gaps: SHARE-02/STORE-06/ADMIN-06
|
||||
|
||||
**Goal**: Close three v1.0 requirements that remain unimplemented — atomic quota decrement on document delete (STORE-06), "Shared with me" virtual folder without recipient quota charge (SHARE-02), and admin audit log viewer with date/user/action type filters (ADMIN-06).
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 6
|
||||
**Requirements**: STORE-06, SHARE-02, ADMIN-06
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Deleting a document atomically decrements the owning user's quota; after deletion the quota reflects the freed bytes with no race condition under concurrent deletes
|
||||
2. A user who receives a shared document sees it appear in a "Shared with me" virtual folder; the recipient's quota usage is not charged for the shared document's storage
|
||||
3. An admin can view the audit log filtered independently by date range, user, and action type; filtered results contain no document content, filenames, or extracted text
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 1** — Test promotion (parallel)
|
||||
|
||||
- [x] 06.1-01-PLAN.md — Promote test_shares.py stubs to real tests + second_auth_user fixture (SHARE-01..05)
|
||||
- [x] 06.1-02-PLAN.md — Promote test_audit.py stubs to real tests (ADMIN-06)
|
||||
|
||||
**Phase gates (must pass before Phase 6.1 is complete):**
|
||||
|
||||
- [ ] `pytest -v` — zero failures; all 7 share tests + 4 audit log tests passing
|
||||
- [ ] Security agent: bandit + pip audit + npm audit all clean
|
||||
- [ ] STORE-06 confirmed: `test_delete_decrements_quota` passes under `INTEGRATION=1`
|
||||
|
||||
---
|
||||
|
||||
### Phase 6.2: Close v1 sharing + cloud-delete + CSV export gaps
|
||||
|
||||
**Goal**: Close remaining v1 gaps — sharing edge cases (SHARE-03/SHARE-05), cloud document deletion propagation to the remote backend, and CSV export + daily export UI for the admin audit log (ADMIN-06).
|
||||
**Mode:** mvp
|
||||
**Depends on**: Phase 6.1
|
||||
**Requirements**: SHARE-03, SHARE-05, ADMIN-06
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. Documents shared with others display a "Shared" badge in the owner's list view (reads doc.is_shared, not doc.share_count)
|
||||
2. Owner can set permission to "view" or "edit" when creating a share and toggle it per-recipient afterward; PATCH /api/shares/{id} enforces IDOR protection (404 on wrong owner)
|
||||
3. Deleting a cloud document propagates the delete to the cloud provider; failure shows a warning modal with "Remove from app" fallback; ?remove_only=true removes only the DB record; cloud docs never affect quota on delete
|
||||
4. Admin can download filtered audit log CSV via fetch+Blob (not window.location.href); audit log entries show user handles instead of raw UUIDs; user filter accepts handles (not UUIDs)
|
||||
5. Admin can list and download Celery-generated daily audit export files from a new section in the Audit Log tab
|
||||
|
||||
**Plans**: 4 plans
|
||||
|
||||
**Wave 0** — Test stubs
|
||||
|
||||
- [x] 06.2-01-PLAN.md — 11 xfail stubs across test_shares.py, test_documents.py, test_audit.py
|
||||
|
||||
**Wave 1** — Feature slices (parallel)
|
||||
|
||||
- [x] 06.2-02-PLAN.md — SHARE-05 badge fix + SHARE-03 permission control (backend PATCH + frontend dropdown + toggle)
|
||||
- [x] 06.2-03-PLAN.md — Cloud-delete propagation + structured error response + remove_only path + DocumentView warning modal
|
||||
|
||||
**Wave 2** — Audit log enrichment
|
||||
|
||||
- [x] 06.2-04-PLAN.md — Audit handle JOIN + user_handle filter + CSV fetch+Blob fix + daily-export list + download endpoints + AuditLogTab UI
|
||||
|
||||
**Phase gates (must pass before Phase 6.2 is complete):**
|
||||
|
||||
- [x] `pytest -v` — 344 passed, 1 pre-existing unrelated failure (test_extract_docx missing module)
|
||||
- [x] Security agent: bandit + pip audit + npm audit all clean (SECURITY.md threats_open: 0)
|
||||
- [x] IDOR on PATCH /api/shares/{id}: test_share_patch_idor passes
|
||||
- [x] Date regex validation confirmed: GET /api/admin/audit-log/daily-exports/invalid-date returns 404
|
||||
- [x] window.location.href removed from AuditLogTab.vue confirmed by grep
|
||||
|
||||
**Status: ✓ Complete (2026-06-01)**
|
||||
|
||||
### Phase 7: Redo and optimize LLM integration
|
||||
|
||||
**Goal:** A fully refactored, production-reliable AI provider layer: AI provider settings live in a new `system_settings` DB table (replacing env-only config), API keys encrypted at rest via HKDF/AES-GCM (same pattern as cloud credentials), all OpenAI-compatible providers (Groq, xAI, DeepSeek, OpenRouter, Gemini-compat, Mistral-compat, Ollama, LMStudio) handled by a single `GenericOpenAIProvider` class, Anthropic uses native `output_config.format.type="json_schema"` structured output, JSON-mode enforced everywhere with `parse_classification()` as fallback, Celery exponential-backoff retry (30s/90s/270s) replaces silent failure on classification errors, per-provider context window size with smart 60/40 truncation replaces the global `MAX_AI_CHARS`, the singleton `_client` pattern restores httpx connection-pool reuse, an admin AI Providers panel allows interactive configuration plus test-connection, and a "Re-analyze" button on the document card re-queues failed classifications.
|
||||
**Mode:** standard
|
||||
**Depends on:** Phase 6.2
|
||||
**Requirements**: D-01..D-18 (CONTEXT.md decisions; no REQ-IDs yet mapped — phase introduces new infrastructure)
|
||||
|
||||
**Success Criteria** (what must be TRUE):
|
||||
|
||||
1. AI provider settings live in `system_settings`; admin can view, edit, and atomically activate any provider via `PUT /api/admin/ai-config`; `GET /api/admin/ai-config` never returns `api_key_enc` or any decrypted key value
|
||||
2. A document whose classification raises an exception is automatically retried by Celery at 30 s, 90 s, and 270 s; after the third failure the document's status remains `classification_failed` and no further retries occur
|
||||
3. All OpenAI-compatible providers route through `GenericOpenAIProvider`; classify/suggest pass `response_format={"type":"json_object"}` when `supports_json_mode` is True (Gemini preset is False and falls back to `parse_classification()`); Anthropic uses `output_config.format.type="json_schema"` with a constrained schema
|
||||
4. `MAX_AI_CHARS` no longer exists in `openai_provider.py`, `anthropic_provider.py`, or `classifier.py`; each provider truncates input text using its own `context_chars` value via a 60% head + 40% tail strategy
|
||||
5. A failed document shows a red "Classification failed" badge and a "Re-analyze" button on the document card; clicking the button calls `POST /api/documents/{id}/classify`, which sets the document to `processing` and re-queues the Celery task
|
||||
|
||||
**Plans**: 5 plans (5 waves)
|
||||
|
||||
**Wave 1** — Foundation: migration, ORM model, encryption helpers, Wave 0 test stubs
|
||||
|
||||
- [x] 07-01-PLAN.md — Alembic migration 0005 (system_settings table) + SystemSettings ORM model + services/ai_config.py (HKDF helpers + load_provider_config + env seed on startup) + Wave 0 xfail stubs for D-01..D-16 (test_ai_providers.py, test_ai_config.py, test_admin_ai_config.py, test_document_tasks.py)
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)* — Provider layer: ProviderConfig, GenericOpenAIProvider, singleton client fix, registry
|
||||
|
||||
- [x] 07-02-PLAN.md — ProviderConfig Pydantic model + PROVIDER_DEFAULTS + SUPPORTS_JSON_MODE + GenericOpenAIProvider(OpenAIProvider) + OpenAIProvider singleton refactor + ollama/lmstudio context_chars + MAX_AI_CHARS removal from openai_provider.py + classifier.py + registry-based ai/__init__.py get_provider(config: ProviderConfig) + anthropic>=0.95.0 pin
|
||||
|
||||
**Wave 3** *(blocked on Wave 2)* — Anthropic + Classifier wiring
|
||||
|
||||
- [x] 07-03-PLAN.md — AnthropicProvider singleton + output_config json_schema + smart truncation + MAX_AI_CHARS removal + classifier.classify_document driven by load_provider_config (no inline _settings dict) + ai_config stub removed in favor of real ProviderConfig + load_provider_config_by_id helper
|
||||
|
||||
**Wave 4** *(blocked on Wave 3)* — Celery retry harness + Re-queue endpoint
|
||||
|
||||
- [x] 07-04-PLAN.md — Celery extract_and_classify decorator gains bind=True + max_retries=3 + _ClassificationError sentinel + 30/90/270 backoff + _mark_classification_failed on exhaustion + POST /api/documents/{id}/classify changes to re-queue Celery (sets status=processing, calls .delay()) — promotes test_retry_backoff, test_exhaustion_sets_failed_status, test_reclassify_requeues_celery
|
||||
|
||||
**Wave 5** *(blocked on Wave 4)* — Admin UI + Frontend badge
|
||||
|
||||
- [x] 07-05-PLAN.md — Admin AI-config backend (GET/PUT/test-connection with whitelist + audit logging + atomic is_active flip) + frontend API client helpers (getAiConfig/saveAiConfig/testAiConnection) + AdminAiConfigTab.vue global System AI Providers section ABOVE existing per-user table + DocumentCard.vue classification_failed badge + Re-analyze button + human checkpoint UAT
|
||||
|
||||
**Cross-cutting constraints:**
|
||||
|
||||
- `api_key_enc` is never returned by any admin endpoint — enforced by `_ai_config_to_dict()` whitelist (Plan 05)
|
||||
- HKDF domain separation: AI settings use `info=b"ai-provider-settings"`, cloud credentials use `info=b"cloud-credentials"` (Plan 01)
|
||||
- `is_active` flip is atomic: single `UPDATE SET is_active = (provider_id = $target)` — never read-then-write (Plan 05)
|
||||
- Provider clients are stored as `self._client` in `__init__` — never recreated per call (Plans 02, 03)
|
||||
- `MAX_AI_CHARS` removal must cover all three locations: openai_provider.py L5, anthropic_provider.py L5, classifier.py L28 (Plans 02 + 03)
|
||||
- Celery `self.retry()` must be raised from the outer sync task body, never inside `asyncio.run()` (Plan 04 — Pitfall 3)
|
||||
- `extra_hosts: ["host.docker.internal:host-gateway"]` already present in docker-compose.yml — D-14 is already done; no docker-compose changes required (RESEARCH.md)
|
||||
- AdminAiConfigTab.vue per-user assignment table is preserved untouched; the new global system section is added ABOVE it (Plan 05 — Pitfall 6)
|
||||
|
||||
**Phase gates (must pass before Phase 7 is complete):**
|
||||
|
||||
- [x] `pytest -v` — zero failures; D-01/D-03/D-05/D-06/D-07/D-09/D-10/D-11/D-12/D-13/D-16 covered by promoted tests (347 passed, 1 pre-existing failure test_extract_docx missing module)
|
||||
- [x] Security agent: bandit -r backend/ (zero HIGH), npm audit --audit-level=high (2 moderate esbuild/vite dev-only — no high/critical); pip-audit not runnable locally (Python 3.9 vs 3.12 requirements), inherited clean gate from Phase 6.2
|
||||
- [x] `_ai_config_to_dict()` confirmed to never include `api_key_enc` (test_get_never_returns_key passes; whitelist at admin.py:62)
|
||||
- [x] HKDF domain separation verified: test_encrypt_api_key_domain_isolation passes (test_ai_config.py:21)
|
||||
- [x] Atomic `is_active` flip verified: test_set_active_provider_atomic passes (test_admin_ai_config.py:77)
|
||||
- [x] Human checkpoint UAT approves admin panel + DocumentCard badge end-to-end (07-UAT.md: 11/11 passed 2026-06-05)
|
||||
|
||||
**UI hint**: yes
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.1: Security: session revocation on privilege change (CR-01..03) (INSERTED)
|
||||
|
||||
**Goal**: Fix the three missing session-revocation calls in `backend/api/auth.py`: `change_password`, `enable_totp`, and `disable_totp` must all call `revoke_all_refresh_tokens()` (excluding the current session). Add `sessions_revoked` to their response shapes and a frontend toast when other sessions are terminated.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7
|
||||
**Requirements**: CR-01, CR-02, CR-03
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 1** — Backend: service + API changes
|
||||
|
||||
- [ ] 07.1-01-PLAN.md — Add skip_token_hash param to revoke_all_refresh_tokens + wire revoke into change_password, enable_totp, disable_totp with sessions_revoked response + audit log
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)*
|
||||
|
||||
- [ ] 07.1-02-PLAN.md — Tests (3 new test_*_revokes_other_sessions) + frontend toast in SettingsAccountTab.vue + TotpEnrollment.vue
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.2: Security — JTI Claim + Redis Access-Token Revocation (INSERTED)
|
||||
|
||||
**Goal**: Add a `jti` (JWT ID) claim to every issued access token. In `get_current_user`, check `redis.get("jti_revoked:{jti}")` and raise 401 if set. Add `revoke_access_token(jti, ttl)` helper called from `change_password`, `enable_totp`, `disable_totp`, and admin account deactivation. Closes the 15-minute window where a revoked session's live access token remains valid.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.1
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"No JTI Claim and No JTI Revocation in Redis"
|
||||
|
||||
**Status:** Not planned yet
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.3: Security — ES256 Algorithm Upgrade (INSERTED)
|
||||
|
||||
**Goal**: Replace HS256 with ES256 (ECDSA P-256) for JWT signing. Generate a P-256 key pair; store private key in `JWT_PRIVATE_KEY` env var, public key in `JWT_PUBLIC_KEY`. Update `create_access_token` and all `decode_*` functions. Rotate all active refresh tokens on first boot after deploy. A leaked public key cannot forge tokens.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.2
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"JWT Algorithm Downgrade: HS256 Instead of ES256"
|
||||
|
||||
**Plans**: 3 plans
|
||||
|
||||
**Wave 0** — Test scaffolds (no production code)
|
||||
|
||||
- [x] 07.3-01-PLAN.md — Wave 0 xfail stubs: test_auth_es256.py (9 stubs covering ES256-01..05 + RM-01..03 + CFG-01 satellite) + extend test_settings_has_jwt_config for refresh_token_expire_hours
|
||||
|
||||
**Wave 1** *(blocked on Wave 0)* — ES256 core + startup rotation
|
||||
|
||||
- [x] 07.3-02-PLAN.md — config.py jwt_private_key/jwt_public_key/refresh_token_expire_hours + services/auth.py 4 sites to ES256 + main.py _rotate_tokens_on_algorithm_change lifespan hook + docker-compose JWT_PRIVATE_KEY/JWT_PUBLIC_KEY + README key-gen snippet + .env.example + version bump 0.1.2
|
||||
|
||||
**Wave 2** *(blocked on Wave 1)* — "Remember me" 16h/30d TTL split
|
||||
|
||||
- [x] 07.3-03-PLAN.md — create_refresh_token remember_me param + LoginRequest.remember_me + _set_refresh_cookie max_age conditional + LoginView.vue "Stay signed in for 30 days" checkbox + stores/auth.js + api/client.js forwarding + human checkpoint
|
||||
|
||||
**Status:** Complete (2026-06-06)
|
||||
|
||||
---
|
||||
|
||||
### Phase 7.4: Security — Token Fingerprinting / Token Binding (INSERTED)
|
||||
|
||||
**Goal**: Add a `fgp` (fingerprint) claim = `hmac(key, User-Agent + Accept-Language)[:16]` to every issued access token. In `get_current_user`, recompute the fingerprint from the request headers and compare with `hmac.compare_digest`. Limits replay of stolen access tokens to the original device/browser context.
|
||||
**Mode:** quick
|
||||
**Depends on**: Phase 7.3
|
||||
**Requirements**: Tracked in `.planning/codebase/CONCERNS.md` §"No Token Fingerprint / Token Binding"
|
||||
|
||||
**Plans**: 2 plans
|
||||
|
||||
**Wave 0** — Test scaffolds (no production code)
|
||||
|
||||
- [x] 07.4-01-PLAN.md — Wave 0 xfail stubs: test_auth_fgp.py (4 stubs covering FGP-01..04)
|
||||
|
||||
**Wave 1** *(blocked on Wave 0)* — Production implementation + test promotion
|
||||
|
||||
- [x] 07.4-02-PLAN.md — _compute_fgp helper + create_access_token fgp claim + get_current_user fgp validation + login/refresh call site updates + promote all 4 FGP tests
|
||||
|
||||
**Status:** Complete (2026-06-06)
|
||||
|
||||
---
|
||||
|
||||
---
|
||||
|
||||
## Milestones
|
||||
|
||||
- ✅ **v0.2 — UI Overhaul and Optimization** — Phases 8–11 (shipped 2026-06-17)
|
||||
|
||||
<details>
|
||||
<summary>✅ v0.2 — UI Overhaul and Optimization (Phases 8–11) — SHIPPED 2026-06-17</summary>
|
||||
|
||||
- [x] Phase 8: Stack Upgrade & Backend Decomposition (8/8 plans) — completed 2026-06-12
|
||||
- [x] Phase 9: Admin Panel Rearchitecture (5/5 plans) — completed 2026-06-13
|
||||
- [x] Phase 10: UX & Interaction (13/13 plans) — completed 2026-06-16
|
||||
- [x] Phase 11: Visual Design, Responsive Layout & Cleanup (7/7 plans) — completed 2026-06-17
|
||||
|
||||
Full archive: `.planning/milestones/v0.2-ROADMAP.md`
|
||||
|
||||
</details>
|
||||
|
||||
## v0.2 + v0.1 Progress Table
|
||||
|
||||
| Phase | Milestone | Plans Complete | Status | Completed |
|
||||
|-------|-----------|----------------|--------|-----------|
|
||||
| 8. Stack Upgrade & Backend Decomposition | v0.2 | 8/8 | Complete | 2026-06-12 |
|
||||
| 9. Admin Panel Rearchitecture | v0.2 | 5/5 | Complete | 2026-06-13 |
|
||||
| 10. UX & Interaction | v0.2 | 13/13 | Complete | 2026-06-16 |
|
||||
| 11. Visual Design, Responsive Layout & Cleanup | v0.2 | 7/7 | Complete | 2026-06-17 |
|
||||
| 1. Infrastructure Foundation | v0.1 | 5/5 | Complete | 2026-05-22 |
|
||||
| 2. Users & Authentication | v0.1 | 6/6 | Complete | 2026-06-01 |
|
||||
| 3. Document Migration & Multi-User Isolation | v0.1 | 5/5 | Complete | 2026-05-25 |
|
||||
| 4. Folders, Sharing, Quotas & Document UX | v0.1 | 9/9 | Complete | 2026-05-28 |
|
||||
| 5. Cloud Storage Backends | v0.1 | 12/12 | Complete | 2026-05-30 |
|
||||
| 6. Performance & Production Hardening | v0.1 | 6/6 | Complete | 2026-05-30 |
|
||||
| 6.1. Close v1.0 audit gaps | v0.1 | 2/2 | Complete | 2026-05-30 |
|
||||
| 6.2. Close v1 sharing + cloud-delete + CSV export gaps | v0.1 | 5/5 | Complete | 2026-05-31 |
|
||||
| 7. Redo and optimize LLM integration | v0.1 | 5/5 | Complete | 2026-06-05 |
|
||||
| 7.1. Security: session revocation on privilege change | v0.1 | 2/2 | Complete | 2026-06-08 |
|
||||
| 7.2. Security: JTI claim + Redis access-token revocation | v0.1 | 3/3 | Complete | 2026-06-05 |
|
||||
| 7.3. Security: ES256 algorithm upgrade | v0.1 | 3/3 | Complete | 2026-06-06 |
|
||||
| 7.4. Security: token fingerprinting / token binding | v0.1 | 2/2 | Complete | 2026-06-06 |
|
||||
*Roadmap proposed: 2026-06-17*
|
||||
|
||||
+27
-23
@@ -1,47 +1,50 @@
|
||||
---
|
||||
gsd_state_version: 1.0
|
||||
milestone: v0.2
|
||||
milestone_name: UI Overhaul and Optimization
|
||||
current_phase: 11
|
||||
status: complete
|
||||
milestone: v0.3
|
||||
milestone_name: Reimagining Cloud Storage integration
|
||||
current_phase: 12
|
||||
status: planning
|
||||
last_updated: "2026-06-17"
|
||||
last_activity: 2026-06-17 -- v0.2 milestone archived
|
||||
last_activity: 2026-06-17 -- v0.3 roadmap proposed
|
||||
progress:
|
||||
total_phases: 4
|
||||
completed_phases: 4
|
||||
total_plans: 33
|
||||
completed_plans: 33
|
||||
percent: 100
|
||||
total_phases: 5
|
||||
completed_phases: 0
|
||||
total_plans: 0
|
||||
completed_plans: 0
|
||||
percent: 0
|
||||
---
|
||||
|
||||
# Project State
|
||||
|
||||
**Project:** DocuVault
|
||||
**Status:** v0.2 milestone complete — ready for next milestone
|
||||
**Status:** v0.3 milestone planning — roadmap awaiting approval
|
||||
**Last Updated:** 2026-06-17
|
||||
|
||||
## Current Position
|
||||
|
||||
Milestone v0.2 shipped 2026-06-17. All 4 phases complete, 40/40 requirements satisfied.
|
||||
Next action: `/gsd:new-milestone` to define v0.3
|
||||
Phase: 12 of 16 — Cloud Resource Foundation
|
||||
Plan: —
|
||||
Status: Roadmap proposed; phase not yet planned
|
||||
Last activity: 2026-06-17 — v0.3 roadmap proposed
|
||||
|
||||
## Phase Status
|
||||
|
||||
| Phase | Requirements | Status |
|
||||
|-------|-------------|--------|
|
||||
| 8. Stack Upgrade & Backend Decomposition | PERF-01, CODE-01, CODE-02, CODE-03, CODE-04, CODE-08 | **Complete (8/8 plans)** |
|
||||
| 9. Admin Panel Rearchitecture | ADMIN-08..12, CODE-06, CODE-09 | **Complete (5/5 plans)** |
|
||||
| 10. UX & Interaction | UX-01..14, CODE-05 | **Complete (13/13 plans)** |
|
||||
| 11. Visual Design, Responsive Layout & Cleanup | VISUAL-01..04, RESP-01..05, CODE-07, PERF-02, PERF-03 | **Complete (7/7 plans)** |
|
||||
| 12. Cloud Resource Foundation | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 | **Not started** |
|
||||
| 13. Virtual-Local Cloud Operations | CONN-01..03, CLOUD-02..07, CLOUD-09 | **Not started** |
|
||||
| 14. Selective Analysis and Byte Cache | ANALYZE-01..07, CACHE-03..05 | **Not started** |
|
||||
| 15. Unified Smart Search | SEARCH-01..07 | **Not started** |
|
||||
| 16. Change Tracking and Reliability | SYNC-02..04 | **Not started** |
|
||||
|
||||
## Performance Metrics
|
||||
|
||||
| Metric | Value |
|
||||
|---|---|
|
||||
| Phases complete | 4 / 4 |
|
||||
| Requirements satisfied | 40 / 40 |
|
||||
| Plans complete | 33 / 33 |
|
||||
| Tests at close | 277 |
|
||||
| Phases complete | 0 / 5 |
|
||||
| Requirements satisfied | 0 / 36 |
|
||||
| Plans complete | 0 / 0 |
|
||||
| Tests at milestone start | 277 |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -66,6 +69,7 @@ Next action: `/gsd:new-milestone` to define v0.3
|
||||
- v0.1 completed: all 7 foundation phases + security hardening (2026-06-06)
|
||||
- v0.2 completed: UI overhaul, admin panel rearchitecture, responsive layout, codebase quality (2026-06-17)
|
||||
- v0.2 archived to `.planning/milestones/v0.2-ROADMAP.md`
|
||||
- v0.3 proposed: virtual-local cloud operations, selective cloud analysis, bounded byte caching, unified smart search, and provider change tracking
|
||||
|
||||
### Open Questions
|
||||
|
||||
@@ -81,7 +85,7 @@ _Updated at each phase transition._
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Last session | 2026-06-17 — v0.2 milestone archived |
|
||||
| Next action | /gsd:new-milestone to define v0.3 |
|
||||
| Last session | 2026-06-17 — v0.3 roadmap proposed |
|
||||
| Next action | Approve roadmap, then discuss or plan Phase 12 |
|
||||
| Pending decisions | None |
|
||||
| Resume file | None |
|
||||
|
||||
@@ -1,508 +1,91 @@
|
||||
# Architecture Patterns
|
||||
# v0.3 Research: Architecture
|
||||
|
||||
**Domain:** v0.2 UI Overhaul and Optimization — integration analysis for existing DocuVault codebase
|
||||
**Researched:** 2026-06-07
|
||||
**Milestone:** v0.3 Reimagining Cloud Storage integration
|
||||
**Date:** 2026-06-17
|
||||
|
||||
---
|
||||
## Target Model
|
||||
|
||||
## How the Layout System Currently Works
|
||||
Cloud files become virtual documents: provider-owned bytes remain in the connected storage, while DocuVault stores metadata, extracted text, topics, embeddings/search data, status, and a bounded cache of temporary bytes.
|
||||
|
||||
App.vue is the layout switch. It checks `route.meta.layout` and renders one of two branches:
|
||||
## Proposed Components
|
||||
|
||||
```
|
||||
route.meta.layout === 'auth' → <AuthLayout /> (renders <router-view /> inside a centered card)
|
||||
(anything else) → AppSidebar + <main><router-view /></main>
|
||||
```
|
||||
### Provider Capability Layer
|
||||
|
||||
There are currently only two layouts:
|
||||
- **AuthLayout.vue** — centered card, no sidebar, used for login/register/password-reset
|
||||
- **Implicit main layout** — App.vue's `v-else` branch: `<AppSidebar>` + scrollable main
|
||||
Add a cloud file operation interface separate from the current object-storage `StorageBackend` contract:
|
||||
|
||||
The admin panel (`/admin`) currently falls into the `v-else` branch. AdminView.vue renders inside the main user sidebar, inheriting AppSidebar's folder tree, quota bar, and user nav — none of which are relevant to an admin.
|
||||
- `list_folder`
|
||||
- `get_metadata`
|
||||
- `download_bytes` / streaming equivalent
|
||||
- `upload_file`
|
||||
- `rename_item`
|
||||
- `move_item`
|
||||
- `delete_item`
|
||||
- `create_folder`
|
||||
- `get_change_cursor` / `list_changes` where available
|
||||
- `capabilities()` returning action support and limitations
|
||||
|
||||
---
|
||||
This avoids forcing local/MinIO object semantics onto provider file-manager behavior.
|
||||
|
||||
## Integration Point 1: FastAPI Admin Router Decomposition
|
||||
### Cloud Item Index
|
||||
|
||||
### Current state
|
||||
Add durable tables for cloud-visible items and analysis/index state:
|
||||
|
||||
`backend/api/admin.py` is a single file with `router = APIRouter(prefix="/api/admin")`. It is registered in `main.py` as:
|
||||
- `cloud_items`: user, connection, provider, provider item id/path, parent id/path, name, kind, MIME/content type, size, etag/version, mtime, path snapshot, last_seen_at, deleted_at.
|
||||
- `cloud_item_analysis`: item, status, extracted_text, topic links, semantic index status, last_analyzed_etag, error, job id, timestamps.
|
||||
- Optional `cloud_file_cache`: item, etag/version, byte location, size, last_accessed_at, expires_at, pin reason, eviction state.
|
||||
|
||||
```python
|
||||
from api.admin import router as admin_router
|
||||
app.include_router(admin_router)
|
||||
```
|
||||
Keep `documents` for DocuVault-owned local/imported files unless a later design intentionally merges local and cloud into one polymorphic table.
|
||||
|
||||
The file mixes four concern areas:
|
||||
- User management (CRUD on users, status, password reset)
|
||||
- Quota management (get/patch quota per user)
|
||||
- AI provider configuration (system-level: get, put, test-connection, get-models)
|
||||
- Audit log was already extracted to `api/audit.py` in v0.1
|
||||
### API Shape
|
||||
|
||||
### Target structure
|
||||
Split `backend/api/cloud.py` when implementation starts:
|
||||
|
||||
```
|
||||
backend/api/admin/
|
||||
__init__.py
|
||||
users.py — /api/admin/users endpoints
|
||||
quotas.py — /api/admin/users/{id}/quota endpoints
|
||||
ai.py — /api/admin/ai-config endpoints
|
||||
```
|
||||
- `api/cloud/connections.py`
|
||||
- `api/cloud/browse.py`
|
||||
- `api/cloud/actions.py`
|
||||
- `api/cloud/analysis.py`
|
||||
- `api/cloud/search.py` or reuse a shared search API
|
||||
- shared schemas in `api/schemas.py` or `api/cloud/schemas.py`
|
||||
|
||||
### How to register without changing URL prefix
|
||||
Every endpoint must assert `connection.user_id == current_user.id` and `cloud_item.user_id == current_user.id`.
|
||||
|
||||
FastAPI supports two-level include: a parent router with the shared prefix, sub-routers without prefix. This is the correct pattern:
|
||||
### Frontend Shape
|
||||
|
||||
```python
|
||||
# backend/api/admin/__init__.py
|
||||
from fastapi import APIRouter
|
||||
from .users import router as users_router
|
||||
from .quotas import router as quotas_router
|
||||
from .ai import router as ai_router
|
||||
`FileManagerView` and `CloudFolderView` remain thin data providers. `StorageBrowser.vue` becomes capability-aware:
|
||||
|
||||
router = APIRouter(prefix="/api/admin", tags=["admin"])
|
||||
router.include_router(users_router)
|
||||
router.include_router(quotas_router)
|
||||
router.include_router(ai_router)
|
||||
```
|
||||
- accepts `capabilities` prop
|
||||
- shows action buttons based on capabilities
|
||||
- emits generic events for open/upload/rename/move/delete/analyze
|
||||
- displays status badges for cached/indexed/analyzing/stale/error
|
||||
|
||||
Each sub-router declares its routes WITHOUT a prefix (no `prefix=` arg):
|
||||
Cloud-specific stores own provider state and API calls; `StorageBrowser` stays store-agnostic.
|
||||
|
||||
```python
|
||||
# backend/api/admin/users.py
|
||||
router = APIRouter()
|
||||
### Byte Cache and Offload
|
||||
|
||||
@router.get("/users") # resolves to /api/admin/users
|
||||
@router.post("/users") # resolves to /api/admin/users
|
||||
@router.patch("/users/{id}/status")
|
||||
...
|
||||
```
|
||||
Cache flow:
|
||||
|
||||
`main.py` changes from:
|
||||
1. User opens/previews/analyzes cloud item.
|
||||
2. Backend verifies ownership and provider metadata.
|
||||
3. Backend downloads bytes into a bounded cache or streams through memory if small.
|
||||
4. Extraction/preview reads from cache.
|
||||
5. Cache entry records item id, etag/version, byte size, last access, expiry.
|
||||
6. Eviction removes bytes once unpinned and stale or over budget.
|
||||
|
||||
```python
|
||||
from api.admin import router as admin_router
|
||||
app.include_router(admin_router)
|
||||
```
|
||||
The cache is not the source of truth. Provider metadata and extracted/indexed data are.
|
||||
|
||||
to the identical line — because `api/admin/__init__.py` exports `router` with the same name. **No URL changes. No consumer changes.**
|
||||
### Analysis Flow
|
||||
|
||||
### What moves where
|
||||
1. User selects file/folder/provider and clicks Analyze.
|
||||
2. Backend resolves selection into cloud items, stores/updates metadata rows, and enqueues Celery jobs.
|
||||
3. Jobs hydrate bytes as needed, extract text, classify topics, optionally create embeddings, and persist analysis state.
|
||||
4. Search reads only persisted text/vector/topic metadata; it should not download cloud bytes during query.
|
||||
5. Opening a result hydrates bytes on demand if preview/download needs them.
|
||||
|
||||
| Endpoints | Target file | Pydantic models that move with it |
|
||||
|---|---|---|
|
||||
| GET/POST /users, PATCH /users/{id}/status, POST /users/{id}/password-reset, DELETE /users/{id} | `admin/users.py` | `UserCreate`, `UserStatusUpdate`, `UserDeleteConfirm`, `_user_to_dict` |
|
||||
| GET/PATCH /users/{id}/quota, PATCH /users/{id}/ai-config | `admin/quotas.py` | `QuotaUpdate`, `UserAiConfigUpdate` |
|
||||
| GET/PUT /ai-config, POST /ai-config/test-connection, GET /ai-config/models | `admin/ai.py` | `SystemAiConfigUpdate`, `TestConnectionRequest`, `_ai_config_to_dict` |
|
||||
## Build Order
|
||||
|
||||
`POST /topics` (create system topic) stays in `admin/users.py` — it is short and has no unique models. Alternatively it can go in its own `admin/topics.py`; a decision either way is fine.
|
||||
|
||||
### Shared helpers
|
||||
|
||||
`_DEFAULT_QUOTA_BYTES`, `CloudConnectionOut`, and any imports shared across sub-routers can live in `backend/api/admin/shared.py` and be imported by each sub-router. Do not duplicate them.
|
||||
|
||||
### Files changed
|
||||
|
||||
| File | Change type |
|
||||
|---|---|
|
||||
| `backend/api/admin.py` | DELETED — replaced by package |
|
||||
| `backend/api/admin/__init__.py` | CREATED — aggregating router |
|
||||
| `backend/api/admin/users.py` | CREATED |
|
||||
| `backend/api/admin/quotas.py` | CREATED |
|
||||
| `backend/api/admin/ai.py` | CREATED |
|
||||
| `backend/api/admin/shared.py` | CREATED (optional, holds shared constants/helpers) |
|
||||
| `backend/main.py` | NO CHANGE — import path `api.admin.router` remains valid |
|
||||
|
||||
---
|
||||
|
||||
## Integration Point 2: Vue Admin Layout
|
||||
|
||||
### Current layout switch
|
||||
|
||||
`App.vue`:
|
||||
|
||||
```html
|
||||
<AuthLayout v-if="route.meta.layout === 'auth'" />
|
||||
<div v-else class="flex h-screen overflow-hidden">
|
||||
<AppSidebar />
|
||||
<main class="flex-1 overflow-y-auto">
|
||||
<router-view />
|
||||
</main>
|
||||
</div>
|
||||
```
|
||||
|
||||
### How to add AdminLayout without breaking anything
|
||||
|
||||
Extend the `v-if` chain to a three-way switch using `route.meta.layout`:
|
||||
|
||||
```html
|
||||
<AuthLayout v-if="route.meta.layout === 'auth'" />
|
||||
<AdminLayout v-else-if="route.meta.layout === 'admin'" />
|
||||
<div v-else class="flex h-screen overflow-hidden">
|
||||
<AppSidebar />
|
||||
<main class="flex-1 overflow-y-auto">
|
||||
<router-view />
|
||||
</main>
|
||||
</div>
|
||||
```
|
||||
|
||||
`AdminLayout.vue` renders its own sidebar and a `<router-view />`. It is structurally identical in shape to the main layout but with admin-specific nav items.
|
||||
|
||||
### Router changes
|
||||
|
||||
Current single admin route:
|
||||
|
||||
```js
|
||||
{ path: '/admin', component: () => import('../views/AdminView.vue'), meta: { requiresAdmin: true } }
|
||||
```
|
||||
|
||||
Replace with a route subtree:
|
||||
|
||||
```js
|
||||
{
|
||||
path: '/admin',
|
||||
component: AdminLayout,
|
||||
meta: { requiresAdmin: true, layout: 'admin' },
|
||||
children: [
|
||||
{ path: '', redirect: '/admin/users' },
|
||||
{ path: 'users', component: () => import('../views/admin/AdminUsersView.vue') },
|
||||
{ path: 'quotas', component: () => import('../views/admin/AdminQuotasView.vue') },
|
||||
{ path: 'ai', component: () => import('../views/admin/AdminAiView.vue') },
|
||||
{ path: 'audit', component: () => import('../views/admin/AdminAuditView.vue') },
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
### Meta inheritance caveat
|
||||
|
||||
Vue Router 4 does NOT automatically merge `meta` from parent to children — `route.meta` on a child route only contains what is declared on that child. `App.vue` currently reads `route.meta.layout` directly. For the child routes to inherit `layout: 'admin'`, the check in App.vue must walk `route.matched`:
|
||||
|
||||
```js
|
||||
// App.vue — layout resolution
|
||||
const layout = computed(() => route.matched.find(r => r.meta.layout)?.meta.layout)
|
||||
```
|
||||
|
||||
Likewise, the navigation guard in `router/index.js` must change from `to.meta.requiresAdmin` to `to.matched.some(r => r.meta.requiresAdmin)`. The current guard already handles `.meta.public` the same way — so this is a one-line change pattern that's already established in the router.
|
||||
|
||||
### AdminLayout.vue structure
|
||||
|
||||
```
|
||||
AdminLayout.vue
|
||||
└── AdminSidebar.vue (nav links: Users, Quotas, AI Config, Audit Log)
|
||||
└── <router-view />
|
||||
```
|
||||
|
||||
`AdminSidebar.vue` is NOT `AppSidebar.vue`. Do not reuse or extend `AppSidebar`. The admin sidebar is a new, independent component with admin-specific nav.
|
||||
|
||||
### Views created
|
||||
|
||||
Each current tab component becomes a thin view wrapper (following the View -> Smart -> Presentational hierarchy):
|
||||
|
||||
| New file | Wraps existing component |
|
||||
|---|---|
|
||||
| `views/admin/AdminUsersView.vue` | `AdminUsersTab.vue` (kept unchanged as smart component) |
|
||||
| `views/admin/AdminQuotasView.vue` | `AdminQuotasTab.vue` |
|
||||
| `views/admin/AdminAiView.vue` | `AdminAiConfigTab.vue` |
|
||||
| `views/admin/AdminAuditView.vue` | `AuditLogTab.vue` |
|
||||
|
||||
The existing tab components (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, etc.) are NOT renamed or restructured in this step. They become the smart components rendered inside the new thin view wrappers.
|
||||
|
||||
### Files changed
|
||||
|
||||
| File | Change type |
|
||||
|---|---|
|
||||
| `frontend/src/App.vue` | MODIFIED — add `v-else-if` for admin layout; meta layout resolution uses `route.matched` |
|
||||
| `frontend/src/router/index.js` | MODIFIED — /admin replaced with children subtree; guard uses `to.matched` |
|
||||
| `frontend/src/layouts/AdminLayout.vue` | CREATED |
|
||||
| `frontend/src/components/layout/AdminSidebar.vue` | CREATED |
|
||||
| `frontend/src/views/AdminView.vue` | DELETED — replaced by subtree |
|
||||
| `frontend/src/views/admin/AdminUsersView.vue` | CREATED (thin wrapper) |
|
||||
| `frontend/src/views/admin/AdminQuotasView.vue` | CREATED (thin wrapper) |
|
||||
| `frontend/src/views/admin/AdminAiView.vue` | CREATED (thin wrapper) |
|
||||
| `frontend/src/views/admin/AdminAuditView.vue` | CREATED (thin wrapper) |
|
||||
| `frontend/src/components/admin/AdminUsersTab.vue` | KEPT (smart component, unchanged) |
|
||||
| `frontend/src/components/admin/AdminQuotasTab.vue` | KEPT |
|
||||
| `frontend/src/components/admin/AdminAiConfigTab.vue` | KEPT |
|
||||
| `frontend/src/components/admin/AuditLogTab.vue` | KEPT |
|
||||
|
||||
---
|
||||
|
||||
## Integration Point 3: client.js Decomposition
|
||||
|
||||
### Current import pattern
|
||||
|
||||
Every consumer does one of:
|
||||
|
||||
```js
|
||||
import * as api from '../api/client.js' // namespace import — stores, views, most components
|
||||
import { specificFn } from '../api/client.js' // named import — a few components
|
||||
```
|
||||
|
||||
35+ files import from `client.js`. A decomposition that changes the import path in all 35 files is a large, purely mechanical refactor with meaningful test/review cost for zero user-visible benefit.
|
||||
|
||||
### Minimal, non-breaking approach: barrel re-export
|
||||
|
||||
Create domain modules alongside `client.js`, then have `client.js` re-export everything so no consumer changes:
|
||||
|
||||
```
|
||||
frontend/src/api/
|
||||
client.js — request() transport function + re-exports from domain files
|
||||
documents.js — document API calls (import request from './client.js')
|
||||
topics.js
|
||||
auth.js
|
||||
admin.js
|
||||
folders.js
|
||||
shares.js
|
||||
cloud.js
|
||||
```
|
||||
|
||||
`client.js` becomes the HTTP transport layer and a re-export barrel:
|
||||
|
||||
```js
|
||||
// client.js (after decomposition) — transport only + re-exports
|
||||
async function request(path, options = {}) { ... }
|
||||
export async function fetchDocumentContent(docId, options = {}) { ... }
|
||||
|
||||
export * from './documents.js'
|
||||
export * from './topics.js'
|
||||
export * from './auth.js'
|
||||
export * from './admin.js'
|
||||
export * from './folders.js'
|
||||
export * from './shares.js'
|
||||
export * from './cloud.js'
|
||||
```
|
||||
|
||||
Each domain file imports `request` from `client.js`:
|
||||
|
||||
```js
|
||||
// documents.js
|
||||
import { request } from './client.js'
|
||||
export function listDocuments(...) { return request('/api/documents?...') }
|
||||
```
|
||||
|
||||
This means `import * as api from '../api/client.js'` continues to work for all 35 existing consumers. The decomposition is fully internal.
|
||||
|
||||
### Test mock compatibility
|
||||
|
||||
Existing Vitest tests mock `api/client.js` at the module level (`vi.mock('../../api/client.js')`). Because the re-exports are in `client.js`, mocking `client.js` still intercepts all domain functions. No test changes needed.
|
||||
|
||||
### Files changed
|
||||
|
||||
| File | Change type |
|
||||
|---|---|
|
||||
| `frontend/src/api/client.js` | MODIFIED — split out domain functions, keep re-exports |
|
||||
| `frontend/src/api/documents.js` | CREATED |
|
||||
| `frontend/src/api/topics.js` | CREATED |
|
||||
| `frontend/src/api/auth.js` | CREATED (name does not conflict with `stores/auth.js` — different path) |
|
||||
| `frontend/src/api/admin.js` | CREATED |
|
||||
| `frontend/src/api/folders.js` | CREATED |
|
||||
| `frontend/src/api/shares.js` | CREATED |
|
||||
| `frontend/src/api/cloud.js` | CREATED |
|
||||
| All 35 consumer files | NO CHANGE |
|
||||
|
||||
---
|
||||
|
||||
## Integration Point 4: Icon Strategy
|
||||
|
||||
### Current state
|
||||
|
||||
There are 58 inline `<svg>` blocks in `frontend/src/components/` and 8 more in `frontend/src/views/`. Each is a 3-6 line block with `stroke`, `viewBox`, `class`, and a `<path>`. The same path data appears multiple times for the same icon (e.g., the document icon in the sidebar, document card, and document view are separate copies of the same path string).
|
||||
|
||||
### Options assessed
|
||||
|
||||
**Option A: SVG sprite sheet**
|
||||
A `<symbol>`-based sprite in `index.html` with `<use href="#icon-name" />` in components. Minimal bundle impact, no component overhead. Drawback: requires a build step or manual maintenance of the sprite file; poor DX for adding new icons during the visual redesign.
|
||||
|
||||
**Option B: `AppIcon.vue` component with a `name` prop**
|
||||
A Vue component that maps `name` to an inline `<svg>` block. All SVG paths live in one file. Zero external dependencies. Easy to add icons.
|
||||
|
||||
**Option C: lucide-vue-next**
|
||||
Purpose-built icon library for Tailwind projects. Tree-shakeable individual icon components: `<FileText class="w-4 h-4" />`. Maintained, typed, comprehensive. Avoids maintaining path data entirely.
|
||||
|
||||
**Recommendation: Option B (`AppIcon.vue`) unless a dependency is explicitly approved.**
|
||||
|
||||
The project already has 66 SVG paths hand-chosen to match the visual style. Switching to lucide means auditing that every lucide icon matches the expected visual. During a visual redesign milestone that may be desirable, but it is a separate decision. The minimal refactor that eliminates duplication without changing visual output is Option B.
|
||||
|
||||
**Named slots are wrong for this problem.** Named slots solve "inject different markup into different zones of a layout". This problem is "render the correct path data for a given icon name". A `name` prop is the correct API.
|
||||
|
||||
**Option B implementation:**
|
||||
|
||||
```
|
||||
frontend/src/components/ui/AppIcon.vue
|
||||
```
|
||||
|
||||
```html
|
||||
<template>
|
||||
<svg :class="$attrs.class" fill="none" stroke="currentColor" viewBox="0 0 24 24"
|
||||
aria-hidden="true">
|
||||
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" :d="paths[name]" />
|
||||
</svg>
|
||||
</template>
|
||||
<script>
|
||||
export default {
|
||||
inheritAttrs: false,
|
||||
props: { name: { type: String, required: true } },
|
||||
data: () => ({
|
||||
paths: {
|
||||
document: 'M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5...',
|
||||
folder: 'M3 7a2 2 0 012-2h4l2 2h6a2 2 0 012 2v...',
|
||||
// etc.
|
||||
}
|
||||
})
|
||||
}
|
||||
</script>
|
||||
```
|
||||
|
||||
Usage: `<AppIcon name="document" class="w-4 h-4 mr-2 shrink-0" />`
|
||||
|
||||
### Files changed (Option B)
|
||||
|
||||
| File | Change type |
|
||||
|---|---|
|
||||
| `frontend/src/components/ui/AppIcon.vue` | CREATED |
|
||||
| ~35 components/views with inline SVGs | MODIFIED — replace `<svg>...</svg>` with `<AppIcon name="..." class="..." />` |
|
||||
|
||||
---
|
||||
|
||||
## Suggested Build Order
|
||||
|
||||
Dependencies between the four changes determine order. A change that other changes depend on must land first.
|
||||
|
||||
```
|
||||
Step 1 — Backend admin router decomposition
|
||||
Step 2 — API client decomposition (client.js -> domain files)
|
||||
Step 3 — Admin layout + route subtree (frontend)
|
||||
Step 4 — Icon refactor
|
||||
```
|
||||
|
||||
### Rationale
|
||||
|
||||
**Step 1 (backend) first:**
|
||||
- No frontend dependencies. All existing tests pass with zero behavior change (same URLs, same auth, same import path in `main.py`).
|
||||
- Risk: lowest. Pure internal restructuring.
|
||||
|
||||
**Step 2 (client.js) second:**
|
||||
- No dependency on the admin layout.
|
||||
- New admin view files created in Step 3 can import from `api/admin.js` specifically rather than the monolith.
|
||||
- Internal-only change. All 35 consumer files unchanged. All existing mock-based tests unchanged.
|
||||
- Risk: low.
|
||||
|
||||
**Step 3 (admin layout + route subtree) third:**
|
||||
- Highest risk: modifies `App.vue` (central layout switch), `router/index.js` (all navigation), creates new layout files.
|
||||
- The existing smart components (`AdminUsersTab.vue` etc.) are not touched — they are simply wrapped by new thin view files.
|
||||
- Navigation guard logic for `requiresAdmin` and `layout` meta must be re-tested after this step.
|
||||
- Risk: medium.
|
||||
|
||||
**Step 4 (icon refactor) last:**
|
||||
- Pure cosmetic. Zero behavior change.
|
||||
- Can be done incrementally (file by file) without blocking anything.
|
||||
- Risk: lowest.
|
||||
|
||||
### Parallelism
|
||||
|
||||
Steps 1 and 2 are independent and can be executed simultaneously — they touch no shared files.
|
||||
|
||||
Steps 3 and 4 overlap on the admin component files (icons are replaced in Step 4, structure is set in Step 3). Complete Step 3 first to avoid merge conflicts.
|
||||
|
||||
---
|
||||
|
||||
## Component Architecture: Existing vs. Target
|
||||
|
||||
### Current (wrong)
|
||||
|
||||
```
|
||||
App.vue
|
||||
└── AppSidebar (always present)
|
||||
└── AdminView.vue (tabs rendered in user sidebar context)
|
||||
└── AdminUsersTab.vue
|
||||
└── AdminQuotasTab.vue
|
||||
└── AdminAiConfigTab.vue
|
||||
└── AuditLogTab.vue
|
||||
```
|
||||
|
||||
### Target (correct)
|
||||
|
||||
```
|
||||
App.vue (layout switch)
|
||||
└── AuthLayout.vue (for meta.layout === 'auth')
|
||||
└── AdminLayout.vue (for meta.layout === 'admin')
|
||||
└── AdminSidebar.vue
|
||||
└── <router-view />
|
||||
→ AdminUsersView.vue (thin — passes props to AdminUsersTab)
|
||||
→ AdminQuotasView.vue
|
||||
→ AdminAiView.vue
|
||||
→ AdminAuditView.vue
|
||||
└── Implicit main layout (v-else)
|
||||
└── AppSidebar.vue
|
||||
└── <router-view />
|
||||
→ FileManagerView, DocumentView, TopicsView, ...
|
||||
```
|
||||
|
||||
The smart components (`AdminUsersTab.vue` etc.) are unchanged. The view layer above them is what changes.
|
||||
|
||||
---
|
||||
|
||||
## What Is New vs. Modified: Complete Inventory
|
||||
|
||||
### New files
|
||||
|
||||
```
|
||||
backend/api/admin/__init__.py
|
||||
backend/api/admin/users.py
|
||||
backend/api/admin/quotas.py
|
||||
backend/api/admin/ai.py
|
||||
backend/api/admin/shared.py (optional — shared constants/helpers)
|
||||
|
||||
frontend/src/layouts/AdminLayout.vue
|
||||
frontend/src/components/layout/AdminSidebar.vue
|
||||
frontend/src/views/admin/AdminUsersView.vue
|
||||
frontend/src/views/admin/AdminQuotasView.vue
|
||||
frontend/src/views/admin/AdminAiView.vue
|
||||
frontend/src/views/admin/AdminAuditView.vue
|
||||
frontend/src/api/documents.js
|
||||
frontend/src/api/topics.js
|
||||
frontend/src/api/auth.js
|
||||
frontend/src/api/admin.js
|
||||
frontend/src/api/folders.js
|
||||
frontend/src/api/shares.js
|
||||
frontend/src/api/cloud.js
|
||||
frontend/src/components/ui/AppIcon.vue (Option B only)
|
||||
```
|
||||
|
||||
### Modified files
|
||||
|
||||
```
|
||||
backend/main.py (no functional change — import stays identical)
|
||||
|
||||
frontend/src/App.vue (add v-else-if; use route.matched for layout)
|
||||
frontend/src/router/index.js (admin route becomes children subtree; guard uses to.matched)
|
||||
frontend/src/api/client.js (gutted to transport + re-exports)
|
||||
All ~35 components with inline SVGs (icons replaced with AppIcon or lucide)
|
||||
```
|
||||
|
||||
### Deleted files
|
||||
|
||||
```
|
||||
backend/api/admin.py (replaced by package)
|
||||
frontend/src/views/AdminView.vue (replaced by route subtree)
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Minimal vs. Proper Assessment
|
||||
|
||||
| Change | Is the minimal solution also the proper one? |
|
||||
|---|---|
|
||||
| Backend decomposition | Yes. Sub-router via `__init__.py` aggregator is both minimal and the standard FastAPI pattern. |
|
||||
| client.js decomposition | Yes. Re-export barrel means zero consumer churn while achieving internal separation. |
|
||||
| Admin layout | Yes. Three-way `v-if` in App.vue + new layout file is 20 lines of change in existing files. |
|
||||
| Icon refactor | The minimal solution (AppIcon.vue) is simpler than lucide but adds maintenance burden for the path map. lucide is more proper but adds a dependency. Call must be made by project owner. |
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- Inspected source: `backend/main.py`, `backend/api/admin.py`, `frontend/src/App.vue`, `frontend/src/router/index.js`, `frontend/src/layouts/AuthLayout.vue`, `frontend/src/api/client.js`, `frontend/src/views/AdminView.vue`, `frontend/src/components/layout/AppSidebar.vue`
|
||||
- FastAPI sub-router `include_router` nesting: standard FastAPI router composition (HIGH confidence)
|
||||
- Vue Router 4 `route.matched` for meta inheritance: Vue Router 4 documented behavior (HIGH confidence)
|
||||
- lucide-vue-next: tree-shakeable Vue 3 icon library (MEDIUM confidence — verified as active, Tailwind-compatible)
|
||||
1. Capability model + provider action interface + tests.
|
||||
2. Cloud item metadata/index schema + browse persistence.
|
||||
3. Unified browser actions for cloud files/folders.
|
||||
4. Cloud analysis jobs and cache/offload service.
|
||||
5. Unified smart search over local and analyzed cloud documents.
|
||||
6. Provider change detection, stale markers, and polish.
|
||||
|
||||
+54
-246
@@ -1,257 +1,65 @@
|
||||
# Feature Landscape — v0.2 UI Overhaul
|
||||
# v0.3 Research: Features
|
||||
|
||||
**Domain:** Self-hosted SaaS document management (DocuVault)
|
||||
**Researched:** 2026-06-07
|
||||
**Scope:** UX patterns and UI feature decisions for v0.2. Existing functional features (upload, extraction, sharing, cloud backends, auth) are not re-researched — only their UX surface.
|
||||
|
||||
---
|
||||
**Milestone:** v0.3 Reimagining Cloud Storage integration
|
||||
**Date:** 2026-06-17
|
||||
|
||||
## Table Stakes
|
||||
|
||||
Features users expect. Missing = product feels incomplete or broken.
|
||||
### Cloud-as-Local File Management
|
||||
|
||||
| Feature | Why Expected | Complexity | Depends On |
|
||||
|---------|--------------|------------|------------|
|
||||
| Standalone admin layout (`/admin/*` routes with own sidebar) | AdminView.vue is currently a tab sheet bolted to the user layout. Admin and user contexts share no chrome. Nextcloud, Mayan EDMS, and every self-hosted SaaS treat admin as a distinct area. | Medium | Router nested routes, new `AdminLayout.vue` |
|
||||
| Admin sidebar with discrete nav items (Users, Quotas, AI Config, Audit Log) | Each admin tab is already a full standalone component. Route-based nav replaces in-view tabs — enables deep linking, browser back button, and future per-section permissions. | Low | Vue Router `/admin/users`, `/admin/quotas`, `/admin/ai`, `/admin/audit` child routes |
|
||||
| Empty states for every zero-content context | File list empty, folder empty, search no-results, shared-with-me empty, topics empty — each needs a distinct, actionable state. Current implementation is plain gray text. Zero-content states are the first thing new users see. | Medium | `StorageBrowser.vue`, `SharedView.vue`, `TopicsView.vue` |
|
||||
| Loading skeletons for document list and sidebar tree | The document list (`StorageBrowser`) and sidebar folder tree currently render a plain "Loading…" text string. Skeletons reduce perceived load time by 30–50% on equivalent wait times per usability studies. The `QuotaBar` already uses `animate-pulse` — extend that pattern. | Medium | `StorageBrowser.vue`, `AppSidebar.vue`, new `SkeletonRow.vue` |
|
||||
| Global search focus shortcut (`/` key) | Table stakes for document management tools. Paperless-ngx, Notion, GitHub, Linear all use `/` to focus search. Users who type `/` and nothing happens feel the app is unpolished. | Low | `SearchBar.vue`, `FileManagerView.vue` global `keydown` listener |
|
||||
| Escape key closes modals and clears search | Universal browser convention. Escape should: close `ShareModal`, close `FolderDeleteModal`, close `DocumentPreviewModal`, clear the inline folder rename input, and clear search field. Several of these already handle `keydown.escape` in inputs but not at the modal overlay level. | Low | `ShareModal.vue`, `FolderDeleteModal.vue`, `DocumentPreviewModal.vue` |
|
||||
| Sidebar collapses on mobile (hamburger toggle, overlay drawer) | On viewports below 1024px the fixed 256px sidebar eats 25–40% of the viewport. The current layout has no mobile handling — sidebar overlaps content on small screens. Minimum viable mobile: hamburger button + slide-in overlay drawer. | Medium | `AppSidebar.vue`, `App.vue`, new mobile toggle state |
|
||||
| Responsive document list column hiding | `StorageBrowser` grid already uses `hidden md:block` for Size and `hidden sm:block` for Modified. This is correct — verify it works and document the pattern as intentional. Ensure the icon + name + actions columns always render. | Low | `StorageBrowser.vue` (verify existing classes) |
|
||||
| Upload trigger keyboard shortcut (`U` key when no input is focused) | Secondary table stakes. Windows Explorer, Nextcloud, and Dropbox support pressing a key to trigger upload when focus is on the file list. Makes keyboard-first workflows viable. | Low | `DropZone.vue` or `FileManagerView.vue` global `keydown` listener |
|
||||
| Visible drag-and-drop affordance on the drop zone at all times | `DropZone.vue` already exists and handles dragover/drop. The gap: the drop zone is a full-width dashed box inside the content area, which is correct, but there is no whole-page drag overlay when a user drags a file from the OS file manager. Users miss the target and the drop is ignored. | Medium | `FileManagerView.vue` document-level dragenter/dragleave listener |
|
||||
| Contextual breadcrumb with admin section label | Admin subroutes need breadcrumbs: `Admin > Users`, `Admin > Audit Log`. Without them, admin pages feel unmoored. The user-facing `FolderBreadcrumb` is already built — admin can use the same component with static segments. | Low | `FolderBreadcrumb.vue`, `AdminLayout.vue` |
|
||||
- Browse connected cloud folders with local-like row actions.
|
||||
- Open/preview files through DocuVault authorization.
|
||||
- Upload files into the visible cloud folder.
|
||||
- Rename files and folders where provider supports it.
|
||||
- Move files/folders within the same provider connection.
|
||||
- Delete files/folders with explicit confirmation and audit logging.
|
||||
- Create folders where provider supports it.
|
||||
- Show unsupported actions disabled with a provider-aware reason.
|
||||
|
||||
---
|
||||
### Provider Health and Maintenance
|
||||
|
||||
- Show connection health per provider.
|
||||
- Detect expired/revoked credentials and guide reconnect.
|
||||
- Invalidate folder/index caches after reconnect, delete, move, rename, or upload.
|
||||
- Preserve credentials encryption and never expose secrets.
|
||||
- Surface provider errors as actionable user-facing messages.
|
||||
|
||||
### Analyze Cloud Documents
|
||||
|
||||
- Analyze a selected file, selected folder, or whole provider connection.
|
||||
- Track analysis progress: queued, downloading/caching, extracting, classifying, indexed, failed.
|
||||
- Avoid duplicate analysis when provider item version/etag has not changed.
|
||||
- Allow retry of failed items.
|
||||
- Keep analysis non-destructive: analyzing a cloud file must not move or rewrite it.
|
||||
|
||||
### Smart Search Across Local and Cloud
|
||||
|
||||
- Search should cover local documents plus analyzed cloud documents.
|
||||
- Keyword/sentence search uses extracted text.
|
||||
- Idea/semantic search should be supported through embeddings if v0.3 includes semantic indexing.
|
||||
- Search results identify source: local, OneDrive, Google Drive, Nextcloud, WebDAV.
|
||||
- Opening a cloud result should hydrate bytes on demand if not cached.
|
||||
|
||||
### Storage Minimization
|
||||
|
||||
- Store provider metadata, extracted text, topics, and embeddings in DocuVault.
|
||||
- Cache full file bytes only while needed for preview/extraction/open.
|
||||
- Evict cached bytes by quota/TTL/LRU and never count transient cache bytes as user-owned imported storage.
|
||||
- Keep an explicit user action for permanent import/copy to local DocuVault storage if implemented later.
|
||||
|
||||
## Differentiators
|
||||
|
||||
Features that add genuine value beyond the baseline. Not expected by most users, but valued when present.
|
||||
- Virtual-local status indicators: online-only, cached, analyzing, indexed, stale, error.
|
||||
- Folder-level analyze action with recursive selection preview.
|
||||
- Change detection via provider deltas where available, falling back to scheduled metadata refresh for WebDAV.
|
||||
- Unified local/cloud search with filters for source, provider, folder, topics, status, and analyzed/not analyzed.
|
||||
- Conflict-aware operations when provider etag/version changes between list and mutate.
|
||||
|
||||
| Feature | Value Proposition | Complexity | Depends On |
|
||||
|---------|-------------------|------------|------------|
|
||||
| Whole-page drop zone (drag files anywhere on the screen to upload) | Removes the requirement to aim at the DropZone rectangle. Paperless-ngx supports this. Implementation: listen for `dragenter` at the document level, show a full-screen overlay with the drop target, then pipe the `drop` event into the existing upload emit chain. | Medium | `FileManagerView.vue`, new `PageDropOverlay.vue`, existing `DropZone` emit chain |
|
||||
| Skeleton placeholders shaped like the real content (5-column grid rows) | The QuotaBar has a simple pulse bar. The document list skeleton should be 5-column grid rows matching the actual layout — icon placeholder, name placeholder, size placeholder, date placeholder. This makes the transition from loading to content feel instantaneous rather than a layout jump. | Medium | New `SkeletonRow.vue`, `StorageBrowser.vue` |
|
||||
| Admin dashboard overview page (`/admin` root — stats at a glance) | User count, storage total across all users, recent audit log entries. Gives admins a reason to visit the admin panel rather than jumping straight to sub-sections. Self-hosted tools consistently provide this at the admin root. | Medium | New `AdminOverviewView.vue`, read-only aggregate API endpoints (may require backend work) |
|
||||
| Inline toast/notification system for upload completion and errors | Currently errors appear in the UploadProgress inline list and are never dismissed. A toast system enables transient success/error messages for any action (rename, delete, share) — not just upload. Constraint: keep it simple, not a full notification center. | Medium | New `ToastManager.vue` in `App.vue`, composable `useToast()` |
|
||||
| Document count badge polish on topic sidebar links | Already has `{{ topic.doc_count }}` in the sidebar. What's missing: show 0 counts as dimmed rather than hidden (users want to know the topic exists even if empty), and truncate to "9+" at double digits. | Low | `AppSidebar.vue` (minor change) |
|
||||
| Upload from keyboard shortcut `U` with accessible file picker trigger | The keyboard shortcut fires `triggerInput()` on DropZone. Accessibility requirement: this is equivalent to clicking the hidden file input — screen readers announce the file dialog opening. Already partially supported by `DropZone.vue`'s `triggerInput()` — wire up the global shortcut. | Low | `DropZone.vue`, `FileManagerView.vue` |
|
||||
| Tablet landscape layout: sidebar + content, no collapse needed | At 1024px (lg breakpoint), the sidebar is fully visible alongside content. This is the natural Tailwind `lg:flex` pattern. Verify it and make the breakpoint logic explicit in the layout wrapper. | Low | `App.vue` layout wrapper |
|
||||
## Anti-Features for v0.3
|
||||
|
||||
---
|
||||
|
||||
## Anti-Features
|
||||
|
||||
Features to explicitly NOT build in v0.2.
|
||||
|
||||
| Anti-Feature | Why Avoid | What to Do Instead |
|
||||
|--------------|-----------|-------------------|
|
||||
| Folder reorder by drag-and-drop (drag folder A above folder B to change sidebar order) | Requires a persistent `position` column in the DB, a backend migration, and drag state tracking across the sidebar tree. The sidebar is already collapsible. Payoff is marginal. | Keep alphabetical or creation-time ordering. Users can rename folders to impose their own order (e.g., "1. Contracts"). |
|
||||
| Reorderable sidebar sections (drag "Folders" above "Cloud Storage") | Same DB persistence problem. Section order is fixed and meaningful — user folders first, then external cloud. No self-hosted document manager reorders sections. | Fixed section order in `AppSidebar.vue`. |
|
||||
| Virtual scrolling for the document list | DocuVault is quota-capped at 100 MB per user. At typical document sizes (10–500 KB each), a user has at most a few hundred documents before hitting quota. Virtual scrolling adds significant complexity (dynamic row height, scroll restoration) for an edge case that the quota cap prevents. | Standard `v-for`. Add server-side pagination in a later milestone if needed. |
|
||||
| Rich text document annotations or in-app editing | Already scoped out in `PROJECT.md`. Mentioned here because UX redesigns often attract "what if we added..." scope creep. The preview modal shows PDF; that is the limit. | Stay read-only. Document content is immutable after upload. |
|
||||
| Dark mode toggle | Requires a systematic color token system across all components plus full testing of the inverted palette. The current light-mode palette is inconsistent — fix that first. Tailwind makes dark mode mechanically possible but the v0.2 payoff is low relative to scope. | Design tokens and CSS variables in v0.2 that would support dark mode in a future milestone. |
|
||||
| Drag-and-drop to reorder documents within a folder | Requires a persistent sort order column per user per folder. Documents are naturally sorted by creation date or name — both useful orderings. Custom ordering has high complexity and low frequency of use in a document vault. | Keep the existing sort controls (created_at, name, size). Server-side sort is already implemented. |
|
||||
| Full-screen modal for file upload | The DropZone is embedded in the file browser content area, which is the correct placement. A modal upload workflow adds navigation overhead with no benefit for a simple file picker. | The whole-page drag overlay (differentiator above) is the right scope. |
|
||||
| Notification center / inbox for async events | Building a polling-based notification center (classification done, quota warning) is a full feature, not UX polish. Celery classification runs in the background and its result is visible when the user next opens the document. | Toast on upload completion. Badge count on the document list is sufficient for v0.2. |
|
||||
| Multi-select with batch operations (select 10 files, delete all) | Requires a selection state layer across the file list, a selection-aware toolbar, and confirmed batch-delete UX. Medium-to-high complexity for a feature that is rarely needed in a personal document vault. | Address in a future milestone if user feedback identifies it as a gap. |
|
||||
| Drag documents from file list to sidebar folder entries | Cross-component drag state requires either a Pinia drag store or provide/inject. The sidebar `FolderTreeItem` and `StorageBrowser` have no shared parent that can coordinate state without significant refactoring. The "Move to folder" picker button already covers this use case. | The existing folder picker button in the file row actions menu. |
|
||||
|
||||
---
|
||||
|
||||
## Admin Panel: Specific Pattern Decisions
|
||||
|
||||
### Route structure
|
||||
|
||||
```
|
||||
/admin → AdminLayout.vue wrapper (persistent sidebar + router-view)
|
||||
/admin/ → AdminOverviewView.vue (stats at a glance)
|
||||
/admin/users → AdminUsersView.vue (wraps AdminUsersTab.vue)
|
||||
/admin/quotas → AdminQuotasView.vue (wraps AdminQuotasTab.vue)
|
||||
/admin/ai → AdminAiConfigView.vue (wraps AdminAiConfigTab.vue)
|
||||
/admin/audit → AdminAuditView.vue (wraps AuditLogTab.vue)
|
||||
```
|
||||
|
||||
The existing tab components (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, `AdminAiConfigTab.vue`, `AuditLogTab.vue`) are promoted to route-level views. The `AdminLayout.vue` replaces `AdminView.vue`. The tab strip becomes a sidebar with router-link active styling. No logic changes inside the tab components — only the container changes. `AdminView.vue` is deleted after the migration.
|
||||
|
||||
Vue Router 4 nested routes pattern (HIGH confidence — verified against official docs):
|
||||
|
||||
```js
|
||||
{
|
||||
path: '/admin',
|
||||
component: AdminLayout, // contains <router-view />
|
||||
meta: { requiresAdmin: true },
|
||||
children: [
|
||||
{ path: '', component: AdminOverviewView },
|
||||
{ path: 'users', component: AdminUsersView },
|
||||
{ path: 'quotas', component: AdminQuotasView },
|
||||
{ path: 'ai', component: AdminAiConfigView },
|
||||
{ path: 'audit', component: AdminAuditView },
|
||||
],
|
||||
}
|
||||
```
|
||||
|
||||
The `requiresAdmin` guard moves from the child route to the parent. All children inherit it automatically.
|
||||
|
||||
### Admin sidebar nav items (in order)
|
||||
1. Overview
|
||||
2. Users
|
||||
3. Quotas
|
||||
4. AI Config
|
||||
5. Audit Log
|
||||
|
||||
A "Back to app" link at the bottom of the admin sidebar returns to `/` — admin should not feel like a dead end.
|
||||
|
||||
### Admin sidebar does NOT share code with `AppSidebar.vue`
|
||||
|
||||
`AdminSidebar.vue` is a separate component. It has no folder tree, no cloud tree, no topics list, no quota bar. Those belong to the user interface. Sharing the sidebar component would create coupling between two logically separate contexts and conflict with the "thin view + smart component" architecture described in CLAUDE.md.
|
||||
|
||||
### Breadcrumb
|
||||
|
||||
Use the existing `FolderBreadcrumb.vue` or a simpler static variant: `Admin > [Section Name]`. No dynamic nesting required. The admin sidebar's active link already signals location; the breadcrumb adds skimmability for users who navigate via URL.
|
||||
|
||||
---
|
||||
|
||||
## Empty State Patterns: Specific Decisions
|
||||
|
||||
Each zero-content context gets a distinct implementation. Generic "No items" text is the anti-pattern.
|
||||
|
||||
| Context | Headline | Subtext | CTA |
|
||||
|---------|----------|---------|-----|
|
||||
| Root folder (no folders yet) | "Start organizing your documents" | "Create a folder to group related files together." | "New folder" button |
|
||||
| Folder is empty (exists but has no contents) | "This folder is empty" | "Upload files here or move documents into this folder." | "Upload files" link (triggers DropZone click) |
|
||||
| Search returns no results | "No matches for '{query}'" | "Try a different search term or clear your search." | "Clear search" button |
|
||||
| Shared with me (no shares yet) | "Nothing shared with you yet" | "When someone shares a document with your handle, it appears here." | None (passive state) |
|
||||
| Topics list empty | "No topics yet" | "Topics are created automatically when you upload and classify a document." | None |
|
||||
| Audit log (admin, no entries yet) | "No audit events recorded" | "Events are recorded when users log in, upload documents, or make changes." | None |
|
||||
| Cloud storage — no connections | "No cloud storage connected" | "Connect OneDrive, Google Drive, Nextcloud, or WebDAV in Settings." | "Open Settings" link |
|
||||
|
||||
Rule: every empty state that the user can resolve with an action gets exactly one CTA. Empty states that are passive (shared-with-me before anyone shares, topics before first upload, audit before any events) get explanatory subtext only — no CTA that leads nowhere useful.
|
||||
|
||||
Implementation: extract a shared `EmptyState.vue` presentational component that accepts `headline`, `subtext`, and an optional `action` slot. Use it in all seven contexts above instead of duplicating the layout.
|
||||
|
||||
---
|
||||
|
||||
## Loading State Patterns: Specific Decisions
|
||||
|
||||
| Component | Current | Target | Rationale |
|
||||
|-----------|---------|--------|-----------|
|
||||
| Document list (`StorageBrowser`) | `<div>Loading…</div>` text | 6–8 skeleton rows (5-col grid, `animate-pulse`) | List loads on every folder navigation. Skeletons match the grid layout exactly, preventing layout jump. |
|
||||
| Sidebar folder tree | `<div>Loading…</div>` text | 3 skeleton nav items (indent + placeholder bar) | Sidebar loads once on mount. Short skeleton is sufficient. |
|
||||
| Sidebar topics list | `<div>Loading…</div>` text | 4 skeleton topic pills (color dot + bar) | Same rationale as folder tree. |
|
||||
| Admin user table | No loading state | Skeleton table rows (3–5 rows, matching column count) | Admin tables are data-heavy; skeleton preserves table chrome during load. |
|
||||
| Audit log table | No loading state | Skeleton rows matching timestamp/user/action columns | Same. |
|
||||
| Upload progress item | Spinner + progress bar (correct) | Keep as-is | Upload is a blocking action, not a content load. Spinners are correct for actions. |
|
||||
| QuotaBar | `animate-pulse` bar (correct) | No change | Already implemented correctly. Document as the canonical skeleton pattern. |
|
||||
| Document preview modal | None | Spinner centered in modal body | PDF proxy fetch is a single-request action. Spinner is correct. |
|
||||
|
||||
Rule: if the loading UI represents content the user will read (list, table, tree), use a skeleton. If the loading UI represents a system action (save, upload, submit, process), use a spinner. Never show a loading state that will be visible for less than 300ms — add a minimum display delay.
|
||||
|
||||
---
|
||||
|
||||
## Drag-and-Drop: Scope Boundaries
|
||||
|
||||
### Build in v0.2
|
||||
|
||||
1. **Whole-page drop overlay for file upload** — listen for `dragenter` at the document level, show a full-screen overlay, delegate the `drop` event to the existing DropZone emit chain. The drop must only activate when files are being dragged (check `e.dataTransfer.types.includes('Files')`).
|
||||
|
||||
2. **Move a document to a folder by dragging the file row onto a folder row** — already implemented in `StorageBrowser.vue`. Verify it functions correctly and polish the drop highlight: `ring-2 ring-inset ring-amber-300` on the folder row is correct. Ensure the file row shows reduced opacity during drag (`opacity-50` already applied).
|
||||
|
||||
### Do Not Build in v0.2
|
||||
|
||||
- Dragging files from the file list onto sidebar folder entries (requires cross-component drag state via Pinia or provide/inject — the "Move to folder" picker already covers this)
|
||||
- Touch drag-and-drop on mobile (HTML5 drag events do not fire on iOS Safari without a polyfill library; defer to the move picker button)
|
||||
- Drag folders to reorder them
|
||||
|
||||
### Required for every drag interaction
|
||||
|
||||
- Visual affordance on dragover (already done on DropZone; add to whole-page overlay)
|
||||
- Non-drag alternative for every drag action: the "Move to folder" picker button covers the drag-to-move case
|
||||
- Drop must be clearly confirmed: file row disappears from source location and appears in target on success
|
||||
|
||||
---
|
||||
|
||||
## Keyboard Shortcuts: Final List
|
||||
|
||||
| Shortcut | Action | Condition |
|
||||
|----------|--------|-----------|
|
||||
| `/` | Focus search bar | Not in an input, textarea, or contenteditable |
|
||||
| `Escape` | Close modal or clear active inline input | Modal open, or search focused with content |
|
||||
| `U` | Trigger file picker (upload) | Not in any input, on a folder/file manager view |
|
||||
| `N` | Start new folder inline input | Not in any input, on a folder/file manager view |
|
||||
|
||||
### Do Not Build
|
||||
|
||||
- `J`/`K` for row-by-row navigation — requires focus management, ARIA selected state, and keyboard event bubbling control. Over-engineered for a vault where opening a document is secondary to browsing and uploading.
|
||||
- `Ctrl+A` / `Cmd+A` for select all — requires multi-select, which is an anti-feature.
|
||||
- Arrow key tree navigation in the sidebar — screen readers handle this natively; custom arrow-key navigation would conflict with browser defaults.
|
||||
- `?` shortcut for a shortcut reference modal — too few shortcuts to warrant a help overlay.
|
||||
|
||||
---
|
||||
|
||||
## Responsive Layout: Breakpoint Decisions
|
||||
|
||||
| Viewport | Tailwind breakpoint | Sidebar | Document list | Admin |
|
||||
|----------|---------------------|---------|--------------|-------|
|
||||
| Mobile portrait | `< 640px` (below `sm`) | Hidden; hamburger button + overlay drawer | Icon + name only; Size (`hidden md:block`) and Modified (`hidden sm:block`) hidden | Same drawer |
|
||||
| Mobile landscape / small tablet | `640–767px` (`sm`) | Hidden; hamburger still active | Icon + name + Modified visible; Size hidden | Same |
|
||||
| Tablet portrait | `768–1023px` (`md`) | Hidden; hamburger still active | Full 5-col grid | Full columns |
|
||||
| Tablet landscape / small desktop | `1024px+` (`lg`) | Always visible (256px fixed) | Full 5-col grid | Full admin sidebar + content |
|
||||
| Desktop | `1280px+` (`xl`) | Always visible | Full grid + comfortable padding | Full layout |
|
||||
|
||||
Key decision: sidebar collapses at the `lg` breakpoint (1024px), using `hidden lg:flex` on the sidebar and `lg:hidden` on the hamburger button. At 1024px the sidebar (256px) leaves 768px for content — sufficient for the 5-column grid with room for padding.
|
||||
|
||||
Touch targets: all inline action icon buttons are currently `p-1.5` (approximately 30px). Increase to `p-2` on `sm:` and below using responsive padding classes. 44px is the WCAG target — not achievable for inline buttons without breaking the grid layout, but 36px is a reasonable minimum.
|
||||
|
||||
---
|
||||
|
||||
## Feature Dependencies (summary)
|
||||
|
||||
```
|
||||
Admin route subtree
|
||||
→ AdminLayout.vue (new) contains AdminSidebar.vue (new) + <router-view>
|
||||
→ Child views wrap existing AdminUsersTab, AdminQuotasTab, AdminAiConfigTab, AuditLogTab
|
||||
→ AdminView.vue deleted after migration
|
||||
|
||||
Empty states
|
||||
→ New EmptyState.vue (presentational, accepts headline/subtext/action slot)
|
||||
→ Used in StorageBrowser.vue, SharedView.vue, TopicsView.vue, CloudStorageView.vue, AuditLogTab.vue
|
||||
|
||||
Skeleton rows
|
||||
→ New SkeletonRow.vue (5-col grid, animate-pulse)
|
||||
→ Used in StorageBrowser.vue (document list), AppSidebar.vue (folder tree, topics list)
|
||||
|
||||
Keyboard shortcuts
|
||||
→ Global keydown listener in FileManagerView.vue
|
||||
→ SearchBar.vue exposes focus() method
|
||||
→ DropZone.vue exposes triggerInput() (already exists as triggerInput())
|
||||
→ StorageBrowser.vue exposes startNewFolder() (already defineExposed)
|
||||
|
||||
Mobile sidebar
|
||||
→ App.vue holds sidebarOpen boolean state
|
||||
→ AppSidebar.vue accepts isOpen prop + emits close
|
||||
→ Hamburger button in App.vue header (mobile only, hidden lg:hidden)
|
||||
|
||||
Whole-page drop overlay
|
||||
→ New PageDropOverlay.vue (full-screen overlay, accepts active prop)
|
||||
→ FileManagerView.vue listens for document dragenter/dragleave
|
||||
→ On drop, delegates to existing upload flow via onFilesSelected
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- Vue Router 4 nested routes: https://router.vuejs.org/guide/essentials/nested-routes
|
||||
- Skeleton screens vs. spinners UX: https://www.onething.design/post/skeleton-screens-vs-loading-spinners
|
||||
- Skeleton loading screen design: https://blog.logrocket.com/ux-design/skeleton-loading-screen-design/
|
||||
- SaaS empty state patterns: https://pixxen.com/saas-empty-state-design/
|
||||
- IBM Carbon Design System empty states: https://carbondesignsystem.com/patterns/empty-states-pattern/
|
||||
- Drag-and-drop SaaS UX: https://www.eleken.co/blog-posts/drag-and-drop-ui
|
||||
- Drag-and-drop NN/g guidelines: https://www.nngroup.com/articles/drag-drop/
|
||||
- Responsive SaaS breakpoints: https://tryhoverify.com/blog/responsive-design-breakpoints-2024-guide/
|
||||
- Keyboard shortcut design for web apps: https://sashika.medium.com/j-k-or-how-to-choose-keyboard-shortcuts-for-web-applications-a7c3b7b408ee
|
||||
- Nextcloud layout patterns: https://docs.nextcloud.com/server/latest/developer_manual/design/layout.html
|
||||
- Paperless-ngx frontend architecture: https://deepwiki.com/paperless-ngx/paperless-ngx/2-frontend-architecture
|
||||
- Native desktop sync client.
|
||||
- Offline-first web app for entire cloud drives.
|
||||
- Cross-provider move/copy as a seamless operation.
|
||||
- Real-time collaborative editing.
|
||||
- Public link sharing.
|
||||
- Automatic whole-cloud analysis without explicit user opt-in and cost warning.
|
||||
|
||||
+34
-258
@@ -1,290 +1,66 @@
|
||||
# Domain Pitfalls — v0.2 UI Overhaul and Optimization
|
||||
# v0.3 Research: Pitfalls
|
||||
|
||||
**Project:** DocuVault
|
||||
**Milestone:** v0.2 — UI Overhaul and Optimization on existing FastAPI + Vue 3 app
|
||||
**Researched:** 2026-06-07
|
||||
**Confidence:** HIGH (all pitfalls grounded in actual source files: router/index.js, App.vue, main.py, client.js, StorageBrowser.vue, DocumentCard.vue, DropZone.vue, AppSidebar.vue, and all api/*.py router files)
|
||||
**Milestone:** v0.3 Reimagining Cloud Storage integration
|
||||
**Date:** 2026-06-17
|
||||
|
||||
---
|
||||
## Major Risks
|
||||
|
||||
## Critical Pitfalls
|
||||
### Full Mirror by Accident
|
||||
|
||||
Mistakes that cause regressions, security gaps, or rewrites.
|
||||
If analysis downloads every cloud file into MinIO, user quota and infrastructure cost explode. Treat bytes as temporary cache unless the user explicitly imports a file.
|
||||
|
||||
---
|
||||
Prevention: separate cloud item metadata/index records from local `documents`; add cache limits and eviction tests early.
|
||||
|
||||
### Pitfall 1: FastAPI Router Splitting — Prefix Already Embedded in APIRouter, Not in include_router
|
||||
### Provider Capability Mismatch
|
||||
|
||||
**Phase:** Backend refactoring (split large router files)
|
||||
OneDrive, Google Drive, Nextcloud, and generic WebDAV do not expose identical semantics. Google Workspace files, WebDAV path IDs, OneDrive item IDs, etag behavior, move/rename semantics, and preview support differ.
|
||||
|
||||
**What goes wrong:** Every existing router in this codebase declares its prefix on the `APIRouter` instance itself — `router = APIRouter(prefix="/api/documents", tags=["documents"])` — NOT on the `include_router(router, prefix=...)` call in `main.py`. When splitting a large file (e.g. `documents.py` at 852 lines, `admin.py` at 934 lines) into sub-modules, the new sub-routers pick up their endpoints via an `include_router` call inside the original module, or they are wired directly in `main.py`. Either way, if you add a prefix on `include_router` in addition to the prefix on the inner `APIRouter`, all URLs double-prefix to `/api/documents/api/documents/...`. Likewise, if you strip the prefix from the inner router and forget to add it to `include_router`, all URLs return 404.
|
||||
Prevention: model provider capabilities explicitly and disable unsupported actions with a clear reason.
|
||||
|
||||
**Why it happens:** The original pattern (`router = APIRouter(prefix=...)` in the module + `app.include_router(router)` without a prefix in `main.py`) is unusual but consistent in this project. Developers splitting a file expect the standard pattern where prefixes go on `include_router`, not on the router definition.
|
||||
### Search That Downloads on Query
|
||||
|
||||
**Consequences:** 404 on all endpoints in the split module; existing tests and the frontend `client.js` (which hard-codes `/api/documents`, `/api/admin`, etc.) silently break.
|
||||
Smart search must query persisted extracted text/embeddings. Downloading cloud bytes during search would be slow, expensive, and fragile.
|
||||
|
||||
**Prevention:** When splitting, create sub-routers with NO prefix: `sub = APIRouter(tags=["documents-upload"])`. Register them inside the parent module with `router.include_router(sub)` so the parent's prefix propagates. The parent module keeps its existing prefix and its existing `app.include_router(parent)` call in `main.py` is unchanged.
|
||||
Prevention: search only analyzed/indexed cloud item records; show unanalyzed items as out of search scope until analyzed.
|
||||
|
||||
**Detection warning signs:** Any test that calls `/api/documents/*` returns 404 immediately after the split; Swagger UI shows doubled path segments.
|
||||
### Stale or Wrong Results
|
||||
|
||||
---
|
||||
Cloud files can change outside DocuVault. Search results become misleading if etags/versions are ignored.
|
||||
|
||||
### Pitfall 2: FastAPI Router Splitting — Circular Import via Shared Dependencies
|
||||
Prevention: store last analyzed etag/version, mark stale when provider metadata changes, and re-analyze changed items.
|
||||
|
||||
**Phase:** Backend refactoring
|
||||
### Destructive Action Surprise
|
||||
|
||||
**What goes wrong:** `folders.py` already exports a `document_move_router = APIRouter(prefix="/api/documents")` — a cross-cutting concern documented in the file (line 439-442). When splitting `documents.py`, if the new sub-module imports anything from `api/folders.py` (e.g. a shared validator or Pydantic schema), Python raises `ImportError: cannot import name X` at startup because both modules are in partial initialization state.
|
||||
Delete/move/rename in cloud mode changes the user's real provider storage, not a DocuVault copy.
|
||||
|
||||
**Why it happens:** FastAPI modules that share models and validators tend to grow cross-references. The `document_move_router` in `folders.py` under the `/api/documents` prefix is the existing example — any new import in the opposite direction closes the import cycle.
|
||||
Prevention: clear confirmations for destructive actions, audit logs, ownership checks, and provider-specific error handling.
|
||||
|
||||
**Consequences:** Application fails to start. Docker container exits immediately.
|
||||
### Credential and Token Drift
|
||||
|
||||
**Prevention:** Shared Pydantic models and validators must live in `db/models.py` or a dedicated `schemas/` module — not in router files. Before splitting, grep for any `from api.X import` inside the router files being split and move the shared symbols to a neutral module first.
|
||||
OAuth tokens expire or are revoked. WebDAV passwords/app tokens change. Background jobs may fail long after the user connected the provider.
|
||||
|
||||
**Detection warning signs:** `ImportError` or `cannot import name` in the uvicorn startup log.
|
||||
Prevention: connection health state, reconnect flows, token refresh persistence, and failure states that stop retry storms.
|
||||
|
||||
---
|
||||
### Blocking Event Loop or Worker Exhaustion
|
||||
|
||||
### Pitfall 3: AdminLayout Vue Router Integration — beforeEach Guard Misses /admin/* Children
|
||||
Provider SDKs may be synchronous and cloud downloads can be large.
|
||||
|
||||
**Phase:** Admin panel rearchitecture
|
||||
Prevention: keep sync SDK calls in thread wrappers, stream/chunk large files, use Celery queues/rate limits, and retry external API failures with backoff.
|
||||
|
||||
**What goes wrong:** The current guard in `router/index.js` (lines 81-93) checks `to.meta.requiresAdmin` on the matched route object. The current `/admin` route is a flat route with that meta set directly. When `/admin` becomes a parent route with nested children (`/admin/users`, `/admin/audit`, etc.), the children do NOT automatically inherit `meta: { requiresAdmin: true }` from the parent in Vue Router 4 unless the guard iterates `to.matched`. If the children are added without repeating the meta, the guard's `to.meta.requiresAdmin` check evaluates to `undefined` (falsy) for every child route — non-admin users can navigate to `/admin/users` directly.
|
||||
### SSRF and URL Handling Regression
|
||||
|
||||
**Why it happens:** Vue Router 4 does NOT merge parent `meta` into child `meta` by default. `to.meta` reflects only the deepest matched route's own meta, not the parent's.
|
||||
WebDAV/Nextcloud URLs are user-supplied. v0.1 already added SSRF guards; new action endpoints must preserve them.
|
||||
|
||||
**Consequences:** Security regression — the admin guard that currently works correctly stops protecting all admin child routes. Non-admin users can access `/admin/users`, `/admin/audit`, etc.
|
||||
Prevention: all WebDAV outbound paths go through existing validation utilities; never join raw user path strings into URLs without provider helper methods.
|
||||
|
||||
**Prevention:** Update the guard to check `to.matched.some(r => r.meta.requiresAdmin)` instead of `to.meta.requiresAdmin`. This is the correct Vue Router 4 idiom for inherited meta and requires one change to the guard. Apply the same fix to the `!to.meta.public` check if any auth routes gain children.
|
||||
### Cache Privacy Leak
|
||||
|
||||
```javascript
|
||||
// router/index.js — change:
|
||||
if (to.meta.requiresAdmin && authStore.user?.role !== 'admin') {
|
||||
// to:
|
||||
if (to.matched.some(r => r.meta.requiresAdmin) && authStore.user?.role !== 'admin') {
|
||||
```
|
||||
Cached bytes are sensitive document content. A shared cache path without user/item scoping could leak across users.
|
||||
|
||||
---
|
||||
Prevention: cache keys include user id, connection id, item id, and etag; cache content is never served without current-user ownership checks.
|
||||
|
||||
### Pitfall 4: AdminLayout Integration — App.vue Layout Switch Breaks for /admin/* Routes
|
||||
### Runaway AI Cost
|
||||
|
||||
**Phase:** Admin panel rearchitecture
|
||||
"Analyze all cloud documents" could enqueue thousands of files.
|
||||
|
||||
**What goes wrong:** `App.vue` currently switches between `AuthLayout` (when `route.meta.layout === 'auth'`) and the main app shell (sidebar + `<router-view>`) for everything else. The admin panel needs a third layout (`AdminLayout`) with its own nav. The naive approach — `v-else-if="route.meta.layout === 'admin'"` in `App.vue` rendering `AdminLayout` which then renders `<router-view />` — creates double rendering: `App.vue`'s `<router-view>` already resolved the full route. The inner `<router-view>` inside `AdminLayout` gets nothing because the route is already consumed.
|
||||
|
||||
**Consequences:** Admin layout renders with an empty content area, or the admin page renders without the admin chrome.
|
||||
|
||||
**Prevention:** The correct structure for a standalone admin layout in Vue Router 4:
|
||||
|
||||
```javascript
|
||||
// router/index.js
|
||||
{
|
||||
path: '/admin',
|
||||
component: AdminLayout, // resolves to the layout shell, which contains <router-view />
|
||||
meta: { requiresAdmin: true },
|
||||
children: [
|
||||
{ path: '', redirect: '/admin/users' },
|
||||
{ path: 'users', component: AdminUsersView },
|
||||
{ path: 'quotas', component: AdminQuotasView },
|
||||
{ path: 'ai', component: AdminAiView },
|
||||
{ path: 'audit', component: AdminAuditView },
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
`AdminLayout.vue` contains `<router-view />` where page content renders. `App.vue` remains unchanged — it renders the component that the route resolves to, which is now `AdminLayout`. No third branch in `App.vue` is needed.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 5: client.js Splitting — noRefreshPaths Breaks When Paths Move to Sub-modules
|
||||
|
||||
**Phase:** Frontend refactoring (client.js split)
|
||||
|
||||
**What goes wrong:** `client.js` has a hardcoded `noRefreshPaths` list inside `request()` (line 26) that guards against infinite refresh loops. If `request()` is moved to a shared `api/core.js` module and the path list stays there, it works. But the three blob-download functions (`adminExportAuditLogCsv`, `adminDownloadDailyExport`, `fetchDocumentContent`) each contain a duplicated copy of the 401-refresh-retry logic. When splitting `client.js` into domain files, those duplicates will be copied into new sub-files. This is the correct moment to eliminate them — but if missed, the bug surface multiplies.
|
||||
|
||||
**Additionally:** If components are changed to import from sub-files directly (e.g. `import { listDocuments } from '../api/documents.js'`) instead of the barrel, every other component that imports from `client.js` still works — but a future refactor that renames a function in the sub-file will miss the barrel re-export and break components that still use the barrel. Both import paths exist simultaneously and diverge.
|
||||
|
||||
**Prevention:** Split in this order:
|
||||
1. Extract `request()` + `noRefreshPaths` into `api/core.js`. Export `request` from there.
|
||||
2. Extract the 401-retry pattern from the three blob-download functions into a single `fetchWithRetry(url, headers)` helper also in `api/core.js`.
|
||||
3. Create domain files (`api/documents.js`, `api/admin.js`, `api/auth.js`, etc.) that import from `api/core.js`.
|
||||
4. Keep `api/client.js` as a re-export barrel: `export * from './documents.js'; export * from './admin.js'` etc. This preserves all existing `import { listDocuments } from '../api/client.js'` calls without touching any component. Never change component imports to point at sub-files directly.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 6: Drag-and-Drop on DocumentCard — dragstart vs @click Conflict
|
||||
|
||||
**Phase:** UX improvements (drag-and-drop)
|
||||
|
||||
**What goes wrong:** `DocumentCard.vue` navigates on `@click="$router.push('/document/' + doc.id)"` (line 4). Browsers fire `click` after `dragend` when the drag distance is below the browser's drag-initiation threshold — typically ~4px. A user who clicks and releases without moving far enough triggers both `dragstart` and then `click`: the card navigates to the document even though the user intended to start a drag.
|
||||
|
||||
**Also:** The action buttons inside `DocumentCard` use `@click.stop`. If a drag handle overlaps an action button area, `@click.stop` prevents the click from propagating but does not prevent `dragstart` from bubbling — behavior becomes inconsistent depending on which pixel the user starts from.
|
||||
|
||||
**Note:** `StorageBrowser.vue` already implements drag-and-drop on list rows correctly (lines 141-145: `draggable`, `@dragstart`, `@dragend`) and does not have this problem because the row's `@click` emits `file-open` rather than navigating imperatively. The risk is specific to adding drag to `DocumentCard.vue` (card grid used in `TopicsView`).
|
||||
|
||||
**Prevention:**
|
||||
1. Track drag state with a component-level `dragging` ref. Set `dragging = true` in `@dragstart`, reset to `false` in `@dragend`. Guard `@click`: `if (this.dragging) return`.
|
||||
2. Preferred: use a dedicated drag handle element (grip icon, `cursor-grab`) and set `draggable="true"` only on that element. Add `@click.stop` on the handle. The rest of the card retains click-to-navigate.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 7: Virtual Scrolling — StorageBrowser DOM Assumptions Break
|
||||
|
||||
**Phase:** Frontend performance (virtual scrolling)
|
||||
|
||||
**What goes wrong:**
|
||||
|
||||
`StorageBrowser.vue` renders its item list with `divide-y divide-gray-100` (line 76). The Tailwind `divide-y` utility inserts borders using adjacent-sibling CSS (`& > * + * { border-top-width: 1px }`). This only works when all children are present in the DOM simultaneously. Virtual scroll libraries keep only visible rows in the DOM — `divide-y` produces borders only between the visible subset, leaving gaps at virtual boundaries.
|
||||
|
||||
The folder picker dropdown inside file rows (lines 185-213) is `absolute right-0 top-full` — positioned relative to its parent row. Virtual scroll repositions rows via `transform: translateY()`. Absolutely positioned children escape their transformed parent and appear at wrong viewport positions.
|
||||
|
||||
CSS transitions on rows (`transition-all`, `transition-colors`) work with `v-for` because Vue animates DOM insertion/removal. Virtual scroll does not insert/remove — it repositions. Vue's `<transition-group>` cannot wrap a virtually scrolled list and will throw warnings.
|
||||
|
||||
**Prevention:**
|
||||
1. Replace `divide-y` with `border-b border-gray-100` on each individual row element. Rows become self-contained and do not depend on DOM adjacency.
|
||||
2. The folder picker dropdown must use `<Teleport to="body">` with `position: fixed` coordinates computed from `getBoundingClientRect()`. This is a prerequisite for virtual scroll adoption, not optional.
|
||||
3. Remove any `<transition-group>` wrapping a virtually scrolled list. Use per-row fade-in via a CSS class added on first mount instead.
|
||||
|
||||
---
|
||||
|
||||
## Moderate Pitfalls
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 8: Responsive Layout — w-64 Sidebar vs Flex-1 Content on Mobile
|
||||
|
||||
**Phase:** Responsive / mobile layout
|
||||
|
||||
**What goes wrong:** `App.vue` uses `flex h-screen overflow-hidden` with `AppSidebar` fixed at `w-64 shrink-0`. On screens narrower than ~320px the content area collapses below usable width. `StorageBrowser`'s 5-column grid `grid-cols-[2rem_1fr_6rem_8rem_6rem]` has `hidden sm:block` and `hidden md:block` to hide size and date columns — but the fixed `2rem` icon and `6rem` action columns still consume 128px on either side of the filename, leaving minimal space on a 375px screen.
|
||||
|
||||
**Prevention:**
|
||||
1. On mobile (`< lg`), hide the sidebar. Use a hamburger button in a mobile-only header bar. Toggle with `translate-x-0 / -translate-x-full` transition on the sidebar element.
|
||||
2. The sidebar open/closed state needs to be in a Pinia store ref (or provided from `App.vue`) so the hamburger button in the header, the sidebar overlay, and the main content area all react to it. Do not put it in a component's `data()`.
|
||||
3. On mobile, simplify the grid to `grid-cols-[2rem_1fr_4rem]` — filename and a single action column. Test at 375px explicitly.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 9: Admin Panel + Responsive — Double Padding After Extracting Tab Content
|
||||
|
||||
**Phase:** Admin panel rearchitecture + responsive layout (interaction pitfall)
|
||||
|
||||
**What goes wrong:** `AdminView.vue` wraps all content in `p-8 max-w-5xl mx-auto`. Each tab component (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, etc.) has its own internal padding. When tab components are extracted into standalone page views (`AdminUsersView.vue`, etc.), the `p-8 max-w-5xl mx-auto` wrapper moves to `AdminLayout.vue`. If the extracted page views retain their own inner padding, content is double-padded. If `AdminLayout` does not include the constraint, the new admin pages render full-bleed against the viewport.
|
||||
|
||||
**Prevention:** When extracting tab components into page views, strip all top-level padding and `max-w` from the extracted component — those belong to `AdminLayout`'s content area. The page components should fill 100% of the space provided by the layout slot.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 10: Options API Refactoring — Prop Mutation via v-model
|
||||
|
||||
**Phase:** Frontend refactoring (Vue component decomposition)
|
||||
|
||||
**What goes wrong:** If a component receives a prop and binds it directly to `<input v-model="propName">`, Vue emits a prop mutation warning in development. In production (no warnings), the mutation silently fails to propagate: the component's displayed value updates (local vdom), but the parent never receives the change because no `update:propName` event was emitted.
|
||||
|
||||
**Specific risk in this codebase:** `AdminUsersTab`, `AdminQuotasTab`, and `AuditLogTab` currently own their filter/search state internally. When extracted into standalone views, these may receive initial state from route query params and accidentally bind those props directly to inputs.
|
||||
|
||||
**Prevention:** Any input the component controls must bind to a local `data()` copy initialized from the prop: `data() { return { localFilter: this.initialFilter } }`. Emit `update:initialFilter` on change. Never write `v-model="$props.X"`.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 11: Options API Refactoring — data() vs computed() in Decomposed Components
|
||||
|
||||
**Phase:** Frontend refactoring
|
||||
|
||||
**What goes wrong:** When extracting a large component, logic that was a `computed()` in the parent often gets copy-pasted into a child's `data()`. A `data()` value initializes once at mount — it does not react to prop changes. If the parent later passes a different prop value (e.g. the document list changes after a search), the child's stale `data()` copy never updates.
|
||||
|
||||
**Prevention:** For every value in a newly created component, ask: "If the parent passes a different value for this prop, should the child update?" Yes → `computed()`. No → `data()`. Initialized from prop but locally modified afterward → `data()` with a `watch` on the prop to reset when it changes externally.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 12: Keyboard Shortcuts — Conflict with Text Inputs and Browser Defaults
|
||||
|
||||
**Phase:** UX improvements (keyboard shortcuts)
|
||||
|
||||
**What goes wrong:** A global `keydown` listener on `window` fires regardless of focused element. Shortcuts like `/` (focus search), `n` (new folder), `Delete` (delete selected) are also typed characters. They must not fire when any `<input>`, `<textarea>`, `<select>`, or `[contenteditable]` element has focus. `Escape` conflicts with native `<dialog>` cancel events if `DocumentPreviewModal` uses a native dialog — both fire, causing a double-close bug.
|
||||
|
||||
**Prevention:**
|
||||
|
||||
```javascript
|
||||
function onKeydown(e) {
|
||||
const tag = document.activeElement?.tagName
|
||||
const isEditing = ['INPUT', 'TEXTAREA', 'SELECT'].includes(tag)
|
||||
|| document.activeElement?.isContentEditable
|
||||
if (isEditing) return
|
||||
|
||||
if (e.key === '/' && !e.ctrlKey && !e.metaKey) {
|
||||
e.preventDefault()
|
||||
searchInputRef.value?.focus()
|
||||
}
|
||||
}
|
||||
// Options API: register in mounted(), remove in beforeUnmount()
|
||||
// Omitting the removeEventListener is the most common bug in this pattern.
|
||||
```
|
||||
|
||||
Never call `preventDefault()` on `F5`, `Backspace`, `Ctrl+F`, or `Cmd+F` — these are browser-reserved. Use a Pinia store to track which shortcut context is active (file manager, document view, modal) so modal shortcuts suppress file manager shortcuts rather than both firing.
|
||||
|
||||
---
|
||||
|
||||
## Minor Pitfalls
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 13: Loading Skeletons — Count Mismatch Causes Layout Shift
|
||||
|
||||
**Phase:** UX improvements (loading skeletons)
|
||||
|
||||
**What goes wrong:** `StorageBrowser.vue` already has a `loading` prop (line 244) that shows "Loading…" text. Replacing this with skeleton rows requires a skeleton count. Hardcoding 5 rows when `per_page=20` causes a jump; using `per_page` as the skeleton count when only 3 items exist also jars. If skeleton rows are different heights from actual rows, the layout shifts when real content loads — this is a Core Web Vitals issue.
|
||||
|
||||
**Prevention:** Render skeletons at the previous page's actual item count (stored in the parent store). Default to 6-8 on first load. Skeleton rows must match the height of `StorageBrowser`'s actual grid rows exactly — use the same `px-4 py-2.5 grid-cols-[2rem_1fr_6rem_8rem_6rem]` structure with gray placeholder blocks.
|
||||
|
||||
---
|
||||
|
||||
### Pitfall 14: Tailwind Purging — Dynamic Class Names in Refactored Components
|
||||
|
||||
**Phase:** Visual redesign (Tailwind component system)
|
||||
|
||||
**What goes wrong:** Vite + Tailwind purges unused CSS at build time by scanning template strings for class names. Dynamic class names constructed at runtime (e.g. `'text-' + color + '-500'`) are invisible to the purger. In development all classes are present; in production the dynamic classes are stripped and topics/provider badges render without color.
|
||||
|
||||
**Specific risk:** `formatters.js` functions `providerColor`, `providerBg`, `providerLabel` return class strings. If those strings are not present as complete literals anywhere in the scanned source, they are purged.
|
||||
|
||||
**Prevention:** Add a `safelist` in `tailwind.config.js`:
|
||||
```javascript
|
||||
safelist: [
|
||||
{ pattern: /bg-(blue|green|purple|orange|gray|indigo|red)-(50|100|500|600)/ },
|
||||
{ pattern: /text-(blue|green|purple|orange|gray|indigo|red)-(500|600|700)/ },
|
||||
]
|
||||
```
|
||||
Alternatively, use a lookup object of complete class strings rather than string concatenation — Tailwind can scan complete strings in object literals.
|
||||
|
||||
---
|
||||
|
||||
## Integration Pitfalls (Where Two Changes Interact Badly)
|
||||
|
||||
---
|
||||
|
||||
### Integration Pitfall A: Admin Rearchitecture + client.js Splitting — Wrong Order Causes Test Blindness
|
||||
|
||||
**Phases:** Admin panel rearchitecture + frontend refactoring
|
||||
|
||||
**What goes wrong:** If `client.js` is split into domain sub-files before the admin routes are moved to `/admin/*`, the new `api/admin.js` sub-file gets created. Components are then updated to point at the new route structure. But if the split happened first and components were simultaneously moved to import from `api/admin.js` directly (bypassing the barrel), the existing tests (which mock `client.js` imports) no longer cover the new import path. Tests pass, bugs hide.
|
||||
|
||||
**Prevention:** Split `client.js` using the re-export barrel pattern first. Verify all existing tests still pass against the barrel. Then rearchitect admin routes. The barrel acts as a compatibility shim throughout both phases. Only remove the barrel after all tests are updated to import from sub-files.
|
||||
|
||||
---
|
||||
|
||||
### Integration Pitfall B: Virtual Scrolling + Drag-and-Drop Cannot Be Added Simultaneously
|
||||
|
||||
**Phases:** Frontend performance + UX improvements
|
||||
|
||||
**What goes wrong:** Virtual scroll and HTML5 drag-and-drop between rows both depend on the same DOM elements. Virtual scroll keeps only visible rows in the DOM. A user dragging a file toward a folder that is currently scrolled out of the virtual window will never trigger `dragover` on that folder row — it is not in the DOM. `StorageBrowser.vue`'s existing drag-into-folder implementation (lines 88-90) relies on `@dragover.prevent` on folder rows. In a virtual list, off-screen folder rows do not receive this event.
|
||||
|
||||
**Prevention:** Decide the list interaction model before implementing either feature. If virtual scrolling is added, redesign drag-to-folder: the sidebar folder tree (always fully rendered, outside the virtual list) becomes the drop target. This is a UX design decision that must be made before either feature is implemented, not a detail to resolve mid-phase.
|
||||
|
||||
---
|
||||
|
||||
### Integration Pitfall C: Responsive Sidebar + Admin Guard Race on Mobile Page Load
|
||||
|
||||
**Phases:** Responsive layout + admin panel rearchitecture
|
||||
|
||||
**What goes wrong:** The current `beforeEach` guard correctly awaits `authStore.refresh()` before checking `authStore.user?.role` (router lines 82-88). This is correct. The risk emerges when adding nested admin child routes: a developer adding a component-level `created()` or `onMounted()` auth check in an admin page component, trying to guard against unauthorized access "just in case." If that component check does not await the refresh (because the token is memory-only and gone on mobile page reload), `authStore.user` is null at that point, the component check redirects the user to `/login`, and the correct guard never gets to run. The result is that admins are bounced to login on every mobile page reload.
|
||||
|
||||
**Prevention:** The single `beforeEach` guard is the only place that should call `refresh()` or check roles. Page components must not independently re-check auth. If a component needs the current user, it reads from `authStore.user` — which is guaranteed to be populated by the time the component mounts, because the guard has already awaited the refresh.
|
||||
Prevention: preflight count/size estimate, explicit confirmation, quotas/rate limits, progress UI, cancellation, and retry controls.
|
||||
|
||||
+33
-307
@@ -1,322 +1,48 @@
|
||||
# Technology Stack — DocuVault v0.2: UI Overhaul and Optimization
|
||||
# v0.3 Research: Stack
|
||||
|
||||
**Milestone:** v0.2 — UI Overhaul and Optimization
|
||||
**Researched:** 2026-06-07
|
||||
**Scope:** New additions only. The v0.1 stack (FastAPI, Vue 3, Pinia, Vue Router 4, Vite, Tailwind CSS, PostgreSQL, MinIO) is unchanged and not re-researched here.
|
||||
**Milestone:** v0.3 Reimagining Cloud Storage integration
|
||||
**Date:** 2026-06-17
|
||||
|
||||
---
|
||||
## Recommendation
|
||||
|
||||
## Existing Stack Baseline (Confirmed, Do Not Change)
|
||||
Keep the existing FastAPI + SQLAlchemy + PostgreSQL + MinIO + Celery + Vue stack. Add a first-class cloud metadata/index layer and a bounded byte cache instead of adding a separate sync server, filesystem mount, or full local mirror.
|
||||
|
||||
| Component | Current Version (package.json / requirements.txt) | Notes |
|
||||
|-----------|--------------------------------------------------|-------|
|
||||
| Vue | `^3.4.0` | Must bump to `^3.5.0` (see VueUse note below) |
|
||||
| Vite | `^5.2.0` | Must bump to `^6.0.0` — Vite 8 is latest; skip 8 (too new) |
|
||||
| Tailwind CSS | `^3.4.0` | Stay on v3; v4 is out but requires config rewrite |
|
||||
| Pinia | `^2.1.0` | No change needed |
|
||||
| Vue Router | `^4.3.0` | No change needed |
|
||||
## Existing Pieces to Extend
|
||||
|
||||
---
|
||||
- `backend/storage/base.py`: current `StorageBackend` is object-storage shaped. v0.3 needs a second interface or capability extension for file-manager operations: list, open/download stream, upload to path, rename, move, delete, mkdir, stat, and provider change cursors where available.
|
||||
- `backend/api/cloud.py`: currently owns connect/list folders. It should split into focused modules as v0.2 did for admin/documents/auth once behavior expands.
|
||||
- `backend/services/cloud_cache.py`: current TTL folder-listing cache is metadata-only. Keep it for short folder listings, but add a durable DB-backed cloud item index and a separate byte-cache service for extraction/preview.
|
||||
- `backend/tasks/document_tasks.py`: extraction/classification already retrieves bytes from cloud backends for cloud-backed `Document` rows. v0.3 should add dedicated cloud analysis jobs that create/update index records from existing provider files without requiring upload into MinIO.
|
||||
- `frontend/src/components/storage/StorageBrowser.vue`: already unifies local/cloud browsing. v0.3 should make actions capability-aware instead of `mode === "local"` gated.
|
||||
|
||||
## New Frontend Dependencies
|
||||
## Provider APIs
|
||||
|
||||
### 1. VueUse — `@vueuse/core` + `@vueuse/integrations`
|
||||
- Microsoft Graph `driveItem` supports list children, update item, upload, download, delete, move, copy, search, preview, and delta tracking. It exposes eTags/cTags and short-lived download URLs that must not be cached. Source: https://learn.microsoft.com/en-us/graph/api/resources/driveitem?view=graph-rest-1.0
|
||||
- Google Drive API supports resumable uploads and file metadata operations. v0.3 should normalize Drive file IDs, parent IDs, MIME types, and Workspace-document export limitations. Source: https://developers.google.com/workspace/drive/api/guides/manage-uploads
|
||||
- Nextcloud/WebDAV supports standard `PROPFIND`, `PUT`, `GET`, `DELETE`, `MOVE`, `COPY`, and metadata response headers such as `OC-Etag` and `OC-FileId`. Source: https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/basic.html
|
||||
|
||||
**Recommended:** `@vueuse/core@^14.3.0`, `@vueuse/integrations@^14.3.0`
|
||||
**Confidence:** HIGH (verified from Context7 docs and npm registry)
|
||||
## Cache and Index Stack
|
||||
|
||||
**Why:** VueUse provides three composables this milestone needs, each saving 30-100 lines of custom code that would otherwise require careful browser-compatibility testing:
|
||||
- Store cloud item metadata in PostgreSQL, not Redis: provider, connection, provider item id/path, parent id/path, name, MIME/content type, size, etag/content hash, mtime, folder/file flag, provider capabilities, analysis/index status, last seen at, and error state.
|
||||
- Store extracted text and classification/search metadata in PostgreSQL, linked to either local `documents` or cloud item records. Avoid copying full bytes to MinIO unless the user explicitly imports or provider capability requires it.
|
||||
- Use MinIO or a local temp directory only as a bounded byte cache for preview/extraction. Cache entries need owner, provider item version, byte size, last access, and eviction status.
|
||||
- Redis can remain a coordination/cache primitive. If used for byte-cache metadata or job locks, configure memory limits and eviction intentionally; Redis recommends choosing policies such as `allkeys-lru` for cache-like workloads. Source: https://redis.io/docs/latest/develop/reference/eviction/
|
||||
|
||||
- `useDropZone` — drop-target zones for drag-and-drop file upload (handles `dragenter`/`dragleave`/`drop`, exposes `isOverDropZone` reactive ref, correct `dataTypes` filtering, Safari limitations documented)
|
||||
- `onKeyStroke` — keyboard shortcut registration without manual `addEventListener`/`removeEventListener` lifecycle management
|
||||
- `useSortable` (from `@vueuse/integrations`) — thin wrapper over SortableJS for list/folder reordering, auto-syncs to reactive array
|
||||
## Background Jobs
|
||||
|
||||
**Bundle impact:** `@vueuse/core` unpacked size is ~870 KB but it is fully tree-shakeable — only imported composables are included in the bundle. Each composable is typically 1-3 KB gzipped. This is the correct way to import:
|
||||
- Use Celery for analysis, indexing, cache hydration, cache eviction, and provider refresh jobs. Celery supports autoretry with exponential backoff and jitter for external services. Source: https://docs.celeryq.dev/en/stable/userguide/tasks.html
|
||||
- Jobs should be idempotent by `(connection_id, provider_item_id, version/etag)` to survive retries, duplicate selections, and provider race conditions.
|
||||
|
||||
```js
|
||||
import { useDropZone, onKeyStroke } from '@vueuse/core'
|
||||
import { useSortable } from '@vueuse/integrations/useSortable'
|
||||
```
|
||||
## Search Stack
|
||||
|
||||
**Critical version constraint:** VueUse v14.0+ requires `vue@^3.5.0`. The current `package.json` pins `^3.4.0`. Vue 3.5 has no breaking changes for Options API components (confirmed from official Vue blog — purely additive release: reactivity improvements, -56% memory usage). Bump Vue to `^3.5.0` as part of this milestone's first phase.
|
||||
- Keep PostgreSQL full-text search for keyword/sentence search over extracted text.
|
||||
- Add semantic search as a scoped phase using `pgvector` if the project wants idea search in v0.3. pgvector keeps vectors in Postgres, supports exact/approximate nearest-neighbor search, and avoids introducing a separate vector database. Source: https://github.com/pgvector/pgvector
|
||||
- Semantic indexing should be optional per analyzed item and provider-independent.
|
||||
|
||||
**`@vueuse/integrations` peer deps:** `useSortable` requires `sortablejs@^1` as a peer dependency. Install `sortablejs` alongside.
|
||||
## What Not to Add
|
||||
|
||||
```bash
|
||||
npm install @vueuse/core@^14.3.0 @vueuse/integrations@^14.3.0 sortablejs@^1.15.7
|
||||
npm install -D @types/sortablejs
|
||||
```
|
||||
|
||||
**Do NOT use VueUse as a global plugin** (`app.use(VueUse)`) — it doesn't have one. Import composables individually, always.
|
||||
|
||||
---
|
||||
|
||||
### 2. vue-virtual-scroller — `vue-virtual-scroller`
|
||||
|
||||
**Recommended:** `vue-virtual-scroller@^3.0.4`
|
||||
**Confidence:** HIGH (verified from Context7 docs and npm registry; published 2026-05-20, actively maintained)
|
||||
|
||||
**Why:** Document lists can reach hundreds or thousands of entries per user. Without virtual scrolling, rendering all `DocumentCard` components simultaneously degrades scroll performance significantly on mobile. `vue-virtual-scroller` is the canonical Vue 3 virtual list library by Guillaume Chau (Akryum, Vue core team contributor). It ships ESM-only, is Vite-native, and requires Vue 3.3+ (compatible with our 3.5 bump).
|
||||
|
||||
**Components used:**
|
||||
- `<RecycleScroller>` — fixed item height, highest performance. Use for the main document grid where item height is deterministic.
|
||||
- `<DynamicScroller>` — variable item height. Use only if needed (e.g. list view with text previews of different lengths).
|
||||
|
||||
**Bundle size:** Unpacked ~461 KB; ESM-only means Rollup tree-shakes unused components. Requires importing its CSS: `import 'vue-virtual-scroller/index.css'`.
|
||||
|
||||
**Integration note:** Because `RecycleScroller` pools and reuses DOM nodes, every item component must be fully reactive from props alone — no internal `mounted()` side effects that assume a fresh component per item. Options API components that follow the props-down / events-up pattern (as required by the existing architecture) are fully compatible.
|
||||
|
||||
```bash
|
||||
npm install vue-virtual-scroller@^3.0.4
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 3. Drag-and-Drop for File Upload — Native HTML5 + VueUse `useDropZone`
|
||||
|
||||
**Recommended:** No additional library. Use `useDropZone` from `@vueuse/core` (already added above).
|
||||
**Confidence:** HIGH
|
||||
|
||||
**Why not a dedicated library:** The drag-and-drop requirement for this milestone is drop-zone file upload (files dragged from OS to a browser area). `useDropZone` covers this completely. Dedicated upload libraries like `dropzone-vue` or `vue-file-uploader` are either poorly maintained, opinionated about UI, or duplicate what the existing upload flow already does. The existing backend upload endpoint and progress tracking are already implemented — adding a library that wraps both drop detection and upload would require replacing working code.
|
||||
|
||||
**For folder/list reordering** (drag document into a different folder), use `useSortable` from `@vueuse/integrations` (already added above via SortableJS). SortableJS handles this precisely: it is a low-level, pure-DOM drag-sort library with no Vue-specific assumptions.
|
||||
|
||||
**Do NOT use `vue-draggable-plus` (0.6.1, last published 2026-01-11):** It wraps SortableJS but adds a Vue component abstraction layer that conflicts with the project's data-provider view pattern. It also adds bundle overhead that `useSortable` avoids. The last publish date of January 2026 is acceptable for a stable library, but there is no reason to prefer it over the VueUse integration that is already in the bundle.
|
||||
|
||||
---
|
||||
|
||||
### 4. Keyboard Shortcuts — VueUse `onKeyStroke`
|
||||
|
||||
**Recommended:** `onKeyStroke` from `@vueuse/core` (already added). No additional library.
|
||||
**Confidence:** HIGH
|
||||
|
||||
**Why no dedicated library:** Libraries like `vue-shortkey` or `hotkeys-js` exist but are unnecessary overhead. `onKeyStroke` from VueUse handles:
|
||||
- Single key listening: `onKeyStroke('/', handler)` (focus search)
|
||||
- Modifier combos: check `event.ctrlKey` / `event.metaKey` inside the handler
|
||||
- Automatic cleanup when the component is unmounted (it wraps `addEventListener`/`removeEventListener` with `onUnmounted`)
|
||||
|
||||
For a document manager, typical shortcuts are: `/` (search focus), `Escape` (close modal/panel), `Delete` (delete selected), `Enter` (open/confirm). All fit `onKeyStroke` with no wrapper library.
|
||||
|
||||
**Implementation pattern:**
|
||||
```js
|
||||
// In setup() or a composable
|
||||
import { onKeyStroke } from '@vueuse/core'
|
||||
|
||||
onKeyStroke('/', (e) => {
|
||||
e.preventDefault()
|
||||
searchInput.value?.focus()
|
||||
})
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### 5. Loading Skeletons — Pure Tailwind CSS
|
||||
|
||||
**Recommended:** No library. Use `animate-pulse` utility with placeholder shapes.
|
||||
**Confidence:** HIGH
|
||||
|
||||
**Why no library:** Skeleton loaders in this app are simple rectangular placeholders — cards, list rows, and a few text lines. Tailwind's `animate-pulse` combined with `bg-gray-200 dark:bg-gray-700` rounded shapes is sufficient and produces zero additional bundle weight. A skeleton library (`vue3-skeleton`, etc.) would add a dependency for something that is 5 lines of Tailwind HTML per component.
|
||||
|
||||
**Recommended pattern for a document card skeleton:**
|
||||
```html
|
||||
<div class="animate-pulse rounded-lg border border-gray-200 p-4">
|
||||
<div class="h-4 bg-gray-200 rounded w-3/4 mb-2"></div>
|
||||
<div class="h-3 bg-gray-200 rounded w-1/2"></div>
|
||||
</div>
|
||||
```
|
||||
|
||||
For a shimmer effect (more polished than pulse), use a CSS `@keyframes` background-position animation — still zero JS dependency. Only add a library if shimmer is required across 10+ distinct skeleton shapes that need consistent management.
|
||||
|
||||
---
|
||||
|
||||
### 6. Tailwind CSS Plugins
|
||||
|
||||
**Recommended:** `@tailwindcss/forms@^0.5.11`
|
||||
**Confidence:** HIGH (official Tailwind Labs plugin, 4,500+ GitHub stars, supports both v3 and v4)
|
||||
|
||||
**Why `@tailwindcss/forms`:** DocuVault has several form-heavy screens (login, registration, upload dialog, admin user management, cloud credential entry). Browser defaults for `<input>`, `<select>`, `<textarea>`, `<checkbox>`, and `<radio>` are inconsistent across platforms and resist Tailwind utility styling. `@tailwindcss/forms` provides a minimal, accessible reset that makes all form elements style-consistent and fully overridable with utilities. Unpacked size is 55 KB; it is a PostCSS plugin with zero runtime JS.
|
||||
|
||||
```bash
|
||||
npm install -D @tailwindcss/forms@^0.5.11
|
||||
```
|
||||
|
||||
```js
|
||||
// tailwind.config.js
|
||||
plugins: [require('@tailwindcss/forms')]
|
||||
```
|
||||
|
||||
**Container queries (`@tailwindcss/container-queries`):** Do NOT add. In Tailwind v3, this would be the `@tailwindcss/container-queries` plugin. However, the v0.2 responsive layout target is standard viewport-based breakpoints (`sm:`, `md:`, `lg:`). Container queries are useful when the same component is rendered in contexts of different widths (sidebar vs. main content), but the DocuVault component architecture uses the data-provider view pattern where layout is controlled by the view, not by individual components. This eliminates the primary use case for container queries in this codebase. If a future phase introduces a component genuinely embedded in variable-width containers, add the plugin then.
|
||||
|
||||
**Typography plugin (`@tailwindcss/typography`):** Do NOT add. There is no long-form rich text content in this app — document content is displayed in a preview iframe, not rendered as styled HTML.
|
||||
|
||||
---
|
||||
|
||||
### 7. Bundle Analysis — `rollup-plugin-visualizer`
|
||||
|
||||
**Recommended:** `rollup-plugin-visualizer@^7.0.1` (dev dependency, build-time only)
|
||||
**Confidence:** MEDIUM-HIGH (standard tool, actively maintained, Vite-native)
|
||||
|
||||
**Why:** The bundle size reduction goal requires knowing what is currently in the bundle. `rollup-plugin-visualizer` generates an interactive treemap that shows each module's contribution. Run once at the start of v0.2 to establish baseline, and again after refactoring to verify improvement. It does not affect the production build.
|
||||
|
||||
```bash
|
||||
npm install -D rollup-plugin-visualizer@^7.0.1
|
||||
```
|
||||
|
||||
```js
|
||||
// vite.config.js — enable only when analyzing, not in every build
|
||||
import { visualizer } from 'rollup-plugin-visualizer'
|
||||
|
||||
export default defineConfig({
|
||||
plugins: [
|
||||
vue(),
|
||||
process.env.ANALYZE ? visualizer({ open: true, filename: 'dist/stats.html' }) : null,
|
||||
].filter(Boolean),
|
||||
})
|
||||
```
|
||||
|
||||
Run with: `ANALYZE=1 npm run build`
|
||||
|
||||
---
|
||||
|
||||
## Vite Upgrade
|
||||
|
||||
**Recommended:** Bump from `^5.2.0` to `^6.4.3` (the stable Vite 6 release; avoid Vite 7/8 beta)
|
||||
**Confidence:** HIGH (Vite 6 released 2024-11; Vite 7 and 8 are in beta as of 2026-06)
|
||||
|
||||
**Why Vite 6 over staying on Vite 5:** Vite 6 is a stable, minimal-breaking-change major release. Breaking changes for this project are:
|
||||
1. **Node.js ≥ 20.0.0 required** — Docker Compose service should already use Node 20+ (verify `FROM node:20-alpine` or equivalent in any Dockerfile that runs the frontend build).
|
||||
2. **No other breaking changes apply** to this project (CSS library output file rename only affects library authors, not app builds; SSR CSS change is not applicable).
|
||||
|
||||
Vite 6 delivers improved dependency pre-bundling and module runner improvements relevant to the performance goals. The built-in code splitting (`import()` dynamic imports for lazy routes) and tree-shaking require no config changes — they are on by default. The manual chunking config for vendor splitting (Pinia, Vue Router) in `build.rollupOptions.output.manualChunks` can be used if the baseline analysis shows large vendor chunks being repeated across async routes.
|
||||
|
||||
**Do NOT upgrade to Vite 7 or 8** — both are in beta as of this research date and are not appropriate for a production milestone.
|
||||
|
||||
```bash
|
||||
npm install -D vite@^6.4.3 @vitejs/plugin-vue@^5.0.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Vue Version Bump
|
||||
|
||||
**Required:** `^3.4.0` → `^3.5.0`
|
||||
**Confidence:** HIGH (no breaking changes for Options API; confirmed from official Vue 3.5 blog post)
|
||||
|
||||
This is required by `@vueuse/core@^14.0`. Vue 3.5 is a drop-in upgrade for Options API components — the release is purely additive (reactivity performance improvements, `useTemplateRef()` for Composition API). No component changes are needed.
|
||||
|
||||
```bash
|
||||
npm install vue@^3.5.0
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Backend — No New Python Packages Needed
|
||||
|
||||
**Confidence:** HIGH
|
||||
|
||||
FastAPI's `APIRouter` (already used) is the correct and sufficient tool for router decomposition. Splitting large router files into smaller modules uses only `APIRouter(prefix=..., tags=[...])` and `app.include_router()` — both already in the project. No new library is warranted.
|
||||
|
||||
**Pattern for large router decomposition:**
|
||||
```
|
||||
backend/routers/
|
||||
documents/
|
||||
__init__.py # from .upload import router as upload_router; etc.
|
||||
upload.py # APIRouter for POST /documents
|
||||
search.py # APIRouter for GET /documents
|
||||
share.py # APIRouter for share endpoints
|
||||
admin/
|
||||
__init__.py
|
||||
users.py
|
||||
audit.py
|
||||
settings.py
|
||||
```
|
||||
|
||||
Include in `main.py`:
|
||||
```python
|
||||
from routers.documents import router as documents_router
|
||||
from routers.admin import router as admin_router
|
||||
|
||||
app.include_router(documents_router, prefix="/documents", tags=["documents"])
|
||||
app.include_router(admin_router, prefix="/admin", tags=["admin"])
|
||||
```
|
||||
|
||||
This is a pure refactor — no new dependencies, no behavior change, no test changes except updating import paths.
|
||||
|
||||
---
|
||||
|
||||
## Full v0.2 Dependency Diff
|
||||
|
||||
### `npm install` (production dependencies)
|
||||
|
||||
```bash
|
||||
npm install \
|
||||
vue@^3.5.0 \
|
||||
@vueuse/core@^14.3.0 \
|
||||
@vueuse/integrations@^14.3.0 \
|
||||
sortablejs@^1.15.7 \
|
||||
vue-virtual-scroller@^3.0.4
|
||||
```
|
||||
|
||||
### `npm install -D` (dev dependencies)
|
||||
|
||||
```bash
|
||||
npm install -D \
|
||||
vite@^6.4.3 \
|
||||
@vitejs/plugin-vue@^5.0.0 \
|
||||
@tailwindcss/forms@^0.5.11 \
|
||||
rollup-plugin-visualizer@^7.0.1 \
|
||||
@types/sortablejs@^1.15.0
|
||||
```
|
||||
|
||||
### Python backend additions
|
||||
|
||||
**None.** All backend work (router decomposition, service extraction) uses FastAPI primitives already present.
|
||||
|
||||
---
|
||||
|
||||
## Alternatives Considered and Rejected
|
||||
|
||||
| Category | Rejected | Reason |
|
||||
|----------|----------|--------|
|
||||
| Drag-and-drop file upload | `dropzone-vue`, `vue-file-uploader` | Opinionated UI, poor maintenance, duplicates existing backend upload flow |
|
||||
| Drag-and-drop sorting | `vue-draggable-plus@0.6.1` | Adds Vue component abstraction over SortableJS that conflicts with data-provider view pattern; VueUse `useSortable` covers the same need with less surface area |
|
||||
| Virtual scrolling | `@tanstack/vue-virtual` | Lower-level API requiring more integration code than `vue-virtual-scroller`; no meaningful bundle advantage; vue-virtual-scroller is Vue-native |
|
||||
| Keyboard shortcuts | `hotkeys-js`, `vue-shortkey` | Unnecessary wrapper over `addEventListener`; VueUse `onKeyStroke` is sufficient and already in the bundle |
|
||||
| Loading skeletons | `vue3-skeleton` | 5-line Tailwind pattern produces identical results; no justification for an additional dependency |
|
||||
| Tailwind container queries | `@tailwindcss/container-queries` | Viewport breakpoints are sufficient for this app's layout; container queries solve a problem this codebase does not have |
|
||||
| Tailwind typography | `@tailwindcss/typography` | No prose HTML content in this app |
|
||||
| Vite 7 / 8 | Latest beta | Both in beta as of research date; not appropriate for a production milestone |
|
||||
|
||||
---
|
||||
|
||||
## Compatibility Matrix (v0.2 additions)
|
||||
|
||||
| Package | Version | Vue Requirement | Vite Compatibility | Tree-Shakeable |
|
||||
|---------|---------|-----------------|-------------------|----------------|
|
||||
| `@vueuse/core` | ^14.3.0 | ^3.5.0 | Vite 5+ | Yes — import individually |
|
||||
| `@vueuse/integrations` | ^14.3.0 | ^3.5.0 | Vite 5+ | Yes |
|
||||
| `sortablejs` | ^1.15.7 | — (plain JS) | Any | Yes |
|
||||
| `vue-virtual-scroller` | ^3.0.4 | ^3.3.0 | Vite (ESM) | Partial (register components individually) |
|
||||
| `@tailwindcss/forms` | ^0.5.11 | — (PostCSS) | Any | N/A (CSS plugin) |
|
||||
| `rollup-plugin-visualizer` | ^7.0.1 | — (build tool) | Vite 5+ | N/A (dev only) |
|
||||
| `vue` | ^3.5.0 | — | — | — |
|
||||
| `vite` | ^6.4.3 | — | — | — |
|
||||
|
||||
---
|
||||
|
||||
## Sources
|
||||
|
||||
- VueUse v14.3.0 docs and peer dependencies: Context7 `/vueuse/vueuse`, verified via npm registry (2026-06-07)
|
||||
- vue-virtual-scroller v3.0.4: Context7 `/akryum/vue-virtual-scroller`, npm registry (last published 2026-05-20)
|
||||
- vue-draggable-plus v0.6.1: npm registry (last published 2026-01-11) — rejected
|
||||
- sortablejs v1.15.7: npm registry (2026-06-07)
|
||||
- Tailwind CSS container queries: official Tailwind CSS announcement — built into v4 core; plugin for v3 would be `@tailwindcss/container-queries` — rejected as out of scope
|
||||
- @tailwindcss/forms: official Tailwind Labs GitHub (tailwindlabs/tailwindcss-forms), 4,500+ stars, supports v3 and v4
|
||||
- rollup-plugin-visualizer: npm registry, v7.0.1 (2026-06-07)
|
||||
- Vue 3.5 no-breaking-changes confirmation: https://blog.vuejs.org/posts/vue-3-5
|
||||
- Vite 6 migration guide: https://v6.vite.dev/guide/migration
|
||||
- FastAPI router decomposition: https://fastapi.tiangolo.com/tutorial/bigger-applications/ (built-in, no new deps)
|
||||
|
||||
---
|
||||
*Stack research for: DocuVault v0.2 — UI Overhaul and Optimization*
|
||||
*Researched: 2026-06-07*
|
||||
- Do not mount user cloud drives into the container filesystem.
|
||||
- Do not build a full Dropbox/Nextcloud sync daemon in v0.3.
|
||||
- Do not mirror entire cloud accounts into MinIO by default.
|
||||
- Do not create a second cloud file grid. Extend `StorageBrowser.vue`.
|
||||
- Do not expose provider download URLs directly to the browser unless proxied and authorized.
|
||||
|
||||
+35
-139
@@ -1,150 +1,46 @@
|
||||
# DocuVault — Research Synthesis
|
||||
# v0.3 Research Summary
|
||||
|
||||
_Last updated: 2026-05-21_
|
||||
**Milestone:** v0.3 Reimagining Cloud Storage integration
|
||||
**Date:** 2026-06-17
|
||||
|
||||
## Executive Summary
|
||||
## Stack Additions
|
||||
|
||||
DocuVault is a brownfield migration of a functional single-user document scanner into a privacy-first, multi-user SaaS platform. The existing system already handles document upload, text extraction, and AI-based topic classification via a well-designed provider abstraction. This milestone replaces the flat-file JSON + filesystem persistence layer with PostgreSQL + MinIO, adds full multi-user authentication (JWT with httpOnly cookies, TOTP 2FA, refresh token rotation), per-user quota enforcement, folder organization, document sharing, and pluggable cloud storage backends — following the same adapter pattern already used for AI providers.
|
||||
- Keep the current stack and add a provider capability/action layer rather than replacing storage.
|
||||
- Add PostgreSQL-backed cloud item metadata and analysis/index state.
|
||||
- Add a bounded byte-cache/offload service for preview/extraction/open flows.
|
||||
- Use Celery for cloud analysis, cache hydration, retries, and provider refresh jobs.
|
||||
- Use PostgreSQL full-text search for keyword/sentence search; consider `pgvector` for idea/semantic search without a separate vector DB.
|
||||
|
||||
---
|
||||
## Feature Table Stakes
|
||||
|
||||
## Confirmed Stack
|
||||
- Cloud files behave locally for open/preview/upload/move/delete/rename/create-folder where provider capabilities allow.
|
||||
- Users can analyze selected files, folders, or whole connections.
|
||||
- Analyzed cloud documents become classified and smart-searchable alongside local documents.
|
||||
- Search never downloads bytes at query time; it reads persisted extracted/indexed data.
|
||||
- DocuVault minimizes local storage by caching bytes only when needed and evicting them safely.
|
||||
|
||||
### Use
|
||||
## Architecture Direction
|
||||
|
||||
| Package | Version | Purpose |
|
||||
|---|---|---|
|
||||
| `pyjwt[crypto]` | ≥2.12.1 | JWT — current FastAPI docs recommendation; replaces python-jose |
|
||||
| `pwdlib[argon2]` | ≥0.2.0 | Password hashing — Argon2 is memory-hard (OWASP 2025) |
|
||||
| `pyotp` | ≥2.9.0 | TOTP 2FA — RFC 6238 reference |
|
||||
| `cryptography` (Fernet) | ≥44.0.0 | Credential encryption — AES-128-CBC + HMAC-SHA256 |
|
||||
| `sqlalchemy[asyncio]` | ≥2.0.36 | ORM — async-native; better brownfield fit than SQLModel |
|
||||
| `psycopg[asyncio,binary]` | ≥3.2.0 | PostgreSQL driver — single driver for async FastAPI + sync Alembic |
|
||||
| `alembic` | ≥1.14.0 | DB migrations |
|
||||
| `minio` | ≥7.2.0 | Object storage — presigned URL flow (FastAPI never proxies bytes) |
|
||||
| `msgraph-sdk` + `azure-identity` | ≥1.0.0 / ≥1.19.0 | OneDrive — official Microsoft SDK |
|
||||
| `google-api-python-client` + `google-auth-oauthlib` | ≥2.150.0 / ≥1.2.0 | Google Drive v3 |
|
||||
| `webdav4` | ≥0.9.8 | Nextcloud + generic WebDAV |
|
||||
- Keep provider-owned cloud bytes as source of truth.
|
||||
- Store metadata, text, topics, embeddings, and statuses in DocuVault.
|
||||
- Separate local/imported `documents` from cloud item index records unless a later phase intentionally unifies them.
|
||||
- Extend `StorageBrowser.vue` with capability-aware actions instead of creating a second file manager.
|
||||
- Split `backend/api/cloud.py` as behavior grows.
|
||||
|
||||
### Do NOT Use
|
||||
## Watch Out For
|
||||
|
||||
- `python-jose` — FastAPI dropped it; use PyJWT
|
||||
- `passlib[bcrypt]` for new hashes — maintenance mode; keep only for migrating existing hashes
|
||||
- `tiangolo/uvicorn-gunicorn-fastapi` Docker image — deprecated; use `python:3.12-slim`
|
||||
- `localStorage` for any auth token — XSS-accessible; httpOnly cookie for refresh, Pinia memory for access token
|
||||
- Single platform Fernet key for all users — HKDF per-user derivation required (catastrophic blast radius otherwise)
|
||||
- `SQLModel` for this migration — async story is thin; SQLAlchemy 2.0 async is better for brownfield
|
||||
- Do not accidentally build a full cloud mirror.
|
||||
- Treat destructive cloud actions as real provider mutations.
|
||||
- Model provider capability differences explicitly.
|
||||
- Track etags/versions so analyzed search data can become stale instead of silently wrong.
|
||||
- Add preflight estimates and confirmation before large "analyze all" jobs.
|
||||
- Preserve SSRF, credential encryption, ownership checks, and admin privacy boundaries.
|
||||
|
||||
---
|
||||
## Key Sources
|
||||
|
||||
## Table-Stakes Features for v1
|
||||
|
||||
### Confirmed (from PROJECT.md)
|
||||
|
||||
- Email + password registration + JWT sessions with refresh tokens
|
||||
- TOTP 2FA + backup codes *(see gap below)*
|
||||
- Password reset via email
|
||||
- Per-user isolated storage (100 MB free tier)
|
||||
- Quota tracking, enforcement at upload, visible indicator
|
||||
- Folder CRUD, move documents, "Shared with me" folder
|
||||
- Share by handle, view-only default, immediate revoke
|
||||
- Cloud OAuth2 connect flow + credential encryption
|
||||
- Admin: user management, quota adjustment, AI provider assignment
|
||||
- Audit log (append-only, metadata only) + admin viewer
|
||||
- In-browser PDF preview
|
||||
|
||||
### Gaps — Items PROJECT.md Missed
|
||||
|
||||
1. **TOTP backup codes** — Every competitor ships these. Without them, a lost phone permanently locks users out. 8–10 single-use codes, stored hashed, acknowledged by user before TOTP is activated.
|
||||
|
||||
2. **Quota warnings at 80% and 95%** — PROJECT.md specifies rejection at 100% only. Pre-emptive warnings are table stakes (Google Drive, Dropbox both do this). In-app banner at 80% (amber) and 95% (red), plus a specific error at 100% showing current usage, rejected file size, and a link to storage settings.
|
||||
|
||||
3. **"Sign out all devices" / session revocation** — Users who believe their account is compromised need forced logout everywhere. Already handled by the `refresh_tokens` table — requires only an endpoint and a UI control.
|
||||
|
||||
4. **Breadcrumb navigation** — Folder CRUD is in PROJECT.md but not the navigation UX. Required for nested folder usability.
|
||||
|
||||
5. **Cloud storage connection status indicator** — PROJECT.md doesn't specify what happens when cloud storage is unreachable. Silent failure = data loss. Must show `ACTIVE | REQUIRES_REAUTH | ERROR` state and fall back to local storage with a clear message.
|
||||
|
||||
6. **Admin impersonation is an explicit architectural exclusion** — Must be documented as excluded, not just left unbuilt. Directly contradicts the privacy-first core value.
|
||||
|
||||
---
|
||||
|
||||
## Critical Architectural Decisions (Lock Before Building)
|
||||
|
||||
These cannot be safely retrofitted:
|
||||
|
||||
**1. JWT in httpOnly cookies**
|
||||
Refresh token: `httpOnly; Secure; SameSite=Strict` cookie. Access token: Pinia memory only. Never `localStorage`. Vue Router guard silently refreshes before redirecting to login. Axios `withCredentials: true`.
|
||||
|
||||
**2. HKDF per-user key derivation for cloud credentials**
|
||||
`HKDF(master_key, salt=user_id_bytes, info=b"cloud-credentials")`. Master key in `CLOUD_CREDS_KEY` env var only. Salt in users table. Design before writing the first line of credential storage — cannot be added later without re-encrypting everything.
|
||||
|
||||
**3. Presigned MinIO URL flow**
|
||||
FastAPI generates signed PUT URL → browser uploads directly to MinIO → FastAPI confirms object and commits quota atomically. FastAPI handles metadata only; bytes never pass through the API layer. Object keys: `{user_id}/{document_id}/{uuid4()}{ext}`. Human-readable filename in DB only.
|
||||
|
||||
**4. Atomic PostgreSQL quota enforcement**
|
||||
`UPDATE quotas SET used_bytes = used_bytes + $delta WHERE user_id = $uid AND (used_bytes + $delta) <= limit_bytes RETURNING used_bytes`. If 0 rows returned, delete the MinIO object and return 413. Never perform quota arithmetic in Python between two DB statements.
|
||||
|
||||
**5. BackgroundTasks replacement before horizontal scaling**
|
||||
FastAPI `BackgroundTasks` is per-instance — classification tasks cannot distribute across containers. Replace with Celery + Redis or pgqueuer (PostgreSQL-backed, no Redis dependency) before scaling to N instances. Decide during Phase 3 planning.
|
||||
|
||||
**Additional locked decisions:**
|
||||
- Refresh tokens are opaque UUIDs stored hashed in DB (not JWTs); access tokens are short-lived JWTs (15 min).
|
||||
- `refresh_tokens` table has `family_id` — on reuse of a rotated token, revoke entire family and emit security alert.
|
||||
- Audit log uses `BIGSERIAL` PK; app DB user has INSERT + SELECT only (no UPDATE/DELETE).
|
||||
- Admin endpoints for cloud connections return only `provider, display_name, connected_at, status` — never `credentials_enc`.
|
||||
- Every document/folder endpoint asserts `resource.user_id == current_user.id` via centralized `assert_document_access()`.
|
||||
|
||||
---
|
||||
|
||||
## 5-Phase Migration Sequence
|
||||
|
||||
### Phase 1 — Infrastructure Foundation
|
||||
Wire PostgreSQL + MinIO into Docker Compose. Create `db/models.py` with full schema. Alembic initial migration. Async session dependency. No API changes — flat-file code still runs. Gate: all services boot cleanly; migrations apply; no behavior change.
|
||||
|
||||
### Phase 2 — Users and Authentication
|
||||
Users, refresh_tokens, quotas tables. Auth endpoints (register, login, refresh, TOTP, password reset, forced logout). TOTP with backup codes. Password reset does NOT auto-login (routes through TOTP gate). `get_current_user` + `get_current_admin` FastAPI dependencies. Admin user management endpoints. Vue auth store (Pinia memory + httpOnly cookie), Router guard, Axios interceptors. Gate: admin JWT returns 403 on document endpoints; backup codes issued and acknowledged at enrollment.
|
||||
|
||||
### Phase 3 — Document Migration to PostgreSQL + MinIO
|
||||
Dual-write window: new uploads write to both stores. Migration script copies historical flat-file data to PostgreSQL + MinIO. Count reconciliation assertion (go/no-go gate). Flip read source to PostgreSQL. Remove JSON write path. Presigned URL flow for all uploads/downloads. `asyncio.to_thread()` wrapping all MinIO SDK calls. Gate: concurrent upload test at 99% quota — only one succeeds.
|
||||
|
||||
### Phase 4 — Multi-User Isolation, Quotas, Folders, Sharing
|
||||
All queries gain `WHERE user_id = current_user.id`. Quota bar (80%/95% warnings). Folder CRUD + breadcrumbs. Document move + sort. Share by handle + "Shared with me" folder. Audit log wired to all events. Admin audit viewer. In-browser PDF preview. Gate: negative-access test (admin cannot retrieve any document content); quota reconciliation drift <1%.
|
||||
|
||||
### Phase 5 — Cloud Storage Backends
|
||||
`StorageBackend` ABC + factory (mirrors `ai/` pattern). `MinIOBackend`, `OneDriveBackend`, `GoogleDriveBackend`, `NextcloudBackend`, `WebDAVBackend`. OAuth2 connect/disconnect flows. Connection status UX. HKDF key derivation for all credentials. `delete_user_files()` on account deletion. Gate: mock `invalid_grant` → REQUIRES_REAUTH (not 500); account deletion asserts `delete_user_files()` per connection.
|
||||
|
||||
---
|
||||
|
||||
## Top 5 Pitfalls by Risk
|
||||
|
||||
| # | Pitfall | Severity | Fix |
|
||||
|---|---|---|---|
|
||||
| 1 | JWT in localStorage — XSS bypasses TOTP entirely | CRITICAL | httpOnly cookie for refresh, Pinia memory for access token |
|
||||
| 2 | Quota race condition — concurrent uploads bypass limit | DATA INTEGRITY | Atomic PostgreSQL `UPDATE ... RETURNING` |
|
||||
| 3 | TOTP bypass via password reset — full 2FA bypass via email compromise | SECURITY | Reset issues `password_reset_pending` state, not a full session |
|
||||
| 4 | Single Fernet key for all cloud credentials — catastrophic on key leak | CATASTROPHIC | HKDF per-user derivation before first credential is stored |
|
||||
| 5 | Path traversal in MinIO keys — cross-user data access | SECURITY | UUID-only MinIO keys; human filename in DB only; never reconstruct key from request parameters |
|
||||
|
||||
---
|
||||
|
||||
## Confidence Assessment
|
||||
|
||||
| Area | Confidence | Notes |
|
||||
|---|---|---|
|
||||
| Stack | MEDIUM-HIGH | Core libraries confirmed from FastAPI official release notes (PyJWT, pwdlib, SQLAlchemy 2.0, psycopg v3). Cloud SDK minor versions — verify on PyPI before pinning. |
|
||||
| Features | MEDIUM | Based on Google Drive, Dropbox, Box, Paperless-ngx knowledge through Aug 2025. |
|
||||
| Architecture | HIGH | FastAPI DI pattern from official docs; S3 presigned URLs and atomic PostgreSQL quota update are industry standards. |
|
||||
| Pitfalls | HIGH | OWASP cheat sheets; RFC 9700 refresh token rotation; GDPR Article 17 stable regulatory text. |
|
||||
|
||||
**Overall: MEDIUM-HIGH**
|
||||
|
||||
---
|
||||
|
||||
## Gaps to Resolve During Planning
|
||||
|
||||
- Verify cloud SDK minor versions on PyPI before pinning
|
||||
- Confirm PyOTP `valid_window` default in current docs (recommend `valid_window=1` for ±30s clock drift)
|
||||
- Decide Celery + Redis vs pgqueuer during Phase 3 (depends on Redis availability in deployment target)
|
||||
- Audit existing codebase for any existing bcrypt hashes before removing `passlib`
|
||||
- Validate MinIO Docker Compose public endpoint in Phase 3 acceptance testing (presigned URLs must use host-accessible address, not internal Docker network name)
|
||||
- Microsoft Graph `driveItem`: https://learn.microsoft.com/en-us/graph/api/resources/driveitem?view=graph-rest-1.0
|
||||
- Google Drive uploads: https://developers.google.com/workspace/drive/api/guides/manage-uploads
|
||||
- Nextcloud WebDAV operations: https://docs.nextcloud.com/server/latest/developer_manual/client_apis/WebDAV/basic.html
|
||||
- Celery task retries: https://docs.celeryq.dev/en/stable/userguide/tasks.html
|
||||
- Redis cache eviction: https://redis.io/docs/latest/develop/reference/eviction/
|
||||
- pgvector: https://github.com/pgvector/pgvector
|
||||
|
||||
Reference in New Issue
Block a user