docs(04): capture phase context

.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md

@@ -0,0 +1,154 @@
+# Phase 4: Folders, Sharing, Quotas & Document UX - Context
+
+**Gathered:** 2026-05-25
+**Status:** Ready for planning
+
+<domain>
+## Phase Boundary
+
+Deliver a complete document management experience on top of the Phase 3 multi-user foundation: folder organization (create/rename/delete/move, breadcrumb navigation, sort), document sharing by exact handle with immediate revocation, quota feedback (already-built QuotaBar, upload rejection with detailed error), PDF in-browser preview proxied through the app with a per-user open-mode preference, full-text search via PostgreSQL tsvector, and an admin audit log viewer with export and daily automated CSV backups.
+
+This phase does NOT include cloud storage backend connectivity (Phase 5). The QuotaBar component and upload rejection flow are already built (Phase 3) — Phase 4 wires the remaining document UX features around them.
+
+</domain>
+
+<decisions>
+## Implementation Decisions
+
+### Folder Navigation Layout
+
+- **D-01:** Hybrid layout — AppSidebar shows top-level folders only (no nested expansion in sidebar). Once inside a folder, the main content area shows sub-folder rows + breadcrumb navigation. Top-level folders in sidebar are clickable to navigate directly.
+- **D-02:** Unlimited nesting depth — no API or UI cap. The `Folder.parent_id` self-referential FK is the authoritative constraint. Breadcrumbs truncate long paths with an ellipsis when path depth > N segments.
+- **D-03:** Non-empty folder delete shows a warning modal that includes the document count (e.g., "This folder contains 5 documents. Deleting it will permanently delete all documents inside."). If the user confirms, cascade-delete all documents (MinIO objects + DB rows + quota decrements) and the folder. If the user cancels, no action taken. Documents are NOT moved to root — they are destroyed with the folder.
+
+### Sharing
+
+- **D-04:** Exact handle input — user types the recipient's handle (no `@` prefix required by API, but UI may accept it). No autocomplete or user search endpoint needed. API returns 404 if handle not found; UI shows a clear "User not found" error.
+- **D-05:** Share button is on the DocumentCard (inline icon button). Clicking opens a sharing modal containing: (a) a text input for the recipient handle, (b) a "Share" submit button, (c) a list of current recipients with their permission level and a "Revoke" button per row.
+- **D-06:** "Shared with me" is a fixed virtual folder entry in AppSidebar, rendered above the user's own folder list. It is not stored as a real folder — the backend filters by `shares.recipient_id = current_user.id`. Shared documents count zero bytes against the recipient's quota (SHARE-02).
+- **D-07:** Share permission is `view` only for Phase 4 (SHARE-03 — owner controls permission level; only `view` implemented; `edit` deferred).
+
+### PDF Proxy & Preview
+
+- **D-08:** Streaming proxy endpoint: `GET /api/documents/{id}/content` retrieves bytes from MinIO via the storage backend and returns them via FastAPI `StreamingResponse`. Supports `Range` headers for partial content (large file performance). `Content-Disposition: inline` with correct `Content-Type` from the document's `content_type` column. No presigned URL is ever generated or exposed to the browser (DOC-02 privacy model).
+- **D-09:** Native browser PDF rendering — no PDF.js library. DOC-02 specifies PDF.js but the user explicitly chose native rendering (intent over literal spec). Content-Type header drives browser rendering. Zero frontend dependencies added.
+- **D-10:** Per-user PDF open preference stored in DB. New column `users.pdf_open_mode` (String, default `'in_app'`, allowed values: `'in_app'` | `'new_tab'`). Exposed via `PATCH /api/me/preferences` endpoint. Setting is shown in SettingsView for all users (not admin-only).
+  - `in_app`: PDF opens in a modal/overlay containing `<iframe src="/api/documents/{id}/content">`.
+  - `new_tab`: Frontend calls `window.open('/api/documents/{id}/content', '_blank')`.
+
+### Full-Text Search
+
+- **D-11:** PostgreSQL `tsvector` index on `documents.extracted_text` (FOLD-05). Index type: GIN. Column: `ts_vector` stored/generated column or updated on extract. Search endpoint: `GET /api/documents?q=<query>` using `to_tsquery()`. Search scope: user's own documents (ownership assertion applies); does not search shared documents received from others.
+- **D-12:** Search UX: search bar in the main document list view (inline filter, not a separate route). Results update as the user types (debounced, minimum 2 chars). Searching within the current folder context is supported; searching from root searches all user documents.
+
+### Audit Log
+
+- **D-13:** All 4 event categories are logged (user decision):
+  - **Auth events:** login success, login failure, logout, password change, TOTP enrolled, TOTP revoked, backup code used, sign-out-all triggered
+  - **Document operations:** document uploaded (no filename/extracted_text in log — only `document_id`, `size_bytes`, `storage_backend`), document deleted, reclassification triggered
+  - **Folder & share operations:** folder created, folder deleted (with `doc_count` in `metadata_`), share granted (`document_id`, `owner_id`, `recipient_id`, `permission`), share revoked
+  - **Admin actions:** user created, user deactivated/activated, quota limit changed (old + new value), AI provider/model assigned
+- **D-14:** Audit log entries written inline in each API handler after the successful operation completes. Shared helper `write_audit_log(session, event_type, user_id, actor_id, resource_id, ip_address, metadata_dict)` — no middleware, no Celery task.
+- **D-15:** Admin audit log viewer (`/admin` → AuditLog tab): paginated table with columns: timestamp, user, action type, IP address. Filters: date range picker, user dropdown, action type dropdown. No document content, filenames, or extracted text in any log entry (ADMIN-06).
+- **D-16:** Manual CSV and JSON export button in the admin audit log view. Exports current filtered result set.
+- **D-17:** Daily automated CSV backup via Celery beat task. Exports the previous day's audit log rows to a CSV file uploaded to a dedicated MinIO `audit-logs` bucket (e.g., key: `audit-logs/YYYY-MM-DD.csv`). Bucket is private (no public access). Admins can download exports from the admin panel. This manages log growth without deleting rows from the DB.
+
+### SEC-08 / SEC-09
+
+- **D-18:** `credentials_enc` excluded from all serializers (SEC-08). Phase 4 adds explicit Pydantic response model enforcement across all user-facing endpoints that touch `CloudConnection`. Defensive even though Phase 5 hasn't created cloud connections yet.
+- **D-19:** Account deletion (admin-triggered) runs `delete_user_files()` for all user documents before removing DB records (SEC-09). Phase 4 implements this cleanup hook since document deletion is in scope here; cloud connection cleanup deferred to Phase 5.
+
+</decisions>
+
+<canonical_refs>
+## Canonical References
+
+**Downstream agents MUST read these before planning or implementing.**
+
+### Requirements
+- `.planning/REQUIREMENTS.md` — FOLD-01 through FOLD-05 (folder CRUD, breadcrumbs, sort, search), SHARE-01 through SHARE-05 (sharing by handle, virtual folder, revocation, indicator), DOC-01 (document metadata view), DOC-02 (PDF proxy, no presigned URLs), SEC-08 (credentials_enc excluded), SEC-09 (account deletion cleanup), ADMIN-06 (audit log viewer, metadata only)
+
+### Roadmap & Success Criteria
+- `.planning/ROADMAP.md` — Phase 4 goal and all 5 success criteria (especially SC2: sharing with quota enforcement on recipient, SC4: PDF bytes proxied with no presigned URL exposure, SC5: admin audit log with no document content)
+
+### Architecture Constraints
+- `CLAUDE.md` — Key Architectural Rules: bytes never through API (DOC-02 is the deliberate exception — proxy endpoint), MinIO key schema, atomic quota UPDATE, ownership assertion pattern, admin endpoints never return document content
+
+### Prior Phase Decisions
+- `.planning/phases/03-document-migration-multi-user-isolation/03-CONTEXT.md` — D-07 (atomic quota UPDATE pattern), D-16 (ownership assertion: 404 not 403 for cross-user access), D-17 (topic namespace model — context for how virtual folder pattern differs)
+
+### ORM Schema (all relevant models already exist)
+- `backend/db/models.py` — `Folder` (parent_id self-FK, user_id, name), `Document` (folder_id FK nullable), `Share` (document_id, owner_id, recipient_id, permission), `AuditLog` (user_id, actor_id, event_type, resource_id, ip_address, metadata_ JSONB), `User` (add pdf_open_mode column in Phase 4 migration)
+
+### Frontend Existing Components
+- `frontend/src/components/layout/AppSidebar.vue` — extend with Folders section + "Shared with me" entry
+- `frontend/src/components/documents/DocumentCard.vue` — add share button
+- `frontend/src/components/layout/QuotaBar.vue` — already complete (Phase 3)
+- `frontend/src/stores/documents.js` — extend with folder/search/sharing actions
+- `frontend/src/api/client.js` — add folder, share, search, PDF proxy, preferences API calls
+
+</canonical_refs>
+
+<code_context>
+## Existing Code Insights
+
+### Reusable Assets
+- `backend/deps/auth.py` — `get_current_user` (regular user) and `get_current_admin` (admin only) — inject into all new folder/share/audit endpoints
+- `backend/db/models.py` — `Folder`, `Share`, `AuditLog` ORM models are complete; `Document.folder_id` FK exists; no new tables needed for Phase 4 except adding `users.pdf_open_mode` column
+- `backend/storage/minio_backend.py` — `MinIOBackend.get_object()` exists (or add it); streaming proxy reads from MinIO and passes bytes to `StreamingResponse`
+- `backend/celery_app.py` — Celery beat schedule: add daily audit log export task alongside existing abandoned-upload cleanup
+- `frontend/src/components/layout/QuotaBar.vue` — already implemented with amber/red thresholds; no changes needed
+- `frontend/src/components/upload/DropZone.vue` — upload 413 rejection error with payload already implemented
+
+### Established Patterns
+- **Ownership assertion** — `doc.user_id == current_user.id` → raise `HTTPException(404)` (D-16 from Phase 3); same pattern applies to folder endpoints
+- **Atomic quota UPDATE** — `UPDATE quotas SET used_bytes = ... WHERE ... RETURNING used_bytes`; use same pattern for quota decrement on folder cascade-delete
+- **`get_regular_user` dep (403 for admin)** — enforced on all document/folder/share endpoints; admin role cannot read document content
+- **`write_audit_log()` helper** — new shared function in `backend/services/audit.py` (or `backend/db/audit.py`); called inline in handlers after successful operation
+- **`asyncio.to_thread()`** — for MinIO sync SDK calls in streaming proxy (established pattern from Phase 1)
+- **Pydantic response models** — explicit whitelist pattern (`_user_to_dict()` equivalent) already established for admin responses; extend for CloudConnection serializers (SEC-08)
+
+### Integration Points
+- `backend/api/documents.py` — add `GET /api/documents/{id}/content` streaming proxy; add `q` query param to list endpoint for tsvector search; add audit log calls to upload/delete/reclassify handlers
+- `backend/api/` — add `backend/api/folders.py` (folder CRUD + move), `backend/api/shares.py` (share grant/revoke/list), `backend/api/audit.py` (admin audit log viewer + export)
+- `backend/api/auth.py` + `backend/api/admin.py` — add `write_audit_log()` calls to auth events and admin user management actions (Phase 4 back-fills audit writing for previously unlogged operations)
+- `backend/migrations/versions/0004_phase4_*.py` — add `users.pdf_open_mode` column; add `tsvector` GIN index on `documents.extracted_text`; create `audit-logs` MinIO bucket in migration post-DDL step
+- `frontend/src/stores/documents.js` — add `fetchFolder(folderId)`, `createFolder()`, `moveDocument()`, `shareDocument()`, `revokeShare()`, `searchDocuments(q)` actions
+- `frontend/src/views/HomeView.vue` — wire folder navigation state (current folder ID), breadcrumb component, folder rows in list
+- `frontend/src/views/SettingsView.vue` — add PDF open mode toggle (currently a static placeholder per Phase 3 Risk 6 decision)
+
+### Constraints from Prior Phases
+- MinIO key schema `{user_id}/{document_id}/{uuid4()}{ext}` is locked — proxy endpoint uses stored `object_key`, never reconstructs it
+- `get_regular_user` returns 403 for admin role on document endpoints — streaming proxy must also use `get_regular_user`
+- Cross-user doc access returns 404 not 403 (D-16) — same rule applies to folder and share access attempts
+- `documents.user_id` is NOT NULL (Phase 3 migration) — no null-user guards needed
+
+</code_context>
+
+<specifics>
+## Specific Ideas
+
+- Folder delete warning modal: shows count string like "This folder contains 5 documents. Deleting it will permanently delete all documents inside. This cannot be undone." — two buttons: "Delete folder and documents" (destructive, red) / "Cancel"
+- Share modal layout: handle input at top → "Share" button → separator → list of current shares with handle + permission badge + "Revoke" button per row. Empty state: "Not shared with anyone yet."
+- "Shared with me" sidebar entry: fixed position above user-created folders, shows inbox-like icon, displays document count badge when non-empty
+- PDF open mode setting in SettingsView: simple toggle/radio in a "Document Preferences" card section ("Open documents in-app" / "Open documents in new tab")
+- Audit log CSV filename convention: `audit-logs/YYYY-MM-DD.csv` in MinIO `audit-logs` bucket
+- tsvector search: `documents.extracted_text` GIN index; query via `plainto_tsquery('english', :q)` for natural-language matching (simpler than `to_tsquery` which requires operators)
+- Breadcrumb truncation: show first segment + "..." + last 2 segments when depth > 4
+
+</specifics>
+
+<deferred>
+## Deferred Ideas
+
+- **Per-day document scan quota tiers** — User wants subscription tiers based on number of AI scans per day rather than (or in addition to) storage bytes. New billing/subscription model capability — separate milestone/phase. Does not affect Phase 4 implementation.
+- **Share permission levels beyond `view`** — SHARE-03 says owner controls permission level; only `view` is implemented in Phase 4. `edit` and `comment` permissions deferred to a future iteration.
+- **Presigned GET URLs for non-PDF document download** — Phase 3 deferred this; Phase 4 adds the proxy endpoint for PDFs. For other formats (DOCX, TXT), streaming proxy via the same endpoint is the approach; no direct presigned URLs exposed.
+- **Audit log row archival/deletion policy** — Daily CSV backup to MinIO manages growth by providing an export, but DB rows are not auto-deleted in Phase 4. Long-term archival/TTL policy deferred.
+
+</deferred>
+
+---
+
+*Phase: 4-Folders-Sharing-Quotas-Document-UX*
+*Context gathered: 2026-05-25*