diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index c495344..fc390af 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -1,6 +1,16 @@ # DocuVault — v1 Roadmap -_Last updated: 2026-05-22_ +_Last updated: 2026-05-25_ + +## Mandatory Cross-Cutting Gates (every phase) + +Before any phase is marked complete, all three gates must pass: + +1. **Test gate** — `pytest -v` passes with zero failures; every new function/endpoint has at least one test; all security invariant tests pass (wrong owner, admin block, token replay) +2. **Security gate** — Security agent runs `bandit -r backend/` (zero HIGH), `pip audit` (zero critical/high), `npm audit --audit-level=high` (zero high/critical); admin endpoints verified to never return `password_hash`, `credentials_enc`, or document content; no hardcoded secrets +3. **Bug fix rule** — Any bug fix during execution must: (a) target the root cause, (b) change ≤50 lines, (c) include a regression test — no workarounds permitted + +--- ## Phases @@ -111,6 +121,11 @@ _Last updated: 2026-05-22_ - AI provider/model resolved only via Celery task DB lookup (Plan 04) - Browser XHR PUT to MinIO sends NO Authorization header (Plan 05) +**Phase gates (must pass before Phase 3 is complete):** +- [ ] `pytest -v` — zero failures; presigned URL, quota enforcement, ownership isolation, and admin-403 all covered +- [ ] Security agent: path traversal check on object key construction; cross-user IDOR tests; quota race condition test +- [ ] Bandit + pip audit + npm audit all clean + **UI hint**: yes --- @@ -125,10 +140,37 @@ _Last updated: 2026-05-22_ 1. A user can create, rename, and delete folders; moving a document between folders preserves its metadata and AI classification; deleting a non-empty folder prompts with the content count before proceeding 2. A user can share a document with another user by handle; the recipient sees it appear in a "Shared with me" virtual folder with no storage quota charged against them; the owner can revoke access and the shared entry disappears immediately for the recipient 3. The sidebar quota bar displays current usage in MB; it turns amber at 80% and red at 95%; an upload that would exceed the limit is rejected with an error showing current usage, the rejected file size, and a link to storage settings -4. Any document in the user's library can be previewed in-browser as a PDF via PDF.js; document bytes are proxied through the app and no presigned URLs are exposed to the browser +4. Any document in the user's library can be previewed in-browser as a PDF; document bytes are proxied through the app and no presigned URLs are exposed to the browser (native browser PDF rendering via Content-Type header) 5. An admin can view the audit log filtered by date range, user, and action type; the log contains no document content, filenames, or extracted text; account deletion triggers cleanup of all user files before DB records are removed -**Plans**: TBD +**Plans**: 9 plans + +**Wave 1** — Test scaffolds + DB migration (parallel) +- [ ] 04-01-PLAN.md — Wave 0 test stubs: test_folders.py + test_shares.py + test_audit.py + proxy stubs in test_documents.py + SEC-08/SEC-09 stubs in test_security.py +- [ ] 04-02-PLAN.md — Alembic migration 0004 (users.pdf_open_mode, GIN FTS index, audit-logs bucket) + MinIOBackend.put_object_raw() + +**Wave 2** *(blocked on Wave 1)* +- [ ] 04-03-PLAN.md — Audit service (write_audit_log) + Folders API (FOLD-01..05): POST/GET/PATCH/DELETE /api/folders + PATCH /api/documents/{id}/folder + document list sort/search/is_shared extension +- [ ] 04-04-PLAN.md — Shares API (SHARE-01..05): POST/GET /api/shares + GET /api/shares/received + DELETE /api/shares/{id} with IDOR protection + +**Wave 3** *(blocked on Wave 2)* +- [ ] 04-05-PLAN.md — PDF streaming proxy GET /api/documents/{id}/content with Range header support + PATCH /api/auth/me/preferences (pdf_open_mode) +- [ ] 04-06-PLAN.md — Admin audit log API (GET /api/admin/audit-log, CSV export) + Celery beat daily audit export task + celery_app.py beat schedule + +**Wave 4** *(blocked on Wave 3)* +- [ ] 04-07-PLAN.md — SEC-08/SEC-09 hardening + audit log backfill into auth.py/admin.py/documents.py + CloudConnectionOut Pydantic model + delete-user file cleanup + +**Wave 5** *(blocked on Wave 4)* +- [ ] 04-08-PLAN.md — Frontend data layer: API client functions + useFoldersStore + documents store extension + Vue Router routes (/folders/:folderId, /shared) + +**Wave 6** *(blocked on Wave 5)* +- [ ] 04-09-PLAN.md — Frontend UI: all new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + view wiring (AppSidebar, DocumentCard, HomeView, FolderView, SharedView, SettingsView, AdminView) + human checkpoint + +**Phase gates (must pass before Phase 4 is complete):** +- [ ] `pytest -v` — zero failures; folder ownership, share revocation, quota bar, PDF proxy (no presigned URL exposure) all covered +- [ ] Security agent: audit log verified to contain zero document content; sharing IDOR tests; PDF proxy verified to not leak presigned URLs or object keys +- [ ] Bandit + pip audit + npm audit all clean + **UI hint**: yes --- @@ -147,6 +189,12 @@ _Last updated: 2026-05-22_ 5. An admin API response for a user's cloud connections returns only `provider, display_name, connected_at, status` — the `credentials_enc` column is never present in any serialized response **Plans**: TBD + +**Phase gates (must pass before Phase 5 is complete):** +- [ ] `pytest -v` — zero failures; SSRF prevention on WebDAV/Nextcloud user-supplied URLs; credential encryption/decryption round-trip; admin response never exposes `credentials_enc`; OAuth invalid_grant handling +- [ ] Security agent: SSRF allowlist verification; credential key derivation correctness; connection status never leaks raw credential values +- [ ] Bandit + pip audit + npm audit all clean + **UI hint**: yes --- @@ -158,5 +206,5 @@ _Last updated: 2026-05-22_ | 1. Infrastructure Foundation | 5/5 | Complete | 2026-05-22 | | 2. Users & Authentication | 5/5 | Complete | 2026-05-22 | | 3. Document Migration & Multi-User Isolation | 0/5 | Not started | - | -| 4. Folders, Sharing, Quotas & Document UX | 0/? | Not started | - | +| 4. Folders, Sharing, Quotas & Document UX | 0/9 | Not started | - | | 5. Cloud Storage Backends | 0/? | Not started | - | diff --git a/.planning/STATE.md b/.planning/STATE.md index d434cf1..90dee6a 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -3,12 +3,12 @@ gsd_state_version: 1.0 milestone: v1.0 milestone_name: milestone current_phase: 4 -status: ready -last_updated: "2026-05-25T00:00:00Z" +status: planned +last_updated: "2026-05-25T16:00:00.000Z" progress: total_phases: 5 completed_phases: 3 - total_plans: 15 + total_plans: 33 completed_plans: 15 percent: 60 --- @@ -32,8 +32,8 @@ progress: ## Current Position -**Phase:** 04-folders-sharing-quotas-document-ux — Ready to start -**Plan:** 0/N — awaiting /gsd:discuss-phase 4 +**Phase:** 04-folders-sharing-quotas-document-ux — Ready to execute +**Plan:** 0/9 — 9 plans created, verification passed **Progress:** ██████░░░░ 60% (3/5 phases complete) ## Performance Metrics @@ -116,6 +116,7 @@ Two mandatory cross-cutting gates added to all phases going forward: **1. Test gate** — every plan must leave `pytest -v` passing with zero failures. Every new function/endpoint/component requires at least one test. All security-invariant negative tests (wrong owner, admin block, token replay) must exist and pass. **2. Security gate** — a security agent runs after every plan execution and is a blocking requirement before phase advancement. It: + - Runs `bandit -r backend/`, `pip audit`, `npm audit --audit-level=high` - Checks for path traversal, IDOR, SSRF, timing attacks, mass assignment, token replay - Verifies admin endpoints never return `password_hash`, `credentials_enc`, or document content @@ -137,6 +138,8 @@ _Updated at each phase transition._ |---|---| | Last session | 2026-05-25 — Phase 3 UAT complete (10/10); security gate passed (3 fixes: bandit B324, Referrer-Policy, IDOR on /topics/suggest); test fix for test_lmstudio.py import | | Last session | 2026-05-25 — Phase 4 context gathered (4 areas: folder nav, sharing, PDF proxy, audit log) | -| Next action | Run `/gsd:plan-phase 4` to create execution plan | +| Last session | 2026-05-25 — Phase 4 UI-SPEC approved (6 dimensions: 2 PASS clean, 3 FLAG non-blocking, 0 BLOCK) | +| Last session | 2026-05-25 — Phase 4 plans created (9 plans, 7 waves) + verification passed (0 blockers, 2 warnings) | +| Next action | Run `/gsd:execute-phase 4` to begin execution (Wave 1: plans 04-01 + 04-02 in parallel) | | Pending decisions | None | -| Resume file | `.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md` | +| Resume file | `.planning/phases/04-folders-sharing-quotas-document-ux/04-UI-SPEC.md` | diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-01-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-01-PLAN.md new file mode 100644 index 0000000..8689edd --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-01-PLAN.md @@ -0,0 +1,219 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 01 +type: execute +wave: 1 +depends_on: [] +files_modified: + - backend/tests/test_folders.py + - backend/tests/test_shares.py + - backend/tests/test_audit.py + - backend/tests/test_documents.py + - backend/tests/test_security.py + - backend/tests/test_migration.py +autonomous: true +requirements: + - FOLD-01 + - FOLD-02 + - FOLD-03 + - FOLD-04 + - FOLD-05 + - SHARE-01 + - SHARE-02 + - SHARE-03 + - SHARE-04 + - SHARE-05 + - SEC-08 + - SEC-09 + - ADMIN-06 + - DOC-01 + - DOC-02 + +must_haves: + truths: + - "Wave 0 test stubs exist for every requirement in Phase 4" + - "All new test functions are xfail(strict=False) so CI stays green before implementation" + - "Shared fixtures from Phase 3 conftest are reused without modification" + artifacts: + - path: "backend/tests/test_folders.py" + provides: "FOLD-01..05 test stubs (create, rename, delete empty, cascade-delete, move, breadcrumb, sort, FTS)" + - path: "backend/tests/test_shares.py" + provides: "SHARE-01..05 stubs + IDOR negative tests" + - path: "backend/tests/test_audit.py" + provides: "ADMIN-06 stubs: viewer, no-doc-content, regular-user-403" + - path: "backend/tests/test_documents.py" + provides: "DOC-02 proxy stubs appended: 200, 206, admin-403, no-presigned-url" + - path: "backend/tests/test_security.py" + provides: "SEC-08 credentials_enc exclusion + SEC-09 delete-user-cleans-files stubs" + key_links: + - from: "backend/tests/test_folders.py" + to: "backend/tests/conftest.py" + via: "auth_user / admin_user / async_client fixtures" + pattern: "from tests.conftest import|fixture" + - from: "backend/tests/test_shares.py" + to: "backend/tests/conftest.py" + via: "auth_user fixture" + pattern: "auth_user|async_client" +--- + + +Create Wave 0 test scaffolds for Phase 4. Every Phase 4 requirement gets at least one +xfail test stub. All stubs use `strict=False` so unexpected passes do not break CI. +No implementation code is written in this plan. + +Purpose: Establish the Nyquist test gate before any backend code is written. +Output: Five test files (three new, two extended) with named xfail stubs for all 22 planned tests. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/PROJECT.md +@.planning/ROADMAP.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-VALIDATION.md +@backend/tests/conftest.py +@backend/tests/test_documents.py +@backend/tests/test_security.py + + + + + + Task 1: Create test_folders.py and test_shares.py stubs + backend/tests/test_folders.py, backend/tests/test_shares.py + + backend/tests/conftest.py — read the entire file to understand available fixtures (auth_user, admin_user, async_client, db_session) and how they are declared; extract fixture names and signatures before writing any test + backend/tests/test_documents.py — read lines 1-50 for established import and fixture-injection patterns + + + - test_create_folder: POST /api/folders creates folder, returns 201 + - test_create_folder_duplicate_name: POST /api/folders with same name under same parent returns 409 + - test_rename_folder: PATCH /api/folders/{id} changes name, returns 200 + - test_rename_folder_wrong_owner: PATCH /api/folders/{id} by non-owner returns 404 + - test_delete_empty_folder: DELETE /api/folders/{id} on empty folder returns 204 + - test_delete_folder_cascade: DELETE /api/folders/{id} on non-empty folder deletes all docs + decrements quota + - test_delete_folder_wrong_owner: DELETE /api/folders/{id} by non-owner returns 404 + - test_move_document: PATCH /api/documents/{id}/folder moves doc to target folder, returns 200 + - test_move_wrong_owner_404: PATCH /api/documents/{id}/folder where doc or target folder belongs to other user returns 404 + - test_breadcrumb_path: GET /api/folders/{id} returns breadcrumb array of {id, name} from root to current + - test_document_sort: GET /api/documents?sort=name|date|size returns correctly ordered results + - test_fts_search: GET /api/documents?q=term returns matching docs only; marked with pytest.mark.skipif for non-PostgreSQL + - test_fts_search_scoped_to_owner: GET /api/documents?q=term does not return other user's matching docs + - test_share_success: POST /api/shares grants share, recipient can see doc via GET /api/shares/received + - test_share_handle_not_found: POST /api/shares with unknown handle returns 404 + - test_shared_with_me: GET /api/shares/received lists docs shared with current user + - test_share_no_quota_impact: share does not increment recipient's quota used_bytes + - test_revoke_share: DELETE /api/shares/{id} removes share; GET /api/shares/received no longer lists the doc + - test_share_revoke_wrong_owner_404: DELETE /api/shares/{id} by non-owner returns 404 + - test_share_duplicate: POST /api/shares same doc+recipient twice returns 409 + + + Create backend/tests/test_folders.py. Header imports: `import pytest` and `pytest.mark.xfail(strict=False)`. + Import `pytest_asyncio` and any fixtures from conftest using injection (no direct import — pytest injects by name). + Every test function body is `pytest.xfail("not implemented yet")` as the first line — no other code. + Name stubs exactly as listed in the behavior block (test_create_folder, test_create_folder_duplicate_name, etc.). + Mark test_fts_search and test_fts_search_scoped_to_owner with both `@pytest.mark.xfail(strict=False)` and a `pytest.mark.skipif` checking for the INTEGRATION env var — pattern: `pytest.mark.skipif(not os.environ.get("INTEGRATION"), reason="requires PostgreSQL")`. + + Create backend/tests/test_shares.py with the same xfail stub pattern. + Stubs: test_share_success, test_share_handle_not_found, test_shared_with_me, test_share_no_quota_impact, test_revoke_share, test_share_revoke_wrong_owner_404, test_share_duplicate. + + Do NOT write any assertion code in stub bodies. One line only: `pytest.xfail("not implemented yet")`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_folders.py tests/test_shares.py -v --no-header 2>&1 | tail -30 + + + - backend/tests/test_folders.py exists with all 13 stub functions listed in the behavior block + - backend/tests/test_shares.py exists with all 7 stub functions listed in the behavior block + - pytest reports all stubs as xfail (x) or xpass — zero failures (F) or errors (E) + - test_fts_search and test_fts_search_scoped_to_owner have both @pytest.mark.xfail and @pytest.mark.skipif decorators + - No import errors: `python -c "import tests.test_folders; import tests.test_shares"` exits 0 + + Both files exist; pytest collects them with zero errors; all tests show as xfail. + + + + Task 2: Extend test_documents.py, test_audit.py, test_security.py, test_migration.py with Phase 4 stubs + backend/tests/test_documents.py, backend/tests/test_audit.py, backend/tests/test_security.py, backend/tests/test_migration.py + + backend/tests/test_documents.py — read the entire file; identify the last test function and existing imports to find the correct append point; note the fixture names in use + backend/tests/test_security.py — read the entire file; note what already exists to avoid duplicate function names; if test_delete_user_cleans_files already exists skip it + backend/tests/test_audit.py — this file does NOT exist yet; create it fresh + + + Additions to test_documents.py: + - test_content_stream_200: GET /api/documents/{id}/content returns 200 with correct Content-Type and Content-Disposition: inline + - test_content_stream_206_range: GET /api/documents/{id}/content with Range header returns 206 and Content-Range header + - test_content_stream_admin_403: GET /api/documents/{id}/content with admin JWT returns 403 + - test_content_stream_no_presigned_url: GET /api/documents/{id}/content response body does not contain any presigned URL token (no "X-Amz-Signature" or similar in body) + + New test_audit.py: + - test_audit_log_viewer: GET /api/admin/audit-log returns paginated entries + - test_audit_log_no_doc_content: audit log entries contain no "filename", "extracted_text" keys in metadata_ + - test_audit_log_regular_user_403: GET /api/admin/audit-log with regular user token returns 403 + - test_audit_log_export_csv: GET /api/admin/audit-log/export?format=csv returns CSV content-type + + Additions to test_security.py: + - test_credentials_enc_not_in_response: no API response for current user includes credentials_enc field + - test_delete_user_cleans_files: admin DELETE /api/admin/users/{id} triggers MinIO object deletion before DB removal + + + For test_documents.py: read the existing file, then APPEND the four new stub functions at the end. Each is `@pytest.mark.xfail(strict=False)` decorated, body is `pytest.xfail("not implemented yet")`. + + Create backend/tests/test_audit.py from scratch with the four stubs plus necessary imports (pytest, pytest_asyncio pattern from conftest — no direct imports, fixture injection only). + + For test_security.py: read the existing file first. Append the two new stubs ONLY if they do not already exist (check for test_credentials_enc_not_in_response and test_delete_user_cleans_files). If test_security.py does not exist, create it with both stubs. + + All new stubs: `@pytest.mark.xfail(strict=False)`, body `pytest.xfail("not implemented yet")`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_documents.py tests/test_audit.py tests/test_security.py -v --no-header 2>&1 | tail -30 + + + - backend/tests/test_audit.py exists with 4 stub functions + - backend/tests/test_documents.py contains test_content_stream_200, test_content_stream_206_range, test_content_stream_admin_403, test_content_stream_no_presigned_url (verified by grep) + - backend/tests/test_security.py contains test_credentials_enc_not_in_response and test_delete_user_cleans_files + - `cd backend && python -m pytest tests/ -v --no-header 2>&1 | grep -E "FAILED|ERROR"` produces zero lines + - Total xfail count increases by at least 10 compared to pre-plan baseline (all new stubs collected) + + All three files contain Phase 4 stubs; full test suite runs with zero failures or errors. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Test code → conftest fixtures | Test stubs must not import production secrets or live services | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-00-01 | Tampering | test stub files | mitigate | xfail(strict=False) ensures stubs cannot falsely pass and mask missing implementation | +| T-04-00-02 | Information Disclosure | test fixture reuse | accept | Phase 3 conftest fixtures already use ephemeral DB + mock MinIO; no real credentials in tests | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +Run full suite: `cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest -v --no-header 2>&1 | tail -20` + +Expected: zero FAILED, zero ERROR. All new stubs appear as xfail (x) in summary. + + + +- Five test files collectively contain all 23 Phase 4 test stubs +- `pytest -v` exits 0 (green) +- No existing passing tests regress +- All stubs are properly named (exact names matching 04-VALIDATION.md Per-Task Verification Map) + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-01-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-02-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-02-PLAN.md new file mode 100644 index 0000000..76d09e5 --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-02-PLAN.md @@ -0,0 +1,216 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 02 +type: execute +wave: 1 +depends_on: [] +files_modified: + - backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py + - backend/storage/minio_backend.py +autonomous: true +requirements: + - FOLD-05 + - DOC-02 + - ADMIN-06 + +must_haves: + truths: + - "Alembic migration 0004 upgrades cleanly; downgrade reverses all DDL changes" + - "users.pdf_open_mode column exists with server_default 'in_app'" + - "GIN expression index ix_documents_fts on documents.extracted_text exists in PostgreSQL" + - "MinIOBackend.put_object_raw() is callable for audit-logs bucket writes" + - "audit-logs MinIO bucket is created when MINIO_ENDPOINT is set" + artifacts: + - path: "backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py" + provides: "Alembic migration: pdf_open_mode column, GIN FTS index, audit-logs bucket creation" + - path: "backend/storage/minio_backend.py" + provides: "put_object_raw(bucket, key, data, length, content_type) method added" + key_links: + - from: "backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py" + to: "backend/db/models.py" + via: "Alembic op.execute GIN index must not collide with ORM model table_args" + pattern: "ix_documents_fts" + - from: "backend/tasks/audit_tasks.py" + to: "backend/storage/minio_backend.py" + via: "put_object_raw called by daily export task" + pattern: "put_object_raw" +--- + + +Create Alembic migration 0004 and add MinIOBackend.put_object_raw(). This plan has no +dependency on the test scaffolds plan (04-01) and can run in parallel. + +Purpose: Establish the database schema additions (pdf_open_mode, FTS index) and storage +method (put_object_raw for audit CSV upload) that later plans depend on. +Output: Migration file 0004 + extended MinIOBackend. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@backend/migrations/versions/0003_multi_user_isolation.py +@backend/storage/minio_backend.py +@backend/db/models.py + + + + + + + + + + + + + + Task 1: Alembic migration 0004 — pdf_open_mode column + GIN FTS index + audit-logs bucket + backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py + + backend/migrations/versions/0003_multi_user_isolation.py — read the entire file; extract the batch_alter_table pattern for adding columns, the MinIO bucket creation pattern gated on MINIO_ENDPOINT env var, the module docstring format, and the downgrade() function pattern + backend/db/models.py — read lines 44-70 (User model) to confirm pdf_open_mode does NOT already exist before adding it + + + Create backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py. + + Module docstring (follow 0003 format): list three changes in order — (1) users.pdf_open_mode column, (2) GIN expression index on documents.extracted_text, (3) audit-logs MinIO bucket creation. + + Set: revision = "0004", down_revision = "0003", branch_labels = None, depends_on = None. + + upgrade() function — three steps in this exact order: + + Step 1: Add users.pdf_open_mode column using batch_alter_table (per D-10, for SQLite compat): + `with op.batch_alter_table("users") as batch_op: batch_op.add_column(sa.Column("pdf_open_mode", sa.String(), nullable=False, server_default="in_app"))` + + Step 2: Create GIN expression index MANUALLY — do NOT use Column Computed() or Index() — use op.execute with raw SQL (per D-11, Pitfall 2, PATTERNS.md). The exact SQL is: + `CREATE INDEX ix_documents_fts ON documents USING GIN (to_tsvector('english', coalesce(extracted_text, '')))` + Add a comment above this line: `# managed manually — do not autogenerate (Alembic issue #1390)`. + + Step 3: Create the audit-logs MinIO bucket gated on MINIO_ENDPOINT env var (per Pitfall 8, PATTERNS.md). Follow the exact same guard pattern from migration 0003: `if os.environ.get("MINIO_ENDPOINT"):` then instantiate Minio client using env vars MINIO_ENDPOINT, MINIO_ACCESS_KEY, MINIO_SECRET_KEY with secure=False. Check `client.bucket_exists("audit-logs")` and call `client.make_bucket("audit-logs")` only if it does not exist. + + downgrade() function: (1) `op.execute("DROP INDEX IF EXISTS ix_documents_fts")`, (2) `with op.batch_alter_table("users") as batch_op: batch_op.drop_column("pdf_open_mode")`. Add a comment: `# MinIO bucket NOT reversed — bucket may contain audit data`. + + Imports needed at module top: `from __future__ import annotations`, `import os`, `import sqlalchemy as sa`, `from alembic import op`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "from migrations.versions import 0004_phase4_pdf_open_mode_tsvector" 2>&1 || python -c "import importlib.util; spec = importlib.util.spec_from_file_location('m', 'migrations/versions/0004_phase4_pdf_open_mode_tsvector.py'); m = importlib.util.module_from_spec(spec); spec.loader.exec_module(m); print('OK')" + + + - File exists at backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py + - `revision = "0004"` and `down_revision = "0003"` present in file + - File contains `batch_alter_table("users")` with pdf_open_mode column, server_default="in_app" + - File contains `CREATE INDEX ix_documents_fts ON documents USING GIN` (grep confirms) + - File contains `# managed manually — do not autogenerate` comment + - File contains `if os.environ.get("MINIO_ENDPOINT")` MinIO gate for audit-logs bucket + - File contains `audit-logs` bucket name (grep confirms) + - File contains `DROP INDEX IF EXISTS ix_documents_fts` in downgrade() + - `python -c "import py_compile; py_compile.compile('migrations/versions/0004_phase4_pdf_open_mode_tsvector.py')"` exits 0 (no syntax errors) + + Migration file is syntactically valid and contains all three upgrade steps + downgrade reversal. + + + + Task 2: Add put_object_raw() to MinIOBackend + backend/storage/minio_backend.py + + backend/storage/minio_backend.py — read the entire file; identify the existing put_object() method signature and asyncio.to_thread() call pattern; identify the class definition and existing imports including io import + backend/storage/base.py — read to see if StorageBackend ABC needs a corresponding abstract method added (it does NOT — put_object_raw is concrete only on MinIOBackend, not on the ABC, because only MinIOBackend has an audit-logs bucket) + + + Add the put_object_raw() async method to the MinIOBackend class in backend/storage/minio_backend.py. + + Place it immediately after the existing put_object() method. + + Method signature: `async def put_object_raw(self, bucket: str, key: str, data: io.BytesIO, length: int, content_type: str) -> None:` + + Docstring: "Upload bytes to an arbitrary bucket+key (used for audit-logs CSV export). Unlike put_object(), does NOT apply the document key schema — the caller supplies the complete key. The main documents bucket is NOT used." + + Implementation: call `await asyncio.to_thread(self._client.put_object, bucket, key, data, length=length, content_type=content_type)`. This mirrors the exact pattern of the existing put_object() method (which uses self._client.put_object via asyncio.to_thread). + + Do NOT add io import if it already exists. Do NOT add this method to StorageBackend ABC (base.py) — this is MinIOBackend-only. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "from storage.minio_backend import MinIOBackend; import inspect; src = inspect.getsource(MinIOBackend.put_object_raw); print('OK')" + + + - MinIOBackend.put_object_raw exists as an async method (grep: `async def put_object_raw`) + - Method accepts: bucket: str, key: str, data: io.BytesIO, length: int, content_type: str + - Method body calls `asyncio.to_thread(self._client.put_object, bucket, key, data, length=length, content_type=content_type)` + - `python -c "from storage.minio_backend import MinIOBackend"` exits 0 (no import errors) + - StorageBackend ABC (base.py) is NOT modified (grep: `put_object_raw` absent from base.py) + + MinIOBackend.put_object_raw() is importable and callable; base.py unchanged. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Migration → MinIO | Bucket creation uses env var credentials; no credentials in code | +| put_object_raw → MinIO SDK | Caller supplies bucket name; method does not validate bucket against allowlist | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-02-01 | Tampering | migration 0004 GIN index | mitigate | Index created via raw SQL (not autogenerate) to prevent Alembic repeat-generation bug; comment documents the decision | +| T-04-02-02 | Information Disclosure | audit-logs MinIO bucket | mitigate | Bucket creation gated on MINIO_ENDPOINT env var (no hardcoded credentials); bucket policy is private-by-default (MinIO default) | +| T-04-02-03 | Tampering | put_object_raw caller-supplied key | accept | put_object_raw is called only from trusted application code (Celery task); key is constructed from application logic, not user input | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. Verify migration syntax: `python -c "import py_compile; py_compile.compile('backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py')"` +2. Verify MinIOBackend import: `cd backend && python -c "from storage.minio_backend import MinIOBackend; print(hasattr(MinIOBackend, 'put_object_raw'))"` +3. Grep checks: `grep -n "ix_documents_fts\|audit-logs\|pdf_open_mode\|put_object_raw" backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py backend/storage/minio_backend.py` + + + +- Migration 0004 file is syntactically valid Python with revision="0004" and down_revision="0003" +- Migration contains all three upgrade steps: pdf_open_mode column, GIN FTS index, audit-logs bucket creation +- MinIOBackend.put_object_raw() is an async method that delegates to asyncio.to_thread +- No existing tests regress: `cd backend && pytest -v --no-header 2>&1 | grep -E "FAILED|ERROR"` returns nothing + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-02-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-03-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-03-PLAN.md new file mode 100644 index 0000000..20d38be --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-03-PLAN.md @@ -0,0 +1,289 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 03 +type: execute +wave: 2 +depends_on: + - "04-01" + - "04-02" +files_modified: + - backend/services/audit.py + - backend/api/folders.py + - backend/api/documents.py + - backend/main.py +autonomous: true +requirements: + - FOLD-01 + - FOLD-02 + - FOLD-03 + - FOLD-04 + - FOLD-05 + +must_haves: + truths: + - "User can create, rename, delete, and list folders via REST API" + - "Deleting a non-empty folder cascade-deletes all documents (MinIO + DB) and decrements quota atomically" + - "GET /api/folders/{id} returns a breadcrumb array from root to the requested folder" + - "Document list supports sort by name, date, size via ?sort= query param" + - "Full-text search via ?q= uses plainto_tsquery on documents.extracted_text GIN index" + - "write_audit_log() helper is available for all Phase 4 handlers" + - "Duplicate folder name under same parent returns 409" + artifacts: + - path: "backend/services/audit.py" + provides: "write_audit_log() async helper — flush-not-commit, never-raises" + exports: ["write_audit_log"] + - path: "backend/api/folders.py" + provides: "FOLD-01..05 endpoints: POST /api/folders, GET /api/folders, GET /api/folders/{id}, PATCH /api/folders/{id}, DELETE /api/folders/{id}, PATCH /api/documents/{id}/folder" + - path: "backend/main.py" + provides: "folders router registered; all Phase 4 routers wired" + key_links: + - from: "backend/api/folders.py" + to: "backend/services/audit.py" + via: "write_audit_log() call after successful folder operations" + pattern: "write_audit_log" + - from: "backend/api/folders.py" + to: "backend/db/models.py" + via: "Folder, Document, Quota ORM models" + pattern: "from db.models import Folder" + - from: "backend/api/documents.py" + to: "backend/api/folders.py" + via: "PATCH /api/documents/{id}/folder uses same ownership assertion as documents.py" + pattern: "get_regular_user" +--- + + +Implement the audit service helper and all folder backend endpoints (FOLD-01..05). +This plan introduces write_audit_log() which subsequent plans (shares, proxy, audit viewer) depend on. + +Purpose: Deliver the complete folder CRUD and document organization API. +Output: backend/services/audit.py + backend/api/folders.py + main.py router registration. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md +@backend/api/documents.py +@backend/db/models.py +@backend/deps/auth.py +@backend/deps/db.py + + + + + + + + + + + + + + + + + + Task 1: Create backend/services/audit.py — write_audit_log() helper + backend/services/audit.py + + backend/services/storage.py — read the entire file; extract the import pattern for async service modules and how AsyncSession is used; note the module-level logger pattern + backend/db/models.py — search for the AuditLog class definition; read it fully to confirm attribute names: event_type, user_id, actor_id, resource_id, ip_address, metadata_ (ORM attribute; DB column name is "metadata" per CLAUDE.md note) + + + - write_audit_log() called with all fields: session, event_type (str), user_id (Optional[uuid.UUID]), actor_id (Optional[uuid.UUID]), resource_id (Optional[uuid.UUID]), ip_address (Optional[str]), metadata_ (Optional[dict]) + - Function creates AuditLog ORM instance, adds to session, calls session.flush() (NOT commit) + - Function NEVER raises — exception caught, logged as warning, then swallowed + - AuditLog.metadata_ ORM attribute (not "metadata") used per CLAUDE.md note about reserved SQLAlchemy attribute name + + + Create backend/services/audit.py. + + Imports: `from __future__ import annotations`, `import logging`, `import uuid`, `from typing import Optional`, `from sqlalchemy.ext.asyncio import AsyncSession`, `from db.models import AuditLog`. + + Module-level: `logger = logging.getLogger(__name__)`. + + Implement async def write_audit_log with signature: `(session: AsyncSession, event_type: str, user_id: Optional[uuid.UUID], actor_id: Optional[uuid.UUID], resource_id: Optional[uuid.UUID], ip_address: Optional[str], metadata_: Optional[dict] = None) -> None`. + + Body: try block that creates `AuditLog(event_type=event_type, user_id=user_id, actor_id=actor_id, resource_id=resource_id, ip_address=ip_address, metadata_=metadata_)`, calls `session.add(entry)`, then `await session.flush()`. Except block: `except Exception as exc: logger.warning("audit log write failed: %s", exc)`. No re-raise. + + Critical: use `session.flush()` not `session.commit()` — this is a hard architectural requirement (D-14). + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "from services.audit import write_audit_log; import inspect; sig = inspect.signature(write_audit_log); print(list(sig.parameters.keys()))" + + + - backend/services/audit.py exists and is importable + - write_audit_log is an async function (grep: `async def write_audit_log`) + - Function uses `await session.flush()` not `await session.commit()` (grep: `session.flush` present; `session.commit` absent from this file) + - Function has a bare `except Exception` that logs and does NOT re-raise (grep: `logger.warning` inside except block) + - `python -c "from services.audit import write_audit_log"` exits 0 + + write_audit_log() is importable; uses flush-not-commit; never raises. + + + + Task 2: Create backend/api/folders.py — all FOLD-01..05 endpoints + backend/api/folders.py, backend/main.py + + backend/api/documents.py — read the entire file; extract the import block, UUID parse pattern (try/except ValueError → 404), ownership assertion pattern (resource is None or resource.user_id != current_user.id → 404), atomic quota UPDATE text(), and the list_documents handler signature for the sort/search extension + backend/main.py — read the entire file; identify the include_router section to know where to add the folders router + backend/db/models.py — read the Folder class and Document class fully to confirm all column names (parent_id, folder_id FK on Document) + + + POST /api/folders: + - Body: {name: str, parent_id: Optional[str] = null} + - Auth: get_regular_user (403 for admin per CLAUDE.md) + - If parent_id provided: assert parent folder exists and belongs to current_user (→ 404 if not) + - Create Folder(user_id=current_user.id, name=name, parent_id=parsed_parent_uuid or None) + - Catch IntegrityError → 409 "A folder with that name already exists here" (Pitfall 6) + - Call write_audit_log after commit: event_type="folder.created", resource_id=folder.id, metadata_={"name": folder.name} + - Return 201 with folder JSON + + GET /api/folders (list top-level): returns folders where user_id=current_user.id and parent_id IS NULL + + GET /api/folders/{id}: returns folder JSON + breadcrumb array + - Breadcrumb: iterative Python walk up parent chain (not WITH RECURSIVE — avoids SQLite incompatibility in unit tests) + - Walk: start from folder, load parent via session.get(Folder, folder.parent_id), repeat until parent_id is None + - Breadcrumb array: [{id: str, name: str}, ...] from root to current folder (root first) + - Response: {id, name, parent_id, user_id, created_at, breadcrumb: [...]} + + PATCH /api/folders/{id}: rename folder + - Body: {name: str} + - Assert folder.user_id == current_user.id → 404 if not + - Catch IntegrityError → 409 on duplicate name + - Call write_audit_log: event_type="folder.renamed" + + DELETE /api/folders/{id}: delete folder + - Assert folder.user_id == current_user.id → 404 if not + - Collect all documents in this folder only (direct children — parent is not responsible for sub-folder docs via cascade; D-03 says cascade-delete but the UI only deletes the selected folder tree, not sub-folders of sub-folders; use WITH RECURSIVE via sqlalchemy text() for PostgreSQL; in test environment without PostgreSQL the recursive CTE will be skipped via a try/except fallback to direct children only) + - Use WITH RECURSIVE CTE (via text()) to find all folder IDs in subtree (Pitfall 1) — wrap in try/except OperationalError for SQLite test compat, fallback collects only direct children + - Sum size_bytes of all collected documents + - Atomic quota decrement with CASE WHEN pattern + - Delete MinIO objects best-effort (try/except per each object, per PATTERNS.md Pattern 2) + - Delete all documents via ORM, then delete folder via session.delete(folder) + - Call write_audit_log: event_type="folder.deleted", metadata_={"doc_count": len(docs), "name": folder.name} + - Return 204 + + PATCH /api/documents/{id}/folder (move document): + - Body: {folder_id: Optional[str]} — null = move to root (no folder) + - Assert doc.user_id == current_user.id → 404 if not + - If folder_id provided: assert target folder.user_id == current_user.id → 404 if not (Pitfall from CONTEXT.md) + - Update doc.folder_id and commit + - Return 200 with updated doc JSON + + Extension to GET /api/documents list endpoint (in documents.py, not folders.py): + - Add query params: sort (str, default "date"), order (str, default "desc"), folder_id (Optional[str]), q (Optional[str]) + - sort=name → order_by(Document.filename), sort=size → order_by(Document.size_bytes), sort=date → order_by(Document.created_at) + - order=asc → .asc(), order=desc → .desc() + - folder_id provided → add .where(Document.folder_id == parsed_uuid) + - q provided and len >= 2: add FTS where clause using func.to_tsvector("english", func.coalesce(Document.extracted_text, "")).op("@@")(func.plainto_tsquery("english", q)) — import sqlalchemy.func + - Add is_shared subquery: select(Share.document_id).where(Share.owner_id == current_user.id).scalar_subquery(); add Document.id.in_(shared_subq).label("is_shared") to select + + + Create backend/api/folders.py with APIRouter(prefix="/api/folders", tags=["folders"]). + + Imports: `from __future__ import annotations`, `import uuid`, `from typing import Optional`, `from fastapi import APIRouter, Depends, HTTPException, Query, Request, status`, `from pydantic import BaseModel`, `from sqlalchemy import select, text, delete, func`, `from sqlalchemy.exc import IntegrityError, OperationalError`, `from sqlalchemy.ext.asyncio import AsyncSession`, `from db.models import Document, Folder, Quota, Share`, `from deps.auth import get_regular_user`, `from deps.db import get_db`, `from storage import get_storage_backend`, `from services.audit import write_audit_log`. + + Pydantic models: FolderCreate(name: str, parent_id: Optional[str] = None), FolderRename(name: str), DocumentMove(folder_id: Optional[str] = None). + + Implement all five endpoints as specified in the behavior block. For the WITH RECURSIVE CTE, use: + `await session.execute(text("WITH RECURSIVE subtree AS (SELECT id FROM folders WHERE id = :root_id AND user_id = :uid UNION ALL SELECT f.id FROM folders f JOIN subtree s ON f.parent_id = s.id WHERE f.user_id = :uid) SELECT id FROM subtree"), {"root_id": str(folder.id), "uid": str(current_user.id)})` + Wrap in try/except OperationalError and fallback to `select(Document).where(Document.folder_id == folder.id, Document.user_id == current_user.id)`. + + For the ip_address extraction pattern (per Pitfall 5): `ip_address = request.headers.get("X-Forwarded-For") or (request.client.host if request.client else None)`. + + After creating backend/api/folders.py, modify backend/main.py: add `from api.folders import router as folders_router` and `app.include_router(folders_router)` in the routes section alongside the other routers. + + Also extend backend/api/documents.py: add sort, order, folder_id, q query params to the list_documents handler. Add import for func and Share. Add the is_shared subquery to the select. Add FTS where clause when q is present. Do NOT break existing tests — these are additive changes. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_folders.py -x -v --no-header 2>&1 | tail -30 + + + - backend/api/folders.py exists with all five endpoint functions (grep: `async def create_folder`, `async def list_folders`, `async def get_folder`, `async def rename_folder`, `async def delete_folder`) + - backend/api/folders.py contains PATCH /api/documents/{id}/folder endpoint (grep: `move_document` or equivalent) + - backend/main.py includes folders router (grep: `folders_router`) + - `python -c "from api.folders import router"` exits 0 + - All folder ownership assertions use 404 not 403 (grep: `status_code=404` in folders.py; no `status_code=403`) + - IntegrityError caught and returns 409 (grep: `IntegrityError` and `409` in folders.py) + - write_audit_log called after folder create, rename, delete (grep: `write_audit_log` appears at least 3 times) + - test_create_folder and test_rename_folder turn green (xpass) or remain xfail — no FAILED + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Folders API endpoints are implemented; document list has sort/search; all existing tests still pass. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Client → POST /api/folders | Untrusted name/parent_id input; Pydantic validates types; IntegrityError → 409 | +| Client → DELETE /api/folders/{id} | Untrusted folder_id path parameter; UUID parse → 404; ownership assertion → 404 | +| Client → GET /api/documents?q= | Untrusted search query; plainto_tsquery is injection-safe (parameterized via SQLAlchemy func) | +| Client → PATCH /api/documents/{id}/folder | Untrusted doc_id + folder_id; both validated via ownership assertion | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-03-01 | Elevation of Privilege | POST/PATCH/DELETE /api/folders | mitigate | get_regular_user dep: admin role returns 403; regular user passes through | +| T-04-03-02 | Information Disclosure | GET /api/documents?q= (FTS scope) | mitigate | FTS query MUST include Document.user_id == current_user.id filter; plainto_tsquery is parameterized (no injection) | +| T-04-03-03 | Tampering | DELETE /api/folders/{id} cascade-delete | mitigate | Ownership assertion on folder before cascade; atomic quota decrement (CASE WHEN); MinIO deletion is best-effort (does not abort primary operation on failure) | +| T-04-03-04 | Information Disclosure | folder IDOR via path parameter | mitigate | All folder endpoints assert folder.user_id == current_user.id → 404 (not 403 — prevents attacker enumeration of folder IDs) | +| T-04-03-05 | Information Disclosure | PATCH /api/documents/{id}/folder cross-user target folder | mitigate | Both document ownership AND target folder ownership asserted → 404 on mismatch | +| T-04-03-06 | Tampering | folder name UniqueConstraint violation surfaced as 500 | mitigate | IntegrityError caught → 409 Conflict with clear error message (Pitfall 6) | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. Endpoint smoke: `cd backend && python -m pytest tests/test_folders.py -v --no-header 2>&1 | tail -30` +2. Full suite: `cd backend && python -m pytest -v --no-header 2>&1 | grep -E "FAILED|ERROR"` — expect empty +3. Import check: `cd backend && python -c "from api.folders import router; from services.audit import write_audit_log; print('OK')"` +4. Security grep: `grep -n "get_regular_user\|status_code=404\|IntegrityError\|write_audit_log\|plainto_tsquery" backend/api/folders.py` + + + +- write_audit_log() is importable from services.audit; uses session.flush(); never raises +- All five folder endpoints exist in api/folders.py; router registered in main.py +- Document list endpoint supports sort, folder_id, q params; is_shared field returned +- test_folders.py tests turn from xfail to xpass (green) or remain xfail — zero FAILED +- Full pytest suite is green + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-03-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-04-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-04-PLAN.md new file mode 100644 index 0000000..cbd2930 --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-04-PLAN.md @@ -0,0 +1,229 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 04 +type: execute +wave: 3 +depends_on: + - "04-01" + - "04-02" + - "04-03" +files_modified: + - backend/api/shares.py + - backend/main.py +autonomous: true +requirements: + - SHARE-01 + - SHARE-02 + - SHARE-03 + - SHARE-04 + - SHARE-05 + +must_haves: + truths: + - "User can share a document with another user by their exact handle" + - "GET /api/shares/received returns documents shared with the current user (virtual folder)" + - "Recipient's quota is never modified by a share operation" + - "Share revocation is immediate: DELETE /api/shares/{id} with owner assertion" + - "Sharing the same document with the same user twice returns 409" + - "Wrong-owner revocation attempt returns 404 (IDOR prevention)" + artifacts: + - path: "backend/api/shares.py" + provides: "POST /api/shares, GET /api/shares, GET /api/shares/received, DELETE /api/shares/{id}" + exports: ["router"] + key_links: + - from: "backend/api/shares.py" + to: "backend/services/audit.py" + via: "write_audit_log called after share grant and revoke" + pattern: "write_audit_log" + - from: "backend/api/shares.py" + to: "backend/db/models.py" + via: "Share ORM model; User.handle for recipient lookup" + pattern: "Share.owner_id|Share.recipient_id|User.handle" +--- + + +Implement the document sharing API: grant share by handle, list owned shares per document, +list "shared with me" virtual folder, and revoke share with IDOR protection. + +Purpose: Deliver SHARE-01 through SHARE-05 and the security invariant test for share IDOR. +Output: backend/api/shares.py + main.py registration. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@backend/api/documents.py +@backend/api/admin.py +@backend/db/models.py +@backend/deps/auth.py +@backend/services/audit.py + + + + + + + + + + + + + + + + + + + + Task 1: Create backend/api/shares.py — full sharing API + backend/api/shares.py, backend/main.py + + backend/db/models.py — read the Share class definition fully; confirm column names (document_id, owner_id, recipient_id, permission); check for UniqueConstraint on (document_id, recipient_id); confirm User.handle column name + backend/api/documents.py — read the UUID parse pattern (try: uid = uuid.UUID(doc_id) except ValueError: raise HTTPException(404)) and the ownership assertion pattern to replicate exactly + backend/api/admin.py — read lines 140-170 for the handle/user lookup pattern used in admin endpoints + backend/services/audit.py — verify the write_audit_log signature (already created in plan 04-03) + backend/main.py — read the include_router section; note current router list to add shares_router + + + POST /api/shares: + - Body: {document_id: str, recipient_handle: str} + - Auth: get_regular_user + - Parse document_id as UUID → 404 on invalid + - Assert document.user_id == current_user.id → 404 (per D-16 ownership rule) + - Look up User by recipient_handle (exact match) → 404 "User not found" if absent (D-04) + - Prevent self-share: if recipient.id == current_user.id → 400 "Cannot share with yourself" + - Create Share(document_id=uid, owner_id=current_user.id, recipient_id=recipient.id, permission="view") + - Catch IntegrityError → 409 "Document already shared with this user" + - Call write_audit_log: event_type="share.granted", resource_id=uid, metadata_={"recipient_id": str(recipient.id)} + - Return 201 with share JSON {id, document_id, owner_id, recipient_id, permission, created_at} + + GET /api/shares?document_id={id} (list shares owned by current user for a document): + - Auth: get_regular_user + - Assert document.user_id == current_user.id → 404 + - Return list of shares for this document: [{id, recipient_id, recipient_handle, permission, created_at}] + - Join Share with User on Share.recipient_id = User.id to get recipient handle + + GET /api/shares/received (virtual "Shared with me" folder — D-06): + - Auth: get_regular_user + - Return documents shared WITH current_user (Share.recipient_id == current_user.id) + - Response: [{doc metadata fields}, owner_handle] — no quota impact, no share_count on this view + - Does NOT return extracted_text (Pitfall 7 — shared docs list shows metadata only) + - Returns: {items: [{id, filename, content_type, size_bytes, created_at, owner_handle}]} + + DELETE /api/shares/{share_id}: + - Auth: get_regular_user + - Parse share_id as UUID → 404 on invalid + - CRITICAL: assert share.owner_id == current_user.id → 404 "Share not found" if mismatch (Pitfall 4, IDOR) + - Delete share; commit + - Call write_audit_log: event_type="share.revoked", resource_id=share.document_id, metadata_={"recipient_id": str(share.recipient_id)} + - Return 204 + + + Create backend/api/shares.py. APIRouter(prefix="/api/shares", tags=["shares"]). + + Imports: `from __future__ import annotations`, `import uuid`, `from typing import Optional`, `from fastapi import APIRouter, Depends, HTTPException, Query, Request`, `from pydantic import BaseModel`, `from sqlalchemy import select`, `from sqlalchemy.exc import IntegrityError`, `from sqlalchemy.ext.asyncio import AsyncSession`, `from db.models import Document, Share, User`, `from deps.auth import get_regular_user`, `from deps.db import get_db`, `from services.audit import write_audit_log`. + + Pydantic request model: ShareCreate(document_id: str, recipient_handle: str). + + Implement all four endpoints in order: POST /api/shares, GET /api/shares, GET /api/shares/received, DELETE /api/shares/{share_id}. + + The GET /api/shares/received endpoint MUST be defined BEFORE DELETE /api/shares/{share_id} in the router — otherwise FastAPI will route GET /api/shares/received as DELETE with share_id="received" (path parameter conflict). + + CRITICAL security requirement: DELETE /api/shares/{share_id} MUST check `share.owner_id != current_user.id` → 404 (not merely that the share exists). This is the IDOR test that the security agent will verify. + + ip_address extraction: `request.headers.get("X-Forwarded-For") or (request.client.host if request.client else None)`. + + After creating shares.py, modify backend/main.py: add `from api.shares import router as shares_router` and `app.include_router(shares_router)`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_shares.py -x -v --no-header 2>&1 | tail -30 + + + - backend/api/shares.py exists with all four endpoint functions + - GET /api/shares/received is defined BEFORE DELETE /api/shares/{share_id} in the file (grep line numbers confirm ordering) + - DELETE /api/shares/{share_id} checks `share.owner_id != current_user.id` → 404 (grep: `share.owner_id != current_user.id` or `share.owner_id == current_user.id` inverted with raise) + - IntegrityError → 409 for duplicate share (grep: `IntegrityError` and `409` in shares.py) + - write_audit_log called for share.granted and share.revoked (grep: `write_audit_log` appears at least twice) + - GET /api/shares/received response does NOT include extracted_text field (grep: `extracted_text` absent from the received endpoint's return dict) + - `python -c "from api.shares import router"` exits 0 + - test_share_revoke_wrong_owner_404 turns green (xpass) or remains xfail — not FAILED + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Shares API is implemented; IDOR protection on revoke confirmed; full test suite passes. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Client → POST /api/shares | Untrusted document_id and recipient_handle; document ownership asserted; handle is exact-match lookup only | +| Client → DELETE /api/shares/{id} | Untrusted share_id; ownership asserted via share.owner_id == current_user.id | +| Client → GET /api/shares/received | Returns only metadata, never extracted_text; scoped to current_user as recipient | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-04-01 | Elevation of Privilege | POST /api/shares | mitigate | get_regular_user dep: admin cannot share documents | +| T-04-04-02 | Information Disclosure | Share IDOR — DELETE /api/shares/{id} | mitigate | Ownership assertion: share.owner_id == current_user.id → 404 if mismatch (not 403 — prevents ID enumeration); test_share_revoke_wrong_owner_404 validates this | +| T-04-04-03 | Information Disclosure | GET /api/shares/received leaking extracted_text | mitigate | Received endpoint returns metadata only: id, filename, content_type, size_bytes, created_at, owner_handle — extracted_text is explicitly excluded | +| T-04-04-04 | Information Disclosure | Recipient quota modified by share | mitigate | Share creation MUST NOT touch quotas table; no quota UPDATE in shares.py | +| T-04-04-05 | Denial of Service | Duplicate share flooding | mitigate | UniqueConstraint(document_id, recipient_id) → IntegrityError → 409; no unbounded inserts | +| T-04-04-06 | Information Disclosure | Share reveals document existence to non-recipients | mitigate | Ownership assertion on POST /api/shares: only document owner can grant; ID enumeration blocked by 404 | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. IDOR test: `cd backend && python -m pytest tests/test_shares.py::test_share_revoke_wrong_owner_404 -v` +2. Full suite: `cd backend && python -m pytest tests/ -v --no-header 2>&1 | grep -E "FAILED|ERROR"` +3. Route order grep: `grep -n "def list_shared_with_me\|def delete_share\|received\|share_id" backend/api/shares.py` +4. extracted_text absence: `grep -n "extracted_text" backend/api/shares.py` — expect empty (no results) + + + +- All four share endpoints implement the behavior contract from CONTEXT.md D-04..D-07 +- IDOR protection: DELETE /api/shares/{id} with wrong owner returns 404 (not 200/204) +- test_shares.py tests turn green or remain xfail — zero FAILED +- Full pytest suite green + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-04-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-05-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-05-PLAN.md new file mode 100644 index 0000000..3099dce --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-05-PLAN.md @@ -0,0 +1,240 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 05 +type: execute +wave: 4 +depends_on: + - "04-03" + - "04-04" +files_modified: + - backend/api/documents.py + - backend/api/auth.py +autonomous: true +requirements: + - DOC-01 + - DOC-02 + - FOLD-04 + +must_haves: + truths: + - "GET /api/documents/{id}/content streams document bytes from MinIO through FastAPI" + - "Range header requests return 206 with correct Content-Range header" + - "Admin cannot access the streaming proxy — get_regular_user dep returns 403" + - "No presigned URL is generated or exposed in the proxy response" + - "Share recipients can access document content via the proxy" + - "PATCH /api/auth/me/preferences stores and returns pdf_open_mode preference" + artifacts: + - path: "backend/api/documents.py" + provides: "GET /api/documents/{id}/content streaming proxy with Range header support" + - path: "backend/api/auth.py" + provides: "PATCH /api/auth/me/preferences endpoint for pdf_open_mode" + key_links: + - from: "backend/api/documents.py" + to: "backend/storage/minio_backend.py" + via: "get_storage_backend().get_object(doc.object_key) — bytes fetched directly, not via presigned URL" + pattern: "get_object" + - from: "backend/api/documents.py" + to: "backend/db/models.py" + via: "Share model for _can_access_document() recipient check" + pattern: "Share.recipient_id" +--- + + +Add the PDF streaming proxy to the documents API (DOC-02) and the pdf_open_mode preferences +endpoint to the auth API (D-10, DOC-01). These can be implemented together since they both +modify existing API modules and neither depends on the other. + +Purpose: Complete document content access — both the proxy stream and the user preference for +how to open PDFs. +Output: Streaming proxy endpoint in documents.py + preferences PATCH in auth.py. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md +@backend/api/documents.py +@backend/api/auth.py +@backend/storage/minio_backend.py +@backend/db/models.py + + + + + + + + + + + + + + + + + + Task 1: Add GET /api/documents/{id}/content streaming proxy to documents.py + backend/api/documents.py + + backend/api/documents.py — read the entire file; identify current imports (Request, StreamingResponse if already present; func, Select, etc.); find the endpoint list to determine where to insert the new endpoint; confirm existing UUID parse pattern and get_regular_user usage + backend/storage/minio_backend.py — read to confirm the exact name and signature of get_object(); confirm it returns bytes (not a stream object) + backend/db/models.py — search for the Share class definition; confirm Share.document_id and Share.recipient_id column names + + + GET /api/documents/{id}/content: + - MUST use get_regular_user dep (admin role → 403 — Pitfall 3, CRITICAL) + - Parse doc_id as UUID → 404 "Document not found" on ValueError + - Load document via session.get(Document, uid) → 404 if None + - Access check: if doc.user_id == current_user.id → proceed; else query Share where Share.document_id == doc.id AND Share.recipient_id == current_user.id; if no share → 404 "Document not found" + - Fetch bytes: `file_bytes = await get_storage_backend().get_object(doc.object_key)` — this calls MinIO directly, NO presigned URL + - Build base headers: content-type = doc.content_type, content-disposition = `inline; filename="{doc.filename}"`, accept-ranges = "bytes", content-length = str(file_size) + - Range header handling: if request.headers.get("range") is truthy, call _parse_range() → on valid range: add content-range header, set content-length to chunk length, return StreamingResponse(iter([chunk]), status_code=206, headers=headers) + - No range: return StreamingResponse(iter([file_bytes]), status_code=200, headers=headers) + - _parse_range() is a module-level helper function, not an endpoint (defined above the endpoint) + + DOC-01 note: GET /api/documents/{id} already returns extracted_text — verify this is still included in the existing response; add it to the response if absent (read the actual endpoint to check before modifying). + + + Modify backend/api/documents.py. + + Add to imports if not already present: `from fastapi import Request` (alongside existing FastAPI imports), `from fastapi.responses import StreamingResponse`, `from fastapi import status`. Confirm `from db.models import Share` is in imports (added in plan 04-03 for the is_shared subquery — if absent, add it now). + + Define _parse_range() helper function at module level (above the stream_document_content endpoint). Implement exactly as specified in the behavior block and RESEARCH.md Pattern 3: parse "bytes=X-Y" format, handle open-ended ranges (end = file_size - 1 when Y is empty), raise HTTPException(status.HTTP_416_REQUESTED_RANGE_NOT_SATISFIABLE) on invalid range. + + Add the stream_document_content endpoint after the existing document endpoints. Implement the access check using an inline query for Share (do NOT call a separate helper function — the access check is in the handler body for clarity and to avoid test mocking complexity). + + CRITICAL: endpoint decorator uses `get_regular_user` not `get_current_user`. This is verified by the security agent and by test_content_stream_admin_403. + + CRITICAL: NEVER call presigned_get_url() or any presigned URL method inside this endpoint. Bytes must flow directly from MinIO through FastAPI. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_documents.py -x -v --no-header 2>&1 | tail -30 + + + - `GET /api/documents/{id}/content` endpoint exists in documents.py (grep: `async def stream_document_content` or similar) + - Endpoint uses `get_regular_user` dep, NOT `get_current_user` (grep: `Depends(get_regular_user)` on the content endpoint; no `get_current_user` on content endpoint) + - _parse_range() helper exists at module level (grep: `def _parse_range`) + - StreamingResponse imported and used (grep: `StreamingResponse` in documents.py) + - No presigned URL call in the handler (grep: `presigned` absent from stream_document_content function body) + - Share access check present (grep: `Share.recipient_id` or `recipient_id` in documents.py) + - test_content_stream_admin_403 turns green (xpass) or remains xfail — not FAILED + - test_content_stream_no_presigned_url turns green or remains xfail — not FAILED + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Streaming proxy delivers bytes via StreamingResponse; admin blocked; Range headers supported; no presigned URL exposure. + + + + Task 2: Add PATCH /api/auth/me/preferences endpoint for pdf_open_mode + backend/api/auth.py + + backend/api/auth.py — read the entire file; identify the /api/auth/me GET endpoint (existing); find the router prefix and how it is declared; confirm get_current_user (not get_regular_user) is used for /me endpoints (admin can read/update their own preferences); find the existing import block to add pydantic BaseModel if needed + backend/db/models.py — confirm users.pdf_open_mode column name and that it exists (added by migration 0004) + + + PATCH /api/auth/me/preferences: + - Auth: get_current_user (both regular users and admins can set their own PDF preference — D-10) + - Body: {pdf_open_mode: str} — validated to be one of ["in_app", "new_tab"] + - If pdf_open_mode not in allowed values → 422 (Pydantic validation handles this via Literal type or custom validator) + - Update current_user.pdf_open_mode = body.pdf_open_mode + - session.add(current_user); await session.commit() + - Return 200: {pdf_open_mode: current_user.pdf_open_mode} + + GET /api/auth/me/preferences (optional but recommended for frontend to load initial value): + - Auth: get_current_user + - Return 200: {pdf_open_mode: current_user.pdf_open_mode} + - If pdf_open_mode column is missing from User model (migration not yet run), return {"pdf_open_mode": "in_app"} as default + + + Modify backend/api/auth.py. + + Add a Pydantic request model: PreferencesUpdate with field `pdf_open_mode: Literal["in_app", "new_tab"]`. Import `Literal` from typing. + + Add two new endpoints at the end of the router function list: + 1. GET /api/auth/me/preferences — returns {pdf_open_mode: str}. Uses `get_current_user` dep. Reads current_user.pdf_open_mode. If AttributeError (column not yet in ORM map, e.g., test env without migration), return {"pdf_open_mode": "in_app"}. + 2. PATCH /api/auth/me/preferences — accepts PreferencesUpdate body. Updates current_user.pdf_open_mode. Commits. Returns {pdf_open_mode: updated value}. + + Both endpoints are placed after the existing /api/auth/me endpoint. No new router prefix needed — they use the same auth router. + + Do NOT break any existing auth tests. Do NOT modify any existing endpoint. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "from api.auth import router; print([r.path for r in router.routes])" 2>&1 + + + - `/api/auth/me/preferences` appears in auth router routes (grep: `me/preferences` in auth.py) + - PATCH endpoint validates pdf_open_mode via Literal["in_app", "new_tab"] (grep: `Literal` and `pdf_open_mode` in auth.py) + - `python -c "from api.auth import router"` exits 0 + - Existing auth tests still pass: `cd backend && python -m pytest tests/test_auth_api.py -v --no-header 2>&1 | grep -E "^FAILED"` returns nothing + - Full suite: `cd backend && python -m pytest -v --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + PATCH /api/auth/me/preferences stores pdf_open_mode; GET returns current value; existing auth tests unaffected. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Client → GET /api/documents/{id}/content | Untrusted doc_id; access checked via ownership or active share; bytes flow from MinIO through FastAPI only | +| Client → Range header | Untrusted byte ranges; _parse_range() validates start/end bounds against file_size; invalid range → 416 | +| Client → PATCH /api/auth/me/preferences | Untrusted pdf_open_mode value; Pydantic Literal validates allowlist | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-05-01 | Broken Access Control | GET /api/documents/{id}/content — admin access | mitigate | MUST use get_regular_user dep: admin role → 403; test_content_stream_admin_403 validates; security agent checks | +| T-04-05-02 | Information Disclosure | Presigned URL exposure in proxy response | mitigate | presigned_get_url() NEVER called in stream_document_content; bytes fetched via get_object() directly; test_content_stream_no_presigned_url validates | +| T-04-05-03 | Information Disclosure | Range header bypass — out-of-bounds access | mitigate | _parse_range() validates: start <= end, start >= 0, end < file_size; → 416 on any violation | +| T-04-05-04 | Information Disclosure | Non-recipient accessing shared document via proxy | mitigate | Access check: doc.user_id == current_user.id OR active Share.recipient_id == current_user.id; neither → 404 | +| T-04-05-05 | Tampering | pdf_open_mode mass assignment | mitigate | Pydantic Literal["in_app", "new_tab"] enforces strict allowlist; no other user fields touched by preferences endpoint | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. Security test: `cd backend && python -m pytest tests/test_documents.py::test_content_stream_admin_403 tests/test_documents.py::test_content_stream_no_presigned_url -v` +2. Full suite: `cd backend && python -m pytest tests/ -v --no-header 2>&1 | grep -E "FAILED|ERROR"` +3. Admin dep grep: `grep -A5 "content" backend/api/documents.py | grep "get_regular_user"` — must match +4. Presigned URL grep: `grep -n "presigned" backend/api/documents.py` — confirms absence from stream handler + + + +- GET /api/documents/{id}/content proxies bytes from MinIO; admin gets 403; Range → 206; no presigned URL +- PATCH /api/auth/me/preferences stores valid pdf_open_mode; GET returns current value +- test_content_stream_* tests turn green or remain xfail; zero FAILED +- Full pytest suite green + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-05-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-06-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-06-PLAN.md new file mode 100644 index 0000000..9e14d68 --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-06-PLAN.md @@ -0,0 +1,274 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 06 +type: execute +wave: 4 +depends_on: + - "04-03" + - "04-04" +files_modified: + - backend/api/audit.py + - backend/tasks/audit_tasks.py + - backend/celery_app.py + - backend/main.py +autonomous: true +requirements: + - ADMIN-06 + +must_haves: + truths: + - "Admin can retrieve paginated audit log entries filtered by date range, user, and action type" + - "Audit log entries never contain document content, filenames, or extracted text" + - "Regular user requesting audit log returns 403" + - "CSV export returns streaming response with correct Content-Disposition: attachment header" + - "Celery beat daily export task runs at midnight UTC and uploads CSV to audit-logs bucket" + - "MinIOBackend.put_object_raw() is used for the daily export (not the documents key scheme)" + artifacts: + - path: "backend/api/audit.py" + provides: "GET /api/admin/audit-log (paginated, filtered), GET /api/admin/audit-log/export (CSV stream)" + - path: "backend/tasks/audit_tasks.py" + provides: "audit_log_daily_export Celery task — queries DB, writes CSV, uploads to MinIO audit-logs bucket" + - path: "backend/celery_app.py" + provides: "beat_schedule extended with audit-log-daily-export at midnight UTC" + key_links: + - from: "backend/api/audit.py" + to: "backend/deps/auth.py" + via: "get_current_admin dep on all audit log endpoints" + pattern: "get_current_admin" + - from: "backend/tasks/audit_tasks.py" + to: "backend/storage/minio_backend.py" + via: "put_object_raw(bucket='audit-logs', key='audit-logs/YYYY-MM-DD.csv', ...)" + pattern: "put_object_raw" +--- + + +Implement the admin audit log viewer API (ADMIN-06) and the daily Celery beat export task (D-17). +This plan runs in parallel with plan 04-05 (both depend on Wave 2 completion, neither depends on each other). + +Purpose: Give admins a paginated, filterable audit log and ensure log data is exported daily to MinIO. +Output: backend/api/audit.py + backend/tasks/audit_tasks.py + celery_app.py + main.py updates. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md +@backend/api/admin.py +@backend/tasks/document_tasks.py +@backend/celery_app.py +@backend/db/models.py + + + + + + + + + + + + + + + + + + Task 1: Create backend/api/audit.py — admin audit log viewer + CSV export + backend/api/audit.py, backend/main.py + + backend/api/admin.py — read the entire file; extract the get_current_admin dep usage pattern, the paginated list query pattern (lines ~140-160), the _user_to_dict() whitelist helper pattern, and how the router is prefixed + backend/db/models.py — read the AuditLog class fully; confirm the ORM attribute name is metadata_ (not metadata); confirm all column names + + + GET /api/admin/audit-log: + - Auth: get_current_admin (regular user → 403) + - Query params: start (Optional[datetime] = None), end (Optional[datetime] = None), user_id (Optional[uuid.UUID] = None), event_type (Optional[str] = None), page (int = 1, ge=1), per_page (int = 50, ge=1, le=500) + - Build SQLAlchemy query: select(AuditLog).order_by(AuditLog.created_at.desc()) + - Apply filters: if start → .where(AuditLog.created_at >= start); if end → .where(AuditLog.created_at <= end); if user_id → .where(AuditLog.user_id == user_id); if event_type → .where(AuditLog.event_type == event_type) + - Apply pagination: .limit(per_page).offset((page - 1) * per_page) + - Also run a COUNT query with same filters (no limit/offset) for total + - Return {items: [_audit_to_dict(e) for e in entries], total: count, page: page, per_page: per_page} + + GET /api/admin/audit-log/export: + - Auth: get_current_admin + - Query params: same filters as viewer (start, end, user_id, event_type), format: str = "csv" + - Query all matching rows (no pagination) + - For format="csv": use csv.DictWriter with io.StringIO; write all rows via _audit_to_dict(); return StreamingResponse(iter([output.getvalue()]), media_type="text/csv", headers={"Content-Disposition": "attachment; filename=audit-export.csv"}) + - CRITICAL: _audit_to_dict() must NEVER include filename, extracted_text, or document content keys + + _audit_to_dict() helper: pure whitelist dict — id, event_type, user_id, actor_id, resource_id, ip_address, metadata_, created_at — no other keys possible. + + Register router in main.py. + + + Create backend/api/audit.py. + + Imports: `from __future__ import annotations`, `import csv`, `import io`, `import uuid`, `from datetime import datetime`, `from typing import Optional`, `from fastapi import APIRouter, Depends, Query`, `from fastapi.responses import StreamingResponse`, `from pydantic import BaseModel`, `from sqlalchemy import select, func`, `from sqlalchemy.ext.asyncio import AsyncSession`, `from db.models import AuditLog`, `from deps.auth import get_current_admin`, `from deps.db import get_db`. + + Router: `router = APIRouter(prefix="/api/admin", tags=["audit"])`. + + Define _audit_to_dict() as a module-level function. The whitelist is: id, event_type, user_id (str or None), actor_id (str or None), resource_id (str or None), ip_address (str or None), metadata_ (the JSONB dict value), created_at (isoformat). The function MUST NOT include filename, extracted_text, or any document content fields. Add a docstring: "Safe audit log serializer — never includes filename, extracted_text, or document content (ADMIN-06, D-15)." + + Define _build_query() helper (or inline) that accepts filters and returns a SQLAlchemy Select statement with filters applied. Reuse for both list and export endpoints. + + Implement GET /api/admin/audit-log and GET /api/admin/audit-log/export as specified. + + After creating audit.py, modify backend/main.py: add `from api.audit import router as audit_router` and `app.include_router(audit_router)`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_audit.py -x -v --no-header 2>&1 | tail -30 + + + - backend/api/audit.py exists with GET /api/admin/audit-log and GET /api/admin/audit-log/export endpoints + - Both endpoints use get_current_admin dep (grep: `Depends(get_current_admin)` appears twice in audit.py) + - _audit_to_dict() whitelist does NOT contain filename, extracted_text, password_hash, or credentials_enc (grep: these strings absent from the dict literal in _audit_to_dict) + - CSV export returns StreamingResponse with Content-Disposition: attachment (grep: `attachment` in audit.py) + - `python -c "from api.audit import router"` exits 0 + - test_audit_log_regular_user_403 turns green or remains xfail — not FAILED + - test_audit_log_no_doc_content turns green or remains xfail — not FAILED + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Audit log viewer and CSV export implemented; admin-only access confirmed; no doc content in serializer. + + + + Task 2: Create backend/tasks/audit_tasks.py + extend celery_app.py beat schedule + backend/tasks/audit_tasks.py, backend/celery_app.py + + backend/tasks/document_tasks.py — read the entire file; extract the Celery task decorator pattern, the asyncio.run(_run()) bridge pattern, the AsyncSessionLocal usage with deferred imports inside the async function body, and the error handling pattern + backend/celery_app.py — read the entire file; extract the beat_schedule dict structure, how _timedelta is imported, and where to add the crontab import and new beat entry; find the task_routes dict + + + audit_log_daily_export Celery task: + - Sync Celery entry point: `@celery_app.task(name="tasks.audit_tasks.audit_log_daily_export") def audit_log_daily_export() -> dict: return asyncio.run(_run_daily_export())` + - _run_daily_export() async function: + - Compute yesterday: `yesterday = date.today() - timedelta(days=1)` using `from datetime import date, datetime, timedelta, timezone` + - start = datetime(yesterday.year, yesterday.month, yesterday.day, tzinfo=timezone.utc) + - end = start + timedelta(days=1) + - Open AsyncSessionLocal() session; query AuditLog where created_at >= start AND created_at < end; order by created_at + - Build CSV using csv.DictWriter + io.StringIO; use the same field list as _audit_to_dict: id, event_type, user_id, actor_id, resource_id, ip_address, metadata_, created_at + - csv_bytes = output.getvalue().encode("utf-8") + - key = f"audit-logs/{yesterday.isoformat()}.csv" + - Call `await get_storage_backend().put_object_raw(bucket="audit-logs", key=key, data=io.BytesIO(csv_bytes), length=len(csv_bytes), content_type="text/csv")` + - Return {"exported": len(rows), "key": key, "date": yesterday.isoformat()} + - Error handling: wrap the entire _run_daily_export in try/except; on exception: log error and return {"exported": 0, "error": str(e)} + + celery_app.py beat_schedule extension: + - Import `from celery.schedules import crontab as _crontab` (alias with underscore like _timedelta) + - Add to beat_schedule dict: "audit-log-daily-export" → {"task": "tasks.audit_tasks.audit_log_daily_export", "schedule": _crontab(hour=0, minute=0)} + - Add to task_routes: "tasks.audit_tasks.*": {"queue": "documents"} (reuse documents worker queue per PATTERNS.md) + + + Create backend/tasks/audit_tasks.py. Module docstring: describe the daily audit export task and its MinIO target. + + Top-level imports: `import asyncio`, `from celery_app import celery_app`. All other imports go inside the async function body to avoid circular imports (per established pattern from document_tasks.py). + + Inside _run_daily_export() body (deferred imports): `from datetime import date, datetime, timedelta, timezone`, `import csv`, `import io`, `from db.session import AsyncSessionLocal`, `from db.models import AuditLog`, `from sqlalchemy import select`, `from storage import get_storage_backend`. + + Implement the task and async function exactly as specified in the behavior block. + + Modify backend/celery_app.py: add `from celery.schedules import crontab as _crontab` to imports. Add the new beat entry to `celery_app.conf.beat_schedule`. Add the task route to `celery_app.conf.task_routes`. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "from tasks.audit_tasks import audit_log_daily_export; print('OK')" + + + - backend/tasks/audit_tasks.py exists and is importable without error + - `python -c "from tasks.audit_tasks import audit_log_daily_export"` exits 0 + - audit_log_daily_export is registered with name "tasks.audit_tasks.audit_log_daily_export" (grep: `name="tasks.audit_tasks.audit_log_daily_export"`) + - _run_daily_export calls put_object_raw with bucket="audit-logs" (grep: `put_object_raw` and `audit-logs` in audit_tasks.py) + - celery_app.py beat_schedule contains "audit-log-daily-export" entry with `_crontab(hour=0, minute=0)` (grep: `audit-log-daily-export` and `crontab` in celery_app.py) + - celery_app.py task_routes contains `tasks.audit_tasks.*` (grep confirms) + - All deferred imports are inside the async function body, not at module top level (grep: import lines inside `async def _run_daily_export`) + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Celery export task importable; beat schedule updated; put_object_raw used for audit-logs bucket writes. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Admin → GET /api/admin/audit-log | Admin-authenticated; regular users cannot access | +| Celery worker → MinIO audit-logs bucket | Service-to-service; uses env-var credentials; bucket is private | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-06-01 | Broken Access Control | GET /api/admin/audit-log access by regular user | mitigate | get_current_admin dep: regular user role → 403; test_audit_log_regular_user_403 validates | +| T-04-06-02 | Sensitive Data Exposure | Audit log returning document content | mitigate | _audit_to_dict() whitelist explicitly excludes filename, extracted_text; no other fields can be added without modifying the dict literal (safe-by-default) | +| T-04-06-03 | Information Disclosure | CSV export containing sensitive data | mitigate | CSV export uses the same _audit_to_dict() whitelist as the JSON viewer; both share the same helper function | +| T-04-06-04 | Tampering | audit-logs MinIO bucket publicly accessible | mitigate | Bucket created without public policy (MinIO default is private); confirmed in migration 0004 | +| T-04-06-05 | Denial of Service | Unbounded CSV export | accept | Export scoped by same date/user/event_type filters as viewer; max rows bounded by time window (1 day per task run); admin-only endpoint | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. Audit log tests: `cd backend && python -m pytest tests/test_audit.py -v --no-header` +2. Admin-only check: `grep -n "get_current_admin" backend/api/audit.py` — must appear on both endpoints +3. No doc content in serializer: `grep -n "filename\|extracted_text\|password_hash\|credentials_enc" backend/api/audit.py` — must return nothing from _audit_to_dict function body +4. Celery task import: `cd backend && python -c "from tasks.audit_tasks import audit_log_daily_export; print('OK')"` +5. Full suite: `cd backend && python -m pytest tests/ -v --no-header 2>&1 | grep -E "FAILED|ERROR"` + + + +- Audit log viewer returns paginated, filtered entries with no document content in any field +- CSV export streams valid CSV with Content-Disposition: attachment +- Daily Celery task exports to audit-logs MinIO bucket using put_object_raw +- Beat schedule updated; task route registered +- test_audit.py tests green or xfail; zero FAILED in full suite + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-06-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-07-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-07-PLAN.md new file mode 100644 index 0000000..2086a6c --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-07-PLAN.md @@ -0,0 +1,271 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 07 +type: execute +wave: 5 +depends_on: + - "04-05" + - "04-06" +files_modified: + - backend/api/auth.py + - backend/api/admin.py + - backend/api/documents.py +autonomous: true +requirements: + - SEC-08 + - SEC-09 + - ADMIN-06 + - DOC-01 + - DOC-02 + - SHARE-01 + - SHARE-02 + - SHARE-03 + - SHARE-04 + - SHARE-05 + +must_haves: + truths: + - "Auth events (login, logout, password change, TOTP, sign-out-all) are written to the audit log" + - "Admin events (user create, deactivate, quota change, AI provider assign) are written to the audit log" + - "credentials_enc field is absent from every serialized response across the entire API" + - "Admin delete-user triggers delete_user_files() — MinIO objects deleted before DB records removed" + - "Document upload and delete events are written to the audit log" + artifacts: + - path: "backend/api/auth.py" + provides: "write_audit_log() calls backfilled into login, logout, password change, TOTP, sign-out-all handlers" + - path: "backend/api/admin.py" + provides: "write_audit_log() calls in user create/deactivate/quota/AI-config handlers; delete-user cleanup; CloudConnectionOut Pydantic model for SEC-08" + - path: "backend/api/documents.py" + provides: "write_audit_log() calls in upload confirm and document delete handlers" + key_links: + - from: "backend/api/auth.py" + to: "backend/services/audit.py" + via: "write_audit_log called after successful auth events" + pattern: "write_audit_log" + - from: "backend/api/admin.py" + to: "backend/db/models.py" + via: "delete_user_files() iterates Document records and calls storage.delete_object" + pattern: "delete_user_files\|delete_object" +--- + + +Back-fill audit log writes into auth and admin handlers (D-13), implement SEC-08 +(credentials_enc exclusion) and SEC-09 (delete-user file cleanup). These are security +hardening tasks — they modify existing handlers without changing their primary behavior. + +Purpose: Complete the audit trail (ADMIN-06) and close the two security requirements +(SEC-08, SEC-09) that are in scope for Phase 4. +Output: Audit writes in auth.py, admin.py, documents.py + CloudConnectionOut Pydantic model ++ delete_user_files() cleanup in admin.py. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@backend/api/auth.py +@backend/api/admin.py +@backend/api/documents.py +@backend/services/audit.py +@backend/db/models.py + + + + + + + + + + + + + + + + + + Task 1: Back-fill audit log writes into auth.py and documents.py handlers + backend/api/auth.py, backend/api/documents.py + + backend/api/auth.py — read the entire file; identify all state-changing endpoints: login (POST /api/auth/login), logout (POST /api/auth/logout), password change (POST /api/auth/change-password), TOTP enroll (POST /api/auth/totp/enroll), TOTP verify/enable, backup code use, sign-out-all (POST /api/auth/sign-out-all); note which already inject Request and which need it added + backend/api/documents.py — read the entire file; identify the confirm endpoint (POST /api/documents/{id}/confirm or similar) and the delete endpoint (DELETE /api/documents/{id}); note existing imports + backend/services/audit.py — confirm write_audit_log is importable + + + Auth events to write (D-13 — all 4 categories required): + + 1. login success → event_type="auth.login", user_id=user.id, actor_id=user.id, resource_id=None, metadata_={"totp_used": bool} + 2. login failure → event_type="auth.login_failed", user_id=None, actor_id=None, resource_id=None, metadata_={"email_hash": hmac of email for correlation without PII exposure — or simply omit metadata_} + NOTE: Do NOT log the email or password in metadata_. Log only the IP and event type. + 3. logout → event_type="auth.logout", user_id=current_user.id + 4. password change → event_type="auth.password_changed", user_id=current_user.id + 5. TOTP enrolled → event_type="auth.totp_enrolled", user_id=current_user.id + 6. TOTP revoked (if endpoint exists) → event_type="auth.totp_revoked" + 7. backup code used → event_type="auth.backup_code_used" (on login with backup code) + 8. sign-out-all → event_type="auth.sign_out_all", user_id=current_user.id + + Document events (D-13): + 9. document uploaded (on confirm) → event_type="document.uploaded", user_id=current_user.id, resource_id=doc.id, metadata_={"size_bytes": doc.size_bytes, "storage_backend": "minio"} — MUST NOT include filename or extracted_text + 10. document deleted → event_type="document.deleted", user_id=current_user.id, resource_id=doc.id, metadata_={"size_bytes": doc.size_bytes} + + write_audit_log is called AFTER the successful operation. If the operation fails (exception raised before commit), write_audit_log is not called. Call it just before the return statement. + + + Modify backend/api/auth.py: + - Add import: `from services.audit import write_audit_log` + - For handlers that do not already inject `request: Request`, add it as a parameter + - After each successful auth operation (commit or token issuance), call write_audit_log with appropriate event_type + - IP extraction: `ip_address = request.headers.get("X-Forwarded-For") or (request.client.host if request.client else None)` + - CRITICAL: login_failed event must NOT log email or password in metadata_ — log only IP + + Modify backend/api/documents.py: + - Add import: `from services.audit import write_audit_log` if not already present + - In the confirm/finalize endpoint (where document upload completes): call write_audit_log with event_type="document.uploaded"; metadata_ contains size_bytes and storage_backend only — NOT filename, NOT extracted_text + - In the delete endpoint: call write_audit_log with event_type="document.deleted"; metadata_ contains size_bytes only + + Do NOT change any existing endpoint return values, status codes, or response schemas. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_auth_api.py tests/test_documents.py -x -v --no-header 2>&1 | tail -30 + + + - `from services.audit import write_audit_log` appears in both auth.py and documents.py + - write_audit_log called in at least 6 distinct places across auth.py (grep: `write_audit_log` count >= 6 in auth.py) + - write_audit_log called in at least 2 distinct places in documents.py (confirm + delete) + - auth.py audit calls for document.uploaded and document.deleted do NOT include filename or extracted_text in metadata_ (grep these strings absent from write_audit_log call sites in documents.py) + - Existing auth tests still pass: `pytest tests/test_auth_api.py -v --no-header 2>&1 | grep -E "^FAILED"` returns nothing + - Full suite: `pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Audit writes backfilled into auth and documents handlers; no sensitive data in metadata_; existing tests pass. + + + + Task 2: SEC-08 / SEC-09 hardening + admin event audit writes in admin.py + backend/api/admin.py + + backend/api/admin.py — read the entire file; identify all state-changing admin endpoints: create user, deactivate user, activate user, reset password (admin-triggered), change quota, assign AI provider; find the delete-user endpoint if it exists (SEC-09 requires file cleanup before DB deletion); identify the _user_to_dict() whitelist pattern; find imports + backend/db/models.py — read the CloudConnection model class fully to confirm credentials_enc column name; read the Document model to identify columns used for cleanup + backend/services/audit.py — confirm write_audit_log signature + + + Admin event audit writes (D-13 category 4): + - user created by admin → event_type="admin.user_created", user_id=new_user.id, actor_id=admin.id + - user deactivated → event_type="admin.user_deactivated", user_id=target_user.id, actor_id=admin.id + - user activated → event_type="admin.user_activated", user_id=target_user.id, actor_id=admin.id + - quota changed → event_type="admin.quota_changed", user_id=target_user.id, actor_id=admin.id, metadata_={"old_bytes": old_limit, "new_bytes": new_limit} + - AI provider assigned → event_type="admin.ai_provider_assigned", user_id=target_user.id, actor_id=admin.id, metadata_={"provider": new_provider, "model": new_model} + + SEC-08 — credentials_enc exclusion (D-18): + - Define CloudConnectionOut Pydantic model: id (str), provider (str), display_name (str), status (str), connected_at (datetime) — credentials_enc is ABSENT; model_config = {"from_attributes": True} + - If any admin endpoint currently returns CloudConnection ORM objects as JSON: add response_model=CloudConnectionOut or use explicit dict serialization that excludes credentials_enc + - Phase 4 note: no cloud connection admin endpoints exist yet (Phase 5). Define the model now so Phase 5 cannot accidentally expose credentials_enc. Place it in admin.py schemas section. + + SEC-09 — delete-user file cleanup (D-19): + - If DELETE /api/admin/users/{id} endpoint exists: before deleting DB records, collect all documents for the user via query; call get_storage_backend().delete_object(doc.object_key) for each (best-effort, try/except); decrement quota for deleted files (or simply rely on CASCADE DELETE of quota row since user is deleted); then delete the user DB record + - If DELETE endpoint does not exist: create it with the cleanup logic + - Endpoint: DELETE /api/admin/users/{user_id} — auth: get_current_admin; assert user is not admin (cannot delete admin accounts); cleanup user files; delete user record; write audit log event_type="admin.user_deleted" + + + Modify backend/api/admin.py: + + 1. Add imports: `from services.audit import write_audit_log`, `from datetime import datetime` (if not present), `from storage import get_storage_backend`, `from sqlalchemy import select` (if not present), `from db.models import Document` (if not already imported). + + 2. Define CloudConnectionOut Pydantic model near the top of the file (after existing request models): fields id, provider, display_name, status, connected_at; add docstring "SEC-08: credentials_enc deliberately excluded". + + 3. Add write_audit_log calls after each successful admin state change as specified in the behavior block. ip_address from request.headers or request.client. + + 4. Implement or extend DELETE /api/admin/users/{user_id}: read all user documents, delete MinIO objects best-effort, then call `await session.delete(user)` and `await session.commit()`. Write audit log event_type="admin.user_deleted". + + 5. Do NOT change _user_to_dict() whitelist — it already excludes credentials_enc and password_hash (per Phase 3 implementation). Only add the new CloudConnectionOut model. + + + cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_admin_api.py tests/test_security.py -x -v --no-header 2>&1 | tail -30 + + + - CloudConnectionOut Pydantic model defined in admin.py with no credentials_enc field (grep: `CloudConnectionOut` in admin.py; grep: `credentials_enc` absent from CloudConnectionOut class body) + - write_audit_log called in at least 4 admin state-changing endpoints (grep: `write_audit_log` count >= 4 in admin.py) + - DELETE /api/admin/users/{id} endpoint exists and calls get_storage_backend().delete_object (grep: `delete_object` in admin.py delete-user handler) + - test_credentials_enc_not_in_response turns green or remains xfail — not FAILED + - test_delete_user_cleans_files turns green or remains xfail — not FAILED + - `cd backend && python -m pytest tests/ -x --no-header 2>&1 | grep -E "^FAILED"` returns nothing + - `cd backend && python -m pytest tests/test_admin_api.py -v --no-header 2>&1 | grep -E "^FAILED"` returns nothing + + Audit writes backfilled; CloudConnectionOut model defined without credentials_enc; delete-user cleanup implemented. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Audit write ← auth handler | write_audit_log must not receive PII (email, password) in metadata_ | +| Admin → DELETE /api/admin/users | Must not delete admin accounts; must clean MinIO before DB | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-07-01 | Sensitive Data Exposure | auth.login_failed logs email or password | mitigate | login_failed event metadata_ MUST be empty or contain only non-PII correlation data; email never logged | +| T-04-07-02 | Sensitive Data Exposure | document.uploaded audit event contains filename or extracted_text | mitigate | metadata_ for document.uploaded contains only size_bytes and storage_backend; filename and extracted_text explicitly excluded | +| T-04-07-03 | Sensitive Data Exposure | credentials_enc in API response (SEC-08) | mitigate | CloudConnectionOut Pydantic model excludes credentials_enc; safe-by-default (whitelist not blacklist) | +| T-04-07-04 | Tampering | Admin deletes their own account | mitigate | DELETE /api/admin/users/{id} asserts target user role != admin before proceeding | +| T-04-07-05 | Information Disclosure | Orphaned MinIO objects after user deletion (SEC-09) | mitigate | delete_user_files logic collects all Document.object_key values and calls delete_object before DB delete | +| T-04-07-06 | Repudiation | Auth events not logged | mitigate | All 8 auth event types logged per D-13; audit log is append-only (no DELETE on audit_log table) | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. SEC-08 test: `cd backend && python -m pytest tests/test_security.py::test_credentials_enc_not_in_response -v` +2. SEC-09 test: `cd backend && python -m pytest tests/test_admin_api.py::test_delete_user_cleans_files -v` +3. PII check on audit writes: `grep -A5 "login_failed\|auth.login_failed" backend/api/auth.py | grep "email\|password"` — expect no match +4. Full suite: `cd backend && python -m pytest tests/ -v --no-header 2>&1 | grep -E "FAILED|ERROR"` + + + +- Audit log has entries for all D-13 event categories (auth, document, folder/share, admin) +- credentials_enc never appears in any serialized response — CloudConnectionOut model enforces this +- delete-user endpoint cleans MinIO objects before removing DB records +- Full pytest suite green — all existing tests pass + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-07-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-08-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-08-PLAN.md new file mode 100644 index 0000000..abec7c8 --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-08-PLAN.md @@ -0,0 +1,243 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 08 +type: execute +wave: 6 +depends_on: + - "04-07" +files_modified: + - frontend/src/api/client.js + - frontend/src/stores/folders.js + - frontend/src/stores/documents.js + - frontend/src/router/index.js +autonomous: true +requirements: + - FOLD-01 + - FOLD-02 + - FOLD-03 + - FOLD-04 + - FOLD-05 + - SHARE-01 + - SHARE-02 + - SHARE-03 + - SHARE-04 + - SHARE-05 + - DOC-01 + - DOC-02 + +must_haves: + truths: + - "All backend API endpoints have matching client.js wrapper functions" + - "useFoldersStore exposes state and actions for folder CRUD and navigation" + - "useDocumentsStore extended with search, sort, folderId, shareDocument, revokeShare, listShares" + - "Vue Router has routes for /folders/:folderId and /shared" + - "Access token injected via existing request() helper — no new auth logic added" + artifacts: + - path: "frontend/src/api/client.js" + provides: "listFolders, createFolder, renameFolder, deleteFolder, moveDocument, createShare, deleteShare, listShares, getSharedWithMe, getMyPreferences, updateMyPreferences, getDocumentContentUrl" + - path: "frontend/src/stores/folders.js" + provides: "useFoldersStore: folders, currentFolderId, breadcrumb, loading, error + folder CRUD actions" + - path: "frontend/src/stores/documents.js" + provides: "extended with folderId, searchQuery, sort, order state + share actions" + - path: "frontend/src/router/index.js" + provides: "/folders/:folderId route + /shared route" + key_links: + - from: "frontend/src/stores/folders.js" + to: "frontend/src/api/client.js" + via: "all folder CRUD actions call api.* functions" + pattern: "import.*api" + - from: "frontend/src/router/index.js" + to: "frontend/src/views/FolderView.vue" + via: "route /folders/:folderId component lazy-loaded" + pattern: "FolderView" +--- + + +Wire the frontend data layer: API client functions, Pinia stores, and router routes for all +Phase 4 backend endpoints. This plan creates no UI components — those are in plan 04-09. +Stores and API client are the contracts the UI components depend on. + +Purpose: Establish frontend data contracts before writing view and component code. +Output: client.js additions + folders store + extended documents store + new routes. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-UI-SPEC.md +@frontend/src/api/client.js +@frontend/src/stores/documents.js +@frontend/src/stores/topics.js +@frontend/src/router/index.js + + + +From frontend/src/api/client.js: read actual file to confirm request() signature. +Pattern: all exported functions call the internal request() helper which injects +the access token from the auth store and handles 401 refresh. + +From frontend/src/stores/topics.js: full defineStore pattern to replicate. +State refs: typed refs, loading, error. Actions: async try/catch/finally pattern. + +From frontend/src/router/index.js: read to find the route array and lazy-import +pattern for view components (e.g., () => import('../views/SomeView.vue')). + + + + + + Task 1: Add Phase 4 API functions to client.js and create folders store + frontend/src/api/client.js, frontend/src/stores/folders.js + + frontend/src/api/client.js — read the ENTIRE file; identify the request() helper signature; find the last exported function; note if any listFolders or share functions already exist + frontend/src/stores/topics.js — read ENTIRE file for the exact defineStore + ref + async action pattern + frontend/src/stores/documents.js — read lines 1-30 to confirm existing imports + + + Modify frontend/src/api/client.js: APPEND these named export functions after existing functions. Follow the existing named-export style exactly. + + Folder API: listFolders(parentId = null) → GET /api/folders with optional ?parent_id= query param; createFolder(name, parentId = null) → POST /api/folders with JSON body {name, parent_id: parentId || null}; getFolder(folderId) → GET /api/folders/{folderId}; renameFolder(folderId, name) → PATCH /api/folders/{folderId} with JSON {name}; deleteFolder(folderId) → DELETE /api/folders/{folderId}; moveDocument(docId, folderId) → PATCH /api/documents/{docId}/folder with JSON {folder_id: folderId || null}. + + Share API: createShare(docId, recipientHandle) → POST /api/shares with JSON {document_id: docId, recipient_handle: recipientHandle}; listShares(docId) → GET /api/shares?document_id={docId}; deleteShare(shareId) → DELETE /api/shares/{shareId}; getSharedWithMe() → GET /api/shares/received. + + Preference API: getMyPreferences() → GET /api/auth/me/preferences; updateMyPreferences(payload) → PATCH /api/auth/me/preferences with JSON payload. + + PDF proxy URL helper: getDocumentContentUrl(docId) → returns the string `/api/documents/${docId}/content` — pure string, no fetch. Used for iframe src or window.open. + + For listDocuments (the existing function): read its current signature. If it does not already accept folderId, q, sort, order params (added in plan 04-03), update it to pass these as query params via URLSearchParams, including params only when non-null and non-empty. + + Create frontend/src/stores/folders.js as a NEW file. Follow the exact useTopicsStore pattern. + Imports: `import { defineStore } from 'pinia'`, `import { ref } from 'vue'`, `import * as api from '../api/client.js'`. + State refs: folders (ref([])), currentFolderId (ref(null)), breadcrumb (ref([])), loading (ref(false)), error (ref(null)). + Actions: + - fetchFolders(parentId = null): calls api.listFolders(parentId); sets folders.value + - createFolder(name, parentId = null): calls api.createFolder(name, parentId); pushes result to folders.value + - renameFolder(folderId, name): calls api.renameFolder; updates matching entry in folders.value + - deleteFolder(folderId): calls api.deleteFolder; removes from folders.value + - navigateTo(folderId): sets currentFolderId.value = folderId; if folderId is not null, calls api.getFolder(folderId) and sets breadcrumb.value = response.breadcrumb; if folderId is null, sets breadcrumb.value = [] + All actions follow the loading/error/finally pattern from topics store. + Export: `export const useFoldersStore = defineStore('folders', () => { ... return { folders, currentFolderId, breadcrumb, loading, error, fetchFolders, createFolder, renameFolder, deleteFolder, navigateTo } })`. + + + Run: grep -n "listFolders\|createShare\|getMyPreferences\|getDocumentContentUrl\|getSharedWithMe" /Users/nik/Documents/Progamming/document_scanner/frontend/src/api/client.js + + Expected: all five function names appear in the grep output. + + Also: grep -n "useFoldersStore\|navigateTo\|currentFolderId" /Users/nik/Documents/Progamming/document_scanner/frontend/src/stores/folders.js + + Expected: all three identifiers appear. + + + - frontend/src/api/client.js contains: listFolders, createFolder, getFolder, renameFolder, deleteFolder, moveDocument, createShare, listShares, deleteShare, getSharedWithMe, getMyPreferences, updateMyPreferences, getDocumentContentUrl (13 new functions — grep confirms each name) + - frontend/src/stores/folders.js exists and exports useFoldersStore + - useFoldersStore returns: folders, currentFolderId, breadcrumb, loading, error, fetchFolders, createFolder, renameFolder, deleteFolder, navigateTo + - listDocuments in client.js accepts folderId, q, sort, order params + - No existing API functions are modified (only appended to) + + client.js has all Phase 4 API wrappers; folders store is created and exported. + + + + Task 2: Extend documents store with folder/search/share actions + add Vue Router routes + frontend/src/stores/documents.js, frontend/src/router/index.js + + frontend/src/stores/documents.js — read the ENTIRE file; identify fetchDocuments signature; find existing actions (upload, remove, etc.); find the return statement at the end to know what to add to it + frontend/src/router/index.js — read the ENTIRE file; identify the routes array; find the lazy import pattern; note auth guard usage (requiresAuth meta, beforeEach hook) to replicate for new routes + + + Modify frontend/src/stores/documents.js: + + Add new refs (insert after existing state refs, before existing actions): + - currentFolderId: ref(null) + - searchQuery: ref('') + - sortField: ref('date') — 'name' | 'date' | 'size' + - sortOrder: ref('desc') — 'asc' | 'desc' + + Add debounced search watcher (RESEARCH.md Pattern 10 — no lodash): + After existing watch declarations (or before return), add: + let _searchTimer = null + watch(searchQuery, (newVal) => { + clearTimeout(_searchTimer) + if (newVal.length < 2) { fetchDocuments({ folderId: currentFolderId.value }); return } + _searchTimer = setTimeout(() => { fetchDocuments({ q: newVal, folderId: currentFolderId.value, sort: sortField.value, order: sortOrder.value }) }, 300) + }) + + Update fetchDocuments() to accept and pass {folderId, q, sort, order} params to api.listDocuments(). + + Add new actions (append before return statement): + - shareDocument(docId, recipientHandle): try { return await api.createShare(docId, recipientHandle) } catch (e) { throw e } + - revokeShare(shareId): try { await api.deleteShare(shareId) } catch (e) { throw e } + - listShares(docId): try { return await api.listShares(docId) } catch (e) { throw e } + + Add new state/actions to the return statement: currentFolderId, searchQuery, sortField, sortOrder, shareDocument, revokeShare, listShares. + + Modify frontend/src/router/index.js: + Add two new route objects to the routes array. Follow the exact lazy-import pattern from existing routes. Apply the same requiresAuth guard as existing protected routes. + + Route 1: { path: '/folders/:folderId', name: 'folder', component: () => import('../views/FolderView.vue'), meta: { requiresAuth: true } } + Route 2: { path: '/shared', name: 'shared', component: () => import('../views/SharedView.vue'), meta: { requiresAuth: true } } + + Note: FolderView.vue and SharedView.vue do not exist yet (created in plan 04-09). The router import is lazy-loaded so missing files do not cause import errors at router init — only at navigation time. + + + Run: grep -n "currentFolderId\|searchQuery\|sortField\|shareDocument\|revokeShare" /Users/nik/Documents/Progamming/document_scanner/frontend/src/stores/documents.js + + Expected: all five identifiers appear. + + Also: grep -n "FolderView\|SharedView\|/folders\|/shared" /Users/nik/Documents/Progamming/document_scanner/frontend/src/router/index.js + + Expected: both view names and both paths appear. + + + - documents.js contains: currentFolderId ref, searchQuery ref, sortField ref, sortOrder ref + - documents.js contains debounced watch on searchQuery with 300ms timeout and 2-char minimum + - documents.js return statement includes: currentFolderId, searchQuery, sortField, sortOrder, shareDocument, revokeShare, listShares + - router/index.js contains /folders/:folderId route and /shared route + - Both new routes use lazy component import and requiresAuth meta (grep confirms) + - No existing routes modified (only new routes appended) + + Documents store extended with folder/search/share state; router has two new protected routes. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Frontend store → backend API | All requests go through request() helper which injects JWT; no auth logic duplicated in new stores | +| Vue Router → view components | New routes use same requiresAuth guard as existing protected routes | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-08-01 | Broken Access Control | /folders/:folderId route without auth guard | mitigate | requiresAuth: true meta applied to both new routes; same router beforeEach guard as existing protected routes | +| T-04-08-02 | Information Disclosure | Access token stored in localStorage via new store | accept | All token handling is in existing auth store + request() helper; folders store and documents store extension do not touch auth state | +| T-04-08-03 | Tampering | Debounced search fires before 2-char minimum | mitigate | searchQuery watch checks newVal.length < 2 before firing API call; short inputs restore full list without API call | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. API functions: grep -c "export function" /Users/nik/Documents/Progamming/document_scanner/frontend/src/api/client.js +2. Folders store: grep -n "useFoldersStore\|navigateTo\|breadcrumb" /Users/nik/Documents/Progamming/document_scanner/frontend/src/stores/folders.js +3. Documents store: grep -n "currentFolderId\|shareDocument\|searchQuery" /Users/nik/Documents/Progamming/document_scanner/frontend/src/stores/documents.js +4. Router: grep -n "FolderView\|SharedView" /Users/nik/Documents/Progamming/document_scanner/frontend/src/router/index.js + + + +- All 13 new API functions exist in client.js (verified by grep) +- useFoldersStore is created and exports folder CRUD actions + navigation state +- documents store return includes all new state/action exports +- Vue Router has /folders/:folderId and /shared routes with requiresAuth guard + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-08-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-09-PLAN.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-09-PLAN.md new file mode 100644 index 0000000..0f8936f --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-09-PLAN.md @@ -0,0 +1,349 @@ +--- +phase: 04-folders-sharing-quotas-document-ux +plan: 09 +type: execute +wave: 7 +depends_on: + - "04-08" +files_modified: + - frontend/src/views/FolderView.vue + - frontend/src/views/SharedView.vue + - frontend/src/views/SettingsView.vue + - frontend/src/views/DocumentView.vue + - frontend/src/views/HomeView.vue + - frontend/src/views/AdminView.vue + - frontend/src/components/folders/FolderRow.vue + - frontend/src/components/folders/FolderBreadcrumb.vue + - frontend/src/components/folders/FolderDeleteModal.vue + - frontend/src/components/sharing/ShareModal.vue + - frontend/src/components/documents/DocumentPreviewModal.vue + - frontend/src/components/documents/SearchBar.vue + - frontend/src/components/documents/SortControls.vue + - frontend/src/components/admin/AuditLogTab.vue + - frontend/src/components/layout/AppSidebar.vue + - frontend/src/components/documents/DocumentCard.vue + - frontend/src/api/client.js +autonomous: false +requirements: + - FOLD-01 + - FOLD-02 + - FOLD-03 + - FOLD-04 + - FOLD-05 + - SHARE-01 + - SHARE-02 + - SHARE-03 + - SHARE-04 + - SHARE-05 + - ADMIN-06 + - DOC-01 + - DOC-02 + +must_haves: + truths: + - "Folders section visible in AppSidebar above topics; top-level folders clickable to navigate" + - "Shared with me entry in AppSidebar above folders; shows count badge when non-empty" + - "Document list shows SearchBar and SortControls above; sort persists until changed" + - "DocumentCard shows share button on hover; shared indicator pill when doc.share_count > 0" + - "Share modal: handle input, Share button, current recipient list with Revoke button" + - "FolderView renders sub-folders as FolderRow components + breadcrumb + document list" + - "FolderDeleteModal shows document count and requires explicit confirmation before delete" + - "PDF documents open in DocumentPreviewModal (in_app mode) or new tab (new_tab mode)" + - "SettingsView Document Preferences card shows pdf_open_mode radio; auto-saves on change" + - "Admin AuditLog tab is accessible from AdminView; filters and CSV export work" + artifacts: + - path: "frontend/src/views/FolderView.vue" + provides: "Folder contents: sub-folder rows + breadcrumb + document list + new subfolder inline input" + - path: "frontend/src/views/SharedView.vue" + provides: "Shared with me virtual folder view" + - path: "frontend/src/components/folders/FolderRow.vue" + provides: "Folder row with three-dot menu (rename inline, delete folder) per UI-SPEC" + - path: "frontend/src/components/folders/FolderBreadcrumb.vue" + provides: "Breadcrumb nav with truncation at depth > 4; each segment clickable" + - path: "frontend/src/components/folders/FolderDeleteModal.vue" + provides: "Destructive confirmation modal with document count" + - path: "frontend/src/components/sharing/ShareModal.vue" + provides: "Share by handle + current recipients list + revoke" + - path: "frontend/src/components/documents/DocumentPreviewModal.vue" + provides: "In-app PDF preview via iframe" + - path: "frontend/src/components/documents/SearchBar.vue" + provides: "Debounced search input; clears on Escape" + - path: "frontend/src/components/documents/SortControls.vue" + provides: "Name / Date / Size sort toggle buttons" + - path: "frontend/src/components/admin/AuditLogTab.vue" + provides: "Paginated audit log table with date/user/action filters + CSV export button" + key_links: + - from: "frontend/src/views/FolderView.vue" + to: "frontend/src/stores/folders.js" + via: "useFoldersStore for folder data and navigation" + pattern: "useFoldersStore" + - from: "frontend/src/components/sharing/ShareModal.vue" + to: "frontend/src/stores/documents.js" + via: "shareDocument, revokeShare, listShares actions" + pattern: "useDocumentsStore" + - from: "frontend/src/components/layout/AppSidebar.vue" + to: "frontend/src/stores/folders.js" + via: "top-level folder list + currentFolderId for active state" + pattern: "useFoldersStore" +--- + + +Build all Phase 4 Vue components and wire them into the existing views. This is the final +frontend plan — all stores, API client, and routes are established by plan 04-08. + +Purpose: Deliver the complete document management UX visible to users and admins. +Output: 10 new components + 6 modified views following 04-UI-SPEC.md exactly. + + + +@$HOME/.claude/get-shit-done/workflows/execute-plan.md +@$HOME/.claude/get-shit-done/templates/summary.md + + + +@.planning/phases/04-folders-sharing-quotas-document-ux/04-UI-SPEC.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-CONTEXT.md +@.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md +@frontend/src/components/layout/AppSidebar.vue +@frontend/src/components/documents/DocumentCard.vue +@frontend/src/views/HomeView.vue +@frontend/src/views/AdminView.vue +@frontend/src/views/SettingsView.vue +@frontend/src/views/DocumentView.vue +@frontend/src/components/admin/AdminUsersTab.vue + + + +From 04-UI-SPEC.md — exact Tailwind classes for each new component are specified. +Executor must read 04-UI-SPEC.md fully before writing any component template. + +Key design tokens from 04-UI-SPEC.md: +- Modal overlay: `fixed inset-0 bg-black/40 flex items-center justify-center z-50` +- Modal panel: `bg-white rounded-2xl shadow-xl p-6 max-w-md w-full mx-4` +- Primary button: `bg-indigo-600 hover:bg-indigo-700 text-white text-sm px-4 py-2 rounded-lg` +- Destructive button: `bg-red-600 hover:bg-red-700 text-white text-sm font-semibold px-4 py-2 rounded-lg min-h-[44px]` +- FolderRow: `flex items-center gap-3 px-4 py-3 bg-white border border-gray-200 rounded-xl hover:border-indigo-300 hover:shadow-sm transition-all cursor-pointer` +- Share badge (SHARE-05): `bg-indigo-50 text-indigo-600 text-xs font-medium px-2 py-1 rounded-full` +- Shared with me icon: `bg-purple-50 text-purple-500` (only purple UI element) +- Section label: `text-xs font-semibold text-gray-400 uppercase tracking-wider mb-1` +- Accessibility: all icon-only buttons have aria-label; modals have role="dialog" aria-modal="true" + +Copywriting (from 04-UI-SPEC.md): +- Share modal title: "Share document" +- Handle input placeholder: "Enter username handle" +- Share submit: "Share document" +- Empty share state: "Not shared with anyone yet." +- User not found: "User not found. Check the handle and try again." +- Non-empty folder delete body: "This folder contains {N} document{s}. Deleting it will permanently delete all documents inside. This cannot be undone." +- Folder delete confirm: "Delete folder and documents" +- Folder delete dismiss: "Keep folder" +- Search placeholder: "Search documents..." +- PDF pref section heading: "Document Preferences" +- PDF open modes: "Open documents in-app" / "Open documents in new tab" +- Audit log tab label: "Audit Log" +- Sort labels: "Name" / "Date" / "Size" + + + + + + Task 1: Create new components (FolderRow, FolderBreadcrumb, FolderDeleteModal, ShareModal, DocumentPreviewModal, SearchBar, SortControls, AuditLogTab) + + frontend/src/components/folders/FolderRow.vue, + frontend/src/components/folders/FolderBreadcrumb.vue, + frontend/src/components/folders/FolderDeleteModal.vue, + frontend/src/components/sharing/ShareModal.vue, + frontend/src/components/documents/DocumentPreviewModal.vue, + frontend/src/components/documents/SearchBar.vue, + frontend/src/components/documents/SortControls.vue, + frontend/src/components/admin/AuditLogTab.vue + + + frontend/src/components/admin/AdminUsersTab.vue — read the ENTIRE file; extract the form pattern (reactive form state, loading/error refs, onMounted fetch, table with filters, action buttons, inline confirm pattern); use this as the template for ShareModal and AuditLogTab + frontend/src/components/ui/ConfirmBlock.vue — read fully for the confirm/cancel button pattern + frontend/src/components/documents/DocumentCard.vue — read for the card layout pattern and SVG icon style + frontend/src/views/AdminView.vue — read to understand how tabs are currently implemented; extract the tab switch pattern + + + Create each component file in the exact directory specified. All use Vue 3 Options API with Composition API (script setup) — follow the same pattern as existing components (read the existing files to confirm which style is used and replicate it exactly). + + FolderRow.vue (per UI-SPEC FolderRow section): + - Props: folder {id, name, doc_count}, onNavigate (function), onRename (function), onDelete (function) + - Template: flex container with folder icon (bg-gray-100), folder name + doc count, three-dot menu button + - Three-dot menu: dropdown with "Rename" and "Delete folder" items; closes on outside click (use @click.outside or a boolean ref + document listener) + - Rename mode: toggles to inline input pre-filled with current name; Enter=save, Escape=cancel, empty=rejected with inline error + - No modal for rename — inline text input replaces the name display + + FolderBreadcrumb.vue (per UI-SPEC FolderBreadcrumb section): + - Props: segments Array of {id, name} + - Emits: navigate(folderId) — folderId is null for root + - Computed: visibleSegments — if segments.length > 4, return [segments[0], {id: null, name: '...'}, ...segments.slice(-2)]; otherwise return segments as-is + - Template: nav with aria-label="Folder navigation", ol list structure (accessibility), separator chevron SVG between segments + - All segments except the last are clickable (emit navigate); last segment is non-clickable (current folder) + - Wrap with nav aria-label="Folder navigation" and ol per UI-SPEC accessibility contract + + FolderDeleteModal.vue (per UI-SPEC FolderDeleteModal section): + - Props: folder {id, name, doc_count}, onConfirm (function), onCancel (function) + - Template: fixed overlay, warning icon (red exclamation), heading "Delete folder?", body text with doc_count interpolated, two buttons: "Keep folder" (neutral) and "Delete folder and documents" (red, min-h-[44px]) + - Emits: confirm, cancel + - Accessibility: role="dialog" aria-modal="true" aria-labelledby="modal-title-id" + + ShareModal.vue (per UI-SPEC ShareModal section): + - Props: doc {id, filename} + - Emits: close + - Script state: handle ref(''), submitting ref(false), error ref(null), shares ref([]) + - onMounted: load current shares via useDocumentsStore().listShares(props.doc.id); set shares.value + - submitShare(): calls store.shareDocument(props.doc.id, handle.value); clears handle on success; updates shares list; shows error for 404/409 + - revokeShare(shareId): calls store.revokeShare(shareId); removes from shares array optimistically; on error re-adds + - Template: overlay, panel, title "Share document", handle input + Share document button, separator, recipient list OR empty state, close X button + - Error messages per UI-SPEC copywriting (User not found, already shared) + - Accessibility: role="dialog" aria-modal="true"; close button aria-label="Close" + + DocumentPreviewModal.vue (per UI-SPEC DocumentPreviewModal section): + - Props: doc {id, filename} + - Emits: close + - Script: proxyUrl computed = `/api/documents/${props.doc.id}/content` + - Template: fixed overlay (bg-black/60 z-50 flex flex-col), header bar with doc name + close button, flex-1 iframe filling the remaining height + - Close on overlay click (check event.target === overlay element) and Escape key (keydown listener on mounted/unmounted) + - Accessibility: role="dialog" aria-modal="true" aria-label="Document preview" + + SearchBar.vue (per UI-SPEC SearchBar section): + - Props: modelValue (String), placeholder defaults to "Search documents..." + - Emits: update:modelValue + - Template: input type="search", role="search" on wrapper div, aria-label="Search documents" + - Clears on Escape key (@keydown.escape) + - Width: w-56 + + SortControls.vue (per UI-SPEC SortControls section): + - Props: sort ('name'|'date'|'size'), order ('asc'|'desc') + - Emits: change({sort, order}) + - Template: three text buttons; active sort has `text-indigo-600 font-semibold` class; direction indicator appended (arrow up/down unicode or ↑/↓ text) + - Clicking the already-active sort toggles order; clicking a different sort switches to it with desc order + - aria-pressed="true" on active button; aria-label includes direction (e.g., "Sort by name, ascending") + + AuditLogTab.vue (per UI-SPEC AuditLogTab section): + - Script state: entries ref([]), total ref(0), page ref(1), loading ref(false), filters reactive({start:'', end:'', user_id:'', event_type:''}) + - onMounted: fetchLog() + - fetchLog(): calls api.adminListAuditLog({...filters, page: page.value}) from api/client.js (add this function — GET /api/admin/audit-log with query params) + - exportCsv(): sets window.location.href to `/api/admin/audit-log/export?format=csv&${new URLSearchParams(activeFilters)}` — triggers browser download + - Template: filter bar (date inputs, user dropdown placeholder, event_type dropdown, Apply button, Export CSV button), paginated table (Timestamp | User | Action Type | IP Address), Previous/Next pagination, empty state + - Action type pill badge colors per UI-SPEC: auth=bg-blue-50 text-blue-600, document=bg-gray-100 text-gray-600, folder/share=bg-purple-50 text-purple-600, admin=bg-amber-50 text-amber-700 + - Table headers with scope="col"; timestamp cell uses font-mono text-xs + + Also add adminListAuditLog function to frontend/src/api/client.js (append): GET /api/admin/audit-log with query params start, end, user_id, event_type, page, per_page. + + + Run: find /Users/nik/Documents/Progamming/document_scanner/frontend/src/components/folders /Users/nik/Documents/Progamming/document_scanner/frontend/src/components/sharing -name "*.vue" 2>/dev/null + + Expected: FolderRow.vue, FolderBreadcrumb.vue, FolderDeleteModal.vue appear; ShareModal.vue appears. + + Also: find /Users/nik/Documents/Progamming/document_scanner/frontend/src/components/documents /Users/nik/Documents/Progamming/document_scanner/frontend/src/components/admin -name "*.vue" | sort + + Expected: DocumentPreviewModal.vue, SearchBar.vue, SortControls.vue, AuditLogTab.vue appear. + + + - All 8 component files exist in their respective directories (verify with find) + - FolderBreadcrumb.vue has nav aria-label="Folder navigation" and ol list structure (grep: `aria-label.*Folder navigation` and `ol` in template) + - ShareModal.vue has role="dialog" and aria-modal="true" (grep: `role="dialog"` and `aria-modal="true"`) + - FolderDeleteModal.vue contains "Keep folder" and "Delete folder and documents" button text (grep) + - DocumentPreviewModal.vue contains iframe with :src bound to proxyUrl (grep: `iframe` and `proxyUrl`) + - SearchBar.vue has @keydown.escape handler (grep: `keydown.escape` or `keydown.esc`) + - SortControls.vue has aria-pressed (grep: `aria-pressed`) + - AuditLogTab.vue has window.location.href for CSV export (grep: `window.location.href`) + - frontend/src/api/client.js contains adminListAuditLog function (grep) + + All new components created following UI-SPEC; accessibility contracts met; CSV export uses window.location.href. + + + + + All frontend views and components for Phase 4: + - AppSidebar.vue extended with "Shared with me" entry (purple icon, count badge) and Folders section with "New folder" CTA; folder links navigate to /folders/:id + - DocumentCard.vue extended with share button (hover, opacity-0 group-hover:opacity-100) and shared indicator pill + - HomeView.vue wired with SearchBar + SortControls above document list; folder-aware fetchDocuments + - FolderView.vue: breadcrumb + sub-folder FolderRow list + document list + inline new-folder input + - SharedView.vue: filtered document list from GET /api/shares/received; empty state + - DocumentView.vue: PDF click triggers DocumentPreviewModal (in_app) or window.open (new_tab) based on pdf_open_mode preference + - SettingsView.vue: Document Preferences card with radio buttons auto-saved via PATCH /api/auth/me/preferences + - AdminView.vue: AuditLog tab added alongside existing tabs + + This checkpoint also triggers human verification of all visual and interactive Phase 4 features before the phase is marked complete. + + + Before this checkpoint is reached, the executor MUST complete the following modifications: + + AppSidebar.vue: read the entire file; add "Shared with me" entry (above folders section, inline inbox icon, purple bg-purple-50 text-purple-500 icon container, count badge when sharedCount > 0); add Folders section below (section label "FOLDERS", "New folder" button, top-level folder list from useFoldersStore); import useFoldersStore; add sharedCount computed using getSharedWithMe() or a dedicated ref. + + DocumentCard.vue: read the entire file; add `group` class to outer container; add share button (opacity-0 group-hover:opacity-100 transition-opacity, aria-label="Share document", min-h-[44px] min-w-[44px]); add ShareModal component import and v-if=showShareModal; add shared indicator pill below metadata line if doc.share_count > 0. + + HomeView.vue: read the entire file; add SearchBar component above document list with v-model=docsStore.searchQuery; add SortControls with sort=docsStore.sortField, order=docsStore.sortOrder, @change handler; add useFoldersStore; update onMounted to also call foldersStore.fetchFolders(null). + + FolderView.vue: create new file; uses useFoldersStore and useDocumentsStore; on route param folderId change, calls foldersStore.navigateTo(folderId) and docsStore.fetchDocuments({folderId}); renders FolderBreadcrumb + FolderRow list + document list + inline new-folder input. + + SharedView.vue: create new file; on mounted calls api.getSharedWithMe(); renders document list using same DocumentCard layout but with owner handle shown; empty state per UI-SPEC copywriting. + + DocumentView.vue: read the entire file; find where PDF documents would be clicked or previewed; add logic — if current_user.pdf_open_mode === 'in_app', show DocumentPreviewModal; if 'new_tab', call window.open(getDocumentContentUrl(doc.id), '_blank'); load preference via api.getMyPreferences() on mount. + + SettingsView.vue: read the entire file; add Document Preferences section after existing sections; radio group with v-model=pdfOpenMode; watch pdfOpenMode to auto-call api.updateMyPreferences. + + AdminView.vue: read the entire file; add "Audit Log" tab button alongside existing tabs; add AuditLogTab component conditional display. + + AFTER completing all view modifications, start docker compose up and navigate to the application: + + 1. Navigate to the home page — verify Folders section and "Shared with me" appear in sidebar + 2. Create a folder via sidebar "New folder" — verify it appears in the folder list + 3. Navigate into the folder — verify FolderView loads with breadcrumb showing the folder name + 4. Upload a document (use existing upload flow) — verify it appears in the current folder + 5. Click the share button on a DocumentCard — verify ShareModal opens with handle input and empty state + 6. In ShareModal, type a non-existent handle — verify "User not found" error appears + 7. Delete a non-empty folder — verify FolderDeleteModal shows the document count and correct copy + 8. Navigate to Settings — verify Document Preferences card with radio buttons appears + 9. Navigate to Admin — verify Audit Log tab appears; click Apply filters — verify table loads + 10. Click Export CSV in audit log — verify file downloads + 11. Open a PDF document (set preference to in_app first) — verify DocumentPreviewModal with iframe appears + + Type "approved" after verifying all 11 checkpoints pass. Describe any issues for the executor to fix before approving. + + + + + +## Trust Boundaries + +| Boundary | Description | +|----------|-------------| +| Browser → /api/documents/{id}/content | All document content access goes through proxy; iframe src is the proxy URL, not a presigned URL | +| Share modal → shares API | Recipient handle is user-supplied text; backend validates existence; no autocomplete/search that could enumerate users | +| Admin audit log export | window.location.href triggers a download authenticated by the existing httpOnly cookie; no token in URL | + +## STRIDE Threat Register + +| Threat ID | Category | Component | Disposition | Mitigation Plan | +|-----------|----------|-----------|-------------|-----------------| +| T-04-09-01 | Information Disclosure | iframe src reveals presigned URL in browser | mitigate | DocumentPreviewModal iframe src = `/api/documents/${docId}/content` (the proxy URL, never presigned); Content-Disposition: inline drives rendering | +| T-04-09-02 | Information Disclosure | Share modal autocomplete reveals user handles | mitigate | D-04: exact handle input only; no autocomplete API; 404 response reveals only that the handle does not exist | +| T-04-09-03 | Broken Access Control | Audit log tab visible to regular users in AdminView | mitigate | AuditLogTab only visible inside AdminView which is already guarded by admin-only route; backend enforces get_current_admin | +| T-04-09-04 | Information Disclosure | window.location.href for CSV export embeds sensitive params in URL | accept | Params are only filter values (dates, event types, user IDs) — not auth tokens; the request is authenticated via httpOnly cookie already set | +| T-04-SC | Tampering | npm/pip/cargo installs | accept | No new packages installed in this plan | + + + +1. Component existence: find frontend/src/components/folders frontend/src/components/sharing -name "*.vue" +2. Security: grep -rn "presigned" frontend/src/components/documents/DocumentPreviewModal.vue — expect zero matches +3. Accessibility: grep -n "role=\"dialog\"\|aria-modal" frontend/src/components/sharing/ShareModal.vue frontend/src/components/folders/FolderDeleteModal.vue +4. Share IDOR prevention: grep -n "aria-label" frontend/src/components/documents/DocumentCard.vue — share button must have aria-label +5. Human checkpoint: all 11 scenarios verified by the developer in a running instance + + + +- All 10 new component files exist +- AppSidebar has "Shared with me" (purple icon) and Folders section +- DocumentCard has share button (hover-reveal) and shared indicator pill +- DocumentPreviewModal uses proxy URL (/api/documents/{id}/content), not presigned URL +- AuditLogTab is accessible only inside AdminView (protected route) +- Developer approves all 11 manual verification checkpoints + + + +Create `.planning/phases/04-folders-sharing-quotas-document-ux/04-09-SUMMARY.md` when done. + diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md new file mode 100644 index 0000000..56b63e9 --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-PATTERNS.md @@ -0,0 +1,1250 @@ +# Phase 4: Folders, Sharing, Quotas & Document UX - Pattern Map + +**Mapped:** 2026-05-25 +**Files analyzed:** 20 (10 new, 10 modified) +**Analogs found:** 19 / 20 + +--- + +## File Classification + +| New/Modified File | Role | Data Flow | Closest Analog | Match Quality | +|---|---|---|---|---| +| `backend/api/folders.py` | router/controller | CRUD + request-response | `backend/api/documents.py` | exact | +| `backend/api/shares.py` | router/controller | CRUD + request-response | `backend/api/topics.py` + `backend/api/documents.py` | role-match | +| `backend/api/audit.py` | router/controller | CRUD + streaming (export) | `backend/api/admin.py` | exact | +| `backend/services/audit.py` | service/utility | request-response | `backend/services/storage.py` | role-match | +| `backend/tasks/audit_tasks.py` | task/worker | batch + file-I/O | `backend/tasks/document_tasks.py` | exact | +| `backend/migrations/versions/0004_phase4_*.py` | migration/config | transform | `backend/migrations/versions/0003_multi_user_isolation.py` | exact | +| `backend/storage/minio_backend.py` (modify) | service | file-I/O | self (existing file) | self | +| `backend/celery_app.py` (modify) | config | event-driven | self (existing file) | self | +| `backend/api/documents.py` (modify) | router/controller | streaming + CRUD | self (existing file) | self | +| `backend/api/auth.py` (modify) | router/controller | request-response | self (existing file) | self | +| `backend/api/admin.py` (modify) | router/controller | CRUD + streaming | self (existing file) | self | +| `frontend/src/stores/folders.js` | store | CRUD + request-response | `frontend/src/stores/topics.js` | exact | +| `frontend/src/components/documents/ShareModal.vue` | component | request-response | `frontend/src/components/admin/AdminUsersTab.vue` | partial | +| `frontend/src/components/layout/BreadcrumbNav.vue` | component | request-response | `frontend/src/components/topics/TopicBadge.vue` | partial | +| `frontend/src/components/admin/AdminAuditLogTab.vue` | component | CRUD + request-response | `frontend/src/components/admin/AdminUsersTab.vue` | exact | +| `frontend/src/components/layout/AppSidebar.vue` (modify) | component | request-response | self (existing file) | self | +| `frontend/src/components/documents/DocumentCard.vue` (modify) | component | request-response | self (existing file) | self | +| `frontend/src/stores/documents.js` (modify) | store | CRUD + request-response | self (existing file) | self | +| `frontend/src/views/HomeView.vue` (modify) | view | request-response | self (existing file) | self | +| `frontend/src/views/SettingsView.vue` (modify) | view | request-response | self (existing file) | self | + +--- + +## Pattern Assignments + +### `backend/api/folders.py` (router/controller, CRUD) + +**Analog:** `backend/api/documents.py` + +**Imports pattern** (lines 17-34 of `backend/api/documents.py`): +```python +from __future__ import annotations + +import uuid +from typing import Optional + +from fastapi import APIRouter, Depends, HTTPException, Query, status +from pydantic import BaseModel +from sqlalchemy import select, text, delete +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession + +from db.models import Document, Folder, Quota, User +from deps.auth import get_regular_user +from deps.db import get_db +from storage import get_storage_backend + +router = APIRouter(prefix="/api/folders", tags=["folders"]) +``` + +**Auth/Guard pattern** — `get_regular_user` on every handler (lines 54-58 of `backend/api/documents.py`): +```python +@router.post("/") +async def create_folder( + body: FolderCreate, + session: AsyncSession = Depends(get_db), + current_user: User = Depends(get_regular_user), +): +``` + +**Ownership assertion pattern** (lines 117-119 of `backend/api/documents.py`): +```python +doc = await session.get(Document, uid) +if doc is None or doc.user_id != current_user.id: + raise HTTPException(status_code=404, detail="Document not found") +``` +Apply as: `folder = await session.get(Folder, folder_id); if folder is None or folder.user_id != current_user.id: raise HTTPException(404, "Folder not found")` + +**UUID parse pattern** (lines 113-115 of `backend/api/documents.py`): +```python +try: + uid = uuid.UUID(doc_id) +except ValueError: + raise HTTPException(status_code=404, detail="Document not found") +``` + +**IntegrityError / 409 pattern** — new for folders, from RESEARCH.md Pitfall 6: +```python +from sqlalchemy.exc import IntegrityError +try: + session.add(folder) + await session.commit() +except IntegrityError: + await session.rollback() + raise HTTPException(409, "A folder with that name already exists here") +``` + +**Atomic quota decrement pattern** (lines 137-146 of `backend/api/documents.py`): +```python +result = await session.execute( + text( + "UPDATE quotas " + "SET used_bytes = used_bytes + :delta " + "WHERE user_id = :uid " + " AND (used_bytes + :delta) <= limit_bytes " + "RETURNING used_bytes, limit_bytes" + ), + {"delta": size, "uid": str(doc.user_id)}, +) +``` +For folder cascade-delete, invert: `CASE WHEN used_bytes > :delta THEN used_bytes - :delta ELSE 0 END` + +**Request models pattern** (lines 46-49 of `backend/api/documents.py`): +```python +class UploadUrlRequest(BaseModel): + filename: str + content_type: str +``` +Apply as: +```python +class FolderCreate(BaseModel): + name: str + parent_id: Optional[str] = None + +class FolderRename(BaseModel): + name: str + +class DocumentMove(BaseModel): + folder_id: Optional[str] = None +``` + +--- + +### `backend/api/shares.py` (router/controller, CRUD) + +**Analog:** `backend/api/documents.py` + `backend/api/admin.py` + +**Imports pattern:** +```python +from __future__ import annotations + +import uuid +from typing import Optional + +from fastapi import APIRouter, Depends, HTTPException, status +from pydantic import BaseModel +from sqlalchemy import select +from sqlalchemy.exc import IntegrityError +from sqlalchemy.ext.asyncio import AsyncSession + +from db.models import Document, Share, User +from deps.auth import get_regular_user +from deps.db import get_db + +router = APIRouter(prefix="/api/shares", tags=["shares"]) +``` + +**Ownership assertion for shares** — owner_id variant (from RESEARCH.md Pitfall 4): +```python +share = await session.get(Share, share_id) +if share is None or share.owner_id != current_user.id: + raise HTTPException(404, "Share not found") +``` + +**Handle lookup pattern** — exact match, 404 if not found (analogous to admin user lookup in lines 231-233 of `backend/api/admin.py`): +```python +result = await session.execute( + select(User).where(User.handle == body.recipient_handle) +) +recipient = result.scalar_one_or_none() +if recipient is None: + raise HTTPException(404, "User not found") +``` + +**Share-recipient list query** (from RESEARCH.md Pattern 4 / D-06): +```python +# GET /api/shares/received — "Shared with me" virtual folder +stmt = ( + select(Document) + .join(Share, Share.document_id == Document.id) + .where(Share.recipient_id == current_user.id) + .order_by(Document.created_at.desc()) +) +result = await session.execute(stmt) +shared_docs = result.scalars().all() +``` + +**IntegrityError / 409 for duplicate share** (same pattern as folder name uniqueness): +```python +try: + session.add(share) + await session.commit() +except IntegrityError: + await session.rollback() + raise HTTPException(409, "Document already shared with this user") +``` + +--- + +### `backend/api/audit.py` (router/controller, CRUD + streaming export) + +**Analog:** `backend/api/admin.py` + +**Imports pattern** (lines 23-40 of `backend/api/admin.py`): +```python +from __future__ import annotations + +import csv +import io +import uuid +from datetime import datetime +from typing import Optional + +from fastapi import APIRouter, Depends, HTTPException, Query +from fastapi.responses import StreamingResponse +from pydantic import BaseModel +from sqlalchemy import select +from sqlalchemy.ext.asyncio import AsyncSession + +from db.models import AuditLog, User +from deps.auth import get_current_admin +from deps.db import get_db + +router = APIRouter(prefix="/api/admin", tags=["admin"]) +``` + +**Admin guard pattern** (line 144 of `backend/api/admin.py`): +```python +@router.get("/audit-log") +async def list_audit_log( + session: AsyncSession = Depends(get_db), + _admin: User = Depends(get_current_admin), + start: Optional[datetime] = Query(None), + end: Optional[datetime] = Query(None), + user_id: Optional[uuid.UUID] = Query(None), + event_type: Optional[str] = Query(None), + page: int = Query(1, ge=1), + per_page: int = Query(50, ge=1, le=500), +) -> dict: +``` + +**Paginated list + total pattern** (lines 151-155 of `backend/api/admin.py`): +```python +result = await session.execute( + select(User).order_by(User.created_at.desc()) +) +users = result.scalars().all() +return {"items": [_user_to_dict(u) for u in users]} +``` +Apply as: +```python +stmt = ( + select(AuditLog) + .order_by(AuditLog.created_at.desc()) + .limit(per_page) + .offset((page - 1) * per_page) +) +``` + +**Safe response helper / whitelist pattern** (lines 54-69 of `backend/api/admin.py`): +```python +def _user_to_dict(user: User) -> dict: + """Return a safe subset of User fields — never includes password_hash, + credentials_enc, totp_secret, or any document content (T-02-27, SEC-07). + """ + return { + "id": str(user.id), + "handle": user.handle, + ... + } +``` +Apply as: +```python +def _audit_to_dict(entry: AuditLog) -> dict: + """Audit log safe serializer — never includes filename, extracted_text, + or document content (ADMIN-06, D-15). + """ + return { + "id": entry.id, + "event_type": entry.event_type, + "user_id": str(entry.user_id) if entry.user_id else None, + "actor_id": str(entry.actor_id) if entry.actor_id else None, + "resource_id": str(entry.resource_id) if entry.resource_id else None, + "ip_address": str(entry.ip_address) if entry.ip_address else None, + "metadata_": entry.metadata_, + "created_at": entry.created_at.isoformat(), + } +``` + +**CSV StreamingResponse export pattern** (from RESEARCH.md Pattern 7): +```python +@router.get("/audit-log/export") +async def export_audit_log( + format: str = Query("csv"), + session: AsyncSession = Depends(get_db), + _admin: User = Depends(get_current_admin), +): + rows = await _query_audit_log(session, ...) + output = io.StringIO() + writer = csv.DictWriter(output, fieldnames=[ + "id", "event_type", "user_id", "actor_id", + "resource_id", "ip_address", "metadata_", "created_at" + ]) + writer.writeheader() + for row in rows: + writer.writerow(_audit_to_dict(row)) + output.seek(0) + return StreamingResponse( + iter([output.getvalue()]), + media_type="text/csv", + headers={"Content-Disposition": "attachment; filename=audit-export.csv"}, + ) +``` + +--- + +### `backend/services/audit.py` (service/utility, request-response) + +**Analog:** `backend/services/storage.py` (pattern for standalone async service functions) + +**Imports pattern** (from RESEARCH.md Pattern 6): +```python +from __future__ import annotations + +import logging +import uuid +from typing import Optional + +from sqlalchemy.ext.asyncio import AsyncSession + +from db.models import AuditLog + +logger = logging.getLogger(__name__) +``` + +**Core write_audit_log function** (RESEARCH.md Pattern 6 — flush-not-commit, never-raises): +```python +async def write_audit_log( + session: AsyncSession, + event_type: str, + user_id: Optional[uuid.UUID], + actor_id: Optional[uuid.UUID], + resource_id: Optional[uuid.UUID], + ip_address: Optional[str], + metadata_: Optional[dict] = None, +) -> None: + """Write an audit log entry. Never raises — audit failure is non-fatal.""" + try: + entry = AuditLog( + event_type=event_type, + user_id=user_id, + actor_id=actor_id, + resource_id=resource_id, + ip_address=ip_address, + metadata_=metadata_, + ) + session.add(entry) + await session.flush() # flush within handler's existing transaction, not commit + except Exception as exc: + logger.warning("audit log write failed: %s", exc) + # Do not re-raise — audit failure must never abort the primary operation +``` + +**IP extraction pattern** for handlers (from RESEARCH.md Pitfall 5): +```python +# In each handler that calls write_audit_log: +ip_address = request.headers.get("X-Forwarded-For", None) or ( + request.client.host if request.client else None +) +await write_audit_log(session, "folder.created", current_user.id, current_user.id, folder.id, ip_address) +``` + +--- + +### `backend/tasks/audit_tasks.py` (task/worker, batch + file-I/O) + +**Analog:** `backend/tasks/document_tasks.py` + +**Task module structure** (lines 1-26 of `backend/tasks/document_tasks.py`): +```python +""" +Celery tasks for audit log export in DocuVault. + +audit_log_daily_export — called by Celery beat at midnight UTC. +The task is a plain sync def (Celery workers have no asyncio event loop); it +bridges into the async body via asyncio.run(). +""" +import asyncio + +from celery_app import celery_app +``` + +**Sync entry-point → asyncio.run() bridge pattern** (lines 22-25 of `backend/tasks/document_tasks.py`): +```python +@celery_app.task(name="tasks.document_tasks.extract_and_classify") +def extract_and_classify(document_id: str) -> dict: + """Synchronous Celery entry-point — delegates to async _run via asyncio.run.""" + return asyncio.run(_run(document_id)) +``` +Apply as: +```python +@celery_app.task(name="tasks.audit_tasks.audit_log_daily_export") +def audit_log_daily_export() -> dict: + return asyncio.run(_run_daily_export()) +``` + +**AsyncSessionLocal usage pattern** (lines 41-43 of `backend/tasks/document_tasks.py`): +```python +from db.session import AsyncSessionLocal +from db.models import Document +from storage import get_storage_backend + +async with AsyncSessionLocal() as session: + doc = await session.get(Document, doc_uuid) +``` + +**Best-effort error handling in tasks** (lines 90-101 of `backend/tasks/document_tasks.py`): +```python +try: + topics = await classifier.classify_document(session, document_id, ...) + return {"document_id": document_id, "status": "classified", ...} +except Exception as e: + doc.status = "classification_failed" + await session.commit() + return {"document_id": document_id, "status": "classification_failed", "error": str(e)} +``` + +**Deferred imports pattern** (lines 38-40 of `backend/tasks/document_tasks.py`): +```python +# All application imports are inside the async function body, not at module top level. +# This avoids circular imports between Celery worker process and FastAPI application. +from db.session import AsyncSessionLocal +from db.models import Document +``` + +--- + +### `backend/migrations/versions/0004_phase4_*.py` (migration/config) + +**Analog:** `backend/migrations/versions/0003_multi_user_isolation.py` + +**Module header pattern** (lines 1-50 of `backend/migrations/versions/0003_multi_user_isolation.py`): +```python +""". + +Revision ID: 0004 +Revises: 0003 +Create Date: + +Changes (in order): + 1. Add users.pdf_open_mode column (String, default 'in_app') + 2. Create GIN expression index on documents.extracted_text (tsvector FTS) + 3. Create audit-logs MinIO bucket (gated on MINIO_ENDPOINT env var) +""" +from __future__ import annotations + +import os + +import sqlalchemy as sa +from sqlalchemy import text +from alembic import op + +revision = "0004" +down_revision = "0003" +branch_labels = None +depends_on = None +``` + +**batch_alter_table for SQLite compat** (lines 95-102 of `backend/migrations/versions/0003_multi_user_isolation.py`): +```python +with op.batch_alter_table("users") as batch_op: + batch_op.add_column( + sa.Column("pdf_open_mode", sa.String(), nullable=False, server_default="in_app") + ) +``` + +**GIN expression index — manual SQL, do NOT use Computed()** (from RESEARCH.md Pattern 5): +```python +# managed manually — do not autogenerate (Alembic issue #1390) +op.execute( + "CREATE INDEX ix_documents_fts ON documents " + "USING GIN (to_tsvector('english', coalesce(extracted_text, '')))" +) +``` + +**MinIO bucket creation gated on env var** (lines 74-88 of `backend/migrations/versions/0003_multi_user_isolation.py`): +```python +if os.environ.get("MINIO_ENDPOINT"): + from minio import Minio + bucket = "audit-logs" + client = Minio( + os.environ.get("MINIO_ENDPOINT", "minio:9000"), + access_key=os.environ.get("MINIO_ACCESS_KEY", ""), + secret_key=os.environ.get("MINIO_SECRET_KEY", ""), + secure=False, + ) + if not client.bucket_exists(bucket): + client.make_bucket(bucket) +``` + +**downgrade() reversal** (lines 117-131 of `backend/migrations/versions/0003_multi_user_isolation.py`): +```python +def downgrade() -> None: + op.execute("DROP INDEX IF EXISTS ix_documents_fts") + with op.batch_alter_table("users") as batch_op: + batch_op.drop_column("pdf_open_mode") + # Note: MinIO bucket creation is NOT reversed — bucket may contain data. +``` + +--- + +### `backend/storage/minio_backend.py` — add `put_object_raw()` (modify) + +**Analog:** Existing `MinIOBackend.put_object()` method (lines 62-86 of `backend/storage/minio_backend.py`). + +**asyncio.to_thread() pattern for sync SDK calls** (lines 78-86 of `backend/storage/minio_backend.py`): +```python +await asyncio.to_thread( + self._client.put_object, + self._bucket, + object_key, + data, + length=len(file_bytes), + content_type=content_type, +) +``` + +**New `put_object_raw()` method** — for audit-logs bucket (different bucket, caller-supplied key): +```python +async def put_object_raw( + self, + bucket: str, + key: str, + data: io.BytesIO, + length: int, + content_type: str, +) -> None: + """Upload bytes to an arbitrary bucket+key (used for audit-logs CSV export). + + Unlike put_object(), does NOT apply the document key schema — the caller + supplies the complete key. The main documents bucket is NOT used. + """ + await asyncio.to_thread( + self._client.put_object, + bucket, + key, + data, + length=length, + content_type=content_type, + ) +``` + +--- + +### `backend/celery_app.py` — add beat_schedule entry (modify) + +**Analog:** Existing `beat_schedule` dict in `backend/celery_app.py` (lines 38-43). + +**beat_schedule entry pattern** (lines 38-43 of `backend/celery_app.py`): +```python +celery_app.conf.beat_schedule = { + "cleanup-abandoned-uploads": { + "task": "tasks.document_tasks.cleanup_abandoned_uploads", + "schedule": _timedelta(minutes=30), + }, +} +``` +**Add** (import `crontab` at top alongside `_timedelta`): +```python +from celery.schedules import crontab as _crontab + +celery_app.conf.beat_schedule = { + "cleanup-abandoned-uploads": { + "task": "tasks.document_tasks.cleanup_abandoned_uploads", + "schedule": _timedelta(minutes=30), + }, + "audit-log-daily-export": { + "task": "tasks.audit_tasks.audit_log_daily_export", + "schedule": _crontab(hour=0, minute=0), # midnight UTC + }, +} +``` + +**task_routes extension** (lines 32-35 of `backend/celery_app.py`): +```python +celery_app.conf.task_routes = { + "tasks.document_tasks.*": {"queue": "documents"}, + "tasks.email_tasks.*": {"queue": "email"}, + "tasks.audit_tasks.*": {"queue": "documents"}, # reuse documents worker queue +} +``` + +--- + +### `backend/api/documents.py` — add streaming proxy + search + audit (modify) + +**Analog:** Self. Also RESEARCH.md Pattern 3 (streaming proxy) and Pattern 5 (FTS query). + +**StreamingResponse import** (add to existing imports): +```python +from fastapi import Request +from fastapi.responses import StreamingResponse +from sqlalchemy import func +``` + +**Streaming proxy endpoint pattern** (RESEARCH.md Pattern 3): +```python +@router.get("/{doc_id}/content") +async def stream_document_content( + doc_id: str, + request: Request, + session: AsyncSession = Depends(get_db), + current_user: User = Depends(get_regular_user), # MUST be get_regular_user (Pitfall 3) +): + try: + uid = uuid.UUID(doc_id) + except ValueError: + raise HTTPException(404, "Document not found") + + doc = await session.get(Document, uid) + if doc is None: + raise HTTPException(404, "Document not found") + + # Access: owner OR active share recipient + if doc.user_id != current_user.id: + share_result = await session.execute( + select(Share).where( + Share.document_id == doc.id, + Share.recipient_id == current_user.id, + ) + ) + if share_result.scalar_one_or_none() is None: + raise HTTPException(404, "Document not found") + + file_bytes = await get_storage_backend().get_object(doc.object_key) + file_size = len(file_bytes) + + range_header = request.headers.get("range") + headers = { + "content-type": doc.content_type, + "content-disposition": f'inline; filename="{doc.filename}"', + "accept-ranges": "bytes", + "content-length": str(file_size), + } + + if range_header: + start, end = _parse_range(range_header, file_size) + chunk = file_bytes[start:end + 1] + headers["content-range"] = f"bytes {start}-{end}/{file_size}" + headers["content-length"] = str(len(chunk)) + return StreamingResponse(iter([chunk]), status_code=206, headers=headers) + + return StreamingResponse(iter([file_bytes]), status_code=200, headers=headers) +``` + +**tsvector search query addition to list_documents** (RESEARCH.md Pattern 5): +```python +# In list_documents, add q: Optional[str] = Query(None) parameter +if q and len(q) >= 2: + stmt = stmt.where( + func.to_tsvector("english", func.coalesce(Document.extracted_text, "")).op("@@")( + func.plainto_tsquery("english", q) + ) + ) +``` + +--- + +### `frontend/src/stores/folders.js` (store, CRUD) + +**Analog:** `frontend/src/stores/topics.js` — exact same Pinia defineStore + ref() + async action pattern. + +**Store structure** (full `frontend/src/stores/topics.js` — lines 1-42): +```javascript +import { defineStore } from 'pinia' +import { ref } from 'vue' +import * as api from '../api/client.js' + +export const useTopicsStore = defineStore('topics', () => { + const topics = ref([]) + const loading = ref(false) + const error = ref(null) + + async function fetchTopics() { + loading.value = true + error.value = null + try { + const data = await api.listTopics() + topics.value = data.topics + } catch (e) { + error.value = e.message + } finally { + loading.value = false + } + } + + async function addTopic(payload) { + const topic = await api.createTopic(payload) + topics.value.push(topic) + return topic + } + // ... + return { topics, loading, error, fetchTopics, addTopic, editTopic, removeTopic } +}) +``` +Apply as `useTopicsStore` → `useFoldersStore`, with state refs: `folders`, `currentFolderId`, `breadcrumb`, `loading`, `error`. Actions: `fetchFolders(parentId)`, `createFolder(name, parentId)`, `renameFolder(id, name)`, `deleteFolder(id)`, `navigateTo(folderId)`, `fetchBreadcrumb(folderId)`. + +**Action error-catching pattern** (lines 10-21 of `frontend/src/stores/topics.js`): +```javascript +async function fetchTopics() { + loading.value = true + error.value = null + try { + const data = await api.listTopics() + topics.value = data.topics + } catch (e) { + error.value = e.message + } finally { + loading.value = false + } +} +``` + +--- + +### `frontend/src/components/documents/ShareModal.vue` (component, request-response) + +**Analog:** `frontend/src/components/admin/AdminUsersTab.vue` — modal with form input + list with action buttons. + +**Script setup with reactive form state** (lines 216-235 of `frontend/src/components/admin/AdminUsersTab.vue`): +```javascript +import { ref, reactive, onMounted } from 'vue' +import * as api from '../../api/client.js' + +const loading = ref(false) +const error = ref(null) +const submitting = ref(false) + +const form = reactive({ + handle: '', +}) +``` + +**Pending action + error pattern** (lines 224-229 of `frontend/src/components/admin/AdminUsersTab.vue`): +```javascript +const pendingAction = reactive({}) +const actionError = ref(null) +const creating = ref(false) +const createError = ref(null) +``` + +**Props + emits pattern** — ShareModal is opened from DocumentCard, needs `doc` prop and `close` emit: +```javascript +const props = defineProps({ + doc: Object, +}) +const emit = defineEmits(['close']) +``` + +**Loading/empty/list states template pattern** (lines 96-108 of `frontend/src/components/admin/AdminUsersTab.vue`): +```html +
...
+
+

Not shared with anyone yet.

+
+
+
+ {{ share.recipient_handle }} + +
+
+``` + +**Inline confirm pattern** (lines 151-168 of `frontend/src/components/admin/AdminUsersTab.vue`): +```html +
+

Deactivate ...? ...

+
+ + +
+
+``` + +--- + +### `frontend/src/components/layout/BreadcrumbNav.vue` (component, request-response) + +**Analog:** `frontend/src/components/topics/TopicBadge.vue` — lightweight display component, props-driven, no store usage. + +**Props-only component pattern** (lines 34-46 of `frontend/src/components/documents/DocumentCard.vue`): +```javascript +const props = defineProps({ + doc: Object, +}) +``` +Apply as: +```javascript +const props = defineProps({ + // Array of {id, name} objects from root to current folder + segments: { + type: Array, + default: () => [], + }, +}) +const emit = defineEmits(['navigate']) +``` + +**Truncation logic** (from RESEARCH.md Specifics: depth > 4, show first + "..." + last 2): +```javascript +const visibleSegments = computed(() => { + if (props.segments.length <= 4) return props.segments + return [ + props.segments[0], + { id: null, name: '...' }, + ...props.segments.slice(-2), + ] +}) +``` + +**Tailwind styling pattern from AppSidebar** (lines 128-132 of `frontend/src/components/layout/AppSidebar.vue`): +```css +.nav-link { + @apply flex items-center px-3 py-2 rounded-lg text-gray-600 hover:bg-gray-100 hover:text-gray-900 transition-colors text-sm font-medium; +} +``` + +--- + +### `frontend/src/components/admin/AdminAuditLogTab.vue` (component, CRUD) + +**Analog:** `frontend/src/components/admin/AdminUsersTab.vue` — exact same paginated table with filters and action buttons. + +**onMounted fetch pattern** (lines 368-378 of `frontend/src/components/admin/AdminUsersTab.vue`): +```javascript +onMounted(async () => { + loading.value = true + try { + const data = await api.adminListUsers() + users.value = data.items || [] + } catch (e) { + actionError.value = e.message + } finally { + loading.value = false + } +}) +``` + +**Table with thead/tbody pattern** (lines 110-136 of `frontend/src/components/admin/AdminUsersTab.vue`): +```html +
+ + + + + + + + + + + + ... + + +
TimestampUserActionIP Address
+
+``` + +**Export button pattern** — triggers a window.location or fetch for CSV download: +```javascript +async function exportCsv() { + // Use window.location so the browser triggers a download + const params = new URLSearchParams(filterState) + window.location.href = `/api/admin/audit-log/export?format=csv&${params}` +} +``` + +**Filter state with reactive** (lines 223-232 of `frontend/src/components/admin/AdminUsersTab.vue`): +```javascript +const filterState = reactive({ + start: '', + end: '', + user_id: '', + event_type: '', +}) +``` + +--- + +### `frontend/src/components/layout/AppSidebar.vue` — extend with folders (modify) + +**Analog:** Self. Extend the existing Topics section pattern (lines 36-53). + +**Topics section pattern to replicate for Folders** (lines 36-53 of `frontend/src/components/layout/AppSidebar.vue`): +```html +
+

Topics

+
Loading…
+
No topics yet
+ + + {{ topic.name }} + {{ topic.doc_count }} + +
+``` + +**"Shared with me" entry** — fixed above folder list, inbox icon, doc count badge: +```html + +``` + +--- + +### `frontend/src/components/documents/DocumentCard.vue` — add share button (modify) + +**Analog:** Self. Add an inline icon button following existing SVG icon pattern. + +**Existing SVG icon button pattern** (lines 8-13 of `frontend/src/components/documents/DocumentCard.vue`): +```html +
+ + + +
+``` + +**Share button placement** — add a small icon button in the card's top-right (stop propagation on card click): +```html + + + +``` + +**is_shared indicator badge** — add shared badge if `doc.is_shared`: +```html + + Shared + +``` + +--- + +### `frontend/src/stores/documents.js` — extend with folder/search/sharing actions (modify) + +**Analog:** Self. Follow existing action pattern from `fetchDocuments`, `upload`, `remove`. + +**fetchDocuments extension** (add `folderId`, `q` params — lines 38-50): +```javascript +async function fetchDocuments({ topic, page = 1, perPage = 20, folderId = null, q = null } = {}) { + loading.value = true + error.value = null + try { + const data = await api.listDocuments({ topic, page, perPage, folderId, q }) + documents.value = data.items + total.value = data.total + } catch (e) { + error.value = e.message + } finally { + loading.value = false + } +} +``` + +**Debounced search pattern** (RESEARCH.md Pattern 10 — no external dependency): +```javascript +const searchQuery = ref('') +let _searchTimer = null + +watch(searchQuery, (newVal) => { + clearTimeout(_searchTimer) + if (newVal.length < 2) { + fetchDocuments({ folderId: currentFolderId.value }) + return + } + _searchTimer = setTimeout(() => { + fetchDocuments({ q: newVal, folderId: currentFolderId.value }) + }, 300) +}) +``` + +**New share actions** — follow same try/catch pattern as `remove` (lines 112-119): +```javascript +async function shareDocument(docId, recipientHandle) { + return await api.createShare(docId, recipientHandle) +} + +async function revokeShare(shareId) { + await api.deleteShare(shareId) +} + +async function listShares(docId) { + return await api.listShares(docId) +} +``` + +--- + +### `frontend/src/views/HomeView.vue` — wire folder navigation + breadcrumb (modify) + +**Analog:** Self. Follow existing `onMounted` + store composition pattern (lines 27-62). + +**onMounted store init pattern** (lines 39-40 of `frontend/src/views/HomeView.vue`): +```javascript +onMounted(() => docsStore.fetchDocuments()) +``` +Extend to: +```javascript +onMounted(async () => { + await foldersStore.fetchFolders(null) // root-level folders + await docsStore.fetchDocuments({ folderId: null }) +}) +``` + +**Store composition pattern** (lines 31-37 of `frontend/src/views/HomeView.vue`): +```javascript +const docsStore = useDocumentsStore() +const topicsStore = useTopicsStore() +const uploadQueue = ref([]) +``` +Add: +```javascript +const foldersStore = useFoldersStore() +const currentFolderId = ref(null) +const breadcrumb = ref([]) +``` + +--- + +### `frontend/src/views/SettingsView.vue` — add PDF preference toggle (modify) + +**Analog:** Self (currently a static placeholder). Follow `frontend/src/components/admin/AdminAiConfigTab.vue` for toggle/radio section pattern. + +**Existing static section pattern** (lines 6-11 of `frontend/src/views/SettingsView.vue`): +```html +
+

AI configuration

+

AI provider and model are managed by your administrator.

+
+``` + +**Add new section for PDF preference:** +```html +
+

Document preferences

+

Choose how PDF documents open when you click them.

+
+ + +
+

{{ saveError }}

+
+``` + +**Script setup with watch for auto-save:** +```javascript +import { ref, watch, onMounted } from 'vue' +import * as api from '../api/client.js' + +const pdfOpenMode = ref('in_app') +const saveError = ref(null) + +onMounted(async () => { + const prefs = await api.getMyPreferences() + pdfOpenMode.value = prefs.pdf_open_mode +}) + +watch(pdfOpenMode, async (newVal) => { + try { + await api.updateMyPreferences({ pdf_open_mode: newVal }) + } catch (e) { + saveError.value = e.message + } +}) +``` + +--- + +## Shared Patterns + +### Authentication / Authorization Guard +**Source:** `backend/deps/auth.py` +**Apply to:** All new backend endpoints in `api/folders.py`, `api/shares.py`; streaming proxy in `api/documents.py` +```python +# get_regular_user — for all document/folder/share/proxy endpoints +# Returns 403 for admin role (CLAUDE.md architectural rule) +current_user: User = Depends(get_regular_user) + +# get_current_admin — for all audit log endpoints in api/audit.py +_admin: User = Depends(get_current_admin) +``` + +### Ownership Assertion (404-not-403) +**Source:** `backend/api/documents.py` lines 117-119 +**Apply to:** All folder CRUD, document move, share grant/revoke, streaming proxy +```python +resource = await session.get(ModelClass, resource_id) +if resource is None or resource.user_id != current_user.id: + raise HTTPException(status_code=404, detail=" not found") +``` + +### UUID Parse + 404 +**Source:** `backend/api/documents.py` lines 113-115 +**Apply to:** All path-parameter ID handlers in `api/folders.py` and `api/shares.py` +```python +try: + uid = uuid.UUID(resource_id) +except ValueError: + raise HTTPException(status_code=404, detail=" not found") +``` + +### Atomic Quota UPDATE +**Source:** `backend/api/documents.py` lines 137-146 (increment) and `backend/services/storage.py` (decrement) +**Apply to:** `api/folders.py` DELETE cascade-delete (decrement), existing confirm endpoint (increment) +```python +# Decrement pattern for folder cascade-delete: +await session.execute( + text( + "UPDATE quotas SET used_bytes = " + "CASE WHEN used_bytes > :delta THEN used_bytes - :delta ELSE 0 END " + "WHERE user_id = :uid" + ), + {"delta": total_bytes, "uid": str(current_user.id)}, +) +``` + +### asyncio.to_thread() for Sync MinIO SDK +**Source:** `backend/storage/minio_backend.py` lines 78-86 and `backend/tasks/document_tasks.py` lines 62-66 +**Apply to:** `backend/storage/minio_backend.py` new `put_object_raw()` method; streaming proxy `get_object()` call +```python +await asyncio.to_thread( + self._client.put_object, + bucket, + key, + data, + length=length, + content_type=content_type, +) +``` + +### Pydantic Whitelist Response Helper +**Source:** `backend/api/admin.py` lines 54-69 (`_user_to_dict`) +**Apply to:** `backend/api/audit.py` (`_audit_to_dict`), `backend/api/admin.py` new `CloudConnectionOut` model (SEC-08) +```python +def _audit_to_dict(entry: AuditLog) -> dict: + # Must NEVER include: filename, extracted_text, file bytes, credentials_enc + return { "id": entry.id, "event_type": entry.event_type, ... } +``` + +### write_audit_log Call Site Pattern +**Source:** `backend/services/audit.py` (new) — called inline after each successful operation +**Apply to:** `api/folders.py`, `api/shares.py`, `api/documents.py` (upload/delete/proxy), `api/auth.py` (login/logout/etc.), `api/admin.py` (user create/deactivate/quota change) +```python +# After session.commit() for the primary operation: +await write_audit_log( + session, + event_type="folder.created", + user_id=current_user.id, + actor_id=current_user.id, + resource_id=folder.id, + ip_address=request.headers.get("X-Forwarded-For") or (request.client.host if request.client else None), + metadata_={"name": folder.name, "parent_id": str(folder.parent_id) if folder.parent_id else None}, +) +``` + +### Pinia Store defineStore Pattern +**Source:** `frontend/src/stores/topics.js` and `frontend/src/stores/documents.js` +**Apply to:** `frontend/src/stores/folders.js` +```javascript +export const useFoldersStore = defineStore('folders', () => { + const state = ref([]) + const loading = ref(false) + const error = ref(null) + + async function action() { + loading.value = true + error.value = null + try { ... } catch (e) { error.value = e.message } finally { loading.value = false } + } + + return { state, loading, error, action } +}) +``` + +### api/client.js Export Function Pattern +**Source:** `frontend/src/api/client.js` — `request()` helper with auth header injection and 401 retry +**Apply to:** All new API functions for folders, shares, audit log, preferences, search +```javascript +// Pattern: named export function calling request() +export function listFolders(parentId = null) { + const params = new URLSearchParams() + if (parentId) params.set('parent_id', parentId) + return request(`/api/folders?${params}`) +} + +export function createShare(docId, recipientHandle) { + return request('/api/shares', { + method: 'POST', + headers: { 'Content-Type': 'application/json' }, + body: JSON.stringify({ document_id: docId, recipient_handle: recipientHandle }), + }) +} +``` + +--- + +## No Analog Found + +| File | Role | Data Flow | Reason | +|---|---|---|---| +| `frontend/src/components/layout/BreadcrumbNav.vue` | component | request-response | No existing breadcrumb component; closest analog is TopicBadge (pure display) but breadcrumb has click-navigation behavior. See Shared Patterns for Tailwind style source. | + +--- + +## Metadata + +**Analog search scope:** `backend/api/`, `backend/services/`, `backend/tasks/`, `backend/migrations/`, `backend/storage/`, `frontend/src/stores/`, `frontend/src/components/`, `frontend/src/views/`, `frontend/src/api/` +**Files scanned:** 35 +**Pattern extraction date:** 2026-05-25 diff --git a/.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md b/.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md new file mode 100644 index 0000000..d94c8ea --- /dev/null +++ b/.planning/phases/04-folders-sharing-quotas-document-ux/04-RESEARCH.md @@ -0,0 +1,1088 @@ +# Phase 4: Folders, Sharing, Quotas & Document UX — Research + +**Researched:** 2026-05-25 +**Domain:** FastAPI folder/share CRUD, PostgreSQL tsvector, MinIO streaming proxy, Celery beat, Vue 3 folder navigation +**Confidence:** HIGH + +--- + + +## User Constraints (from CONTEXT.md) + +### Locked Decisions + +**D-01** Hybrid layout — AppSidebar shows top-level folders only. Sub-folders + breadcrumb in main content area. Top-level folders clickable directly in sidebar. + +**D-02** Unlimited nesting depth (no API or UI cap). `Folder.parent_id` self-referential FK is authoritative. Breadcrumbs truncate at depth > 4 (show first + "..." + last 2). + +**D-03** Non-empty folder delete: warning modal with document count. Confirm → cascade-delete all documents (MinIO + DB + quota). Cancel → no action. Documents are NOT moved to root — they are destroyed. + +**D-04** Exact handle input — no autocomplete. API returns 404 if handle not found; UI shows "User not found" error. + +**D-05** Share button on DocumentCard (inline icon button). Modal: (a) handle input, (b) Share button, (c) current recipients list with Revoke per row. + +**D-06** "Shared with me" is a fixed virtual folder entry in AppSidebar, rendered above the user's own folder list. Filtered by `shares.recipient_id = current_user.id`. Zero quota charged to recipient. + +**D-07** Share permission is `view` only for Phase 4. `edit` deferred. + +**D-08** Streaming proxy endpoint: `GET /api/documents/{id}/content`. Returns bytes via FastAPI `StreamingResponse`. Supports `Range` headers. `Content-Disposition: inline`. No presigned URL ever generated or exposed. Uses `get_regular_user` dep. + +**D-09** Native browser PDF rendering — no PDF.js. Zero frontend dependencies added. Content-Type header drives browser rendering. + +**D-10** `users.pdf_open_mode` column (String, default `'in_app'`). Exposed via `PATCH /api/me/preferences`. `in_app` = modal with `