feat(phase-4): complete UX redesign — FileManagerView, FolderTreeItem, test suite, and all Phase 4 fixes
Adds the unified file manager view (Windows Explorer-style), collapsible folder tree sidebar item, full vitest test suite (55 tests, 4 files), and commits all Phase 4 backend/frontend fixes that were staged but uncommitted. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
curo1305
@@ -1,14 +1,15 @@
|
||||
---
|
||||
status: testing
|
||||
status: complete
|
||||
phase: 03-document-migration-multi-user-isolation
|
||||
source: 03-01-SUMMARY.md, 03-02-SUMMARY.md, 03-03-SUMMARY.md, 03-04-SUMMARY.md, 03-05-SUMMARY.md
|
||||
started: 2026-05-24T00:00:00Z
|
||||
updated: 2026-05-24T00:00:00Z
|
||||
updated: 2026-05-25T00:00:00Z
|
||||
completed: 2026-05-25T00:00:00Z
|
||||
---
|
||||
|
||||
## Current Test
|
||||
|
||||
UAT-3 — QuotaBar visual in sidebar (needs browser confirmation)
|
||||
All tests complete — Phase 3 UAT passed.
|
||||
|
||||
## Tests
|
||||
|
||||
@@ -23,7 +24,8 @@ reported: "User confirmed upload works after fixes."
|
||||
|
||||
### 3. QuotaBar displays in sidebar
|
||||
expected: After the upload completes, look at the left sidebar. A quota bar widget is visible below the navigation links. It shows used/total storage (e.g. "1.2 MB / 100 MB") with an indigo-colored fill bar. No error state or broken layout.
|
||||
result: [pending]
|
||||
result: pass
|
||||
reported: "QuotaBar visible in sidebar with indigo fill bar. Confirmed by user 2026-05-25."
|
||||
|
||||
### 4. Quota rejection error block
|
||||
expected: Upload a file that would push usage over the user's quota limit (create a user via admin with a very small quota, e.g. 1 byte, or use an account already near-full). The upload row shows a red "Not enough storage" error block with role="alert", showing the rejected file size, current used bytes, and quota limit. A "Manage storage →" link appears. The quota bar does NOT increase past the limit.
|
||||
@@ -62,9 +64,9 @@ reported: "document_tasks.py _run() resolves ai_provider from user.ai_provider w
|
||||
## Summary
|
||||
|
||||
total: 10
|
||||
passed: 9
|
||||
passed: 10
|
||||
issues: 0
|
||||
pending: 1
|
||||
pending: 0
|
||||
skipped: 0
|
||||
blocked: 0
|
||||
|
||||
|
||||
@@ -0,0 +1,158 @@
|
||||
---
|
||||
phase: 04-folders-sharing-quotas-document-ux
|
||||
plan: "05"
|
||||
subsystem: backend-api
|
||||
tags: [streaming-proxy, content-delivery, preferences, pdf, range-requests, doc-02, doc-01]
|
||||
dependency_graph:
|
||||
requires:
|
||||
- 04-02 # pdf_open_mode migration 0004 adds users.pdf_open_mode column
|
||||
- 04-04 # Share model with recipient_id for access control
|
||||
provides:
|
||||
- "GET /api/documents/{id}/content — streaming proxy from MinIO"
|
||||
- "GET /api/auth/me/preferences — read pdf_open_mode"
|
||||
- "PATCH /api/auth/me/preferences — update pdf_open_mode"
|
||||
affects:
|
||||
- 04-09 # frontend uses content URL + preferences PATCH for PDF display
|
||||
tech_stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "StreamingResponse with iter([bytes]) for zero-copy streaming"
|
||||
- "_parse_range() module-level helper for RFC 7233 byte range parsing"
|
||||
- "Literal['in_app', 'new_tab'] Pydantic field for allowlist enforcement (T-04-05-05)"
|
||||
- "get_regular_user dep blocks admin access to content proxy (T-04-05-01)"
|
||||
key_files:
|
||||
created: []
|
||||
modified:
|
||||
- path: backend/api/documents.py
|
||||
role: "Added _parse_range() + stream_document_content endpoint"
|
||||
- path: backend/api/auth.py
|
||||
role: "Added PreferencesUpdate model + GET/PATCH /me/preferences endpoints"
|
||||
- path: backend/db/models.py
|
||||
role: "Added pdf_open_mode column to User ORM model"
|
||||
- path: backend/tests/test_documents.py
|
||||
role: "Replaced xfail stubs with 8 real streaming proxy tests"
|
||||
- path: backend/tests/test_auth_api.py
|
||||
role: "Added 7 preferences endpoint tests"
|
||||
decisions:
|
||||
- "Use get_regular_user (not get_current_user) for content proxy: admin role blocked at dep level (T-04-05-01)"
|
||||
- "Fetch bytes via get_object() directly — presigned_get_url() forbidden in proxy handler (T-04-05-02)"
|
||||
- "Access check inline in handler body (not helper function) for test mocking simplicity"
|
||||
- "HTTP_416_RANGE_NOT_SATISFIABLE used instead of deprecated HTTP_416_REQUESTED_RANGE_NOT_SATISFIABLE"
|
||||
- "pdf_open_mode added to User ORM model (migration 0004 already added the DB column)"
|
||||
- "GET /me/preferences uses AttributeError guard for env without migration run"
|
||||
metrics:
|
||||
duration: "~15 minutes"
|
||||
completed: "2026-05-25"
|
||||
tasks_completed: 2
|
||||
tasks_total: 2
|
||||
files_modified: 5
|
||||
---
|
||||
|
||||
# Phase 04 Plan 05: Document Streaming Proxy + PDF Preferences Summary
|
||||
|
||||
**One-liner:** MinIO streaming proxy for document content with Range header support (DOC-02) and pdf_open_mode user preference endpoint (DOC-01) backed by Literal validation.
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Task 1: GET /api/documents/{id}/content (DOC-02)
|
||||
|
||||
Added a streaming proxy endpoint to `backend/api/documents.py`:
|
||||
|
||||
- **`_parse_range(range_header, file_size)`** module-level helper that parses RFC 7233 `bytes=X-Y` syntax, handles open-ended ranges (`bytes=X-` and `bytes=-Y`), and raises HTTP 416 on any invalid or out-of-bounds range
|
||||
- **`stream_document_content`** endpoint at `GET /api/documents/{id}/content`:
|
||||
- Uses `get_regular_user` dep — admin role → 403 (T-04-05-01, CRITICAL)
|
||||
- Parses doc_id as UUID → 404 on ValueError
|
||||
- Loads Document via session.get → 404 if None
|
||||
- Access: `doc.user_id == current_user.id` OR `Share.recipient_id == current_user.id`; neither → 404 (T-04-05-04)
|
||||
- Fetches bytes via `get_storage_backend().get_object(doc.object_key)` — no presigned URL (T-04-05-02)
|
||||
- Returns `StreamingResponse` with `content-type`, `content-disposition: inline`, `accept-ranges: bytes`, `content-length`
|
||||
- Range header present → 206 with `content-range: bytes {start}-{end}/{total}` (T-04-05-03)
|
||||
- No Range → 200
|
||||
|
||||
Also added `pdf_open_mode` column to `User` ORM model (migration 0004 already added the DB column).
|
||||
|
||||
### Task 2: Preferences Endpoints (DOC-01)
|
||||
|
||||
Added to `backend/api/auth.py`:
|
||||
|
||||
- **`PreferencesUpdate`** Pydantic model with `pdf_open_mode: Literal["in_app", "new_tab"]`
|
||||
- **`GET /api/auth/me/preferences`** — returns `{"pdf_open_mode": ...}` using `get_current_user` (both roles)
|
||||
- **`PATCH /api/auth/me/preferences`** — validates via Literal, updates `current_user.pdf_open_mode`, commits, returns updated value
|
||||
|
||||
## Commits
|
||||
|
||||
| Hash | Message |
|
||||
|------|---------|
|
||||
| `8e6cb6e` | test(phase-4-05): add failing tests for document streaming proxy (DOC-02) — RED phase |
|
||||
| `f868a4e` | feat(phase-4-05): document streaming proxy GET /api/documents/{id}/content (DOC-02) — GREEN phase |
|
||||
| `2a0df32` | feat(phase-4-05): PATCH /api/auth/me/preferences for pdf_open_mode (DOC-01) |
|
||||
|
||||
## Test Results
|
||||
|
||||
**Before this plan:** 85 passed, 10 xfailed (document tests only had xfail stubs)
|
||||
**After this plan:** 137 passed, 35 xfailed — all new tests green
|
||||
|
||||
| Test | Result |
|
||||
|------|--------|
|
||||
| test_content_stream_200 | PASSED |
|
||||
| test_content_stream_206_range | PASSED |
|
||||
| test_content_stream_admin_403 | PASSED |
|
||||
| test_content_stream_no_presigned_url | PASSED |
|
||||
| test_content_stream_share_recipient_200 | PASSED |
|
||||
| test_content_stream_not_found | PASSED |
|
||||
| test_content_stream_invalid_id | PASSED |
|
||||
| test_parse_range_416 | PASSED |
|
||||
| test_get_preferences_default | PASSED |
|
||||
| test_patch_preferences_in_app | PASSED |
|
||||
| test_patch_preferences_new_tab | PASSED |
|
||||
| test_patch_preferences_invalid_value | PASSED |
|
||||
| test_patch_preferences_persists | PASSED |
|
||||
| test_preferences_requires_auth | PASSED |
|
||||
| test_patch_preferences_requires_auth | PASSED |
|
||||
|
||||
Pre-existing failure: `test_extract_docx` (missing `python-docx` module in local env — not introduced by this plan).
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 1 - Bug] Deprecated HTTP_416 status constant**
|
||||
- **Found during:** Task 1 implementation
|
||||
- **Issue:** `HTTP_416_REQUESTED_RANGE_NOT_SATISFIABLE` is deprecated in the installed FastAPI version; `HTTP_416_RANGE_NOT_SATISFIABLE` is the current constant
|
||||
- **Fix:** Used `HTTP_416_RANGE_NOT_SATISFIABLE` throughout
|
||||
- **Files modified:** `backend/api/documents.py`
|
||||
- **Commit:** f868a4e
|
||||
|
||||
**2. [Rule 2 - Missing functionality] pdf_open_mode absent from User ORM model**
|
||||
- **Found during:** Task 2 — plan noted "migration 0004 adds column" but the ORM `User` class in `models.py` did not declare `pdf_open_mode`
|
||||
- **Fix:** Added `pdf_open_mode: Mapped[str]` with `server_default="in_app"` to the User class
|
||||
- **Files modified:** `backend/db/models.py`
|
||||
- **Commit:** f868a4e
|
||||
|
||||
## Security Invariants Verified
|
||||
|
||||
| Threat ID | Status |
|
||||
|-----------|--------|
|
||||
| T-04-05-01: Admin blocked at content proxy | VERIFIED — `get_regular_user` dep; `test_content_stream_admin_403` passes |
|
||||
| T-04-05-02: No presigned URL in proxy | VERIFIED — `presigned_mock.assert_not_called()` in `test_content_stream_no_presigned_url` |
|
||||
| T-04-05-03: Range validation bounds | VERIFIED — `test_parse_range_416` confirms 416 on out-of-bounds |
|
||||
| T-04-05-04: Non-recipient 404 | VERIFIED — unshared doc returns 404; share recipient gets 200 |
|
||||
| T-04-05-05: pdf_open_mode Literal | VERIFIED — `test_patch_preferences_invalid_value` confirms 422 on invalid value |
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None — all behavior is fully implemented and wired.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None — no new network endpoints, auth paths, or file access patterns beyond what was planned in the threat model.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- [x] `backend/api/documents.py` — stream_document_content endpoint exists
|
||||
- [x] `backend/api/auth.py` — /me/preferences routes registered (GET + PATCH)
|
||||
- [x] `backend/db/models.py` — pdf_open_mode column on User model
|
||||
- [x] `backend/tests/test_documents.py` — 8 streaming proxy tests pass
|
||||
- [x] `backend/tests/test_auth_api.py` — 7 preferences tests pass
|
||||
- [x] Commits: 8e6cb6e (RED), f868a4e (GREEN Task 1), 2a0df32 (Task 2)
|
||||
- [x] Full suite: 137 passed, 1 pre-existing failure (test_extract_docx — missing docx module)
|
||||
@@ -0,0 +1,109 @@
|
||||
---
|
||||
phase: 04-folders-sharing-quotas-document-ux
|
||||
plan: "06"
|
||||
subsystem: admin-audit
|
||||
tags: [audit-log, admin-api, celery, csv-export, minio, security]
|
||||
dependency_graph:
|
||||
requires: ["04-03", "04-04"]
|
||||
provides: ["ADMIN-06", "D-17"]
|
||||
affects: ["backend/api/audit.py", "backend/tasks/audit_tasks.py", "backend/celery_app.py", "backend/main.py"]
|
||||
tech_stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "Admin-only audit log viewer with paginated, filtered SQLAlchemy query"
|
||||
- "Streaming CSV export via FastAPI StreamingResponse + csv.DictWriter"
|
||||
- "Celery beat crontab schedule at midnight UTC for daily MinIO export"
|
||||
- "Deferred imports inside async task body to prevent circular imports"
|
||||
- "_audit_to_dict() safe whitelist serializer pattern (mirrors _user_to_dict)"
|
||||
key_files:
|
||||
created:
|
||||
- backend/api/audit.py
|
||||
- backend/tasks/audit_tasks.py
|
||||
modified:
|
||||
- backend/celery_app.py
|
||||
- backend/main.py
|
||||
decisions:
|
||||
- "CSV export reuses _audit_to_dict() whitelist helper — single source of truth for safe field set"
|
||||
- "audit_tasks.* routed to documents queue — reuses existing documents worker (no new queue needed)"
|
||||
- "crontab alias uses _crontab (underscore prefix) consistent with existing _timedelta alias"
|
||||
metrics:
|
||||
duration_seconds: 262
|
||||
completed_date: "2026-05-25"
|
||||
tasks_completed: 2
|
||||
files_created: 2
|
||||
files_modified: 2
|
||||
---
|
||||
|
||||
# Phase 4 Plan 06: Admin Audit Log API + Celery Daily Export Summary
|
||||
|
||||
**One-liner:** Admin-only paginated/filtered audit log viewer with CSV streaming export (ADMIN-06) and midnight-UTC Celery beat task uploading daily CSVs to MinIO audit-logs bucket (D-17).
|
||||
|
||||
## Tasks Completed
|
||||
|
||||
| Task | Name | Commit | Files |
|
||||
|------|------|--------|-------|
|
||||
| 1 | Admin audit log viewer + CSV export | 364447d | backend/api/audit.py, backend/main.py |
|
||||
| 2 | Celery daily export task + beat schedule | f89f787 | backend/tasks/audit_tasks.py, backend/celery_app.py |
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Task 1: backend/api/audit.py
|
||||
|
||||
Two admin-only endpoints protected by `Depends(get_current_admin)`:
|
||||
|
||||
- `GET /api/admin/audit-log` — paginated (page/per_page), filtered (start, end, user_id, event_type). Returns `{items, total, page, per_page}`. Runs a separate COUNT query for total using the same filters.
|
||||
- `GET /api/admin/audit-log/export` — same filter params, no pagination; streams CSV with `Content-Disposition: attachment; filename=audit-export.csv`.
|
||||
|
||||
The `_audit_to_dict()` helper is the single source of truth for the safe field set: `id, event_type, user_id, actor_id, resource_id, ip_address, metadata_, created_at`. The dict literal contains no `filename`, `extracted_text`, `password_hash`, or `credentials_enc` keys. Both the JSON and CSV paths use this same helper.
|
||||
|
||||
### Task 2: backend/tasks/audit_tasks.py + celery_app.py
|
||||
|
||||
- `audit_log_daily_export` Celery task: sync entry point → `asyncio.run(_run_daily_export())`.
|
||||
- `_run_daily_export()`: queries yesterday's `AuditLog` rows (UTC midnight to midnight), writes CSV via `csv.DictWriter`, uploads to MinIO via `put_object_raw(bucket="audit-logs", key="audit-logs/YYYY-MM-DD.csv", ...)`. Wraps everything in try/except — returns `{"exported": 0, "error": str(e)}` on failure.
|
||||
- All imports deferred inside `_run_daily_export()` body (same circular-import-prevention pattern as `document_tasks._run`).
|
||||
- `celery_app.py`: `_crontab` aliased import, beat entry `"audit-log-daily-export"` at `_crontab(hour=0, minute=0)`, task route `"tasks.audit_tasks.*": {"queue": "documents"}`.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
None — plan executed exactly as written.
|
||||
|
||||
## Security Invariants Verified
|
||||
|
||||
| Threat ID | Check | Result |
|
||||
|-----------|-------|--------|
|
||||
| T-04-06-01 | `Depends(get_current_admin)` on both endpoints (grep: 2 occurrences at lines 94, 129) | PASS |
|
||||
| T-04-06-02 | `_audit_to_dict()` dict literal contains no forbidden keys (grep: filename/extracted_text only in comments) | PASS |
|
||||
| T-04-06-03 | CSV export uses same `_audit_to_dict()` helper as JSON viewer | PASS |
|
||||
| T-04-06-04 | `put_object_raw` uses `bucket="audit-logs"` (not documents bucket) | PASS |
|
||||
|
||||
## Test Results
|
||||
|
||||
```
|
||||
tests/test_audit.py: 4 xfailed (stub tests from Wave 0 — plan 04-06 implements the API,
|
||||
detailed integration tests will be written in the full TDD pass)
|
||||
Full suite: 1 failed (test_extractor.py::test_extract_docx — pre-existing missing module,
|
||||
out of scope), 130 passed, 7 skipped, 35 xfailed
|
||||
```
|
||||
|
||||
Pre-existing failures (not caused by this plan):
|
||||
- `test_extractor.py::test_extract_docx` — missing python-docx module in local env
|
||||
- `test_documents.py::test_content_stream_200` — intentional TDD RED from plan 04-05 (commit 8e6cb6e)
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None — both endpoints are fully implemented and wired.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None — no new network endpoints or trust boundaries beyond those documented in the plan's threat model.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- [x] `backend/api/audit.py` exists: FOUND
|
||||
- [x] `backend/tasks/audit_tasks.py` exists: FOUND
|
||||
- [x] Task 1 commit 364447d: FOUND
|
||||
- [x] Task 2 commit f89f787: FOUND
|
||||
- [x] `python3 -c "from api.audit import router"` exits 0: PASS
|
||||
- [x] `python3 -c "from tasks.audit_tasks import audit_log_daily_export"` exits 0: PASS
|
||||
- [x] `beat_schedule` contains `audit-log-daily-export`: PASS
|
||||
- [x] `task_routes` contains `tasks.audit_tasks.*`: PASS
|
||||
Reference in New Issue
Block a user