curo1305
87a32b7ee8
Adds the unified file manager view (Windows Explorer-style), collapsible folder tree sidebar item, full vitest test suite (55 tests, 4 files), and commits all Phase 4 backend/frontend fixes that were staged but uncommitted. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
110 lines
5.2 KiB
Markdown
110 lines
5.2 KiB
Markdown
---
|
|
phase: 04-folders-sharing-quotas-document-ux
|
|
plan: "06"
|
|
subsystem: admin-audit
|
|
tags: [audit-log, admin-api, celery, csv-export, minio, security]
|
|
dependency_graph:
|
|
requires: ["04-03", "04-04"]
|
|
provides: ["ADMIN-06", "D-17"]
|
|
affects: ["backend/api/audit.py", "backend/tasks/audit_tasks.py", "backend/celery_app.py", "backend/main.py"]
|
|
tech_stack:
|
|
added: []
|
|
patterns:
|
|
- "Admin-only audit log viewer with paginated, filtered SQLAlchemy query"
|
|
- "Streaming CSV export via FastAPI StreamingResponse + csv.DictWriter"
|
|
- "Celery beat crontab schedule at midnight UTC for daily MinIO export"
|
|
- "Deferred imports inside async task body to prevent circular imports"
|
|
- "_audit_to_dict() safe whitelist serializer pattern (mirrors _user_to_dict)"
|
|
key_files:
|
|
created:
|
|
- backend/api/audit.py
|
|
- backend/tasks/audit_tasks.py
|
|
modified:
|
|
- backend/celery_app.py
|
|
- backend/main.py
|
|
decisions:
|
|
- "CSV export reuses _audit_to_dict() whitelist helper — single source of truth for safe field set"
|
|
- "audit_tasks.* routed to documents queue — reuses existing documents worker (no new queue needed)"
|
|
- "crontab alias uses _crontab (underscore prefix) consistent with existing _timedelta alias"
|
|
metrics:
|
|
duration_seconds: 262
|
|
completed_date: "2026-05-25"
|
|
tasks_completed: 2
|
|
files_created: 2
|
|
files_modified: 2
|
|
---
|
|
|
|
# Phase 4 Plan 06: Admin Audit Log API + Celery Daily Export Summary
|
|
|
|
**One-liner:** Admin-only paginated/filtered audit log viewer with CSV streaming export (ADMIN-06) and midnight-UTC Celery beat task uploading daily CSVs to MinIO audit-logs bucket (D-17).
|
|
|
|
## Tasks Completed
|
|
|
|
| Task | Name | Commit | Files |
|
|
|------|------|--------|-------|
|
|
| 1 | Admin audit log viewer + CSV export | 364447d | backend/api/audit.py, backend/main.py |
|
|
| 2 | Celery daily export task + beat schedule | f89f787 | backend/tasks/audit_tasks.py, backend/celery_app.py |
|
|
|
|
## What Was Built
|
|
|
|
### Task 1: backend/api/audit.py
|
|
|
|
Two admin-only endpoints protected by `Depends(get_current_admin)`:
|
|
|
|
- `GET /api/admin/audit-log` — paginated (page/per_page), filtered (start, end, user_id, event_type). Returns `{items, total, page, per_page}`. Runs a separate COUNT query for total using the same filters.
|
|
- `GET /api/admin/audit-log/export` — same filter params, no pagination; streams CSV with `Content-Disposition: attachment; filename=audit-export.csv`.
|
|
|
|
The `_audit_to_dict()` helper is the single source of truth for the safe field set: `id, event_type, user_id, actor_id, resource_id, ip_address, metadata_, created_at`. The dict literal contains no `filename`, `extracted_text`, `password_hash`, or `credentials_enc` keys. Both the JSON and CSV paths use this same helper.
|
|
|
|
### Task 2: backend/tasks/audit_tasks.py + celery_app.py
|
|
|
|
- `audit_log_daily_export` Celery task: sync entry point → `asyncio.run(_run_daily_export())`.
|
|
- `_run_daily_export()`: queries yesterday's `AuditLog` rows (UTC midnight to midnight), writes CSV via `csv.DictWriter`, uploads to MinIO via `put_object_raw(bucket="audit-logs", key="audit-logs/YYYY-MM-DD.csv", ...)`. Wraps everything in try/except — returns `{"exported": 0, "error": str(e)}` on failure.
|
|
- All imports deferred inside `_run_daily_export()` body (same circular-import-prevention pattern as `document_tasks._run`).
|
|
- `celery_app.py`: `_crontab` aliased import, beat entry `"audit-log-daily-export"` at `_crontab(hour=0, minute=0)`, task route `"tasks.audit_tasks.*": {"queue": "documents"}`.
|
|
|
|
## Deviations from Plan
|
|
|
|
None — plan executed exactly as written.
|
|
|
|
## Security Invariants Verified
|
|
|
|
| Threat ID | Check | Result |
|
|
|-----------|-------|--------|
|
|
| T-04-06-01 | `Depends(get_current_admin)` on both endpoints (grep: 2 occurrences at lines 94, 129) | PASS |
|
|
| T-04-06-02 | `_audit_to_dict()` dict literal contains no forbidden keys (grep: filename/extracted_text only in comments) | PASS |
|
|
| T-04-06-03 | CSV export uses same `_audit_to_dict()` helper as JSON viewer | PASS |
|
|
| T-04-06-04 | `put_object_raw` uses `bucket="audit-logs"` (not documents bucket) | PASS |
|
|
|
|
## Test Results
|
|
|
|
```
|
|
tests/test_audit.py: 4 xfailed (stub tests from Wave 0 — plan 04-06 implements the API,
|
|
detailed integration tests will be written in the full TDD pass)
|
|
Full suite: 1 failed (test_extractor.py::test_extract_docx — pre-existing missing module,
|
|
out of scope), 130 passed, 7 skipped, 35 xfailed
|
|
```
|
|
|
|
Pre-existing failures (not caused by this plan):
|
|
- `test_extractor.py::test_extract_docx` — missing python-docx module in local env
|
|
- `test_documents.py::test_content_stream_200` — intentional TDD RED from plan 04-05 (commit 8e6cb6e)
|
|
|
|
## Known Stubs
|
|
|
|
None — both endpoints are fully implemented and wired.
|
|
|
|
## Threat Flags
|
|
|
|
None — no new network endpoints or trust boundaries beyond those documented in the plan's threat model.
|
|
|
|
## Self-Check: PASSED
|
|
|
|
- [x] `backend/api/audit.py` exists: FOUND
|
|
- [x] `backend/tasks/audit_tasks.py` exists: FOUND
|
|
- [x] Task 1 commit 364447d: FOUND
|
|
- [x] Task 2 commit f89f787: FOUND
|
|
- [x] `python3 -c "from api.audit import router"` exits 0: PASS
|
|
- [x] `python3 -c "from tasks.audit_tasks import audit_log_daily_export"` exits 0: PASS
|
|
- [x] `beat_schedule` contains `audit-log-daily-export`: PASS
|
|
- [x] `task_routes` contains `tasks.audit_tasks.*`: PASS
|