- cloud_cache.py: module-level TTLCache(maxsize=1000, ttl=60) singleton with
threading.Lock for concurrent access safety (RESEARCH.md Pattern 8 / D-16)
- get_cloud_folders_cached(): async function; calls fetch_fn OUTSIDE the lock
to avoid blocking the event loop during cloud API calls
- invalidate_provider_cache(): removes all cache entries for a user+provider prefix
- storage/__init__.py: adds get_storage_backend_for_document() async factory
— returns MinIOBackend for minio docs; queries CloudConnection (scoped to user.id),
decrypts credentials, and lazy-imports cloud backends to avoid circular imports
— raises HTTPException(503) if connection missing or not ACTIVE (T-05-02-04)
- validate_cloud_url(): blocks RFC-1918 (10.x, 172.16.x, 192.168.x), loopback (127.x),
link-local (169.254.x), IPv6 loopback (::1), ULA (fc00::/7), and 'localhost' string;
resolves DNS via socket.getaddrinfo BEFORE IP check (anti-DNS-rebinding per D-17)
- _derive_fernet_key(): creates fresh HKDF-SHA256 instance per call (AlreadyFinalized
pitfall avoided per RESEARCH.md Pitfall 3); uses user_id as salt for per-user isolation
- encrypt_credentials(): Fernet-encrypts JSON-serialised credentials dict; returns str
- decrypt_credentials(): decrypts Fernet token back to original dict
- [Rule 1 - Bug] Fixed test_allows_public_https to use 8.8.8.8 IP (cloud.example.com
does not resolve in offline CI environments)
Adds the unified file manager view (Windows Explorer-style), collapsible
folder tree sidebar item, full vitest test suite (55 tests, 4 files), and
commits all Phase 4 backend/frontend fixes that were staged but uncommitted.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- Add CloudConnectionOut Pydantic model (SEC-08): credentials_enc deliberately excluded
- Implement DELETE /api/admin/users/{id} (SEC-09): collects user docs, deletes MinIO
objects best-effort before DB delete; audit log written within same transaction
- Add write_audit_log calls to: create_user (admin.user_created), update_user_status
(admin.user_deactivated/admin.user_activated), update_user_quota (admin.quota_changed),
update_ai_config (admin.ai_provider_assigned), delete_user (admin.user_deleted)
- Add Request param to all admin state-changing handlers for IP extraction
- Fix test_admin_impersonation_not_found: accept 405 in addition to 404/422
(expected: DELETE /users/{id} exists now, so GET returns 405 — no impersonation
route still satisfied, just a different HTTP status for non-existent method)
- Add PreferencesUpdate Pydantic model with Literal['in_app', 'new_tab'] validation
- Add GET /api/auth/me/preferences — returns current pdf_open_mode
- Add PATCH /api/auth/me/preferences — validates + stores + returns updated value
- Both endpoints use get_current_user (admin can set own prefs, D-10)
- Add 7 preference tests: default GET, in_app, new_tab, invalid 422, persist,
and two unauthenticated 401 tests
- Add _parse_range() helper: validates Range header bounds, raises 416 on invalid
- Add stream_document_content endpoint with get_regular_user dep (admin → 403)
- Access check: owner OR Share.recipient_id; neither → 404
- Bytes fetched via get_object() only — presigned_get_url() never called
- Range requests return 206 + Content-Range header
- Add pdf_open_mode column to User ORM model (migration 0004 already applied)
- Use HTTP_416_RANGE_NOT_SATISFIABLE (non-deprecated constant)
- POST /api/shares: grant share by recipient_handle; 400 self-share, 404 bad UUID/doc/user, 409 duplicate
- GET /api/shares?document_id: list shares owned by current user for a document
- GET /api/shares/received: virtual "shared with me" folder — metadata only (no extracted_text)
- DELETE /api/shares/{share_id}: revoke with IDOR protection (share.owner_id != current_user.id → 404)
- IntegrityError on UniqueConstraint(document_id, recipient_id) → 409
- write_audit_log called for share.granted and share.revoked (D-14)
- /received defined before /{share_id} in router to prevent FastAPI path parameter conflict
- No quota table touched — recipient quota never modified by share operations (T-04-04-04)
- backend/api/folders.py: POST /api/folders (create), GET /api/folders (list),
GET /api/folders/{id} (breadcrumb), PATCH /api/folders/{id} (rename),
DELETE /api/folders/{id} (cascade-delete + atomic quota decrement),
PATCH /api/documents/{id}/folder (move document)
- All folder endpoints use get_regular_user (admin gets 403); 404 for IDOR
- IntegrityError caught -> 409 on duplicate folder name under same parent
- WITH RECURSIVE CTE for subtree collection with SQLite fallback (OperationalError)
- Atomic quota decrement with CASE WHEN pattern (SQLite compat)
- MinIO object deletion best-effort (per-object try/except)
- write_audit_log called after folder.created, folder.renamed, folder.deleted
- backend/api/documents.py: add sort, order, folder_id, q params to list_documents;
add is_shared field to each document in response using Share subquery
- backend/main.py: register folders_router and document_move_router
- Creates backend/services/audit.py with write_audit_log() function
- Uses session.flush() not session.commit() per D-14 architectural requirement
- Catches and logs all exceptions (never re-raises) so audit failure is non-fatal
- Correct AuditLog ORM attribute metadata_ (not metadata) per models.py
- Add users.pdf_open_mode column via batch_alter_table (server_default='in_app')
- Create GIN expression index ix_documents_fts on documents.extracted_text via raw SQL (Alembic #1390)
- Create audit-logs MinIO bucket gated on MINIO_ENDPOINT env var
- Add MinIOBackend.put_object_raw() for caller-supplied bucket+key uploads (audit CSV export)
Bugs fixed:
- minio_backend.py: generate_presigned_put_url and presigned_get_url used internal
_client (minio:9000) instead of _public_client (localhost:9000). Browser received
ERR_NAME_NOT_RESOLVED. Fixed by using _public_client with region='us-east-1' to
skip region-discovery HTTP request from inside the container.
- docker-compose.yml: MINIO_API_CORS_ALLOW_ORIGIN was set from CORS_ORIGINS which
uses pydantic JSON list format '["http://localhost:5173"]'. MinIO expected a plain
string and never matched the origin. Fixed to use FRONTEND_URL instead.
- admin.py: All write handlers (create_user, update_user_status, update_user_quota,
update_ai_config) used session.flush() without session.commit(). Changes appeared
to succeed (response reflected in-memory state) but rolled back on session close.
Fixed by replacing flush() with commit() in all four write handlers.
- auth.js: Concurrent refresh() calls from QuotaBar and App.vue on page reload caused
a token rotation race — first call rotated the cookie, second arrived with stale
cookie and cleared accessToken. Fixed by deduplicating with a shared in-flight
promise (_refreshInFlight).
Phase 3 UAT: 9/10 pass. UAT-3 (QuotaBar visual) pending browser confirmation.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Includes planning artifacts (03-CONTEXT, 03-DISCUSSION-LOG, 03-02-SUMMARY),
integration test script, MinIO/auth/docker fixes, and local dev account reference.
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
- revision="0003", down_revision="0002"
- upgrade(): collects null-user object_keys, deletes document_topics cascade,
deletes null-user documents, removes MinIO objects (skip if MINIO_ENDPOINT unset),
deletes all topics (D-10), alters documents.user_id NOT NULL via batch_alter_table,
creates ix_topics_user_id index, reconciles quotas.used_bytes from SUM(size_bytes)
- downgrade(): drops ix_topics_user_id, reverts user_id to nullable; documents not restored
- batch_alter_table ensures SQLite compatibility for test suite
- MinIO step gated on MINIO_ENDPOINT env var for safe SQLite test runs
- GET /api/admin/users: list users (safe fields only, ordered by created_at)
- POST /api/admin/users: create user (password_must_change=True, quota init)
- PATCH /api/admin/users/{id}/status: deactivate/reactivate with sole-admin guard
- POST /api/admin/users/{id}/password-reset: Celery email dispatch (no token returned)
- GET /api/admin/users/{id}/quota: quota view with MB helpers
- PATCH /api/admin/users/{id}/quota: quota adjust with below-usage warning
- PATCH /api/admin/users/{id}/ai-config: assign AI provider/model per user
- _user_to_dict() whitelist helper prevents password_hash/credentials_enc leakage
- No impersonation endpoint (ADMIN-07 enforced by omission)
- get_current_admin Depends() on every handler (SEC-07)
- Updated backend/main.py to include admin_router
- Fixed test: mock send_reset_email.delay to avoid Redis in unit tests
- Add BackupCode model to db/models.py with user_id FK, code_hash (Argon2), used_at (nullable)
- Add ix_backup_codes_user_id index on backup_codes.user_id
- Add password_must_change BOOLEAN NOT NULL DEFAULT false to User model (ADMIN-01)
- Extend config.py Settings with JWT, SMTP, admin bootstrap, and CORS fields (D-01, D-04, D-09)
- Add env_list_separator=',' for cors_origins env var parsing
- Append PyJWT, pwdlib[argon2], pyotp, aioredis, slowapi to requirements.txt
- Add .env.example entries for SECRET_KEY, ADMIN_EMAIL, SMTP_*, CORS_ORIGINS
- Create migration 0002 adding backup_codes table and password_must_change column
- Add TDD tests for all Task 1 acceptance criteria (7 tests pass)
- Delete backend/data/ tracked files (D-04): flat-file metadata, settings.json,
topics.json, and uploaded files removed from git; backend/data/ added to
.gitignore (empty dir remains on macOS due to ACL — no tracked files remain)
- Prune backend/config.py: remove DATA_DIR, UPLOADS_DIR, METADATA_DIR,
TOPICS_FILE, ensure_data_dirs(); rebase SETTINGS_FILE as derived path from
settings.data_dir (Phase 1 flat-file settings kept per plan decision)
- Prune backend/tests/conftest.py: remove isolated_data_dir autouse fixture
and sync TestClient client fixture; add SQLite type compatibility shim
(visit_INET/JSONB) so in-memory db_session can create tables with
PostgreSQL-specific column types; add live_services_available fixture
- Rewrite backend/tests/test_documents.py: delete all legacy sync tests,
remove all @pytest.mark.xfail markers; async-only document tests now
use async_client + storage service directly for topic wiring
- Rewrite backend/tests/test_health.py: delete legacy sync test_health(client);
remove @pytest.mark.xfail from test_health_checks_postgres_and_minio
- Port backend/tests/test_topics.py to async_client (sync client removed)
- Port backend/tests/test_settings.py to async_client with monkeypatch for
SETTINGS_FILE isolation (settings remain flat-file in Phase 1)
- Add backend/celery_app.py: Celery("docuvault") with Redis broker, JSON
serialization, and tasks.document_tasks.* routed to documents queue;
reads REDIS_URL directly from os.environ (no config import — Pitfall 7)
- Add backend/tasks/__init__.py: empty package marker
- Add backend/tasks/document_tasks.py: sync extract_and_classify Celery task
that calls asyncio.run(_run()) to retrieve bytes from MinIO, extract text
via extractor, and classify via classifier; classification failure is non-fatal
- Update backend/services/classifier.py: classify_document and
suggest_topics_for_document now accept session: AsyncSession as first arg;
all storage.* calls updated to async session-injection pattern
- Add extract_text_from_bytes helper to services/extractor.py for bytes-based
extraction (used by Celery worker, which retrieves bytes from MinIO)
- backend/alembic.ini: script_location=migrations, sqlalchemy.url=%(DATABASE_MIGRATE_URL)s
- backend/migrations/env.py: async_engine_from_config + Base.metadata wiring;
runtime os.environ.get("DATABASE_MIGRATE_URL") injection (alembic.ini interpolation
does not read OS env directly)
- backend/migrations/versions/0001_initial_schema.py: creates all 11 tables in
dependency order with correct FKs, indexes, and named constraints
- documents.user_id is nullable=True per D-03; Phase 2 adds NOT NULL
- Ends with GRANT + ALTER DEFAULT PRIVILEGES for docuvault_app (Pitfall 4)
- Also grants USAGE/SELECT on sequences (audit_log.id autoincrement)
- downgrade() drops all tables in reverse dependency order
- Add @pytest_asyncio.fixture db_session: in-memory SQLite via aiosqlite,
expire_on_commit=False, skips gracefully (ImportError) before Plan 03
- Add @pytest_asyncio.fixture async_client: httpx.AsyncClient with
ASGITransport, overrides deps.db.get_db, skips before Plan 03
- Retain all legacy sync fixtures (isolated_data_dir, client, sample_txt,
sample_pdf) unchanged for backward compatibility through Plan 04
curo1305