46 Commits
Author SHA1 Message Date
curo1305 52acd5634b feat(14.1-02): force re-analyze flag + single-item retry-job creation
- Add force: bool = False to AnalysisEnqueueRequest in cloud/schemas.py (D-11)
- Extend enqueue_analysis_job to accept force param; already_current check
  becomes (already_current and not force) — bypass for supported items (D-11, ANALYZE-06)
- Add retry_or_create_single_item_job service helper in cloud_analysis.py that
  creates a single-item force enqueue job when no active job exists (D-12)
- Add POST /analysis/connections/{id}/items/{cloud_item_id}/retry route
  in analysis.py returning AnalysisEnqueueOut with queued_count >= 1
- Wire force= through enqueue_job route handler (body.force)
- All 8 test_cloud_reanalyze_force tests pass; 26 test_cloud_analysis_contract
  tests pass (no idempotency regression); 872/873 backend tests pass
2026-06-26 21:59:50 +02:00
curo1305 a0d5c1d6c4 feat(14.1-02): CloudItemDetailOut schema + resolve_owned_cloud_item_detail + GET detail route
- Add CloudItemDetailOut allowlist schema to backend/api/cloud/schemas.py
  (excludes credentials_enc, object_key, version_key, raw provider URLs — T-14.1-03)
- Add CloudItemDetail dataclass and resolve_owned_cloud_item_detail service helper
  to backend/services/cloud_items.py (owner-scoped, metadata-only, no byte hydration)
- Add GET /connections/{id}/items/{item_id:path}/detail route to operations.py
  with get_regular_user (admin 403), ConnectionNotFound → 404, CloudItemNotFound → 404
- All 8 test_cloud_detail_parity tests now pass (D-05, D-07, D-16, T-14.1-03/04/06)
2026-06-26 21:54:28 +02:00
curo1305 dfc335022f feat(14-06): route preview/download bytes through durable byte cache
- Add hydrate_and_cache_bytes to services/cloud_cache.py (Plan 06 cache lifecycle)
  - Cache-hit path: pin existing entry, read from MinIO, release pin in finally
  - Cache-miss path: fetch from provider, store in MinIO, create/reactivate entry, pin during response, release pin in finally
  - CacheQuotaExceeded and MinIO failures are non-fatal (bytes returned from provider path)
  - evict_lru_entries run after pin release to stay within user cache limit
- Update preview_cloud_file and download_cloud_file in operations.py to call hydrate_and_cache_bytes
  - Version key computed from CloudItem metadata (no provider call required for key resolution)
  - Shared _make_minio_cache_wrapper helper eliminates duplicate adapter code
  - Falls through to direct provider fetch when no CloudItem row found
- Add _make_minio_cache_wrapper helper to operations.py (DRY, shared by preview and download)
- Add 5 new tests to test_cloud_cache.py: cache-miss fetches provider, cache-hit skips provider, pin released on hit, pin released on miss, user isolation
- Add 3 new security tests to test_cloud_security.py: preview response excludes object_key, download response excludes object_key, foreign-user IDOR via download blocked
2026-06-23 16:28:06 +02:00
curo1305 cea9980aaf feat(14-05): add cloud analysis processing service (Task 1)
- Implement services/cloud_analysis_processing.py with full pipeline:
  queued -> downloading -> extracting -> classifying -> indexed
- Stale guard: version key comparison fires before provider byte fetch (T-14-13)
- Cache pin lifecycle: pin acquired before bytes used, released in finally block (T-14-12)
- No provider mutation methods called: adapter only used via get_object (T-14-03)
- No local Document row created: classification targets CloudItem directly
- Cooperative cancellation check at job/item level before each stage
- ProcessingResult dataclass with status, cache_entry_id, error_code fields
- Add 4 contract tests: status_transitions, stale_detection, pin_release, no_mutations
2026-06-23 16:02:44 +02:00
curo1305 3f26cd2059 feat(14-04): add cloud_analysis orchestration service
- estimate_scope: file/selection/folder/connection scope from cached metadata only
- enqueue_analysis_job: create CloudAnalysisJob + items; already_current skip without bytes
- already_current detection: indexed analysis_status + live metadata check (metadata-only, T-14-04)
- cancel_job, cancel_job_item, skip_job_item, retry_job_item: full control operations
- check_scan_quota: seam for future tier enforcement (always True in Phase 14)
- _is_supported: content type + extension list; explicit unsupported-override for .exe etc
- get_adapter / resolve_provider_metadata: stubs for Plan 05 provider integration
- No provider mutation methods called (T-14-03); no bytes downloaded (T-14-04)
- 16/16 test_cloud_analysis_contract.py tests pass
2026-06-23 15:50:24 +02:00
curo1305 5bdd23f3bc feat(14-03): add cache orchestration functions and tests
- retain_or_reuse_cache_entry: reuse non-evicted entry or reactivate evicted row
- pin_cache_entry / release_cache_entry: pin_count lifecycle management
- update_cache_access: touch last_accessed_at for LRU ordering
- Ownership assertions in all new service functions (T-14-06)
- 9 new integration tests covering retain/reuse, pin/release, and isolation
2026-06-23 15:24:13 +02:00
curo1305 1df61040c6 feat(14-02): add version-key, settings helpers and Phase 14 cache service functions
- cloud_analysis_versioning.py: compute_version_key with version>etag>fingerprint
  precedence (D-19); compute_fingerprint helper; content_hash is optional post-
  hydration only (D-20); raises ValueError only, no HTTPException
- cloud_analysis_settings.py: get_or_create_user_analysis_settings (upsert on
  first access), update_user_analysis_settings with enum validation and tier-capped
  cache limit, build_default_settings for API responses; raises ValueError only
- cloud_cache.py: Phase 14 durable cache functions added alongside existing Phase 12
  in-memory TTL cache — create_cache_entry, list_cache_entries (owner-scoped),
  evict_lru_entries (LRU, skips pin_count>0 and active_job_count>0), increment/
  decrement_quota_for_cache (atomic UPDATE pattern); re-exports compute_version_key
  from cloud_analysis_versioning so tests have one import site
2026-06-23 15:14:19 +02:00
curo1305 e1ce4cbe14 feat(13-05): implement keep_both_name() for D-03/D-05 collision naming
- keep_both_name(filename, counter) inserts counter before first extension
- Handles compound extensions (archive.tar.gz → archive (1).tar.gz)
- Handles extensionless files (README → README (1))
- Lives in services/cloud_operations as shared upload-queue helper
- All 23 upload contract + keep-both naming tests pass
2026-06-22 19:17:59 +02:00
curo1305 6784d3bdb7 feat(13-03): add cloud operations service and reconnect/health/test routes
- Create backend/services/cloud_operations.py as the single Phase 13 orchestration seam:
  get_connection_health (D-12), test_connection (D-13), reconnect_connection (CONN-01/02/03,
  D-14), disconnect_connection (D-16, explicit CloudItem cascade for SQLite compatibility)
- Add POST /connections/{id}/reconnect route (CONN-01/02/03, D-14)
- Add GET /connections/{id}/health route (D-12, T-13-02)
- Add POST /connections/{id}/test route (D-13, T-13-02)
- Update DELETE /connections/{id} to use service-layer disconnect with explicit
  CloudItem deletion (D-16; covers FK-cascade-less environments like SQLite tests)
- Fix (Rule 2 - T-13-02): add _scrub_audit_metadata() to admin audit log API to
  remove credential fields before returning audit rows to admin callers
- All 14 reconnect tests pass + 110 provider-contract tests pass
- Pre-existing failures in test_cloud_mutations and test_extractor are unrelated
2026-06-22 18:43:50 +02:00
curo1305 bfa4dc502a feat(12.1-02): centralize listing freshness gate — apply_listing_and_finalize
- Add ListingResult dataclass to cloud_items.py
- Add apply_listing_and_finalize() — single shared gate for complete/incomplete semantics
- complete=True: upserts, soft-deletes unseen children, marks fresh, advances last_refreshed_at
- complete=False: upserts seen items only, never deletes unseen, preserves prior timestamp, warns
- browse.py: replace unconditional update_folder_state('fresh') with apply_listing_and_finalize
- cloud_tasks.py: replace unconditional fresh transition with apply_listing_and_finalize
- Celery task returns structured warning status on incomplete (no credentials/provider item names)
- Fix timezone naive/aware comparison in test (SQLite strips tzinfo)
2026-06-22 08:29:41 +02:00
curo1305andClaude Sonnet 4.6 b6911fb4ed fix(12): resolve critical code review findings (CR-01 through CR-04)
- CR-01: fix browse URL from /folders/{id} to /items?parent_ref= (broken feature)
- CR-02: remove raw provider exception text from _TerminalProviderError messages
- CR-03: add user_id predicate to get_or_create_folder_state query (ownership boundary)
- CR-04: add max_length=255 to ConnectionRenameRequest.display_name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 02:00:05 +02:00
curo1305 e186019066 feat(12-02): decompose api/cloud.py into package with connection-ID browse endpoint
- Add api/cloud/ package: connections.py, browse.py, schemas.py, __init__.py
- Add GET /api/cloud/connections/{connection_id}/items (T-12-01 IDOR, T-12-03 cred-free)
- Add PATCH /api/cloud/connections/{id} for display_name_override rename
- Add display_name_override ORM field to CloudConnection model
- Add CloudResourceAdapter service layer with str/UUID coercion
- Fix UUID type compatibility: test_cloud_items.py now uses UUID(as_uuid=True)
  matching conftest — removes String(36) patch that caused type incompatibility
- Add 11 Phase 12 integration tests (IDOR, admin block, credential exclusion,
  duplicate providers, rename, malformed UUID)
- Remove deleted api/cloud.py (replaced by api/cloud/ package)
2026-06-18 23:10:30 +02:00
curo1305 718fb2c2b5 feat(12-01): durable owner-scoped cloud metadata schema (migration 0006 + models)
- Migration 0006: cloud_items, cloud_item_topics, cloud_folder_states tables
- cloud_connections: add display_name_override column for same-provider disambiguation
- CloudItem, CloudItemTopic, CloudFolderState ORM models with owner/connection indexes
- Unique (connection_id, provider_item_id) boundary; no MinIO object_key field
- Root folder state representable as parent_ref='' without CloudItem parent row
- services/cloud_items.py: resolve_owned_connection, upsert, list, reconcile, folder state
- 17 unit/integration tests covering model fields, isolation, quota invariant, idempotency
2026-06-18 22:37:28 +02:00
curo1305 e97ca164d7 Refactor backend and frontend cleanup paths 2026-06-16 11:50:17 +02:00
curo1305 4d7157d7fc feat(08-04): add validate_provider_id helper to services/ai_config.py (D-11)
- Migrate inline provider_id validator to service layer per CLAUDE.md
- Raises ValueError (not HTTPException) — service-layer rule
- PROVIDER_DEFAULTS already imported; placed after load_provider_config helpers
2026-06-10 18:39:03 +02:00
curo1305andClaude Sonnet 4.6 c77b97b6d3 fix(fgp): add null-byte separator in HMAC input to prevent concatenation collision
Without a separator, UA='foobar'+AL='' and UA='foo'+AL='bar' produced the same
HMAC input, allowing a token fingerprint to match a different header split.
Fix: use '\x00' as separator between user_agent and accept_lang fields.

Regression test added: test_fgp_no_concatenation_collision (FGP-05).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-06 23:03:48 +02:00
curo1305 25c9142fe0 feat(07.4-02): add _compute_fgp helper + extend create_access_token with fgp claim
- Add _compute_fgp(user_agent, accept_lang) helper returning 16-char hex
  HMAC-SHA256 fingerprint (D-04); uses existing hmac + hashlib imports
- Extend create_access_token signature with user_agent/accept_lang params
  (empty-string defaults for backward compatibility, D-05)
- Embed fgp claim in every issued access token payload
2026-06-06 21:54:47 +02:00
curo1305 9cc11b5446 feat(07.3-03): backend remember_me — TTL split + cookie Max-Age + 3 promoted tests
- services/auth.py: create_refresh_token gains remember_me=False param; selects 16h or 30d TTL (D-09, D-10, D-11)
- api/auth.py: LoginRequest.remember_me bool field; _set_refresh_cookie remember_me param for conditional max_age; login handler threads remember_me through both calls (D-11, RM-03)
- test_auth_es256.py: promote RM-01, RM-02, RM-03 stubs — all 9 phase tests now PASSED
2026-06-06 17:21:14 +02:00
curo1305 8be792ab4c feat(07.3-02): ES256 JWT algorithm upgrade + startup rotation hook
- config.py: add refresh_token_expire_hours=16, jwt_private_key, jwt_public_key fields (D-01, D-09)
- services/auth.py: swap all 4 JWT sites to ES256 via base64-decoded PEM keys; remove HS256 (D-02, D-03)
- main.py: add _rotate_tokens_on_algorithm_change lifespan hook — bulk-revokes refresh tokens on algorithm change; idempotent on repeat boots (D-04, D-05)
- test_auth_es256.py: promote ES256-01..05 + CFG-01 stubs to 6 passing tests; RM-01..03 remain xfail
- docker-compose.yml: inject JWT_PRIVATE_KEY + JWT_PUBLIC_KEY into backend + celery-worker (D-07)
- README.md: add JWT key env vars + key generation Python one-liner snippet
- .env.example: add JWT_PRIVATE_KEY= and JWT_PUBLIC_KEY= lines
- Version bump to 0.1.2
2026-06-06 17:19:50 +02:00
curo1305 c00c1fbaa7 feat(07.2-02): add jti UUID claim to create_access_token
- Added "jti": str(uuid.uuid4()) to the access token payload in services/auth.py
- Promoted Wave 0 xfail stub test_create_access_token_includes_jti_claim to PASS
- Added test_create_access_token_jti_is_unique_per_call uniqueness guard
- uuid module was already imported at module top — no new imports needed
2026-06-05 19:15:01 +02:00
curo1305andClaude Sonnet 4.6 c38c6b1c01 feat(07.1): session revocation on privilege change — CR-01/CR-02/CR-03
- revoke_all_refresh_tokens: add skip_token_hash optional param (exclude
  current session while revoking others)
- change_password, enable_totp, disable_totp: call revoke with skip hash
  derived from refresh cookie; return sessions_revoked in response and
  write to audit log metadata_
- 3 new tests: test_{change_password,enable_totp,disable_totp}_revokes_other_sessions
  — all PASSED; 373 total passing, 0 regressions
- Frontend toasts: SettingsAccountTab + TotpEnrollment show
  "Other sessions have been terminated." when sessions_revoked > 0
- Companion fixes: rate_limiting get_client_ip refactor, deps/auth.py
  request.state.current_user, locustfile refresh-token task removal
- Version bump: 0.1.0 → 0.1.1

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 12:47:16 +02:00
curo1305andClaude Sonnet 4.6 76ebc3e96e fix(07): add status field to _doc_to_dict + regression test
DocumentCard classification_failed badge requires doc.status from the
API list endpoint. _doc_to_dict was omitting the field; this fix adds
it and adds a regression test that would have caught the omission.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-05 09:30:03 +02:00
curo1305 e678930b8d feat(07-05): admin AI-config endpoints + load_provider_config_by_id + atomic is_active flip
- Add GET/PUT /api/admin/ai-config + GET /api/admin/ai-config/test-connection
- _ai_config_to_dict whitelist excludes api_key_enc (T-07-01)
- PUT: HKDF-encrypt api_key, atomic UPDATE SET is_active = (provider_id = target) (T-07-03)
- PUT: upsert with PROVIDER_DEFAULTS for omitted fields; audit log records fields_changed only (T-07-14)
- All 3 endpoints require get_current_admin (T-07-15)
- Add load_provider_config_by_id to services/ai_config.py (ignores is_active)
- Promote 3 xfail tests in test_admin_ai_config.py — all 3 pass
2026-06-04 23:19:40 +02:00
curo1305 95c386f764 feat(07-03): classifier wired to load_provider_config + ai_config stub removed — D-04/D-06
- Remove _ProviderConfigStub from services/ai_config.py (replaced by real ProviderConfig)
- Add module-level import: from ai.provider_config import ProviderConfig, PROVIDER_DEFAULTS
- load_provider_config() now returns ProviderConfig (no lazy import inside function body)
- classifier.py: replace inline _settings dict with load_provider_config(session) call (D-06)
- Per-user override path builds ProviderConfig from PROVIDER_DEFAULTS (no api_key — T-07-06)
- Fallback to app_settings defaults when load_provider_config returns None (D-15)
- Truncation delegated to provider._truncate() — no more text slices in classifier (D-12/D-13)
- Promote test_api_key_encrypt_decrypt to passing (round-trip + domain salt isolation)
- Update test_classifier.py: test_per_user_provider + test_default_provider_fallback use ProviderConfig assertions
2026-06-04 19:14:48 +02:00
curo1305 02bcbb9143 feat(07-02): singleton OpenAIProvider + GenericOpenAIProvider + MAX_AI_CHARS removal
- openai_provider.py: singleton self._client=AsyncOpenAI(...) in __init__ (D-07),
  _truncate() 60/40 smart truncation (D-13), removed MAX_AI_CHARS constant,
  changed __init__ signature to include context_chars, removed _client() method
- generic_openai_provider.py: new class, subclasses OpenAIProvider, conditional
  response_format={"type":"json_object"} on supports_json_mode flag (D-01/D-02),
  imports parse_classification + parse_suggestions from ai.utils (D-02 contract)
- ollama_provider.py: added context_chars kwarg with default 8000, passes through
- lmstudio_provider.py: added context_chars kwarg with default 8000, passes through
- classifier.py: removed MAX_AI_CHARS constant and text[:MAX_AI_CHARS] slices;
  truncation now handled inside each provider via _truncate()
- requirements.txt: bumped anthropic floor to >=0.95.0 (D-03 output_config support)
2026-06-04 18:58:19 +02:00
curo1305 0fd6930b41 feat(07-01): services/ai_config.py HKDF helpers, ProviderConfig loader, env seed
- Create backend/services/ai_config.py with:
  - _derive_ai_settings_key(master_key, provider_id): fresh HKDF per call, salt=provider_id.encode("utf-8"), info=b"ai-provider-settings" (domain-separated from b"cloud-credentials")
  - encrypt_api_key / decrypt_api_key: Fernet round-trip without JSON wrapping
  - load_provider_config(session): reads is_active=True row from system_settings, decrypts api_key_enc; returns stub ProviderConfig (real class lands in Plan 02)
  - seed_system_settings_from_env(session): idempotent insert of default provider on first boot
  - _ProviderConfigStub: minimal Pydantic model stub until ai/provider_config.py exists (Plan 02)
- Update backend/main.py: import seed_system_settings_from_env, call in lifespan with try/except so missing table (pre-migration fresh boot) doesn't crash startup
- Round-trip smoke test: encrypt_api_key → decrypt_api_key == original plaintext
2026-06-04 18:50:01 +02:00
curo1305andClaude Sonnet 4.6 f98cab30e9 chore: merge executor worktree (06-05) — trusted-proxy rate limiting + per-account limiter
Resolved merge conflicts:
- backend/main.py: keep both setup_logging (06-02) and account_limiter (06-05) imports
- backend/tests/test_rate_limiting.py: take 06-05 version (all 8 xfails promoted)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 18:31:46 +02:00
curo1305andClaude Sonnet 4.6 a826738e18 feat(06-05): trusted-proxy get_client_ip, per-account rate limiter, promote 8 xfail tests (D-11/D-12/D-13)
- D-11: replace get_client_ip body with trusted-proxy CIDR check (127/8, 172.16/12,
  192.168/16, ::1/128); untrusted peers always return their own IP, preventing XFF
  spoofing by external callers
- D-12: create services/rate_limiting.py with _account_key() keyed by user.id
  (falls back to peer IP when no authenticated user in request.state); exports
  account_limiter = Limiter(key_func=_account_key)
- Wire: auth.py switches from get_remote_address to get_client_ip; main.py imports
  account_limiter; all 9 documents.py endpoints and 7 cloud.py endpoints decorated
  @account_limiter.limit("100/minute") with request.state.current_user = current_user
  as the first handler statement (A1 ordering invariant)
- Promote all 8 xfail stubs to real assertions; add autouse fixture in conftest.py
  to reset MemoryStorage between tests preventing cross-test 429 contamination

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-04 18:27:49 +02:00
curo1305 9fa74a91f5 feat(06-02): add structlog dependency + create services/logging.py
- Append structlog>=25.5.0 to requirements.txt under Phase 6 D-01 comment
- Create backend/services/logging.py with setup_logging(json_logs, log_level)
- shared_processors: merge_contextvars first, then add_log_level, add_logger_name,
  PositionalArgumentsFormatter, ExtraAdder, TimeStamper(iso), StackInfoRenderer
- JSON branch appends format_exc_info to shared_processors
- JSONRenderer when json_logs=True, ConsoleRenderer when False
- ProcessorFormatter bridges uvicorn/stdlib loggers through same chain
- Idempotent: clears root handlers before installing new one
- uvicorn.access propagate=False — middleware owns request logging
2026-06-03 18:44:37 +02:00
curo1305andClaude Sonnet 4.6 a548266461 refactor(backend): extract shared helper modules per architecture rules
- Add backend/ai/utils.py — parse_classification, parse_suggestions, strip_code_fences
  shared by all AI providers; removes duplicated private functions from
  anthropic_provider.py and openai_provider.py
- Add backend/deps/utils.py — get_client_ip, parse_uuid request-parsing helpers;
  removes local _ip() variants from admin.py, auth.py, shares.py, folders.py
- Add backend/storage/exceptions.py — canonical CloudConnectionError definition;
  all routers and backends import from here instead of redefining
- Move validate_password_strength to backend/services/auth.py; removes duplicated
  _validate_password_strength from admin.py and auth.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 16:10:35 +02:00
curo1305andClaude Sonnet 4.6 b245fcc527 fix(phase-03): use UUID.hex in raw SQL to fix SQLite UUID format mismatch
str(uuid_obj) produces dashed 36-char format; SQLite stores UUID as 32-char
hex without dashes, so WHERE user_id = :uid never matched. Using .hex fixes
confirm_upload (api/documents.py) and delete_document (services/storage.py).
Removes stale xfail from test_delete_decrements_quota — now passes on SQLite.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 15:43:01 +02:00
curo1305 2072c3ddcd fix(06.2): WR-08 delete_document defers commit so audit log writes in same transaction 2026-06-01 14:30:31 +02:00
curo1305andClaude Sonnet 4.6 95c7ed786a feat(06.2-03): backend — cloud-aware delete routing + skip_quota + remove_only param
- storage.delete_document gains skip_quota=False param; quota decrement gated on it
- DELETE /api/documents/{id} gains remove_only=bool query param
- Cloud docs (storage_backend != minio): attempt cloud backend delete_object first
  - On failure: return HTTP 200 {success: false, cloud_delete_failed: true} (not 4xx)
  - On success or remove_only: delete DB row with skip_quota=True
- Cloud creds/exception message never included in response body (T-06.2-03-02)
- Promote 3 xfail stubs to real tests (propagates, failure, remove_only)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-31 15:09:44 +02:00
curo1305 fb803795fa feat(05-02): implement cloud_cache.py and extend storage factory
- cloud_cache.py: module-level TTLCache(maxsize=1000, ttl=60) singleton with
  threading.Lock for concurrent access safety (RESEARCH.md Pattern 8 / D-16)
- get_cloud_folders_cached(): async function; calls fetch_fn OUTSIDE the lock
  to avoid blocking the event loop during cloud API calls
- invalidate_provider_cache(): removes all cache entries for a user+provider prefix
- storage/__init__.py: adds get_storage_backend_for_document() async factory
  — returns MinIOBackend for minio docs; queries CloudConnection (scoped to user.id),
  decrypts credentials, and lazy-imports cloud backends to avoid circular imports
  — raises HTTPException(503) if connection missing or not ACTIVE (T-05-02-04)
2026-05-28 21:00:48 +02:00
curo1305andClaude Sonnet 4.6 87a32b7ee8 feat(phase-4): complete UX redesign — FileManagerView, FolderTreeItem, test suite, and all Phase 4 fixes
Adds the unified file manager view (Windows Explorer-style), collapsible
folder tree sidebar item, full vitest test suite (55 tests, 4 files), and
commits all Phase 4 backend/frontend fixes that were staged but uncommitted.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-28 17:10:52 +02:00
curo1305 259a1542d8 feat(phase-4): add write_audit_log() async helper (flush-not-commit, never-raises)
- Creates backend/services/audit.py with write_audit_log() function
- Uses session.flush() not session.commit() per D-14 architectural requirement
- Catches and logs all exceptions (never re-raises) so audit failure is non-fatal
- Correct AuditLog ORM attribute metadata_ (not metadata) per models.py
2026-05-25 18:33:31 +02:00
curo1305andClaude Sonnet 4.6 a5994d9ff4 chore: commit pending phase-3 work and add TEST_ACCOUNTS.md
Includes planning artifacts (03-CONTEXT, 03-DISCUSSION-LOG, 03-02-SUMMARY),
integration test script, MinIO/auth/docker fixes, and local dev account reference.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-24 11:30:56 +02:00
curo1305 6849ebd1e6 feat(03-04): retire flat-file settings; wire per-user AI config via DB lookup
- config.py: Remove SETTINGS_FILE, DEFAULT_SYSTEM_PROMPT, DEFAULT_SETTINGS
  constants; add system_prompt, default_ai_provider, default_ai_model to Settings
- services/classifier.py: Add _DEFAULT_SYSTEM_PROMPT module constant; classify_document
  and suggest_topics_for_document accept ai_provider/ai_model kwargs; no longer calls
  storage.load_settings() — uses app_settings defaults with DB-supplied overrides (D-14, D-15)
- services/storage.py: Delete load_settings, save_settings, mask_api_key, settings_masked;
  remove from __all__; remove import copy, json, DEFAULT_SETTINGS, SETTINGS_FILE (D-12)
- tasks/document_tasks.py: _run resolves user.ai_provider/ai_model via session.get(User,
  doc.user_id) and passes through to classifier; task signature unchanged (T-03-19)
- api/settings.py: Deleted — /api/settings endpoint removed (D-12)
- main.py: Remove settings_router import and include_router call
- tests/test_settings.py: Replace all tests with test_settings_endpoint_removed (404, green)
- tests/test_classifier.py: Implement test_per_user_provider, test_celery_task_uses_user_provider,
  test_default_provider_fallback; remove xfail markers (DOC-03, DOC-05)
2026-05-23 20:32:55 +02:00
curo1305 5950a3f5c2 feat(03-03): wire get_current_user into /api/topics/*; add load_topics_for_user; POST /api/admin/topics
- api/topics.py: add get_current_user dep to all 5 handlers (list, create, update, delete, suggest)
- list_topics: uses load_topics_for_user (system topics + user's own) with user-scoped doc counts
- create_topic: passes user_id=current_user.id (never creates system topics via regular endpoint)
- update_topic/delete_topic: ownership assertion — system topics and other users' topics return 404
- api/admin.py: add SystemTopicCreate model + POST /api/admin/topics (user_id=NULL, admin-only)
- services/storage.py: add or_ import; load_topics_for_user (D-17); create_topic gains user_id param with namespace-scoped dedup; topic_doc_counts gains optional user_id for user-scoped counts; add load_topics_for_user to __all__
- services/classifier.py: replace load_topics with load_topics_for_user(doc.user_id); pass user_id=doc.user_id to create_topic for AI-suggested topics (D-11)
- Tests: update all topic tests to pass auth headers; implement test_topic_namespace, test_admin_create_system_topic, test_regular_user_cannot_create_system_topic, test_topics_require_auth
2026-05-23 20:15:44 +02:00
curo1305 b28bb01995 feat(03-03): add get_regular_user dep; wire auth + ownership into /api/documents/*
- Add get_regular_user FastAPI dep (rejects admin with 403) to deps/auth.py
- Wire Depends(get_regular_user) into all 6 /api/documents/* handlers
- upload-url: replace null-user/... object_key with str(current_user.id)/...; set user_id=current_user.id
- confirm: remove Wave 2 doc.user_id is None guard — quota runs unconditionally; add ownership assertion (404 on cross-user)
- list: filter by user_id=current_user.id via storage.list_metadata(user_id=...)
- get/delete/classify: ownership assertion (doc.user_id != current_user.id → 404)
- storage.list_metadata: add required user_id param + Document.user_id == user_id filter
- storage.delete_document: remove if doc.user_id is not None guard; use CASE WHEN for SQLite-compat quota decrement
- Tests: update existing tests to pass auth headers; implement test_cross_user_access_404, test_admin_cannot_access_documents, test_documents_require_auth; mark test_confirm_endpoint xfail(strict=False) for SQLite UUID mismatch
2026-05-23 20:05:34 +02:00
curo1305 0d51d023ce feat(03-02): implement presigned upload flow, quota enforcement, cleanup task
- Replace POST /api/documents/upload with POST /api/documents/upload-url + /{id}/confirm
- upload-url: create pending Document row with user_id=None (Wave 2), return presigned PUT URL
- confirm: stat MinIO for authoritative size (T-03-05), atomic quota UPDATE (T-03-06, STORE-03)
- Confirm returns 413 with {used_bytes, limit_bytes, rejected_bytes} on quota exceeded (STORE-05)
- Wave 2 guard: skip quota UPDATE when doc.user_id is None (Plan 03-03 removes this)
- Add GET /api/auth/me/quota to api/auth.py (STORE-04)
- services/storage.py: remove save_upload (D-04); add GREATEST(0, used_bytes-delta) quota decrement to delete_document (STORE-06)
- tasks/document_tasks.py: add cleanup_abandoned_uploads Celery beat task (D-06)
- celery_app.py: add beat_schedule for cleanup-abandoned-uploads every 30 minutes
- tests/test_documents.py: replace legacy /upload tests with xfail; add real test logic for upload-url/confirm/get-quota
- tests/test_quota.py: implement real test logic with xfail for PostgreSQL-specific SQL
2026-05-23 14:32:12 +02:00
curo1305andClaude Sonnet 4.6 1882edfff6 feat(02-02): auth API endpoints + security hardening + Python 3.9 compat
- backend/api/auth.py: register, login (TOTP+backup), refresh, logout,
  me, change-password; per-account Redis rate limit; HIBP check
- backend/main.py: Origin validation middleware, CSP headers middleware,
  CORS locked to settings.cors_origins, Redis lifespan (app.state.redis),
  admin bootstrap, auth router included, slowapi SlowAPIMiddleware
- backend/services/email.py: already created in Plan 01 (verified exists)
- Python 3.9 compat: fixed match statement in ai/__init__.py,
  str|None union syntax in openai_provider.py, api/documents.py,
  api/topics.py, api/settings.py, services/classifier.py

All 17 tests in test_auth_api.py pass.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 19:35:38 +02:00
curo1305 9fc820d893 feat(02-01): implement services/auth.py full auth service layer and email_tasks.py
- services/auth.py: Argon2 password hashing (pwdlib), constant-time verify (SEC-06)
- JWT create/decode for access tokens and password-reset tokens (typ claim validation, T-02-01)
- Refresh token lifecycle: create, rotate, revoke-all with family revocation (AUTH-07, RFC 9700)
- Family revocation enqueues send_security_alert_email.delay on token reuse (T-02-02)
- TOTP provisioning (pyotp) and verification with Redis replay prevention, valid_window=1 (AUTH-08)
- Backup code generation (8-char hex uppercase), storage (Argon2 hashed), constant-time verify (T-02-03)
- HIBP k-anonymity check via SHA-1 prefix (T-02-05), fail-open on network error (T-02-06)
- Admin bootstrap: idempotent, logs WARNING if env vars missing (D-04/D-05/D-06)
- services/email.py: SMTP send + dev stdout fallback (D-01/D-02)
- tasks/email_tasks.py: send_reset_email and send_security_alert_email Celery tasks
- celery_app.py: add email queue route for tasks.email_tasks.*
- TDD tests: 17 tests covering all auth primitives and family revocation
2026-05-22 19:23:42 +02:00
curo1305 32d67de1ca feat(01-05): introduce celery_app + tasks/document_tasks + session-aware classifier
- Add backend/celery_app.py: Celery("docuvault") with Redis broker, JSON
  serialization, and tasks.document_tasks.* routed to documents queue;
  reads REDIS_URL directly from os.environ (no config import — Pitfall 7)
- Add backend/tasks/__init__.py: empty package marker
- Add backend/tasks/document_tasks.py: sync extract_and_classify Celery task
  that calls asyncio.run(_run()) to retrieve bytes from MinIO, extract text
  via extractor, and classify via classifier; classification failure is non-fatal
- Update backend/services/classifier.py: classify_document and
  suggest_topics_for_document now accept session: AsyncSession as first arg;
  all storage.* calls updated to async session-injection pattern
- Add extract_text_from_bytes helper to services/extractor.py for bytes-based
  extraction (used by Celery worker, which retrieves bytes from MinIO)
2026-05-22 09:45:33 +02:00
curo1305 3e4b1f1f91 feat(01-04): rewrite services/storage.py as async SQLAlchemy + MinIO orchestrator
- Replaced entire flat-file + filelock implementation with async ORM + MinIO
- All 14 DB-touching functions are async def accepting AsyncSession as first param
- load_settings/save_settings/mask_api_key/settings_masked remain sync (flat-file, Phase 2 will migrate)
- save_upload uses null-user D-03 sentinel; object_key via MinIO put_object
- update_document_topics auto-creates missing topics via create_topic deduplication
- No filelock, no METADATA_DIR/UPLOADS_DIR/TOPICS_FILE references remain
- Added __all__ listing all 18 public functions
- Updated conftest.py: removed filelock patching no longer needed
- Fixed test_object_key_schema: removed unused db_session param (SQLite INET type conflict)
2026-05-22 09:39:32 +02:00
curo1305andClaude Sonnet 4.6 7a34807fa0 chore: initial commit — existing single-user document scanner codebase
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-05-22 08:53:28 +02:00