12 Commits
Author SHA1 Message Date
curo1305 f235d894fb feat(14-05): add Celery cloud analysis task with retry harness (Task 2)
- Implement tasks/cloud_analysis_tasks.py: sync Celery bridge to async processing
- Broker payload contains IDs only — credentials decrypted in worker (T-14-10)
- _TransientError and _ClassificationRetry sentinels for bounded retry (max=3)
- Auth errors are terminal (not retried); MaxRetriesExceeded writes failure status
- celery_app.py: add cloud_analysis_tasks route and import registration
- Add 6 contract tests: task registration, broker payload shape, queue routing,
  bounded retries, sentinel pattern, connection-not-found graceful drop
2026-06-23 16:06:07 +02:00
curo1305 bfa4dc502a feat(12.1-02): centralize listing freshness gate — apply_listing_and_finalize
- Add ListingResult dataclass to cloud_items.py
- Add apply_listing_and_finalize() — single shared gate for complete/incomplete semantics
- complete=True: upserts, soft-deletes unseen children, marks fresh, advances last_refreshed_at
- complete=False: upserts seen items only, never deletes unseen, preserves prior timestamp, warns
- browse.py: replace unconditional update_folder_state('fresh') with apply_listing_and_finalize
- cloud_tasks.py: replace unconditional fresh transition with apply_listing_and_finalize
- Celery task returns structured warning status on incomplete (no credentials/provider item names)
- Fix timezone naive/aware comparison in test (SQLite strips tzinfo)
2026-06-22 08:29:41 +02:00
curo1305andClaude Sonnet 4.6 b6911fb4ed fix(12): resolve critical code review findings (CR-01 through CR-04)
- CR-01: fix browse URL from /folders/{id} to /items?parent_ref= (broken feature)
- CR-02: remove raw provider exception text from _TerminalProviderError messages
- CR-03: add user_id predicate to get_or_create_folder_state query (ownership boundary)
- CR-04: add max_length=255 to ConnectionRenameRequest.display_name

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-19 02:00:05 +02:00
curo1305 c6237ca57f feat(12-02): add refresh_cloud_folder Celery task, staleness trigger, version 0.1.5
- Add tasks/cloud_tasks.py: durable refresh_cloud_folder task with 3-retry
  bounded backoff (30s/90s/270s +jitter), credential decryption in worker only
- Register tasks.cloud_tasks.* on documents queue in celery_app.py
- Add stale-while-revalidate staleness trigger in browse.py (5-min threshold)
- Add 4 Task 3 tests: idempotency, cached-row retention on failure, task structure,
  no-byte-download contract; add background-refresh scheduling integration test
- Bump backend version 0.1.4 → 0.1.5, frontend package.json 0.1.4 → 0.1.5
- Update AGENTS.md with Phase 12 Plan 02 state and new shared module map entries
- Update README with connection-ID browse API table and v0.1.5
2026-06-18 23:17:34 +02:00
curo1305 e9ee5d4ba5 feat(07-04): Celery retry harness + _ClassificationError sentinel (D-09/D-10)
- Add _ClassificationError sentinel raised by _run() for classification failures
- Add _mark_classification_failed() async helper for final status writeback
- Change decorator to bind=True, max_retries=3
- Outer sync task catches _ClassificationError and calls self.retry(countdown=[30,90,270])
- MaxRetriesExceededError caught in nested try/except to avoid re-raise from exc block
- Promote test_retry_backoff and test_exhaustion_sets_failed_status from xfail to passing
- Tests use push_request(retries=N) + patch.object(task, "retry") to verify countdown values
- test_exhaustion_sets_failed_status uses assert_called_once_with (asyncio.run calling convention)
2026-06-04 23:08:16 +02:00
curo1305andClaude Sonnet 4.6 a548266461 refactor(backend): extract shared helper modules per architecture rules
- Add backend/ai/utils.py — parse_classification, parse_suggestions, strip_code_fences
  shared by all AI providers; removes duplicated private functions from
  anthropic_provider.py and openai_provider.py
- Add backend/deps/utils.py — get_client_ip, parse_uuid request-parsing helpers;
  removes local _ip() variants from admin.py, auth.py, shares.py, folders.py
- Add backend/storage/exceptions.py — canonical CloudConnectionError definition;
  all routers and backends import from here instead of redefining
- Move validate_password_strength to backend/services/auth.py; removes duplicated
  _validate_password_strength from admin.py and auth.py

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-02 16:10:35 +02:00
curo1305 6d094d17f0 feat(05-09): PATCH /documents/{id} endpoint + cloud-aware Celery re-analyze
- Add DocumentPatch Pydantic model with filename and folder_id optional fields
- Add PATCH /api/documents/{doc_id} endpoint: ownership guard, model_fields_set
  to distinguish absent vs null folder_id, returns updated metadata dict
- Update _run() in document_tasks.py to use get_storage_backend_for_document
  for non-MinIO backends instead of hardcoded MinIO path
- CloudConnectionError caught in cloud path: returns extract_failed status
- Update test to use pure unit mocks (no PostgreSQL) for _run() cloud routing
- All 3 plan tests pass; 23 test_cloud.py tests pass
2026-05-30 11:16:01 +02:00
curo1305 f89f787656 feat(phase-4-06): Celery daily audit export task + beat schedule (D-17)
- Create backend/tasks/audit_tasks.py: audit_log_daily_export task
  queries yesterday's AuditLog rows, writes CSV, uploads to MinIO
  audit-logs bucket via put_object_raw(bucket='audit-logs', ...)
- All imports deferred inside _run_daily_export() to prevent circular imports
- celery_app.py: add crontab import, beat entry at midnight UTC,
  tasks.audit_tasks.* routed to documents queue
2026-05-25 18:50:50 +02:00
curo1305 6849ebd1e6 feat(03-04): retire flat-file settings; wire per-user AI config via DB lookup
- config.py: Remove SETTINGS_FILE, DEFAULT_SYSTEM_PROMPT, DEFAULT_SETTINGS
  constants; add system_prompt, default_ai_provider, default_ai_model to Settings
- services/classifier.py: Add _DEFAULT_SYSTEM_PROMPT module constant; classify_document
  and suggest_topics_for_document accept ai_provider/ai_model kwargs; no longer calls
  storage.load_settings() — uses app_settings defaults with DB-supplied overrides (D-14, D-15)
- services/storage.py: Delete load_settings, save_settings, mask_api_key, settings_masked;
  remove from __all__; remove import copy, json, DEFAULT_SETTINGS, SETTINGS_FILE (D-12)
- tasks/document_tasks.py: _run resolves user.ai_provider/ai_model via session.get(User,
  doc.user_id) and passes through to classifier; task signature unchanged (T-03-19)
- api/settings.py: Deleted — /api/settings endpoint removed (D-12)
- main.py: Remove settings_router import and include_router call
- tests/test_settings.py: Replace all tests with test_settings_endpoint_removed (404, green)
- tests/test_classifier.py: Implement test_per_user_provider, test_celery_task_uses_user_provider,
  test_default_provider_fallback; remove xfail markers (DOC-03, DOC-05)
2026-05-23 20:32:55 +02:00
curo1305 0d51d023ce feat(03-02): implement presigned upload flow, quota enforcement, cleanup task
- Replace POST /api/documents/upload with POST /api/documents/upload-url + /{id}/confirm
- upload-url: create pending Document row with user_id=None (Wave 2), return presigned PUT URL
- confirm: stat MinIO for authoritative size (T-03-05), atomic quota UPDATE (T-03-06, STORE-03)
- Confirm returns 413 with {used_bytes, limit_bytes, rejected_bytes} on quota exceeded (STORE-05)
- Wave 2 guard: skip quota UPDATE when doc.user_id is None (Plan 03-03 removes this)
- Add GET /api/auth/me/quota to api/auth.py (STORE-04)
- services/storage.py: remove save_upload (D-04); add GREATEST(0, used_bytes-delta) quota decrement to delete_document (STORE-06)
- tasks/document_tasks.py: add cleanup_abandoned_uploads Celery beat task (D-06)
- celery_app.py: add beat_schedule for cleanup-abandoned-uploads every 30 minutes
- tests/test_documents.py: replace legacy /upload tests with xfail; add real test logic for upload-url/confirm/get-quota
- tests/test_quota.py: implement real test logic with xfail for PostgreSQL-specific SQL
2026-05-23 14:32:12 +02:00
curo1305 9fc820d893 feat(02-01): implement services/auth.py full auth service layer and email_tasks.py
- services/auth.py: Argon2 password hashing (pwdlib), constant-time verify (SEC-06)
- JWT create/decode for access tokens and password-reset tokens (typ claim validation, T-02-01)
- Refresh token lifecycle: create, rotate, revoke-all with family revocation (AUTH-07, RFC 9700)
- Family revocation enqueues send_security_alert_email.delay on token reuse (T-02-02)
- TOTP provisioning (pyotp) and verification with Redis replay prevention, valid_window=1 (AUTH-08)
- Backup code generation (8-char hex uppercase), storage (Argon2 hashed), constant-time verify (T-02-03)
- HIBP k-anonymity check via SHA-1 prefix (T-02-05), fail-open on network error (T-02-06)
- Admin bootstrap: idempotent, logs WARNING if env vars missing (D-04/D-05/D-06)
- services/email.py: SMTP send + dev stdout fallback (D-01/D-02)
- tasks/email_tasks.py: send_reset_email and send_security_alert_email Celery tasks
- celery_app.py: add email queue route for tasks.email_tasks.*
- TDD tests: 17 tests covering all auth primitives and family revocation
2026-05-22 19:23:42 +02:00
curo1305 32d67de1ca feat(01-05): introduce celery_app + tasks/document_tasks + session-aware classifier
- Add backend/celery_app.py: Celery("docuvault") with Redis broker, JSON
  serialization, and tasks.document_tasks.* routed to documents queue;
  reads REDIS_URL directly from os.environ (no config import — Pitfall 7)
- Add backend/tasks/__init__.py: empty package marker
- Add backend/tasks/document_tasks.py: sync extract_and_classify Celery task
  that calls asyncio.run(_run()) to retrieve bytes from MinIO, extract text
  via extractor, and classify via classifier; classification failure is non-fatal
- Update backend/services/classifier.py: classify_document and
  suggest_topics_for_document now accept session: AsyncSession as first arg;
  all storage.* calls updated to async session-injection pattern
- Add extract_text_from_bytes helper to services/extractor.py for bytes-based
  extraction (used by Celery worker, which retrieves bytes from MinIO)
2026-05-22 09:45:33 +02:00