--- phase: "14" plan: "02" subsystem: cloud-analysis tags: [schema, migration, services, cache, versioning, settings] status: complete requires: - phases/14-selective-analysis-and-byte-cache/14-01 provides: - Alembic migration 0007: cloud_byte_cache_entries, cloud_analysis_jobs, cloud_analysis_job_items, user_analysis_settings - ORM models: CloudByteCacheEntry, CloudAnalysisJob, CloudAnalysisJobItem, UserAnalysisSettings - services/cloud_analysis_versioning.py: compute_version_key, compute_fingerprint - services/cloud_analysis_settings.py: get_or_create_user_analysis_settings, update_user_analysis_settings, build_default_settings - services/cloud_cache.py: Phase 14 durable cache functions (create_cache_entry, list_cache_entries, evict_lru_entries, increment_quota_for_cache, decrement_quota_for_cache) + re-export of compute_version_key affects: - backend/migrations/versions/0007_cloud_analysis_cache.py - backend/db/models.py - backend/services/cloud_analysis_versioning.py - backend/services/cloud_analysis_settings.py - backend/services/cloud_cache.py tech-stack: added: [] patterns: - version-key precedence (version > etag > metadata fingerprint > content hash) - LRU eviction with pin_count/active_job_count guards - atomic quota UPDATE pattern for cache retain/release - upsert-on-first-access for user settings row - re-export pattern to unify import sites for compute_version_key key-files: created: - backend/migrations/versions/0007_cloud_analysis_cache.py - backend/services/cloud_analysis_versioning.py - backend/services/cloud_analysis_settings.py modified: - backend/db/models.py - backend/services/cloud_cache.py decisions: - compute_version_key defined in cloud_analysis_versioning.py and re-exported from cloud_cache.py so tests have one canonical import site - content_hash accepted as optional post-hydration supplement — never required as a pre-download precondition (D-20) - user_analysis_settings is distinct from system_settings — single settings authority for per-user analysis preferences - evict_lru_entries stamps evicted_at only — caller is responsible for MinIO object deletion and quota decrement (separation of concerns) - increment_quota_for_cache uses atomic UPDATE…RETURNING pattern (same as upload quota enforcement) — raises CacheQuotaExceeded on overflow metrics: duration: "~12 minutes" completed: "2026-06-23" tasks: 2 files: 5 --- # Phase 14 Plan 02: Schema and Service Helpers Summary One-liner: Alembic migration 0007 adds four Phase 14 tables (byte cache, analysis jobs, job items, user settings) with ORM models, version-key precedence helper, tier-seamed settings service, and LRU eviction cache functions. ## What Was Built ### Task 1: Migration and ORM Models **migration 0007_cloud_analysis_cache.py:** - `cloud_byte_cache_entries`: owner-scoped durable byte cache backed by MinIO. Unique on (user_id, connection_id, cloud_item_id, version_key). Fields include pin_count, active_job_count, last_accessed_at, evicted_at. Two LRU eviction indexes (ix_cache_entries_user_evicted_accessed, ix_cache_entries_user_active). - `cloud_analysis_jobs`: user-visible batch rows with scope_kind (file/selection/ folder/connection), 11 aggregate item counters, provider_bytes_estimate, status, and failure_behavior. - `cloud_analysis_job_items`: per-item idempotency rows with the full status vocabulary (queued/downloading/extracting/classifying/indexed/already_current/ cancelled/failed/unsupported/stale), controlled error fields, cache_entry_id FK, fingerprint snapshot, and retry_count. Unique on (job_id, cloud_item_id). Fast duplicate-current index: (user_id, cloud_item_id, version_key). - `user_analysis_settings`: per-user preferences (analysis_progress_detail, analysis_failure_behavior, cloud_cache_limit_bytes). Separate from system_settings. - Reversible downgrade drops Phase 14 tables only in reverse-creation order. **db/models.py additions:** - CloudByteCacheEntry, CloudAnalysisJob, CloudAnalysisJobItem, UserAnalysisSettings ORM models mirror the migration exactly. ### Task 2: Service Helpers **services/cloud_analysis_versioning.py** (new): - `compute_version_key(provider_item_id, version, etag, size, modified_at, content_type, content_hash=None)`: canonical version-key helper with v:/e:/fp:/ch: prefix tags. Precedence: provider version → etag → SHA-256 metadata fingerprint → optional post-hydration content hash suffix. Deterministic and raises ValueError only. - `compute_fingerprint(...)`: public entry point for metadata-only fingerprint. - No HTTPException import or raise (CLAUDE.md service-layer rule). **services/cloud_analysis_settings.py** (new): - `get_or_create_user_analysis_settings(session, user_id)`: upsert on first access. - `update_user_analysis_settings(session, user_id, ...)`: validates enum fields (VALID_PROGRESS_DETAIL, VALID_FAILURE_BEHAVIOR) and bounds-checks cache limit against MIN_CACHE_LIMIT_BYTES and caller-supplied max_cache_limit_bytes. - `build_default_settings()`: returns defaults dict without touching the DB. - Tier/admin maximum is an explicit parameter seam (not hardcoded in the service). - No HTTPException import or raise. **services/cloud_cache.py** (extended): - Phase 14 functions added after the existing Phase 12 in-memory TTL cache code: `create_cache_entry`, `list_cache_entries` (owner-scoped, excludes evicted), `evict_lru_entries` (LRU, skips pin_count>0 or active_job_count>0), `increment_quota_for_cache` (atomic UPDATE…RETURNING, raises CacheQuotaExceeded), `decrement_quota_for_cache` (GREATEST(0, used_bytes - delta)). - Re-exports `compute_version_key` from cloud_analysis_versioning.py so test/caller import sites are unified. - Phase 12 get_cloud_folders_cached / invalidate_provider_cache unchanged. ## Test Results | File | Pass | Fail | Notes | |------|------|------|-------| | test_cloud_cache.py | 10 | 2 | 2 failures require api.cloud.analysis (future plan) | | test_cloud_analysis_contract.py | 0 | 16 | All require api.cloud.analysis (future plan) | | Overall suite (excl. Phase 14 RED) | 610+ | 1 pre-existing | test_extractor.py docx env issue — pre-existing | ## Deviations from Plan None — plan executed exactly as written. ## Known Stubs None. ## Threat Flags None — this plan adds schema, migration, and service layer only. No new network endpoints, auth paths, or file access patterns. ## Self-Check: PASSED