kite/.planning/phases/14-selective-analysis-and-byte-cache/14-06-SUMMARY.md

---

phase: "14" plan: "06" subsystem: cloud-byte-cache tags: [cache, preview, download, security, pin, hydration, minio] status: complete

requires:

• phases/14-selective-analysis-and-byte-cache/14-03
• phases/14-selective-analysis-and-byte-cache/14-05

provides:

• services/cloud_cache.py: hydrate_and_cache_bytes — full cache lifecycle for open/preview/download: cache-hit avoids provider fetch, cache-miss stores bytes in MinIO and pins entry, pin always released in finally block (PATTERNS.md steps 3, 7, 8)
• api/cloud/operations.py: _make_minio_cache_wrapper — shared MinIO adapter for cache operations (DRY, used by both preview and download endpoints)
• api/cloud/operations.py: preview_cloud_file and download_cloud_file now route through hydrate_and_cache_bytes; version_key computed from CloudItem metadata, no provider call required for cache key resolution

affects:

• backend/services/cloud_cache.py
• backend/api/cloud/operations.py
• backend/tests/test_cloud_cache.py
• backend/tests/test_cloud_security.py
• frontend/src/views/tests/CloudFolderOpenPreview.test.js

tech-stack: added: [] patterns: - cache-hit-pin-read-release: hydrate_and_cache_bytes pins before MinIO read, releases in finally — eviction cannot occur mid-response (PATTERNS.md steps 7, 8) - cache-miss-fetch-store-pin-release: provider fetch → MinIO put → retain_or_reuse → pin → release; all steps non-fatal (bytes returned even on MinIO/quota failure) - quota-bypass-on-failure: CacheQuotaExceeded causes MinIO object deletion and immediate return of provider bytes without caching — quota is never left inconsistent - version-key-from-metadata: compute_version_key called with CloudItem.version/etag/ provider_size/modified_at — no provider round-trip needed for key resolution - shared-minio-wrapper: _make_minio_cache_wrapper in operations.py eliminates duplicate adapter code at two call sites (DRY) - cache-transparent-frontend: response shapes from preview and download routes unchanged — frontend never sees object_key, MinIO paths, or cache metadata

key-files: created: [] modified: - backend/services/cloud_cache.py - backend/api/cloud/operations.py - backend/tests/test_cloud_cache.py - backend/tests/test_cloud_security.py - frontend/src/views/tests/CloudFolderOpenPreview.test.js

decisions:

• hydrate_and_cache_bytes lives in services/cloud_cache.py (service layer), not in the router — CLAUDE.md service-layer rule; router only calls the helper
• MinIO write failure and quota failure are non-fatal — provider bytes returned without caching; caller experience is identical (just no cache benefit on miss)
• Cache-hit MinIO read failure falls through to provider fetch — robust degradation so a corrupted MinIO entry does not break user access to the file
• Pin is acquired before MinIO read on cache hit so eviction cannot occur mid-response
• evict_lru_entries called after pin release on cache-miss path — post-hydration LRU maintenance stays within user's configured cache limit
• Frontend tests required no response-shape changes — all 8 pre-existing tests pass unchanged; 1 new test added to document T-14-02 cache transparency invariant
• _make_minio_cache_wrapper helper avoids inline class duplication at preview/download

metrics: duration: "~10 minutes" completed: "2026-06-23" tasks: 2 files: 5

Phase 14 Plan 06: Cache-Backed Open/Preview/Download Summary

One-liner: Preview and download routes now hydrate provider bytes through the Phase 14 durable byte cache — cache hits avoid provider round-trips, active entries are pinned during response streaming, and pins are always released in finally blocks.

What Was Built

Task 1: Backend open/preview/download routes use cache leases

services/cloud_cache.py — hydrate_and_cache_bytes (new, ~140 lines):

The canonical cache lifecycle helper for content routes (PATTERNS.md steps 3, 7, 8).

Cache-hit path:

1. Query CloudByteCacheEntry for a non-evicted matching row by (user_id, connection_id, cloud_item_id, version_key).
2. Pin the entry before reading (pin_count += 1) so LRU eviction cannot occur mid-response (T-14-12 equivalent for preview/download).
3. Touch last_accessed_at to update LRU position.
4. Read bytes from MinIO via the private object_key.
5. Release pin in a finally block on all exit paths.
6. Return (bytes, cache_entry_id).

Cache-miss path:

1. Call fetch_fn() to download bytes from the provider.
2. Write bytes to MinIO under a private UUID key.
3. Attempt atomic quota increment via increment_quota_for_cache.
4. If quota exceeded: delete the MinIO object, return provider bytes without caching.
5. Create or reactivate the cache entry via retain_or_reuse_cache_entry.
6. Pin the entry for the response duration.
7. Release pin in a finally block.
8. Run evict_lru_entries to stay within the user's configured byte limit.
9. Return (bytes, cache_entry_id).

Non-fatal failure policy: MinIO write failure, quota exhaustion, and any other non-provider error return the bytes from the provider fetch path without bubbling up an exception. The caller always gets bytes.

---

api/cloud/operations.py — three changes:

1. _make_minio_cache_wrapper(backend) helper (new): Returns a _MinIOCacheWrapper instance adapting MinIOBackend to the hydrate_and_cache_bytes interface (get_object, put_object, delete_object). Shared by both preview_cloud_file and download_cloud_file (DRY).

2. preview_cloud_file updated (Steps 4-5 refactored):

   • Step 4 now resolves the adapter only (no get_object call inline).
   • Step 5 computes version_key from CloudItem.version/etag/provider_size/modified_at — no provider call needed.
   • Reads user's cloud_cache_limit_bytes from UserAnalysisSettings (default 512 MB).
   • Calls hydrate_and_cache_bytes; falls through to direct adapter.get_object() if no CloudItem row found.
   • Binary-only preview enforcement (D-18) unchanged — PreviewSupport.is_supported() still gates before the adapter is resolved.

3. download_cloud_file updated:

   • Same pattern: version_key from metadata, hydrate through cache, fallback to direct provider fetch for items without metadata rows.
   • Error handling preserved — _normalize_error on adapter still surfaces reauth_required as 401.

New tests (Task 1):

tests/test_cloud_cache.py — 5 new tests (prefixed test_hydrate_*):

• test_hydrate_cache_miss_fetches_from_provider: cache miss calls fetch_fn once, stores in MinIO, creates cache entry
• test_hydrate_cache_hit_skips_provider: cache hit reads from MinIO, fetch_fn not called (T-14-03 equivalent)
• test_hydrate_pin_released_after_cache_hit: pin_count == 0 after successful hit (PATTERNS.md step 8)
• test_hydrate_cache_miss_pin_released: no pin leak after miss path completes
• test_hydrate_foreign_user_gets_separate_cache_entry: foreign user cannot reuse owner's cache entry (T-14-06)

tests/test_cloud_security.py — 3 new tests:

• test_preview_cache_hit_response_excludes_object_key: response body/headers contain no object_key or credentials_enc (T-14-02)
• test_download_response_excludes_object_key: download headers contain no cache internals (T-14-02)
• test_foreign_user_cannot_download_via_cache: IDOR — ownership check runs before cache lookup (T-13-01)

Task 2: Frontend tests preserve open/preview behavior

No response schema changes were required — the byte cache is entirely backend-transparent. The existing 8 frontend tests in CloudFolderOpenPreview.test.js pass unchanged.

New test added (1):

• test_hydrate_cache_hit_skips_provider in cache_backed_response_has_no_object_key describe block: verifies that the openCloudFile response processed by CloudFolderView never exposes object_key, MinIO cache paths, or credentials_enc in rendered HTML or API call arguments (T-14-02 frontend invariant).

Test Results

File	Tests	Pass	Notes
test_cloud_cache.py	30 → 35	35	5 new hydrate tests
test_cloud_security.py -k cache/preview/download	10 → 13	13	3 new Plan 06 security tests
CloudFolderOpenPreview.test.js	8 → 9	9	1 new cache transparency test
Overall backend (excl. test_extractor.py docx env)	693	693	No regressions

Pre-existing failures (not Plan 06 scope):

• test_extractor.py::test_extract_docx: pre-existing docx env issue (ModuleNotFoundError)

Deviations from Plan

Auto-fixed Issues

1. [Rule 1 - Bug] CacheQuotaExceeded with no Quota row in test SQLite DB

• Found during: Task 1 test authoring
• Issue: hydrate_and_cache_bytes calls increment_quota_for_cache which uses a raw SQL UPDATE quotas SET... — but test fixtures use random UUIDs without creating User+Quota rows. SQLite returns 0 rows → CacheQuotaExceeded raised → cache entry never created → assertion len(entries) == 1 fails.
• Fix: Patched services.cloud_cache.increment_quota_for_cache with AsyncMock() in the three tests that need cache entry creation (test_hydrate_cache_miss_fetches_from_provider, test_hydrate_cache_miss_pin_released, test_hydrate_foreign_user_gets_separate_cache_entry). The quota contract is already tested by test_cache_retain_increments_quota and test_cache_eviction_decrements_quota in the existing RED suite.
• Files modified: backend/tests/test_cloud_cache.py
• Commit: dfc3350

2. [Rule 2 - Missing] Shared MinIO wrapper to avoid code duplication

• Found during: Task 1 implementation (both preview and download needed identical wrapper)
• Issue: Both preview_cloud_file and download_cloud_file would require identical inline _MinIOWrapper class definitions.
• Fix: Extracted _make_minio_cache_wrapper(backend) helper function at the top of operations.py, used at both call sites.
• Files modified: backend/api/cloud/operations.py
• Commit: dfc3350

Known Stubs

None. The byte cache integration is complete for the preview and download routes. Caching is non-fatal on MinIO failure or quota exhaustion — the routes always return provider bytes as a fallback.

Threat Flags

None new. The plan's mitigations are fully implemented:

Threat	Mitigation	Status
T-14-02 object_key secrecy	object_key never in any response body/header	Verified by test
T-13-01 IDOR	ownership check (resolve_owned_connection) runs before cache lookup	Verified by test
T-14-12 cache pin leak	finally block releases pin on all exit paths (success, failure, MinIO error)	Implemented + tested
Provider URL exposure	No provider URL in open/preview/download response — DocuVault URLs only	Verified by rg check

Self-Check: PASSED