docs(14-03): complete cache service and API plan
This commit is contained in:
curo1305
@@ -0,0 +1,161 @@
|
||||
---
|
||||
phase: "14"
|
||||
plan: "03"
|
||||
subsystem: cloud-analysis
|
||||
tags: [cache, api, security, quota, pinning, settings]
|
||||
status: complete
|
||||
|
||||
requires:
|
||||
- phases/14-selective-analysis-and-byte-cache/14-02
|
||||
|
||||
provides:
|
||||
- services/cloud_cache.py: retain_or_reuse_cache_entry, pin_cache_entry,
|
||||
release_cache_entry, update_cache_access — Phase 14 cache lifecycle
|
||||
orchestration (PATTERNS.md steps 3, 7, 8)
|
||||
- api/cloud/cache.py: GET /analysis/cache, PATCH /analysis/cache/settings
|
||||
— aggregate cache usage/settings, no object_key or credentials in response
|
||||
- api/cloud/analysis.py: analysis router aggregator (parent for Phase 14
|
||||
plans 04-09)
|
||||
- api/cloud/schemas.py: CacheStatusOut, CacheSettingsUpdateRequest
|
||||
|
||||
affects:
|
||||
- backend/services/cloud_cache.py
|
||||
- backend/api/cloud/cache.py (new)
|
||||
- backend/api/cloud/analysis.py (new)
|
||||
- backend/api/cloud/schemas.py
|
||||
- backend/api/cloud/__init__.py
|
||||
- backend/tests/test_cloud_cache.py
|
||||
- backend/tests/test_cloud_security.py
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- retain-or-reactivate: unique constraint means evicted entries are
|
||||
reactivated (evicted_at cleared, object_key updated) rather than
|
||||
duplicated — semantically equivalent to "cache miss requires new bytes"
|
||||
- pin_count lifecycle: pin_cache_entry / release_cache_entry with
|
||||
GREATEST(0, ...) clamp for defensive double-release handling
|
||||
- analysis router aggregator: api/cloud/analysis.py is the future-proof
|
||||
parent for all Phase 14 analysis routes (estimates, jobs, controls)
|
||||
- CacheStatusOut allowlist: schema enforces T-14-02/T-14-08 by design —
|
||||
object_key and credentials_enc absent at type level
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- backend/api/cloud/cache.py
|
||||
- backend/api/cloud/analysis.py
|
||||
modified:
|
||||
- backend/services/cloud_cache.py
|
||||
- backend/api/cloud/schemas.py
|
||||
- backend/api/cloud/__init__.py
|
||||
- backend/tests/test_cloud_cache.py
|
||||
- backend/tests/test_cloud_security.py
|
||||
|
||||
decisions:
|
||||
- retain_or_reuse_cache_entry reactivates evicted rows (clears evicted_at,
|
||||
updates object_key) rather than inserting a new row — the unique constraint
|
||||
on (user_id, connection_id, cloud_item_id, version_key) makes insertion
|
||||
impossible; reactivation is semantically correct (fresh bytes, same version)
|
||||
- api/cloud/analysis.py created as the aggregator so tests can
|
||||
`from api.cloud.analysis import router` as required by Plan 01 RED tests
|
||||
- CacheStatusOut has no per-entry fields — only aggregate totals — ensuring
|
||||
T-14-02 compliance at schema design level rather than runtime filtering
|
||||
|
||||
metrics:
|
||||
duration: "~18 minutes"
|
||||
completed: "2026-06-23"
|
||||
tasks: 2
|
||||
files: 7
|
||||
---
|
||||
|
||||
# Phase 14 Plan 03: Cache Service and API Summary
|
||||
|
||||
One-liner: Cache orchestration functions (retain/reuse, pin, release, touch) and credential-free cache status/settings API with admin block and object_key exclusion.
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Task 1: Cache orchestration functions
|
||||
|
||||
**services/cloud_cache.py** — four new Phase 14 orchestration functions added after the Plan 02 primitives:
|
||||
|
||||
- `retain_or_reuse_cache_entry(session, *, user_id, connection_id, cloud_item_id, provider_item_id, version_key, object_key, ...)`: PATTERNS.md step 3 implementation. Checks for a non-evicted matching entry first (reuse path). If the only matching entry is evicted, reactivates it (clears evicted_at, updates object_key/size) because the unique constraint prevents inserting a duplicate. Returns `(entry, created_new)` tuple. Ownership enforced — foreign user always gets a separate row.
|
||||
|
||||
- `pin_cache_entry(session, *, entry_id, user_id)`: Increments `pin_count` to mark an active preview or download lease (PATTERNS.md step 7). Ownership asserted — raises `ValueError` for foreign entry IDs.
|
||||
|
||||
- `release_cache_entry(session, *, entry_id, user_id)`: Decrements `pin_count` with `GREATEST(0, ...)` clamp (PATTERNS.md step 8). Ownership asserted.
|
||||
|
||||
- `update_cache_access(session, *, entry_id, user_id)`: Touches `last_accessed_at` after a cache hit to maintain accurate LRU ordering. Ownership asserted.
|
||||
|
||||
All four functions raise `ValueError` only — never `HTTPException` (CLAUDE.md service-layer rule).
|
||||
|
||||
**Tests added to tests/test_cloud_cache.py** (9 new tests, Task 1):
|
||||
- retain_or_reuse returns existing active entry
|
||||
- retain_or_reuse creates new entry when absent
|
||||
- retain_or_reuse reactivates evicted entry (not duplicate)
|
||||
- retain_or_reuse isolates foreign users (T-14-06)
|
||||
- pin increments pin_count
|
||||
- release decrements pin_count
|
||||
- release clamps at zero (double-release defense)
|
||||
- pin raises ValueError on foreign entry (T-14-06)
|
||||
- update_cache_access refreshes last_accessed_at timestamp
|
||||
|
||||
### Task 2: Cache settings/status API
|
||||
|
||||
**api/cloud/cache.py** (new):
|
||||
- `GET /analysis/cache` — returns `CacheStatusOut` with aggregate entry_count, total_bytes, cache_limit_bytes, tier_cap_bytes, analysis_progress_detail, analysis_failure_behavior. Calls `get_or_create_user_analysis_settings` for defaults on first access. Rate-limited at 120/minute. Blocked for admin (get_regular_user).
|
||||
- `PATCH /analysis/cache/settings` — accepts `CacheSettingsUpdateRequest` (three explicit fields only — mass assignment prevention). Service layer validates enums and bounds; route re-raises ValueError as 422. Returns fresh CacheStatusOut. Rate-limited at 30/minute. Blocked for admin.
|
||||
|
||||
**api/cloud/analysis.py** (new):
|
||||
- Aggregator router for `/api/cloud/analysis/*` namespace.
|
||||
- Includes `cache_router` now; will include estimate/jobs/controls routers in Plans 04-09.
|
||||
- Importable as `api.cloud.analysis` so Plan 01 RED tests can import the router.
|
||||
|
||||
**api/cloud/schemas.py** (extended):
|
||||
- `CacheStatusOut`: explicit allowlist with 6 fields — no object_key, credentials_enc, or per-entry data possible.
|
||||
- `CacheSettingsUpdateRequest`: 3 explicit optional fields with `ge=1` constraint on byte limit.
|
||||
|
||||
**api/cloud/__init__.py** (updated):
|
||||
- Imports and includes `analysis_router` from `api.cloud.analysis`.
|
||||
|
||||
**Security tests added to tests/test_cloud_security.py** (7 new tests, Task 2):
|
||||
- admin blocked from GET /analysis/cache (403)
|
||||
- admin blocked from PATCH /analysis/cache/settings (403)
|
||||
- unauthenticated request blocked (401/403)
|
||||
- response excludes object_key and credentials_enc (T-14-02, T-14-08)
|
||||
- response includes expected aggregate fields
|
||||
- invalid enum value returns 422
|
||||
- cache limit above tier cap returns 422
|
||||
|
||||
## Test Results
|
||||
|
||||
| File | Tests | Pass | Notes |
|
||||
|------|-------|------|-------|
|
||||
| test_cloud_cache.py | 21 | 21 | All pass including 9 new Task 1 tests |
|
||||
| test_cloud_security.py -k cache | 8 | 8 | 7 new + 1 pre-existing |
|
||||
| Overall suite (excl. RED tests) | 808 | 808 | Pre-existing RED failures unchanged |
|
||||
|
||||
Pre-existing RED failures (not Plan 03 scope):
|
||||
- test_cloud_analysis_api.py: 10 failures — estimate/jobs/controls routes (Plans 04+)
|
||||
- test_cloud_analysis_contract.py: 14 failures — analysis contract (Plans 04+)
|
||||
- test_extractor.py::test_extract_docx: pre-existing docx env issue
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 1 - Bug] Unique constraint prevents duplicate insert for evicted entries**
|
||||
- **Found during:** Task 1 implementation
|
||||
- **Issue:** `retain_or_reuse_cache_entry` attempted to create a new row for an evicted entry. The unique constraint on `(user_id, connection_id, cloud_item_id, version_key)` prevents duplicate inserts, raising `IntegrityError`.
|
||||
- **Fix:** When an evicted entry exists for the same version_key, reactivate it (clear `evicted_at`, update `object_key` and `size_bytes`) instead of inserting. This is semantically correct: the bytes needed to be re-hydrated (created_new=True) but the DB row identity is preserved.
|
||||
- **Files modified:** `backend/services/cloud_cache.py`, `backend/tests/test_cloud_cache.py` (test updated to match corrected semantics)
|
||||
- **Commit:** 5bdd23f
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
None — no new network endpoints beyond the intended /analysis/cache routes documented in the plan. Admin block and object_key exclusion enforced at schema and dependency level.
|
||||
|
||||
## Self-Check: PASSED
|
||||
Reference in New Issue
Block a user