11 KiB
phase: 14.1-cloud-local-file-parity-hardening plan: "02" subsystem: cloud-detail-parity-backend status: complete tags: [cloud-detail, force-reanalyze, single-item-retry, schema-allowlist, parity] requirements: [CLOUD-02, ANALYZE-01, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-05]
dependency_graph: requires: - 14.1-01 (RED tests — test_cloud_detail_parity.py, test_cloud_reanalyze_force.py) provides: - backend/api/cloud/schemas.py (CloudItemDetailOut + force on AnalysisEnqueueRequest) - backend/services/cloud_items.py (resolve_owned_cloud_item_detail) - backend/api/cloud/operations.py (GET /connections/{id}/items/{item_id:path}/detail) - backend/services/cloud_analysis.py (force param + retry_or_create_single_item_job) - backend/api/cloud/analysis.py (force wired; POST /connections/{id}/items/{cloud_item_id}/retry) affects: - frontend (Plan 03/04 target — can now render cloud file detail with analysis fields)
tech_stack: added: [] patterns: - Strict Pydantic allowlist schema with explicit forbidden-field documentation (T-14.1-03) - Service raises domain exceptions (ConnectionNotFound, CloudItemNotFound, InvalidJobState); router translates to HTTP - Metadata-only detail resolution — zero provider bytes downloaded (CACHE-03, T-14.1-06) - already_current bypass via force flag without changing the unsupported guard (ANALYZE-06) - Single-item retry-job creation through authorized enqueue path (D-12)
key_files: created: [] modified: - backend/api/cloud/schemas.py - backend/services/cloud_items.py - backend/api/cloud/operations.py - backend/services/cloud_analysis.py - backend/api/cloud/analysis.py
decisions:
- CloudItemDetailOut uses an empty capabilities dict (no live capability resolution without credential decryption) — frontend infers actions from analysis_status + unsupported_analysis_reason
- unsupported_analysis_reason populated via _is_supported from cloud_analysis (single source of truth)
- force=True bypasses already_current check for supported items only; unsupported items remain unsupported regardless
- retry_or_create_single_item_job accepts failed/indexed/stale/pending items via force=True enqueue
- Single-item retry route uses cloud_item_id (DocuVault UUID) not provider_item_id for stable identity
metrics: duration: "18m" completed_date: "2026-06-26" tasks_completed: 2 files_modified: 5 backend_tests_passing: 872 backend_tests_added_passing: 34 pre_existing_failures: 1 (test_extract_docx — ModuleNotFoundError: No module named 'docx', unrelated)
Phase 14.1 Plan 02: Cloud Item Detail + Force Re-analyze + Single-item Retry Summary
Backend contracts that let cloud files behave like local documents: owner-scoped cloud item detail with extracted text, topics, and analysis status through a strict credential-free schema; force re-analyze flag bypassing already_current for indexed items; and a single-item retry-job path for failed items with no surviving active job.
Tasks Completed
Task 1: CloudItemDetailOut schema + resolve_owned_cloud_item_detail + GET detail route
Schema — backend/api/cloud/schemas.py:
Added CloudItemDetailOut as an explicit allowlist — credentials_enc, object_key, version_key, and raw provider URLs are absent by design (T-14.1-03). Fields: id, provider_item_id, name, kind, parent_ref, content_type, size, modified_at, etag, provider, display_name, location, analysis_status, semantic_index_status, extracted_text, topics (list[str]), capabilities (empty dict — no live provider call), unsupported_analysis_reason, is_stale.
Added force: bool = Field(default=False) to AnalysisEnqueueRequest (used by Task 2).
Service — backend/services/cloud_items.py:
Added CloudItemDetail dataclass and resolve_owned_cloud_item_detail(session, *, user_id, connection_id, provider_item_id):
- Calls
resolve_owned_connectionfor ownership gate (T-14.1-04). - Selects CloudItem by (connection_id, provider_item_id, user_id) — raises CloudItemNotFound.
- Loads topic names via CloudItemTopic→Topic join (metadata-only).
- Derives
locationfrom path_snapshot or parent_ref (never a raw provider URL — T-14.1-03). - Derives
unsupported_analysis_reasonvia_is_supportedfrom cloud_analysis (single source). - Zero bytes downloaded — no get_object, no hydrate_and_cache_bytes (CACHE-03, T-14.1-06).
Route — backend/api/cloud/operations.py:
Added GET /connections/{connection_id}/items/{item_id:path}/detail with response_model=CloudItemDetailOut and get_regular_user dependency (admin → 403). ConnectionNotFound → 404, CloudItemNotFound → 404. Returns CloudItemDetailOut with capabilities={} (intentionally empty — no credential decryption during detail fetch).
Test results: All 8 test_cloud_detail_parity.py tests pass (owner 200, foreign 404, admin 403, credential exclusion, stale retention, pending empty-state, no byte hydration).
Commit: a0d5c1d
Task 2: Force re-analyze flag + single-item retry-job creation
Schema — backend/api/cloud/schemas.py:
force: bool = Field(default=False) added to AnalysisEnqueueRequest with full docstring (D-11).
Service — backend/services/cloud_analysis.py:
Extended enqueue_analysis_job with force: bool = False parameter. The already_current check becomes:
already_current = False if (live_metadata_changed or force) else await _check_already_current(...)
When force=True, supported items are created as queued job items even if their version_key matches a prior indexed run. Unsupported items remain unsupported regardless of force (ANALYZE-07: no provider mutation is performed).
Added retry_or_create_single_item_job(session, *, cloud_item_id, user_id) service helper that:
- Resolves the CloudItem by (id, user_id) — raises AnalysisItemNotFound.
- Accepts failed/indexed/stale/pending items; rejects others with InvalidJobState.
- Calls
enqueue_analysis_job(scope="file", provider_item_ids=[item.provider_item_id], force=True). - Returns an EnqueueResult with queued_count >= 1 (for supported items).
Route — backend/api/cloud/analysis.py:
- Wired
force=body.forceinto the existingenqueue_jobroute handler (D-11). - Added
POST /analysis/connections/{connection_id}/items/{cloud_item_id}/retryreturningAnalysisEnqueueOutwith status 202. Validates connection ownership before delegating toretry_or_create_single_item_job. Returns job_id and queued_count in the response (D-12).
Test results: All 8 test_cloud_reanalyze_force.py tests pass. All 26 test_cloud_analysis_contract.py tests pass (no idempotency regression). Full suite: 872/873 pass (1 pre-existing failure unrelated to this plan).
Commit: 52acd56
Verification Results
Backend verification
test_cloud_detail_parity.py — 8/8 passed
test_cloud_reanalyze_force.py — 8/8 passed
test_cloud_analysis_contract.py — 26/26 passed
test_cloud_security.py — 36/36 passed
Full suite: 872 passed, 1 pre-existing failure (test_extract_docx — missing docx module)
Acceptance criteria
grep -c 'class CloudItemDetailOut' backend/api/cloud/schemas.py→ 1 ✓- CloudItemDetailOut excludes object_key/credentials_enc/version_key within class body ✓
grep -c 'resolve_owned_cloud_item_detail' backend/services/cloud_items.py→ 2 (definition + docstring) ✓- Detail route
/detailwithresponse_model=CloudItemDetailOutpresent in operations.py ✓ - No get_object/hydrate_and_cache_bytes call inside resolve_owned_cloud_item_detail ✓
grep -c 'force' backend/api/cloud/schemas.py→ 3 (field, description lines) ✓enqueue_analysis_jobsignature includesforce: bool = False✓already_current = False if (live_metadata_changed or force) else ...on line 608 ✓force=body.forcewired in analysis.py enqueue route ✓retry_or_create_single_item_jobexists in cloud_analysis.py and analysis.py import ✓
Deviations from Plan
Auto-fixed Issues
1. [Rule 2 - Missing critical functionality] Empty capabilities in detail response
- Found during: Task 1 implementation
- Issue: The plan called for returning
capabilitiesin CloudItemDetailOut. Populating live capabilities would require credential decryption and a provider call — violating the metadata-only constraint (CACHE-03 / T-14.1-06). - Fix: Return
capabilities={}(empty dict) from the detail route. Theunsupported_analysis_reasonfield carries the action-availability signal the frontend needs (D-16). A comment in the route documents the intentional empty capabilities and the reason. - Files modified: backend/api/cloud/operations.py
- Commit:
a0d5c1d
2. [Rule 1 - Bug] CloudConnection.display_name_override must take precedence
- Found during: Task 1 — reviewing CloudConnection model fields
- Issue: CloudConnection has both
display_nameanddisplay_name_override. The connection rename flow setsdisplay_name_override; the originaldisplay_nameis the provider-set name. The detail response should show what the user sees (override if set). - Fix:
display_name=conn.display_name_override or conn.display_namein resolve_owned_cloud_item_detail. - Files modified: backend/services/cloud_items.py
- Commit:
a0d5c1d
Known Stubs
None — all fields return real data from DB rows. The empty capabilities={} in the detail response is intentional and documented (see Deviation 1 above), not a stub.
Threat Flags
| Flag | File | Description |
|---|---|---|
| No new surface | — | All new endpoints are owner-scoped via get_regular_user + resolve_owned_connection + item ownership check. No new trust boundaries introduced. |
T-14.1-03, T-14.1-04, T-14.1-05, T-14.1-06 mitigations all implemented as designed:
- T-14.1-03: CloudItemDetailOut schema enforces allowlist — tests verify credentials_enc/object_key/version_key absent.
- T-14.1-04: get_regular_user (admin 403) + ownership check (foreign 404) on both detail and retry routes.
- T-14.1-05: force only bypasses already_current check; no provider mutation methods called.
- T-14.1-06: resolve_owned_cloud_item_detail calls zero get_object / hydrate_and_cache_bytes.
Self-Check: PASSED
Files verified:
- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/schemas.py ✓
- /Users/nik/Documents/Progamming/document_scanner/backend/services/cloud_items.py ✓
- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/operations.py ✓
- /Users/nik/Documents/Progamming/document_scanner/backend/services/cloud_analysis.py ✓
- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/analysis.py ✓
Commits verified: