docs(14.1-02): complete cloud-detail-parity-backend plan

.planning/ROADMAP.md

@@ -26,7 +26,7 @@ Before any phase is marked complete:
 | 12 | 6/6 | Complete   | 2026-06-21 |
 | 13 | 11/11 | Complete   | 2026-06-23 |
 | 14 | 9/9 | Complete   | 2026-06-23 |
-| 14.1 | 1/5 | In Progress|  |
+| 14.1 | 2/5 | In Progress|  |
 | 14.2 | Cross-Codebase Review and Cleanup | Cross-reference backend/frontend code, remove duplication and dead code, consolidate shared paths, and fix inefficiencies without behavior changes | Cross-cutting quality gates |
 | 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
 | 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
@@ -141,13 +141,13 @@ Plans:
 **Goal:** Cloud files opened, viewed, downloaded, analyzed, and displayed in results use the same user-facing behavior as local files, while preserving provider ownership, cache boundaries, authorization, and no-provider-mutation guarantees.
 **Requirements:** CLOUD-02, ANALYZE-01, ANALYZE-02, ANALYZE-03, ANALYZE-04, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-04, CACHE-05
 **Depends on:** Phase 14
-**Plans:** 1/5 plans executed
+**Plans:** 2/5 plans executed
 **Execution waves:** Wave 1: 14.1-01 (RED tests); Wave 2: 14.1-02 after 01 (backend detail/force/retry); Wave 3: 14.1-03 after 01-02 (shared detail surface + cloud route/view); Wave 4: 14.1-04 after 01-03 (browser row parity + cloud row navigation); Wave 5: 14.1-05 after 01-04 (gates, security, docs, version, commit).
 
 Plans:
 
 - [x] 14.1-01-PLAN.md — RED backend + frontend parity tests (cloud detail endpoint, force re-analyze, single-item retry, route + row parity).
-- [ ] 14.1-02-PLAN.md — Backend cloud detail endpoint (CloudItemDetailOut + resolve_owned_cloud_item_detail), force re-analyze flag, single-item retry-job creation.
+- [x] 14.1-02-PLAN.md — Backend cloud detail endpoint (CloudItemDetailOut + resolve_owned_cloud_item_detail), force re-analyze flag, single-item retry-job creation.
 - [ ] 14.1-03-PLAN.md — Shared DocumentDetailSurface extraction, DocumentView refactor (Re-analyze copy), CloudDetailView + cloud-file-detail route + getCloudItemDetail.
 - [ ] 14.1-04-PLAN.md — Browse row topics/analysis_status/stale, StorageBrowser row parity via store translator, cloud row click → cloud detail route (no auto preview/download).
 - [ ] 14.1-05-PLAN.md — Full test suites, security gate, dependency/secret audits, CLAUDE.md/README updates, version bump, atomic commit.

.planning/STATE.md

@@ -6,14 +6,14 @@ current_phase: 14.1
 current_phase_name: cloud-local-file-parity-hardening
 status: executing
 stopped_at: Phase 14.1 UI-SPEC approved
-last_updated: "2026-06-26T19:49:54.552Z"
+last_updated: "2026-06-26T20:03:08.999Z"
 last_activity: 2026-06-26
 last_activity_desc: Phase 14.1 execution started
 progress:
   total_phases: 8
   completed_phases: 4
   total_plans: 35
-  completed_plans: 31
+  completed_plans: 32
   percent: 50
 ---
 
@@ -26,7 +26,7 @@ progress:
 ## Current Position
 
 Phase: 14.1 (cloud-local-file-parity-hardening) — EXECUTING
-Plan: 2 of 5
+Plan: 3 of 5
 Status: Ready to execute
 Last activity: 2026-06-26 — Phase 14.1 execution started
 
@@ -74,6 +74,7 @@ Last activity: 2026-06-26 — Phase 14.1 execution started
 | Phase 14 P08 | 10m | 2 tasks | 3 files |
 | Phase 14 P09 | 11m | 2 tasks | 9 files |
 | Phase 14.1 P01 | 13m | 2 tasks | 4 files |
+| Phase 14.1 P02 | 18m | 2 tasks | 5 files |
 
 ## Accumulated Context
 
@@ -96,6 +97,9 @@ Last activity: 2026-06-26 — Phase 14.1 execution started
 | Phase 13 cloud_operations.py orchestration | All mutation logic routes through services/cloud_operations.py — never inline in routers |
 | testCloudConnection explicit-only | Never called as navigation side effect (D-13); only from user action or post-failure retest |
 | v0.3.0 version bump | Phase 13 completion warrants minor version bump per CLAUDE.md versioning protocol |
+| CloudItemDetailOut empty capabilities | Live capability resolution requires credential decryption — violates CACHE-03 metadata-only constraint; frontend infers actions from analysis_status + unsupported_analysis_reason |
+| force=true bypasses already_current for supported items | Unsupported items remain unsupported regardless of force (ANALYZE-06, ANALYZE-07) |
+| Single-item retry uses cloud_item_id | DocuVault UUID is stable across provider rename/move; provider_item_id could change (D-12) |
 
 ### Roadmap Evolution
 
@@ -117,13 +121,13 @@ None.
 
 ## Session Continuity
 
-**Stopped at:** Phase 14.1 UI-SPEC approved
+**Stopped at:** Phase 14.1 Plan 02 complete — awaiting Plan 03 (frontend cloud detail view)
 
 _Updated at each phase transition._
 
 | Field | Value |
 |---|---|
-| Last session | 2026-06-26T19:49:54.547Z |
+| Last session | 2026-06-26T20:03:08.994Z |
 | Next action | Plan 14.1 |
 | Pending decisions | None |
 | Resume file | .planning/phases/14.1-cloud-local-file-parity-hardening/14.1-UI-SPEC.md |

.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-02-SUMMARY.md

@@ -0,0 +1,186 @@
+---
+phase: 14.1-cloud-local-file-parity-hardening
+plan: "02"
+subsystem: cloud-detail-parity-backend
+status: complete
+tags: [cloud-detail, force-reanalyze, single-item-retry, schema-allowlist, parity]
+requirements: [CLOUD-02, ANALYZE-01, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-05]
+
+dependency_graph:
+  requires:
+    - 14.1-01 (RED tests — test_cloud_detail_parity.py, test_cloud_reanalyze_force.py)
+  provides:
+    - backend/api/cloud/schemas.py (CloudItemDetailOut + force on AnalysisEnqueueRequest)
+    - backend/services/cloud_items.py (resolve_owned_cloud_item_detail)
+    - backend/api/cloud/operations.py (GET /connections/{id}/items/{item_id:path}/detail)
+    - backend/services/cloud_analysis.py (force param + retry_or_create_single_item_job)
+    - backend/api/cloud/analysis.py (force wired; POST /connections/{id}/items/{cloud_item_id}/retry)
+  affects:
+    - frontend (Plan 03/04 target — can now render cloud file detail with analysis fields)
+
+tech_stack:
+  added: []
+  patterns:
+    - Strict Pydantic allowlist schema with explicit forbidden-field documentation (T-14.1-03)
+    - Service raises domain exceptions (ConnectionNotFound, CloudItemNotFound, InvalidJobState); router translates to HTTP
+    - Metadata-only detail resolution — zero provider bytes downloaded (CACHE-03, T-14.1-06)
+    - already_current bypass via force flag without changing the unsupported guard (ANALYZE-06)
+    - Single-item retry-job creation through authorized enqueue path (D-12)
+
+key_files:
+  created: []
+  modified:
+    - backend/api/cloud/schemas.py
+    - backend/services/cloud_items.py
+    - backend/api/cloud/operations.py
+    - backend/services/cloud_analysis.py
+    - backend/api/cloud/analysis.py
+
+decisions:
+  - CloudItemDetailOut uses an empty capabilities dict (no live capability resolution without credential decryption) — frontend infers actions from analysis_status + unsupported_analysis_reason
+  - unsupported_analysis_reason populated via _is_supported from cloud_analysis (single source of truth)
+  - force=True bypasses already_current check for supported items only; unsupported items remain unsupported regardless
+  - retry_or_create_single_item_job accepts failed/indexed/stale/pending items via force=True enqueue
+  - Single-item retry route uses cloud_item_id (DocuVault UUID) not provider_item_id for stable identity
+
+metrics:
+  duration: "18m"
+  completed_date: "2026-06-26"
+  tasks_completed: 2
+  files_modified: 5
+  backend_tests_passing: 872
+  backend_tests_added_passing: 34
+  pre_existing_failures: 1 (test_extract_docx — ModuleNotFoundError: No module named 'docx', unrelated)
+---
+
+# Phase 14.1 Plan 02: Cloud Item Detail + Force Re-analyze + Single-item Retry Summary
+
+Backend contracts that let cloud files behave like local documents: owner-scoped cloud item detail with extracted text, topics, and analysis status through a strict credential-free schema; force re-analyze flag bypassing already_current for indexed items; and a single-item retry-job path for failed items with no surviving active job.
+
+## Tasks Completed
+
+### Task 1: CloudItemDetailOut schema + resolve_owned_cloud_item_detail + GET detail route
+
+**Schema — `backend/api/cloud/schemas.py`:**
+
+Added `CloudItemDetailOut` as an explicit allowlist — credentials_enc, object_key, version_key, and raw provider URLs are absent by design (T-14.1-03). Fields: id, provider_item_id, name, kind, parent_ref, content_type, size, modified_at, etag, provider, display_name, location, analysis_status, semantic_index_status, extracted_text, topics (list[str]), capabilities (empty dict — no live provider call), unsupported_analysis_reason, is_stale.
+
+Added `force: bool = Field(default=False)` to `AnalysisEnqueueRequest` (used by Task 2).
+
+**Service — `backend/services/cloud_items.py`:**
+
+Added `CloudItemDetail` dataclass and `resolve_owned_cloud_item_detail(session, *, user_id, connection_id, provider_item_id)`:
+- Calls `resolve_owned_connection` for ownership gate (T-14.1-04).
+- Selects CloudItem by (connection_id, provider_item_id, user_id) — raises CloudItemNotFound.
+- Loads topic names via CloudItemTopic→Topic join (metadata-only).
+- Derives `location` from path_snapshot or parent_ref (never a raw provider URL — T-14.1-03).
+- Derives `unsupported_analysis_reason` via `_is_supported` from cloud_analysis (single source).
+- Zero bytes downloaded — no get_object, no hydrate_and_cache_bytes (CACHE-03, T-14.1-06).
+
+**Route — `backend/api/cloud/operations.py`:**
+
+Added `GET /connections/{connection_id}/items/{item_id:path}/detail` with `response_model=CloudItemDetailOut` and `get_regular_user` dependency (admin → 403). ConnectionNotFound → 404, CloudItemNotFound → 404. Returns CloudItemDetailOut with capabilities={} (intentionally empty — no credential decryption during detail fetch).
+
+**Test results:** All 8 test_cloud_detail_parity.py tests pass (owner 200, foreign 404, admin 403, credential exclusion, stale retention, pending empty-state, no byte hydration).
+
+**Commit:** a0d5c1d
+
+### Task 2: Force re-analyze flag + single-item retry-job creation
+
+**Schema — `backend/api/cloud/schemas.py`:**
+
+`force: bool = Field(default=False)` added to `AnalysisEnqueueRequest` with full docstring (D-11).
+
+**Service — `backend/services/cloud_analysis.py`:**
+
+Extended `enqueue_analysis_job` with `force: bool = False` parameter. The already_current check becomes:
+```python
+already_current = False if (live_metadata_changed or force) else await _check_already_current(...)
+```
+When force=True, supported items are created as queued job items even if their version_key matches a prior indexed run. Unsupported items remain unsupported regardless of force (ANALYZE-07: no provider mutation is performed).
+
+Added `retry_or_create_single_item_job(session, *, cloud_item_id, user_id)` service helper that:
+- Resolves the CloudItem by (id, user_id) — raises AnalysisItemNotFound.
+- Accepts failed/indexed/stale/pending items; rejects others with InvalidJobState.
+- Calls `enqueue_analysis_job(scope="file", provider_item_ids=[item.provider_item_id], force=True)`.
+- Returns an EnqueueResult with queued_count >= 1 (for supported items).
+
+**Route — `backend/api/cloud/analysis.py`:**
+
+- Wired `force=body.force` into the existing `enqueue_job` route handler (D-11).
+- Added `POST /analysis/connections/{connection_id}/items/{cloud_item_id}/retry` returning `AnalysisEnqueueOut` with status 202. Validates connection ownership before delegating to `retry_or_create_single_item_job`. Returns job_id and queued_count in the response (D-12).
+
+**Test results:** All 8 test_cloud_reanalyze_force.py tests pass. All 26 test_cloud_analysis_contract.py tests pass (no idempotency regression). Full suite: 872/873 pass (1 pre-existing failure unrelated to this plan).
+
+**Commit:** 52acd56
+
+## Verification Results
+
+### Backend verification
+```
+test_cloud_detail_parity.py      — 8/8 passed
+test_cloud_reanalyze_force.py    — 8/8 passed
+test_cloud_analysis_contract.py  — 26/26 passed
+test_cloud_security.py           — 36/36 passed
+Full suite: 872 passed, 1 pre-existing failure (test_extract_docx — missing docx module)
+```
+
+### Acceptance criteria
+
+- `grep -c 'class CloudItemDetailOut' backend/api/cloud/schemas.py` → 1 ✓
+- CloudItemDetailOut excludes object_key/credentials_enc/version_key within class body ✓
+- `grep -c 'resolve_owned_cloud_item_detail' backend/services/cloud_items.py` → 2 (definition + docstring) ✓
+- Detail route `/detail` with `response_model=CloudItemDetailOut` present in operations.py ✓
+- No get_object/hydrate_and_cache_bytes call inside resolve_owned_cloud_item_detail ✓
+- `grep -c 'force' backend/api/cloud/schemas.py` → 3 (field, description lines) ✓
+- `enqueue_analysis_job` signature includes `force: bool = False` ✓
+- `already_current = False if (live_metadata_changed or force) else ...` on line 608 ✓
+- `force=body.force` wired in analysis.py enqueue route ✓
+- `retry_or_create_single_item_job` exists in cloud_analysis.py and analysis.py import ✓
+
+## Deviations from Plan
+
+### Auto-fixed Issues
+
+**1. [Rule 2 - Missing critical functionality] Empty capabilities in detail response**
+- **Found during:** Task 1 implementation
+- **Issue:** The plan called for returning `capabilities` in CloudItemDetailOut. Populating live capabilities would require credential decryption and a provider call — violating the metadata-only constraint (CACHE-03 / T-14.1-06).
+- **Fix:** Return `capabilities={}` (empty dict) from the detail route. The `unsupported_analysis_reason` field carries the action-availability signal the frontend needs (D-16). A comment in the route documents the intentional empty capabilities and the reason.
+- **Files modified:** backend/api/cloud/operations.py
+- **Commit:** a0d5c1d
+
+**2. [Rule 1 - Bug] CloudConnection.display_name_override must take precedence**
+- **Found during:** Task 1 — reviewing CloudConnection model fields
+- **Issue:** CloudConnection has both `display_name` and `display_name_override`. The connection rename flow sets `display_name_override`; the original `display_name` is the provider-set name. The detail response should show what the user sees (override if set).
+- **Fix:** `display_name=conn.display_name_override or conn.display_name` in resolve_owned_cloud_item_detail.
+- **Files modified:** backend/services/cloud_items.py
+- **Commit:** a0d5c1d
+
+## Known Stubs
+
+None — all fields return real data from DB rows. The empty `capabilities={}` in the detail response is intentional and documented (see Deviation 1 above), not a stub.
+
+## Threat Flags
+
+| Flag | File | Description |
+|------|------|-------------|
+| No new surface | — | All new endpoints are owner-scoped via get_regular_user + resolve_owned_connection + item ownership check. No new trust boundaries introduced. |
+
+T-14.1-03, T-14.1-04, T-14.1-05, T-14.1-06 mitigations all implemented as designed:
+- T-14.1-03: CloudItemDetailOut schema enforces allowlist — tests verify credentials_enc/object_key/version_key absent.
+- T-14.1-04: get_regular_user (admin 403) + ownership check (foreign 404) on both detail and retry routes.
+- T-14.1-05: force only bypasses already_current check; no provider mutation methods called.
+- T-14.1-06: resolve_owned_cloud_item_detail calls zero get_object / hydrate_and_cache_bytes.
+
+## Self-Check: PASSED
+
+Files verified:
+- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/schemas.py ✓
+- /Users/nik/Documents/Progamming/document_scanner/backend/services/cloud_items.py ✓
+- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/operations.py ✓
+- /Users/nik/Documents/Progamming/document_scanner/backend/services/cloud_analysis.py ✓
+- /Users/nik/Documents/Progamming/document_scanner/backend/api/cloud/analysis.py ✓
+
+Commits verified:
+- a0d5c1d (Task 1) present in git log ✓
+- 52acd56 (Task 2) present in git log ✓