diff --git a/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-CONTEXT.md b/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-CONTEXT.md new file mode 100644 index 0000000..019c271 --- /dev/null +++ b/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-CONTEXT.md @@ -0,0 +1,148 @@ +# Phase 14.1: cloud-local-file-parity-hardening - Context + +**Gathered:** 2026-06-26 +**Status:** Ready for planning + + +## Phase Boundary + +Phase 14.1 hardens the already-shipped cloud file experience so cloud files opened, viewed, downloaded, analyzed, retried, re-analyzed, and displayed after analysis behave like local documents where the user-facing concepts match. Cloud files remain provider-owned `CloudItem` records, not imported local `Document` rows. All cloud bytes continue through DocuVault authorization and the existing cache lifecycle; provider URLs, credentials, cache object keys, cross-provider transfer, unified search, and broader provider change tracking remain out of scope. + + + + +## Implementation Decisions + +### Open/view/download parity +- **D-01:** Cloud file rows/cards open into a document-like detail view backed by `CloudItem`, not a local `Document` import. +- **D-02:** The cloud detail view exists before analysis. If a cloud file is not analyzed yet, the detail view offers Analyze; after analysis, the same view fills in extracted text, topics, and status. +- **D-03:** Preview and download controls in the cloud detail view mirror local document detail controls and placement, while calling authorized cloud preview/download handlers that hydrate bytes through the cache lifecycle. +- **D-04:** The cloud detail view stays visually aligned with local document detail and uses only subtle source metadata, such as provider, location, and current/stale status, to show that the file is provider-owned. + +### Analyzed metadata display +- **D-05:** Once analysis completes, cloud detail shows the same core sections as local documents: extracted text, topics, analysis status, preview/download controls, and subtle source metadata. +- **D-06:** `StorageBrowser.vue` rows/cards should match local rows/cards wherever equivalent data exists, including topic badges, analysis/current/stale state, and available actions in the same positions. +- **D-07:** Stale cloud analysis keeps prior extracted text and topics visible, marks them stale, and offers Re-analyze. +- **D-08:** Partial analysis results remain visible. For example, extracted text can display while failed classification/topics are marked clearly with a targeted Retry analysis or Re-analyze action. + +### Retry and re-analyze states +- **D-09:** Rename visible "Re-classify" copy to "Re-analyze" everywhere. Preserve existing local behavior/API unless planning finds an internal rename is necessary. +- **D-10:** Re-analyze appears consistently in local and cloud detail/row action locations when analysis exists or is stale/failed. +- **D-11:** Explicit Re-analyze on an already-current cloud file allows the user to force fresh extraction/classification after confirmation, instead of silently skipping as already-current. +- **D-12:** Failed analysis retry from rows/cards/detail uses the existing job retry semantics where possible; if no active job exists, create a single-item retry job. + +### Unsupported and provider-limited actions +- **D-13:** Unsupported cloud preview/download/analyze cases use the same shared action positions as local files, disabled or typed with clear backend-provided reasons. +- **D-14:** If a cloud file cannot be previewed in-app but can be downloaded, show Preview unavailable with the reason and keep authorized Download active. Do not auto-download from Preview. +- **D-15:** If a provider lacks reliable version/etag metadata, use the existing fallback metadata fingerprint and surface uncertainty only when it affects a user action. +- **D-16:** Unsupported analysis appears everywhere the Analyze action would appear, disabled with the same reason text in row/card/detail. + +### Parity tests +- **D-17:** Frontend parity tests use paired local/cloud assertions in shared components and detail surfaces, checking equivalent controls, labels, topic/status placement, and disabled states. +- **D-18:** Backend Phase 14.1 tests explicitly cover cache/auth boundaries: no provider URL, credentials, or `object_key` leakage; owner/admin negatives; preview/download through cache hydration; forced Re-analyze through the authorized analysis path; and no provider mutation. +- **D-19:** Tests require route-level parity: a cloud detail route opens from browser rows and renders the same core sections/actions as local detail. +- **D-20:** Provider coverage uses mocked provider contracts for deterministic parity/security tests and preserves opt-in live tests only where existing patterns already support them. + +### Codex's Discretion +- Choose exact route name/path, component extraction shape, copy, icons, and compact source metadata presentation while preserving local/cloud parity and provider-ownership clarity. +- Choose whether cloud detail shares an existing local detail component directly or extracts a shared detail surface first, as long as duplicate layouts/actions are not created. +- Choose internal naming changes only when needed for coherence or testability; visible copy must use Re-analyze. + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Milestone and phase contracts +- `.planning/ROADMAP.md` - Phase 14.1 goal, requirements, success criteria, and Phase 14.2/15/16 boundaries. +- `.planning/REQUIREMENTS.md` - canonical CLOUD-02, ANALYZE-01 through ANALYZE-07, and CACHE-03 through CACHE-05 requirements. +- `.planning/PROJECT.md` - virtual-local cloud storage model, privacy boundary, shared-component direction, and no-rewrite constraint. +- `.planning/phases/14-selective-analysis-and-byte-cache/14-CONTEXT.md` - analysis scope, byte cache, idempotency, queue, cache, and no-provider-mutation decisions inherited by Phase 14.1. +- `.planning/phases/13-virtual-local-cloud-operations/13-CONTEXT.md` - authorized cloud open/preview/download, shared browser operations, and provider limitation decisions inherited by Phase 14.1. +- `.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-CONTEXT.md` - truthful provider metadata, fallback correctness, owner scoping, and no-byte browse constraints. +- `AGENTS.md` - non-negotiable shared-module, frontend architecture, security, testing, documentation, worktree, and Git rules. + +### Frontend parity surfaces +- `frontend/src/components/storage/StorageBrowser.vue` - single shared browser; row/card action placement, topic/status display, disabled action states, analysis queue controls, and local/cloud event emissions must converge here. +- `frontend/src/views/FileManagerView.vue` - local document browser behavior and row/card action reference. +- `frontend/src/views/CloudFolderView.vue` - thin cloud data-provider that must route cloud row/card open/analyze/retry events without duplicating browser layout. +- `frontend/src/stores/cloudConnections.js` - cloud analysis status translation, queue state, retry/cancel/skip actions, and cache/settings state. +- `frontend/src/api/cloud.js` - authorized cloud open/preview/download, analysis estimate/enqueue/status/control, and cache/settings API client surface. + +### Backend cloud, cache, and analysis surfaces +- `backend/db/models.py` - `CloudItem`, `CloudItemTopic`, `CloudByteCacheEntry`, `CloudAnalysisJob`, and `CloudAnalysisJobItem` persistence contracts. +- `backend/api/cloud/operations.py` - authorized cloud open/preview/download routes that must keep response shapes credential-free and cache-backed. +- `backend/api/cloud/analysis.py` - cloud analysis route aggregator for estimate, enqueue, status, cancel, skip, and retry controls. +- `backend/api/cloud/schemas.py` - credential-free response schemas; must continue excluding `object_key` and `credentials_enc`. +- `backend/services/cloud_cache.py` - `hydrate_and_cache_bytes` and cache lifecycle helpers; open/preview/download must not bypass them. +- `backend/services/cloud_analysis.py` - estimate/enqueue/status/cancel/skip/retry orchestration and idempotency/current-state behavior. +- `backend/services/cloud_analysis_processing.py` - per-item processing, stale guard, extraction/classification, and cooperative cancellation. +- `backend/tasks/cloud_analysis_tasks.py` - Celery task boundary; UUID-only payload and worker-side credential revalidation. + +### Verification references +- `backend/tests/test_cloud_analysis_contract.py` - cloud analysis scope, idempotency, no-byte estimate, processing, and task contract coverage. +- `backend/tests/test_cloud_security.py` - owner/admin/credential/no-byte/cache response negative tests to extend for Phase 14.1. +- `backend/tests/test_cloud_audit.py` - metadata-only cloud operation audit patterns. +- `backend/tests/test_cloud.py` - cloud API integration patterns. +- `frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js` - shared disabled-capability/action rendering coverage. +- `frontend/src/components/storage/__tests__/StorageBrowser.cloud-queue.test.js` - cloud queue behavior to preserve while adding parity assertions. +- `frontend/src/views/__tests__/CloudFolderOpenPreview.test.js` - authorized open/preview/download frontend contract. +- `frontend/src/views/__tests__/CloudFolderView.test.js` - rendered cloud view behavior and mocked store/API patterns. +- `frontend/src/views/__tests__/FileManagerView.test.js` - local file manager behavior reference for paired parity tests. + + + + +## Existing Code Insights + +### Reusable Assets +- `StorageBrowser.vue`: already owns local/cloud rows, selection, toolbar actions, topic badge rendering, analysis actions, queue controls, upload queue dialogs, disabled capability UI, and event emissions. +- `CloudFolderView.vue`: already maps cloud browse state, byte availability, analysis estimates, active queue, and cloud open/analyze/control events into `StorageBrowser`. +- `FileManagerView.vue`: local file browser reference for row click behavior, topic color lookup, and document open navigation. +- `cloudConnections.js`: single source for cloud analysis status translation and queue/control API calls. +- `CloudItem` and `CloudItemTopic`: already persist extracted text, analysis status, semantic index status/data, provider version/fingerprint context, and topics without creating local `Document` rows. +- `hydrate_and_cache_bytes`: existing cache lifecycle entry point for provider byte hydration; Phase 14.1 should reuse it rather than opening provider bytes directly. + +### Established Patterns +- Views own stores and route params; smart components own interaction/layout and emit upward. +- Local and cloud browser behavior must share components when the user-facing action is the same. +- Cloud files use connection UUID plus opaque `provider_item_id`; Vue must not parse provider paths or expose provider URLs. +- Provider-owned bytes remain authoritative. Cached bytes are temporary, owner-scoped, and private. +- Credentials are decrypted only at provider/task boundaries and never enter API responses, logs, browser state, broker payloads, or planning artifacts. +- Service code raises domain exceptions or `ValueError`; routers translate to HTTP/typed responses. + +### Integration Points +- Add or reuse a cloud detail route that opens from `StorageBrowser` cloud rows/cards and renders the same core detail sections/actions as local document detail. +- Extract a shared detail component if needed so local and cloud detail behavior does not fork into parallel layouts. +- Extend cloud item response shape or add an owner-scoped detail endpoint so analyzed cloud files can supply extracted text, topics, status, stale/current state, provider/location metadata, and supported action reasons. +- Add force Re-analyze behavior to the cloud analysis route/service contract while preserving normal idempotent enqueue behavior for non-forced jobs. +- Extend frontend tests with paired local/cloud fixtures and backend tests with cache/auth/no-mutation invariants. + + + + +## Specific Ideas + +- The cloud detail view should exist even before analysis; it should offer Analyze and then fill in extracted text/topics/status after analysis completes. +- Visible copy should say Re-analyze everywhere instead of Re-classify. +- Stale or partial analysis should not erase useful prior data; users should see what exists and get a clear action to refresh or retry it. +- Unsupported preview should not auto-download. Users explicitly choose Download through the authorized DocuVault endpoint. + + + + +## Deferred Ideas + +- Unified keyword/semantic search across local and analyzed cloud documents remains Phase 15. +- Provider delta feeds, external-delete handling, and broad sync/change reliability remain Phase 16. +- Permanent local import/pinning of provider files remains future `IMPORT-01`; Phase 14.1 must not convert cloud files into local `Document` rows. +- Cross-provider copy/move and live parity testing for every provider remain out of scope. + + + +--- + +*Phase: 14.1-cloud-local-file-parity-hardening* +*Context gathered: 2026-06-26* diff --git a/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-DISCUSSION-LOG.md b/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-DISCUSSION-LOG.md new file mode 100644 index 0000000..c5d937b --- /dev/null +++ b/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-DISCUSSION-LOG.md @@ -0,0 +1,87 @@ +# Phase 14.1: cloud-local-file-parity-hardening - Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md - this log preserves the alternatives considered. + +**Date:** 2026-06-26 +**Phase:** 14.1-cloud-local-file-parity-hardening +**Areas discussed:** Open/view/download parity, Analyzed metadata display, Retry/reanalyze states, Unsupported/provider-limited actions, Parity tests + +--- + +## Open/view/download parity + +| Question | Options Considered | User's Choice | +|----------|--------------------|---------------| +| How should cloud file open/view behave when a cloud file has been analyzed? | Route to a document-like detail view; keep direct open/download; hybrid explicit View analysis; other | Route to a document-like detail view backed by `CloudItem`, not a local `Document` import. | +| What should happen for a cloud file that has not been analyzed yet? | Open/preview first with analyze prompt; always show detail view; split by file type; other | Always show the detail view; if not analyzed, offer Analyze. After analysis, fill the view with extracted text/topics/status. Rename Re-classify to Re-analyze everywhere. | +| For preview and download from that cloud detail view, how should the controls behave? | Mirror local detail controls; keep separate cloud labels; preview embedded/download secondary; other | Mirror local detail controls and placement, using authorized cloud preview/download handlers. | +| How should the app distinguish cloud files from local files on the detail view? | Subtle source metadata; strong cloud banner; breadcrumbs only; other | Use subtle provider/location/status metadata. | + +**Notes:** User wants the cloud detail view to be the main file-open target even before analysis, not just a post-analysis route. + +--- + +## Analyzed metadata display + +| Question | Options Considered | User's Choice | +|----------|--------------------|---------------| +| What should the cloud detail view show once analysis has completed? | Same sections as local documents; compact summary first; cloud-specific panel; other | Same sections as local documents. | +| How should analyzed cloud files appear in `StorageBrowser.vue` rows/cards? | Match local rows/cards where data exists; status badge only; separate cloud indicators; other | Match local rows/cards where equivalent data exists. | +| How should stale cloud analysis be presented when provider version/etag changed? | Show stale state with Re-analyze; hide stale analysis; treat stale as failed; other | Keep prior extracted text/topics visible, mark stale, and offer Re-analyze. | +| If cloud analysis partially succeeds, what should the UI show? | Show available data with targeted retry; failed state only; queue-only failure; other | Show available partial data and mark failed classification/topics with targeted retry/Re-analyze. | + +**Notes:** Prior data should remain useful even when stale or partially failed. + +--- + +## Retry/reanalyze states + +| Question | Options Considered | User's Choice | +|----------|--------------------|---------------| +| Where should the Re-analyze action appear? | Same places local re-analysis appears plus cloud detail; detail only; queue only; other | Same local/cloud detail and row action locations when analysis exists or is stale/failed. | +| What should Re-analyze do for an already-current analyzed cloud file? | Force fresh analysis after confirmation; keep idempotent skip; only reclassify existing text; other | Force fresh extraction/classification after confirmation. | +| How should failed analysis retries behave from detail view or row/card? | Retry as part of existing job model; always create new job; queue only; other | Use existing job retry semantics where possible; create a single-item retry job when no active job exists. | +| How should local document wording change alongside cloud parity? | Rename only action label; rename action and internal API/code; cloud only; other | Rename visible Re-classify copy to Re-analyze everywhere while preserving existing behavior/API unless planning requires internal rename. | + +**Notes:** The label change applies globally to visible UI copy. + +--- + +## Unsupported/provider-limited actions + +| Question | Options Considered | User's Choice | +|----------|--------------------|---------------| +| How should unsupported cloud preview/download/analyze cases appear in shared UI? | Shared disabled/typed state; hide unavailable actions; cloud warning panel; other | Same shared action positions, disabled or typed with backend-provided reasons. | +| When cloud format cannot be previewed but can be downloaded, what should happen? | Show preview unavailable and keep Download active; auto-download fallback; hide Preview; other | Show preview unavailable and keep authorized Download active. | +| If a provider cannot supply reliable version/etag metadata, how should states behave? | Use fallback fingerprint and label uncertainty subtly; always require manual Re-analyze; hide freshness; other | Use fallback fingerprint; surface uncertainty only when it affects a user action. | +| If analysis is unsupported for a file type, where should that be shown? | Everywhere Analyze would appear; detail only; queue/estimate only; other | Everywhere the Analyze action would appear, disabled with the same reason text. | + +**Notes:** Unsupported preview must not trigger an automatic download from Preview. + +--- + +## Parity tests + +| Question | Options Considered | User's Choice | +|----------|--------------------|---------------| +| What should be the core frontend parity test shape? | Paired local/cloud assertions; cloud-only regression tests; E2E browser flow only; other | Paired local/cloud assertions in shared components and detail surfaces. | +| Which backend security/cache invariants must be explicit? | All cache/auth boundaries; only new detail endpoints; reuse Phase 14 coverage; other | Cover no provider URL/credentials/object_key leakage, owner/admin negatives, cache hydration, forced Re-analyze path, and no provider mutation. | +| Should tests require a new cloud-file detail route? | Yes, route-level parity; component parity only; no route tests; other | Yes, route-level parity from browser row to cloud detail route. | +| How broad should live/provider-style coverage be? | Mocked provider contracts plus existing opt-in live pattern; live tests for every provider; frontend-only provider coverage; other | Mocked providers for deterministic tests and preserve opt-in live tests only where existing patterns support them. | + +**Notes:** The test contract should force local/cloud parity rather than allowing a cloud-only UI fork. + +--- + +## Codex's Discretion + +- Choose exact route naming/path, shared detail component shape, source metadata presentation, and internal naming changes where necessary. +- Keep local/cloud parity and provider ownership boundaries as the deciding constraints. + +## Deferred Ideas + +- Unified smart search remains Phase 15. +- Provider change tracking and broader sync reliability remain Phase 16. +- Permanent local import/pinning remains future `IMPORT-01`. +- Live parity testing for every provider is out of scope.