kite/.planning/phases/14.1-cloud-local-file-parity-hardening/14.1-RESEARCH.md

Phase 14.1 Research: Cloud/Local File Parity Hardening

Execution note: produced via Codex generic-agent workaround for gsd-phase-researcher because typed GSD agent dispatch failed before launch.

User Constraints

Phase Boundary

Phase 14.1 hardens the already-shipped cloud file experience so cloud files opened, viewed, downloaded, analyzed, retried, re-analyzed, and displayed after analysis behave like local documents where the user-facing concepts match. Cloud files remain provider-owned CloudItem records, not imported local Document rows. All cloud bytes continue through DocuVault authorization and the existing cache lifecycle; provider URLs, credentials, cache object keys, cross-provider transfer, unified search, and broader provider change tracking remain out of scope.

Implementation Decisions

Open/view/download parity

• D-01: Cloud file rows/cards open into a document-like detail view backed by CloudItem, not a local Document import.
• D-02: The cloud detail view exists before analysis. If a cloud file is not analyzed yet, the detail view offers Analyze; after analysis, the same view fills in extracted text, topics, and status.
• D-03: Preview and download controls in the cloud detail view mirror local document detail controls and placement, while calling authorized cloud preview/download handlers that hydrate bytes through the cache lifecycle.
• D-04: The cloud detail view stays visually aligned with local document detail and uses only subtle source metadata, such as provider, location, and current/stale status, to show that the file is provider-owned.

Analyzed metadata display

• D-05: Once analysis completes, cloud detail shows the same core sections as local documents: extracted text, topics, analysis status, preview/download controls, and subtle source metadata.
• D-06: StorageBrowser.vue rows/cards should match local rows/cards wherever equivalent data exists, including topic badges, analysis/current/stale state, and available actions in the same positions.
• D-07: Stale cloud analysis keeps prior extracted text and topics visible, marks them stale, and offers Re-analyze.
• D-08: Partial analysis results remain visible. For example, extracted text can display while failed classification/topics are marked clearly with a targeted Retry analysis or Re-analyze action.

Retry and re-analyze states

• D-09: Rename visible "Re-classify" copy to "Re-analyze" everywhere. Preserve existing local behavior/API unless planning finds an internal rename is necessary.
• D-10: Re-analyze appears consistently in local and cloud detail/row action locations when analysis exists or is stale/failed.
• D-11: Explicit Re-analyze on an already-current cloud file allows the user to force fresh extraction/classification after confirmation, instead of silently skipping as already-current.
• D-12: Failed analysis retry from rows/cards/detail uses the existing job retry semantics where possible; if no active job exists, create a single-item retry job.

Unsupported and provider-limited actions

• D-13: Unsupported cloud preview/download/analyze cases use the same shared action positions as local files, disabled or typed with clear backend-provided reasons.
• D-14: If a cloud file cannot be previewed in-app but can be downloaded, show Preview unavailable with the reason and keep authorized Download active. Do not auto-download from Preview.
• D-15: If a provider lacks reliable version/etag metadata, use the existing fallback metadata fingerprint and surface uncertainty only when it affects a user action.
• D-16: Unsupported analysis appears everywhere the Analyze action would appear, disabled with the same reason text in row/card/detail.

Parity tests

• D-17: Frontend parity tests use paired local/cloud assertions in shared components and detail surfaces, checking equivalent controls, labels, topic/status placement, and disabled states.
• D-18: Backend Phase 14.1 tests explicitly cover cache/auth boundaries: no provider URL, credentials, or object_key leakage; owner/admin negatives; preview/download through cache hydration; forced Re-analyze through the authorized analysis path; and no provider mutation.
• D-19: Tests require route-level parity: a cloud detail route opens from browser rows and renders the same core sections/actions as local detail.
• D-20: Provider coverage uses mocked provider contracts for deterministic parity/security tests and preserves opt-in live tests only where existing patterns already support them.

Codex's Discretion

• Choose exact route name/path, component extraction shape, copy, icons, and compact source metadata presentation while preserving local/cloud parity and provider-ownership clarity.
• Choose whether cloud detail shares an existing local detail component directly or extracts a shared detail surface first, as long as duplicate layouts/actions are not created.
• Choose internal naming changes only when needed for coherence or testability; visible copy must use Re-analyze.

Deferred Ideas

• Unified keyword/semantic search across local and analyzed cloud documents remains Phase 15.
• Provider delta feeds, external-delete handling, and broad sync/change reliability remain Phase 16.
• Permanent local import/pinning of provider files remains future IMPORT-01; Phase 14.1 must not convert cloud files into local Document rows.
• Cross-provider copy/move and live parity testing for every provider remain out of scope.

Project Constraints (from AGENTS.md)

• Use the existing stack: FastAPI/Python 3.12, SQLAlchemy async/PostgreSQL, MinIO, Vue 3 Options API/Pinia/Vue Router/Vite/Tailwind. [VERIFIED: AGENTS.md]
• JWT stays in Pinia memory only; refresh tokens stay httpOnly/Secure/SameSite=Strict. Do not add browser persistence for auth, cloud jobs, credentials, or object keys. [VERIFIED: AGENTS.md]
• Cloud credentials remain HKDF-encrypted per user; credentials decrypt only at provider/task boundaries and never enter API responses, logs, broker payloads, or frontend state. [VERIFIED: AGENTS.md]
• Every document, folder, connection, item, cache, and job path must enforce owner scoping; admin accounts must not receive document/cloud content. [VERIFIED: AGENTS.md]
• Cloud browse/refresh must not download bytes or mutate quota; open/preview/download/analysis are the only byte-hydrating paths. [VERIFIED: AGENTS.md]
• Router code imports shared helpers rather than creating local variants: get_client_ip, CloudConnectionError, AI parsing helpers, and password validation all have canonical modules. [VERIFIED: AGENTS.md]
• All cloud mutation orchestration remains in backend/services/cloud_operations.py; all byte cache lifecycle calls route through hydrate_and_cache_bytes in backend/services/cloud_cache.py. [VERIFIED: AGENTS.md]
• CacheStatusOut and AnalysisJobOut must never include object_key or credentials_enc; new cloud detail/browse schemas need the same strict allowlist style. [VERIFIED: AGENTS.md]
• Frontend formatting and provider styling must come from frontend/src/utils/formatters.js; StorageBrowser.vue remains the single file browser. [VERIFIED: AGENTS.md]
• FileManagerView.vue and CloudFolderView.vue must stay thin data providers; shared layout/action behavior belongs in smart components or shared detail components. [VERIFIED: AGENTS.md]
• Files with no active route/import must be deleted in the same commit. HomeView.vue and FolderView.vue must not be recreated. [VERIFIED: AGENTS.md]
• Any feature/bug fix requires focused tests; backend pytest -v and frontend tests must pass before phase advancement. [VERIFIED: AGENTS.md]
• Major shipped work must update AGENTS/README when user-facing/API/rule/version behavior changes, bump app versions, run security gates, commit, and push. [VERIFIED: AGENTS.md]
• Security gate must include dependency audits, owner/admin negatives, credential secrecy, no hardcoded secrets, header/CSP/auth invariants, and no high/critical CVEs. [VERIFIED: AGENTS.md]

Standard Stack

• Use existing FastAPI routers under backend/api/cloud/ for cloud item detail and analysis controls; no new backend framework or package is needed. [VERIFIED: codebase]
• Use SQLAlchemy ORM joins/selects over CloudItem, CloudItemTopic, Topic, CloudConnection, and CloudAnalysisJobItem; do not add raw SQL string interpolation. [VERIFIED: backend/db/models.py, AGENTS.md]
• Use existing Vue Router, Pinia, and shared API client barrel (frontend/src/api/client.js) for the cloud detail route and calls. [VERIFIED: frontend/src/router/index.js, frontend/src/api/client.js]
• Use StorageBrowser.vue for browser row/card parity and extract a shared detail surface from DocumentView.vue rather than creating a parallel cloud-only layout. [VERIFIED: frontend/src/views/DocumentView.vue, frontend/src/components/storage/StorageBrowser.vue]
• Use existing cache and analysis services: hydrate_and_cache_bytes, estimate_scope, enqueue_analysis_job, retry_job_item, and process_cloud_analysis_item. [VERIFIED: backend/services/cloud_cache.py, backend/services/cloud_analysis.py, backend/tasks/cloud_analysis_tasks.py]

Current Runtime State Inventory

1. Routes: local detail exists at /document/:id; cloud list routes exist at /cloud and /cloud/:connectionId/:folderId(.*); no cloud detail route currently exists. [VERIFIED: frontend/src/router/index.js]
2. Row open behavior: local FileManagerView routes file-open to /document/${file.id}; cloud CloudFolderView handles file-open by calling openCloudFile and may auto-fallback to download for unsupported preview. [VERIFIED: frontend/src/views/FileManagerView.vue, frontend/src/views/CloudFolderView.vue]
3. Detail layout: DocumentView.vue owns local header, topics card, Re-classify visible copy, suggestions, preview modal, delete, and extracted text layout directly. There is no shared detail component yet. [VERIFIED: frontend/src/views/DocumentView.vue]
4. Browser display: StorageBrowser.vue already renders local/cloud file rows, topic badges when file.topics exists, cloud analysis toolbar/actions, cloud queue, selection, and capability-disabled action buttons. It also has a local translateStatus duplicate despite the store exposing translateAnalysisStatus. [VERIFIED: frontend/src/components/storage/StorageBrowser.vue, frontend/src/stores/cloudConnections.js]
5. Backend data: CloudItem already stores extracted_text, analysis_status, semantic_index_status, etag/version/fingerprint inputs, and CloudItemTopic links topics without creating a Document row. [VERIFIED: backend/db/models.py]
6. Backend response gap: CloudItemOut currently exposes id/provider/name/kind/parent/content type/size/modified/etag/capabilities only; it does not expose extracted text, analysis status, semantic status, topics, provider display name, source metadata, or unsupported analysis reason. [VERIFIED: backend/api/cloud/schemas.py, backend/api/cloud/browse.py]
7. Cloud content boundary: open returns a DocuVault download URL; preview/download route through hydrate_and_cache_bytes when a CloudItem and version key exist, but fallback direct adapter.get_object still exists for missing metadata. Phase 14.1 should keep tests focused on the metadata-backed path and decide whether missing-metadata fallback is acceptable for detail/open parity. [VERIFIED: backend/api/cloud/operations.py]
8. Analysis idempotency: enqueue_analysis_job marks unchanged indexed items as already_current. There is no force flag in AnalysisEnqueueRequest or service signature, so D-11 requires an explicit API/service extension. [VERIFIED: backend/api/cloud/schemas.py, backend/services/cloud_analysis.py]
9. Retry semantics: retry_job_item retries by cloud_item_id within an existing job and allows failed or queued; if no active/known job exists, D-12 needs a single-item job creation path. [VERIFIED: backend/services/cloud_analysis.py, backend/api/cloud/analysis.py]
10. Processing behavior: process_job_item writes extracted text to CloudItem, classifies to CloudItemTopic, marks CloudItem.analysis_status = "indexed", and counts stale as a failed UI outcome while preserving data fields. [VERIFIED: backend/services/cloud_analysis_processing.py]

Architecture Patterns

• Add a cloud detail route keyed by connection UUID and opaque provider item ID, for example named route cloud-file-detail under /cloud/:connectionId/item/:itemId(.*) or an equivalent non-conflicting path. Pass provider IDs as opaque route params/query values and let Vue Router encode them; never split or decode provider paths in Vue. [VERIFIED: frontend/src/router/index.js, frontend/src/views/CloudFolderView.vue]
• Change cloud row open to navigate to the cloud detail route, not immediately preview/download. Keep explicit preview/download buttons on row/detail actions if added; do not trigger auto-download from Preview. [VERIFIED: frontend/src/views/CloudFolderView.vue, 14.1-CONTEXT.md]
• Extract a shared document detail surface from DocumentView.vue with props/events for: title, metadata line, source metadata, topics, status/stale badges, extracted text, preview availability, download availability, analyze/re-analyze/retry actions, and optional delete/share/suggest controls. Local and cloud views then provide data and event handlers. [VERIFIED: frontend/src/views/DocumentView.vue, AGENTS.md]
• Keep DocumentView.vue and the new cloud detail view as data-provider views. They should call stores/API and feed the shared detail surface; they should not duplicate the card/layout markup. [VERIFIED: AGENTS.md, frontend/src/views/FileManagerView.vue]
• Extend CloudItemOut only if browse rows need the added fields; otherwise add CloudItemDetailOut for the detail endpoint and add a lighter row extension for topics/status. In either case, schemas must be explicit allowlists with no credentials_enc, object_key, provider URL, raw tokens, or cache key fields. [VERIFIED: backend/api/cloud/schemas.py]
• Use a service helper under backend/services/cloud_items.py for owner-scoped cloud item detail resolution and topic loading if multiple routers need it; routers should translate ValueError/domain exceptions into HTTP responses. [VERIFIED: AGENTS.md, backend/services/cloud_items.py]
• For row/card parity, have browse responses include enough metadata for StorageBrowser.vue: topics, analysis_status, current/stale state if known, unsupported analysis reason, and existing size/modified fields. This lets local and cloud rows share topic badge/status placement. [VERIFIED: frontend/src/components/storage/StorageBrowser.vue, backend/api/cloud/browse.py]
• Use the Pinia store's translateAnalysisStatus as the single frontend status translation source. Remove or route around the duplicate translateStatus in StorageBrowser.vue during implementation. [VERIFIED: frontend/src/stores/cloudConnections.js, frontend/src/components/storage/StorageBrowser.vue]
• Add force or force_reanalyze to the cloud enqueue request/service path for explicit Re-analyze only. Default enqueue remains idempotent and skips already-current items. [VERIFIED: backend/api/cloud/schemas.py, backend/services/cloud_analysis.py]
• For forced Re-analyze, preserve no-provider-mutation and cache lifecycle boundaries: service may enqueue byte work despite already-current metadata, but processing still uses process_cloud_analysis_item and hydrate_and_cache_bytes; no local Document row is created. [VERIFIED: backend/services/cloud_analysis_processing.py, backend/services/cloud_cache.py]

Don't Hand-Roll

• Do not build a second cloud file grid, card list, action row, or queue UI; extend StorageBrowser.vue and shared detail components. [VERIFIED: AGENTS.md]
• Do not create local Document rows for cloud detail, preview, analysis, re-analysis, retry, or topic display. CloudItem and CloudItemTopic are the canonical cloud analysis records. [VERIFIED: backend/db/models.py, 14.1-CONTEXT.md]
• Do not call provider SDKs directly from frontend code or expose raw provider URLs. Cloud preview/download must go through DocuVault API endpoints. [VERIFIED: frontend/src/api/cloud.js, backend/api/cloud/operations.py]
• Do not bypass hydrate_and_cache_bytes in open/preview/download or forced analysis paths. [VERIFIED: AGENTS.md, backend/api/cloud/operations.py]
• Do not duplicate formatters/provider colors/status translation in components. Use utils/formatters.js and the store/shared helpers. [VERIFIED: AGENTS.md, frontend/src/stores/cloudConnections.js]
• Do not introduce package dependencies for this phase. The needed primitives already exist in FastAPI, SQLAlchemy, Vue, Pinia, Vue Router, and the current codebase. [VERIFIED: package files not required by phase scope]

Common Pitfalls

• Auto-downloading unsupported previews would violate D-14. Existing CloudFolderView.onFileOpen currently calls downloadCloudFile on unsupported_preview; the detail flow should replace this with visible "Preview unavailable" plus explicit Download. [VERIFIED: frontend/src/views/CloudFolderView.vue]
• Extending CloudItemOut with extracted_text for every browse row could inflate list payloads. Prefer row-level lightweight fields and full extracted text on the detail endpoint unless the planner intentionally accepts larger browse payloads. [VERIFIED: backend/api/cloud/schemas.py]
• AnalysisJobItemOut currently has no name field, while the frontend maps item.name from job status. Either tests already mock around this or the status/detail API needs alignment before relying on job item names in new detail controls. [VERIFIED: backend/api/cloud/schemas.py, frontend/src/stores/cloudConnections.js]
• CacheStatusOut.analysis_progress_detail is documented and typed as a string, while the frontend treats it as boolean in several places. Avoid compounding this mismatch during parity work; normalize at the store boundary or adjust tests deliberately. [VERIFIED: backend/api/cloud/schemas.py, frontend/src/stores/cloudConnections.js]
• The cloud item content routes have direct provider fetch fallbacks when no CloudItem metadata/version key exists. Phase 14.1 tests should prove the normal detail/open path requires owner-scoped metadata and uses cache hydration, and should decide whether to remove or explicitly constrain fallback behavior. [VERIFIED: backend/api/cloud/operations.py]
• Stale cloud analysis should not clear CloudItem.extracted_text or topic links. Processing already writes fields independently; UI should mark stale/failed states while rendering existing content. [VERIFIED: backend/services/cloud_analysis_processing.py, 14.1-CONTEXT.md]
• Provider item IDs can contain reserved URL/path characters. New cloud detail routes and API calls must use named routes/encodeURIComponent consistently and must not reconstruct breadcrumbs or locations by parsing provider IDs. [VERIFIED: frontend/src/views/CloudFolderView.vue, frontend/src/api/cloud.js]
• Local visible copy still says Re-classify; Phase 14.1 requires visible Re-analyze without necessarily renaming internal classifyDocument APIs. [VERIFIED: frontend/src/views/DocumentView.vue, 14.1-CONTEXT.md]

Security Domain

• Security enforcement is required. Phase 14.1 touches auth-protected document/cloud content, cloud credentials, cache entries, and analysis jobs. [VERIFIED: AGENTS.md, 14.1-CONTEXT.md]
• ASVS V1/V2 auth-session constraints apply indirectly: do not add persistent tokens or local/session storage for access tokens; keep API calls through authenticated client helpers. [VERIFIED: AGENTS.md, frontend/src/api/utils.js]
• ASVS V4 access control applies directly: cloud detail, row metadata, preview, download, force re-analyze, retry, and job status must resolve connection_id, provider_item_id/cloud_item_id, job ID, and cache entry under current_user.id; foreign user and admin-negative tests are mandatory. [VERIFIED: AGENTS.md, backend/api/cloud/analysis.py, backend/api/cloud/operations.py]
• ASVS V5 validation applies to route/body fields: provider item IDs are opaque strings, UUIDs are parsed as UUIDs, and request schemas must explicitly declare accepted fields with no mass assignment. [VERIFIED: backend/api/cloud/schemas.py]
• ASVS V8 data protection applies to credentials_enc, provider tokens, provider URLs, MinIO object keys, cache metadata, extracted text, and admin access. Response schemas must remain allowlists and tests must assert no leakage. [VERIFIED: backend/api/cloud/schemas.py, backend/tests/test_cloud_security.py]
• ASVS V10 malicious code/dependency concerns are low for this phase because no package installation is needed. Keep no-new-package as the default. [VERIFIED: codebase]
• ASVS V12 file/resource handling applies to preview/download: filenames in headers must stay encoded, content disposition must match preview/download intent, and bytes must route through cache with owner checks. [VERIFIED: backend/api/cloud/operations.py]

Verification Targets

• Backend API tests: owner can fetch cloud detail with analysis fields; foreign user gets 404; admin/regular-user boundary follows get_regular_user; response excludes credentials_enc, object_key, raw provider URLs, and provider tokens. [VERIFIED: backend/tests/test_cloud_security.py patterns]
• Backend cache tests: cloud detail does not hydrate bytes; preview/download from detail hydrate through hydrate_and_cache_bytes; browse/detail estimate paths do not call adapter.get_object; forced Re-analyze enqueues authorized analysis work without provider mutation. [VERIFIED: backend/tests/test_cloud_cache.py, backend/tests/test_cloud_analysis_contract.py]
• Backend force tests: default enqueue skips already-current; explicit force queues an already-current item and preserves no-provider-mutation counters. [VERIFIED: backend/services/cloud_analysis.py]
• Backend partial/stale tests: existing extracted text/topics remain returned when analysis_status is stale or latest job item is failed; retry/re-analyze response is typed and owner-scoped. [VERIFIED: backend/services/cloud_analysis_processing.py]
• Frontend route tests: clicking a cloud file row in StorageBrowser via CloudFolderView opens a cloud detail route, not a direct preview/download; local file rows still route to /document/:id. [VERIFIED: frontend/src/views/FileManagerView.vue, frontend/src/views/CloudFolderView.vue]
• Frontend paired parity tests: local/cloud detail surfaces render equivalent sections and action positions for title, metadata, preview/download, topics, analysis status, extracted text, and Re-analyze. [VERIFIED: frontend/src/views/DocumentView.vue]
• Frontend unsupported tests: unsupported cloud preview shows reason and keeps explicit Download active; Analyze/Re-analyze disabled states use shared action positions and backend-provided reasons. [VERIFIED: frontend/src/components/storage/tests/StorageBrowser.capabilities.test.js]
• Frontend row/card tests: paired local/cloud fixtures in StorageBrowser.vue assert topic badges and analysis/current/stale badges appear in the same relative area, and visible copy uses Re-analyze. [VERIFIED: frontend/src/components/storage/StorageBrowser.vue]
• Regression grep: no visible Re-classify remains after implementation except in historical planning/test snapshots where intentionally preserved. [VERIFIED: frontend/src/views/DocumentView.vue]

Code Examples

Cloud Detail Fetch Pattern

# backend/api/cloud/detail.py or operations.py
@router.get("/connections/{connection_id}/items/{item_id:path}/detail", response_model=CloudItemDetailOut)
async def get_cloud_item_detail(...):
    item = await resolve_owned_cloud_item_detail(
        session,
        user_id=current_user.id,
        connection_id=connection_id,
        provider_item_id=item_id,
    )
    return CloudItemDetailOut(...)

Use a service helper for resolve_owned_cloud_item_detail; include topic names via CloudItemTopic -> Topic, connection/provider display metadata, and no cache/object credential fields. [VERIFIED: backend/services/cloud_items.py, backend/api/cloud/schemas.py]

Forced Re-Analyze Pattern

class AnalysisEnqueueRequest(BaseModel):
    scope: str
    provider_item_ids: Optional[list[str]] = None
    recursive: bool = False
    failure_behavior: str = "pause_batch"
    force: bool = False

force=False keeps current idempotency. force=True bypasses already_current only for an explicit user action and still validates owner/connection/item and unsupported type. [VERIFIED: backend/api/cloud/schemas.py, backend/services/cloud_analysis.py]

Frontend Shared Detail Surface Pattern

<DocumentDetailSurface
  :title="detailTitle"
  :metadata="metadata"
  :topics="topics"
  :analysis-status="analysisStatus"
  :extracted-text="extractedText"
  :source="sourceMetadata"
  :preview-state="previewState"
  :download-state="downloadState"
  @preview="preview"
  @download="download"
  @reanalyze="reanalyze"
  @retry-analysis="retryAnalysis"
/>

Local and cloud views provide handlers; the surface owns layout and action placement. [VERIFIED: frontend/src/views/DocumentView.vue, AGENTS.md]

Package Legitimacy Audit

No external packages are recommended for Phase 14.1. The implementation should use existing backend/frontend dependencies and local services/components only. [VERIFIED: package scope/codebase]

Research Confidence

• HIGH: Existing code paths, models, schemas, routes, stores, and tests cited above were read directly from this worktree. [VERIFIED: codebase]
• HIGH: User decisions, deferred scope, project constraints, and security/testing rules were read from phase context, ROADMAP, REQUIREMENTS, STATE, PROJECT, and AGENTS. [VERIFIED: planning docs]
• MEDIUM: Specific route path recommendation is discretionary; the planner may choose an equivalent path if it preserves opaque provider IDs and route-level parity. [ASSUMED]
• LOW: No external ecosystem/library claims are made; no web/package lookup was necessary because the phase is an in-repo parity hardening task. [ASSUMED]