curo/kite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
90e1b525f1b69d06bf77bf00933c47b5488a61f4
kite/.planning/ROADMAP.md
T
curo1305andClaude Sonnet 4.6 90e1b525f1 docs(14-09): Phase 14 closeout — docs, version bump v0.3.0→v0.4.0, security evidence 
- CLAUDE.md/AGENTS.md: update current-state line to v0.4.0 Phase 14 complete;
  add Phase 14 shared module map entries (cloud_cache, cloud_analysis,
  cloud_analysis_processing, cloud_analysis_tasks, analysis router, cache API);
  add Phase 14 non-negotiable rules (hydrate_and_cache_bytes single entry point,
  IDs-only broker payload, object_key/credentials_enc excluded at schema level)
- README.md: bump to v0.4.0; add Selective cloud analysis and Cloud byte cache
  features; add Analysis API table (Phase 14 endpoints)
- SECURITY.md: add Phase 14 threat register (T-14-01..T-14-15) with evidence,
  security gate results, and accepted risks
- backend/main.py: bump FastAPI version to 0.4.0
- frontend/package.json: bump version to 0.4.0
- ROADMAP.md: mark Phase 14 complete (9/9 plans), plan 14-09 checked off
- 14-VALIDATION.md: gate evidence, requirement coverage, acceptance criteria check
- 14-SECURITY.md: security evidence summary and gate checklist

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-23 20:05:49 +02:00

176 lines
13 KiB
Markdown
Raw Blame History

# DocuVault Roadmap: v0.3 Reimagining Cloud Storage integration
**Status:** Proposed
**Phases:** 12-16
**Requirements:** 36
**Defined:** 2026-06-17
## Milestone Goal
Make connected cloud storage behave like first-class DocuVault storage: users can navigate and manage provider-owned files as if they were local, selectively analyze them, and find them through unified keyword and semantic search while DocuVault minimizes locally cached file bytes.
## Mandatory Cross-Cutting Gates
Before any phase is marked complete:
1. Backend and frontend test suites pass with zero failures; every new function, endpoint, store, and component has focused coverage.
2. Security gate verifies user/connection/item ownership, credential secrecy, SSRF defenses, metadata-only audit logs, cache isolation, and no admin document-content access.
3. Dependency audits report no high/critical vulnerabilities.
4. Provider integration tests cover supported capabilities and explicit unsupported behavior.
5. User-facing changes update `AGENTS.md`, relevant README sections, and application patch/minor versions according to project protocol.
## Phase Summary
| Phase | Name | Goal | Requirements |
|------:|------|------|--------------|
| 12 | 6/6 | Complete | 2026-06-21 |
| 13 | 11/11 | Complete | 2026-06-23 |
| 14 | 9/9 | Complete | 2026-06-23 |
| 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
| 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
## Phase Details
### Phase 12: Cloud Resource Foundation
**Goal:** DocuVault has a provider-neutral cloud file capability contract and durable per-user cloud item index, and the shared browser can render cloud resources and supported actions without copying full provider files into local storage.
**Depends on:** Phase 11
**Requirements:** CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01
**Success Criteria:**
1. Every supported provider reports normalized capabilities for browse, open, preview, upload, folder creation, rename, move, delete, and change tracking.
2. Browsing any connected provider persists normalized, owner-scoped cloud item metadata including provider ID/path, parent, type, size, modification time, and version/etag.
3. `StorageBrowser.vue` remains the single file browser and renders cloud actions from capabilities, including disabled actions with provider-specific explanations.
4. Provider-owned bytes remain outside DocuVault while metadata, extracted text placeholders, topic links, and semantic-index state can be persisted independently.
5. Ownership and admin-negative tests prove cloud connection and cloud item metadata cannot cross user boundaries.
### Phase 12.1: Fix Nextcloud root listing and sync visibility (INSERTED)
**Goal:** Restore visible, truthful, metadata-only cloud browsing by repairing Nextcloud's adapter contract, applying one completeness/freshness contract across all providers, and aligning the shared frontend browser with normalized provider identities.
**Requirements:** CONN-04, CLOUD-01, CACHE-01, SYNC-01
**Depends on:** Phase 12
**Plans:** 4/4 plans complete
**Execution waves:** Wave 1: 12.1-01; Wave 2: 12.1-02 after 01; Wave 3: 12.1-03 after 02; Wave 4: 12.1-04 after 01-03.
Plans:
- [x] 12.1-01-PLAN.md
- [x] 12.1-02-PLAN.md
- [x] 12.1-03-PLAN.md
- [x] 12.1-04-PLAN.md
- [x] 12.1-01 (Wave 1) — Restore the shared four-provider adapter contract, secure Nextcloud URL/redirect handling, and metadata-only listing behavior.
- [x] 12.1-02 (Wave 2; after 12.1-01) — Centralize complete/incomplete reconciliation so incomplete provider results cannot become fresh or delete cached metadata.
- [x] 12.1-03 (Wave 3; after 12.1-02) — Align the shared browser and trees with `kind`, opaque `provider_item_id` routing, lineage-based breadcrumbs, and server freshness.
- [ ] 12.1-04 (Wave 4; after 12.1-01 through 12.1-03) — Run sanitized two-mode live Nextcloud acceptance, rendered/security/secret gates, and phase closeout.
**Success Criteria:**
1. Nextcloud and generic WebDAV expose one canonical `CloudResourceAdapter.list_folder` implementation, and all four providers pass the same runtime identity/completeness/no-byte/no-mutation contract with provider-specific pagination fixtures.
2. Every outbound Nextcloud/WebDAV target, including redirects, is rejected unless each hop is HTTPS, same-origin, and passes URL plus resolved-address SSRF validation; automatic redirects are disabled.
3. `complete=False` never marks a folder fresh, advances `last_refreshed_at`, or authorizes unseen-item deletion in synchronous or worker paths; cached rows remain usable with a controlled warning.
4. The frontend classifies by `kind`, navigates through named routes/query serialization with opaque `provider_item_id` values, builds breadcrumbs from explicit navigation lineage rather than splitting provider IDs, and renders backend freshness verbatim.
5. Live Nextcloud validation first runs a nonblocking sanitized diagnostic; exact names/kinds become a blocking gate only after the owner-confirmed manifest is stored in a tracked, non-secret fixture. Unexpected live names are never printed or persisted.
6. Automated backend/frontend/rendered-flow tests, dedicated per-plan security reviews, dependency audits, an executable secret scan, documentation/version updates, and one atomic commit/push per plan pass without exposing credentials or provider bytes.
### Phase 13: Virtual-Local Cloud Operations
**Goal:** Users can maintain healthy cloud connections and perform the main file-management workflows in connected storage with the same shared browser interactions used for local files.
**Depends on:** Phase 12
**Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09
**Plans:** 11/11 plans complete
**Execution waves:** Wave 0: 13-01 and 13-02 in parallel; Wave 1: 13-03 after 13-01; Wave 2: 13-04 after 13-01 and 13-03; Wave 3: 13-05 after 13-01, 13-03, and 13-04; Wave 4: 13-06 after 13-05; Wave 5: 13-07 and 13-08 in parallel (13-07 after 13-02, 13-04, and 13-06; 13-08 after 13-03, 13-04, and 13-06); Wave 6: 13-09 after 13-08; Wave 7: 13-10 after 13-02, 13-04, 13-07, and 13-09; Wave 8: 13-11 after 13-03 through 13-10.
Plans:
- [x] 13-01-PLAN.md — Create red backend and provider-contract suites for reconnect, content, mutations, and audit secrecy.
- [x] 13-02-PLAN.md — Create red frontend/store suites for queue, preview, health UX, broader Drive consent, and no-probe-on-navigation.
- [x] 13-03-PLAN.md — Build the mutable cloud contract and orchestration seam without breaking centralized reconciliation.
- [x] 13-04-PLAN.md — Implement connection-ID reconnect, explicit health test, broader Drive scope handling, and authorized content routes.
- [x] 13-05-PLAN.md — Implement backend upload provider mechanics, typed route results, and refreshed-credential handoff.
- [x] 13-06-PLAN.md — Complete upload reconcile, freshness, and metadata-only audit follow-through before frontend queue wiring.
- [x] 13-07-PLAN.md — Wire the shared browser queue and binary-only preview or download fallback through thin cloud view handlers.
- [x] 13-08-PLAN.md — Implement backend create-folder and rename semantics with collision, stale, and stable-identity safeguards.
- [x] 13-09-PLAN.md — Implement backend move and delete semantics with same-connection, disclosure, security, and audit safeguards.
- [x] 13-10-PLAN.md — Finish shared-browser mutation UX, store-backed health behavior, reconnect copy, and the no-probe invariant.
- [x] 13-11-PLAN.md — Run closeout-only docs, versions, full gates, explicit secret scan, and ship-readiness checks.
**Success Criteria:**
1. Users can connect, test, reconnect, and disconnect providers while seeing current health and actionable reauthentication/error states.
2. Users can open or preview supported cloud documents and upload files into the current cloud folder through authorized DocuVault endpoints.
3. Users can create folders, rename items, move items within one connection, and delete items after confirmation wherever the provider supports each action.
4. Successful mutations immediately invalidate affected listings/index entries, refresh the shared browser, and write metadata-only audit events.
5. Provider-specific tests verify operation semantics, conflict/error responses, token refresh persistence, SSRF protection, and explicit unsupported capabilities.
### Phase 14: Selective Analysis and Byte Cache
**Goal:** Users can analyze individual files, selected scopes, or whole connections through observable and controllable background jobs, while file bytes are hydrated only on demand and evicted safely.
**Depends on:** Phase 13
**Requirements:** ANALYZE-01, ANALYZE-02, ANALYZE-03, ANALYZE-04, ANALYZE-05, ANALYZE-06, ANALYZE-07, CACHE-03, CACHE-04, CACHE-05
**Plans:** 9/9 plans complete
**Execution waves:** Wave 0: 14-01; Wave 1: 14-02 after 14-01; Wave 2: 14-03 after 14-02; Wave 3: 14-04 after 14-02 and 14-03; Wave 4: 14-05 after 14-03 and 14-04; Wave 5: 14-06 after 14-03 and 14-05; Wave 6: 14-07 after 14-04 and 14-05; Wave 7: 14-08 after 14-03 and 14-07; Wave 8: 14-09 after 14-01 through 14-08.
Plans:
- [x] 14-01-PLAN.md — Create red backend/frontend tests for analysis scopes, estimates, cache invariants, queue states, and security negatives.
- [x] 14-02-PLAN.md — Add durable cache/job/settings schema plus version-key and settings helpers.
- [x] 14-03-PLAN.md — Implement owner-scoped byte cache service, quota accounting, pinning, eviction, and cache settings/status API.
- [x] 14-04-PLAN.md — Implement estimate/enqueue/status/cancel/skip/retry analysis APIs with idempotent job creation.
- [x] 14-05-PLAN.md — Implement Celery-backed cloud analysis processing, cache hydration, extraction, classification, and cancellation.
- [x] 14-06-PLAN.md — Route existing cloud open/preview/download byte hydration through the bounded cache lifecycle.
- [x] 14-07-PLAN.md — Wire shared-browser analysis actions, estimates, aggregate progress, expandable queue, and controls.
- [x] 14-08-PLAN.md — Add Settings controls for cache limit, progress detail, and failure behavior.
- [x] 14-09-PLAN.md — Run closeout docs, versions, full test/security gates, validation, commit, and push.
**Success Criteria:**
1. Users can analyze one file, a multi-selection, a folder tree, or a whole connection after reviewing recursive file-count and byte-size estimates.
2. The UI exposes aggregate and per-item queued, downloading, extracting, classifying, indexed, cancelled, and failed states with retry and queued-work cancellation.
3. Analysis jobs are idempotent by cloud item and provider version/etag, and they never mutate the provider-owned file.
4. Cloud bytes are cached only for active opening, preview, or analysis work and are evicted by configurable limits without interrupting pinned active jobs.
5. Cache ownership, cache-key versioning, cancellation, duplicate-job, failure-retry, and eviction behavior have dedicated tests.
### Phase 15: Unified Smart Search
**Goal:** One search experience finds local and analyzed cloud documents by keywords, sentences, or semantic ideas and opens cloud results through on-demand hydration.
**Depends on:** Phase 14
**Requirements:** SEARCH-01, SEARCH-02, SEARCH-03, SEARCH-04, SEARCH-05, SEARCH-06, SEARCH-07
**Success Criteria:**
1. A single query returns authorized local and analyzed cloud results ranked through full-text and semantic relevance.
2. Keyword and sentence searches match persisted extracted text, while idea searches use persisted embeddings or equivalent semantic indexes.
3. Results clearly show source, provider, cloud location, topics, and current analysis/stale status and can be filtered by those fields.
4. Search execution performs no provider file download; only opening or previewing a result may hydrate bytes.
5. Search isolation, ranking, filtering, stale-result handling, and no-download-on-query invariants have automated coverage.
### Phase 16: Change Tracking and Reliability
**Goal:** DocuVault remains accurate when users or other applications modify connected cloud storage outside DocuVault.
**Depends on:** Phase 15
**Requirements:** SYNC-02, SYNC-03, SYNC-04
**Success Criteria:**
1. Provider delta feeds or bounded metadata refreshes detect changed versions and mark previously analyzed items stale until reanalysis.
2. Items deleted outside DocuVault disappear from navigation and are excluded from search without deleting unrelated user data.
3. Refresh jobs are idempotent, cursor-aware where supported, rate-limited, and retry transient failures with bounded exponential backoff.
4. Users can see refresh/stale/error state and deliberately reanalyze changed items.
5. End-to-end UAT verifies cloud-as-local operations, selective analysis, cache offload, unified smart search, and external-change recovery across supported provider classes.
## Coverage
- v0.3 requirements: 36
- Mapped to phases: 36
- Unmapped: 0
- Duplicate mappings: 0
---
*Roadmap proposed: 2026-06-17*
Reference in New Issue View Git Blame Copy Permalink