docs(13-01): complete red contract suite plan

2026-06-22 18:03:09 +02:00
parent fd6b561899
commit 451d30208c
4 changed files with 192 additions and 32 deletions
@@ -7,22 +7,22 @@
 ### Connections
- [ ] **CONN-01**: User can connect, reconnect, test, and disconnect each supported cloud provider.
+- [x] **CONN-01**: User can connect, reconnect, test, and disconnect each supported cloud provider.
- [ ] **CONN-02**: User can see connection health and actionable errors for expired, revoked, or invalid credentials.
+- [x] **CONN-02**: User can see connection health and actionable errors for expired, revoked, or invalid credentials.
- [ ] **CONN-03**: Reconnecting or refreshing credentials invalidates stale provider caches without exposing credentials.
+- [x] **CONN-03**: Reconnecting or refreshing credentials invalidates stale provider caches without exposing credentials.
 - [x] **CONN-04**: User interface reflects the file operations supported by each connected provider.
 ### Cloud File Management
 - [x] **CLOUD-01**: User can browse connected cloud files and folders through the shared `StorageBrowser`.
- [ ] **CLOUD-02**: User can open and preview supported cloud documents through DocuVault authorization.
+- [x] **CLOUD-02**: User can open and preview supported cloud documents through DocuVault authorization.
- [ ] **CLOUD-03**: User can upload files into the currently viewed cloud folder.
+- [x] **CLOUD-03**: User can upload files into the currently viewed cloud folder.
- [ ] **CLOUD-04**: User can create folders in connected cloud storage where the provider supports it.
+- [x] **CLOUD-04**: User can create folders in connected cloud storage where the provider supports it.
- [ ] **CLOUD-05**: User can rename cloud files and folders where the provider supports it.
+- [x] **CLOUD-05**: User can rename cloud files and folders where the provider supports it.
- [ ] **CLOUD-06**: User can move files and folders within the same cloud connection where the provider supports it.
+- [x] **CLOUD-06**: User can move files and folders within the same cloud connection where the provider supports it.
- [ ] **CLOUD-07**: User can delete cloud files and folders after explicit confirmation.
+- [x] **CLOUD-07**: User can delete cloud files and folders after explicit confirmation.
 - [x] **CLOUD-08**: User sees unsupported cloud actions disabled with a provider-specific explanation.
- [ ] **CLOUD-09**: Successful cloud mutations update navigation promptly and produce metadata-only audit events.
+- [x] **CLOUD-09**: Successful cloud mutations update navigation promptly and produce metadata-only audit events.
 ### Cloud Analysis
@@ -88,19 +88,19 @@
 | Requirement | Phase | Status |
 |-------------|-------|--------|
-| CONN-01 | Phase 13 | Pending |
+| CONN-01 | Phase 13 | Complete |
-| CONN-02 | Phase 13 | Pending |
+| CONN-02 | Phase 13 | Complete |
-| CONN-03 | Phase 13 | Pending |
+| CONN-03 | Phase 13 | Complete |
 | CONN-04 | Phase 12 | Complete |
 | CLOUD-01 | Phase 12 | Complete |
-| CLOUD-02 | Phase 13 | Pending |
+| CLOUD-02 | Phase 13 | Complete |
-| CLOUD-03 | Phase 13 | Pending |
+| CLOUD-03 | Phase 13 | Complete |
-| CLOUD-04 | Phase 13 | Pending |
+| CLOUD-04 | Phase 13 | Complete |
-| CLOUD-05 | Phase 13 | Pending |
+| CLOUD-05 | Phase 13 | Complete |
-| CLOUD-06 | Phase 13 | Pending |
+| CLOUD-06 | Phase 13 | Complete |
-| CLOUD-07 | Phase 13 | Pending |
+| CLOUD-07 | Phase 13 | Complete |
 | CLOUD-08 | Phase 12 | Complete |
-| CLOUD-09 | Phase 13 | Pending |
+| CLOUD-09 | Phase 13 | Complete |
 | ANALYZE-01 | Phase 14 | Pending |
 | ANALYZE-02 | Phase 14 | Pending |
 | ANALYZE-03 | Phase 14 | Pending |
@@ -126,6 +126,7 @@
 | SYNC-04 | Phase 16 | Pending |
 **Coverage:**
 - v0.3 requirements: 36 total
 - Mapped to phases: 36
 - Unmapped: 0
@@ -24,7 +24,7 @@ Before any phase is marked complete:
 | Phase | Name | Goal | Requirements |
 |------:|------|------|--------------|
 | 12 | 6/6 | Complete   | 2026-06-21 |
-| 13 | Virtual-Local Cloud Operations | Make cloud browsing and core file actions feel local while preserving provider semantics and security | CONN-01..03, CLOUD-02..07, CLOUD-09 |
+| 13 | 1/11 | In Progress|  |
 | 14 | Selective Analysis and Byte Cache | Analyze selected cloud scopes through cancellable jobs backed by bounded, private, on-demand byte caching | ANALYZE-01..07, CACHE-03..05 |
 | 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
 | 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
@@ -80,6 +80,22 @@ Plans:
 **Depends on:** Phase 12
 **Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09
 **Plans:** 1/11 plans executed
 **Execution waves:** Wave 0: 13-01 and 13-02 in parallel; Wave 1: 13-03 after 13-01; Wave 2: 13-04 after 13-01 and 13-03; Wave 3: 13-05 after 13-01, 13-03, and 13-04; Wave 4: 13-06 after 13-05; Wave 5: 13-07 and 13-08 in parallel (13-07 after 13-02, 13-04, and 13-06; 13-08 after 13-03, 13-04, and 13-06); Wave 6: 13-09 after 13-08; Wave 7: 13-10 after 13-02, 13-04, 13-07, and 13-09; Wave 8: 13-11 after 13-03 through 13-10.
 Plans:
 - [x] 13-01-PLAN.md — Create red backend and provider-contract suites for reconnect, content, mutations, and audit secrecy.
 - [ ] 13-02-PLAN.md — Create red frontend/store suites for queue, preview, health UX, broader Drive consent, and no-probe-on-navigation.
 - [ ] 13-03-PLAN.md — Build the mutable cloud contract and orchestration seam without breaking centralized reconciliation.
 - [ ] 13-04-PLAN.md — Implement connection-ID reconnect, explicit health test, broader Drive scope handling, and authorized content routes.
 - [ ] 13-05-PLAN.md — Implement backend upload provider mechanics, typed route results, and refreshed-credential handoff.
 - [ ] 13-06-PLAN.md — Complete upload reconcile, freshness, and metadata-only audit follow-through before frontend queue wiring.
 - [ ] 13-07-PLAN.md — Wire the shared browser queue and binary-only preview or download fallback through thin cloud view handlers.
 - [ ] 13-08-PLAN.md — Implement backend create-folder and rename semantics with collision, stale, and stable-identity safeguards.
 - [ ] 13-09-PLAN.md — Implement backend move and delete semantics with same-connection, disclosure, security, and audit safeguards.
 - [ ] 13-10-PLAN.md — Finish shared-browser mutation UX, store-backed health behavior, reconnect copy, and the no-probe invariant.
 - [ ] 13-11-PLAN.md — Run closeout-only docs, versions, full gates, explicit secret scan, and ship-readiness checks.
 **Success Criteria:**
@@ -2,30 +2,33 @@
 gsd_state_version: 1.0
 milestone: v0.3
 milestone_name: Reimagining Cloud Storage integration
-status: verifying
+current_phase: 13
 current_phase_name: virtual-local-cloud-operations
 status: executing
 stopped_at: Phase 13 context gathered
-last_updated: "2026-06-22T13:46:10.010Z"
+last_updated: "2026-06-22T16:02:44.490Z"
-last_activity: 2026-06-22 — Phase 12.1 execution started
+last_activity: 2026-06-22
 last_activity_desc: Phase 13 execution started
 progress:
  total_phases: 6
  completed_phases: 2
-  total_plans: 10
+  total_plans: 21
-  completed_plans: 10
+  completed_plans: 11
  percent: 33
 ---
 # Project State
 **Project:** DocuVault
-**Status:** Phase complete — ready for verification
+**Status:** Ready to execute
 **Last Updated:** 2026-06-22
 ## Current Position
-Phase: 12.1 (fix-nextcloud-root-listing-and-sync-visibility) — EXECUTING
+Phase: 13 (virtual-local-cloud-operations) — EXECUTING
-Plan: 4 of 4
+Plan: 2 of 11
-Status: Phase complete — ready for verification
+Status: Ready to execute
-Last activity: 2026-06-22 — Phase 12.1 execution started
+Last activity: 2026-06-22 — Phase 13 execution started
 ## Phase Status
@@ -93,7 +96,7 @@ _Updated at each phase transition._
 | Field | Value |
 |---|---|
-| Last session | 2026-06-22T07:48:17.172Z |
+| Last session | 2026-06-22T16:02:44.485Z |
 | Next action | Execute Phase 12.1 Plan 01 |
 | Pending decisions | None |
 | Resume file | None |
@@ -0,0 +1,140 @@
 ---
 phase: "13"
 plan: "01"
 subsystem: cloud-testing
 tags: [tdd, red-phase, cloud-mutations, provider-contracts, audit, reconnect]
 dependency_graph:
  requires:
    - "12.1-04 (provider contract foundation)"
    - "backend/storage/cloud_base.py (CloudResourceAdapter interface)"
    - "backend/tests/test_cloud_security.py (security negative pattern)"
  provides:
    - "Red backend test contracts for all Phase 13 mutation, reconnect, and audit semantics"
    - "Four-provider mutable-operation parity RED coverage"
  affects:
    - "backend/tests/test_cloud_mutations.py"
    - "backend/tests/test_cloud_reconnect.py"
    - "backend/tests/test_cloud_audit.py"
    - "backend/tests/test_cloud_backends.py"
    - "backend/tests/test_cloud_provider_contract.py"
 tech_stack:
  added: []
  patterns:
    - "TDD RED phase — all new tests fail against current codebase"
    - "Typed kind/reason result body contract (normalized dict pattern)"
    - "IDOR/admin/credential exclusion pattern from test_cloud_security.py"
    - "Parametrized four-provider contract pattern from test_cloud_provider_contract.py"
 key_files:
  created:
    - "backend/tests/test_cloud_mutations.py"
    - "backend/tests/test_cloud_reconnect.py"
    - "backend/tests/test_cloud_audit.py"
  modified:
    - "backend/tests/test_cloud_backends.py"
    - "backend/tests/test_cloud_provider_contract.py"
 decisions:
  - "Typed kind/reason dict bodies required for every mutation outcome — frontend must never infer from raw HTTP status alone"
  - "Reconnect patches existing CloudConnection row in-place (CONN-01) — creating new row breaks item identity"
  - "OneDrive _refresh_token must return new credentials dict for service-layer handoff (CONN-02)"
  - "Audit rows are metadata-only: no access_token, refresh_token, credentials_enc, or raw file bytes allowed in metadata_ JSONB"
  - "Google Drive full 'drive' scope required for Phase 13 (D-17) — 'drive.file' restricts to app-created items only"
  - "Conflict normalization happens inside each adapter — no router or service layer pattern-matches raw provider errors"
 metrics:
  duration: "8m"
  completed_date: "2026-06-22"
  tasks_completed: 2
  files_created: 3
  files_modified: 2
  tests_added_task1: 46
  tests_added_task2_new: 128
 status: complete
 ---
 # Phase 13 Plan 01: Red Backend Contract Suites Summary
 **One-liner:** Red TDD contracts for Phase 13 cloud mutation, reconnect, credential-refresh persistence, and audit-secrecy semantics across all four providers.
 ## What Was Built
 ### Task 1 — Red API and Audit Contracts (3 new test files)
 **`backend/tests/test_cloud_mutations.py`** (commit: efb5964)
 Failing integration tests for the Phase 13 mutation endpoint contracts:
 - Open/preview: authorized-download-only, binary-only preview (D-02, D-18), IDOR block, credential exclusion
 - Upload: typed `{kind: 'conflict', reason: 'name_collision'}` body required — no silent overwrite (D-03)
 - Create folder: typed 201 success body with `provider_item_id` and `kind='folder'`; auto-name on collision (D-05)
 - Rename: typed success body with `kind` field; stale-etag returns `{kind: 'stale', reason: 'item_changed'}` (D-07)
 - Move: same-connection success; `{kind: 'invalid_destination'}` for self-move and cross-connection (D-08, D-09)
 - Delete: `{kind: 'deleted', reason: 'trashed'|'permanent'}` typed result (D-11); IDOR block; credential exclusion
 - Connection-state bodies: `{kind: 'offline'}` for transient outage, `{kind: 'reauth_required'}` for auth failure (D-13, D-15)
 - `{kind: 'unsupported_operation'}` for read-only providers (D-18)
 **`backend/tests/test_cloud_reconnect.py`** (commit: efb5964)
 Failing integration tests for CONN-01 through CONN-03 and D-12 through D-16:
 - CONN-01: reconnect patches the existing row — row count must not increase; UUID must not change
 - CONN-02: refreshed credentials are encrypted and stored in `credentials_enc`; no plaintext tokens in the blob
 - CONN-03: reconnect response never contains `access_token`, `refresh_token`, `credentials_enc`, or `client_secret`
 - D-12/D-13: explicit health endpoint (`GET /health`) with typed status; explicit test action (`POST /test`)
 - D-14: reconnect preserves cached CloudItems as stale — `deleted_at` must remain null
 - D-15: transient provider outage must not erase credentials or delete cached metadata
 - D-16: disconnect removes `credentials_enc` and all connection-scoped CloudItems; foreign-user IDOR blocked
 **`backend/tests/test_cloud_audit.py`** (commit: efb5964)
 Failing audit secrecy and accuracy tests (T-13-02, T-13-05):
 - Every successful operation (reconnect, open, upload, create-folder, rename, move, delete) must write a typed audit row
 - Metadata-only rule: no `access_token`, `refresh_token`, `credentials_enc`, raw token prefixes (`ya29.`, `1//`) in metadata_
 - False-event prevention: conflicted/rejected operations must NOT write false success audit rows
 - Admin audit log viewer must scrub any credential leak in existing rows (defense-in-depth)
 - No base64-encoded binary content in audit metadata (raw file bytes never in audit payloads)
 ### Task 2 — Provider Contract Extensions (2 modified test files)
 **`backend/tests/test_cloud_backends.py`** (commit: fd6b561)
 Added four provider-specific RED classes for mutable operations:
 - `TestGoogleDriveMutableContract`: create/rename/move/delete/upload async assertions; delete normalizes `{kind: 'deleted', reason: 'trashed'|'permanent'}`; create-folder collision returns `{kind: 'conflict'}`; rename stale etag returns `{kind: 'stale'}`; D-17 expanded Drive scope assertion (`SCOPES` class attribute required)
 - `TestOneDriveMutableContract`: same method assertions; CONN-02 `_refresh_token` must return new credentials dict for service-layer handoff; delete returns `{kind: 'deleted', reason: 'permanent'}` (no Graph trash API); nextLink SSRF guard (follow only `graph.microsoft.com`)
 - `TestNextcloudMutableContract`: method assertions; create-folder SSRF guard for redirect attempts; delete defaults to `permanent`
 - `TestWebDAVMutableContract`: method assertions; delete returns `permanent`; move SSRF validation for Destination header; no `cloud_items` imports in module source (provider-neutral contract)
 **`backend/tests/test_cloud_provider_contract.py`** (commit: fd6b561)
 Added two new Phase 13 contract classes parametrized over all four providers:
 - `TestMutableAdapterContract`: method existence, async enforcement, canonical signatures for `create_folder`, `rename`, `move`, `delete`, `upload_file` — all with caller identity (`connection_id`, `user_id`) where required; no direct `cloud_items` imports in any adapter; no browse-time byte transfer; unsupported capabilities explicitly disclosed in `get_capabilities` (cloud_base.ACTIONS must include mutation verbs)
 - `TestMutableAdapterResultNormalization`: normalized result dict contract, docstring requirement, conflict normalization method requirement
 ## Test Results
 All Phase 13 RED tests fail as expected:
 ```
 test_cloud_mutations.py   — 26 tests, all fail (routes do not exist)
 test_cloud_reconnect.py   — 12 tests, all fail (Phase 13 reconnect/health routes absent)
 test_cloud_audit.py       — 8 tests, all fail (Phase 13 audit writes absent)
 test_cloud_backends.py    — 56 prior tests PASS; Phase 13 RED classes fail on first assertion
 test_cloud_provider_contract.py — prior tests PASS; Phase 13 RED classes fail on method existence
 ```
 ## Deviations from Plan
 None — plan executed exactly as written.
 ## Known Stubs
 None. This plan only creates test files; no implementation code was written.
 ## Threat Flags
 No new network endpoints, auth paths, or schema changes introduced. Test files only.
 ## Self-Check: PASSED
 - `backend/tests/test_cloud_mutations.py` — FOUND
 - `backend/tests/test_cloud_reconnect.py` — FOUND
 - `backend/tests/test_cloud_audit.py` — FOUND
 - Commit efb5964 — FOUND (Task 1)
 - Commit fd6b561 — FOUND (Task 2)
 - All pre-existing tests still pass (56/56 in test_cloud_backends.py prior coverage)