diff --git a/.planning/ROADMAP.md b/.planning/ROADMAP.md index 5262178..fc4c93a 100644 --- a/.planning/ROADMAP.md +++ b/.planning/ROADMAP.md @@ -24,7 +24,7 @@ Before any phase is marked complete: | Phase | Name | Goal | Requirements | |------:|------|------|--------------| | 12 | 6/6 | Complete | 2026-06-21 | -| 13 | 7/11 | In Progress| | +| 13 | 8/11 | In Progress| | | 14 | Selective Analysis and Byte Cache | Analyze selected cloud scopes through cancellable jobs backed by bounded, private, on-demand byte caching | ANALYZE-01..07, CACHE-03..05 | | 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 | | 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 | @@ -80,7 +80,7 @@ Plans: **Depends on:** Phase 12 **Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09 -**Plans:** 7/11 plans executed +**Plans:** 8/11 plans executed **Execution waves:** Wave 0: 13-01 and 13-02 in parallel; Wave 1: 13-03 after 13-01; Wave 2: 13-04 after 13-01 and 13-03; Wave 3: 13-05 after 13-01, 13-03, and 13-04; Wave 4: 13-06 after 13-05; Wave 5: 13-07 and 13-08 in parallel (13-07 after 13-02, 13-04, and 13-06; 13-08 after 13-03, 13-04, and 13-06); Wave 6: 13-09 after 13-08; Wave 7: 13-10 after 13-02, 13-04, 13-07, and 13-09; Wave 8: 13-11 after 13-03 through 13-10. Plans: @@ -92,7 +92,7 @@ Plans: - [x] 13-05-PLAN.md — Implement backend upload provider mechanics, typed route results, and refreshed-credential handoff. - [x] 13-06-PLAN.md — Complete upload reconcile, freshness, and metadata-only audit follow-through before frontend queue wiring. - [x] 13-07-PLAN.md — Wire the shared browser queue and binary-only preview or download fallback through thin cloud view handlers. -- [ ] 13-08-PLAN.md — Implement backend create-folder and rename semantics with collision, stale, and stable-identity safeguards. +- [x] 13-08-PLAN.md — Implement backend create-folder and rename semantics with collision, stale, and stable-identity safeguards. - [ ] 13-09-PLAN.md — Implement backend move and delete semantics with same-connection, disclosure, security, and audit safeguards. - [ ] 13-10-PLAN.md — Finish shared-browser mutation UX, store-backed health behavior, reconnect copy, and the no-probe invariant. - [ ] 13-11-PLAN.md — Run closeout-only docs, versions, full gates, explicit secret scan, and ship-readiness checks. diff --git a/.planning/STATE.md b/.planning/STATE.md index a5dfd90..343c54e 100644 --- a/.planning/STATE.md +++ b/.planning/STATE.md @@ -6,14 +6,14 @@ current_phase: 13 current_phase_name: virtual-local-cloud-operations status: executing stopped_at: Completed 13-07-PLAN.md -last_updated: "2026-06-22T17:45:49.748Z" +last_updated: "2026-06-22T17:53:10.202Z" last_activity: 2026-06-22 last_activity_desc: Phase 13 execution started progress: total_phases: 6 completed_phases: 2 total_plans: 21 - completed_plans: 17 + completed_plans: 18 percent: 33 --- @@ -26,7 +26,7 @@ progress: ## Current Position Phase: 13 (virtual-local-cloud-operations) — EXECUTING -Plan: 8 of 11 +Plan: 9 of 11 Status: Ready to execute Last activity: 2026-06-22 — Phase 13 execution started @@ -58,6 +58,7 @@ Last activity: 2026-06-22 — Phase 13 execution started | Phase 13 P04 | 180m | 2 tasks | 7 files | | Phase 13 P06 | 5m | 2 tasks | 3 files | | Phase 13 P07 | 30m | 2 tasks | 5 files | +| Phase 13 P08 | 20m | 2 tasks | 2 files | ## Accumulated Context @@ -101,7 +102,7 @@ _Updated at each phase transition._ | Field | Value | |---|---| -| Last session | 2026-06-22T17:45:49.742Z | +| Last session | 2026-06-22T17:52:59.873Z | | Next action | Execute Phase 12.1 Plan 01 | | Pending decisions | None | | Resume file | None | @@ -125,3 +126,7 @@ _Updated at each phase transition._ - [Phase ?]: CloudFolderView uses api.* barrel imports so vi.mock intercepts correctly - [Phase ?]: UploadProgress suppressed in cloud mode (v-if) — cloud queue dialogs replace it per D-03/D-04 - [Phase ?]: StorageBrowser emits upload-queue-resolve with typed action; CloudFolderView handles all five resolution paths (keep_both/replace/skip/retry/cancel_all) +- [Phase ?]: Rename collision surfaced to user (no auto-retry) — user chose name explicitly +- [Phase ?]: Create-folder bounded retry up to 5 attempts with keep_both_name counter suffix D-05/D-06 +- [Phase ?]: Stale guard for create-folder and rename calls update_folder_state before returning typed stale body D-07 +- [Phase ?]: Reconcile-before-return for create-folder and rename: upsert_cloud_item + update_folder_state T-13-26 diff --git a/.planning/phases/13-virtual-local-cloud-operations/13-08-SUMMARY.md b/.planning/phases/13-virtual-local-cloud-operations/13-08-SUMMARY.md new file mode 100644 index 0000000..3d5d504 --- /dev/null +++ b/.planning/phases/13-virtual-local-cloud-operations/13-08-SUMMARY.md @@ -0,0 +1,174 @@ +--- +phase: "13" +plan: "08" +subsystem: cloud-operations +status: complete +tags: + - cloud + - create-folder + - rename + - collision-retry + - stale-guard + - reconciliation + - tdd + +dependency_graph: + requires: + - "13-03 (MutableCloudResourceAdapter + mutable provider implementations)" + - "13-04 (operations.py route layer + typed result vocabulary)" + - "13-06 (upload reconcile-before-return pattern)" + provides: + - "Bounded collision retry for create-folder (D-05, D-06)" + - "Stale guard with folder state refresh for create-folder and rename (D-07)" + - "Reconcile-before-return for create-folder and rename success (T-13-26)" + - "10 new behavioral tests in test_cloud_mutations.py" + affects: + - "13-09 through 13-11 (move, delete, audit plans)" + +tech_stack: + added: + - "Bounded collision retry loop in create_cloud_folder route (D-06)" + - "_CREATE_FOLDER_MAX_RETRIES = 5 constant" + - "upsert_cloud_item called from create_cloud_folder success path" + - "upsert_cloud_item called from rename_cloud_item success path" + - "update_folder_state called with warning/create_folder_mutated on create-folder success" + - "update_folder_state called with warning/rename_mutated on rename success" + - "update_folder_state called with warning/stale_listing on stale create-folder or rename" + - "keep_both_name imported from services.cloud_operations for counter suffix generation" + patterns: + - "Reconcile-before-return: upsert + folder invalidation before success response" + - "Bounded retry: up to 5 collision suffix attempts before returning conflict" + - "Stale-guard: folder state invalidation on stale result before returning typed stale body" + - "Non-success bypass: conflict/offline/reauth/stale never reach reconciliation upsert" + +key_files: + modified: + - path: "backend/api/cloud/operations.py" + change: "create_cloud_folder: bounded retry + stale guard + reconcile-before-return; rename_cloud_item: stale guard + reconcile-before-return; new imports" + - path: "backend/tests/test_cloud_mutations.py" + change: "Added 10 behavioral tests (RED then GREEN) for Plan 08" + +decisions: + - "Rename collision surfaced to user (they chose the name explicitly) — no auto-retry for rename" + - "Create-folder retry uses keep_both_name counter suffix: 'Projects (1)', 'Projects (2)', etc." + - "Stale guard for both operations uses update_folder_state with warning/stale_listing error_code" + - "Reconciliation pattern follows Plan 06 upload pattern: upsert + folder state + session.commit in one atomic transaction" + - "rename reconcile resolves parent_ref from existing CloudItem (if present) to invalidate correct folder" + +metrics: + duration: "~20 minutes" + completed: "2026-06-22" + tasks_completed: 2 + tasks_planned: 2 + files_changed: 2 + files_created: 0 + tests_added: 10 + tests_passing: 755 +--- + +# Phase 13 Plan 08: Create-Folder and Rename Collision, Stale Guard, and Reconciliation Summary + +**One-liner:** Bounded collision retry (D-05/D-06), stale-guard with folder refresh (D-07), and reconcile-before-return reconciliation (T-13-26) for create-folder and rename mutations. + +## Tasks Completed + +| Task | Name | Commit | Key Files | +|------|------|--------|-----------| +| 1 (RED) | Add failing collision retry, stale guard, and provider normalization tests | 9ad9946 | test_cloud_mutations.py | +| 1/2 (GREEN) | Implement bounded retry, stale guard, and reconcile-before-return | aaa63c1 | api/cloud/operations.py | + +## What Was Built + +### Task 1: Collision Retry, Stale Guard, Provider Normalization (GREEN) + +**`create_cloud_folder` in `backend/api/cloud/operations.py`:** + +Added a bounded retry loop (up to `_CREATE_FOLDER_MAX_RETRIES = 5` attempts) around the adapter call: + +```python +for attempt in range(_CREATE_FOLDER_MAX_RETRIES + 1): + result = await adapter.create_folder(parent_ref=..., name=candidate_name, ...) + if kind == MUT_KIND_FOLDER: + # reconcile and return + if kind == MUT_KIND_CONFLICT and attempt < _CREATE_FOLDER_MAX_RETRIES: + candidate_name = keep_both_name(body.name, attempt + 1) # 'Projects (1)', etc. + continue + if kind == MUT_KIND_STALE: + # update_folder_state with warning/stale_listing and return stale body + break +``` + +- D-05/D-06: Collision auto-suffixes `Projects (1)`, `Projects (2)`, etc. via `keep_both_name`; retries up to 5 times before surfacing conflict to client +- D-07: Stale precondition calls `update_folder_state(warning, stale_listing)` before returning typed `{kind: 'stale'}` + +**`rename_cloud_item` in `backend/api/cloud/operations.py`:** + +- D-07: Stale result triggers `update_folder_state(warning, stale_listing)` using the existing CloudItem's `parent_ref` before returning typed `{kind: 'stale'}` +- Rename collision is surfaced directly (user chose the name — no auto-retry) + +### Task 2: Reconcile Create-Folder and Rename Success (GREEN) + +Both `create_cloud_folder` and `rename_cloud_item` now follow the reconcile-before-return pattern established by Plan 06 (upload): + +**create_cloud_folder success path:** +```python +resource = CloudResource(provider_item_id=..., kind="folder", ...) +await upsert_cloud_item(session, user_id=..., resource=resource) +await update_folder_state(session, ..., refresh_state="warning", error_code="create_folder_mutated") +await session.commit() +``` + +**rename_cloud_item success path:** +```python +# Resolve existing item for parent_ref, kind, content_type, size +existing_item = await session.execute(select(CloudItem).where(...)) +resource = CloudResource(provider_item_id=..., name=resolved_name, ...) +await upsert_cloud_item(session, user_id=..., resource=resource) +await update_folder_state(session, ..., refresh_state="warning", error_code="rename_mutated") +await session.commit() +``` + +Key design decisions: +- `upsert_cloud_item` uses `provider_item_id` as stable identity key — existing rows are updated in place, preserving the DocuVault UUID +- `update_folder_state` invalidates the parent folder (not the item itself) so the next browse triggers a provider re-list +- Non-success paths (conflict, offline, reauth, stale) never reach the upsert/folder-state calls — preserving T-13-26 (no phantom metadata mutations on failure) + +## Deviations from Plan + +### Auto-fixed Issues + +None. The plan was executed exactly as written. + +**Scope note:** The plan listed `backend/services/cloud_operations.py` and `backend/services/cloud_items.py` as files modified. In practice, neither was modified — the existing `upsert_cloud_item`, `update_folder_state`, and `keep_both_name` helpers were sufficient. The route layer in `operations.py` consumed them directly via imports, consistent with the CLAUDE.md shared module map. + +## Test Results + +``` +755 passed, 17 skipped, 4 deselected, 12 xfailed +``` + +- `test_cloud_mutations.py`: 10 new tests, all pass +- `test_cloud_backends.py`: unchanged, all pass +- `test_cloud_provider_contract.py`: unchanged, all pass +- Full backend suite: no regressions + +## Known Stubs + +None. Create-folder and rename reconciliation are fully functional. The reconcile-before-return pattern is complete for create-folder and rename (analogous to Plan 06 for upload). + +## Threat Flags + +No new security surfaces introduced. T-13-24, T-13-25, T-13-26 are mitigated: + +- **T-13-24 (collision handling):** Bounded retry loop enforces D-06; tests verify retry behavior and collision exhaustion. +- **T-13-25 (stale mutation safety):** Stale guards stop mutations, refresh affected folder state, require retry — never forcing a stale create-folder or rename. +- **T-13-26 (identity reconciliation):** Successful create-folder and rename route through `upsert_cloud_item` before success is returned. + +## Self-Check: PASSED + +- FOUND: `backend/api/cloud/operations.py` with bounded retry, stale guards, and reconcile-before-return for both create_cloud_folder and rename_cloud_item +- FOUND: `backend/tests/test_cloud_mutations.py` with 10 new behavioral tests +- FOUND: `.planning/phases/13-virtual-local-cloud-operations/13-08-SUMMARY.md` +- FOUND commit: `9ad9946` (Task 1 RED) +- FOUND commit: `aaa63c1` (Task 1/2 GREEN) +- Full suite: 755 passed, 17 skipped, 4 deselected, 12 xfailed