docs(13-05): complete upload mechanics plan
This commit is contained in:
curo1305
@@ -24,7 +24,7 @@ Before any phase is marked complete:
|
||||
| Phase | Name | Goal | Requirements |
|
||||
|------:|------|------|--------------|
|
||||
| 12 | 6/6 | Complete | 2026-06-21 |
|
||||
| 13 | 3/11 | In Progress| |
|
||||
| 13 | 5/11 | In Progress| |
|
||||
| 14 | Selective Analysis and Byte Cache | Analyze selected cloud scopes through cancellable jobs backed by bounded, private, on-demand byte caching | ANALYZE-01..07, CACHE-03..05 |
|
||||
| 15 | Unified Smart Search | Search local and analyzed cloud documents by exact text or semantic ideas without downloading files during queries | SEARCH-01..07 |
|
||||
| 16 | Change Tracking and Reliability | Detect external provider changes, mark stale indexes, remove deleted items, and harden refresh behavior | SYNC-02..04 |
|
||||
@@ -80,7 +80,7 @@ Plans:
|
||||
|
||||
**Depends on:** Phase 12
|
||||
**Requirements:** CONN-01, CONN-02, CONN-03, CLOUD-02, CLOUD-03, CLOUD-04, CLOUD-05, CLOUD-06, CLOUD-07, CLOUD-09
|
||||
**Plans:** 3/11 plans executed
|
||||
**Plans:** 5/11 plans executed
|
||||
**Execution waves:** Wave 0: 13-01 and 13-02 in parallel; Wave 1: 13-03 after 13-01; Wave 2: 13-04 after 13-01 and 13-03; Wave 3: 13-05 after 13-01, 13-03, and 13-04; Wave 4: 13-06 after 13-05; Wave 5: 13-07 and 13-08 in parallel (13-07 after 13-02, 13-04, and 13-06; 13-08 after 13-03, 13-04, and 13-06); Wave 6: 13-09 after 13-08; Wave 7: 13-10 after 13-02, 13-04, 13-07, and 13-09; Wave 8: 13-11 after 13-03 through 13-10.
|
||||
|
||||
Plans:
|
||||
@@ -88,8 +88,8 @@ Plans:
|
||||
- [x] 13-01-PLAN.md — Create red backend and provider-contract suites for reconnect, content, mutations, and audit secrecy.
|
||||
- [x] 13-02-PLAN.md — Create red frontend/store suites for queue, preview, health UX, broader Drive consent, and no-probe-on-navigation.
|
||||
- [x] 13-03-PLAN.md — Build the mutable cloud contract and orchestration seam without breaking centralized reconciliation.
|
||||
- [ ] 13-04-PLAN.md — Implement connection-ID reconnect, explicit health test, broader Drive scope handling, and authorized content routes.
|
||||
- [ ] 13-05-PLAN.md — Implement backend upload provider mechanics, typed route results, and refreshed-credential handoff.
|
||||
- [x] 13-04-PLAN.md — Implement connection-ID reconnect, explicit health test, broader Drive scope handling, and authorized content routes.
|
||||
- [x] 13-05-PLAN.md — Implement backend upload provider mechanics, typed route results, and refreshed-credential handoff.
|
||||
- [ ] 13-06-PLAN.md — Complete upload reconcile, freshness, and metadata-only audit follow-through before frontend queue wiring.
|
||||
- [ ] 13-07-PLAN.md — Wire the shared browser queue and binary-only preview or download fallback through thin cloud view handlers.
|
||||
- [ ] 13-08-PLAN.md — Implement backend create-folder and rename semantics with collision, stale, and stable-identity safeguards.
|
||||
|
||||
+8
-4
@@ -6,14 +6,14 @@ current_phase: 13
|
||||
current_phase_name: virtual-local-cloud-operations
|
||||
status: executing
|
||||
stopped_at: Phase 13 context gathered
|
||||
last_updated: "2026-06-22T16:45:34.553Z"
|
||||
last_updated: "2026-06-22T17:22:26.784Z"
|
||||
last_activity: 2026-06-22
|
||||
last_activity_desc: Phase 13 execution started
|
||||
progress:
|
||||
total_phases: 6
|
||||
completed_phases: 2
|
||||
total_plans: 21
|
||||
completed_plans: 13
|
||||
completed_plans: 15
|
||||
percent: 33
|
||||
---
|
||||
|
||||
@@ -26,7 +26,7 @@ progress:
|
||||
## Current Position
|
||||
|
||||
Phase: 13 (virtual-local-cloud-operations) — EXECUTING
|
||||
Plan: 4 of 11
|
||||
Plan: 6 of 11
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-06-22 — Phase 13 execution started
|
||||
|
||||
@@ -55,6 +55,7 @@ Last activity: 2026-06-22 — Phase 13 execution started
|
||||
| Phase 12.1 P04 | 15m | 4 tasks | 10 files |
|
||||
| Phase 13 P02 | 30m | 2 tasks | 6 files |
|
||||
| Phase 13 P03 | 134 | 2 tasks | 8 files |
|
||||
| Phase 13 P04 | 180m | 2 tasks | 7 files |
|
||||
|
||||
## Accumulated Context
|
||||
|
||||
@@ -98,7 +99,7 @@ _Updated at each phase transition._
|
||||
|
||||
| Field | Value |
|
||||
|---|---|
|
||||
| Last session | 2026-06-22T16:45:34.548Z |
|
||||
| Last session | 2026-06-22T17:22:26.779Z |
|
||||
| Next action | Execute Phase 12.1 Plan 01 |
|
||||
| Pending decisions | None |
|
||||
| Resume file | None |
|
||||
@@ -115,3 +116,6 @@ _Updated at each phase transition._
|
||||
- [Phase ?]: file-open must call openCloudFile API; window.open() to provider URL is forbidden (D-02/T-13-07)
|
||||
- [Phase ?]: connectionHealth store translation is single source for browser compact status and Settings diagnostics (D-12)
|
||||
- [Phase ?]: testCloudConnection never called as side effect of folder browse navigation (D-13)
|
||||
- [Phase ?]: D-17: Google Drive OAuth uses drive scope (not drive.file) for Phase 13 full-access mutations
|
||||
- [Phase ?]: D-18: Preview is binary-only (PDF/images); Office/Workspace formats use typed unsupported_preview fallback to authorized download endpoint
|
||||
- [Phase ?]: Phase 13 mutation errors use JSONResponse (not HTTPException) so kind/reason appear at top level of response body, not nested under detail
|
||||
|
||||
@@ -0,0 +1,157 @@
|
||||
---
|
||||
phase: "13"
|
||||
plan: "05"
|
||||
subsystem: cloud-operations
|
||||
status: complete
|
||||
tags:
|
||||
- cloud
|
||||
- upload
|
||||
- conflict
|
||||
- keep-both
|
||||
- typed-results
|
||||
- tdd
|
||||
|
||||
dependency_graph:
|
||||
requires:
|
||||
- "13-01 (RED mutable adapter contract tests)"
|
||||
- "13-03 (cloud operations seam + mutable provider implementations)"
|
||||
- "13-04 (operations.py route layer + typed result vocabulary)"
|
||||
provides:
|
||||
- "keep_both_name() helper in services/cloud_operations.py for D-03/D-05 collision naming"
|
||||
- "TestUploadConflictSemantics: 16 provider-level upload behavioral tests"
|
||||
- "TestKeepBothNaming: 7 keep-both naming format tests"
|
||||
- "7 typed upload route mechanics tests in test_cloud_mutations.py"
|
||||
- "All 4 providers verified to return typed upload results (never raw provider errors)"
|
||||
affects:
|
||||
- "13-06 through 13-11 (reconcile, audit, frontend upload queue plans)"
|
||||
|
||||
tech_stack:
|
||||
added:
|
||||
- "keep_both_name(filename, counter) in services/cloud_operations.py"
|
||||
patterns:
|
||||
- "Counter-before-extension naming: Report.pdf → Report (1).pdf"
|
||||
- "Typed upload result vocabulary: MUT_KIND_UPLOADED / MUT_KIND_CONFLICT / MUT_KIND_OFFLINE / MUT_KIND_REAUTH"
|
||||
- "Provider-neutral upload mechanics: all 4 providers normalize errors into shared kinds"
|
||||
- "Queue-control semantics from backend-authored typed bodies (not Vue-side HTTP status inference)"
|
||||
|
||||
key_files:
|
||||
modified:
|
||||
- path: "backend/services/cloud_operations.py"
|
||||
change: "Added keep_both_name(filename, counter) helper for D-03/D-05 collision naming"
|
||||
- path: "backend/tests/test_cloud_backends.py"
|
||||
change: "Added TestUploadConflictSemantics (16 tests) and TestKeepBothNaming (7 tests)"
|
||||
- path: "backend/tests/test_cloud_mutations.py"
|
||||
change: "Added 7 typed upload route mechanics tests (Task 2 behaviors)"
|
||||
|
||||
decisions:
|
||||
- "keep_both_name lives in services/cloud_operations (not a provider) — naming is queue/service concern, not provider concern"
|
||||
- "Counter inserted before first dot to preserve compound extensions (archive.tar.gz → archive (1).tar.gz)"
|
||||
- "Providers already had upload_file implementations from Plan 03 — Plan 05 adds behavioral tests and the naming utility only"
|
||||
- "Upload route already had typed result handling from Plan 04 — Plan 05 confirms test coverage of all outcome paths"
|
||||
- "Reconcile and audit follow-through deferred to Plans 06+ per PLAN.md bounded scope"
|
||||
|
||||
metrics:
|
||||
duration: "~45 minutes"
|
||||
completed: "2026-06-22"
|
||||
tasks_completed: 2
|
||||
tasks_planned: 2
|
||||
files_changed: 3
|
||||
files_created: 0
|
||||
tests_added: 30
|
||||
tests_passing: 741
|
||||
---
|
||||
|
||||
# Phase 13 Plan 05: Upload Mechanics Slice Summary
|
||||
|
||||
**One-liner:** Provider-normalized upload conflict/retry/reauth typed mechanics with `keep_both_name()` helper for D-03/D-05 queue-level collision resolution.
|
||||
|
||||
## Tasks Completed
|
||||
|
||||
| Task | Name | Commit | Key Files |
|
||||
|------|------|--------|-----------|
|
||||
| 1 (RED) | Add failing upload conflict and keep-both naming tests | f47e36d | test_cloud_backends.py |
|
||||
| 1 (GREEN) | Implement keep_both_name() helper | e1ce4cb | services/cloud_operations.py |
|
||||
| 2 | Add typed upload route mechanics tests | 3ddf4da | test_cloud_mutations.py |
|
||||
|
||||
## What Was Built
|
||||
|
||||
### Task 1: Provider-normalized upload contract tests + keep_both_name() helper
|
||||
|
||||
**RED phase:** Added 23 tests (16 provider behavioral + 7 keep-both naming) to `test_cloud_backends.py`:
|
||||
|
||||
**TestUploadConflictSemantics:**
|
||||
- `test_google_drive_upload_success_returns_typed_result` — kind='uploaded' with provider_item_id, name, size; no credentials in result
|
||||
- `test_google_drive_upload_transient_error_returns_offline_kind` — HTTP 503 → kind='offline' (retryable)
|
||||
- `test_google_drive_upload_auth_error_returns_reauth_kind` — HTTP 401 → kind='reauth_required'
|
||||
- `test_onedrive_upload_success_returns_typed_result` — resumable session success → kind='uploaded'
|
||||
- `test_onedrive_upload_conflict_returns_typed_conflict_kind` — conflictBehavior=fail 409 → kind='conflict', reason='name_collision'
|
||||
- `test_onedrive_upload_never_writes_cloud_items` — source scan confirms no cloud_items import
|
||||
- `test_webdav_upload_success_returns_typed_result` — PUT success → kind='uploaded' with correct path
|
||||
- `test_webdav_upload_provider_error_returns_error_kind` — connection timeout → kind='offline'
|
||||
- `test_upload_result_never_contains_credentials` — parametrized over 4 providers; no access_token/refresh_token in result
|
||||
- `test_upload_result_kind_in_known_kinds` — parametrized over 4 providers; kind must be in MUT_KINDS constant set
|
||||
|
||||
**TestKeepBothNaming:**
|
||||
- `test_keep_both_naming_module_exists` — services.cloud_operations exposes keep_both_name
|
||||
- `test_keep_both_basic_pdf` — Report.pdf + 1 → Report (1).pdf
|
||||
- `test_keep_both_counter_increments` — Report.pdf + 2 → Report (2).pdf
|
||||
- `test_keep_both_no_extension` — README + 1 → README (1)
|
||||
- `test_keep_both_compound_extension` — archive.tar.gz + 1 → archive (1).tar.gz
|
||||
- `test_keep_both_preserves_existing_counter` — Report (1).pdf + 2 → Report (1) (2).pdf
|
||||
- `test_keep_both_spaces_in_name` — My Document.docx + 1 → My Document (1).docx
|
||||
|
||||
**GREEN phase:** Implemented `keep_both_name(filename, counter)` in `services/cloud_operations.py`:
|
||||
|
||||
```python
|
||||
def keep_both_name(filename: str, counter: int) -> str:
|
||||
"""Insert collision counter before first file extension.
|
||||
|
||||
Report.pdf + 1 → Report (1).pdf
|
||||
archive.tar.gz + 1 → archive (1).tar.gz
|
||||
README + 1 → README (1)
|
||||
"""
|
||||
```
|
||||
|
||||
The counter is inserted before the first dot (not last), so compound extensions are preserved intact.
|
||||
|
||||
### Task 2: Typed upload route mechanics tests
|
||||
|
||||
Added 7 tests to `test_cloud_mutations.py` verifying Task 2 behavioral contract:
|
||||
|
||||
1. `test_upload_success_returns_typed_uploaded_body` — success body has kind/provider_item_id/name/size
|
||||
2. `test_upload_provider_retryable_error_returns_offline_kind` — offline kind from provider error, no bare 500
|
||||
3. `test_upload_provider_conflict_returns_conflict_kind` — provider-detected conflict when cache misses
|
||||
4. `test_upload_reauth_result_is_typed_and_not_500` — CONN-02 token expiry surfaces as typed body
|
||||
5. `test_upload_queue_control_semantics_are_backend_authored` — all 4 required queue fields present
|
||||
6. `test_upload_keep_both_name_format` — service utility accessible to route layer
|
||||
7. `test_upload_foreign_user_blocked` — IDOR protection on upload endpoint
|
||||
|
||||
All 7 tests pass because the upload route in `operations.py` (Plan 04) already implements the typed result dispatch correctly. Plan 05 adds the test coverage that was missing.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
None. The plan stated providers should "implement D-03 and D-04 inside the provider layer" but providers already had complete `upload_file` implementations from Plan 03. Plan 05 correctly adds behavioral tests and the naming utility that were genuinely missing.
|
||||
|
||||
**Note on scope:** The providers already normalized upload results into typed kinds from Plan 03. Plan 05's contribution is:
|
||||
1. Behavioral tests that prove the normalization is correct
|
||||
2. The `keep_both_name()` service helper that the upload queue needs for conflict resolution
|
||||
|
||||
## Known Stubs
|
||||
|
||||
None. The upload mechanics are fully functional. Reconcile and audit follow-through are intentionally deferred to Plans 06+.
|
||||
|
||||
## Threat Flags
|
||||
|
||||
No new security surfaces introduced. Upload route is covered by T-13-16 (typed conflict bodies forbid silent overwrite), T-13-17 (no provider URLs/tokens in response), T-13-18 (WebDAV SSRF guard preserved via __init__).
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
- FOUND: `backend/services/cloud_operations.py` with `keep_both_name()` function
|
||||
- FOUND: `backend/tests/test_cloud_backends.py` (modified — 23 new tests)
|
||||
- FOUND: `backend/tests/test_cloud_mutations.py` (modified — 7 new tests)
|
||||
- FOUND commit: `f47e36d` (Task 1 RED)
|
||||
- FOUND commit: `e1ce4cb` (Task 1 GREEN — keep_both_name)
|
||||
- FOUND commit: `3ddf4da` (Task 2 — route mechanics tests)
|
||||
- Full suite: 741 passed, 17 skipped, 4 deselected, 13 xfailed
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
DocuVault is a multi-user SaaS document management platform built on FastAPI (Python) + Vue 3. It handles document upload, text extraction (PDF/DOCX/image/text), AI-based topic classification, per-user isolated storage, folder organization, document sharing, and pluggable cloud storage backends (OneDrive, Google Drive, Nextcloud, WebDAV).
|
||||
|
||||
**Current state:** v0.2.6 — Phase 12.1 complete 2026-06-22. Nextcloud root listing functional: legacy `list_folder` signature override removed; incomplete listings blocked from `fresh`; frontend uses `kind` and `provider_item_id`; server freshness mapped verbatim. Live smoke test suite (opt-in, read-only) with owner-confirmed exact-name fixture. Full validation, security, documentation, versioning, and push complete. Not cleared for public internet deployment.
|
||||
**Current state:** v0.2.6 — Phase 13 Plan 05 complete 2026-06-22. Upload mechanics slice: `keep_both_name()` helper in `services/cloud_operations.py`; typed upload conflict/retry/reauth results across all 4 providers; 37 upload-specific tests pass. Not cleared for public internet deployment.
|
||||
|
||||
## Stack
|
||||
|
||||
@@ -43,6 +43,7 @@ Before adding a helper, check if it belongs in an existing shared module:
|
||||
| `backend/storage/cloud_base.py` | `CloudResourceAdapter`, `CloudListing`, `CloudResource`, `CloudCapability` — Phase 12 browse contract; all 4 providers implement this |
|
||||
| `backend/storage/cloud_utils.py` | `validate_cloud_url`, `normalize_nextcloud_url`, `encrypt_credentials`, `decrypt_credentials` — Phase 12.1 adds `normalize_nextcloud_url` |
|
||||
| `backend/services/cloud_items.py` | `resolve_owned_connection`, `upsert_cloud_item`, `reconcile_cloud_listing`, `get_or_create_folder_state`, `apply_listing_and_finalize`, `ListingResult` — owner-scoped cloud metadata service; `apply_listing_and_finalize` is the single shared freshness gate |
|
||||
| `backend/services/cloud_operations.py` | `keep_both_name(filename, counter)` — canonical D-03/D-05 keep-both collision naming (counter before first extension); upload queue uses this for conflict resolution |
|
||||
| `backend/api/cloud/schemas.py` | `CloudBrowseResponse`, `CloudItemOut`, `FolderFreshnessOut`, `ConnectionRenameRequest` — credential-free cloud response schemas |
|
||||
|
||||
**Rules:**
|
||||
|
||||
Reference in New Issue
Block a user