docs(12): create phase plan
This commit is contained in:
curo1305
+7
-8
@@ -2,14 +2,13 @@
|
||||
gsd_state_version: 1.0
|
||||
milestone: v0.3
|
||||
milestone_name: Reimagining Cloud Storage integration
|
||||
current_phase: 12
|
||||
status: planning
|
||||
last_updated: "2026-06-17"
|
||||
last_activity: 2026-06-17 -- v0.3 roadmap proposed
|
||||
status: executing
|
||||
last_updated: "2026-06-18T20:29:32.914Z"
|
||||
last_activity: 2026-06-18 -- Phase 12 planning complete
|
||||
progress:
|
||||
total_phases: 5
|
||||
completed_phases: 0
|
||||
total_plans: 0
|
||||
total_plans: 4
|
||||
completed_plans: 0
|
||||
percent: 0
|
||||
---
|
||||
@@ -17,15 +16,15 @@ progress:
|
||||
# Project State
|
||||
|
||||
**Project:** DocuVault
|
||||
**Status:** v0.3 milestone planning — roadmap awaiting approval
|
||||
**Status:** Ready to execute
|
||||
**Last Updated:** 2026-06-17
|
||||
|
||||
## Current Position
|
||||
|
||||
Phase: 12 of 16 — Cloud Resource Foundation
|
||||
Plan: —
|
||||
Status: Roadmap proposed; phase not yet planned
|
||||
Last activity: 2026-06-17 — v0.3 roadmap proposed
|
||||
Status: Ready to execute
|
||||
Last activity: 2026-06-18 -- Phase 12 planning complete
|
||||
|
||||
## Phase Status
|
||||
|
||||
|
||||
@@ -0,0 +1,169 @@
|
||||
---
|
||||
phase: "12"
|
||||
plan: "01"
|
||||
type: execute
|
||||
wave: 1
|
||||
depends_on: []
|
||||
files_modified:
|
||||
- backend/migrations/versions/0006_cloud_resource_foundation.py
|
||||
- backend/db/models.py
|
||||
- backend/storage/cloud_base.py
|
||||
- backend/services/cloud_items.py
|
||||
- backend/api/schemas.py
|
||||
- backend/tests/test_cloud_capabilities.py
|
||||
- backend/tests/test_cloud_items.py
|
||||
autonomous: true
|
||||
requirements:
|
||||
- CONN-04
|
||||
- CACHE-01
|
||||
- CACHE-02
|
||||
- SYNC-01
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "D-02/D-03: connection UUID, not provider slug, is the durable identity and multiple same-provider accounts remain distinct"
|
||||
- "D-08/D-09/D-10: every normalized action reports supported, unsupported, or temporarily_unavailable with a stable reason and safe message"
|
||||
- "D-18: provider metadata size never changes document quota and no cloud bytes are persisted by metadata operations"
|
||||
- "A cloud item keeps one DocuVault UUID across provider rename, move, and metadata refresh"
|
||||
- "Cloud items and folder freshness rows cannot cross user or connection boundaries"
|
||||
artifacts:
|
||||
- path: "backend/storage/cloud_base.py"
|
||||
provides: "Normalized cloud resource, capability, listing, and adapter contracts"
|
||||
contains: "class CloudResourceAdapter"
|
||||
- path: "backend/migrations/versions/0006_cloud_resource_foundation.py"
|
||||
provides: "cloud_items and cloud_folder_states tables plus connection identity support"
|
||||
contains: "cloud_items"
|
||||
- path: "backend/services/cloud_items.py"
|
||||
provides: "Owner-scoped metadata upsert, list, and complete-list reconciliation"
|
||||
contains: "reconcile_cloud_listing"
|
||||
- path: "backend/tests/test_cloud_items.py"
|
||||
provides: "Stable identity, ownership, reconciliation, and metadata-only regression tests"
|
||||
contains: "test_same_provider_item_isolated_by_connection"
|
||||
key_links:
|
||||
- from: "backend/services/cloud_items.py"
|
||||
to: "backend/db/models.py"
|
||||
via: "owner and connection scoped SQLAlchemy statements"
|
||||
pattern: "CloudItem.user_id == user_id"
|
||||
- from: "backend/storage/cloud_base.py"
|
||||
to: "backend/tests/test_cloud_capabilities.py"
|
||||
via: "fixed action/state/reason contract tests"
|
||||
pattern: "temporarily_unavailable"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Create the provider-neutral cloud resource contract and durable metadata foundation. This plan establishes stable connection/item identity, normalized capability semantics, owner-scoped persistence, and metadata-only quota invariants before any provider or UI integration depends on them.
|
||||
|
||||
Scope fence: do not implement cloud mutations, byte download/cache, analysis jobs, semantic search, or broader OAuth consent.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/PROJECT.md
|
||||
@.planning/REQUIREMENTS.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-CONTEXT.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-RESEARCH.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-PATTERNS.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Define and test normalized cloud resource capabilities</name>
|
||||
<files>backend/storage/cloud_base.py, backend/tests/test_cloud_capabilities.py</files>
|
||||
<read_first>
|
||||
- backend/storage/base.py — existing byte-oriented ABC that must remain separate
|
||||
- backend/storage/exceptions.py — canonical cloud exception
|
||||
- .planning/phases/12-cloud-resource-foundation/12-CONTEXT.md — D-06 through D-10 capability decisions
|
||||
- .planning/phases/12-cloud-resource-foundation/12-RESEARCH.md — capability levels and provider notes
|
||||
</read_first>
|
||||
<action>
|
||||
Add a cloud-resource contract separate from `StorageBackend`. Define fixed action keys for browse, open, preview, upload, create_folder, rename, move, delete, and change_tracking; capability states `supported`, `unsupported`, `temporarily_unavailable`; stable reason codes including provider_unsupported, insufficient_scope, read_only, reauth_required, offline, and item_restricted; immutable normalized `CloudCapability`, `CloudResource`, and complete/paginated `CloudListing` values; and an abstract `CloudResourceAdapter` with read-only `list_folder`, `get_capabilities`, and optional item-capability merge behavior. Messages are controlled adapter output, credentials are never fields, and the interface contains no mutation methods until Phase 13. Add unit tests for vocabulary completeness, state validation, connection defaults plus item overrides, exact reason/message preservation, and a fake adapter proving capability discovery invokes no put/delete/move/rename method.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- `cloud_base.py` defines all nine action keys and exactly three capability states
|
||||
- normalized resources support provider ID, parent reference, type, size, MIME type, modified time, etag/version, and item capabilities
|
||||
- no method in the Phase 12 adapter contract mutates provider content
|
||||
- `pytest -q tests/test_cloud_capabilities.py` exits 0
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud_capabilities.py</automated></verify>
|
||||
<done>Normalized read-only capability/resource contract exists and its focused tests pass.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Add durable owner-scoped cloud metadata schema</name>
|
||||
<files>backend/migrations/versions/0006_cloud_resource_foundation.py, backend/db/models.py, backend/tests/test_cloud_items.py</files>
|
||||
<read_first>
|
||||
- backend/migrations/versions/0005_system_settings.py — current Alembic revision and typed-table style
|
||||
- backend/db/models.py — CloudConnection, Document, Folder, Quota ownership/index patterns
|
||||
- backend/tests/conftest.py — SQLite compatibility and integration conventions
|
||||
- .planning/phases/12-cloud-resource-foundation/12-RESEARCH.md — recommended fields and uniqueness boundary
|
||||
</read_first>
|
||||
<action>
|
||||
Create revision `0006` with `down_revision="0005"`. Extend `cloud_connections` only as needed for distinct same-provider accounts and user-defined display names; do not add `(user_id, provider)` uniqueness. Add `cloud_items` with UUID id, required user_id/connection_id FKs using CASCADE, provider_item_id, nullable parent provider reference/path snapshot, name, file/folder kind, nullable content type, provider-reported size, etag/version/modified_at, last_seen_at/deleted_at, nullable extracted_text, analysis_status, semantic_index_status, and nullable semantic_index_data JSONB reserved for provider-independent search artifacts. Add `cloud_item_topics` as an association from cloud items to existing owner topics without requiring a local `Document` row. Add `cloud_folder_states` for root or parent reference freshness, last successful refresh, refresh state, and safe error code/message. Enforce unique `(connection_id, provider_item_id)` and `(connection_id, parent_ref)` boundaries plus owner/connection/parent indexes. Mirror models and relationships. Tests must show metadata, extracted text, topic links, and semantic data persist with no object key/cache row; identical provider item IDs coexist across connections; foreign-owner queries do not match; cascades work in PostgreSQL integration; and provider size fields do not alter `Quota.used_bytes`.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- migration upgrade and downgrade are structurally reversible and revision chain is 0005 -> 0006
|
||||
- `cloud_items` has explicit user and connection ownership plus unique connection/provider-item identity
|
||||
- root folder freshness can be represented without a CloudItem parent row
|
||||
- no MinIO object key or retained-byte field is required to persist cloud metadata, extracted text, topic links, or semantic index data
|
||||
- focused model tests pass; real PostgreSQL constraint test is marked/configured per project conventions
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud_items.py -k "model or schema or isolation or quota"</automated></verify>
|
||||
<done>Revision 0006 and matching ORM models persist owner-scoped metadata/search state without byte storage or quota mutation.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Implement owner-scoped metadata reconciliation service</name>
|
||||
<files>backend/services/cloud_items.py, backend/tests/test_cloud_items.py</files>
|
||||
<read_first>
|
||||
- backend/db/models.py — models created in Task 2
|
||||
- backend/deps/utils.py — shared UUID parsing stays at router boundary
|
||||
- backend/api/folders.py — owner-scoped SQL patterns only; do not copy private helpers or HTTPException into service
|
||||
- .planning/phases/12-cloud-resource-foundation/12-CONTEXT.md — D-11 through D-14 freshness behavior
|
||||
</read_first>
|
||||
<action>
|
||||
Add service functions to resolve an owned connection by `(connection_id, user_id)`, list durable children by `(user_id, connection_id, parent_ref)`, upsert normalized resources by `(connection_id, provider_item_id)` while preserving CloudItem UUID, and reconcile a listing. Reconciliation updates parent/name/type/size/mtime/etag/version and last_seen_at; it marks missing children deleted only when `CloudListing.complete` is true and the provider refresh succeeded. Failed or partial listings retain rows. Folder-state helpers transition refreshing/fresh/warning, retain last successful timestamp on failure, and store only controlled error codes/messages. Service functions raise domain exceptions or `ValueError`, never FastAPI `HTTPException`. Tests cover rename/move stable identity, complete removal, partial/failure retention, owner isolation, and idempotent repeated reconciliation.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- repeated reconciliation of unchanged metadata creates no duplicate row
|
||||
- rename/move updates metadata without changing the DocuVault CloudItem UUID
|
||||
- incomplete or failed listing cannot mark an unseen child deleted
|
||||
- all item/list/folder-state queries include both owner and connection scope
|
||||
- `pytest -q tests/test_cloud_items.py` exits 0
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud_items.py</automated></verify>
|
||||
<done>Owner-scoped reconciliation is idempotent, stable across rename/move, and safe for failed or partial listings.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
| Threat ID | Threat | Mitigation and evidence |
|
||||
|-----------|--------|-------------------------|
|
||||
| T-12-01 | Connection/item IDOR | Composite owner+connection predicates and negative service tests |
|
||||
| T-12-02 | Same-provider account collision | Connection UUID in uniqueness, indexes, and every cache/persistence boundary |
|
||||
| T-12-06 | Destructive capability probing | Read-only interface and fake-adapter test asserting no mutation calls |
|
||||
| T-12-07 | Partial-list metadata deletion | Deletion reconciliation gated on successful `complete=true` listing |
|
||||
| T-12-09 | Quota corruption | Metadata size never invokes quota service; regression test asserts unchanged usage |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
1. `cd backend && pytest -q tests/test_cloud_capabilities.py tests/test_cloud_items.py`
|
||||
2. Run migration upgrade/downgrade against PostgreSQL integration environment.
|
||||
3. Confirm `rg "HTTPException" backend/services/cloud_items.py` returns no matches.
|
||||
4. Confirm no Phase 12 task writes provider file bytes or MinIO objects.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Normalized provider-neutral capability and resource contracts are test-covered.
|
||||
- Durable cloud item/folder freshness metadata is isolated by user and connection.
|
||||
- Same-provider accounts and same provider item IDs do not collide.
|
||||
- Metadata operations preserve provider-byte source of truth and quota usage.
|
||||
</success_criteria>
|
||||
|
||||
<output>Create `.planning/phases/12-cloud-resource-foundation/12-01-SUMMARY.md` when complete.</output>
|
||||
@@ -0,0 +1,192 @@
|
||||
---
|
||||
phase: "12"
|
||||
plan: "02"
|
||||
type: execute
|
||||
wave: 2
|
||||
depends_on:
|
||||
- "12-01"
|
||||
files_modified:
|
||||
- backend/storage/cloud_backend_factory.py
|
||||
- backend/storage/google_drive_backend.py
|
||||
- backend/storage/onedrive_backend.py
|
||||
- backend/storage/nextcloud_backend.py
|
||||
- backend/storage/webdav_backend.py
|
||||
- backend/api/cloud/__init__.py
|
||||
- backend/api/cloud/connections.py
|
||||
- backend/api/cloud/browse.py
|
||||
- backend/api/cloud/schemas.py
|
||||
- backend/api/cloud.py
|
||||
- backend/tasks/cloud_tasks.py
|
||||
- backend/celery_app.py
|
||||
- backend/tests/test_cloud_backends.py
|
||||
- backend/tests/test_cloud.py
|
||||
- backend/tests/test_cloud_items.py
|
||||
- backend/main.py
|
||||
- frontend/package.json
|
||||
- frontend/package-lock.json
|
||||
- AGENTS.md
|
||||
- README.md
|
||||
autonomous: true
|
||||
requirements:
|
||||
- CONN-04
|
||||
- CLOUD-01
|
||||
- CLOUD-08
|
||||
- CACHE-01
|
||||
- CACHE-02
|
||||
- SYNC-01
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "D-01/D-02/D-03: API lists distinct connection roots and browses by owned connection UUID, including duplicate provider types"
|
||||
- "D-08: capability reasons come from provider metadata, granted permissions, and safe read-only checks only"
|
||||
- "D-11/D-12: previously indexed folders return durable rows promptly and refresh in the background"
|
||||
- "D-13/D-14: successful refresh reconciles silently; failures retain cached rows, warning state, last-success time, and bounded retries"
|
||||
- "D-18/CACHE-01: browse and refresh never download file bytes, write MinIO, or mutate quota"
|
||||
artifacts:
|
||||
- path: "backend/api/cloud/browse.py"
|
||||
provides: "Owner-scoped connection-ID browse response and refresh scheduling"
|
||||
contains: "connection_id"
|
||||
- path: "backend/tasks/cloud_tasks.py"
|
||||
provides: "Idempotent cloud metadata refresh with bounded retries"
|
||||
contains: "refresh_cloud_folder"
|
||||
- path: "backend/api/cloud/schemas.py"
|
||||
provides: "Whitelisted capability, item, freshness, and connection schemas"
|
||||
contains: "CloudBrowseResponse"
|
||||
- path: "backend/tests/test_cloud.py"
|
||||
provides: "Multiple account, IDOR/admin, credential exclusion, and no-byte browse tests"
|
||||
contains: "test_browse_connection_rejects_foreign_owner"
|
||||
key_links:
|
||||
- from: "backend/api/cloud/browse.py"
|
||||
to: "backend/services/cloud_items.py"
|
||||
via: "owned connection resolution and durable cached listing"
|
||||
pattern: "list_cloud_children"
|
||||
- from: "backend/tasks/cloud_tasks.py"
|
||||
to: "backend/storage/cloud_backend_factory.py"
|
||||
via: "connection provider selects CloudResourceAdapter after ownership lookup"
|
||||
pattern: "build_cloud_resource_adapter"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Normalize all four providers into the Phase 12 resource contract and expose durable, owner-scoped, connection-ID browsing with safe stale-while-revalidate behavior. Decompose the cloud router without changing its public connection-management behavior.
|
||||
|
||||
Scope fence: capability reporting is active, but create/rename/move/delete/open/preview implementations remain Phase 13; file byte hydration and cache accounting remain Phase 14.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-cloud-resource-foundation/12-01-SUMMARY.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-CONTEXT.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-RESEARCH.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-PATTERNS.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Normalize provider listings and capability evidence</name>
|
||||
<files>backend/storage/cloud_backend_factory.py, backend/storage/google_drive_backend.py, backend/storage/onedrive_backend.py, backend/storage/nextcloud_backend.py, backend/storage/webdav_backend.py, backend/tests/test_cloud_backends.py</files>
|
||||
<read_first>
|
||||
- backend/storage/cloud_base.py — contract created by Plan 12-01
|
||||
- backend/storage/cloud_backend_factory.py — existing lazy provider construction
|
||||
- backend/api/cloud.py — current inline Google/OneDrive listing implementations to relocate
|
||||
- backend/storage/nextcloud_backend.py — current N+1 listing implementation and SSRF guard
|
||||
- backend/tests/test_cloud_backends.py — provider contract test conventions
|
||||
</read_first>
|
||||
<action>
|
||||
Make each provider expose `CloudResourceAdapter` behavior and normalize all pages into `CloudListing`. Google requests explicit files fields for id, parents, name, mimeType, size, modifiedTime, version/etag-equivalent fields, and item capabilities; follows `nextPageToken`; represents native Google documents with nullable/zero size without treating them as folders; and reports `insufficient_scope` when current `drive.file` credentials cannot browse intended content. OneDrive follows `@odata.nextLink` and normalizes id, drive/parent reference, file/folder facet, size, modified time, eTag/cTag, and effective permission state. Nextcloud/WebDAV use SSRF-validated non-mutating PROPFIND/OPTIONS metadata, normalize getetag/getlastmodified/getcontentlength/getcontenttype/resourcetype, and treat absent properties as unknown rather than unsupported. Factory adds `build_cloud_resource_adapter`. Fixture tests assert equivalent normalized results, complete pagination, safe capability evidence, and zero byte-download/mutation calls.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- all supported providers return the same normalized resource and capability schema
|
||||
- Google and OneDrive pagination is followed before `complete=true`
|
||||
- every WebDAV/Nextcloud outbound request retains `validate_cloud_url()` protection
|
||||
- provider tests verify list/capability flows never call get_object/put/delete/move/rename
|
||||
- `pytest -q tests/test_cloud_backends.py` exits 0
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud_backends.py</automated></verify>
|
||||
<done>All four providers pass the normalized metadata/capability and complete-pagination contract without byte or mutation calls.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Decompose cloud API and add connection-ID browse schemas</name>
|
||||
<files>backend/api/cloud/__init__.py, backend/api/cloud/connections.py, backend/api/cloud/browse.py, backend/api/cloud/schemas.py, backend/api/cloud.py, backend/main.py, backend/tests/test_cloud.py</files>
|
||||
<read_first>
|
||||
- backend/api/cloud.py — preserve all connection/OAuth/WebDAV/default-storage behavior while splitting
|
||||
- backend/api/documents/__init__.py — no-prefix sub-router aggregator pattern and FastAPI empty-path warning
|
||||
- backend/api/schemas.py — explicit response whitelist pattern
|
||||
- backend/deps/utils.py — canonical `parse_uuid` and `get_client_ip`
|
||||
- backend/services/cloud_items.py — owner-scoped service API
|
||||
</read_first>
|
||||
<action>
|
||||
Convert `api/cloud.py` into an `api/cloud/` package with one parent router owning `/api/cloud`; sub-routers carry no prefix. Preserve existing endpoint URLs as compatibility routes where unambiguous, but introduce canonical `GET /api/cloud/connections/{connection_id}/items` with optional encoded parent reference and owner-scoped `PATCH /api/cloud/connections/{connection_id}` accepting only a validated nonblank `display_name` field. Parse UUIDs with `deps.utils.parse_uuid`, resolve the connection by current owner, and return explicit schemas for connection root, items, merged capabilities, freshness state, and last refresh timestamp. List connections keeps every same-provider account and never returns credentials. For an indexed folder, return durable rows and set/schedule refresh; for an empty first visit, perform one bounded metadata fetch so the user receives content. Map service/provider errors to controlled router errors. Admin remains blocked by `get_regular_user`. Add tests for renaming one connection without affecting another, duplicate providers, foreign connection/item IDs, admin access, credentials/token/password exclusion, malformed UUID, mass-assignment rejection, and compatibility endpoint behavior.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- cloud package aggregator is the only router with `/api/cloud` prefix
|
||||
- canonical browse endpoint uses connection UUID and cannot select by provider alone
|
||||
- response schemas cannot serialize `credentials_enc` or decrypted credential fields
|
||||
- two active Google Drive connections for one user are independently browsable and independently renameable
|
||||
- focused cloud API tests pass
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud.py -k "connection or browse or credential or admin"</automated></verify>
|
||||
<done>Canonical connection-ID browse and display-name APIs preserve existing connection flows and pass ownership/credential negatives.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Add durable metadata refresh task, retry behavior, and plan documentation</name>
|
||||
<files>backend/tasks/cloud_tasks.py, backend/celery_app.py, backend/tests/test_cloud_items.py, backend/tests/test_cloud.py, backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md</files>
|
||||
<read_first>
|
||||
- backend/tasks/document_tasks.py — sync Celery entry point bridging async service with bounded retry
|
||||
- backend/celery_app.py — JSON-only task registration and queue routing
|
||||
- backend/services/cloud_items.py — idempotent reconciliation/folder-state functions
|
||||
- AGENTS.md — documentation, version, testing, and security protocol
|
||||
- README.md — cloud feature and API documentation sections
|
||||
</read_first>
|
||||
<action>
|
||||
Add `refresh_cloud_folder(user_id, connection_id, parent_ref)` as a JSON-only Celery task on the documents queue. The task opens its own async DB session, revalidates owner/connection, decrypts credentials inside the worker (never broker payload), marks refreshing, obtains a complete normalized listing, reconciles idempotently, and marks fresh. Retry only transient provider/network failures using bounded increasing countdowns with jitter or the established explicit retry harness; terminal auth/scope errors become warning state with safe reason/remedy. Update task registration. Add tests for idempotency, retry countdown bounds, failed refresh retaining rows/last-success time, and no provider byte calls/quota changes. Apply project protocol for this user-visible/API plan: bump backend/frontend/package-lock patch version from 0.1.4 to 0.1.5, update AGENTS current state/shared module map if new canonical modules were added, and document the connection-ID browse/capability foundation in README.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- task payload contains IDs/references only and no credential or provider byte content
|
||||
- transient retries are bounded; auth/scope failures do not retry forever
|
||||
- warning state preserves cached rows and last successful refresh timestamp
|
||||
- browse/refresh tests assert no `get_object`, MinIO put, or quota update
|
||||
- versions are exactly 0.1.5 in backend main, package.json, and package-lock root/package entry
|
||||
- focused backend tests pass
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -q tests/test_cloud.py tests/test_cloud_backends.py tests/test_cloud_items.py && cd ../frontend && npm run build</automated></verify>
|
||||
<done>Durable refresh retries safely, retains cached metadata on failure, performs no byte/quota work, and plan documentation/version 0.1.5 is complete.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
| Threat ID | Threat | Mitigation and evidence |
|
||||
|-----------|--------|-------------------------|
|
||||
| T-12-01 | Connection/item IDOR | `get_regular_user` plus owner-scoped connection UUID lookup; foreign/admin negative tests |
|
||||
| T-12-03 | Credential disclosure | Explicit schemas and broker payload tests exclude credentials/tokens/passwords |
|
||||
| T-12-04 | WebDAV SSRF | Preserve URL validation before every request and redirect; regression tests |
|
||||
| T-12-05 | Cache collision | Connection UUID replaces provider in route/task/durable identity |
|
||||
| T-12-07 | Data loss from partial pagination | `complete=true` only after all pages; reconciliation test |
|
||||
| T-12-08 | Provider error XSS/leak | Stable reason codes and controlled messages; no raw credential-bearing exception response |
|
||||
| T-12-09 | Byte/quota side effects | Negative mocks assert metadata browse never invokes byte or quota APIs |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
1. `cd backend && pytest -q tests/test_cloud.py tests/test_cloud_backends.py tests/test_cloud_items.py`
|
||||
2. `cd backend && pytest -v`
|
||||
3. `cd frontend && npm run build`
|
||||
4. Inspect API response fixtures for zero credential fields.
|
||||
5. Confirm Phase 13 mutation and Phase 14 byte-cache methods are absent.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Every provider supplies normalized metadata and capabilities.
|
||||
- Browsing is keyed by owned connection UUID and supports duplicate provider accounts.
|
||||
- Durable cached rows return without provider-byte downloads and refresh safely in background.
|
||||
- Failure states preserve navigation, retry boundedly, and expose safe actionable reasons.
|
||||
</success_criteria>
|
||||
|
||||
<output>Create `.planning/phases/12-cloud-resource-foundation/12-02-SUMMARY.md` when complete.</output>
|
||||
@@ -0,0 +1,189 @@
|
||||
---
|
||||
phase: "12"
|
||||
plan: "03"
|
||||
type: execute
|
||||
wave: 3
|
||||
depends_on:
|
||||
- "12-02"
|
||||
files_modified:
|
||||
- frontend/src/api/cloud.js
|
||||
- frontend/src/stores/cloudConnections.js
|
||||
- frontend/src/router/index.js
|
||||
- frontend/src/views/CloudStorageView.vue
|
||||
- frontend/src/views/CloudFolderView.vue
|
||||
- frontend/src/components/settings/SettingsCloudTab.vue
|
||||
- frontend/src/components/storage/StorageBrowser.vue
|
||||
- frontend/src/components/ui/BreadcrumbBar.vue
|
||||
- frontend/src/components/ui/AppIcon.vue
|
||||
- frontend/src/utils/formatters.js
|
||||
- frontend/src/stores/__tests__/cloudConnections.test.js
|
||||
- frontend/src/views/__tests__/CloudStorageView.test.js
|
||||
- frontend/src/views/__tests__/CloudFolderView.test.js
|
||||
- frontend/src/components/settings/__tests__/SettingsCloudTab.test.js
|
||||
- frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js
|
||||
- frontend/src/components/ui/__tests__/BreadcrumbBar.test.js
|
||||
- backend/main.py
|
||||
- frontend/package.json
|
||||
- frontend/package-lock.json
|
||||
- AGENTS.md
|
||||
- README.md
|
||||
autonomous: true
|
||||
requirements:
|
||||
- CONN-04
|
||||
- CLOUD-01
|
||||
- CLOUD-08
|
||||
- CACHE-01
|
||||
- CACHE-02
|
||||
- SYNC-01
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "D-01/D-03: Cloud Storage shows every connection as a customizable, disambiguated top-level root"
|
||||
- "D-04/D-05: breadcrumbs begin with connection name and folder memory lasts only for the browser session"
|
||||
- "D-06/D-07: unsupported actions remain in stable positions and explain themselves on hover, focus, and touch tap"
|
||||
- "D-09/D-10: permanent limitations are gray; temporary/remediable blocks are amber with exact guidance"
|
||||
- "D-12/D-14/D-16: folder refresh state is subtle, cached rows remain usable, and warning copy includes last refresh/retry"
|
||||
- "D-15/D-17: cloud rows match local rows, repeat source only contextually, and show a compact marker only for cached bytes"
|
||||
artifacts:
|
||||
- path: "frontend/src/components/storage/StorageBrowser.vue"
|
||||
provides: "Single capability-aware local/cloud row and action renderer"
|
||||
contains: "aria-disabled"
|
||||
- path: "frontend/src/components/ui/BreadcrumbBar.vue"
|
||||
provides: "Connection-root breadcrumb and refreshing/fresh/warning sync indicator"
|
||||
contains: "folderFreshness"
|
||||
- path: "frontend/src/views/CloudFolderView.vue"
|
||||
provides: "Thin connection-ID route data provider"
|
||||
contains: "connectionId"
|
||||
- path: "frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js"
|
||||
provides: "Pointer, keyboard, touch, action suppression, and local regression coverage"
|
||||
contains: "temporarily_unavailable"
|
||||
key_links:
|
||||
- from: "frontend/src/router/index.js"
|
||||
to: "frontend/src/views/CloudFolderView.vue"
|
||||
via: "/cloud/:connectionId/:folderId route"
|
||||
pattern: ":connectionId"
|
||||
- from: "frontend/src/views/CloudFolderView.vue"
|
||||
to: "frontend/src/components/storage/StorageBrowser.vue"
|
||||
via: "capabilities, connectionRoot, folderFreshness, lastRefreshedAt, byteAvailability props"
|
||||
pattern: ":capabilities"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Make connected accounts and durable cloud metadata feel like the local file manager by extending the one shared `StorageBrowser` and breadcrumb system. Render provider capabilities and freshness without duplicating a cloud grid or implementing Phase 13 actions.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-cloud-resource-foundation/12-02-SUMMARY.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-CONTEXT.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-PATTERNS.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Route and store cloud navigation by connection identity</name>
|
||||
<files>frontend/src/api/cloud.js, frontend/src/stores/cloudConnections.js, frontend/src/router/index.js, frontend/src/views/CloudStorageView.vue, frontend/src/views/CloudFolderView.vue, frontend/src/components/settings/SettingsCloudTab.vue, frontend/src/stores/__tests__/cloudConnections.test.js, frontend/src/views/__tests__/CloudStorageView.test.js, frontend/src/views/__tests__/CloudFolderView.test.js, frontend/src/components/settings/__tests__/SettingsCloudTab.test.js</files>
|
||||
<read_first>
|
||||
- frontend/src/api/cloud.js — current provider-keyed folder URL
|
||||
- frontend/src/stores/cloudConnections.js — existing Pinia fetch/disconnect pattern
|
||||
- frontend/src/router/index.js — current `/cloud/:provider/:folderId` route
|
||||
- frontend/src/views/CloudStorageView.vue — connection list currently bypasses StorageBrowser
|
||||
- frontend/src/views/CloudFolderView.vue — thin-view boundary and provider-keyed browse
|
||||
- frontend/src/components/settings/SettingsCloudTab.vue — existing connection maintenance UI where display-name editing belongs
|
||||
- .planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md — root list and session navigation contracts
|
||||
</read_first>
|
||||
<action>
|
||||
Change the canonical cloud route to `/cloud/:connectionId/:folderId(.*)` and call the connection-ID browse endpoint through `api/cloud.js`. Add `renameCloudConnection(id, displayName)` and a Pinia `rename` action that updates only the returned connection after the API succeeds. Add accessible display-name editing to the existing SettingsCloudTab connection maintenance surface; default names remain plain provider labels and duplicate defaults show a four-character connection UUID suffix in the cloud root until customized. Extend the Pinia store with selected connection, browse result, capabilities, freshness, last refresh, and safe error state while keeping API calls out of smart components. Render all connection roots through `StorageBrowser` data rather than a parallel row grid. `CloudFolderView` passes normalized props/events only. Save the last folder in `sessionStorage` under a namespaced key containing connection UUID; entering from the root may resume during the same session, while absent session state starts at root. Do not store JWTs, credentials, cloud item content, or persistent navigation in browser storage. Add tests for rename validation/success/failure, two same-provider roots, connection UUID URLs/API calls, root reset/fresh session, same-session resume, and thin-view behavior.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- no frontend browse request or route selects a connection by provider slug alone
|
||||
- duplicate provider connections render as separate roots, default-name suffixes are deterministic, and Settings can customize each name independently
|
||||
- only folder navigation state is stored in sessionStorage; no token or credential data is stored
|
||||
- CloudStorageView and CloudFolderView contain no file-grid markup
|
||||
- focused store/view tests pass
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd frontend && npm test -- --run cloudConnections CloudStorageView CloudFolderView SettingsCloudTab</automated></verify>
|
||||
<done>Connection roots, custom names, connection-ID routing, and session-only folder memory pass store/view/settings tests.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Replace mode-gated actions with accessible capability rendering</name>
|
||||
<files>frontend/src/components/storage/StorageBrowser.vue, frontend/src/components/ui/AppIcon.vue, frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js</files>
|
||||
<read_first>
|
||||
- frontend/src/components/storage/StorageBrowser.vue — all current `mode === 'local'` gates, emits, and touch targets
|
||||
- frontend/src/components/ui/AppIcon.vue — centralized icon registry
|
||||
- frontend/src/components/storage/__tests__/StorageBrowser.dragmove.test.js — shared browser mount/event pattern
|
||||
- .planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md — exact capability colors, props, interaction, copy, accessibility
|
||||
</read_first>
|
||||
<action>
|
||||
Add typed/defaulted props for capabilities, connectionRoot, folderFreshness, lastRefreshedAt, and byteAvailability. Supply a full-supported local capability set so local and cloud use one action-rendering path. Keep action controls in stable locations. Supported controls emit existing generic events; unsupported controls render gray with `aria-disabled="true"`; temporarily unavailable controls render amber; neither uses native `disabled`; Enter, Space, pointer click, and touch tap suppress the action and surface the capability message through a shared explanation event/state. Use tooltips for hover/focus and an inline dismissible notice for touch per UI-SPEC. Do not expose Phase 13 mutation handlers: controls may be present but capability-disabled. Add a compact cached-byte icon and exact quota tooltip only when availability is `cached`; cloud_only renders no marker. Add AppIcon entries only if existing names are insufficient. Tests cover all three states, exact message/remedy, click/keyboard/touch action suppression, active event emission, touch target classes, cached marker, no cloud-only marker, and unchanged local drag/move/share/delete behavior.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- no action visibility block depends solely on `mode === 'local'`
|
||||
- `aria-disabled` controls remain focusable and never emit the underlying action
|
||||
- unsupported and temporary states use gray and amber UI-SPEC tokens respectively
|
||||
- local capability defaults preserve all pre-Phase-12 local actions
|
||||
- all StorageBrowser tests pass
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd frontend && npm test -- --run StorageBrowser</automated></verify>
|
||||
<done>One shared browser renders and safely explains every capability state while preserving local behavior.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Add connection breadcrumb freshness and finish UI plan protocol</name>
|
||||
<files>frontend/src/components/ui/BreadcrumbBar.vue, frontend/src/components/ui/__tests__/BreadcrumbBar.test.js, frontend/src/utils/formatters.js, frontend/src/views/CloudFolderView.vue, backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md</files>
|
||||
<read_first>
|
||||
- frontend/src/components/ui/BreadcrumbBar.vue — existing generic segment/root API
|
||||
- frontend/src/components/ui/__tests__/BreadcrumbBar.test.js — current test conventions
|
||||
- frontend/src/utils/formatters.js — single source for date/size/provider formatting
|
||||
- .planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md — sync indicator states, copy, typography, spacing
|
||||
- AGENTS.md — documentation and version protocol
|
||||
</read_first>
|
||||
<action>
|
||||
Extend BreadcrumbBar generically with optional connection root icon/name and freshness metadata. Render refreshing spinner with accessible label, fresh checkmark that fades after three seconds without layout shift, and persistent warning indicator/banner with exact UI-SPEC copy and relative last-update time. Keep cached rows interactive. Use shared formatter utilities; add one reusable relative-time formatter there only if absent. Ensure no repeated provider badge appears on rows within a connection. Add fake-timer tests for fade behavior plus labels/tooltips/warning copy. Apply plan protocol: bump versions from 0.1.5 to 0.1.6 in backend main, package.json, and package-lock root/package entry; update AGENTS current state/shared component map if contracts changed; update README feature text for unified connection-root browsing and capability explanations.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- breadcrumb root is connection display name, with no redundant `Cloud` segment
|
||||
- freshness exposes exact accessible labels for refreshing, up to date, and failed
|
||||
- fresh indicator fades without resizing the breadcrumb; warning persists until state changes
|
||||
- versions are exactly 0.1.6 across backend/frontend lock sources
|
||||
- focused frontend suite and production build pass
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd frontend && npm test -- --run BreadcrumbBar StorageBrowser CloudFolderView && npm run build</automated></verify>
|
||||
<done>Breadcrumb freshness, cached-byte status, documentation, version 0.1.6, focused tests, and production build are complete.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
| Threat ID | Threat | Mitigation and evidence |
|
||||
|-----------|--------|-------------------------|
|
||||
| T-12-01 | Client route IDOR | Backend remains authority; frontend uses opaque connection UUID and tests never infer owner/provider access |
|
||||
| T-12-08 | Stored XSS via filename/provider message | Vue text interpolation only; no `v-html`; controlled capability message schema |
|
||||
| T-12-10 | Disabled action executes on keyboard/touch | `aria-disabled` guard suppresses click/Enter/Space and tests assert no action emit |
|
||||
| T-12-11 | Secret/token persistence | sessionStorage contains folder references only; negative source/test assertions |
|
||||
| T-12-12 | UI source confusion | connection root breadcrumb and contextual provider icon; no misleading per-row duplication |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
1. `cd frontend && npm test`
|
||||
2. `cd frontend && npm run build`
|
||||
3. Playwright desktop/mobile inspection: tooltip focus/hover/tap, no overlap, cached rows usable during refresh.
|
||||
4. Source scan confirms StorageBrowser remains the only file grid and views stay thin.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Multiple provider accounts appear as distinct connection roots and navigate by connection UUID.
|
||||
- Local and cloud files share one row/action implementation.
|
||||
- Unsupported/temporary actions explain themselves accessibly without executing.
|
||||
- Breadcrumb freshness and cached-byte status match the approved UI contract.
|
||||
</success_criteria>
|
||||
|
||||
<output>Create `.planning/phases/12-cloud-resource-foundation/12-03-SUMMARY.md` when complete.</output>
|
||||
@@ -0,0 +1,182 @@
|
||||
---
|
||||
phase: "12"
|
||||
plan: "04"
|
||||
type: execute
|
||||
wave: 4
|
||||
depends_on:
|
||||
- "12-02"
|
||||
- "12-03"
|
||||
files_modified:
|
||||
- backend/tests/test_cloud.py
|
||||
- backend/tests/test_cloud_items.py
|
||||
- backend/tests/test_cloud_security.py
|
||||
- frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js
|
||||
- frontend/src/views/__tests__/CloudFolderView.test.js
|
||||
- backend/main.py
|
||||
- frontend/package.json
|
||||
- frontend/package-lock.json
|
||||
- AGENTS.md
|
||||
- README.md
|
||||
- SECURITY.md
|
||||
- RUNBOOK.md
|
||||
autonomous: true
|
||||
requirements:
|
||||
- CONN-04
|
||||
- CLOUD-01
|
||||
- CLOUD-08
|
||||
- CACHE-01
|
||||
- CACHE-02
|
||||
- SYNC-01
|
||||
|
||||
must_haves:
|
||||
truths:
|
||||
- "All D-01 through D-18 Phase 12 decisions have automated or explicit rendered-browser evidence"
|
||||
- "All six Phase 12 requirements pass owner, admin-negative, provider-contract, metadata-only, and shared-browser tests"
|
||||
- "No cloud browse path exposes credentials, downloads file bytes, changes MinIO, or changes user quota"
|
||||
- "Security gates report zero blocking high-severity findings before Phase 12 is marked complete"
|
||||
- "Project documentation and phase-completion version are synchronized with shipped behavior"
|
||||
artifacts:
|
||||
- path: "backend/tests/test_cloud_security.py"
|
||||
provides: "Cross-user/admin/credential/SSRF/no-byte negative integration suite"
|
||||
contains: "test_foreign_user_cannot_browse_cloud_item"
|
||||
- path: "SECURITY.md"
|
||||
provides: "Phase 12 threat mitigations and security-gate evidence"
|
||||
contains: "Phase 12"
|
||||
- path: "AGENTS.md"
|
||||
provides: "Current phase state and new canonical shared-module rules"
|
||||
contains: "Phase 12"
|
||||
key_links:
|
||||
- from: "backend/tests/test_cloud_security.py"
|
||||
to: "backend/api/cloud/browse.py"
|
||||
via: "authenticated API negative tests"
|
||||
pattern: "404|403"
|
||||
- from: "12-CONTEXT.md decisions"
|
||||
to: "backend/frontend focused tests"
|
||||
via: "decision coverage matrix in plan verification"
|
||||
pattern: "D-18"
|
||||
---
|
||||
|
||||
<objective>
|
||||
Close Phase 12 with cross-layer regression, security, real-PostgreSQL schema evidence, rendered-browser UI verification, and synchronized documentation/versioning. This plan verifies the foundation as a coherent whole; it does not add Phase 13 operations or Phase 14 byte caching.
|
||||
</objective>
|
||||
|
||||
<execution_context>
|
||||
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
||||
@$HOME/.claude/get-shit-done/templates/summary.md
|
||||
</execution_context>
|
||||
|
||||
<context>
|
||||
@.planning/phases/12-cloud-resource-foundation/12-01-SUMMARY.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-02-SUMMARY.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-03-SUMMARY.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-CONTEXT.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md
|
||||
@.planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md
|
||||
</context>
|
||||
|
||||
<tasks>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 1: Complete cross-layer and security-negative coverage</name>
|
||||
<files>backend/tests/test_cloud.py, backend/tests/test_cloud_items.py, backend/tests/test_cloud_security.py, frontend/src/components/storage/__tests__/StorageBrowser.capabilities.test.js, frontend/src/views/__tests__/CloudFolderView.test.js</files>
|
||||
<read_first>
|
||||
- backend/api/cloud/browse.py — final browse trust boundary and response models
|
||||
- backend/services/cloud_items.py — final owner/reconciliation logic
|
||||
- backend/storage/cloud_utils.py — SSRF and credential encryption invariants
|
||||
- backend/tests/conftest.py — auth/admin and real-service fixture conventions
|
||||
- .planning/phases/12-cloud-resource-foundation/12-VALIDATION.md — requirement-to-evidence map
|
||||
- .planning/phases/12-cloud-resource-foundation/12-CONTEXT.md — D-01 through D-18
|
||||
</read_first>
|
||||
<action>
|
||||
Fill any remaining validation matrix gaps. Add full API negatives for wrong user connection/item IDs, admin browse denial, malformed IDs, disabled/deleted connection, credential field exclusion, provider error sanitization, SSRF preservation, and same provider/item IDs across users/connections. Assert browse/refresh mocks never call get_object, MinIO put/delete, or quota mutation. Add a real PostgreSQL integration case for unique constraints, owner FKs/indexes, and cascade behavior rather than relying on SQLite. Complete frontend regressions for every capability state and local parity. Add a decision coverage table in test comments or verification notes mapping D-01..D-18 to concrete tests without duplicating tests solely for bookkeeping.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- every security invariant has a passing negative test
|
||||
- PostgreSQL integration proves constraints/cascades used by production
|
||||
- no admin response contains document metadata/content or cloud credentials
|
||||
- all six requirement IDs and D-01..D-18 have named evidence
|
||||
- backend and frontend full suites pass with zero failures
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && pytest -v && cd ../frontend && npm test && npm run build</automated></verify>
|
||||
<done>All requirement, decision, ownership, admin, credential, provider, quota, and shared-browser evidence is green.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 2: Run security/dependency gates and rendered-browser UAT</name>
|
||||
<files>SECURITY.md, RUNBOOK.md</files>
|
||||
<read_first>
|
||||
- AGENTS.md — mandatory security gate checklist
|
||||
- SECURITY.md — existing security documentation format
|
||||
- RUNBOOK.md — operational verification format
|
||||
- .planning/phases/12-cloud-resource-foundation/12-UI-SPEC.md — visual/accessibility acceptance contract
|
||||
</read_first>
|
||||
<action>
|
||||
Run `bandit -r backend/`, `pip audit`, `npm audit --audit-level=high`, full security-invariant tests, and the repository’s secret scanner where available. Resolve all Phase 12-introduced high/critical findings; do not suppress them. Use the local app and Playwright at desktop and mobile widths to verify connection roots, duplicate-name disambiguation, refreshing/fresh/warning breadcrumb states, hover/focus/tap capability explanations, minimum touch targets, cached-byte marker, no overlap, and retained row usability during delayed refresh. Record commands/results and operational refresh/retry behavior in SECURITY.md/RUNBOOK.md without credentials or document content.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- Bandit reports zero HIGH findings introduced by Phase 12
|
||||
- pip/npm audits report zero high/critical vulnerabilities or a documented external blocker requiring owner escalation
|
||||
- secret scan finds no committed credentials/tokens
|
||||
- desktop/mobile Playwright evidence covers all UI-SPEC state transitions with no overlap
|
||||
- SECURITY.md and RUNBOOK.md contain Phase 12 evidence and safe operational guidance
|
||||
</acceptance_criteria>
|
||||
<verify><automated>cd backend && bandit -r . && pip audit && cd ../frontend && npm audit --audit-level=high && npm test && npm run build</automated></verify>
|
||||
<done>Security/dependency/secret gates and rendered desktop/mobile UI verification have no unresolved blocking findings.</done>
|
||||
</task>
|
||||
|
||||
<task type="auto">
|
||||
<name>Task 3: Finalize Phase 12 documentation and phase version</name>
|
||||
<files>backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md, SECURITY.md, RUNBOOK.md</files>
|
||||
<read_first>
|
||||
- AGENTS.md — current state, shared module map, docs/version protocol
|
||||
- README.md — features, cloud provider, API, environment sections
|
||||
- backend/main.py — backend version source of truth
|
||||
- frontend/package.json — frontend version source of truth
|
||||
- .planning/ROADMAP.md — exact Phase 12 goal/success criteria
|
||||
</read_first>
|
||||
<action>
|
||||
Update AGENTS current state to Phase 12 complete and add any new canonical cloud contract/service/frontend helpers to shared module maps with non-negotiable no-duplication rules. Update README features/API architecture for multiple same-provider connections, connection-ID virtual browsing, durable metadata, capability explanations, and provider-byte source-of-truth; do not claim Phase 13 mutations or Phase 14 caching are shipped. Update SECURITY/RUNBOOK references to final module and command names. Apply the full-phase minor version bump from 0.1.6 to 0.2.0 in backend main, package.json, and package-lock root/package entry. Run source checks that all three report 0.2.0.
|
||||
</action>
|
||||
<acceptance_criteria>
|
||||
- AGENTS says Phase 12 complete and names the actual shared modules introduced
|
||||
- README accurately distinguishes shipped Phase 12 foundation from future cloud operations/cache
|
||||
- backend main, frontend package.json, and package-lock root/package entry all equal 0.2.0
|
||||
- no documentation includes credentials, raw provider errors, or unshipped capability claims
|
||||
- final full suites and build remain green after documentation/version changes
|
||||
</acceptance_criteria>
|
||||
<verify><automated>rg -n '0\.2\.0' backend/main.py frontend/package.json frontend/package-lock.json && cd backend && pytest -v && cd ../frontend && npm test && npm run build</automated></verify>
|
||||
<done>Phase documentation is accurate, shared-module rules are current, versions equal 0.2.0, and final suites/build pass.</done>
|
||||
</task>
|
||||
|
||||
</tasks>
|
||||
|
||||
<threat_model>
|
||||
| Threat ID | Threat | Mitigation and evidence |
|
||||
|-----------|--------|-------------------------|
|
||||
| T-12-01 | IDOR/admin content access | Dedicated foreign-user/admin negative integration suite |
|
||||
| T-12-03 | Credential disclosure | Response/broker/log field assertions and secret scanning |
|
||||
| T-12-04 | SSRF | Existing validator regression plus provider-request tests |
|
||||
| T-12-07 | Metadata loss | Pagination/completeness and failed-refresh retention tests |
|
||||
| T-12-08 | XSS/error leakage | Controlled messages, Vue escaping, rendered tests, no raw exception docs |
|
||||
| T-12-09 | Quota corruption | no-byte/no-quota cross-layer mocks and DB assertions |
|
||||
| T-12-10 | Disabled control action | pointer/keyboard/touch suppression tests and rendered UAT |
|
||||
| T-12-SC | Supply-chain/secrets | pip/npm audits and repository secret scan before completion |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
1. Backend and frontend full suites pass with zero failures.
|
||||
2. Production frontend build succeeds.
|
||||
3. PostgreSQL integration verifies production constraints.
|
||||
4. Security gate checklist in AGENTS is fully satisfied.
|
||||
5. Rendered-browser UAT satisfies the UI-SPEC on desktop and mobile.
|
||||
6. Requirement coverage is 6/6 and decision coverage is 18/18.
|
||||
</verification>
|
||||
|
||||
<success_criteria>
|
||||
- Phase 12 success criteria are proven across adapters, database, API, shared browser, and security boundaries.
|
||||
- No cloud bytes are copied or quota-charged by browsing.
|
||||
- Documentation accurately describes the completed foundation and future phase boundaries.
|
||||
- Version sources are synchronized at 0.2.0 and all release gates pass.
|
||||
</success_criteria>
|
||||
|
||||
<output>Create `.planning/phases/12-cloud-resource-foundation/12-04-SUMMARY.md` when complete.</output>
|
||||
@@ -0,0 +1,103 @@
|
||||
# Phase 12: Cloud Resource Foundation - Pattern Map
|
||||
|
||||
**Mapped:** 2026-06-18
|
||||
**Status:** Complete
|
||||
|
||||
## Purpose
|
||||
|
||||
Map Phase 12 files and data flow to existing DocuVault patterns so implementation extends established architecture without duplicating helpers, grids, or provider-specific UI logic.
|
||||
|
||||
## Backend Pattern Map
|
||||
|
||||
| New/changed area | Closest analog | Pattern to preserve |
|
||||
|------------------|----------------|---------------------|
|
||||
| `backend/migrations/versions/0006_cloud_resource_foundation.py` | `0005_system_settings.py` | Typed Alembic table creation, named constraints/indexes, timezone timestamps, exact `down_revision="0005"`, reversible downgrade |
|
||||
| `CloudItem` and connection fields in `backend/db/models.py` | `CloudConnection`, `Document`, `Folder` | UUID PKs, explicit owner FKs with cascade, compound owner/navigation indexes, human names in DB only |
|
||||
| `backend/storage/cloud_base.py` | `backend/storage/base.py`, `backend/ai/base.py` | Small abstract interface; normalized domain values; provider adapters own translation |
|
||||
| Provider capability/metadata normalization | Existing Google/OneDrive/Nextcloud adapters | Keep SDK/HTTP details inside provider files, wrap synchronous SDK calls with `asyncio.to_thread()`, preserve canonical `CloudConnectionError` import |
|
||||
| `backend/services/cloud_items.py` | Owner-scoped query patterns in `api/folders.py`, service exception policy in AGENTS.md | Filter by owner in every statement; service raises `ValueError`/domain exceptions, never `HTTPException`; use shared `deps.utils.parse_uuid` at router boundary |
|
||||
| `backend/api/cloud/` package | `backend/api/documents/` and `backend/api/admin/` | One aggregator owns `/api/cloud`; sub-routers have no prefix; root empty-path routes register on parent if FastAPI requires it |
|
||||
| Cloud response schemas | `CloudConnectionOut` in `backend/api/schemas.py` | Explicit field whitelist with `from_attributes`; never serialize ORM objects or `credentials_enc` directly |
|
||||
| Metadata refresh/backoff | Celery retry conventions in document tasks | Bounded exponential retry, idempotent upsert/reconciliation, no retry around ownership/validation failures |
|
||||
|
||||
### Required Backend Corrections
|
||||
|
||||
- Current `_upsert_cloud_connection()` and `_get_active_connection()` use `(user_id, provider)` and must not be copied. Phase 12 uses connection UUID as identity to support multiple same-provider accounts.
|
||||
- Current listing cache key `{user_id}:{provider}:{folder_id}` is insufficient. Any transitional key includes connection UUID; durable metadata belongs in PostgreSQL.
|
||||
- Current provider listing dictionaries contain only `id`, `name`, `is_dir`, `size`. New adapters normalize parent, MIME/type, mtime, etag/version, per-item capabilities, and completeness/pagination.
|
||||
- `api/folders.py` contains a private `_parse_uuid`; new cloud routers must import `parse_uuid` from `backend/deps/utils.py` under the project’s shared-helper rule.
|
||||
|
||||
## Frontend Pattern Map
|
||||
|
||||
| New/changed area | Closest analog | Pattern to preserve |
|
||||
|------------------|----------------|---------------------|
|
||||
| Capability-aware rows | `frontend/src/components/storage/StorageBrowser.vue` | This remains the only grid; props drive layout/state and generic events travel upward |
|
||||
| Sync indicator | `frontend/src/components/ui/BreadcrumbBar.vue` | Extend shared breadcrumb rather than adding cloud-only breadcrumb markup |
|
||||
| Connection roots | Current `CloudStorageView.vue` plus `StorageBrowser.vue` | View owns store/router only; feed normalized root rows into shared browser |
|
||||
| Cloud folder route | `CloudFolderView.vue`, `FileManagerView.vue` | Thin data provider; no duplicated toolbar, row, action, or tooltip layout |
|
||||
| API functions | `frontend/src/api/cloud.js` + `client.js` barrel | Change route arguments behind the domain module; consumers retain barrel imports |
|
||||
| Connection state | `frontend/src/stores/cloudConnections.js` | Pinia owns connection/folder state and errors; components stay store-agnostic |
|
||||
| Shared formatting | `frontend/src/utils/formatters.js` | Reuse `formatDate`, `formatSize`, and provider helpers; do not redefine locally |
|
||||
| Icons | `frontend/src/components/ui/AppIcon.vue` | Add registry entries only when an existing icon name cannot express status/action |
|
||||
|
||||
### UI Contract Patterns
|
||||
|
||||
- Unsupported actions use `aria-disabled="true"`, remain focusable/tappable, and suppress action emission while surfacing explanation.
|
||||
- Permanent unsupported state is gray; temporary unavailable state is amber. Do not hide actions based on `mode === "local"`.
|
||||
- `StorageBrowser` receives capabilities, folder freshness, connection root, last refresh time, and byte availability as props.
|
||||
- Local storage supplies a supported capability set so local/cloud share one rendering path.
|
||||
- Minimum touch target stays 36x36 px and all controls use existing indigo focus-ring utilities.
|
||||
- Session-only last folder state uses `sessionStorage` keyed by connection UUID; JWT tokens remain Pinia-memory-only.
|
||||
|
||||
## Testing Pattern Map
|
||||
|
||||
| Coverage | Existing analog | Phase 12 use |
|
||||
|----------|-----------------|--------------|
|
||||
| API/ownership | `backend/tests/test_cloud.py` | Multiple same-provider accounts, foreign connection/item 404, admin rejection, credential exclusion |
|
||||
| Adapter contracts | `backend/tests/test_cloud_backends.py` | Shared normalized fixtures, async methods, pagination/completeness, no destructive probes |
|
||||
| DB/service | async fixtures in `backend/tests/conftest.py` | Fast SQLite unit loop plus marked real-PostgreSQL constraint/index/cascade tests |
|
||||
| Pinia | `frontend/src/stores/__tests__/cloudConnections.test.js` | Connection UUID routes, duplicate providers, freshness/error transitions |
|
||||
| Storage browser | `StorageBrowser.dragmove/showSearch/skeleton.test.js` | Capability rendering, action suppression, focus/tap explanations, local regression |
|
||||
| Breadcrumb | `components/ui/__tests__/BreadcrumbBar.test.js` | Refreshing/fresh/warning labels, timestamps, fade behavior |
|
||||
| Thin views | `views/__tests__/FileManagerView.test.js` | Cloud views pass props/handle events without grid duplication |
|
||||
|
||||
## Data Flow
|
||||
|
||||
1. Router authenticates regular user and parses `connection_id` with shared request helpers.
|
||||
2. Service resolves `(connection_id, user_id)` and reads durable children by connection/parent.
|
||||
3. Provider adapter normalizes a complete paginated listing and capability states.
|
||||
4. Service upserts metadata using `(connection_id, provider_item_id)` and reconciles only after complete success.
|
||||
5. Explicit Pydantic schemas return rows, capability messages, freshness, and connection identity without credentials.
|
||||
6. Pinia/view maps API data into `StorageBrowser`; the browser renders shared rows/actions and emits generic commands.
|
||||
7. Browse never calls provider byte download, writes MinIO, or changes quota.
|
||||
|
||||
## Files Likely Modified
|
||||
|
||||
### Backend
|
||||
|
||||
- `backend/migrations/versions/0006_cloud_resource_foundation.py`
|
||||
- `backend/db/models.py`
|
||||
- `backend/storage/cloud_base.py`
|
||||
- `backend/storage/cloud_backend_factory.py`
|
||||
- provider backend modules
|
||||
- `backend/services/cloud_items.py`
|
||||
- `backend/api/cloud/__init__.py`, `connections.py`, `browse.py`, `schemas.py`
|
||||
- `backend/main.py` only if aggregator import changes
|
||||
- focused backend tests and PostgreSQL integration coverage
|
||||
|
||||
### Frontend
|
||||
|
||||
- `frontend/src/api/cloud.js`
|
||||
- `frontend/src/stores/cloudConnections.js`
|
||||
- `frontend/src/router/index.js`
|
||||
- `frontend/src/views/CloudStorageView.vue`
|
||||
- `frontend/src/views/CloudFolderView.vue`
|
||||
- `frontend/src/components/storage/StorageBrowser.vue`
|
||||
- `frontend/src/components/ui/BreadcrumbBar.vue`
|
||||
- focused component/store/view tests
|
||||
|
||||
### Documentation/Versioning
|
||||
|
||||
- `AGENTS.md`, `README.md`, `backend/main.py`, and `frontend/package.json` per the project documentation/version protocol after user-facing plan completion.
|
||||
|
||||
## PATTERN MAPPING COMPLETE
|
||||
Reference in New Issue
Block a user