kite/.planning/phases/12-cloud-resource-foundation/12-01-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
12	01	execute	1		backend/migrations/versions/0006_cloud_resource_foundation.py backend/db/models.py backend/storage/cloud_base.py backend/services/cloud_items.py backend/api/schemas.py backend/tests/test_cloud_capabilities.py backend/tests/test_cloud_items.py	true	CONN-04 CACHE-01 CACHE-02 SYNC-01	truths artifacts key_links D-02/D-03: connection UUID, not provider slug, is the durable identity and multiple same-provider accounts remain distinct D-08/D-09/D-10: every normalized action reports supported, unsupported, or temporarily_unavailable with a stable reason and safe message D-18: provider metadata size never changes document quota and no cloud bytes are persisted by metadata operations A cloud item keeps one DocuVault UUID across provider rename, move, and metadata refresh Cloud items and folder freshness rows cannot cross user or connection boundaries path provides contains backend/storage/cloud_base.py Normalized cloud resource, capability, listing, and adapter contracts class CloudResourceAdapter path provides contains backend/migrations/versions/0006_cloud_resource_foundation.py cloud_items and cloud_folder_states tables plus connection identity support cloud_items path provides contains backend/services/cloud_items.py Owner-scoped metadata upsert, list, and complete-list reconciliation reconcile_cloud_listing path provides contains backend/tests/test_cloud_items.py Stable identity, ownership, reconciliation, and metadata-only regression tests test_same_provider_item_isolated_by_connection from to via pattern backend/services/cloud_items.py backend/db/models.py owner and connection scoped SQLAlchemy statements CloudItem.user_id == user_id from to via pattern backend/storage/cloud_base.py backend/tests/test_cloud_capabilities.py fixed action/state/reason contract tests temporarily_unavailable

Create the provider-neutral cloud resource contract and durable metadata foundation. This plan establishes stable connection/item identity, normalized capability semantics, owner-scoped persistence, and metadata-only quota invariants before any provider or UI integration depends on them.

Scope fence: do not implement cloud mutations, byte download/cache, analysis jobs, semantic search, or broader OAuth consent.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/REQUIREMENTS.md @.planning/phases/12-cloud-resource-foundation/12-CONTEXT.md @.planning/phases/12-cloud-resource-foundation/12-RESEARCH.md @.planning/phases/12-cloud-resource-foundation/12-PATTERNS.md @.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md Task 1: Define and test normalized cloud resource capabilities backend/storage/cloud_base.py, backend/tests/test_cloud_capabilities.py - backend/storage/base.py — existing byte-oriented ABC that must remain separate - backend/storage/exceptions.py — canonical cloud exception - .planning/phases/12-cloud-resource-foundation/12-CONTEXT.md — D-06 through D-10 capability decisions - .planning/phases/12-cloud-resource-foundation/12-RESEARCH.md — capability levels and provider notes Add a cloud-resource contract separate from `StorageBackend`. Define fixed action keys for browse, open, preview, upload, create_folder, rename, move, delete, and change_tracking; capability states `supported`, `unsupported`, `temporarily_unavailable`; stable reason codes including provider_unsupported, insufficient_scope, read_only, reauth_required, offline, and item_restricted; immutable normalized `CloudCapability`, `CloudResource`, and complete/paginated `CloudListing` values; and an abstract `CloudResourceAdapter` with read-only `list_folder`, `get_capabilities`, and optional item-capability merge behavior. Messages are controlled adapter output, credentials are never fields, and the interface contains no mutation methods until Phase 13. Add unit tests for vocabulary completeness, state validation, connection defaults plus item overrides, exact reason/message preservation, and a fake adapter proving capability discovery invokes no put/delete/move/rename method. - `cloud_base.py` defines all nine action keys and exactly three capability states - normalized resources support provider ID, parent reference, type, size, MIME type, modified time, etag/version, and item capabilities - no method in the Phase 12 adapter contract mutates provider content - `pytest -q tests/test_cloud_capabilities.py` exits 0 cd backend && pytest -q tests/test_cloud_capabilities.py Normalized read-only capability/resource contract exists and its focused tests pass. Task 2: Add durable owner-scoped cloud metadata schema backend/migrations/versions/0006_cloud_resource_foundation.py, backend/db/models.py, backend/tests/test_cloud_items.py - backend/migrations/versions/0005_system_settings.py — current Alembic revision and typed-table style - backend/db/models.py — CloudConnection, Document, Folder, Quota ownership/index patterns - backend/tests/conftest.py — SQLite compatibility and integration conventions - .planning/phases/12-cloud-resource-foundation/12-RESEARCH.md — recommended fields and uniqueness boundary Create revision `0006` with `down_revision="0005"`. Extend `cloud_connections` only as needed for distinct same-provider accounts and user-defined display names; do not add `(user_id, provider)` uniqueness. Add `cloud_items` with UUID id, required user_id/connection_id FKs using CASCADE, provider_item_id, nullable parent provider reference/path snapshot, name, file/folder kind, nullable content type, provider-reported size, etag/version/modified_at, last_seen_at/deleted_at, nullable extracted_text, analysis_status, semantic_index_status, and nullable semantic_index_data JSONB reserved for provider-independent search artifacts. Add `cloud_item_topics` as an association from cloud items to existing owner topics without requiring a local `Document` row. Add `cloud_folder_states` for root or parent reference freshness, last successful refresh, refresh state, and safe error code/message. Enforce unique `(connection_id, provider_item_id)` and `(connection_id, parent_ref)` boundaries plus owner/connection/parent indexes. Mirror models and relationships. Tests must show metadata, extracted text, topic links, and semantic data persist with no object key/cache row; identical provider item IDs coexist across connections; foreign-owner queries do not match; cascades work in PostgreSQL integration; and provider size fields do not alter `Quota.used_bytes`. - migration upgrade and downgrade are structurally reversible and revision chain is 0005 -> 0006 - `cloud_items` has explicit user and connection ownership plus unique connection/provider-item identity - root folder freshness can be represented without a CloudItem parent row - no MinIO object key or retained-byte field is required to persist cloud metadata, extracted text, topic links, or semantic index data - focused model tests pass; real PostgreSQL constraint test is marked/configured per project conventions cd backend && pytest -q tests/test_cloud_items.py -k "model or schema or isolation or quota" Revision 0006 and matching ORM models persist owner-scoped metadata/search state without byte storage or quota mutation. Task 3: Implement owner-scoped metadata reconciliation service backend/services/cloud_items.py, backend/tests/test_cloud_items.py - backend/db/models.py — models created in Task 2 - backend/deps/utils.py — shared UUID parsing stays at router boundary - backend/api/folders.py — owner-scoped SQL patterns only; do not copy private helpers or HTTPException into service - .planning/phases/12-cloud-resource-foundation/12-CONTEXT.md — D-11 through D-14 freshness behavior Add service functions to resolve an owned connection by `(connection_id, user_id)`, list durable children by `(user_id, connection_id, parent_ref)`, upsert normalized resources by `(connection_id, provider_item_id)` while preserving CloudItem UUID, and reconcile a listing. Reconciliation updates parent/name/type/size/mtime/etag/version and last_seen_at; it marks missing children deleted only when `CloudListing.complete` is true and the provider refresh succeeded. Failed or partial listings retain rows. Folder-state helpers transition refreshing/fresh/warning, retain last successful timestamp on failure, and store only controlled error codes/messages. Service functions raise domain exceptions or `ValueError`, never FastAPI `HTTPException`. Tests cover rename/move stable identity, complete removal, partial/failure retention, owner isolation, and idempotent repeated reconciliation. - repeated reconciliation of unchanged metadata creates no duplicate row - rename/move updates metadata without changing the DocuVault CloudItem UUID - incomplete or failed listing cannot mark an unseen child deleted - all item/list/folder-state queries include both owner and connection scope - `pytest -q tests/test_cloud_items.py` exits 0 cd backend && pytest -q tests/test_cloud_items.py Owner-scoped reconciliation is idempotent, stable across rename/move, and safe for failed or partial listings.

<threat_model>

Threat ID	Threat	Mitigation and evidence
T-12-01	Connection/item IDOR	Composite owner+connection predicates and negative service tests
T-12-02	Same-provider account collision	Connection UUID in uniqueness, indexes, and every cache/persistence boundary
T-12-06	Destructive capability probing	Read-only interface and fake-adapter test asserting no mutation calls
T-12-07	Partial-list metadata deletion	Deletion reconciliation gated on successful complete=true listing
T-12-09	Quota corruption	Metadata size never invokes quota service; regression test asserts unchanged usage
</threat_model>

1. `cd backend && pytest -q tests/test_cloud_capabilities.py tests/test_cloud_items.py` 2. Run migration upgrade/downgrade against PostgreSQL integration environment. 3. Confirm `rg "HTTPException" backend/services/cloud_items.py` returns no matches. 4. Confirm no Phase 12 task writes provider file bytes or MinIO objects.

<success_criteria>

• Normalized provider-neutral capability and resource contracts are test-covered.
• Durable cloud item/folder freshness metadata is isolated by user and connection.
• Same-provider accounts and same provider item IDs do not collide.
• Metadata operations preserve provider-byte source of truth and quota usage. </success_criteria>

Create .planning/phases/12-cloud-resource-foundation/12-01-SUMMARY.md when complete.