kite/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-PATTERNS.md

Phase 12.1 Pattern Map: Cloud Listing Correctness

Purpose

Map the existing shared patterns and exact change seams for making root and nested cloud listings correct across Nextcloud, generic WebDAV, Google Drive, and OneDrive. This is a static repository map only: no .env access, credentials, provider calls, or network operations were used.

The supplied Nextcloud root names are acceptance evidence, not configuration or test secrets:

• Folders: Documents, docuvault, Photos, Templates
• Files: Nextcloud.png, Nextcloud Intro.mp4, Manual Nextcloud.png, Readme.md, Reasons to use Nextcloud.pdf, Template credits.md

Root-Cause Map

1. Nextcloud breaks the shared adapter contract

backend/storage/cloud_base.py defines the canonical interface:

await adapter.list_folder(connection_id, user_id, parent_ref=None, page_token=None)
    -> CloudListing

backend/storage/webdav_backend.py implements that interface and already contains the reusable WebDAV/Nextcloud PROPFIND normalization into CloudResource and CloudListing.

backend/storage/nextcloud_backend.py, however, overrides list_folder with the legacy signature list_folder(folder_path="") -> list[dict]. The canonical API and Celery task call it with connection/user UUIDs plus parent_ref, so Nextcloud can raise a signature error before listing. It also returns the legacy shape instead of CloudListing.

Fix pattern: do not add another Nextcloud listing implementation. Remove the incompatible override and inherit WebDAVBackend.list_folder, or move the legacy behavior behind an explicitly named compatibility method used only by the legacy route. The canonical CloudResourceAdapter.list_folder must have one signature and one return type for every provider.

2. The frontend consumes the legacy item shape

backend/api/cloud/schemas.py exposes canonical items as:

id                  DocuVault stable UUID
provider_item_id    provider folder/file reference
kind                "folder" or "file"

frontend/src/views/CloudFolderView.vue still filters by is_dir and navigates with item.id. Consequently:

• canonical folders are not recognized as folders;
• nested navigation sends the DocuVault metadata UUID as parent_ref instead of the provider reference;
• this fails for every provider, not just Nextcloud.

Fix pattern: normalize once at the API/view boundary or consume the canonical schema directly. In CloudFolderView.vue, derive folders/files from item.kind, and navigate using item.provider_item_id. Do not teach StorageBrowser.vue provider-specific IDs or add provider branches.

3. The UI invents freshness instead of rendering provider state

backend/api/cloud/browse.py returns freshness from durable CloudFolderState, including refresh_state, last_refreshed_at, and controlled errors. frontend/src/views/CloudFolderView.vue ignores it and sets freshness: 'fresh' plus the browser's current time after every HTTP 200.

An empty listing caused by a provider failure can therefore appear synchronized.

Fix pattern: map data.freshness.refresh_state into the store's UI vocabulary and use data.freshness.last_refreshed_at; preserve error_code/error_message as the user-visible warning source. Never infer provider freshness from HTTP success.

4. Empty complete listings and failed listings must remain distinguishable

CloudListing.complete in backend/storage/cloud_base.py is the authoritative distinction:

• complete=True: full listing; reconciliation may soft-delete unseen children.
• complete=False: partial/failed listing; reconciliation must retain cached children.

backend/services/cloud_items.py::reconcile_cloud_listing already implements this correctly. Keep it as the only metadata reconciliation entry point. Adapter/API fixes must not bypass it or update CloudItem rows directly.

The synchronous first-load path in backend/api/cloud/browse.py should only mark a folder fresh after an authoritative result. A CloudListing(complete=False) must produce a controlled warning/stale state and must not be reported as fresh. Apply the same decision in backend/tasks/cloud_tasks.py so synchronous and background refreshes cannot disagree.

Shared Pipeline and Exact Files

Adapter contract and normalization

• backend/storage/cloud_base.py
  • Preserve CloudResourceAdapter.list_folder as the sole browse contract.
  • Preserve normalized CloudResource identity fields and CloudListing.complete semantics.
  • If validation is strengthened, add provider-neutral checks here rather than in four adapters (for example, reject duplicate provider IDs within one authoritative page/listing).
• backend/storage/webdav_backend.py
  • Keep one DAV listing implementation for both WebDAV and Nextcloud.
  • Root is parent_ref=None, translated internally to the SDK's empty relative path.
  • Continue returning provider-relative paths as provider_item_id and the requested parent reference as parent_ref.
  • Keep SSRF revalidation before outbound calls and the no-byte-download invariant.
• backend/storage/nextcloud_backend.py
  • Remove/rename the legacy list_folder(folder_path) override so the inherited canonical implementation is used.
  • Retain only genuinely Nextcloud-specific behavior such as root health-check path conventions.
  • Do not duplicate the PROPFIND normalization loop.
• backend/storage/google_drive_backend.py
  • Preserve root translation None -> "root", provider IDs for navigation, full pagination, and complete=False on provider failure.
• backend/storage/onedrive_backend.py
  • Preserve root translation to /me/drive/root/children, item IDs for navigation, @odata.nextLink exhaustion, and incomplete failure semantics.
• backend/storage/cloud_backend_factory.py
  • Continue constructing all providers behind build_cloud_resource_adapter.
  • Add/retain a defensive contract assertion, but do not special-case list behavior by provider.

Service and task patterns

• backend/services/cloud_items.py
  • resolve_owned_connection: owner boundary before adapter access.
  • upsert_cloud_item: stable metadata identity by (connection_id, provider_item_id).
  • reconcile_cloud_listing: only reconciliation entry point; soft-delete only for complete listings.
  • list_cloud_children: query by the exact canonical parent_ref used during reconciliation.
  • update_folder_state: canonical controlled freshness/error state; never raw provider exception text.
• backend/tasks/cloud_tasks.py
  • Keep credential decryption inside the worker.
  • Call only the canonical adapter method.
  • Treat complete=False as refresh failure/warning rather than successful freshness.
  • Preserve bounded retries, owner revalidation, metadata-only operation, and no quota mutation/download.

API patterns

• backend/api/cloud/browse.py
  • Keep /api/cloud/connections/{connection_id}/items as the canonical owner-scoped endpoint.
  • Preserve whitelisted response conversion through _item_out, _capability_out, and _freshness_out.
  • First visit may fetch synchronously, but must not mark an incomplete listing fresh.
  • Cached visits continue stale-while-revalidate through refresh_cloud_folder.delay.
  • Retire or isolate _fetch_*_folders compatibility helpers; do not let the legacy route define new listing semantics.
• backend/api/cloud/connections.py
  • The /folders/{provider}/{folder_id} route is legacy. Migrate remaining consumers/tests to the connection-ID endpoint, then remove it when unreferenced rather than maintaining two provider listing pipelines.
• backend/api/cloud/schemas.py
  • Keep the explicit credential-free schema.
  • kind and provider_item_id are canonical; avoid restoring is_dir solely to accommodate stale frontend code unless a deliberate transitional compatibility field is documented and tested.

Frontend patterns

• frontend/src/api/cloud.js
  • Keep getCloudFoldersByConnectionId(connectionId, parentRef) as the single browse client.
  • Preserve URL encoding of opaque/path-like provider references.
  • Remove deprecated getCloudFolders(provider, folderId) once no active consumer remains.
• frontend/src/views/CloudFolderView.vue
  • Remain a thin data provider.
  • Split items using kind and navigate folders using provider_item_id.
  • Feed backend freshness and controlled warning data into the store; do not synthesize “fresh”.
  • Keep connection UUID routing and session-only folder memory.
  • Do not add provider switches or listing layout.
• frontend/src/stores/cloudConnections.js
  • Keep shared browse state here.
  • If freshness translation is non-trivial, add one shared mapper/action here and test it once; views should not each invent status semantics.
• frontend/src/components/storage/StorageBrowser.vue
  • Remain the only local/cloud file browser.
  • Receive already normalized folders/files and capability/freshness props.
  • No provider-specific path parsing or Nextcloud-only rendering.

Test Design

Provider contract suite: one parametrized contract, provider fixtures only

Extend backend/tests/test_cloud_backends.py with a shared adapter contract harness covering Google Drive, OneDrive, WebDAV, and Nextcloud. Each provider supplies mocked SDK/HTTP responses; assertions stay provider-neutral:

1. Root call accepts (connection_id, user_id, parent_ref=None) and returns CloudListing.
2. Every child is a CloudResource with the supplied connection/user IDs.
3. Folder/file classification, names, provider IDs, parent refs, sizes, MIME types, timestamps, and etags normalize correctly.
4. Nested listing accepts a returned folder's provider_item_id as the next parent_ref.
5. Pagination is exhausted before complete=True (where applicable).
6. Provider/network failure returns or is translated to complete=False without byte downloads or mutation calls.
7. No adapter invokes get_object, upload, delete, move, rename, or quota code while browsing.

Add an explicit Nextcloud regression in backend/tests/test_cloud_backends.py:

• instantiate NextcloudBackend with mocked webdav3 client;
• call the canonical four-argument adapter method;
• return the ten supplied root entries from mocked PROPFIND metadata;
• assert exactly four normalized folders and six normalized files by the expected names;
• assert root children have parent_ref is None;
• select Documents and prove its provider_item_id can be passed back for a nested listing.

This test uses only expected names as fixture evidence. It must not contain a URL, username, app-password, encrypted credentials, or network marker.

Service reconciliation suite

Extend backend/tests/test_cloud_items.py:

• authoritative root listing upserts all ten unique Nextcloud fixture names;
• repeating the listing is idempotent;
• renamed/moved metadata preserves DocuVault item UUID by provider ID;
• complete=False retains cached rows and records no deletions;
• empty complete=True is allowed to remove known children, proving failure and true-empty semantics remain distinct;
• root (None) and nested parent refs never collide across connection/user boundaries.

Task suite

Extend backend/tests/test_cloud_items.py or create backend/tests/test_cloud_tasks.py if task behavior becomes substantial:

• Nextcloud adapter receives canonical arguments through _run;
• complete listing reconciles and marks fresh;
• incomplete listing marks warning/stale and does not delete cached items;
• transient exception retries, terminal auth failure does not retry;
• credentials stay out of broker payload and logs;
• no byte or quota APIs are called.

API integration suite

Extend backend/tests/test_cloud.py and backend/tests/test_cloud_security.py:

• first root browse with a mocked provider contract returns all normalized items;
• response exposes kind and provider_item_id, never credentials;
• incomplete first fetch returns controlled non-fresh status rather than a false fresh status;
• cached listing is returned while refresh is scheduled;
• nested browse forwards the exact provider reference;
• all four provider values pass through the same endpoint;
• wrong-owner/admin requests remain 404/403 and never invoke adapters;
• browse never downloads bytes or mutates quota.

Frontend suite

Extend frontend/src/views/__tests__/CloudFolderView.test.js:

• canonical kind: 'folder' and kind: 'file' entries are passed to StorageBrowser in the right props;
• clicking Documents routes with its provider_item_id, not its DocuVault id;
• root request omits/translates the root sentinel consistently to canonical parent_ref=None at the API boundary;
• backend freshness.refresh_state, timestamp, and warning message populate store/UI state;
• HTTP 200 with warning/incomplete state never becomes “fresh” merely because the request resolved;
• expected ten-name Nextcloud fixture is rendered through the same shared browser as every provider.

Extend frontend/src/stores/__tests__/cloudConnections.test.js if freshness mapping is placed in the store. Preserve existing connection UUID and session-storage secrecy tests.

Acceptance Matrix

Layer	Provider-neutral acceptance
Adapter	All four providers implement the exact CloudResourceAdapter signature and return CloudListing
Root identity	UI root sentinel becomes backend parent_ref=None; adapters translate root internally
Child identity	provider_item_id is used for provider navigation; id remains DocuVault metadata identity
Classification	kind is the sole canonical folder/file discriminator
Reconciliation	Only reconcile_cloud_listing writes listing metadata; incomplete results never delete cached rows
Freshness	Backend folder state is authoritative; incomplete/error results cannot display as fresh
Security	Owner scoping, credential secrecy, SSRF checks, no byte download, and no quota mutation remain intact
Nextcloud evidence	Root shows exactly the four supplied folders and six supplied files under mocked acceptance data

Implementation Order

1. Add the shared provider contract tests and failing Nextcloud signature/root regression.
2. Remove the Nextcloud legacy override in favor of inherited DAV normalization.
3. Make incomplete-listing handling consistent in synchronous API and background task paths.
4. Fix CloudFolderView.vue to use kind, provider_item_id, and backend freshness.
5. Add API and frontend regressions for root, nested navigation, warning truthfulness, and all-provider parity.
6. Remove the legacy provider-slug browse path/helpers when repository search proves no active consumer remains.

No provider-specific file grid, reconciliation function, freshness mapper, or Nextcloud-only listing parser should be introduced.