kite/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-PATTERNS.md

# Phase 12.1 Pattern Map: Cloud Listing Correctness

## Purpose

Map the existing shared patterns and exact change seams for making root and nested cloud listings correct across Nextcloud, generic WebDAV, Google Drive, and OneDrive. This is a static repository map only: no `.env` access, credentials, provider calls, or network operations were used.

The supplied Nextcloud root names are acceptance evidence, not configuration or test secrets:

- Folders: `Documents`, `docuvault`, `Photos`, `Templates`
- Files: `Nextcloud.png`, `Nextcloud Intro.mp4`, `Manual Nextcloud.png`, `Readme.md`, `Reasons to use Nextcloud.pdf`, `Template credits.md`

## Root-Cause Map

### 1. Nextcloud breaks the shared adapter contract

`backend/storage/cloud_base.py` defines the canonical interface:

```python
await adapter.list_folder(connection_id, user_id, parent_ref=None, page_token=None)
    -> CloudListing
```

`backend/storage/webdav_backend.py` implements that interface and already contains the reusable WebDAV/Nextcloud PROPFIND normalization into `CloudResource` and `CloudListing`.

`backend/storage/nextcloud_backend.py`, however, overrides `list_folder` with the legacy signature `list_folder(folder_path="") -> list[dict]`. The canonical API and Celery task call it with connection/user UUIDs plus `parent_ref`, so Nextcloud can raise a signature error before listing. It also returns the legacy shape instead of `CloudListing`.

**Fix pattern:** do not add another Nextcloud listing implementation. Remove the incompatible override and inherit `WebDAVBackend.list_folder`, or move the legacy behavior behind an explicitly named compatibility method used only by the legacy route. The canonical `CloudResourceAdapter.list_folder` must have one signature and one return type for every provider.

### 2. The frontend consumes the legacy item shape

`backend/api/cloud/schemas.py` exposes canonical items as:

```text
id                  DocuVault stable UUID
provider_item_id    provider folder/file reference
kind                "folder" or "file"
```

`frontend/src/views/CloudFolderView.vue` still filters by `is_dir` and navigates with `item.id`. Consequently:

- canonical folders are not recognized as folders;
- nested navigation sends the DocuVault metadata UUID as `parent_ref` instead of the provider reference;
- this fails for every provider, not just Nextcloud.

**Fix pattern:** normalize once at the API/view boundary or consume the canonical schema directly. In `CloudFolderView.vue`, derive folders/files from `item.kind`, and navigate using `item.provider_item_id`. Do not teach `StorageBrowser.vue` provider-specific IDs or add provider branches.

### 3. The UI invents freshness instead of rendering provider state

`backend/api/cloud/browse.py` returns `freshness` from durable `CloudFolderState`, including `refresh_state`, `last_refreshed_at`, and controlled errors. `frontend/src/views/CloudFolderView.vue` ignores it and sets `freshness: 'fresh'` plus the browser's current time after every HTTP 200.

An empty listing caused by a provider failure can therefore appear synchronized.

**Fix pattern:** map `data.freshness.refresh_state` into the store's UI vocabulary and use `data.freshness.last_refreshed_at`; preserve `error_code/error_message` as the user-visible warning source. Never infer provider freshness from HTTP success.

### 4. Empty complete listings and failed listings must remain distinguishable

`CloudListing.complete` in `backend/storage/cloud_base.py` is the authoritative distinction:

- `complete=True`: full listing; reconciliation may soft-delete unseen children.
- `complete=False`: partial/failed listing; reconciliation must retain cached children.

`backend/services/cloud_items.py::reconcile_cloud_listing` already implements this correctly. Keep it as the only metadata reconciliation entry point. Adapter/API fixes must not bypass it or update `CloudItem` rows directly.

The synchronous first-load path in `backend/api/cloud/browse.py` should only mark a folder fresh after an authoritative result. A `CloudListing(complete=False)` must produce a controlled warning/stale state and must not be reported as fresh. Apply the same decision in `backend/tasks/cloud_tasks.py` so synchronous and background refreshes cannot disagree.

## Shared Pipeline and Exact Files

### Adapter contract and normalization

- `backend/storage/cloud_base.py`
  - Preserve `CloudResourceAdapter.list_folder` as the sole browse contract.
  - Preserve normalized `CloudResource` identity fields and `CloudListing.complete` semantics.
  - If validation is strengthened, add provider-neutral checks here rather than in four adapters (for example, reject duplicate provider IDs within one authoritative page/listing).
- `backend/storage/webdav_backend.py`
  - Keep one DAV listing implementation for both WebDAV and Nextcloud.
  - Root is `parent_ref=None`, translated internally to the SDK's empty relative path.
  - Continue returning provider-relative paths as `provider_item_id` and the requested parent reference as `parent_ref`.
  - Keep SSRF revalidation before outbound calls and the no-byte-download invariant.
- `backend/storage/nextcloud_backend.py`
  - Remove/rename the legacy `list_folder(folder_path)` override so the inherited canonical implementation is used.
  - Retain only genuinely Nextcloud-specific behavior such as root health-check path conventions.
  - Do not duplicate the PROPFIND normalization loop.
- `backend/storage/google_drive_backend.py`
  - Preserve root translation `None -> "root"`, provider IDs for navigation, full pagination, and `complete=False` on provider failure.
- `backend/storage/onedrive_backend.py`
  - Preserve root translation to `/me/drive/root/children`, item IDs for navigation, `@odata.nextLink` exhaustion, and incomplete failure semantics.
- `backend/storage/cloud_backend_factory.py`
  - Continue constructing all providers behind `build_cloud_resource_adapter`.
  - Add/retain a defensive contract assertion, but do not special-case list behavior by provider.

### Service and task patterns

- `backend/services/cloud_items.py`
  - `resolve_owned_connection`: owner boundary before adapter access.
  - `upsert_cloud_item`: stable metadata identity by `(connection_id, provider_item_id)`.
  - `reconcile_cloud_listing`: only reconciliation entry point; soft-delete only for complete listings.
  - `list_cloud_children`: query by the exact canonical `parent_ref` used during reconciliation.
  - `update_folder_state`: canonical controlled freshness/error state; never raw provider exception text.
- `backend/tasks/cloud_tasks.py`
  - Keep credential decryption inside the worker.
  - Call only the canonical adapter method.
  - Treat `complete=False` as refresh failure/warning rather than successful freshness.
  - Preserve bounded retries, owner revalidation, metadata-only operation, and no quota mutation/download.

### API patterns

- `backend/api/cloud/browse.py`
  - Keep `/api/cloud/connections/{connection_id}/items` as the canonical owner-scoped endpoint.
  - Preserve whitelisted response conversion through `_item_out`, `_capability_out`, and `_freshness_out`.
  - First visit may fetch synchronously, but must not mark an incomplete listing fresh.
  - Cached visits continue stale-while-revalidate through `refresh_cloud_folder.delay`.
  - Retire or isolate `_fetch_*_folders` compatibility helpers; do not let the legacy route define new listing semantics.
- `backend/api/cloud/connections.py`
  - The `/folders/{provider}/{folder_id}` route is legacy. Migrate remaining consumers/tests to the connection-ID endpoint, then remove it when unreferenced rather than maintaining two provider listing pipelines.
- `backend/api/cloud/schemas.py`
  - Keep the explicit credential-free schema.
  - `kind` and `provider_item_id` are canonical; avoid restoring `is_dir` solely to accommodate stale frontend code unless a deliberate transitional compatibility field is documented and tested.

### Frontend patterns

- `frontend/src/api/cloud.js`
  - Keep `getCloudFoldersByConnectionId(connectionId, parentRef)` as the single browse client.
  - Preserve URL encoding of opaque/path-like provider references.
  - Remove deprecated `getCloudFolders(provider, folderId)` once no active consumer remains.
- `frontend/src/views/CloudFolderView.vue`
  - Remain a thin data provider.
  - Split items using `kind` and navigate folders using `provider_item_id`.
  - Feed backend freshness and controlled warning data into the store; do not synthesize “fresh”.
  - Keep connection UUID routing and session-only folder memory.
  - Do not add provider switches or listing layout.
- `frontend/src/stores/cloudConnections.js`
  - Keep shared browse state here.
  - If freshness translation is non-trivial, add one shared mapper/action here and test it once; views should not each invent status semantics.
- `frontend/src/components/storage/StorageBrowser.vue`
  - Remain the only local/cloud file browser.
  - Receive already normalized folders/files and capability/freshness props.
  - No provider-specific path parsing or Nextcloud-only rendering.

## Test Design

### Provider contract suite: one parametrized contract, provider fixtures only

Extend `backend/tests/test_cloud_backends.py` with a shared adapter contract harness covering Google Drive, OneDrive, WebDAV, and Nextcloud. Each provider supplies mocked SDK/HTTP responses; assertions stay provider-neutral:

1. Root call accepts `(connection_id, user_id, parent_ref=None)` and returns `CloudListing`.
2. Every child is a `CloudResource` with the supplied connection/user IDs.
3. Folder/file classification, names, provider IDs, parent refs, sizes, MIME types, timestamps, and etags normalize correctly.
4. Nested listing accepts a returned folder's `provider_item_id` as the next `parent_ref`.
5. Pagination is exhausted before `complete=True` (where applicable).
6. Provider/network failure returns or is translated to `complete=False` without byte downloads or mutation calls.
7. No adapter invokes `get_object`, upload, delete, move, rename, or quota code while browsing.

Add an explicit Nextcloud regression in `backend/tests/test_cloud_backends.py`:

- instantiate `NextcloudBackend` with mocked `webdav3` client;
- call the canonical four-argument adapter method;
- return the ten supplied root entries from mocked PROPFIND metadata;
- assert exactly four normalized folders and six normalized files by the expected names;
- assert root children have `parent_ref is None`;
- select `Documents` and prove its `provider_item_id` can be passed back for a nested listing.

This test uses only expected names as fixture evidence. It must not contain a URL, username, app-password, encrypted credentials, or network marker.

### Service reconciliation suite

Extend `backend/tests/test_cloud_items.py`:

- authoritative root listing upserts all ten unique Nextcloud fixture names;
- repeating the listing is idempotent;
- renamed/moved metadata preserves DocuVault item UUID by provider ID;
- `complete=False` retains cached rows and records no deletions;
- empty `complete=True` is allowed to remove known children, proving failure and true-empty semantics remain distinct;
- root (`None`) and nested parent refs never collide across connection/user boundaries.

### Task suite

Extend `backend/tests/test_cloud_items.py` or create `backend/tests/test_cloud_tasks.py` if task behavior becomes substantial:

- Nextcloud adapter receives canonical arguments through `_run`;
- complete listing reconciles and marks fresh;
- incomplete listing marks warning/stale and does not delete cached items;
- transient exception retries, terminal auth failure does not retry;
- credentials stay out of broker payload and logs;
- no byte or quota APIs are called.

### API integration suite

Extend `backend/tests/test_cloud.py` and `backend/tests/test_cloud_security.py`:

- first root browse with a mocked provider contract returns all normalized items;
- response exposes `kind` and `provider_item_id`, never credentials;
- incomplete first fetch returns controlled non-fresh status rather than a false fresh status;
- cached listing is returned while refresh is scheduled;
- nested browse forwards the exact provider reference;
- all four provider values pass through the same endpoint;
- wrong-owner/admin requests remain 404/403 and never invoke adapters;
- browse never downloads bytes or mutates quota.

### Frontend suite

Extend `frontend/src/views/__tests__/CloudFolderView.test.js`:

- canonical `kind: 'folder'` and `kind: 'file'` entries are passed to `StorageBrowser` in the right props;
- clicking `Documents` routes with its `provider_item_id`, not its DocuVault `id`;
- root request omits/translates the `root` sentinel consistently to canonical `parent_ref=None` at the API boundary;
- backend `freshness.refresh_state`, timestamp, and warning message populate store/UI state;
- HTTP 200 with warning/incomplete state never becomes “fresh” merely because the request resolved;
- expected ten-name Nextcloud fixture is rendered through the same shared browser as every provider.

Extend `frontend/src/stores/__tests__/cloudConnections.test.js` if freshness mapping is placed in the store. Preserve existing connection UUID and session-storage secrecy tests.

## Acceptance Matrix

| Layer | Provider-neutral acceptance |
|---|---|
| Adapter | All four providers implement the exact `CloudResourceAdapter` signature and return `CloudListing` |
| Root identity | UI root sentinel becomes backend `parent_ref=None`; adapters translate root internally |
| Child identity | `provider_item_id` is used for provider navigation; `id` remains DocuVault metadata identity |
| Classification | `kind` is the sole canonical folder/file discriminator |
| Reconciliation | Only `reconcile_cloud_listing` writes listing metadata; incomplete results never delete cached rows |
| Freshness | Backend folder state is authoritative; incomplete/error results cannot display as fresh |
| Security | Owner scoping, credential secrecy, SSRF checks, no byte download, and no quota mutation remain intact |
| Nextcloud evidence | Root shows exactly the four supplied folders and six supplied files under mocked acceptance data |

## Implementation Order

1. Add the shared provider contract tests and failing Nextcloud signature/root regression.
2. Remove the Nextcloud legacy override in favor of inherited DAV normalization.
3. Make incomplete-listing handling consistent in synchronous API and background task paths.
4. Fix `CloudFolderView.vue` to use `kind`, `provider_item_id`, and backend freshness.
5. Add API and frontend regressions for root, nested navigation, warning truthfulness, and all-provider parity.
6. Remove the legacy provider-slug browse path/helpers when repository search proves no active consumer remains.

No provider-specific file grid, reconciliation function, freshness mapper, or Nextcloud-only listing parser should be introduced.