kite/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-02-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
12.1	02	tdd	2	12.1-01	backend/services/cloud_items.py backend/api/cloud/browse.py backend/tasks/cloud_tasks.py backend/api/cloud/schemas.py backend/tests/test_cloud_items.py backend/tests/test_cloud.py backend/tests/test_cloud_security.py backend/main.py frontend/package.json frontend/package-lock.json AGENTS.md README.md RUNBOOK.md SECURITY.md	true	CLOUD-01 CACHE-01 SYNC-01	truths artifacts key_links CloudListing.complete=False can never transition a folder to fresh in either synchronous browse or Celery refresh A complete authoritative empty listing may become fresh and reconcile deletions Incomplete/error refresh retains cached children and the previous last successful refresh timestamp The synchronous API and background worker use one service-level reconciliation/completion gate Connection-ID ownership, credential secrecy, zero-byte browse, and zero quota mutation remain enforced path provides contains backend/services/cloud_items.py Single listing application gate shared by API and worker reconcile_cloud_listing path provides contains backend/tests/test_cloud_items.py Complete-versus-incomplete reconciliation and timestamp regression tests incomplete_listing path provides contains backend/tests/test_cloud.py Connection-ID browse freshness integration tests complete_false from to via pattern backend/api/cloud/browse.py backend/services/cloud_items.py shared listing application/finalization service reconcile_cloud_listing from to via pattern backend/tasks/cloud_tasks.py backend/services/cloud_items.py the same service gate used by synchronous browse reconcile_cloud_listing from to via pattern CloudListing.complete CloudFolderState.refresh_state fresh only for authoritative completion complete.*fresh

Make backend freshness truthful. Centralize how normalized listings are reconciled and finalized so incomplete, ambiguous, or failed provider results retain metadata and produce a controlled warning rather than a false fresh state.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-CONTEXT.md @.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-RESEARCH.md @.planning/phases/12-cloud-resource-foundation/12-SECURITY.md @backend/services/cloud_items.py @backend/api/cloud/browse.py @backend/tasks/cloud_tasks.py Task 1: Specify complete, incomplete, and empty reconciliation semantics backend/tests/test_cloud_items.py, backend/tests/test_cloud.py, backend/tests/test_cloud_security.py - backend/services/cloud_items.py — current reconciliation and update_folder_state behavior - backend/api/cloud/browse.py — first-visit synchronous fresh transition - backend/tasks/cloud_tasks.py — worker fresh transition and retry paths - backend/tests/test_cloud_items.py — existing incomplete-retention tests - backend/tests/test_cloud.py — browse response/freshness fixtures Add red tests named `test_incomplete_listing_never_marks_folder_fresh`, `test_incomplete_listing_retains_cached_rows_and_last_success`, `test_complete_empty_listing_is_authoritative_and_fresh`, `test_partial_items_upsert_without_deleting_unseen_children`, `test_sync_browse_returns_warning_for_complete_false`, and `test_worker_returns_warning_for_complete_false`. Seed a previous successful timestamp and prove it is unchanged on incomplete/error results. Distinguish a provider-confirmed `CloudListing(items=(), complete=True)` from `complete=False`; only the former may soft-delete unseen children and update last_refreshed_at.

Add API/security assertions that responses remain whitelisted, warning messages are controlled, raw provider errors/URLs are absent, cached items remain owner scoped, and no get_object/download/MinIO/quota mutation occurs. Exercise both root `parent_ref=None` and a nested opaque provider reference.

- tests fail on the current unconditional fresh transitions - tests cover service, synchronous API, and Celery worker paths - complete-empty and incomplete-empty results have explicitly different outcomes - cached rows and prior last_refreshed_at survive incomplete/error results - owner/admin/credential/no-byte/no-quota assertions remain in the focused run cd backend && pytest -q tests/test_cloud_items.py tests/test_cloud.py tests/test_cloud_security.py -k 'incomplete or complete_false or complete_empty or refresh or no_byte or quota' Executable red tests define truthful completion semantics at every backend entry point. Task 2: Centralize listing application and freshness finalization backend/services/cloud_items.py, backend/api/cloud/browse.py, backend/tasks/cloud_tasks.py, backend/api/cloud/schemas.py, backend/tests/test_cloud_items.py, backend/tests/test_cloud.py - backend/services/cloud_items.py — sole reconciliation entry point and service exception conventions - backend/api/cloud/schemas.py — credential-free freshness response contract - backend/api/cloud/browse.py — cached-first behavior and controlled warning copy - backend/tasks/cloud_tasks.py — terminal/transient classification and retry behavior - AGENTS.md — service layer must not raise HTTPException Add one service-level operation in `services/cloud_items.py` that calls `reconcile_cloud_listing` and finalizes CloudFolderState according to listing completeness. Both browse.py and cloud_tasks.py must call it; neither caller may independently mark a listing fresh. Preserve `reconcile_cloud_listing` as the sole metadata reconciliation entry point. A complete listing sets fresh, clears controlled errors, and advances last_refreshed_at. An incomplete listing may upsert seen items but cannot delete unseen items, cannot advance last_refreshed_at, and sets warning with a stable code such as `incomplete_listing` and credential-free message.

Keep provider exceptions controlled: auth/credential failures remain terminal warnings; transient transport errors retain cached data and bounded retry behavior. Do not serialize raw exceptions, response bodies, full URLs, or credentials. Return a structured service result/domain exception—not HTTPException—so the API can still return cached metadata and authoritative FolderFreshnessOut. Ensure task return status differentiates complete success from incomplete warning without placing credentials or provider item names in broker/task results.

- no unconditional `refresh_state="fresh"` remains after adapter.list_folder in API or worker - both paths use one service-level listing completion gate - incomplete results expose controlled warning state and retain the previous successful timestamp - complete empty results are accepted as fresh and reconcile correctly - task arguments/results and API schemas contain no credentials or raw provider details cd backend && pytest -q tests/test_cloud_items.py tests/test_cloud.py tests/test_cloud_security.py && ! rg -U 'listing = await adapter\.list_folder[\s\S]{0,800}refresh_state="fresh"' api/cloud/browse.py tasks/cloud_tasks.py Synchronous and background refreshes share one authoritative completion gate and cannot report an incomplete listing as fresh. Task 3: Document, security-review, commit, and push Plan 02 backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md, RUNBOOK.md, SECURITY.md, .planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-02-SUMMARY.md Update required documentation for truthful freshness and the shared listing-application service. Bump the backend/frontend patch version once because synchronization status is user-visible, keeping package-lock metadata synchronized. Run a dedicated security agent over this plan's diff, resolve or escalate every finding, then run focused/full backend tests, Bandit, pip audit, Compose validation, and diff checks. Stage only Plan 02 files, exclude `.env` and unrelated work, commit atomically as `fix(12.1): make cloud freshness authoritative`, and push immediately. docker compose config --quiet && cd backend && pytest -v && bandit -r . && pip audit && cd .. && git diff --check Plan 02 has independent documentation, security approval, green gates, commit, and push.

<threat_model>

Threat ID	Threat	Mitigation and evidence
T-12.1-06	False fresh state hides provider failure	Shared completion gate; complete=False warning tests in API and worker
T-12.1-07	Incomplete listing deletes cached metadata	Reconcile deletion remains guarded by complete=True
T-12.1-08	Error overwrites last known-good evidence	Prior timestamp preserved on incomplete/error refresh
T-12.1-09	Provider errors leak secrets or URLs	Stable codes and controlled messages only
T-12.1-10	Refresh changes bytes/quota	Existing and expanded no-byte/no-MinIO/no-quota assertions
</threat_model>

1. Demonstrate the new incomplete tests fail before implementation. 2. Run service tests for complete, complete-empty, partial, incomplete-empty, and exception paths. 3. Run connection-ID API and Celery task tests for the same state transitions. 4. Run owner/admin/credential/no-byte/no-quota security negatives. 5. Search proves callers no longer set fresh independently after list_folder.

<success_criteria>

• Backend freshness means the provider supplied a complete authoritative listing.
• Cached metadata and last successful refresh evidence survive ambiguity/failure.
• Complete empty folders remain valid and reconcile correctly.
• Shared architecture and security boundaries remain intact. </success_criteria>

Create .planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-02-SUMMARY.md when complete.