curo1305andClaude Sonnet 4.6
73328eee0b
Plans 01–04, CONTEXT, PATTERNS, RESEARCH, UAT, and .gitkeep for the fix-nextcloud-root-listing-and-sync-visibility sub-phase. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
169 lines
11 KiB
Markdown
169 lines
11 KiB
Markdown
---
|
|
phase: "12.1"
|
|
plan: "02"
|
|
type: tdd
|
|
wave: 2
|
|
depends_on:
|
|
- "12.1-01"
|
|
files_modified:
|
|
- backend/services/cloud_items.py
|
|
- backend/api/cloud/browse.py
|
|
- backend/tasks/cloud_tasks.py
|
|
- backend/api/cloud/schemas.py
|
|
- backend/tests/test_cloud_items.py
|
|
- backend/tests/test_cloud.py
|
|
- backend/tests/test_cloud_security.py
|
|
- backend/main.py
|
|
- frontend/package.json
|
|
- frontend/package-lock.json
|
|
- AGENTS.md
|
|
- README.md
|
|
- RUNBOOK.md
|
|
- SECURITY.md
|
|
autonomous: true
|
|
requirements:
|
|
- CLOUD-01
|
|
- CACHE-01
|
|
- SYNC-01
|
|
|
|
must_haves:
|
|
truths:
|
|
- "CloudListing.complete=False can never transition a folder to fresh in either synchronous browse or Celery refresh"
|
|
- "A complete authoritative empty listing may become fresh and reconcile deletions"
|
|
- "Incomplete/error refresh retains cached children and the previous last successful refresh timestamp"
|
|
- "The synchronous API and background worker use one service-level reconciliation/completion gate"
|
|
- "Connection-ID ownership, credential secrecy, zero-byte browse, and zero quota mutation remain enforced"
|
|
artifacts:
|
|
- path: "backend/services/cloud_items.py"
|
|
provides: "Single listing application gate shared by API and worker"
|
|
contains: "reconcile_cloud_listing"
|
|
- path: "backend/tests/test_cloud_items.py"
|
|
provides: "Complete-versus-incomplete reconciliation and timestamp regression tests"
|
|
contains: "incomplete_listing"
|
|
- path: "backend/tests/test_cloud.py"
|
|
provides: "Connection-ID browse freshness integration tests"
|
|
contains: "complete_false"
|
|
key_links:
|
|
- from: "backend/api/cloud/browse.py"
|
|
to: "backend/services/cloud_items.py"
|
|
via: "shared listing application/finalization service"
|
|
pattern: "reconcile_cloud_listing"
|
|
- from: "backend/tasks/cloud_tasks.py"
|
|
to: "backend/services/cloud_items.py"
|
|
via: "the same service gate used by synchronous browse"
|
|
pattern: "reconcile_cloud_listing"
|
|
- from: "CloudListing.complete"
|
|
to: "CloudFolderState.refresh_state"
|
|
via: "fresh only for authoritative completion"
|
|
pattern: "complete.*fresh"
|
|
---
|
|
|
|
<objective>
|
|
Make backend freshness truthful. Centralize how normalized listings are reconciled and finalized so incomplete, ambiguous, or failed provider results retain metadata and produce a controlled warning rather than a false fresh state.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
|
|
@$HOME/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-CONTEXT.md
|
|
@.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-RESEARCH.md
|
|
@.planning/phases/12-cloud-resource-foundation/12-SECURITY.md
|
|
@backend/services/cloud_items.py
|
|
@backend/api/cloud/browse.py
|
|
@backend/tasks/cloud_tasks.py
|
|
</context>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto">
|
|
<name>Task 1: Specify complete, incomplete, and empty reconciliation semantics</name>
|
|
<files>backend/tests/test_cloud_items.py, backend/tests/test_cloud.py, backend/tests/test_cloud_security.py</files>
|
|
<read_first>
|
|
- backend/services/cloud_items.py — current reconciliation and update_folder_state behavior
|
|
- backend/api/cloud/browse.py — first-visit synchronous fresh transition
|
|
- backend/tasks/cloud_tasks.py — worker fresh transition and retry paths
|
|
- backend/tests/test_cloud_items.py — existing incomplete-retention tests
|
|
- backend/tests/test_cloud.py — browse response/freshness fixtures
|
|
</read_first>
|
|
<action>
|
|
Add red tests named `test_incomplete_listing_never_marks_folder_fresh`, `test_incomplete_listing_retains_cached_rows_and_last_success`, `test_complete_empty_listing_is_authoritative_and_fresh`, `test_partial_items_upsert_without_deleting_unseen_children`, `test_sync_browse_returns_warning_for_complete_false`, and `test_worker_returns_warning_for_complete_false`. Seed a previous successful timestamp and prove it is unchanged on incomplete/error results. Distinguish a provider-confirmed `CloudListing(items=(), complete=True)` from `complete=False`; only the former may soft-delete unseen children and update last_refreshed_at.
|
|
|
|
Add API/security assertions that responses remain whitelisted, warning messages are controlled, raw provider errors/URLs are absent, cached items remain owner scoped, and no get_object/download/MinIO/quota mutation occurs. Exercise both root `parent_ref=None` and a nested opaque provider reference.
|
|
</action>
|
|
<acceptance_criteria>
|
|
- tests fail on the current unconditional fresh transitions
|
|
- tests cover service, synchronous API, and Celery worker paths
|
|
- complete-empty and incomplete-empty results have explicitly different outcomes
|
|
- cached rows and prior last_refreshed_at survive incomplete/error results
|
|
- owner/admin/credential/no-byte/no-quota assertions remain in the focused run
|
|
</acceptance_criteria>
|
|
<verify><automated>cd backend && pytest -q tests/test_cloud_items.py tests/test_cloud.py tests/test_cloud_security.py -k 'incomplete or complete_false or complete_empty or refresh or no_byte or quota'</automated></verify>
|
|
<done>Executable red tests define truthful completion semantics at every backend entry point.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 2: Centralize listing application and freshness finalization</name>
|
|
<files>backend/services/cloud_items.py, backend/api/cloud/browse.py, backend/tasks/cloud_tasks.py, backend/api/cloud/schemas.py, backend/tests/test_cloud_items.py, backend/tests/test_cloud.py</files>
|
|
<read_first>
|
|
- backend/services/cloud_items.py — sole reconciliation entry point and service exception conventions
|
|
- backend/api/cloud/schemas.py — credential-free freshness response contract
|
|
- backend/api/cloud/browse.py — cached-first behavior and controlled warning copy
|
|
- backend/tasks/cloud_tasks.py — terminal/transient classification and retry behavior
|
|
- AGENTS.md — service layer must not raise HTTPException
|
|
</read_first>
|
|
<action>
|
|
Add one service-level operation in `services/cloud_items.py` that calls `reconcile_cloud_listing` and finalizes CloudFolderState according to listing completeness. Both browse.py and cloud_tasks.py must call it; neither caller may independently mark a listing fresh. Preserve `reconcile_cloud_listing` as the sole metadata reconciliation entry point. A complete listing sets fresh, clears controlled errors, and advances last_refreshed_at. An incomplete listing may upsert seen items but cannot delete unseen items, cannot advance last_refreshed_at, and sets warning with a stable code such as `incomplete_listing` and credential-free message.
|
|
|
|
Keep provider exceptions controlled: auth/credential failures remain terminal warnings; transient transport errors retain cached data and bounded retry behavior. Do not serialize raw exceptions, response bodies, full URLs, or credentials. Return a structured service result/domain exception—not HTTPException—so the API can still return cached metadata and authoritative FolderFreshnessOut. Ensure task return status differentiates complete success from incomplete warning without placing credentials or provider item names in broker/task results.
|
|
</action>
|
|
<acceptance_criteria>
|
|
- no unconditional `refresh_state="fresh"` remains after adapter.list_folder in API or worker
|
|
- both paths use one service-level listing completion gate
|
|
- incomplete results expose controlled warning state and retain the previous successful timestamp
|
|
- complete empty results are accepted as fresh and reconcile correctly
|
|
- task arguments/results and API schemas contain no credentials or raw provider details
|
|
</acceptance_criteria>
|
|
<verify><automated>cd backend && pytest -q tests/test_cloud_items.py tests/test_cloud.py tests/test_cloud_security.py && ! rg -U 'listing = await adapter\.list_folder[\s\S]{0,800}refresh_state="fresh"' api/cloud/browse.py tasks/cloud_tasks.py</automated></verify>
|
|
<done>Synchronous and background refreshes share one authoritative completion gate and cannot report an incomplete listing as fresh.</done>
|
|
</task>
|
|
|
|
<task type="auto">
|
|
<name>Task 3: Document, security-review, commit, and push Plan 02</name>
|
|
<files>backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md, RUNBOOK.md, SECURITY.md, .planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-02-SUMMARY.md</files>
|
|
<action>Update required documentation for truthful freshness and the shared listing-application service. Bump the backend/frontend patch version once because synchronization status is user-visible, keeping package-lock metadata synchronized. Run a dedicated security agent over this plan's diff, resolve or escalate every finding, then run focused/full backend tests, Bandit, pip audit, Compose validation, and diff checks. Stage only Plan 02 files, exclude `.env` and unrelated work, commit atomically as `fix(12.1): make cloud freshness authoritative`, and push immediately.</action>
|
|
<verify><automated>docker compose config --quiet && cd backend && pytest -v && bandit -r . && pip audit && cd .. && git diff --check</automated></verify>
|
|
<done>Plan 02 has independent documentation, security approval, green gates, commit, and push.</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<threat_model>
|
|
| Threat ID | Threat | Mitigation and evidence |
|
|
|-----------|--------|-------------------------|
|
|
| T-12.1-06 | False fresh state hides provider failure | Shared completion gate; complete=False warning tests in API and worker |
|
|
| T-12.1-07 | Incomplete listing deletes cached metadata | Reconcile deletion remains guarded by complete=True |
|
|
| T-12.1-08 | Error overwrites last known-good evidence | Prior timestamp preserved on incomplete/error refresh |
|
|
| T-12.1-09 | Provider errors leak secrets or URLs | Stable codes and controlled messages only |
|
|
| T-12.1-10 | Refresh changes bytes/quota | Existing and expanded no-byte/no-MinIO/no-quota assertions |
|
|
</threat_model>
|
|
|
|
<verification>
|
|
1. Demonstrate the new incomplete tests fail before implementation.
|
|
2. Run service tests for complete, complete-empty, partial, incomplete-empty, and exception paths.
|
|
3. Run connection-ID API and Celery task tests for the same state transitions.
|
|
4. Run owner/admin/credential/no-byte/no-quota security negatives.
|
|
5. Search proves callers no longer set fresh independently after list_folder.
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- Backend freshness means the provider supplied a complete authoritative listing.
|
|
- Cached metadata and last successful refresh evidence survive ambiguity/failure.
|
|
- Complete empty folders remain valid and reconcile correctly.
|
|
- Shared architecture and security boundaries remain intact.
|
|
</success_criteria>
|
|
|
|
<output>Create `.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-02-SUMMARY.md` when complete.</output>
|