diff --git a/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-VALIDATION.md b/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-VALIDATION.md index 179beb7..6d04807 100644 --- a/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-VALIDATION.md +++ b/.planning/phases/12.1-fix-nextcloud-root-listing-and-sync-visibility/12.1-VALIDATION.md @@ -1,6 +1,6 @@ # Phase 12.1: Fix Nextcloud Root Listing and Sync Visibility — Validation Matrix -**Status:** Plan 04 Task 3 complete. All regression, security, and rendered-flow gates pass. +**Status:** Plan 04 complete. Phase 12.1 closed. All gates pass, docs updated, v0.2.6 committed. **Updated:** 2026-06-22 --- diff --git a/CLAUDE.md b/CLAUDE.md index 97cba0e..bcddc6a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -4,7 +4,7 @@ DocuVault is a multi-user SaaS document management platform built on FastAPI (Python) + Vue 3. It handles document upload, text extraction (PDF/DOCX/image/text), AI-based topic classification, per-user isolated storage, folder organization, document sharing, and pluggable cloud storage backends (OneDrive, Google Drive, Nextcloud, WebDAV). -**Current state:** v0.2.5 — Phase 12.1 Plan 03 complete 2026-06-22. Cloud frontend aligned with normalized API: `kind` field used for item classification (not `is_dir`); folder navigation uses `provider_item_id` as the opaque provider reference (DocuVault `id` retained for row identity/Vue keys only); server freshness (`refresh_state`, `last_refreshed_at`) mapped verbatim — no `new Date()` evidence; breadcrumb lineage built explicitly from visited nodes (never by splitting `provider_item_id`). Phase 12.1 in progress. Not cleared for public internet deployment. +**Current state:** v0.2.6 — Phase 12.1 complete 2026-06-22. Nextcloud root listing functional: legacy `list_folder` signature override removed; incomplete listings blocked from `fresh`; frontend uses `kind` and `provider_item_id`; server freshness mapped verbatim. Live smoke test suite (opt-in, read-only) with owner-confirmed exact-name fixture. Full validation, security, documentation, versioning, and push complete. Not cleared for public internet deployment. ## Stack diff --git a/README.md b/README.md index 7059abf..c6e49f3 100644 --- a/README.md +++ b/README.md @@ -1,6 +1,6 @@ # DocuVault -**Version 0.2.5 — Alpha** +**Version 0.2.6 — Alpha** > **Not production-ready.** DocuVault is functional for local and self-hosted use but has not been audited or hardened for public internet exposure. APIs, environment variables, and the database schema may change without notice until a stable 1.0 release is declared. @@ -323,6 +323,24 @@ Each connected account is independently addressable by its connection UUID: Browsing returns durable cached rows immediately and schedules a background `refresh_cloud_folder` Celery task to reconcile provider changes. First-visit fetches are synchronous and bounded. All browse responses are credential-free — `credentials_enc` is never serialized in the response. +**Freshness states:** Browse responses include `freshness.refresh_state` (`fresh` | `warning` | `refreshing` | `stale`). `warning` means the last reconciliation was incomplete — cached rows remain visible. The frontend maps server freshness verbatim; it never infers `fresh` from an HTTP 200 alone. + +**Phase 12.1 corrections (v0.2.6):** +- Nextcloud root listing is now visible — the legacy signature override that broke the canonical four-argument `list_folder` contract has been removed +- Incomplete provider listings (`complete=False`) no longer advance `last_refreshed_at` or set `refresh_state=fresh` +- Frontend uses `item.kind` (`folder`|`file`) and `item.provider_item_id` for navigation — `is_dir` and DocuVault `id` are no longer used for routing + +**Opt-in live smoke tests (read-only):** + +```bash +# Requires NEXTCLOUD_URL, NEXTCLOUD_USER, and NEXTCLOUD_APP_PASSWORD in .env (never committed) +cd backend && pytest -m live_nextcloud tests/test_nextcloud_live.py +``` + +The live suite is excluded from ordinary CI runs. It is read-only — PROPFIND metadata requests only. No bytes are downloaded and no provider mutations are made. Missing variables produce a safe skip. + +**Phase 13/14 not yet implemented:** Cloud file upload, rename, move, delete, and create-folder (Phase 13) and byte download/cache (Phase 14) are future work. Current browse is read-only. + --- ## Security Highlights diff --git a/RUNBOOK.md b/RUNBOOK.md index 5e946ef..70c59a7 100644 --- a/RUNBOOK.md +++ b/RUNBOOK.md @@ -734,6 +734,54 @@ WHERE cc.id IS NULL; - **Credential fields:** `credentials_enc` is never included in browse API responses. Verify with: `grep -r "credentials_enc" backend/api/cloud/` — must only appear in connection create/update service code, never in response schemas. - **SSRF guard:** `storage/cloud_utils.validate_cloud_url` blocks RFC1918, link-local, file://, and cloud metadata IPs. All WebDAV/Nextcloud URLs pass through this guard before any connection is established. +### Phase 12.1 Live Smoke Tests (Opt-In, Read-Only) + +The `live_nextcloud` pytest marker selects opt-in tests that use real Nextcloud credentials. These tests are **excluded from ordinary CI runs** (`addopts = -m "not live_nextcloud"` in `pytest.ini`). + +**Running the live smoke:** +```bash +# Set in .env (never committed): +# NEXTCLOUD_URL= +# NEXTCLOUD_USER= +# NEXTCLOUD_APP_PASSWORD= + +cd backend +pytest -m live_nextcloud tests/test_nextcloud_live.py +``` + +**What the live tests do:** +- `test_nextcloud_adapter_read_only_root_metadata` — invokes production adapter directly via `PROPFIND`; asserts no byte/mutation method exists; verifies item ownership identity +- `test_nextcloud_root_diagnostic` — sanitized count/kind diagnostic; prints total, expected-match count, missing expected names, unexpected count (names withheld); exits successfully even if candidate manifest has drift +- `test_nextcloud_connection_id_browse_as_designated_user` — end-to-end browse via `GET /api/cloud/connections/{id}/items` in isolated test DB; asserts foreign-user 403/404 and admin 403/404 +- `test_nextcloud_expected_root_manifest` — exact set and kind equality against owner-confirmed fixture (requires `-k expected_root_manifest` or running all live tests) + +**What they do NOT do:** +- Never download file bytes +- Never issue PUT/POST/PATCH/DELETE/MKCOL/MOVE/COPY +- Never print credential values, full URLs, or unexpected provider names +- Never change MinIO objects, quota, or provider content +- Missing credentials produce a descriptive skip, not a failure + +**Fixture reconciliation procedure:** +If the expected root manifest (`backend/tests/fixtures/cloud/nextcloud_expected_root.json`) drifts: +1. Run `pytest -m live_nextcloud -k root_diagnostic` and read the sanitized output (missing expected names only) +2. Compare the missing names with the Nextcloud web UI (do not print the unexpected names) +3. Update the fixture with the owner-confirmed correct names and set `owner_confirmed: true` +4. Never record unexpected live names in any committed artifact + +### Phase 12 / 12.1 Warning Semantics + +A `warning` freshness state in the browse response means the last reconciliation was incomplete or failed. **The UI shows a warning indicator; cached rows remain visible.** This is not an error — it is honest state: + +| State | Meaning | UI behavior | +|-------|---------|-------------| +| `fresh` | Last `complete=True` listing succeeded | Green freshness indicator | +| `refreshing` | Background refresh in progress | Spinner | +| `warning` | Last listing was incomplete or failed | Warning banner; cached rows visible | +| `stale` | Transport error during load | Error message; no freshness indicator | + +`complete=False` from any provider ALWAYS produces `warning`, never `fresh`. `last_refreshed_at` is not advanced for incomplete listings. + ### Phase 12 Deferred Items | Item | Current state | Future path | diff --git a/SECURITY.md b/SECURITY.md index 9db731a..a0e8424 100644 --- a/SECURITY.md +++ b/SECURITY.md @@ -459,3 +459,37 @@ npm audit --audit-level=high: 0 vulnerabilities npm run build: built in ~500ms, 0 errors ``` No bandit-equivalent issues: frontend-only plan; TypeScript/ESLint static analysis implicit via Vite build. + +--- + +## Phase 12.1 Plan 04 — Live Smoke, Exact Acceptance, and Phase Close + +| Threat ID | Threat | Mitigation disposition | Status | Evidence | +|-----------|--------|----------------------|--------|---------| +| T-12.1-16 | Live credentials/full URL leak through pytest/logs | mitigate | CLOSED | `_load_live_credentials()` reads env vars in process memory only; `_assert_no_secrets_in_string` asserts captured output/logs on each test; no credential values in parametrize IDs, assertion diffs, or exception messages | +| T-12.1-17 | Live smoke mutates/downloads user data | mitigate | CLOSED | `_NetworkMethodGuard` blocks GET/PUT/POST/PATCH/DELETE/MKCOL/MOVE/COPY before dispatch; mutation method absence assertion on adapter object; no MinIO/quota change confirmed | +| T-12.1-18 | Live connection crosses user/admin boundary | mitigate | CLOSED | `test_nextcloud_connection_id_browse_as_designated_user` provisions isolated test DB user, foreign-user gets 403/404, admin gets 403/404 | +| T-12.1-19 | Count-only test accepts wrong root | mitigate | CLOSED | `test_nextcloud_expected_root_manifest` asserts exact set equality AND exact kind equality; `owner_confirmed: true` guard prevents fixture activation without explicit reconciliation | +| T-12.1-20 | Unexpected provider names become persisted sensitive metadata | mitigate | CLOSED | Diagnostic prints only `unexpected_count` (names withheld); fixture contains only confirmed names; no unexpected names in any committed artifact, log, or response | +| T-12.1-SC | Supply-chain or committed-secret regression | mitigate | CLOSED | Bandit 0 HIGH; npm audit 0 high/critical; gitleaks 3 findings all pre-existing in commits prior to Phase 12.1 (May 2026), none from Phase 12.1 files; no `.env` in git index | + +### Phase 12.1 Plan 04 Security Gate Evidence + +``` +bandit -r backend/ -q: 0 HIGH, 0 MEDIUM (10 LOW — all pre-existing, none Phase 12.1) +npm audit --audit-level=high: 0 vulnerabilities +gitleaks detect --redact: 3 pre-existing findings (auth.test.js 2026-05-31, + test_cloud_utils.py 2026-05-28, test_auth_deps.py 2026-05-22); none in Phase 12.1 files +git ls-files '.env' '.env.*': only .env.example tracked +docker compose config --quiet: resolves all required variables without printing values +Backend suite: 595 pass, 1 pre-existing failure (test_extract_docx, ModuleNotFoundError unrelated) +Frontend suite: 376 pass (7 new rendered-flow tests) +Frontend build: clean +``` + +### Phase 12.1 Exclusions + +Phase 12.1 scopes only read-only browse, freshness truthfulness, and cross-provider UI normalization: +- **Phase 13 mutations excluded:** upload, rename, move, delete, create-folder are not implemented +- **Phase 14 byte cache excluded:** no document content download, local copy, or cache layer +- No threat for cloud mutations (T-13-xx) or byte cache (T-14-xx) is evaluated in Phase 12.1 diff --git a/backend/main.py b/backend/main.py index da10412..770f6f2 100644 --- a/backend/main.py +++ b/backend/main.py @@ -244,7 +244,7 @@ async def lifespan(app: FastAPI): # ── Application factory ─────────────────────────────────────────────────────── -app = FastAPI(title="Document Scanner API", version="0.2.5", lifespan=lifespan) +app = FastAPI(title="Document Scanner API", version="0.2.6", lifespan=lifespan) # Rate limiter state (slowapi) app.state.limiter = auth_limiter diff --git a/frontend/package.json b/frontend/package.json index 6d5391b..e75c4e6 100644 --- a/frontend/package.json +++ b/frontend/package.json @@ -1,6 +1,6 @@ { "name": "document-scanner-frontend", - "version": "0.2.5", + "version": "0.2.6", "type": "module", "scripts": { "dev": "vite",