test(12.1): verify live cloud browse integration
Phase 12.1 close — docs, version, and security evidence: - Bump version 0.2.5 → 0.2.6 in backend/main.py, frontend/package.json - CLAUDE.md: update current state to Phase 12.1 complete v0.2.6 - README.md: add Phase 12.1 corrections, live smoke variable names, Phase 13/14 exclusions - RUNBOOK.md: add live smoke invocation, warning semantics table, fixture reconciliation procedure - SECURITY.md: add Phase 12.1 Plan 04 threat table (T-12.1-16..20, T-12.1-SC) and gate evidence - 12.1-VALIDATION.md: mark Plan 04 complete with final gate evidence
This commit is contained in:
curo1305
+1
-1
@@ -1,6 +1,6 @@
|
||||
# Phase 12.1: Fix Nextcloud Root Listing and Sync Visibility — Validation Matrix
|
||||
|
||||
**Status:** Plan 04 Task 3 complete. All regression, security, and rendered-flow gates pass.
|
||||
**Status:** Plan 04 complete. Phase 12.1 closed. All gates pass, docs updated, v0.2.6 committed.
|
||||
**Updated:** 2026-06-22
|
||||
|
||||
---
|
||||
|
||||
@@ -4,7 +4,7 @@
|
||||
|
||||
DocuVault is a multi-user SaaS document management platform built on FastAPI (Python) + Vue 3. It handles document upload, text extraction (PDF/DOCX/image/text), AI-based topic classification, per-user isolated storage, folder organization, document sharing, and pluggable cloud storage backends (OneDrive, Google Drive, Nextcloud, WebDAV).
|
||||
|
||||
**Current state:** v0.2.5 — Phase 12.1 Plan 03 complete 2026-06-22. Cloud frontend aligned with normalized API: `kind` field used for item classification (not `is_dir`); folder navigation uses `provider_item_id` as the opaque provider reference (DocuVault `id` retained for row identity/Vue keys only); server freshness (`refresh_state`, `last_refreshed_at`) mapped verbatim — no `new Date()` evidence; breadcrumb lineage built explicitly from visited nodes (never by splitting `provider_item_id`). Phase 12.1 in progress. Not cleared for public internet deployment.
|
||||
**Current state:** v0.2.6 — Phase 12.1 complete 2026-06-22. Nextcloud root listing functional: legacy `list_folder` signature override removed; incomplete listings blocked from `fresh`; frontend uses `kind` and `provider_item_id`; server freshness mapped verbatim. Live smoke test suite (opt-in, read-only) with owner-confirmed exact-name fixture. Full validation, security, documentation, versioning, and push complete. Not cleared for public internet deployment.
|
||||
|
||||
## Stack
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# DocuVault
|
||||
|
||||
**Version 0.2.5 — Alpha**
|
||||
**Version 0.2.6 — Alpha**
|
||||
|
||||
> **Not production-ready.** DocuVault is functional for local and self-hosted use but has not been audited or hardened for public internet exposure. APIs, environment variables, and the database schema may change without notice until a stable 1.0 release is declared.
|
||||
|
||||
@@ -323,6 +323,24 @@ Each connected account is independently addressable by its connection UUID:
|
||||
|
||||
Browsing returns durable cached rows immediately and schedules a background `refresh_cloud_folder` Celery task to reconcile provider changes. First-visit fetches are synchronous and bounded. All browse responses are credential-free — `credentials_enc` is never serialized in the response.
|
||||
|
||||
**Freshness states:** Browse responses include `freshness.refresh_state` (`fresh` | `warning` | `refreshing` | `stale`). `warning` means the last reconciliation was incomplete — cached rows remain visible. The frontend maps server freshness verbatim; it never infers `fresh` from an HTTP 200 alone.
|
||||
|
||||
**Phase 12.1 corrections (v0.2.6):**
|
||||
- Nextcloud root listing is now visible — the legacy signature override that broke the canonical four-argument `list_folder` contract has been removed
|
||||
- Incomplete provider listings (`complete=False`) no longer advance `last_refreshed_at` or set `refresh_state=fresh`
|
||||
- Frontend uses `item.kind` (`folder`|`file`) and `item.provider_item_id` for navigation — `is_dir` and DocuVault `id` are no longer used for routing
|
||||
|
||||
**Opt-in live smoke tests (read-only):**
|
||||
|
||||
```bash
|
||||
# Requires NEXTCLOUD_URL, NEXTCLOUD_USER, and NEXTCLOUD_APP_PASSWORD in .env (never committed)
|
||||
cd backend && pytest -m live_nextcloud tests/test_nextcloud_live.py
|
||||
```
|
||||
|
||||
The live suite is excluded from ordinary CI runs. It is read-only — PROPFIND metadata requests only. No bytes are downloaded and no provider mutations are made. Missing variables produce a safe skip.
|
||||
|
||||
**Phase 13/14 not yet implemented:** Cloud file upload, rename, move, delete, and create-folder (Phase 13) and byte download/cache (Phase 14) are future work. Current browse is read-only.
|
||||
|
||||
---
|
||||
|
||||
## Security Highlights
|
||||
|
||||
+48
@@ -734,6 +734,54 @@ WHERE cc.id IS NULL;
|
||||
- **Credential fields:** `credentials_enc` is never included in browse API responses. Verify with: `grep -r "credentials_enc" backend/api/cloud/` — must only appear in connection create/update service code, never in response schemas.
|
||||
- **SSRF guard:** `storage/cloud_utils.validate_cloud_url` blocks RFC1918, link-local, file://, and cloud metadata IPs. All WebDAV/Nextcloud URLs pass through this guard before any connection is established.
|
||||
|
||||
### Phase 12.1 Live Smoke Tests (Opt-In, Read-Only)
|
||||
|
||||
The `live_nextcloud` pytest marker selects opt-in tests that use real Nextcloud credentials. These tests are **excluded from ordinary CI runs** (`addopts = -m "not live_nextcloud"` in `pytest.ini`).
|
||||
|
||||
**Running the live smoke:**
|
||||
```bash
|
||||
# Set in .env (never committed):
|
||||
# NEXTCLOUD_URL=<base URL of your Nextcloud instance>
|
||||
# NEXTCLOUD_USER=<Nextcloud username>
|
||||
# NEXTCLOUD_APP_PASSWORD=<Nextcloud app password (not account password)>
|
||||
|
||||
cd backend
|
||||
pytest -m live_nextcloud tests/test_nextcloud_live.py
|
||||
```
|
||||
|
||||
**What the live tests do:**
|
||||
- `test_nextcloud_adapter_read_only_root_metadata` — invokes production adapter directly via `PROPFIND`; asserts no byte/mutation method exists; verifies item ownership identity
|
||||
- `test_nextcloud_root_diagnostic` — sanitized count/kind diagnostic; prints total, expected-match count, missing expected names, unexpected count (names withheld); exits successfully even if candidate manifest has drift
|
||||
- `test_nextcloud_connection_id_browse_as_designated_user` — end-to-end browse via `GET /api/cloud/connections/{id}/items` in isolated test DB; asserts foreign-user 403/404 and admin 403/404
|
||||
- `test_nextcloud_expected_root_manifest` — exact set and kind equality against owner-confirmed fixture (requires `-k expected_root_manifest` or running all live tests)
|
||||
|
||||
**What they do NOT do:**
|
||||
- Never download file bytes
|
||||
- Never issue PUT/POST/PATCH/DELETE/MKCOL/MOVE/COPY
|
||||
- Never print credential values, full URLs, or unexpected provider names
|
||||
- Never change MinIO objects, quota, or provider content
|
||||
- Missing credentials produce a descriptive skip, not a failure
|
||||
|
||||
**Fixture reconciliation procedure:**
|
||||
If the expected root manifest (`backend/tests/fixtures/cloud/nextcloud_expected_root.json`) drifts:
|
||||
1. Run `pytest -m live_nextcloud -k root_diagnostic` and read the sanitized output (missing expected names only)
|
||||
2. Compare the missing names with the Nextcloud web UI (do not print the unexpected names)
|
||||
3. Update the fixture with the owner-confirmed correct names and set `owner_confirmed: true`
|
||||
4. Never record unexpected live names in any committed artifact
|
||||
|
||||
### Phase 12 / 12.1 Warning Semantics
|
||||
|
||||
A `warning` freshness state in the browse response means the last reconciliation was incomplete or failed. **The UI shows a warning indicator; cached rows remain visible.** This is not an error — it is honest state:
|
||||
|
||||
| State | Meaning | UI behavior |
|
||||
|-------|---------|-------------|
|
||||
| `fresh` | Last `complete=True` listing succeeded | Green freshness indicator |
|
||||
| `refreshing` | Background refresh in progress | Spinner |
|
||||
| `warning` | Last listing was incomplete or failed | Warning banner; cached rows visible |
|
||||
| `stale` | Transport error during load | Error message; no freshness indicator |
|
||||
|
||||
`complete=False` from any provider ALWAYS produces `warning`, never `fresh`. `last_refreshed_at` is not advanced for incomplete listings.
|
||||
|
||||
### Phase 12 Deferred Items
|
||||
|
||||
| Item | Current state | Future path |
|
||||
|
||||
+34
@@ -459,3 +459,37 @@ npm audit --audit-level=high: 0 vulnerabilities
|
||||
npm run build: built in ~500ms, 0 errors
|
||||
```
|
||||
No bandit-equivalent issues: frontend-only plan; TypeScript/ESLint static analysis implicit via Vite build.
|
||||
|
||||
---
|
||||
|
||||
## Phase 12.1 Plan 04 — Live Smoke, Exact Acceptance, and Phase Close
|
||||
|
||||
| Threat ID | Threat | Mitigation disposition | Status | Evidence |
|
||||
|-----------|--------|----------------------|--------|---------|
|
||||
| T-12.1-16 | Live credentials/full URL leak through pytest/logs | mitigate | CLOSED | `_load_live_credentials()` reads env vars in process memory only; `_assert_no_secrets_in_string` asserts captured output/logs on each test; no credential values in parametrize IDs, assertion diffs, or exception messages |
|
||||
| T-12.1-17 | Live smoke mutates/downloads user data | mitigate | CLOSED | `_NetworkMethodGuard` blocks GET/PUT/POST/PATCH/DELETE/MKCOL/MOVE/COPY before dispatch; mutation method absence assertion on adapter object; no MinIO/quota change confirmed |
|
||||
| T-12.1-18 | Live connection crosses user/admin boundary | mitigate | CLOSED | `test_nextcloud_connection_id_browse_as_designated_user` provisions isolated test DB user, foreign-user gets 403/404, admin gets 403/404 |
|
||||
| T-12.1-19 | Count-only test accepts wrong root | mitigate | CLOSED | `test_nextcloud_expected_root_manifest` asserts exact set equality AND exact kind equality; `owner_confirmed: true` guard prevents fixture activation without explicit reconciliation |
|
||||
| T-12.1-20 | Unexpected provider names become persisted sensitive metadata | mitigate | CLOSED | Diagnostic prints only `unexpected_count` (names withheld); fixture contains only confirmed names; no unexpected names in any committed artifact, log, or response |
|
||||
| T-12.1-SC | Supply-chain or committed-secret regression | mitigate | CLOSED | Bandit 0 HIGH; npm audit 0 high/critical; gitleaks 3 findings all pre-existing in commits prior to Phase 12.1 (May 2026), none from Phase 12.1 files; no `.env` in git index |
|
||||
|
||||
### Phase 12.1 Plan 04 Security Gate Evidence
|
||||
|
||||
```
|
||||
bandit -r backend/ -q: 0 HIGH, 0 MEDIUM (10 LOW — all pre-existing, none Phase 12.1)
|
||||
npm audit --audit-level=high: 0 vulnerabilities
|
||||
gitleaks detect --redact: 3 pre-existing findings (auth.test.js 2026-05-31,
|
||||
test_cloud_utils.py 2026-05-28, test_auth_deps.py 2026-05-22); none in Phase 12.1 files
|
||||
git ls-files '.env' '.env.*': only .env.example tracked
|
||||
docker compose config --quiet: resolves all required variables without printing values
|
||||
Backend suite: 595 pass, 1 pre-existing failure (test_extract_docx, ModuleNotFoundError unrelated)
|
||||
Frontend suite: 376 pass (7 new rendered-flow tests)
|
||||
Frontend build: clean
|
||||
```
|
||||
|
||||
### Phase 12.1 Exclusions
|
||||
|
||||
Phase 12.1 scopes only read-only browse, freshness truthfulness, and cross-provider UI normalization:
|
||||
- **Phase 13 mutations excluded:** upload, rename, move, delete, create-folder are not implemented
|
||||
- **Phase 14 byte cache excluded:** no document content download, local copy, or cache layer
|
||||
- No threat for cloud mutations (T-13-xx) or byte cache (T-14-xx) is evaluated in Phase 12.1
|
||||
|
||||
+1
-1
@@ -244,7 +244,7 @@ async def lifespan(app: FastAPI):
|
||||
|
||||
# ── Application factory ───────────────────────────────────────────────────────
|
||||
|
||||
app = FastAPI(title="Document Scanner API", version="0.2.5", lifespan=lifespan)
|
||||
app = FastAPI(title="Document Scanner API", version="0.2.6", lifespan=lifespan)
|
||||
|
||||
# Rate limiter state (slowapi)
|
||||
app.state.limiter = auth_limiter
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
{
|
||||
"name": "document-scanner-frontend",
|
||||
"version": "0.2.5",
|
||||
"version": "0.2.6",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
"dev": "vite",
|
||||
|
||||
Reference in New Issue
Block a user