Compare commits

...
5 Commits
Author SHA1 Message Date
curo1305 a6e8a597e7 docs(12.1-01): complete Plan 01 summary and state updates 2026-06-22 08:20:06 +02:00
curo1305 62128730e8 fix(12.1-01): restore asyncio import, bump version to 0.2.3, update docs
- Restore asyncio import removed when legacy list_folder was deleted (health_check
  still uses asyncio.to_thread); fixes test_nextcloud_connect_persists regression
- Bump backend/main.py and frontend/package.json version to 0.2.3
- Update CLAUDE.md: note Phase 12.1 Plan 01 complete, add normalize_nextcloud_url
  to shared module map, add NextcloudBackend no-override rule
- Update README.md: version 0.2.3, note canonical Nextcloud URL normalization and
  OneDrive nextLink origin guard
- Full backend suite: 585 passed, 1 pre-existing failure (missing python-docx module)
2026-06-22 08:18:23 +02:00
curo1305 805fe44bfb feat(12.1-01): add cross-origin nextLink guard and Drive/OneDrive contract tests
- Validate @odata.nextLink hostname against graph.microsoft.com before following
  (T-12.1-03); cross-origin nextLink returns complete=False with prior items retained
- Add TestOneDriveCrossOriginNextLink: cross-origin rejection, same-origin follows,
  page failure retains prior items
- Add TestGoogleDriveAuthFailureControl: 401 returns complete=False, page1+page2
  error retains page1 items
- All 163 four-provider contract tests pass
2026-06-22 08:12:39 +02:00
curo1305 2b46f74329 feat(12.1-01): repair Nextcloud adapter contract and add URL normalization
- Remove incompatible NextcloudBackend.list_folder(folder_path="") override
  so Nextcloud inherits canonical WebDAVBackend.list_folder(connection_id,
  user_id, parent_ref=None, page_token=None) -> CloudListing
- Add normalize_nextcloud_url() to cloud_utils.py: idempotent canonical DAV
  root derivation, username percent-encoding, https-only, rejects userinfo/
  query/fragment, validates via SSRF guard before return
- Use normalize_nextcloud_url in cloud_backend_factory for nextcloud provider
- Add TestNextcloudBackendNoListFolderOverride and TestNextcloudUrlNormalization
  to test_webdav_backend.py
- All 9 Nextcloud contract failures now pass; full focused suite: 158 passed
2026-06-22 08:11:06 +02:00
curo1305 eb68facd6c test(12.1-01): add failing four-provider contract suite and fixtures
- Create test_cloud_provider_contract.py with parametrized suite for
  Nextcloud, WebDAV, Google Drive, OneDrive
- Assert canonical (connection_id, user_id, parent_ref=None, page_token=None)
  signature, CloudListing return type, trusted caller identity propagation,
  metadata normalization, pagination completeness, and forbidden-operation spies
- Add synthetic credential-free fixtures: nextcloud_root.xml, webdav_root.xml,
  google_drive_pages.json, onedrive_pages.json
- 9 tests fail for nextcloud (expected RED state — incompatible list_folder override)
- All webdav, google_drive, onedrive contract tests pass
2026-06-22 08:09:00 +02:00
19 changed files with 1911 additions and 124 deletions
+12 -12
View File
@@ -10,18 +10,18 @@
- [ ] **CONN-01**: User can connect, reconnect, test, and disconnect each supported cloud provider. - [ ] **CONN-01**: User can connect, reconnect, test, and disconnect each supported cloud provider.
- [ ] **CONN-02**: User can see connection health and actionable errors for expired, revoked, or invalid credentials. - [ ] **CONN-02**: User can see connection health and actionable errors for expired, revoked, or invalid credentials.
- [ ] **CONN-03**: Reconnecting or refreshing credentials invalidates stale provider caches without exposing credentials. - [ ] **CONN-03**: Reconnecting or refreshing credentials invalidates stale provider caches without exposing credentials.
- [ ] **CONN-04**: User interface reflects the file operations supported by each connected provider. - [x] **CONN-04**: User interface reflects the file operations supported by each connected provider.
### Cloud File Management ### Cloud File Management
- [ ] **CLOUD-01**: User can browse connected cloud files and folders through the shared `StorageBrowser`. - [x] **CLOUD-01**: User can browse connected cloud files and folders through the shared `StorageBrowser`.
- [ ] **CLOUD-02**: User can open and preview supported cloud documents through DocuVault authorization. - [ ] **CLOUD-02**: User can open and preview supported cloud documents through DocuVault authorization.
- [ ] **CLOUD-03**: User can upload files into the currently viewed cloud folder. - [ ] **CLOUD-03**: User can upload files into the currently viewed cloud folder.
- [ ] **CLOUD-04**: User can create folders in connected cloud storage where the provider supports it. - [ ] **CLOUD-04**: User can create folders in connected cloud storage where the provider supports it.
- [ ] **CLOUD-05**: User can rename cloud files and folders where the provider supports it. - [ ] **CLOUD-05**: User can rename cloud files and folders where the provider supports it.
- [ ] **CLOUD-06**: User can move files and folders within the same cloud connection where the provider supports it. - [ ] **CLOUD-06**: User can move files and folders within the same cloud connection where the provider supports it.
- [ ] **CLOUD-07**: User can delete cloud files and folders after explicit confirmation. - [ ] **CLOUD-07**: User can delete cloud files and folders after explicit confirmation.
- [ ] **CLOUD-08**: User sees unsupported cloud actions disabled with a provider-specific explanation. - [x] **CLOUD-08**: User sees unsupported cloud actions disabled with a provider-specific explanation.
- [ ] **CLOUD-09**: Successful cloud mutations update navigation promptly and produce metadata-only audit events. - [ ] **CLOUD-09**: Successful cloud mutations update navigation promptly and produce metadata-only audit events.
### Cloud Analysis ### Cloud Analysis
@@ -46,15 +46,15 @@
### Cache and Storage Minimization ### Cache and Storage Minimization
- [ ] **CACHE-01**: Provider-owned file bytes remain the source of truth unless the user explicitly imports a file into local DocuVault storage. - [x] **CACHE-01**: Provider-owned file bytes remain the source of truth unless the user explicitly imports a file into local DocuVault storage.
- [ ] **CACHE-02**: DocuVault persists cloud metadata, extracted text, topics, and semantic search data independently of cached file bytes. - [x] **CACHE-02**: DocuVault persists cloud metadata, extracted text, topics, and semantic search data independently of cached file bytes.
- [ ] **CACHE-03**: DocuVault caches cloud file bytes only when required for opening, preview, or analysis. - [ ] **CACHE-03**: DocuVault caches cloud file bytes only when required for opening, preview, or analysis.
- [ ] **CACHE-04**: Configurable cache limits and safe eviction minimize local storage use without interrupting active work. - [ ] **CACHE-04**: Configurable cache limits and safe eviction minimize local storage use without interrupting active work.
- [ ] **CACHE-05**: Cached cloud content is isolated by user and connection and is never served without an ownership check. - [ ] **CACHE-05**: Cached cloud content is isolated by user and connection and is never served without an ownership check.
### Change Tracking ### Change Tracking
- [ ] **SYNC-01**: DocuVault records provider item identifiers, parent/location, version or etag, size, and modification time for indexed cloud items. - [x] **SYNC-01**: DocuVault records provider item identifiers, parent/location, version or etag, size, and modification time for indexed cloud items.
- [ ] **SYNC-02**: DocuVault marks analyzed cloud items stale when their provider version changes. - [ ] **SYNC-02**: DocuVault marks analyzed cloud items stale when their provider version changes.
- [ ] **SYNC-03**: Files deleted outside DocuVault are removed from cloud navigation and excluded from search results after refresh. - [ ] **SYNC-03**: Files deleted outside DocuVault are removed from cloud navigation and excluded from search results after refresh.
- [ ] **SYNC-04**: Provider refresh jobs are idempotent and retry transient failures with bounded exponential backoff. - [ ] **SYNC-04**: Provider refresh jobs are idempotent and retry transient failures with bounded exponential backoff.
@@ -91,15 +91,15 @@
| CONN-01 | Phase 13 | Pending | | CONN-01 | Phase 13 | Pending |
| CONN-02 | Phase 13 | Pending | | CONN-02 | Phase 13 | Pending |
| CONN-03 | Phase 13 | Pending | | CONN-03 | Phase 13 | Pending |
| CONN-04 | Phase 12 | Pending | | CONN-04 | Phase 12 | Complete |
| CLOUD-01 | Phase 12 | Pending | | CLOUD-01 | Phase 12 | Complete |
| CLOUD-02 | Phase 13 | Pending | | CLOUD-02 | Phase 13 | Pending |
| CLOUD-03 | Phase 13 | Pending | | CLOUD-03 | Phase 13 | Pending |
| CLOUD-04 | Phase 13 | Pending | | CLOUD-04 | Phase 13 | Pending |
| CLOUD-05 | Phase 13 | Pending | | CLOUD-05 | Phase 13 | Pending |
| CLOUD-06 | Phase 13 | Pending | | CLOUD-06 | Phase 13 | Pending |
| CLOUD-07 | Phase 13 | Pending | | CLOUD-07 | Phase 13 | Pending |
| CLOUD-08 | Phase 12 | Pending | | CLOUD-08 | Phase 12 | Complete |
| CLOUD-09 | Phase 13 | Pending | | CLOUD-09 | Phase 13 | Pending |
| ANALYZE-01 | Phase 14 | Pending | | ANALYZE-01 | Phase 14 | Pending |
| ANALYZE-02 | Phase 14 | Pending | | ANALYZE-02 | Phase 14 | Pending |
@@ -115,12 +115,12 @@
| SEARCH-05 | Phase 15 | Pending | | SEARCH-05 | Phase 15 | Pending |
| SEARCH-06 | Phase 15 | Pending | | SEARCH-06 | Phase 15 | Pending |
| SEARCH-07 | Phase 15 | Pending | | SEARCH-07 | Phase 15 | Pending |
| CACHE-01 | Phase 12 | Pending | | CACHE-01 | Phase 12 | Complete |
| CACHE-02 | Phase 12 | Pending | | CACHE-02 | Phase 12 | Complete |
| CACHE-03 | Phase 14 | Pending | | CACHE-03 | Phase 14 | Pending |
| CACHE-04 | Phase 14 | Pending | | CACHE-04 | Phase 14 | Pending |
| CACHE-05 | Phase 14 | Pending | | CACHE-05 | Phase 14 | Pending |
| SYNC-01 | Phase 12 | Pending | | SYNC-01 | Phase 12 | Complete |
| SYNC-02 | Phase 16 | Pending | | SYNC-02 | Phase 16 | Pending |
| SYNC-03 | Phase 16 | Pending | | SYNC-03 | Phase 16 | Pending |
| SYNC-04 | Phase 16 | Pending | | SYNC-04 | Phase 16 | Pending |
+28
View File
@@ -46,6 +46,34 @@ Before any phase is marked complete:
4. Provider-owned bytes remain outside DocuVault while metadata, extracted text placeholders, topic links, and semantic-index state can be persisted independently. 4. Provider-owned bytes remain outside DocuVault while metadata, extracted text placeholders, topic links, and semantic-index state can be persisted independently.
5. Ownership and admin-negative tests prove cloud connection and cloud item metadata cannot cross user boundaries. 5. Ownership and admin-negative tests prove cloud connection and cloud item metadata cannot cross user boundaries.
### Phase 12.1: Fix Nextcloud root listing and sync visibility (INSERTED)
**Goal:** Restore visible, truthful, metadata-only cloud browsing by repairing Nextcloud's adapter contract, applying one completeness/freshness contract across all providers, and aligning the shared frontend browser with normalized provider identities.
**Requirements:** CONN-04, CLOUD-01, CACHE-01, SYNC-01
**Depends on:** Phase 12
**Plans:** 1/4 plans executed
**Execution waves:** Wave 1: 12.1-01; Wave 2: 12.1-02 after 01; Wave 3: 12.1-03 after 02; Wave 4: 12.1-04 after 01-03.
Plans:
- [x] 12.1-01-PLAN.md
- [ ] 12.1-02-PLAN.md
- [ ] 12.1-03-PLAN.md
- [ ] 12.1-04-PLAN.md
- [x] 12.1-01 (Wave 1) — Restore the shared four-provider adapter contract, secure Nextcloud URL/redirect handling, and metadata-only listing behavior.
- [ ] 12.1-02 (Wave 2; after 12.1-01) — Centralize complete/incomplete reconciliation so incomplete provider results cannot become fresh or delete cached metadata.
- [ ] 12.1-03 (Wave 3; after 12.1-02) — Align the shared browser and trees with `kind`, opaque `provider_item_id` routing, lineage-based breadcrumbs, and server freshness.
- [ ] 12.1-04 (Wave 4; after 12.1-01 through 12.1-03) — Run sanitized two-mode live Nextcloud acceptance, rendered/security/secret gates, and phase closeout.
**Success Criteria:**
1. Nextcloud and generic WebDAV expose one canonical `CloudResourceAdapter.list_folder` implementation, and all four providers pass the same runtime identity/completeness/no-byte/no-mutation contract with provider-specific pagination fixtures.
2. Every outbound Nextcloud/WebDAV target, including redirects, is rejected unless each hop is HTTPS, same-origin, and passes URL plus resolved-address SSRF validation; automatic redirects are disabled.
3. `complete=False` never marks a folder fresh, advances `last_refreshed_at`, or authorizes unseen-item deletion in synchronous or worker paths; cached rows remain usable with a controlled warning.
4. The frontend classifies by `kind`, navigates through named routes/query serialization with opaque `provider_item_id` values, builds breadcrumbs from explicit navigation lineage rather than splitting provider IDs, and renders backend freshness verbatim.
5. Live Nextcloud validation first runs a nonblocking sanitized diagnostic; exact names/kinds become a blocking gate only after the owner-confirmed manifest is stored in a tracked, non-secret fixture. Unexpected live names are never printed or persisted.
6. Automated backend/frontend/rendered-flow tests, dedicated per-plan security reviews, dependency audits, an executable secret scan, documentation/version updates, and one atomic commit/push per plan pass without exposing credentials or provider bytes.
### Phase 13: Virtual-Local Cloud Operations ### Phase 13: Virtual-Local Cloud Operations
**Goal:** Users can maintain healthy cloud connections and perform the main file-management workflows in connected storage with the same shared browser interactions used for local files. **Goal:** Users can maintain healthy cloud connections and perform the main file-management workflows in connected storage with the same shared browser interactions used for local files.
+30 -20
View File
@@ -2,36 +2,39 @@
gsd_state_version: 1.0 gsd_state_version: 1.0
milestone: v0.3 milestone: v0.3
milestone_name: Reimagining Cloud Storage integration milestone_name: Reimagining Cloud Storage integration
status: ready_to_plan current_phase: 12.1
last_updated: 2026-06-19T04:04:58.418Z current_phase_name: fix-nextcloud-root-listing-and-sync-visibility
last_activity: 2026-06-18 -- Phase 12 execution started status: executing
last_updated: "2026-06-22T06:19:49.338Z"
last_activity: 2026-06-22
last_activity_desc: Phase 12.1 execution started
progress: progress:
total_phases: 5 total_phases: 6
completed_phases: 1 completed_phases: 1
total_plans: 4 total_plans: 10
completed_plans: 69 completed_plans: 7
percent: 20 percent: 17
stopped_at: Phase 12 complete (4/4) — ready to discuss Phase 13
--- ---
# Project State # Project State
**Project:** DocuVault **Project:** DocuVault
**Status:** Ready to plan **Status:** Ready to execute
**Last Updated:** 2026-06-17 **Last Updated:** 2026-06-22
## Current Position ## Current Position
Phase: 13 Phase: 12.1 (fix-nextcloud-root-listing-and-sync-visibility) — EXECUTING
Plan: Not started Plan: 2 of 4
Status: Executing Phase 12 Status: Ready to execute
Last activity: 2026-06-19 Last activity: 2026-06-22 — Phase 12.1 execution started
## Phase Status ## Phase Status
| Phase | Requirements | Status | | Phase | Requirements | Status |
|-------|-------------|--------| |-------|-------------|--------|
| 12. Cloud Resource Foundation | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 | **Not started** | | 12. Cloud Resource Foundation | CONN-04, CLOUD-01, CLOUD-08, CACHE-01, CACHE-02, SYNC-01 | **Complete** |
| 12.1 Fix Nextcloud Root Listing and Sync Visibility | CONN-04, CLOUD-01, CACHE-01, SYNC-01 | **Planned** |
| 13. Virtual-Local Cloud Operations | CONN-01..03, CLOUD-02..07, CLOUD-09 | **Not started** | | 13. Virtual-Local Cloud Operations | CONN-01..03, CLOUD-02..07, CLOUD-09 | **Not started** |
| 14. Selective Analysis and Byte Cache | ANALYZE-01..07, CACHE-03..05 | **Not started** | | 14. Selective Analysis and Byte Cache | ANALYZE-01..07, CACHE-03..05 | **Not started** |
| 15. Unified Smart Search | SEARCH-01..07 | **Not started** | | 15. Unified Smart Search | SEARCH-01..07 | **Not started** |
@@ -41,10 +44,11 @@ Last activity: 2026-06-19
| Metric | Value | | Metric | Value |
|---|---| |---|---|
| Phases complete | 0 / 5 | | Phases complete | 1 / 6 |
| Requirements satisfied | 0 / 36 | | Requirements satisfied | 6 / 36 |
| Plans complete | 0 / 0 | | Plans complete | 6 / 10 |
| Tests at milestone start | 277 | | Tests at milestone start | 277 |
| Phase 12.1 P01 | 823 | 4 tasks | 14 files |
## Accumulated Context ## Accumulated Context
@@ -70,6 +74,7 @@ Last activity: 2026-06-19
- v0.2 completed: UI overhaul, admin panel rearchitecture, responsive layout, codebase quality (2026-06-17) - v0.2 completed: UI overhaul, admin panel rearchitecture, responsive layout, codebase quality (2026-06-17)
- v0.2 archived to `.planning/milestones/v0.2-ROADMAP.md` - v0.2 archived to `.planning/milestones/v0.2-ROADMAP.md`
- v0.3 proposed: virtual-local cloud operations, selective cloud analysis, bounded byte caching, unified smart search, and provider change tracking - v0.3 proposed: virtual-local cloud operations, selective cloud analysis, bounded byte caching, unified smart search, and provider change tracking
- Phase 12.1 inserted after Phase 12: Fix Nextcloud root listing and sync visibility (URGENT)
### Open Questions ### Open Questions
@@ -85,7 +90,12 @@ _Updated at each phase transition._
| Field | Value | | Field | Value |
|---|---| |---|---|
| Last session | 2026-06-17 — v0.3 roadmap proposed | | Last session | 2026-06-22T06:19:43.071Z |
| Next action | Approve roadmap, then discuss or plan Phase 12 | | Next action | Execute Phase 12.1 Plan 01 |
| Pending decisions | None | | Pending decisions | None |
| Resume file | None | | Resume file | None |
## Decisions
- [Phase ?]: Remove NextcloudBackend.list_folder override — inherit canonical WebDAVBackend method
- [Phase ?]: OneDrive nextLink restricted to graph.microsoft.com host
@@ -0,0 +1,156 @@
---
phase: "12.1"
plan: "01"
subsystem: cloud-storage-backends
status: complete
tags: [nextcloud, webdav, google-drive, onedrive, cloud-resource-adapter, contract-tests, security]
completed: "2026-06-22"
duration_seconds: 823
task_count: 4
file_count: 14
requires: []
provides:
- "Canonical four-provider CloudResourceAdapter contract suite"
- "Nextcloud list_folder signature defect eliminated"
- "normalize_nextcloud_url shared helper"
- "OneDrive @odata.nextLink cross-origin guard"
affects: [cloud-browse-api, cloud-backend-factory, provider-tests]
tech_stack:
added: []
patterns:
- "TDD RED/GREEN — contract tests written first, production defect confirmed, then implementation fixed"
- "normalize_nextcloud_url idempotent URL derivation with SSRF passthrough"
- "OneDrive cross-origin nextLink validation before following"
key_files:
created:
- backend/tests/test_cloud_provider_contract.py
- backend/tests/fixtures/cloud/nextcloud_root.xml
- backend/tests/fixtures/cloud/webdav_root.xml
- backend/tests/fixtures/cloud/google_drive_pages.json
- backend/tests/fixtures/cloud/onedrive_pages.json
modified:
- backend/storage/nextcloud_backend.py
- backend/storage/cloud_utils.py
- backend/storage/cloud_backend_factory.py
- backend/storage/onedrive_backend.py
- backend/tests/test_cloud_backends.py
- backend/tests/test_webdav_backend.py
- backend/main.py
- frontend/package.json
- CLAUDE.md
- README.md
decisions:
- "Remove NextcloudBackend.list_folder override so Nextcloud inherits canonical WebDAVBackend method (direct fix for P0)"
- "normalize_nextcloud_url lives in cloud_utils.py — called at factory construction, not in every request"
- "OneDrive nextLink restricted to graph.microsoft.com origin — cross-origin returns complete=False with prior items retained"
- "asyncio import restored to nextcloud_backend.py (health_check uses asyncio.to_thread)"
---
# Phase 12.1 Plan 01: Restore Cloud Adapter Contract — Summary
**One-liner:** Removed incompatible `NextcloudBackend.list_folder(folder_path="")` override, added `normalize_nextcloud_url()`, cross-origin OneDrive nextLink guard, and parametrized four-provider contract suite.
## What Was Built
### Task 1 — TDD RED: Four-provider contract tests and fixtures
- `backend/tests/test_cloud_provider_contract.py` — 48 parametrized tests across Nextcloud, WebDAV, Google Drive, OneDrive covering:
- Canonical signature verification (`connection_id, user_id, parent_ref=None, page_token=None`)
- Return type is always `CloudListing` (never `list[dict]`)
- Trusted caller identity (`connection_id`, `user_id`) is never overrideable by provider response
- `provider_item_id` is set; `kind` is exactly "file" or "folder"
- `parent_ref` is propagated to each resource
- Absent optional metadata normalizes to `None`
- All pages consumed before `complete=True`; page failure → `complete=False`
- Forbidden-operation spies for `get_object`, `put_object`, `delete_object`, etc.
- Synthetic credential-free fixtures: `nextcloud_root.xml`, `webdav_root.xml`, `google_drive_pages.json`, `onedrive_pages.json`
- 9 Nextcloud-specific tests confirmed RED (TypeError on canonical invocation) as expected
### Task 2 — GREEN: Repair Nextcloud/WebDAV listing
- Removed `async def list_folder(self, folder_path: str = "") -> list[dict]` from `NextcloudBackend` — root cause of P0 defect
- `NextcloudBackend` now inherits `WebDAVBackend.list_folder` via standard Python inheritance
- Added `normalize_nextcloud_url(base_url, username)` to `cloud_utils.py`:
- Accepts bare origin, origin with subpath, already-canonical URL
- Username percent-encoded as single path segment
- HTTPS-only; rejects userinfo/query/fragment
- Idempotent — no double-append of WebDAV suffix
- SSRF-validated via existing `validate_cloud_url()` before return
- `cloud_backend_factory.py` calls `normalize_nextcloud_url` before `NextcloudBackend` construction; `ValueError` from DNS failures is caught gracefully
- Added `TestNextcloudBackendNoListFolderOverride` and `TestNextcloudUrlNormalization` to `test_webdav_backend.py`
- Restored `asyncio` import (was inadvertently removed with the legacy `list_folder`; `health_check` still uses `asyncio.to_thread`)
### Task 3 — Drive/OneDrive contract completion
- Added `_GRAPH_HOST` constant to `onedrive_backend.py`; `@odata.nextLink` hostname validated against `graph.microsoft.com` before following; cross-origin → `complete=False` with prior items retained
- Added `TestOneDriveCrossOriginNextLink`: cross-origin rejection, same-origin follows, page failure retains prior items
- Added `TestGoogleDriveAuthFailureControl`: 401 → `complete=False`, page1+page2 error retains page1 items
### Task 4 — Documentation and version bump
- Version bumped to `0.2.3` in `backend/main.py` and `frontend/package.json`
- `CLAUDE.md` updated: current state, shared module map, Nextcloud no-override rule
- `README.md` updated: version, Nextcloud URL normalization note, OneDrive nextLink origin guard
## Deviations from Plan
### Auto-fixed Issues
**1. [Rule 1 - Bug] Missing asyncio import in NextcloudBackend after legacy method removal**
- **Found during:** Task 4 (full test suite run)
- **Issue:** Removing the legacy `list_folder` method also removed the only active use of `asyncio` in the file, but the `health_check` method still calls `asyncio.to_thread`. This caused `test_nextcloud_connect_persists` and `test_same_provider_connections_are_independent` to fail.
- **Fix:** Re-added `import asyncio` to `nextcloud_backend.py`
- **Files modified:** `backend/storage/nextcloud_backend.py`
- **Commit:** 6212873
**2. [Rule 2 - Missing security] OneDrive @odata.nextLink cross-origin validation**
- **Found during:** Task 3 (plan specifies "reject cross-origin/untrusted continuation URLs")
- **Issue:** `onedrive_backend.py` followed `@odata.nextLink` without verifying the target stays on `graph.microsoft.com`; an SSRF-adjacent CSRF vector
- **Fix:** Added `_GRAPH_HOST` constant and `urllib.parse` hostname check before following any pagination URL
- **Files modified:** `backend/storage/onedrive_backend.py`
- **Commit:** 805fe44
## Test Evidence
| Suite | Tests | Status |
|-------|-------|--------|
| `test_cloud_provider_contract.py` | 48 | All pass |
| `test_cloud_backends.py` | 67 | All pass |
| `test_webdav_backend.py` | 29 | All pass (9 warnings for sync tests with asyncio mark — module-level `pytestmark` issue, tests behave correctly) |
| `test_cloud_security.py` | 19 | All pass |
| Full backend suite | 585 | 1 pre-existing failure (`test_extract_docx` — missing `python-docx` module, unrelated) |
## Security Verification
Bandit scanned `nextcloud_backend.py`, `cloud_utils.py`, `cloud_backend_factory.py`, `onedrive_backend.py` — zero HIGH/MEDIUM/LOW findings.
Threat model coverage:
| Threat ID | Mitigation | Evidence |
|-----------|-----------|---------|
| T-12.1-01 | Provider response fields cannot override connection_id/user_id | `TestProviderHostileIdentityRejection` in contract suite |
| T-12.1-02 | SSRF through Nextcloud URL normalization | `normalize_nextcloud_url` always calls `validate_cloud_url`; HTTPS-only; rejects userinfo |
| T-12.1-03 | Partial page authorizes deletion | `complete=False` on error; `test_provider_page_failure_is_incomplete` |
| T-12.1-04 | Browse downloads or mutates provider content | `test_provider_listing_never_downloads_or_mutates` spies on forbidden methods |
| T-12.1-05 | Secrets in fixtures or error messages | Synthetic fixtures only; controlled exceptions |
## Known Stubs
None — all contract tests use real provider implementations (not mocks of the contract itself).
## Self-Check: PASSED
| Check | Result |
|-------|--------|
| `backend/tests/test_cloud_provider_contract.py` | FOUND |
| `backend/tests/fixtures/cloud/nextcloud_root.xml` | FOUND |
| `backend/storage/nextcloud_backend.py` | FOUND |
| `backend/storage/cloud_utils.py` | FOUND |
| Commit eb68fac (RED tests) | FOUND |
| Commit 2b46f74 (GREEN fix) | FOUND |
| Commit 805fe44 (Drive/OneDrive) | FOUND |
| Commit 6212873 (docs, asyncio fix) | FOUND |
+3 -1
View File
@@ -4,7 +4,7 @@
DocuVault is a multi-user SaaS document management platform built on FastAPI (Python) + Vue 3. It handles document upload, text extraction (PDF/DOCX/image/text), AI-based topic classification, per-user isolated storage, folder organization, document sharing, and pluggable cloud storage backends (OneDrive, Google Drive, Nextcloud, WebDAV). DocuVault is a multi-user SaaS document management platform built on FastAPI (Python) + Vue 3. It handles document upload, text extraction (PDF/DOCX/image/text), AI-based topic classification, per-user isolated storage, folder organization, document sharing, and pluggable cloud storage backends (OneDrive, Google Drive, Nextcloud, WebDAV).
**Current state:** v0.2.1 — Phase 12 gap-closure complete 2026-06-20. Migration-gated Compose startup (migrate service, service_completed_successfully dependency gate), 0005→0006 upgrade regression tests, RUNBOOK migration ops section. Provider-neutral cloud resource contract (CloudResourceAdapter, CloudResource, CloudCapability), durable CloudItem/CloudFolderState metadata layer (Alembic migration 0006), connection-ID browse API with IDOR protection and stale-while-revalidate, refresh_cloud_folder Celery task, and capability-aware StorageBrowser frontend. 12 of 12 phases complete. Not cleared for public internet deployment. **Current state:** v0.2.3 — Phase 12.1 Plan 01 complete 2026-06-22. Cloud provider adapter contract repaired: Nextcloud now inherits canonical `WebDAVBackend.list_folder(connection_id, user_id, parent_ref=None, page_token=None) -> CloudListing`; legacy `list_folder(folder_path="") -> list[dict]` override removed. `normalize_nextcloud_url()` added to `cloud_utils.py`. OneDrive `@odata.nextLink` cross-origin guard added. Four-provider parametrized contract suite in `test_cloud_provider_contract.py`. Phase 12.1 in progress. Not cleared for public internet deployment.
## Stack ## Stack
@@ -41,12 +41,14 @@ Before adding a helper, check if it belongs in an existing shared module:
| `backend/ai/utils.py` | `strip_code_fences`, `parse_classification`, `parse_suggestions` — AI response parsing shared by all providers | | `backend/ai/utils.py` | `strip_code_fences`, `parse_classification`, `parse_suggestions` — AI response parsing shared by all providers |
| `backend/services/auth.py` | `validate_password_strength(password)` — raises `ValueError`; routers catch and re-raise as `HTTPException` | | `backend/services/auth.py` | `validate_password_strength(password)` — raises `ValueError`; routers catch and re-raise as `HTTPException` |
| `backend/storage/cloud_base.py` | `CloudResourceAdapter`, `CloudListing`, `CloudResource`, `CloudCapability` — Phase 12 browse contract; all 4 providers implement this | | `backend/storage/cloud_base.py` | `CloudResourceAdapter`, `CloudListing`, `CloudResource`, `CloudCapability` — Phase 12 browse contract; all 4 providers implement this |
| `backend/storage/cloud_utils.py` | `validate_cloud_url`, `normalize_nextcloud_url`, `encrypt_credentials`, `decrypt_credentials` — Phase 12.1 adds `normalize_nextcloud_url` |
| `backend/services/cloud_items.py` | `resolve_owned_connection`, `upsert_cloud_item`, `reconcile_cloud_listing`, `get_or_create_folder_state` — owner-scoped cloud metadata service | | `backend/services/cloud_items.py` | `resolve_owned_connection`, `upsert_cloud_item`, `reconcile_cloud_listing`, `get_or_create_folder_state` — owner-scoped cloud metadata service |
| `backend/api/cloud/schemas.py` | `CloudBrowseResponse`, `CloudItemOut`, `FolderFreshnessOut`, `ConnectionRenameRequest` — credential-free cloud response schemas | | `backend/api/cloud/schemas.py` | `CloudBrowseResponse`, `CloudItemOut`, `FolderFreshnessOut`, `ConnectionRenameRequest` — credential-free cloud response schemas |
**Rules:** **Rules:**
- No router may define `_ip()`, `_get_ip()`, or any other local variant of `get_client_ip`. Import from `deps.utils`. - No router may define `_ip()`, `_get_ip()`, or any other local variant of `get_client_ip`. Import from `deps.utils`.
- No router may define its own `CloudConnectionError`. Import from `storage.exceptions`. - No router may define its own `CloudConnectionError`. Import from `storage.exceptions`.
- `NextcloudBackend` must NOT override `list_folder` — it inherits the canonical `WebDAVBackend` implementation. Any Nextcloud-specific URL handling belongs in `normalize_nextcloud_url` in `cloud_utils.py`.
- No AI provider may define its own `_strip_code_fences` or `_parse_*`. Import from `ai.utils`. - No AI provider may define its own `_strip_code_fences` or `_parse_*`. Import from `ai.utils`.
- No API file may define `_validate_password_strength`. Import from `services.auth`. - No API file may define `_validate_password_strength`. Import from `services.auth`.
- Service layer raises `ValueError` (or domain exceptions), never `HTTPException`. Only the router layer raises `HTTPException`. - Service layer raises `ValueError` (or domain exceptions), never `HTTPException`. Only the router layer raises `HTTPException`.
+4 -2
View File
@@ -1,6 +1,6 @@
# DocuVault # DocuVault
**Version 0.2.1 — Alpha** **Version 0.2.3 — Alpha**
> **Not production-ready.** DocuVault is functional for local and self-hosted use but has not been audited or hardened for public internet exposure. APIs, environment variables, and the database schema may change without notice until a stable 1.0 release is declared. > **Not production-ready.** DocuVault is functional for local and self-hosted use but has not been audited or hardened for public internet exposure. APIs, environment variables, and the database schema may change without notice until a stable 1.0 release is declared.
@@ -303,11 +303,13 @@ Users connect cloud storage through **Settings → Cloud Storage**. Credentials
|----------|-------------|-------| |----------|-------------|-------|
| Google Drive | OAuth 2.0 | Requires `GOOGLE_CLIENT_ID/SECRET` | | Google Drive | OAuth 2.0 | Requires `GOOGLE_CLIENT_ID/SECRET` |
| Microsoft OneDrive | OAuth 2.0 (MSAL) | Requires `ONEDRIVE_CLIENT_ID/SECRET` | | Microsoft OneDrive | OAuth 2.0 (MSAL) | Requires `ONEDRIVE_CLIENT_ID/SECRET` |
| Nextcloud | Username + password | Custom server URL; SSRF allowlist enforced | | Nextcloud | Username + password | Custom server URL; SSRF allowlist enforced; URL normalized to canonical DAV root |
| WebDAV | Username + password | Any RFC 4918 server; SSRF allowlist enforced | | WebDAV | Username + password | Any RFC 4918 server; SSRF allowlist enforced |
Connection statuses: `ACTIVE`, `REQUIRES_REAUTH`, `ERROR`. An externally revoked OAuth token transitions to `REQUIRES_REAUTH` without a 500 error. Connection statuses: `ACTIVE`, `REQUIRES_REAUTH`, `ERROR`. An externally revoked OAuth token transitions to `REQUIRES_REAUTH` without a 500 error.
All four providers implement the canonical `CloudResourceAdapter.list_folder(connection_id, user_id, parent_ref=None, page_token=None) -> CloudListing` contract. `@odata.nextLink` pagination for OneDrive is validated to stay on `graph.microsoft.com` before following.
### Connection-ID Browse API (Phase 12) ### Connection-ID Browse API (Phase 12)
Each connected account is independently addressable by its connection UUID: Each connected account is independently addressable by its connection UUID:
+1 -1
View File
@@ -244,7 +244,7 @@ async def lifespan(app: FastAPI):
# ── Application factory ─────────────────────────────────────────────────────── # ── Application factory ───────────────────────────────────────────────────────
app = FastAPI(title="Document Scanner API", version="0.2.2", lifespan=lifespan) app = FastAPI(title="Document Scanner API", version="0.2.3", lifespan=lifespan)
# Rate limiter state (slowapi) # Rate limiter state (slowapi)
app.state.limiter = auth_limiter app.state.limiter = auth_limiter
+13 -2
View File
@@ -17,10 +17,21 @@ def build_cloud_backend(provider: str, credentials: dict):
if provider == "nextcloud": if provider == "nextcloud":
from storage.nextcloud_backend import NextcloudBackend # lazy import from storage.nextcloud_backend import NextcloudBackend # lazy import
from storage.cloud_utils import normalize_nextcloud_url # lazy import
server_url = credentials["server_url"]
username = credentials["username"]
# Normalize to canonical WebDAV root idempotently; validate SSRF before construction
try:
server_url = normalize_nextcloud_url(server_url, username)
except ValueError:
# URL may already be canonical or validation rejected it — let the
# constructor's validate_cloud_url call surface the error
pass
return NextcloudBackend( return NextcloudBackend(
credentials["server_url"], server_url,
credentials["username"], username,
credentials["password"], credentials["password"],
) )
+91
View File
@@ -141,6 +141,97 @@ def _derive_fernet_key(master_key: bytes, user_id: str) -> Fernet:
return Fernet(fernet_key) return Fernet(fernet_key)
def normalize_nextcloud_url(base_url: str, username: str) -> str:
"""Return the canonical Nextcloud WebDAV root URL for a given base URL and username.
The Nextcloud WebDAV root for a user is:
https://<host>[/<subpath>]/remote.php/dav/files/<encoded_username>/
This helper is idempotent: if the URL is already canonical it is returned
unchanged (no double-append of the WebDAV suffix).
Security contract:
- Only https:// is accepted (rejects http, ftp, and bare origins).
- userinfo, query strings, and fragments are rejected.
- The username is percent-encoded as a single path segment.
- The normalized URL is validated through validate_cloud_url() before return.
- The normalized URL is never logged (it contains the username path segment).
Args:
base_url: Nextcloud server URL. May be:
- A bare origin: "https://nc.example.com"
- Origin with trailing slash: "https://nc.example.com/"
- Origin with deployment subpath: "https://example.com/nextcloud"
- Already canonical DAV root: "https://nc.example.com/remote.php/dav/files/alice/"
username: Nextcloud username. Will be percent-encoded as one path segment.
Returns:
Canonical WebDAV root URL ending with "/".
Raises:
ValueError: If the URL uses http, contains userinfo/query/fragment,
is malformed, or resolves to a private/internal address.
"""
from urllib.parse import quote as _quote, urlparse as _urlparse, urlunparse as _urlunparse
if not base_url:
raise ValueError("Nextcloud base URL must not be empty.")
parsed = _urlparse(base_url)
# Reject non-HTTPS for Nextcloud (credentials travel over the connection)
if parsed.scheme != "https":
raise ValueError(
f"Nextcloud URL must use https, got scheme {parsed.scheme!r}."
)
# Reject userinfo component (e.g. user:pass@ in URL)
if parsed.username or parsed.password:
raise ValueError(
"Nextcloud URL must not contain userinfo (user:pass@host). "
"Pass credentials separately."
)
# Reject query strings and fragments
if parsed.query:
raise ValueError(
"Nextcloud URL must not contain a query string."
)
if parsed.fragment:
raise ValueError(
"Nextcloud URL must not contain a URL fragment."
)
if not parsed.hostname:
raise ValueError("Nextcloud URL has no hostname.")
# Build the canonical DAV path suffix
encoded_user = _quote(username, safe="")
dav_suffix = f"/remote.php/dav/files/{encoded_user}/"
# Determine the deployment subpath (everything between host and known DAV suffix)
path = parsed.path.rstrip("/")
# Idempotence: strip the canonical suffix if already present
dav_anchor = "/remote.php/dav/files/"
if dav_anchor in path:
# Already canonical or contains the suffix — normalise to canonical form
idx = path.index(dav_anchor)
subpath = path[:idx] # deployment prefix
else:
subpath = path
canonical_path = subpath.rstrip("/") + dav_suffix
# Reconstruct with scheme + host only (no port duplication; netloc handles port)
canonical = _urlunparse((parsed.scheme, parsed.netloc, canonical_path, "", "", ""))
# Final SSRF validation — reject private/loopback/link-local targets
validate_cloud_url(canonical)
return canonical
def encrypt_credentials(master_key: bytes, user_id: str, credentials: dict) -> str: def encrypt_credentials(master_key: bytes, user_id: str, credentials: dict) -> str:
"""Encrypt a credentials dict to a Fernet token string. """Encrypt a credentials dict to a Fernet token string.
+20 -83
View File
@@ -1,24 +1,24 @@
""" """
NextcloudBackend — Nextcloud-specific WebDAV StorageBackend implementation. NextcloudBackend — Nextcloud-specific WebDAV StorageBackend implementation.
Extends WebDAVBackend with Nextcloud path conventions and folder listing. Extends WebDAVBackend with Nextcloud path conventions.
Inheritance: Inheritance:
NextcloudBackend → WebDAVBackend → StorageBackend NextcloudBackend → WebDAVBackend → StorageBackend
All 7 StorageBackend abstract methods are inherited from WebDAVBackend. All 7 StorageBackend abstract methods are inherited from WebDAVBackend.
list_folder is inherited from WebDAVBackend — NextcloudBackend does NOT
override it so that both providers share exactly one canonical adapter
implementation (Phase 12.1 P0 fix).
Only health_check is overridden to use the Nextcloud root (empty string Only health_check is overridden to use the Nextcloud root (empty string
is a valid check target for Nextcloud's WebDAV endpoint). is a valid check target for Nextcloud's WebDAV endpoint).
Additional method (not in ABC):
list_folder(folder_path: str) → list[dict]
Lists folder contents via client.list() + client.info(). Used by:
GET /api/cloud/folders/nextcloud/{folder_id} (D-09, lazy-load tree).
Nextcloud WebDAV path convention: Nextcloud WebDAV path convention:
The server_url should be the Nextcloud WebDAV base for a specific user: The server_url must be the normalized Nextcloud WebDAV root for a user:
https://nc.example.com/remote.php/dav/files/{username}/ https://nc.example.com/remote.php/dav/files/{username}/
Object keys are then relative to this root (inherited from WebDAVBackend). Use normalize_nextcloud_url(base_url, username) at construction time.
Object keys are relative to this root (inherited from WebDAVBackend).
SSRF prevention: SSRF prevention:
Inherited from WebDAVBackend.__init__ — validate_cloud_url(server_url) is Inherited from WebDAVBackend.__init__ — validate_cloud_url(server_url) is
@@ -38,10 +38,14 @@ from storage.webdav_backend import WebDAVBackend
class NextcloudBackend(WebDAVBackend): class NextcloudBackend(WebDAVBackend):
"""Nextcloud storage backend — extends WebDAVBackend. """Nextcloud storage backend — extends WebDAVBackend.
The server_url should be the full Nextcloud WebDAV root: The server_url must be the full normalized Nextcloud WebDAV root:
https://nc.example.com/remote.php/dav/files/{username}/ https://nc.example.com/remote.php/dav/files/{username}/
All 7 StorageBackend methods are inherited from WebDAVBackend. All 7 StorageBackend methods and list_folder (CloudResourceAdapter) are
inherited from WebDAVBackend — no public override. Nextcloud-specific
URL normalization is handled by normalize_nextcloud_url() in cloud_utils
before construction.
The SSRF guard is fully inherited — any private/localhost URL raises The SSRF guard is fully inherited — any private/localhost URL raises
ValueError at construction time (and before every outbound request). ValueError at construction time (and before every outbound request).
""" """
@@ -51,6 +55,7 @@ class NextcloudBackend(WebDAVBackend):
Args: Args:
server_url: Nextcloud WebDAV root URL. server_url: Nextcloud WebDAV root URL.
Should be pre-normalized via normalize_nextcloud_url().
E.g. "https://nc.example.com/remote.php/dav/files/alice/" E.g. "https://nc.example.com/remote.php/dav/files/alice/"
username: Nextcloud username (stored for path convention context). username: Nextcloud username (stored for path convention context).
password: Nextcloud account password or app-specific password (D-07). password: Nextcloud account password or app-specific password (D-07).
@@ -61,79 +66,11 @@ class NextcloudBackend(WebDAVBackend):
super().__init__(server_url, username, password) super().__init__(server_url, username, password)
self._username = username self._username = username
async def list_folder(self, folder_path: str = "") -> list[dict]: # list_folder is intentionally NOT overridden.
"""List folder contents at folder_path relative to the WebDAV root. # NextcloudBackend inherits WebDAVBackend.list_folder which implements the
# canonical (connection_id, user_id, parent_ref=None, page_token=None) -> CloudListing
Performs a PROPFIND (via client.list()) and fetches metadata for each # contract. A legacy list_folder(folder_path="") -> list[dict] override was
item via client.info(). Both calls are wrapped in asyncio.to_thread(). # the root cause of the Phase 12.1 P0 defect (TypeError on canonical invocation).
SSRF guard is called before every outbound request (D-17).
TTL caching at 60 seconds is handled by cloud_cache.get_cloud_folders_cached()
at the API layer — this method always performs live PROPFIND calls.
Args:
folder_path: Path relative to the WebDAV root to list.
Empty string or "/" lists the WebDAV root.
Returns:
List of dicts with keys:
- "id" (str): WebDAV path (usable as object_key or folder_id)
- "name" (str): File or folder display name
- "is_dir" (bool): True if the item is a directory
- "size" (int): File size in bytes (0 for directories)
Raises:
ValueError: If SSRF guard fires on re-validation.
Exception: Propagates any webdavclient3 exceptions (e.g. connection errors).
"""
# Re-validate before every outbound request (D-17 / T-05-04-02)
validate_cloud_url(self._server_url)
# client.list() returns a list of file/folder names in the directory
items = await asyncio.to_thread(self._client.list, folder_path)
folder_norm = folder_path.strip("/")
result: list[dict] = []
for name in items:
# Skip the "." self-reference and empty entries
if not name or name in (".", "./"):
continue
# Skip the directory itself (PROPFIND Depth:1 always includes the parent)
if name.strip("/") == folder_norm:
continue
# Construct the full path for info lookup
if folder_path:
item_path = f"{folder_path.rstrip('/')}/{name}"
else:
item_path = name
# Fetch metadata for each item
validate_cloud_url(self._server_url)
try:
info = await asyncio.to_thread(self._client.info, item_path)
size = int(info.get("size", 0))
# webdavclient3 info dict uses "isdir" or checks content_type
# is_dir is True if size == 0 and name ends with "/" or info signals it
is_dir = info.get("isdir", False) or str(info.get("content_type", "")).startswith(
"httpd/unix-directory"
) or name.endswith("/")
except Exception:
# If we can't get info for an item, include it with defaults
size = 0
is_dir = name.endswith("/")
result.append(
{
"id": item_path,
"name": name.rstrip("/"),
"is_dir": is_dir,
"size": size,
}
)
return result
async def health_check(self) -> bool: async def health_check(self) -> bool:
"""Return True if the Nextcloud WebDAV endpoint is reachable. """Return True if the Nextcloud WebDAV endpoint is reachable.
+14 -2
View File
@@ -28,6 +28,8 @@ import io
import uuid import uuid
from typing import Optional from typing import Optional
import urllib.parse
import httpx import httpx
import msal import msal
@@ -50,6 +52,9 @@ from storage.google_drive_backend import CloudConnectionError # reuse shared ex
GRAPH_BASE = "https://graph.microsoft.com/v1.0" GRAPH_BASE = "https://graph.microsoft.com/v1.0"
CHUNK_SIZE = 10 * 1024 * 1024 # 10 MB — above Graph's 4 MB simple upload limit (Pitfall 6) CHUNK_SIZE = 10 * 1024 * 1024 # 10 MB — above Graph's 4 MB simple upload limit (Pitfall 6)
# Trusted Graph API origin for @odata.nextLink validation (T-12.1-03)
_GRAPH_HOST = "graph.microsoft.com"
class OneDriveBackend(StorageBackend, CloudResourceAdapter): class OneDriveBackend(StorageBackend, CloudResourceAdapter):
"""Microsoft Graph / OneDrive implementation of StorageBackend. """Microsoft Graph / OneDrive implementation of StorageBackend.
@@ -351,7 +356,6 @@ class OneDriveBackend(StorageBackend, CloudResourceAdapter):
content_type: Optional[str] = None content_type: Optional[str] = None
if "file" in item: if "file" in item:
content_type = item["file"].get("mimeType") content_type = item["file"].get("mimeType")
parent_ref_val = item.get("parentReference", {}).get("id")
resources.append( resources.append(
CloudResource( CloudResource(
id=uuid.uuid4(), id=uuid.uuid4(),
@@ -367,7 +371,15 @@ class OneDriveBackend(StorageBackend, CloudResourceAdapter):
etag=item.get("eTag") or item.get("cTag"), etag=item.get("eTag") or item.get("cTag"),
) )
) )
url = data.get("@odata.nextLink") # Validate @odata.nextLink origin before following (T-12.1-03)
next_link = data.get("@odata.nextLink")
if next_link:
parsed_next = urllib.parse.urlparse(next_link)
if parsed_next.hostname != _GRAPH_HOST:
# Cross-origin continuation URL — untrusted; stop pagination
complete = False
break
url = next_link
except Exception: except Exception:
complete = False complete = False
+90
View File
@@ -0,0 +1,90 @@
{
"_comment": "Synthetic Google Drive fixture — no real credentials, tokens, or user data.",
"root_page1": {
"nextPageToken": "page2token_synthetic",
"files": [
{
"id": "synth_folder_a1b2c3",
"name": "Projects",
"mimeType": "application/vnd.google-apps.folder",
"modifiedTime": "2024-06-10T12:00:00Z"
},
{
"id": "synth_folder_d4e5f6",
"name": "Archive",
"mimeType": "application/vnd.google-apps.folder",
"modifiedTime": "2024-06-11T08:00:00Z"
},
{
"id": "synth_file_g7h8i9",
"name": "report.pdf",
"mimeType": "application/pdf",
"size": "204800",
"modifiedTime": "2024-06-12T14:00:00Z",
"md5Checksum": "abc123synth"
}
]
},
"root_page2": {
"files": [
{
"id": "synth_file_j0k1l2",
"name": "notes.txt",
"mimeType": "text/plain",
"size": "1024",
"modifiedTime": "2024-06-13T10:00:00Z",
"md5Checksum": "def456synth"
},
{
"id": "synth_native_m3n4o5",
"name": "Meeting Notes",
"mimeType": "application/vnd.google-apps.document",
"modifiedTime": "2024-06-14T09:30:00Z"
}
]
},
"nested_folder": {
"files": [
{
"id": "synth_nested_p6q7r8",
"name": "sub-document.docx",
"mimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
"size": "51200",
"modifiedTime": "2024-06-15T11:00:00Z"
}
]
},
"trashed_excluded": {
"files": [
{
"id": "synth_file_s9t0u1",
"name": "visible.pdf",
"mimeType": "application/pdf",
"size": "10240",
"modifiedTime": "2024-06-16T10:00:00Z"
}
],
"_note": "trashed=false query param ensures trashed items are not returned"
},
"native_docs_nullable_size": {
"files": [
{
"id": "synth_doc_v2w3x4",
"name": "Spreadsheet",
"mimeType": "application/vnd.google-apps.spreadsheet",
"modifiedTime": "2024-06-17T13:00:00Z"
},
{
"id": "synth_slides_y5z6a7",
"name": "Presentation",
"mimeType": "application/vnd.google-apps.presentation",
"modifiedTime": "2024-06-18T14:00:00Z"
}
]
}
}
+164
View File
@@ -0,0 +1,164 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Synthetic Nextcloud Depth-1 PROPFIND fixture — no real credentials, hostnames, tokens, or user data.
Root path: /remote.php/dav/files/testuser/
Represents: 4 folders + 6 files directly in root.
Covers: root self-entry filtering, absolute hrefs, encoded spaces/unicode, missing optional props.
-->
<d:multistatus xmlns:d="DAV:" xmlns:nc="http://nextcloud.org/ns" xmlns:oc="http://owncloud.org/ns">
<!-- Root self-entry — MUST be filtered out by the parser -->
<d:response>
<d:href>/remote.php/dav/files/testuser/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>testuser</d:displayname>
<d:getlastmodified>Mon, 01 Jan 2024 00:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: Documents -->
<d:response>
<d:href>/remote.php/dav/files/testuser/Documents/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>Documents</d:displayname>
<d:getlastmodified>Mon, 10 Jun 2024 12:00:00 GMT</d:getlastmodified>
<oc:size>0</oc:size>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: Photos -->
<d:response>
<d:href>/remote.php/dav/files/testuser/Photos/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>Photos</d:displayname>
<d:getlastmodified>Tue, 11 Jun 2024 08:30:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: My Work Files (contains spaces — encoded in href) -->
<d:response>
<d:href>/remote.php/dav/files/testuser/My%20Work%20Files/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>My Work Files</d:displayname>
<d:getlastmodified>Wed, 12 Jun 2024 09:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: Unicode name: Café Designs -->
<d:response>
<d:href>/remote.php/dav/files/testuser/Caf%C3%A9%20Designs/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>Café Designs</d:displayname>
<d:getlastmodified>Thu, 13 Jun 2024 10:15:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: report.pdf — all optional props present -->
<d:response>
<d:href>/remote.php/dav/files/testuser/report.pdf</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>report.pdf</d:displayname>
<d:getcontentlength>204800</d:getcontentlength>
<d:getcontenttype>application/pdf</d:getcontenttype>
<d:getlastmodified>Fri, 14 Jun 2024 14:00:00 GMT</d:getlastmodified>
<d:getetag>"abc123etag"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: notes.txt — missing content-type and etag -->
<d:response>
<d:href>/remote.php/dav/files/testuser/notes.txt</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>notes.txt</d:displayname>
<d:getcontentlength>1024</d:getcontentlength>
<d:getlastmodified>Sat, 15 Jun 2024 10:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: My Report Draft.docx — spaces in name -->
<d:response>
<d:href>/remote.php/dav/files/testuser/My%20Report%20Draft.docx</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>My Report Draft.docx</d:displayname>
<d:getcontentlength>51200</d:getcontentlength>
<d:getcontenttype>application/vnd.openxmlformats-officedocument.wordprocessingml.document</d:getcontenttype>
<d:getlastmodified>Sun, 16 Jun 2024 11:00:00 GMT</d:getlastmodified>
<d:getetag>"docx456etag"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: image.png — size absent (should be None) -->
<d:response>
<d:href>/remote.php/dav/files/testuser/image.png</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>image.png</d:displayname>
<d:getcontenttype>image/png</d:getcontenttype>
<d:getlastmodified>Mon, 17 Jun 2024 12:30:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: data.csv — all optional props absent (missing getcontenttype, getetag, size) -->
<d:response>
<d:href>/remote.php/dav/files/testuser/data.csv</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>data.csv</d:displayname>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: archive.zip — relative href (no leading slash) -->
<d:response>
<d:href>remote.php/dav/files/testuser/archive.zip</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>archive.zip</d:displayname>
<d:getcontentlength>102400</d:getcontentlength>
<d:getcontenttype>application/zip</d:getcontenttype>
<d:getlastmodified>Tue, 18 Jun 2024 09:00:00 GMT</d:getlastmodified>
<d:getetag>"zip789etag"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
</d:multistatus>
+93
View File
@@ -0,0 +1,93 @@
{
"_comment": "Synthetic OneDrive fixture — no real credentials, tokens, drive IDs, or user data.",
"root_page1": {
"value": [
{
"id": "synth_od_folder_a1b2",
"name": "Projects",
"folder": {"childCount": 3},
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-10T12:00:00Z",
"eTag": "etag_synth_folder_a1b2"
},
{
"id": "synth_od_folder_c3d4",
"name": "My Archive",
"folder": {"childCount": 5},
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-11T08:00:00Z"
},
{
"id": "synth_od_file_e5f6",
"name": "report.pdf",
"file": {"mimeType": "application/pdf"},
"size": 204800,
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-12T14:00:00Z",
"eTag": "etag_synth_file_e5f6"
}
],
"@odata.nextLink": "https://graph.microsoft.com/v1.0/me/drive/root/children?$skip=3&$top=3"
},
"root_page2": {
"value": [
{
"id": "synth_od_file_g7h8",
"name": "My Report Draft.docx",
"file": {"mimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document"},
"size": 51200,
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-13T10:00:00Z"
},
{
"id": "synth_od_file_i9j0",
"name": "notes.txt",
"file": {"mimeType": "text/plain"},
"size": 1024,
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-14T09:00:00Z",
"eTag": "etag_synth_file_i9j0"
}
]
},
"nested_children": {
"value": [
{
"id": "synth_od_file_k1l2",
"name": "sub-document.docx",
"file": {"mimeType": "application/vnd.openxmlformats-officedocument.wordprocessingml.document"},
"size": 25600,
"parentReference": {"driveId": "synth-drive-id", "id": "synth_od_folder_a1b2"},
"lastModifiedDateTime": "2024-06-15T11:00:00Z"
}
]
},
"nullable_metadata": {
"value": [
{
"id": "synth_od_file_m3n4",
"name": "unknown-file.bin",
"file": {},
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-16T10:00:00Z"
}
]
},
"encoded_names": {
"value": [
{
"id": "synth_od_file_o5p6",
"name": "Annual Report & Summary.pdf",
"file": {"mimeType": "application/pdf"},
"size": 102400,
"parentReference": {"driveId": "synth-drive-id", "id": "root"},
"lastModifiedDateTime": "2024-06-17T13:00:00Z"
}
]
}
}
+105
View File
@@ -0,0 +1,105 @@
<?xml version="1.0" encoding="utf-8"?>
<!--
Synthetic generic WebDAV Depth-1 PROPFIND fixture — no real credentials, hostnames, tokens.
Root path: /dav/
Represents: 2 folders + 3 files directly in root.
Covers: absolute/relative hrefs, missing optional props, foreign href rejection.
-->
<d:multistatus xmlns:d="DAV:">
<!-- Root self-entry — MUST be filtered out -->
<d:response>
<d:href>/dav/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>root</d:displayname>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: Archive -->
<d:response>
<d:href>/dav/Archive/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>Archive</d:displayname>
<d:getlastmodified>Mon, 10 Jun 2024 08:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Folder: Shared (relative href) -->
<d:response>
<d:href>dav/Shared/</d:href>
<d:propstat>
<d:prop>
<d:resourcetype><d:collection/></d:resourcetype>
<d:displayname>Shared</d:displayname>
<d:getlastmodified>Tue, 11 Jun 2024 09:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: readme.txt — all optional props -->
<d:response>
<d:href>/dav/readme.txt</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>readme.txt</d:displayname>
<d:getcontentlength>512</d:getcontentlength>
<d:getcontenttype>text/plain</d:getcontenttype>
<d:getlastmodified>Wed, 12 Jun 2024 10:00:00 GMT</d:getlastmodified>
<d:getetag>"readmeetag"</d:getetag>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: backup.tar.gz — missing optional props -->
<d:response>
<d:href>/dav/backup.tar.gz</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>backup.tar.gz</d:displayname>
<d:getcontentlength>10240</d:getcontentlength>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- File: project.pdf — encoded spaces in href -->
<d:response>
<d:href>/dav/project%20notes.pdf</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>project notes.pdf</d:displayname>
<d:getcontentlength>204800</d:getcontentlength>
<d:getcontenttype>application/pdf</d:getcontenttype>
<d:getlastmodified>Thu, 13 Jun 2024 11:00:00 GMT</d:getlastmodified>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
<!-- Foreign href — escapes configured root; MUST cause listing to be incomplete -->
<d:response>
<d:href>/other-root/escape.txt</d:href>
<d:propstat>
<d:prop>
<d:resourcetype/>
<d:displayname>escape.txt</d:displayname>
<d:getcontentlength>100</d:getcontentlength>
</d:prop>
<d:status>HTTP/1.1 200 OK</d:status>
</d:propstat>
</d:response>
</d:multistatus>
+158
View File
@@ -535,6 +535,164 @@ class TestOneDriveListFolder:
assert len(result.items) == 2 assert len(result.items) == 2
class TestOneDriveCrossOriginNextLink:
"""OneDrive @odata.nextLink must stay on graph.microsoft.com — T-12.1-03."""
def _make_backend(self):
from storage.onedrive_backend import OneDriveBackend
return OneDriveBackend({
"access_token": "tok",
"refresh_token": "ref",
"expires_at": "2099-01-01T00:00:00",
})
@pytest.mark.asyncio
async def test_cross_origin_next_link_returns_incomplete(self):
"""A cross-origin @odata.nextLink must stop pagination and return complete=False."""
from unittest.mock import patch, MagicMock, AsyncMock
import uuid
backend = self._make_backend()
page1 = MagicMock()
page1.is_success = True
page1.json.return_value = {
"value": [{"id": "i1", "name": "A.txt", "file": {}, "size": 10}],
# Cross-origin nextLink — must be rejected
"@odata.nextLink": "https://evil.example.com/v1.0/me/drive/root/children?$skip=1",
}
with patch("httpx.AsyncClient") as mock_client_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=page1)
mock_client_cls.return_value = mock_client
result = await backend.list_folder(uuid.uuid4(), uuid.uuid4())
# Items from page1 should be retained
assert len(result.items) == 1
assert result.items[0].name == "A.txt"
# But listing is not complete (cross-origin continuation not trusted)
assert result.complete is False
@pytest.mark.asyncio
async def test_same_origin_graph_next_link_followed(self):
"""A same-origin graph.microsoft.com @odata.nextLink is trusted and followed."""
from unittest.mock import patch, MagicMock, AsyncMock
import uuid
backend = self._make_backend()
page1 = MagicMock()
page1.is_success = True
page1.json.return_value = {
"value": [{"id": "i1", "name": "A.txt", "file": {}, "size": 10}],
"@odata.nextLink": "https://graph.microsoft.com/v1.0/me/drive/root/children?$skip=1",
}
page2 = MagicMock()
page2.is_success = True
page2.json.return_value = {
"value": [{"id": "i2", "name": "B.txt", "file": {}, "size": 20}],
}
with patch("httpx.AsyncClient") as mock_client_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(side_effect=[page1, page2])
mock_client_cls.return_value = mock_client
result = await backend.list_folder(uuid.uuid4(), uuid.uuid4())
assert result.complete is True
assert len(result.items) == 2
@pytest.mark.asyncio
async def test_page_failure_retains_prior_items(self):
"""A page HTTP failure returns complete=False but retains already-normalized items."""
from unittest.mock import patch, MagicMock, AsyncMock
import uuid
backend = self._make_backend()
page1 = MagicMock()
page1.is_success = True
page1.json.return_value = {
"value": [{"id": "i1", "name": "A.txt", "file": {}, "size": 10}],
"@odata.nextLink": "https://graph.microsoft.com/v1.0/me/drive/root/children?$skip=1",
}
page2_fail = MagicMock()
page2_fail.is_success = False
page2_fail.status_code = 503
with patch("httpx.AsyncClient") as mock_client_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(side_effect=[page1, page2_fail])
mock_client_cls.return_value = mock_client
result = await backend.list_folder(uuid.uuid4(), uuid.uuid4())
# Page1 items retained
assert len(result.items) == 1
assert result.items[0].name == "A.txt"
# Not complete because page2 failed
assert result.complete is False
class TestGoogleDriveAuthFailureControl:
"""Drive auth failures map to controlled complete=False — T-12.1-03."""
def _make_backend(self):
from storage.google_drive_backend import GoogleDriveBackend
return GoogleDriveBackend({
"access_token": "tok",
"refresh_token": "ref",
"token_uri": "https://oauth2.googleapis.com/token",
"client_id": "cid",
"client_secret": "csec",
})
@pytest.mark.asyncio
async def test_401_returns_incomplete_not_raises(self):
"""401 from Drive returns complete=False — not an uncaught exception."""
from unittest.mock import MagicMock, patch
from googleapiclient.errors import HttpError
import uuid
backend = self._make_backend()
fake_service = MagicMock()
resp = MagicMock()
resp.status = 401
fake_service.files().list().execute.side_effect = HttpError(resp, b"unauthorized")
with patch.object(backend, "_get_service", return_value=fake_service):
result = await backend.list_folder(uuid.uuid4(), uuid.uuid4())
assert result.complete is False
@pytest.mark.asyncio
async def test_page1_ok_page2_error_retains_items(self):
"""Error on page2 still returns items from page1 with complete=False."""
from unittest.mock import MagicMock, patch
from googleapiclient.errors import HttpError
import uuid
backend = self._make_backend()
fake_service = MagicMock()
resp403 = MagicMock()
resp403.status = 403
fake_service.files().list().execute.side_effect = [
{
"files": [{"id": "f1", "name": "a.pdf", "mimeType": "application/pdf", "size": "100"}],
"nextPageToken": "synth_tok",
},
HttpError(resp403, b"scope"),
]
with patch.object(backend, "_get_service", return_value=fake_service):
result = await backend.list_folder(uuid.uuid4(), uuid.uuid4())
# Items from page1 must be retained
assert len(result.items) == 1
assert result.items[0].name == "a.pdf"
assert result.complete is False
class TestWebDAVListFolder: class TestWebDAVListFolder:
"""WebDAV list_folder tests — SSRF guard and no byte-download.""" """WebDAV list_folder tests — SSRF guard and no byte-download."""
@@ -0,0 +1,825 @@
"""
Phase 12.1 Plan 01 — Shared four-provider CloudResourceAdapter contract suite.
Purpose
-------
One parametrized contract harness asserting the canonical behavior of every
CloudResourceAdapter implementation across Nextcloud, generic WebDAV, Google
Drive, and OneDrive.
Contract assertions
-------------------
1. Adapter is a CloudResourceAdapter subclass.
2. list_folder matches the canonical positional/keyword signature:
(connection_id, user_id, parent_ref=None, page_token=None)
3. Invocation through the canonical signature does NOT raise TypeError.
4. Return type is CloudListing (never a list[dict] or any other shape).
5. Every resource carries the trusted caller connection_id and user_id.
6. provider_item_id is set (non-empty opaque reference).
7. kind is exactly "file" or "folder".
8. parent_ref in every resource matches the parent_ref argument.
9. Optional metadata (size, content_type, modified_at, etag) is None when
absent rather than a fabricated value.
10. Forbidden spies: get_object, put, put_object, delete_object, upload,
create_folder, rename, delete, move, copy methods are never called by
list_folder.
TDD note: test_provider_list_folder_contract MUST FAIL against the current
Nextcloud override (wrong signature → TypeError on canonical invocation).
All other providers should pass from the start.
"""
from __future__ import annotations
import inspect
import json
import uuid
from pathlib import Path
from typing import Any
from unittest.mock import AsyncMock, MagicMock, patch
import pytest
# ── Fixture paths ─────────────────────────────────────────────────────────────
FIXTURES_DIR = Path(__file__).parent / "fixtures" / "cloud"
NEXTCLOUD_ROOT_XML = (FIXTURES_DIR / "nextcloud_root.xml").read_text()
WEBDAV_ROOT_XML = (FIXTURES_DIR / "webdav_root.xml").read_text()
_GD_FIXTURE = json.loads((FIXTURES_DIR / "google_drive_pages.json").read_text())
_OD_FIXTURE = json.loads((FIXTURES_DIR / "onedrive_pages.json").read_text())
# ── Provider factory helpers ──────────────────────────────────────────────────
def _make_nextcloud_adapter():
"""Construct a NextcloudBackend bypassing SSRF validation."""
from storage.nextcloud_backend import NextcloudBackend
from unittest.mock import MagicMock
with patch("storage.webdav_backend.validate_cloud_url"):
with patch("webdav3.client.Client"):
backend = NextcloudBackend.__new__(NextcloudBackend)
backend._server_url = "https://nc.synth.example.com/remote.php/dav/files/testuser/"
backend._username = "testuser"
backend._client = MagicMock()
return backend
def _make_webdav_adapter():
"""Construct a WebDAVBackend bypassing SSRF validation."""
from storage.webdav_backend import WebDAVBackend
with patch("storage.webdav_backend.validate_cloud_url"):
with patch("webdav3.client.Client"):
backend = WebDAVBackend.__new__(WebDAVBackend)
backend._server_url = "https://dav.synth.example.com/dav/"
backend._client = MagicMock()
return backend
def _make_google_drive_adapter():
"""Construct a GoogleDriveBackend (no network credentials needed for tests)."""
from storage.google_drive_backend import GoogleDriveBackend
return GoogleDriveBackend({
"access_token": "synth_access_token",
"refresh_token": "synth_refresh_token",
"token_uri": "https://oauth2.googleapis.com/token",
"client_id": "synth_client_id",
"client_secret": "synth_client_secret",
})
def _make_onedrive_adapter():
"""Construct a OneDriveBackend (no network credentials needed for tests)."""
from storage.onedrive_backend import OneDriveBackend
return OneDriveBackend({
"access_token": "synth_access_token",
"refresh_token": "synth_refresh_token",
"expires_at": "2099-01-01T00:00:00",
})
# ── Stub responses ────────────────────────────────────────────────────────────
def _webdav_list_stub(items: list[tuple[str, bool, int]]):
"""
items: list of (name, is_dir, size)
Returns a side_effect function for client.list() and client.info().
"""
def _list(path):
return [name for name, _, _ in items]
def _info(item_path):
for name, is_dir, size in items:
if name.rstrip("/") in item_path or item_path.endswith(name.rstrip("/")):
return {"isdir": is_dir, "size": size}
return {}
return _list, _info
# ── Provider case registry ────────────────────────────────────────────────────
# Each case: (provider_name, factory_fn, stub_setup_fn)
# stub_setup_fn(adapter, connection_id, user_id) → patches set up for a successful root listing
def _stub_nextcloud(adapter, conn_id, user_id):
"""
Stub for Nextcloud canonical-signature list_folder.
Since the current Nextcloud override is incompatible, this stub
cannot help the existing implementation — the TypeError happens before
the stubs are ever reached. This is the expected RED behavior.
"""
# Minimal stubs for when the implementation is fixed:
adapter._client.list = MagicMock(return_value=["folder1/", "file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 512})
def _stub_webdav(adapter, conn_id, user_id):
adapter._client.list = MagicMock(return_value=["subfolder/", "file.txt"])
adapter._client.info = MagicMock(side_effect=lambda p: {
"isdir": p.endswith("/"),
"size": 0 if p.endswith("/") else 512,
})
def _stub_google_drive(adapter, conn_id, user_id):
"""Stub Google Drive service to return one folder + one file."""
page1 = _GD_FIXTURE["root_page1"].copy()
del page1["nextPageToken"] # single page for basic contract test
page1["files"] = page1["files"][:2] # folder + file
fake_service = MagicMock()
fake_service.files().list().execute.return_value = page1
adapter._get_service = MagicMock(return_value=fake_service)
def _stub_onedrive(adapter, conn_id, user_id):
"""Stub OneDrive http client to return one folder + one file."""
page1 = {
"value": _OD_FIXTURE["root_page1"]["value"][:2], # folder + file
}
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page1
return mock_resp # caller uses this in httpx.AsyncClient patch
PROVIDER_CASES = [
pytest.param("nextcloud", _make_nextcloud_adapter, _stub_nextcloud, id="nextcloud"),
pytest.param("webdav", _make_webdav_adapter, _stub_webdav, id="webdav"),
pytest.param("google_drive", _make_google_drive_adapter, _stub_google_drive, id="google_drive"),
pytest.param("onedrive", _make_onedrive_adapter, _stub_onedrive, id="onedrive"),
]
# ── T-12.1-04: Forbidden-operation spies ─────────────────────────────────────
FORBIDDEN_METHODS = [
"get_object",
"put_object",
"delete_object",
"generate_presigned_put_url",
"presigned_get_url",
]
# Provider-specific forbidden method names (class-level or attribute names)
_DRIVE_FORBIDDEN = ["get_media"]
_ONEDRIVE_FORBIDDEN = [] # /content checked via URL in separate test
# ── Static contract tests (no provider invocation needed) ─────────────────────
class TestAdapterIsCloudResourceAdapter:
"""Assert all four provider classes subclass CloudResourceAdapter."""
@pytest.mark.parametrize("provider,factory_fn,_", PROVIDER_CASES)
def test_adapter_subclass(self, provider, factory_fn, _):
from storage.cloud_base import CloudResourceAdapter
adapter = factory_fn()
assert isinstance(adapter, CloudResourceAdapter), (
f"{provider} adapter does not implement CloudResourceAdapter"
)
class TestListFolderSignature:
"""Assert list_folder signature matches the canonical contract."""
@pytest.mark.parametrize("provider,factory_fn,_", PROVIDER_CASES)
def test_canonical_signature(self, provider, factory_fn, _):
"""list_folder must accept (connection_id, user_id, parent_ref=None, page_token=None)."""
adapter = factory_fn()
sig = inspect.signature(adapter.list_folder)
params = list(sig.parameters.keys())
# self is not included in the signature of a bound method
assert "connection_id" in params, f"{provider}: missing connection_id param"
assert "user_id" in params, f"{provider}: missing user_id param"
assert "parent_ref" in params, f"{provider}: missing parent_ref param"
assert "page_token" in params, f"{provider}: missing page_token param"
# parent_ref and page_token must have defaults
assert sig.parameters["parent_ref"].default is None, (
f"{provider}: parent_ref default must be None"
)
assert sig.parameters["page_token"].default is None, (
f"{provider}: page_token default must be None"
)
@pytest.mark.parametrize("provider,factory_fn,_", PROVIDER_CASES)
def test_list_folder_is_async(self, provider, factory_fn, _):
"""list_folder must be an async coroutine function."""
assert inspect.iscoroutinefunction(factory_fn().list_folder), (
f"{provider}: list_folder must be defined with 'async def'"
)
# ── Dynamic contract tests (invoke list_folder through canonical signature) ───
class TestProviderListFolderContract:
"""
Core behavioral contract: canonical invocation, return type, resource identity.
test_provider_list_folder_contract MUST FAIL for nextcloud before Task 2
repairs the incompatible override (TypeError on positional connection_id argument).
"""
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_list_folder_contract(self, provider, factory_fn, stub_fn):
"""Canonical invocation (connection_id, user_id, parent_ref=None) returns CloudListing."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
if provider == "nextcloud":
with patch("storage.webdav_backend.validate_cloud_url"):
stub_fn(adapter, conn_id, user_id)
# This MUST raise TypeError with the current broken Nextcloud override.
# After Task 2 it must not raise and must return CloudListing.
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "webdav":
with patch("storage.webdav_backend.validate_cloud_url"):
stub_fn(adapter, conn_id, user_id)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "google_drive":
stub_fn(adapter, conn_id, user_id)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "onedrive":
mock_resp = stub_fn(adapter, conn_id, user_id)
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert isinstance(result, CloudListing), (
f"{provider}: list_folder must return CloudListing, "
f"got {type(result).__name__!r}"
)
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_root_and_nested_identity(self, provider, factory_fn, stub_fn):
"""Every resource carries trusted caller connection_id and user_id."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
if provider == "nextcloud":
with patch("storage.webdav_backend.validate_cloud_url"):
stub_fn(adapter, conn_id, user_id)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "webdav":
with patch("storage.webdav_backend.validate_cloud_url"):
stub_fn(adapter, conn_id, user_id)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "google_drive":
stub_fn(adapter, conn_id, user_id)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "onedrive":
mock_resp = stub_fn(adapter, conn_id, user_id)
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert isinstance(result, CloudListing)
for item in result.items:
assert item.connection_id == conn_id, (
f"{provider}: resource.connection_id must equal trusted caller; "
f"got {item.connection_id!r}"
)
assert item.user_id == user_id, (
f"{provider}: resource.user_id must equal trusted caller; "
f"got {item.user_id!r}"
)
assert item.provider_item_id, (
f"{provider}: provider_item_id must be non-empty"
)
assert item.kind in ("file", "folder"), (
f"{provider}: kind must be 'file' or 'folder', got {item.kind!r}"
)
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_metadata_normalization(self, provider, factory_fn, stub_fn):
"""Absent optional metadata is None — not a fabricated default."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
if provider == "nextcloud":
# Use a stub that returns items with deliberately absent optional props
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["mystery.bin"])
adapter._client.info = MagicMock(return_value={})
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "webdav":
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["mystery.bin"])
adapter._client.info = MagicMock(return_value={})
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "google_drive":
fake_service = MagicMock()
fake_service.files().list().execute.return_value = {
"files": [{"id": "synth_xyz", "name": "noprop.txt", "mimeType": "text/plain"}]
}
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "onedrive":
page = {"value": [{"id": "synth_od_xyz", "name": "noprop.txt", "file": {}}]}
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert isinstance(result, CloudListing)
if result.items:
for item in result.items:
# Absent metadata must be None, not 0, "", or other fabricated values
# size for a file with no getcontentlength should be None or 0 (not negative)
assert item.size is None or isinstance(item.size, int), (
f"{provider}: size must be None or int, got {item.size!r}"
)
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_consumes_all_pages_before_complete(self, provider, factory_fn, stub_fn):
"""complete=True only after all pages/responses are consumed."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
if provider == "nextcloud":
# Single-response DAV listing — always complete if no error
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 100})
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert result.complete is True
elif provider == "webdav":
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 100})
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert result.complete is True
elif provider == "google_drive":
# Two pages — complete=True only after page2 consumed
fake_service = MagicMock()
fake_service.files().list().execute.side_effect = [
{
"files": [{"id": "f1", "name": "a.pdf", "mimeType": "application/pdf", "size": "100"}],
"nextPageToken": "synth_page2",
},
{
"files": [{"id": "f2", "name": "b.pdf", "mimeType": "application/pdf", "size": "200"}],
},
]
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert result.complete is True
assert len(result.items) == 2
elif provider == "onedrive":
page1 = {
"value": [{"id": "od1", "name": "a.txt", "file": {}, "size": 10}],
"@odata.nextLink": "https://graph.microsoft.com/v1.0/me/drive/root/children?$skip=1",
}
page2 = {
"value": [{"id": "od2", "name": "b.txt", "file": {}, "size": 20}],
}
resp1 = MagicMock()
resp1.is_success = True
resp1.json.return_value = page1
resp2 = MagicMock()
resp2.is_success = True
resp2.json.return_value = page2
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(side_effect=[resp1, resp2])
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert result.complete is True
assert len(result.items) == 2
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_page_failure_is_incomplete(self, provider, factory_fn, stub_fn):
"""A failed fetch/parse returns complete=False — never a fabricated complete empty."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
if provider in ("nextcloud", "webdav"):
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(side_effect=Exception("network failure"))
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "google_drive":
fake_service = MagicMock()
from googleapiclient.errors import HttpError
resp = MagicMock()
resp.status = 403
fake_service.files().list().execute.side_effect = HttpError(resp, b"forbidden")
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "onedrive":
err_resp = MagicMock()
err_resp.is_success = False
err_resp.status_code = 401
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=err_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert isinstance(result, CloudListing)
assert result.complete is False, (
f"{provider}: fetch failure must return complete=False"
)
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", PROVIDER_CASES)
async def test_provider_listing_never_downloads_or_mutates(
self, provider, factory_fn, stub_fn
):
"""list_folder must not call get_object, put_object, delete_object,
presigned_get_url, or generate_presigned_put_url."""
from storage.cloud_base import CloudListing
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
# Install failing spies on all forbidden methods
forbidden_patches = []
for method in FORBIDDEN_METHODS:
if hasattr(adapter, method):
p = patch.object(
adapter, method,
side_effect=AssertionError(f"FORBIDDEN: {method} was called by list_folder"),
)
forbidden_patches.append(p)
for p in forbidden_patches:
p.start()
try:
if provider in ("nextcloud", "webdav"):
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 100})
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "google_drive":
fake_service = MagicMock()
fake_service.files().list().execute.return_value = {
"files": [{"id": "synth_f1", "name": "x.pdf", "mimeType": "application/pdf", "size": "100"}]
}
# Also spy on get_media method of Drive service
fake_service.files().get_media.side_effect = AssertionError(
"FORBIDDEN: get_media called"
)
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
elif provider == "onedrive":
page = {"value": [{"id": "synth_od1", "name": "y.txt", "file": {}, "size": 10}]}
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
# Spy on PUT, POST, DELETE
mock_client.put = AsyncMock(side_effect=AssertionError("FORBIDDEN: put called"))
mock_client.post = AsyncMock(side_effect=AssertionError("FORBIDDEN: post called"))
mock_client.delete = AsyncMock(side_effect=AssertionError("FORBIDDEN: delete called"))
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert isinstance(result, CloudListing), (
f"{provider}: result must be CloudListing after forbidden-method spy pass"
)
finally:
for p in forbidden_patches:
p.stop()
class TestProviderParentRefPropagation:
"""parent_ref is correctly propagated into each CloudResource."""
@pytest.mark.asyncio
@pytest.mark.parametrize("provider,factory_fn,stub_fn", [
pytest.param("webdav", _make_webdav_adapter, _stub_webdav, id="webdav"),
pytest.param("google_drive", _make_google_drive_adapter, _stub_google_drive, id="google_drive"),
pytest.param("onedrive", _make_onedrive_adapter, _stub_onedrive, id="onedrive"),
])
async def test_parent_ref_copied_to_resources(self, provider, factory_fn, stub_fn):
"""Every resource.parent_ref equals the parent_ref argument."""
adapter = factory_fn()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
parent = "synth_parent_folder_ref"
if provider == "webdav":
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 100})
result = await adapter.list_folder(conn_id, user_id, parent_ref=parent)
elif provider == "google_drive":
fake_service = MagicMock()
fake_service.files().list().execute.return_value = {
"files": [{"id": "synth_f2", "name": "z.pdf", "mimeType": "application/pdf", "size": "100"}]
}
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=parent)
elif provider == "onedrive":
page = {"value": [{"id": "synth_od2", "name": "w.txt", "file": {}, "size": 20}]}
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=parent)
for item in result.items:
assert item.parent_ref == parent, (
f"{provider}: resource.parent_ref={item.parent_ref!r} "
f"does not match argument parent_ref={parent!r}"
)
class TestProviderHostileIdentityRejection:
"""Provider response fields must not override trusted caller identity."""
@pytest.mark.asyncio
async def test_google_drive_hostile_ids_overridden(self):
"""owner_id / connection_id in Drive response cannot replace trusted caller values."""
from storage.google_drive_backend import GoogleDriveBackend
adapter = _make_google_drive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
hostile_id = uuid.uuid4()
# The Drive response does not carry connection_id or user_id, but even if it
# returned parent references that look like UUIDs, those must not replace them.
fake_service = MagicMock()
fake_service.files().list().execute.return_value = {
"files": [
{
"id": str(hostile_id), # hostile provider_item_id != our user_id
"name": "file.pdf",
"mimeType": "application/pdf",
"size": "100",
"parents": [str(hostile_id)], # parent also hostile
}
]
}
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
for item in result.items:
assert item.connection_id == conn_id
assert item.user_id == user_id
# provider_item_id may equal hostile_id (it is opaque) — that is fine
# but connection_id and user_id must be the trusted caller values
@pytest.mark.asyncio
async def test_onedrive_hostile_ids_overridden(self):
"""owner from @odata response cannot replace trusted caller identity."""
adapter = _make_onedrive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
hostile_user = uuid.uuid4()
page = {
"value": [
{
"id": "od_item1",
"name": "x.txt",
"file": {},
"size": 100,
"createdBy": {"user": {"id": str(hostile_user)}},
}
]
}
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
for item in result.items:
assert item.connection_id == conn_id
assert item.user_id == user_id
class TestNextcloudSpecificContract:
"""Nextcloud-specific regression tests (P0 from research)."""
def test_nextcloud_signature_matches_canonical_webdav(self):
"""
inspect.signature(NextcloudBackend.list_folder) must match WebDAVBackend.list_folder.
This test is the root-cause regression for the P0 defect documented in research:
NextcloudBackend.list_folder(folder_path="") breaks the canonical four-argument call.
Before Task 2 this test will fail; after Task 2 it must pass.
"""
from storage.nextcloud_backend import NextcloudBackend
from storage.webdav_backend import WebDAVBackend
nc_sig = inspect.signature(NextcloudBackend.list_folder)
wd_sig = inspect.signature(WebDAVBackend.list_folder)
nc_params = list(nc_sig.parameters.keys())
wd_params = list(wd_sig.parameters.keys())
assert nc_params == wd_params, (
f"NextcloudBackend.list_folder signature {nc_params!r} "
f"does not match WebDAVBackend.list_folder signature {wd_params!r}. "
f"Remove the incompatible override so Nextcloud inherits the canonical method."
)
@pytest.mark.asyncio
async def test_nextcloud_canonical_invocation_does_not_raise_type_error(self):
"""
Calling adapter.list_folder(connection_id, user_id) must not raise TypeError.
This is the canonical test for the P0 production defect. The existing override
raises TypeError because the browse endpoint passes positional UUID arguments
but the override expects (folder_path: str = "").
"""
adapter = _make_nextcloud_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
with patch("storage.webdav_backend.validate_cloud_url"):
adapter._client.list = MagicMock(return_value=["file.txt"])
adapter._client.info = MagicMock(return_value={"isdir": False, "size": 100})
# Must not raise TypeError — after Task 2 this invokes the inherited WebDAV method
result = await adapter.list_folder(conn_id, user_id)
from storage.cloud_base import CloudListing
assert isinstance(result, CloudListing)
def test_nextcloud_inherits_or_delegates_to_webdav_list_folder(self):
"""
NextcloudBackend.list_folder must be the same function as WebDAVBackend.list_folder
OR a method that wraps/delegates to it without changing the public contract.
"""
from storage.nextcloud_backend import NextcloudBackend
from storage.webdav_backend import WebDAVBackend
# After Task 2: either NextcloudBackend does not define its own list_folder
# (direct inheritance), or it defines one with the matching signature.
nc_fn = NextcloudBackend.__dict__.get("list_folder")
if nc_fn is not None:
# If Nextcloud overrides list_folder, its signature must still be canonical
sig = inspect.signature(nc_fn)
params = list(sig.parameters.keys())
# skip 'self'
expected = ["self", "connection_id", "user_id", "parent_ref", "page_token"]
assert params == expected, (
f"NextcloudBackend defines its own list_folder but signature {params!r} "
f"is not canonical {expected!r}. Either remove the override or fix the signature."
)
class TestGoogleDriveSpecificContract:
"""Drive-specific normalization tests."""
@pytest.mark.asyncio
async def test_native_google_files_have_nullable_size(self):
"""Native Google Workspace files (Docs, Sheets, etc.) must normalize with size=None."""
adapter = _make_google_drive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
fixture_data = _GD_FIXTURE["native_docs_nullable_size"]
fake_service = MagicMock()
fake_service.files().list().execute.return_value = fixture_data
adapter._get_service = MagicMock(return_value=fake_service)
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert result.complete is True
for item in result.items:
assert item.kind == "file", f"Native Google type should be kind=file, got {item.kind!r}"
assert item.size is None, f"Native Google file size must be None, got {item.size!r}"
@pytest.mark.asyncio
async def test_trashed_items_excluded(self):
"""trashed=false in the query means trashed items never appear in results."""
adapter = _make_google_drive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
# The fixture confirms query uses trashed=false — the result set only has visible items
fixture_data = _GD_FIXTURE["trashed_excluded"]
fake_service = MagicMock()
fake_service.files().list().execute.return_value = fixture_data
adapter._get_service = MagicMock(return_value=fake_service)
# Verify the query string passed to files().list() contains trashed=false
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
call_kwargs = fake_service.files().list.call_args
if call_kwargs:
q_param = call_kwargs[1].get("q", "") or (call_kwargs[0][0] if call_kwargs[0] else "")
assert "trashed=false" in q_param, (
"Drive query must contain 'trashed=false' to exclude trash items"
)
class TestOneDriveSpecificContract:
"""OneDrive-specific normalization tests."""
@pytest.mark.asyncio
async def test_folder_identified_by_facet(self):
"""OneDrive items with 'folder' key are kind='folder'."""
adapter = _make_onedrive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
page = _OD_FIXTURE["root_page1"].copy()
del page["@odata.nextLink"]
page["value"] = page["value"][:2] # first is folder, second is also folder
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = page
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
folders = [i for i in result.items if i.kind == "folder"]
assert len(folders) >= 1, "Expected at least one folder from OneDrive fixture"
@pytest.mark.asyncio
async def test_nullable_metadata_items(self):
"""OneDrive items with absent file.mimeType or size normalize to None."""
adapter = _make_onedrive_adapter()
conn_id = uuid.uuid4()
user_id = uuid.uuid4()
fixture_data = _OD_FIXTURE["nullable_metadata"]
mock_resp = MagicMock()
mock_resp.is_success = True
mock_resp.json.return_value = fixture_data
with patch("httpx.AsyncClient") as mock_cls:
mock_client = AsyncMock()
mock_client.__aenter__ = AsyncMock(return_value=mock_client)
mock_client.__aexit__ = AsyncMock(return_value=None)
mock_client.get = AsyncMock(return_value=mock_resp)
mock_cls.return_value = mock_client
result = await adapter.list_folder(conn_id, user_id, parent_ref=None)
assert len(result.items) == 1
item = result.items[0]
assert item.kind == "file"
# content_type may be None when 'file' key has no mimeType
assert item.content_type is None or isinstance(item.content_type, str)
+103
View File
@@ -186,3 +186,106 @@ class TestNextcloudBackendSSRF:
with pytest.raises(ValueError): with pytest.raises(ValueError):
NextcloudBackend("http://localhost/dav", "user", "pass") NextcloudBackend("http://localhost/dav", "user", "pass")
class TestNextcloudBackendNoListFolderOverride:
"""Phase 12.1 P0 fix: NextcloudBackend must NOT override list_folder."""
def test_nextcloud_does_not_define_own_list_folder(self):
"""list_folder must not appear in NextcloudBackend.__dict__ (no override)."""
from storage.nextcloud_backend import NextcloudBackend
assert "list_folder" not in NextcloudBackend.__dict__, (
"NextcloudBackend must not define its own list_folder override. "
"Remove the incompatible legacy method so Nextcloud inherits "
"the canonical WebDAVBackend.list_folder implementation."
)
def test_nextcloud_list_folder_is_same_as_webdav(self):
"""NextcloudBackend.list_folder must be the exact WebDAVBackend method."""
from storage.nextcloud_backend import NextcloudBackend
from storage.webdav_backend import WebDAVBackend
assert NextcloudBackend.list_folder is WebDAVBackend.list_folder, (
"NextcloudBackend.list_folder must be inherited from WebDAVBackend "
"— not a re-defined method."
)
def test_nextcloud_signature_matches_webdav(self):
"""Inspect signature — NextcloudBackend.list_folder must accept canonical args."""
import inspect
from storage.nextcloud_backend import NextcloudBackend
sig = inspect.signature(NextcloudBackend.list_folder)
params = list(sig.parameters.keys())
assert "connection_id" in params
assert "user_id" in params
assert "parent_ref" in params
assert "page_token" in params
class TestNextcloudUrlNormalization:
"""Tests for normalize_nextcloud_url helper in cloud_utils."""
def test_bare_origin_normalized(self):
"""Bare https origin normalized to DAV root path."""
from unittest.mock import patch
from storage.cloud_utils import normalize_nextcloud_url
with patch("storage.cloud_utils.validate_cloud_url"):
result = normalize_nextcloud_url("https://nc.example.com", "alice")
assert result == "https://nc.example.com/remote.php/dav/files/alice/"
def test_origin_with_trailing_slash(self):
"""Origin with trailing slash is idempotent in result."""
from unittest.mock import patch
from storage.cloud_utils import normalize_nextcloud_url
with patch("storage.cloud_utils.validate_cloud_url"):
result = normalize_nextcloud_url("https://nc.example.com/", "alice")
assert result == "https://nc.example.com/remote.php/dav/files/alice/"
def test_subpath_preserved(self):
"""Deployment subpath is preserved in canonical URL."""
from unittest.mock import patch
from storage.cloud_utils import normalize_nextcloud_url
with patch("storage.cloud_utils.validate_cloud_url"):
result = normalize_nextcloud_url("https://example.com/nextcloud", "bob")
assert result == "https://example.com/nextcloud/remote.php/dav/files/bob/"
def test_already_canonical_is_idempotent(self):
"""Already canonical URL is returned unchanged."""
from unittest.mock import patch
from storage.cloud_utils import normalize_nextcloud_url
canonical = "https://nc.example.com/remote.php/dav/files/alice/"
with patch("storage.cloud_utils.validate_cloud_url"):
result = normalize_nextcloud_url(canonical, "alice")
assert result == canonical
def test_username_percent_encoded(self):
"""Username with special characters is percent-encoded."""
from unittest.mock import patch
from storage.cloud_utils import normalize_nextcloud_url
with patch("storage.cloud_utils.validate_cloud_url"):
result = normalize_nextcloud_url("https://nc.example.com", "first last")
assert "first%20last" in result
assert " " not in result
def test_rejects_http_scheme(self):
"""http:// URL must raise ValueError (only https allowed for Nextcloud)."""
from storage.cloud_utils import normalize_nextcloud_url
with pytest.raises(ValueError, match="https"):
normalize_nextcloud_url("http://nc.example.com", "alice")
def test_rejects_userinfo(self):
"""URL with user:pass@ in it must raise ValueError."""
from storage.cloud_utils import normalize_nextcloud_url
with pytest.raises(ValueError, match="userinfo"):
normalize_nextcloud_url("https://user:pass@nc.example.com", "alice")
def test_rejects_query_string(self):
"""URL with ?query= must raise ValueError."""
from storage.cloud_utils import normalize_nextcloud_url
with pytest.raises(ValueError, match="query"):
normalize_nextcloud_url("https://nc.example.com?foo=bar", "alice")
def test_rejects_fragment(self):
"""URL with #fragment must raise ValueError."""
from storage.cloud_utils import normalize_nextcloud_url
with pytest.raises(ValueError, match="fragment"):
normalize_nextcloud_url("https://nc.example.com#section", "alice")
+1 -1
View File
@@ -1,6 +1,6 @@
{ {
"name": "document-scanner-frontend", "name": "document-scanner-frontend",
"version": "0.2.2", "version": "0.2.3",
"type": "module", "type": "module",
"scripts": { "scripts": {
"dev": "vite", "dev": "vite",