baa5bed7e2
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
288 lines
14 KiB
Markdown
288 lines
14 KiB
Markdown
---
|
|
phase: 05-cloud-storage-backends
|
|
plan: 01
|
|
type: execute
|
|
wave: 1
|
|
depends_on: []
|
|
files_modified:
|
|
- backend/requirements.txt
|
|
- backend/config.py
|
|
- backend/tests/test_cloud.py
|
|
- backend/tests/conftest.py
|
|
autonomous: true
|
|
requirements:
|
|
- CLOUD-01
|
|
- CLOUD-02
|
|
- CLOUD-03
|
|
- CLOUD-04
|
|
- CLOUD-05
|
|
- CLOUD-06
|
|
- CLOUD-07
|
|
|
|
must_haves:
|
|
truths:
|
|
- "All 15 Phase 5 test stubs exist in test_cloud.py and xfail with strict=False"
|
|
- "conftest.py has mock_google_drive_creds, mock_onedrive_creds, mock_webdav_client, cloud_connection_factory fixtures"
|
|
- "requirements.txt includes all 6 new packages with correct version pins"
|
|
- "config.py has CLOUD_CREDS_KEY, GOOGLE_CLIENT_ID/SECRET, ONEDRIVE_CLIENT_ID/SECRET/TENANT_ID, BACKEND_URL settings"
|
|
- "pytest -v passes with zero failures after Wave 0 (stubs xfail, not fail)"
|
|
artifacts:
|
|
- path: "backend/tests/test_cloud.py"
|
|
provides: "All Phase 5 xfail test stubs"
|
|
contains: "test_credential_round_trip"
|
|
- path: "backend/tests/conftest.py"
|
|
provides: "cloud_connection_factory and mock fixtures"
|
|
contains: "cloud_connection_factory"
|
|
- path: "backend/requirements.txt"
|
|
provides: "New package dependencies"
|
|
contains: "cryptography"
|
|
- path: "backend/config.py"
|
|
provides: "Phase 5 settings"
|
|
contains: "cloud_creds_key"
|
|
key_links:
|
|
- from: "backend/tests/test_cloud.py"
|
|
to: "backend/tests/conftest.py"
|
|
via: "fixture injection"
|
|
pattern: "cloud_connection_factory"
|
|
---
|
|
|
|
<objective>
|
|
Wave 0 Nyquist scaffold: create all Phase 5 test stubs, conftest fixtures, new package dependencies, and config settings before any implementation begins.
|
|
|
|
Purpose: Establish the Nyquist validation scaffolding so every subsequent plan has a test to turn green. Per-phase gate — all stubs must xfail (strict=False), never fail.
|
|
Output: test_cloud.py with 15 xfail stubs, updated conftest.py with 4 new fixtures, requirements.txt with 6 new packages, config.py with Phase 5 env vars.
|
|
</objective>
|
|
|
|
<execution_context>
|
|
@/Users/nik/.claude/get-shit-done/workflows/execute-plan.md
|
|
@/Users/nik/.claude/get-shit-done/templates/summary.md
|
|
</execution_context>
|
|
|
|
<context>
|
|
@.planning/PROJECT.md
|
|
@.planning/ROADMAP.md
|
|
@.planning/STATE.md
|
|
@.planning/phases/05-cloud-storage-backends/05-CONTEXT.md
|
|
@.planning/phases/05-cloud-storage-backends/05-VALIDATION.md
|
|
@.planning/phases/05-cloud-storage-backends/05-RESEARCH.md
|
|
</context>
|
|
|
|
<interfaces>
|
|
<!-- From backend/tests/conftest.py — existing fixture patterns -->
|
|
From backend/tests/conftest.py:
|
|
- db_session: async SQLite in-memory session fixture (pytest_asyncio.fixture)
|
|
- async_client: AsyncClient with db_session override (pytest_asyncio.fixture)
|
|
- live_services_available: session-scoped, checks ports 5432/9000/6379
|
|
|
|
From backend/db/models.py:
|
|
- CloudConnection: id (UUID), user_id (UUID FK), provider (String), display_name (Text),
|
|
credentials_enc (Text), status (String, default="ACTIVE"), connected_at (TIMESTAMP)
|
|
- User: id (UUID), handle (String), email (String), role (String), is_active (Boolean),
|
|
default_storage_backend (String, default="minio")
|
|
|
|
From backend/api/admin.py:
|
|
- CloudConnectionOut: id (str), provider (str), display_name (str), status (str),
|
|
connected_at (datetime) — credentials_enc absent by omission (SEC-08)
|
|
</interfaces>
|
|
|
|
<tasks>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 1: Add Phase 5 packages to requirements.txt and settings to config.py</name>
|
|
<files>backend/requirements.txt, backend/config.py</files>
|
|
<read_first>
|
|
- backend/requirements.txt — current packages and version format
|
|
- backend/config.py — SettingsConfigDict pattern, existing field declarations
|
|
</read_first>
|
|
<behavior>
|
|
- requirements.txt contains all 6 new packages with their exact version pins
|
|
- config.py Settings class has: cloud_creds_key (str, default "CHANGEME-32-bytes-padded!!"), google_client_id (str, default ""), google_client_secret (str, default ""), onedrive_client_id (str, default ""), onedrive_client_secret (str, default ""), onedrive_tenant_id (str, default "common"), backend_url (str, default "http://localhost:8000")
|
|
- All new settings have empty-string or safe defaults so the app boots without cloud credentials configured
|
|
</behavior>
|
|
<action>
|
|
Append to backend/requirements.txt (per D-02 research decisions, all versions VERIFIED on PyPI):
|
|
cryptography>=41.0.0, google-auth-oauthlib>=1.3.1, google-api-python-client>=2.196.0,
|
|
msal>=1.36.0, webdavclient3>=3.14.7, cachetools>=5.3.0.
|
|
|
|
In backend/config.py, inside the Settings class add a "# Cloud Storage (Phase 5)" comment block
|
|
followed by these fields:
|
|
- cloud_creds_key: str = "CHANGEME-32-bytes-padded!!" (CLOUD_CREDS_KEY env var — master key for HKDF)
|
|
- google_client_id: str = "" (GOOGLE_CLIENT_ID)
|
|
- google_client_secret: str = "" (GOOGLE_CLIENT_SECRET)
|
|
- onedrive_client_id: str = "" (ONEDRIVE_CLIENT_ID)
|
|
- onedrive_client_secret: str = "" (ONEDRIVE_CLIENT_SECRET)
|
|
- onedrive_tenant_id: str = "common" (ONEDRIVE_TENANT_ID — "common" works for personal + org accounts)
|
|
- backend_url: str = "http://localhost:8000" (BACKEND_URL — used to construct OAuth callback URLs)
|
|
|
|
.env.example should have the CLOUD_CREDS_KEY, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET,
|
|
ONEDRIVE_CLIENT_ID, ONEDRIVE_CLIENT_SECRET, ONEDRIVE_TENANT_ID, BACKEND_URL entries
|
|
(create .env.example if it doesn't exist, or append if it does).
|
|
</action>
|
|
<verify>
|
|
<automated>cd /Users/nik/Documents/Progamming/document_scanner/backend && grep -c "cryptography" requirements.txt && grep -c "google-auth-oauthlib" requirements.txt && grep -c "msal" requirements.txt && grep -c "webdavclient3" requirements.txt && grep -c "cachetools" requirements.txt && python -c "from config import settings; print(settings.cloud_creds_key)"</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- backend/requirements.txt contains lines matching: cryptography>=41.0.0, google-auth-oauthlib>=1.3.1, google-api-python-client>=2.196.0, msal>=1.36.0, webdavclient3>=3.14.7, cachetools>=5.3.0
|
|
- backend/config.py contains `cloud_creds_key: str` and `google_client_id: str` and `backend_url: str`
|
|
- `python -c "from config import settings; print(settings.cloud_creds_key)"` prints without ImportError
|
|
</acceptance_criteria>
|
|
<done>requirements.txt has all 6 Phase 5 package lines; config.py imports and Settings loads without error; all 7 new cloud settings accessible via settings.{field_name}</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 2: Create test_cloud.py with all 15 xfail stubs</name>
|
|
<files>backend/tests/test_cloud.py</files>
|
|
<read_first>
|
|
- backend/tests/test_folders.py — xfail stub pattern (pytest.mark.xfail(strict=False), single-line body)
|
|
- .planning/phases/05-cloud-storage-backends/05-VALIDATION.md — exact test names for all 15 stubs
|
|
- backend/db/models.py — CloudConnection model fields
|
|
</read_first>
|
|
<behavior>
|
|
- File exists at backend/tests/test_cloud.py
|
|
- Contains all 15 test stubs from the VALIDATION.md per-task verification map
|
|
- All stubs decorated with @pytest.mark.xfail(strict=False, reason="not implemented yet")
|
|
- Body of each stub is only: pytest.xfail("not implemented yet") — no assertions
|
|
- pytest tests/test_cloud.py -v exits 0 with all tests xfailed (not failed)
|
|
</behavior>
|
|
<action>
|
|
Create backend/tests/test_cloud.py with a module docstring and the following stubs, each
|
|
decorated with @pytest.mark.xfail(strict=False, reason="not implemented yet"):
|
|
|
|
From CLOUD-01:
|
|
- test_connect_google_drive
|
|
- test_oauth_callback_valid_state
|
|
- test_oauth_callback_invalid_state
|
|
- test_webdav_connect_validates
|
|
|
|
From CLOUD-02:
|
|
- test_credential_round_trip
|
|
- test_credentials_enc_not_exposed
|
|
|
|
From CLOUD-03:
|
|
- test_cloud_upload_no_presigned
|
|
|
|
From CLOUD-04:
|
|
- test_connection_status_display
|
|
|
|
From CLOUD-05:
|
|
- test_invalid_grant_sets_requires_reauth
|
|
|
|
From CLOUD-06:
|
|
- test_disconnect_deletes_credentials
|
|
|
|
From CLOUD-07:
|
|
- test_factory_returns_correct_backend
|
|
|
|
From D-17 SSRF:
|
|
- test_ssrf_validation (parametrized with @pytest.mark.parametrize over RFC-1918, loopback, link-local, valid URLs)
|
|
- test_ssrf_link_local
|
|
|
|
From SEC/IDOR:
|
|
- test_admin_cannot_see_credentials
|
|
- test_cross_user_idor
|
|
|
|
Import pytest at the top. Add `pytestmark = pytest.mark.asyncio` at module level.
|
|
Each test function takes no arguments for now (fixtures added in Task 3 when stubs
|
|
are promoted to real tests).
|
|
</action>
|
|
<verify>
|
|
<automated>cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_cloud.py -v 2>&1 | tail -5</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- backend/tests/test_cloud.py exists
|
|
- `pytest tests/test_cloud.py -v` exits with code 0 (xfailed is not failure)
|
|
- Output contains "xfailed" for all 15 stubs
|
|
- No test shows status "FAILED" or "ERROR"
|
|
- test_ssrf_validation is present (parametrize decorator may be skipped until implementation — single stub is fine)
|
|
</acceptance_criteria>
|
|
<done>pytest tests/test_cloud.py exits 0; all 15 stubs xfailed; no failures or collection errors</done>
|
|
</task>
|
|
|
|
<task type="auto" tdd="true">
|
|
<name>Task 3: Add cloud fixtures to conftest.py</name>
|
|
<files>backend/tests/conftest.py</files>
|
|
<read_first>
|
|
- backend/tests/conftest.py — existing fixture patterns (db_session, async_client, live_services_available)
|
|
- backend/db/models.py — CloudConnection fields (id, user_id, provider, display_name, credentials_enc, status, connected_at)
|
|
</read_first>
|
|
<behavior>
|
|
- conftest.py gains 4 new fixtures without breaking any existing fixtures
|
|
- mock_google_drive_creds returns a dict with keys: access_token, refresh_token, expiry, token_uri, client_id, client_secret
|
|
- mock_onedrive_creds returns a dict with keys: access_token, refresh_token, expires_at
|
|
- mock_webdav_client is a MagicMock with upload_to, download_from, list, check methods mocked (no real connection)
|
|
- cloud_connection_factory is a callable fixture factory that creates CloudConnection ORM rows in the db_session for arbitrary provider/status values
|
|
- pytest -v with the existing test suite exits 0 after adding these fixtures
|
|
</behavior>
|
|
<action>
|
|
Append to backend/tests/conftest.py (after the existing fixtures, do NOT modify any existing code):
|
|
|
|
1. mock_google_drive_creds fixture (scope="function"): returns a dict
|
|
{"access_token": "ya29.test_access", "refresh_token": "1//test_refresh",
|
|
"expiry": "2099-12-31T23:59:59", "token_uri": "https://oauth2.googleapis.com/token",
|
|
"client_id": "test_client_id", "client_secret": "test_client_secret"}
|
|
|
|
2. mock_onedrive_creds fixture (scope="function"): returns a dict
|
|
{"access_token": "test_ms_access", "refresh_token": "test_ms_refresh",
|
|
"expires_at": "2099-12-31T23:59:59"}
|
|
|
|
3. mock_webdav_client fixture (scope="function"): returns a MagicMock with
|
|
.upload_to, .download_from, .list, .check all set to MagicMock(return_value=None).
|
|
Import MagicMock from unittest.mock.
|
|
|
|
4. cloud_connection_factory fixture (scope="function"): a factory function that
|
|
accepts (session, user_id, provider="google_drive", status="ACTIVE",
|
|
display_name=None, credentials_enc="fake_encrypted_creds") and creates +
|
|
flushes a CloudConnection row. Returns the CloudConnection instance.
|
|
The factory accepts db_session as a pytest fixture dependency via the fixture
|
|
mechanism (fixture returning an inner async function that takes session as first arg).
|
|
Use pytest_asyncio.fixture decorator. The factory's inner function should be async.
|
|
</action>
|
|
<verify>
|
|
<automated>cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest -v --co -q 2>&1 | grep "ERROR" | wc -l</automated>
|
|
</verify>
|
|
<acceptance_criteria>
|
|
- backend/tests/conftest.py contains `def cloud_connection_factory`
|
|
- backend/tests/conftest.py contains `def mock_google_drive_creds`
|
|
- backend/tests/conftest.py contains `def mock_onedrive_creds`
|
|
- backend/tests/conftest.py contains `def mock_webdav_client`
|
|
- `pytest -v --co -q` collection phase produces 0 ERROR lines
|
|
- `pytest tests/test_cloud.py -v` still exits 0 with all stubs xfailed
|
|
</acceptance_criteria>
|
|
<done>4 new fixtures in conftest.py; collection error count = 0; existing test suite unaffected</done>
|
|
</task>
|
|
|
|
</tasks>
|
|
|
|
<threat_model>
|
|
## Trust Boundaries
|
|
|
|
| Boundary | Description |
|
|
|----------|-------------|
|
|
| test code → production code | Tests import production modules; config loading must not fail when cloud creds are absent |
|
|
| requirements.txt → PyPI | Package names and versions must match PyPI exactly; wrong names install typosquats |
|
|
|
|
## STRIDE Threat Register
|
|
|
|
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
|
|-----------|----------|-----------|-------------|-----------------|
|
|
| T-05-01-01 | Tampering | requirements.txt package names | mitigate | All 6 packages verified via slopcheck [OK] in RESEARCH.md Package Legitimacy Audit — no [SLOP] or [SUS] verdict |
|
|
| T-05-01-02 | Information Disclosure | config.py cloud_creds_key default | mitigate | Default is "CHANGEME-32-bytes-padded!!" — clearly a placeholder; production requires env var override; Settings raises no error but the HKDF in cloud_utils will produce useless keys without real master key |
|
|
| T-05-01-SC | Tampering | npm/pip/cargo installs | mitigate | All 6 new packages are [OK] per RESEARCH.md slopcheck audit; no blocking human checkpoint required |
|
|
</threat_model>
|
|
|
|
<verification>
|
|
cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_cloud.py -v && python -m pytest -v --tb=short 2>&1 | tail -10
|
|
</verification>
|
|
|
|
<success_criteria>
|
|
- pytest tests/test_cloud.py exits 0; all 15 stubs show xfailed
|
|
- pytest -v (full suite) exits 0 with zero failures
|
|
- requirements.txt contains all 6 new package lines
|
|
- config.py Settings loads without error; cloud_creds_key, google_client_id, backend_url all accessible
|
|
- conftest.py has 4 new fixtures: mock_google_drive_creds, mock_onedrive_creds, mock_webdav_client, cloud_connection_factory
|
|
</success_criteria>
|
|
|
|
<output>
|
|
Create `.planning/phases/05-cloud-storage-backends/05-01-SUMMARY.md` when done
|
|
</output>
|