d13801538d
B1: Mark RESEARCH.md Open Questions as (RESOLVED) with decision text for all 3
B2: Backends now stateless — raise CloudConnectionError(reason=) only; API layer
in cloud.py owns token refresh + DB update via _call_cloud_op helper
B3: Add Task 3 to Plan 05 — cloud connection + object cleanup on account deletion (SEC-09)
B4: Add frontend_url setting to Plan 01 Task 1; Plan 05 uses settings.frontend_url
for OAuth callback redirects
W1: ROADMAP.md Phase 5 now correctly labels Plans 03+04 as Wave 3 (not Wave 2)
W2: Plan 06 invalid_grant test now asserts both 503 HTTP response AND DB REQUIRES_REAUTH
W3: Plan 06 Task 2 split into unit tests (4, cloud_utils.py) and integration tests (11, HTTP)
W4: Plan 07 adds Vitest tests for cloudConnections store (4 tests) and SettingsCloudTab
mount test (2 tests) per CLAUDE.md testing protocol
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
14 KiB
14 KiB
phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves
| phase | plan | type | wave | depends_on | files_modified | autonomous | requirements | must_haves | |||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 05-cloud-storage-backends | 01 | execute | 1 |
|
true |
|
|
Purpose: Establish the Nyquist validation scaffolding so every subsequent plan has a test to turn green. Per-phase gate — all stubs must xfail (strict=False), never fail. Output: test_cloud.py with 15 xfail stubs, updated conftest.py with 4 new fixtures, requirements.txt with 6 new packages, config.py with Phase 5 env vars.
<execution_context> @/Users/nik/.claude/get-shit-done/workflows/execute-plan.md @/Users/nik/.claude/get-shit-done/templates/summary.md </execution_context>
@.planning/PROJECT.md @.planning/ROADMAP.md @.planning/STATE.md @.planning/phases/05-cloud-storage-backends/05-CONTEXT.md @.planning/phases/05-cloud-storage-backends/05-VALIDATION.md @.planning/phases/05-cloud-storage-backends/05-RESEARCH.md From backend/tests/conftest.py: - db_session: async SQLite in-memory session fixture (pytest_asyncio.fixture) - async_client: AsyncClient with db_session override (pytest_asyncio.fixture) - live_services_available: session-scoped, checks ports 5432/9000/6379From backend/db/models.py:
- CloudConnection: id (UUID), user_id (UUID FK), provider (String), display_name (Text), credentials_enc (Text), status (String, default="ACTIVE"), connected_at (TIMESTAMP)
- User: id (UUID), handle (String), email (String), role (String), is_active (Boolean), default_storage_backend (String, default="minio")
From backend/api/admin.py:
- CloudConnectionOut: id (str), provider (str), display_name (str), status (str), connected_at (datetime) — credentials_enc absent by omission (SEC-08)
In backend/config.py, inside the Settings class add a "# Cloud Storage (Phase 5)" comment block
followed by these fields:
- cloud_creds_key: str = "CHANGEME-32-bytes-padded!!" (CLOUD_CREDS_KEY env var — master key for HKDF)
- google_client_id: str = "" (GOOGLE_CLIENT_ID)
- google_client_secret: str = "" (GOOGLE_CLIENT_SECRET)
- onedrive_client_id: str = "" (ONEDRIVE_CLIENT_ID)
- onedrive_client_secret: str = "" (ONEDRIVE_CLIENT_SECRET)
- onedrive_tenant_id: str = "common" (ONEDRIVE_TENANT_ID — "common" works for personal + org accounts)
- backend_url: str = "http://localhost:8000" (BACKEND_URL — used to construct OAuth callback URLs)
- frontend_url: str = "http://localhost:5173" (FRONTEND_URL — used to construct OAuth success/error redirect to Vue app; per B4 fix)
.env.example should have the CLOUD_CREDS_KEY, GOOGLE_CLIENT_ID, GOOGLE_CLIENT_SECRET,
ONEDRIVE_CLIENT_ID, ONEDRIVE_CLIENT_SECRET, ONEDRIVE_TENANT_ID, BACKEND_URL, FRONTEND_URL entries
(create .env.example if it doesn't exist, or append if it does).
cd /Users/nik/Documents/Progamming/document_scanner/backend && grep -c "cryptography" requirements.txt && grep -c "google-auth-oauthlib" requirements.txt && grep -c "msal" requirements.txt && grep -c "webdavclient3" requirements.txt && grep -c "cachetools" requirements.txt && python -c "from config import settings; print(settings.cloud_creds_key)"
- backend/requirements.txt contains lines matching: cryptography>=41.0.0, google-auth-oauthlib>=1.3.1, google-api-python-client>=2.196.0, msal>=1.36.0, webdavclient3>=3.14.7, cachetools>=5.3.0
- backend/config.py contains `cloud_creds_key: str` and `google_client_id: str` and `backend_url: str` and `frontend_url: str`
- `python -c "from config import settings; print(settings.cloud_creds_key)"` prints without ImportError
requirements.txt has all 6 Phase 5 package lines; config.py imports and Settings loads without error; all 8 new cloud settings accessible via settings.{field_name}
Task 2: Create test_cloud.py with all 15 xfail stubs
backend/tests/test_cloud.py
- backend/tests/test_folders.py — xfail stub pattern (pytest.mark.xfail(strict=False), single-line body)
- .planning/phases/05-cloud-storage-backends/05-VALIDATION.md — exact test names for all 15 stubs
- backend/db/models.py — CloudConnection model fields
- File exists at backend/tests/test_cloud.py
- Contains all 15 test stubs from the VALIDATION.md per-task verification map
- All stubs decorated with @pytest.mark.xfail(strict=False, reason="not implemented yet")
- Body of each stub is only: pytest.xfail("not implemented yet") — no assertions
- pytest tests/test_cloud.py -v exits 0 with all tests xfailed (not failed)
Create backend/tests/test_cloud.py with a module docstring and the following stubs, each
decorated with @pytest.mark.xfail(strict=False, reason="not implemented yet"):
From CLOUD-01:
- test_connect_google_drive
- test_oauth_callback_valid_state
- test_oauth_callback_invalid_state
- test_webdav_connect_validates
From CLOUD-02:
- test_credential_round_trip
- test_credentials_enc_not_exposed
From CLOUD-03:
- test_cloud_upload_no_presigned
From CLOUD-04:
- test_connection_status_display
From CLOUD-05:
- test_invalid_grant_sets_requires_reauth
From CLOUD-06:
- test_disconnect_deletes_credentials
From CLOUD-07:
- test_factory_returns_correct_backend
From D-17 SSRF:
- test_ssrf_validation (parametrized with @pytest.mark.parametrize over RFC-1918, loopback, link-local, valid URLs)
- test_ssrf_link_local
From SEC/IDOR:
- test_admin_cannot_see_credentials
- test_cross_user_idor
Import pytest at the top. Add `pytestmark = pytest.mark.asyncio` at module level.
Each test function takes no arguments for now (fixtures added in Task 3 when stubs
are promoted to real tests).
cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest tests/test_cloud.py -v 2>&1 | tail -5
- backend/tests/test_cloud.py exists
- `pytest tests/test_cloud.py -v` exits with code 0 (xfailed is not failure)
- Output contains "xfailed" for all 15 stubs
- No test shows status "FAILED" or "ERROR"
- test_ssrf_validation is present (parametrize decorator may be skipped until implementation — single stub is fine)
pytest tests/test_cloud.py exits 0; all 15 stubs xfailed; no failures or collection errors
Task 3: Add cloud fixtures to conftest.py
backend/tests/conftest.py
- backend/tests/conftest.py — existing fixture patterns (db_session, async_client, live_services_available)
- backend/db/models.py — CloudConnection fields (id, user_id, provider, display_name, credentials_enc, status, connected_at)
- conftest.py gains 4 new fixtures without breaking any existing fixtures
- mock_google_drive_creds returns a dict with keys: access_token, refresh_token, expiry, token_uri, client_id, client_secret
- mock_onedrive_creds returns a dict with keys: access_token, refresh_token, expires_at
- mock_webdav_client is a MagicMock with upload_to, download_from, list, check methods mocked (no real connection)
- cloud_connection_factory is a callable fixture factory that creates CloudConnection ORM rows in the db_session for arbitrary provider/status values
- pytest -v with the existing test suite exits 0 after adding these fixtures
Append to backend/tests/conftest.py (after the existing fixtures, do NOT modify any existing code):
1. mock_google_drive_creds fixture (scope="function"): returns a dict
{"access_token": "ya29.test_access", "refresh_token": "1//test_refresh",
"expiry": "2099-12-31T23:59:59", "token_uri": "https://oauth2.googleapis.com/token",
"client_id": "test_client_id", "client_secret": "test_client_secret"}
2. mock_onedrive_creds fixture (scope="function"): returns a dict
{"access_token": "test_ms_access", "refresh_token": "test_ms_refresh",
"expires_at": "2099-12-31T23:59:59"}
3. mock_webdav_client fixture (scope="function"): returns a MagicMock with
.upload_to, .download_from, .list, .check all set to MagicMock(return_value=None).
Import MagicMock from unittest.mock.
4. cloud_connection_factory fixture (scope="function"): a factory function that
accepts (session, user_id, provider="google_drive", status="ACTIVE",
display_name=None, credentials_enc="fake_encrypted_creds") and creates +
flushes a CloudConnection row. Returns the CloudConnection instance.
The factory accepts db_session as a pytest fixture dependency via the fixture
mechanism (fixture returning an inner async function that takes session as first arg).
Use pytest_asyncio.fixture decorator. The factory's inner function should be async.
cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest -v --co -q 2>&1 | grep "ERROR" | wc -l
- backend/tests/conftest.py contains `def cloud_connection_factory`
- backend/tests/conftest.py contains `def mock_google_drive_creds`
- backend/tests/conftest.py contains `def mock_onedrive_creds`
- backend/tests/conftest.py contains `def mock_webdav_client`
- `pytest -v --co -q` collection phase produces 0 ERROR lines
- `pytest tests/test_cloud.py -v` still exits 0 with all stubs xfailed
4 new fixtures in conftest.py; collection error count = 0; existing test suite unaffected
<threat_model>
Trust Boundaries
| Boundary | Description |
|---|---|
| test code → production code | Tests import production modules; config loading must not fail when cloud creds are absent |
| requirements.txt → PyPI | Package names and versions must match PyPI exactly; wrong names install typosquats |
STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|---|---|---|---|---|
| T-05-01-01 | Tampering | requirements.txt package names | mitigate | All 6 packages verified via slopcheck [OK] in RESEARCH.md Package Legitimacy Audit — no [SLOP] or [SUS] verdict |
| T-05-01-02 | Information Disclosure | config.py cloud_creds_key default | mitigate | Default is "CHANGEME-32-bytes-padded!!" — clearly a placeholder; production requires env var override; Settings raises no error but the HKDF in cloud_utils will produce useless keys without real master key |
| T-05-01-SC | Tampering | npm/pip/cargo installs | mitigate | All 6 new packages are [OK] per RESEARCH.md slopcheck audit; no blocking human checkpoint required |
| </threat_model> |
<success_criteria>
- pytest tests/test_cloud.py exits 0; all 15 stubs show xfailed
- pytest -v (full suite) exits 0 with zero failures
- requirements.txt contains all 6 new package lines
- config.py Settings loads without error; cloud_creds_key, google_client_id, backend_url, frontend_url all accessible
- conftest.py has 4 new fixtures: mock_google_drive_creds, mock_onedrive_creds, mock_webdav_client, cloud_connection_factory </success_criteria>