fix(05): revise Phase 5 plans based on checker feedback — B1-B4, W1-W4

B1: Mark RESEARCH.md Open Questions as (RESOLVED) with decision text for all 3
B2: Backends now stateless — raise CloudConnectionError(reason=) only; API layer
    in cloud.py owns token refresh + DB update via _call_cloud_op helper
B3: Add Task 3 to Plan 05 — cloud connection + object cleanup on account deletion (SEC-09)
B4: Add frontend_url setting to Plan 01 Task 1; Plan 05 uses settings.frontend_url
    for OAuth callback redirects
W1: ROADMAP.md Phase 5 now correctly labels Plans 03+04 as Wave 3 (not Wave 2)
W2: Plan 06 invalid_grant test now asserts both 503 HTTP response AND DB REQUIRES_REAUTH
W3: Plan 06 Task 2 split into unit tests (4, cloud_utils.py) and integration tests (11, HTTP)
W4: Plan 07 adds Vitest tests for cloudConnections store (4 tests) and SettingsCloudTab
    mount test (2 tests) per CLAUDE.md testing protocol

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This commit is contained in:
curo1305
2026-05-28 19:55:28 +02:00
parent baa5bed7e2
commit d13801538d
7 changed files with 328 additions and 59 deletions
@@ -9,6 +9,7 @@ depends_on:
files_modified:
- backend/api/cloud.py
- backend/main.py
- backend/api/auth.py
autonomous: true
requirements:
- CLOUD-01
@@ -17,11 +18,12 @@ requirements:
- CLOUD-04
- CLOUD-05
- CLOUD-06
- SEC-09
must_haves:
truths:
- "GET /api/cloud/oauth/initiate/{provider} redirects to provider OAuth URL; state token in Redis with 30-min TTL"
- "GET /api/cloud/oauth/callback/{provider} validates state, exchanges code, encrypts credentials, saves CloudConnection, redirects to /settings?cloud_connected={provider}"
- "GET /api/cloud/oauth/callback/{provider} validates state, exchanges code, encrypts credentials, saves CloudConnection, redirects to {settings.frontend_url}/settings?cloud_connected={provider}"
- "POST /api/cloud/connections/webdav validates URL (SSRF), tests connection (PROPFIND), encrypts + saves credentials"
- "GET /api/cloud/connections returns CloudConnectionOut list — no credentials_enc"
- "DELETE /api/cloud/connections/{id} deletes credentials_enc row; subsequent use returns 503"
@@ -30,6 +32,8 @@ must_haves:
- "All endpoints use get_regular_user dep — admin blocked (403)"
- "OAuth callback invalid state returns 400; invalid provider returns 400"
- "write_audit_log called on connect, disconnect, and REQUIRES_REAUTH transitions"
- "_call_cloud_op(conn, user, session, op_fn) helper in cloud.py wraps all cloud ops: retries once on token_expired (refresh+DB update), sets REQUIRES_REAUTH+HTTPException(503) on invalid_grant"
- "Account deletion purges all CloudConnection rows and calls delete_object on cloud-stored documents (SEC-09)"
artifacts:
- path: "backend/api/cloud.py"
provides: "All /api/cloud/* endpoints + /api/users/me/default-storage"
@@ -101,6 +105,7 @@ From backend/config.py (after Plan 01):
settings.google_client_id, google_client_secret: str
settings.onedrive_client_id, onedrive_client_secret, onedrive_tenant_id: str
settings.backend_url: str (used in OAuth callback redirect_uri)
settings.frontend_url: str (used in OAuth callback success/error redirect to Vue — per B4 fix)
From backend/storage/cloud_utils.py:
def encrypt_credentials(master_key: bytes, user_id: str, credentials: dict) -> str
@@ -160,6 +165,22 @@ From backend/services/cloud_cache.py: get_cloud_folders_cached(user_id, provider
router = APIRouter(prefix="/api/cloud", tags=["cloud"])
users_router = APIRouter(prefix="/api/users", tags=["users"])
_call_cloud_op helper (add as a module-level async function in cloud.py, per B2 design):
async def _call_cloud_op(conn: CloudConnection, user: User, session: AsyncSession, op_fn):
"""Wraps a cloud operation with transparent token refresh (D-05) and invalid_grant handling (D-06).
1. Calls op_fn() — a zero-argument async callable that performs the cloud operation.
2. On CloudConnectionError(reason="token_expired"): decrypt current creds, refresh via provider,
encrypt new creds, update conn.credentials_enc in DB, rebuild backend, retry op_fn() once.
3. On CloudConnectionError(reason="invalid_grant"): set conn.status="REQUIRES_REAUTH",
await session.commit(), call write_audit_log(event_type="cloud.requires_reauth"),
raise HTTPException(503, "Cloud connection requires re-authentication. Please reconnect in Settings.").
4. Propagates all other exceptions unchanged.
"""
All upload/download/list calls in cloud.py MUST go through _call_cloud_op.
op_fn is a zero-argument async lambda that already has the backend instance captured in closure.
The backend instance is rebuilt after refresh using the new credentials dict.
Pydantic request models:
class WebDAVConnectRequest(BaseModel): server_url: str; username: str; password: str; provider: str
class DefaultStorageRequest(BaseModel): backend: str
@@ -273,6 +294,84 @@ assert len(cloud_routes) >= 5, f'Expected 5+ cloud routes, got {len(cloud_routes
<done>Both cloud routers registered in main.py; all cloud routes visible in app.routes; full pytest suite passes</done>
</task>
<task type="auto" tdd="true">
<name>Task 3: Cloud connection cleanup on account deletion (SEC-09)</name>
<files>backend/api/auth.py</files>
<read_first>
- backend/api/auth.py — find the DELETE /api/users/me endpoint (account self-deletion), verify it exists from Phase 2; if it does not exist, check backend/api/admin.py for DELETE /api/admin/users/{id}
- backend/db/models.py — CloudConnection (user_id, provider, status), Document (user_id, storage_backend, object_key)
- backend/storage/__init__.py — get_storage_backend_for_document signature
</read_first>
<behavior>
- When a user deletes their account (DELETE /api/users/me or admin DELETE /api/admin/users/{id}):
1. Query all CloudConnection rows for the user
2. For each connection, query all Document rows for that user where storage_backend == connection.provider
3. For each such document, call get_storage_backend_for_document(doc, user, session) and await backend.delete_object(doc.object_key) — catch and log exceptions but do NOT abort the deletion
4. Delete all CloudConnection rows for the user (credentials_enc purged)
- This runs BEFORE the user row is deleted (FK cascade would remove connections anyway, but credentials must be actively purged from the cloud provider)
- Runs in the same DB transaction as user deletion — if user deletion succeeds, cloud cleanup has completed
- No orphaned credentials_enc rows after account deletion (SEC-09)
</behavior>
<action>
Read backend/api/auth.py to locate the account deletion endpoint. Also check backend/api/admin.py for admin-initiated user deletion.
In the account deletion handler (DELETE /api/users/me), add a cloud cleanup block BEFORE the user row deletion:
1. Import at top of file (if not already present):
from db.models import CloudConnection, Document
from storage import get_storage_backend_for_document
from sqlalchemy import select
2. Cloud cleanup block (insert before the DELETE user statement):
cloud_conns_result = await session.execute(
select(CloudConnection).where(CloudConnection.user_id == current_user.id)
)
cloud_conns = cloud_conns_result.scalars().all()
for conn in cloud_conns:
# Delete cloud objects for this provider
docs_result = await session.execute(
select(Document).where(
Document.user_id == current_user.id,
Document.storage_backend == conn.provider,
)
)
for doc in docs_result.scalars().all():
try:
backend = await get_storage_backend_for_document(doc, current_user, session)
await backend.delete_object(doc.object_key)
except Exception:
pass # Do not abort user deletion on cloud error
await session.delete(conn)
await session.flush() # Flush connection deletes before user delete
If DELETE /api/users/me does not exist in auth.py, check admin.py for the admin-delete endpoint and add the same cleanup block there. Document which file was modified in the summary.
write_audit_log call: add event_type="cloud.credentials_purged" after the cleanup loop,
with metadata_={"providers": [c.provider for c in cloud_conns]}.
</action>
<verify>
<automated>cd /Users/nik/Documents/Progamming/document_scanner/backend && python -c "
import ast
import os
for fname in ['api/auth.py', 'api/admin.py']:
if os.path.exists(fname):
with open(fname) as f:
src = f.read()
if 'cloud_conns' in src or 'CloudConnection' in src:
print(f'OK: cloud cleanup found in {fname}')
" && python -m pytest -v --tb=short 2>&1 | tail -5</automated>
</verify>
<acceptance_criteria>
- Either backend/api/auth.py or backend/api/admin.py contains cloud connection cleanup logic before user deletion
- CloudConnection rows are deleted for the user as part of account deletion
- delete_object called for each cloud-stored document before credentials are purged
- write_audit_log called with event_type="cloud.credentials_purged"
- pytest -v exits 0 with 0 failures
- No orphaned credentials_enc rows after account deletion (SEC-09)
</acceptance_criteria>
<done>Cloud connection cleanup wired into account deletion; credentials_enc purged; SEC-09 satisfied</done>
</task>
</tasks>
<threat_model>
@@ -308,6 +407,7 @@ cd /Users/nik/Documents/Progamming/document_scanner/backend && python -m pytest
- main.py: both routers registered; all routes visible in app.routes
- pytest -v exits 0, 0 failures
- test_cloud.py stubs transition from xfail to green for test_credentials_enc_not_exposed, test_connection_status_display, test_disconnect_deletes_credentials, test_ssrf_validation, test_cross_user_idor, test_admin_cannot_see_credentials
- SEC-09: account deletion endpoint purges CloudConnection rows and cloud-stored document objects before deleting user row
</success_criteria>
<output>