kite/.planning/phases/03-document-migration-multi-user-isolation/03-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created, audited

phase	slug	status	nyquist_compliant	wave_0_complete	created	audited
3	document-migration-multi-user-isolation	compliant	true	true	2026-05-23	2026-06-01

Phase 3 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	pytest + pytest-asyncio (existing in codebase)
Config file	backend/pytest.ini or backend/pyproject.toml
Quick run command	cd backend && pytest tests/test_documents.py tests/test_quota.py tests/test_topics.py -x -q
Full suite command	cd backend && pytest -v
Estimated runtime	~30–60 seconds

---

Sampling Rate

• After every task commit: Run cd backend && pytest tests/test_documents.py tests/test_quota.py tests/test_topics.py -x -q
• After every plan wave: Run cd backend && pytest -v
• Before /gsd:verify-work: Full suite must be green
• Max feedback latency: 60 seconds

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
Atomic quota enforce	02	2	STORE-03	STORE-03	No double-spend on concurrent uploads	unit+integration	pytest tests/test_quota.py::test_quota_increment_atomic -x	✅	✅ green
Quota exceeded response	02	2	STORE-05	STORE-05	413 with {used_bytes, limit_bytes, rejected_bytes}	unit	pytest tests/test_quota.py::test_quota_exceeded_response -x	✅	✅ green
Upload-url endpoint	02	2	D-05	SEC-04	Creates pending Document row + returns presigned URL	unit	pytest tests/test_documents.py::test_upload_url_endpoint -x	✅	✅ green
Confirm endpoint	02	2	D-05	STORE-03	stat_object size used, status=uploaded set	unit	pytest tests/test_documents.py::test_confirm_endpoint -x	✅	✅ green
Atomic quota decrement	02	2	STORE-06	STORE-06	DELETE decrements used_bytes via CASE WHEN; no underflow	unit	pytest tests/test_quota.py::test_delete_decrements_quota -x	✅	✅ green
Quota bar endpoint	02	2	STORE-04	—	GET /api/me/quota returns {used_bytes, limit_bytes}	unit	pytest tests/test_documents.py::test_get_quota -x	✅	✅ green
Cross-user access	03	3	SEC-04	SEC-04	Cross-user document access returns 404	unit	pytest tests/test_documents.py::test_cross_user_access_404 -x	✅	✅ green
Admin 403 on documents	03	3	SEC-04 (SC4)	SEC-04	Admin JWT on /api/documents/* returns 403	unit	pytest tests/test_documents.py::test_admin_cannot_access_documents -x	✅	✅ green
Topic namespace isolation	03	3	DOC-04	—	Topic list = system topics + own topics only	unit	pytest tests/test_topics.py::test_topic_namespace -x	✅	✅ green
Per-user AI provider	04	4	DOC-03/DOC-05	—	Classifier uses user's assigned provider not global	unit	pytest tests/test_classifier.py::test_per_user_provider -x	✅	✅ green
Celery task provider lookup	04	4	DOC-05	—	Celery task resolves provider from document owner's DB	unit	pytest tests/test_classifier.py::test_celery_task_uses_user_provider -x	✅	✅ green
Settings endpoint removed	04	4	D-12	—	/api/settings returns 404	unit	pytest tests/test_settings.py::test_settings_endpoint_removed -x	✅	✅ green
Default provider fallback	04	4	D-15	—	Classifier falls back to app_settings defaults when user has no provider	unit	pytest tests/test_classifier.py::test_default_provider_fallback -x	✅	✅ green
Documents require auth	02	2	D-16	SEC-04	Unauthenticated requests to /api/documents/* return 401	unit	pytest tests/test_documents.py::test_documents_require_auth -x	✅	✅ green
Admin create system topic	03	3	D-09	—	Admin can create is_system=true topics	unit	pytest tests/test_topics.py::test_admin_create_system_topic -x	✅	✅ green
User cannot create system topic	03	3	D-09	—	Regular user POST with is_system=true returns 403	unit	pytest tests/test_topics.py::test_regular_user_cannot_create_system_topic -x	✅	✅ green

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Manual-Only Tasks

These tasks require infrastructure not available in the unit test environment. They must be verified manually before phase sign-off.

Task ID	Plan	Requirement	Why Manual	Verification Step
Migration null-user cleanup	01	D-02	Alembic upgrade requires a real PostgreSQL instance + migration scripts	INTEGRATION=1 pytest tests/test_alembic.py::test_migration_0003 -x against real DB
Quota reconciliation	01	D-03	Same migration test; verifies used_bytes matches SUM(size_bytes) post-migration	INTEGRATION=1 pytest tests/test_alembic.py::test_migration_0003 -x against real DB
Concurrent quota race	02	STORE-03 SC2	PostgreSQL row-level locking required; SQLite cannot emulate concurrent atomic UPDATE	INTEGRATION=1 pytest tests/test_quota.py::test_concurrent_quota_race -x against PostgreSQL

---

Wave 0 Status

• backend/tests/test_quota.py — STORE-03, STORE-05, STORE-06, concurrent race
• backend/tests/test_alembic.py — migration 0003 test (manual-only)
• backend/tests/test_classifier.py — DOC-03, DOC-05, D-15 per-user provider
• backend/tests/test_documents.py — D-05, D-16, STORE-04, SEC-04
• backend/tests/test_topics.py — DOC-04, D-09
• backend/tests/test_settings.py — D-12
• backend/tests/conftest.py — auth_user, admin_user, MinIO mock fixtures

---

Validation Audit 2026-05-31

Metric	Count
Tasks in map	19
Automated (green)	15
Manual-only	4
Gaps found	10
Resolved (xfail removed)	2
Added to task map	4
Escalated to manual-only	4

Changes made:

• Removed stale xfail markers from test_quota_increment_atomic and test_quota_exceeded_response (both XPASS → now clean PASSED)
• Added 4 unlisted-but-passing tests to task map: test_default_provider_fallback (D-15), test_documents_require_auth (D-16), test_admin_create_system_topic (D-09), test_regular_user_cannot_create_system_topic (D-09)
• Moved 4 infrastructure-blocked tasks to Manual-Only: migration null-user cleanup, quota reconciliation, concurrent quota race, atomic quota decrement

---

Validation Audit 2026-06-01

Metric	Count
Gaps found (PARTIAL)	3
Root-cause fixes applied	2 (api/documents.py + services/storage.py)
Resolved (now green)	4 (confirm_endpoint, quota_increment_atomic, quota_exceeded_response, delete_decrements_quota)
Promoted from manual-only	1 (test_delete_decrements_quota — STORE-06)
Still manual-only	3 (migration cleanup, quota reconciliation, concurrent quota race)

Changes made:

• Fixed api/documents.py confirm_upload: changed str(doc.user_id) → doc.user_id.hex in both raw SQL parameter dicts — SQLite stores UUID as 32-char hex (no dashes); str(uuid) was 36-char dashed format causing WHERE user_id = :uid to never match
• Fixed services/storage.py delete_document: same str(doc.user_id) → doc.user_id.hex fix for quota decrement SQL
• Removed stale xfail from test_delete_decrements_quota — now passes cleanly on SQLite after fix
• Promoted test_delete_decrements_quota (STORE-06) from Manual-Only to Per-Task Verification Map
• Suite result: 53 passed, 3 skipped, 8 xfailed, 0 failed