kite/.planning/phases/06-performance-production-hardening/06-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created, audited

phase	slug	status	nyquist_compliant	wave_0_complete	created	audited
6	performance-production-hardening	audited	true	true	2026-06-02	2026-06-05

Phase 6 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	pytest 8.2 + pytest-asyncio (asyncio_mode=auto)
Config file	backend/pytest.ini
Quick run command	cd backend && pytest tests/ -v -x --tb=short
Full suite command	cd backend && pytest tests/ -v
Estimated runtime	~45 seconds

---

Sampling Rate

• After every task commit: Run cd backend && pytest tests/ -v -x --tb=short
• After every plan wave: Run cd backend && pytest tests/ -v
• Before /gsd:verify-work: Full suite must be green
• Max feedback latency: 60 seconds

---

Per-Task Verification Map

Task ID	Plan	Wave	Requirement	Threat Ref	Secure Behavior	Test Type	Automated Command	File Exists	Status
06-W0-01	Wave 0	0	D-01	—	structlog emits JSON with correlation_id	unit	pytest tests/test_logging.py -x	✅	✅ green
06-W0-02	Wave 0	0	D-11	T-06-01	get_client_ip returns direct IP when peer is untrusted	unit	pytest tests/test_rate_limiting.py::test_get_client_ip_untrusted_returns_direct_peer -x	✅	✅ green
06-W0-03	Wave 0	0	D-11	T-06-01	get_client_ip reads XFF when peer is trusted proxy	unit	pytest tests/test_rate_limiting.py::test_get_client_ip_trusted_proxy_reads_xff_leftmost -x	✅	✅ green
06-W0-04	Wave 0	0	D-12	T-06-01	per-account limiter key is user.id not IP	unit	pytest tests/test_rate_limiting.py::test_account_limiter_key_uses_user_id -x	✅	✅ green
06-W0-05	Wave 0	0	D-12	T-06-01	authenticated endpoint returns 429 after 100 req/min	integration	pytest tests/test_rate_limiting.py::test_authenticated_endpoint_429_after_100_per_minute -x	✅	✅ green
06-W0-06	Wave 0	0	D-04..D-06	—	Locust locustfile.py exists and is discoverable	smoke	ls backend/load_tests/locustfile.py	✅	✅ green
06-LOG-01	structlog	1	D-01/D-02	—	JSON log line contains correlation_id and method	unit	pytest tests/test_logging.py -x	✅	✅ green
06-LOG-02	structlog	1	D-01	—	structlog contextvars cleared between requests	unit	pytest tests/test_logging.py::test_contextvars_cleared_between_requests -x	✅	✅ green
06-RL-01	rate limiting	5	D-11	T-06-01	IP rate limiter uses get_client_ip not get_remote_address	unit	pytest tests/test_rate_limiting.py -x	✅	✅ green
06-RL-02	rate limiting	5	D-12	T-06-01	per-account 429 after 100 req/min on documents endpoint	integration	pytest tests/test_rate_limiting.py::test_authenticated_endpoint_429_after_100_per_minute -x	✅	✅ green
06-SCOUT-01	docker scout	manual	D-10	CVE	Zero critical CVEs in built image	manual	docker scout cves local://docuvault-backend:latest --only-severity critical --exit-code	N/A manual	⬜ pending
06-DOCKER-01	Dockerfile	manual	D-07	EoP	Container runs as uid=1000 not root	manual	docker run --rm docuvault-backend:latest id outputs uid=1000	N/A manual	⬜ pending
06-DOCKER-02	docker-compose	manual	D-08	EoP	read_only container can write to /tmp	manual	docker compose up backend + upload a document — no PermissionError	N/A manual	⬜ pending
06-LOCUST-01	Locust	manual	D-06	—	SLA: p95 < 200ms, p99 < 500ms at 50 users	load test	locust --headless --users 50 --spawn-rate 10 --run-time 5m -f backend/load_tests/locustfile.py --host http://localhost:8000	N/A manual	⬜ pending

Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky

---

Wave 0 Requirements

• backend/tests/test_logging.py — 5 tests: JSON renderer, correlation ID middleware, response header, context cleared, uvicorn suppressed (D-01, D-02) — all green
• backend/tests/test_rate_limiting.py — 8 tests: get_client_ip (4 cases), account key (2 cases), ordering assumption, 429 integration (D-11, D-12, A1) — all green
• backend/load_tests/__init__.py — empty marker file present
• backend/load_tests/locustfile.py — full self-bootstrapping Locust HttpUser with SLA csv export (D-04, D-05, D-06)

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Container runs as uid=1000	D-07	Requires built Docker image, cannot be run in unit test	docker build -t docuvault-backend:latest backend/ && docker run --rm docuvault-backend:latest id — must show uid=1000(appuser)
read_only fs allows /tmp writes	D-08	Requires docker compose stack	docker compose up backend → upload a document → confirm no PermissionError in logs
SLA targets met at 50 concurrent users	D-06	Load test requires running stack	locust --headless --users 50 --spawn-rate 10 --run-time 5m -f backend/load_tests/locustfile.py --host http://localhost:8000 — must exit 0
docker scout reports zero critical CVEs	D-10	Requires built image + Docker Hub login	docker login && docker scout cves local://docuvault-backend:latest --only-severity critical --exit-code — must exit 0
Grafana dashboard shows Loki logs	D-02	UI verification	Open http://localhost:3000, Explore → Loki datasource → query {service="backend"} — should show JSON log lines with correlation_id

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 60s
• nyquist_compliant: true set in frontmatter

Approval: 2026-06-05 — 13/13 automated tests green; 4 manual items pending (Docker/Locust/Scout require running stack)

---

Validation Audit 2026-06-05

Metric	Count
Gaps found	10 (all tasks were "pending")
Resolved	10 (all automated tasks confirmed green)
Escalated to manual-only	0 (manual tasks were already classified)
Total automated tests	13 (test_logging: 5, test_rate_limiting: 8)
Manual-only items	4 (Docker uid, read-only fs, Locust SLA, docker scout CVE)