curo/kite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs(12): plan UAT migration gap closure

Browse Source
This commit is contained in:
curo1305
2026-06-20 10:40:37 +02:00
parent a7e17d69b4
commit 66d8634b17
1 changed files with 174 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/phases/12-cloud-resource-foundation/12-05-PLAN.md
+174
View File
@@ -0,0 +1,174 @@
---
phase: "12"
plan: "05"
type: gap_closure
wave: 1
depends_on:
- "12-04"
files_modified:
- docker-compose.yml
- backend/tests/test_compose_migrations.py
- backend/tests/test_migration_0006.py
- backend/main.py
- frontend/package.json
- frontend/package-lock.json
- AGENTS.md
- README.md
- RUNBOOK.md
- SECURITY.md
autonomous: true
requirements:
- CONN-04
- CLOUD-01
must_haves:
truths:
- "A clean `docker compose up` applies Alembic through head before backend, Celery worker, or Celery beat starts"
- "An existing database at revision 0005 upgrades to 0006 and contains display_name_override plus all Phase 12 cloud metadata tables"
- "GET /api/cloud/connections no longer fails with UndefinedColumn after a normal Compose restart"
- "POST /api/cloud/connections/webdav can persist a valid Nextcloud connection after migration"
- "A failed migration blocks application process startup instead of serving with schema drift"
artifacts:
- path: "docker-compose.yml"
provides: "One-shot Alembic migration service and service_completed_successfully dependency gate"
contains: "alembic upgrade head"
- path: "backend/tests/test_compose_migrations.py"
provides: "Static/Compose configuration regression for migration ordering"
contains: "service_completed_successfully"
- path: "backend/tests/test_migration_0006.py"
provides: "PostgreSQL upgrade regression from revision 0005 to head"
contains: "display_name_override"
- path: "RUNBOOK.md"
provides: "Normal migration lifecycle, revision verification, and schema-drift recovery"
contains: "alembic current"
key_links:
- from: "docker-compose.yml migrate service"
to: "backend/celery-worker/celery-beat services"
via: "depends_on migrate condition service_completed_successfully"
pattern: "service_completed_successfully"
- from: "backend/migrations/versions/0006_cloud_resource_foundation.py"
to: "backend/tests/test_migration_0006.py"
via: "real PostgreSQL revision and information_schema assertions"
pattern: "0005.*0006|display_name_override"
---
<objective>
Close the Phase 12 UAT blockers caused by application/schema drift. Make database migration a mandatory Compose startup gate, add a real upgrade regression, repair the running environment, and verify both cloud connection listing and Nextcloud connection creation before UAT resumes.
Root cause: `.planning/debug/phase-12-cloud-schema-cold-start.md` proves the live database remained at Alembic `0005` because Compose started application processes without running `alembic upgrade head`.
</objective>
<execution_context>
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
</execution_context>
<context>
@.planning/phases/12-cloud-resource-foundation/12-UAT.md
@.planning/debug/phase-12-cloud-schema-cold-start.md
@.planning/phases/12-cloud-resource-foundation/12-VALIDATION.md
@backend/migrations/versions/0006_cloud_resource_foundation.py
@docker-compose.yml
</context>
<tasks>
<task type="auto">
<name>Task 1: Add a migration-gated Compose startup path</name>
<files>docker-compose.yml, backend/tests/test_compose_migrations.py</files>
<read_first>
- docker-compose.yml — current backend/worker/beat dependencies, hardening, volumes, and environment
- backend/migrations/env.py — runtime `DATABASE_MIGRATE_URL` selection
- backend/Dockerfile — image user/workdir and Alembic availability
- .planning/debug/phase-12-cloud-schema-cold-start.md — confirmed deployment root cause
</read_first>
<action>
Add a one-shot `migrate` service built from `./backend` that runs `alembic upgrade head` from `/app`, receives `DATABASE_MIGRATE_URL` and only migration-required environment, mounts backend source consistently for development, waits for PostgreSQL health (and any migration-time MinIO prerequisite required by revision 0004), and uses the same non-root/read-only/tmpfs/cap-drop/no-new-privileges hardening as backend. Change backend, celery-worker, and celery-beat so each has `depends_on.migrate.condition: service_completed_successfully` in addition to its runtime dependencies. Do not run Alembic independently in every replica and do not grant DDL privileges to `docuvault_app`. Add focused tests that load/render the Compose configuration and assert the migrate command, migrate-role URL, one-shot dependency, and absence of `alembic upgrade` from ordinary backend/worker commands. Include a failure-path assertion showing a nonzero migration exit prevents dependent services from starting.
</action>
<acceptance_criteria>
- `docker compose config` succeeds and contains exactly one migration service command `alembic upgrade head`
- backend, celery-worker, and celery-beat each wait for successful migration completion
- migration service uses `DATABASE_MIGRATE_URL`; application services continue using `DATABASE_URL`
- migration service preserves container hardening and stores no credentials in source
- `pytest -q tests/test_compose_migrations.py` passes
</acceptance_criteria>
<verify><automated>docker compose config >/tmp/docuvault-compose.yml && docker compose run --rm backend pytest -q tests/test_compose_migrations.py</automated></verify>
<done>Compose has one hardened migration gate, every application process depends on it, and configuration tests pass.</done>
</task>
<task type="auto">
<name>Task 2: Prove PostgreSQL upgrade from 0005 to Phase 12 head</name>
<files>backend/tests/test_migration_0006.py, RUNBOOK.md</files>
<read_first>
- backend/migrations/versions/0005_system_settings.py — starting revision
- backend/migrations/versions/0006_cloud_resource_foundation.py — required upgrade objects
- backend/tests/test_alembic.py — existing migration tests and their limitations
- docker/postgres/initdb.d/01-init-users.sql — migrate/app role privileges
- RUNBOOK.md — existing database operations guidance
</read_first>
<action>
Add an integration test that uses a disposable PostgreSQL database or isolated test schema, applies Alembic through `0005`, then upgrades to head. Assert `alembic_version` is `0006`; `cloud_connections.display_name_override` exists; `cloud_items`, `cloud_item_topics`, and `cloud_folder_states` exist; named unique constraints/indexes exist; and `docuvault_app` can perform intended DML but not DDL. The test must never downgrade or mutate the developer's normal database and must skip with an explicit prerequisite message when a disposable integration database cannot be provisioned. Add a cold-start Compose test path suitable for CI that starts from an isolated project/volume, waits for migrate success, checks revision/head, exercises `/health`, and tears down only that isolated project. Document normal upgrade, revision inspection, failed-migration recovery, and the immediate `0005` to head remediation commands in RUNBOOK.
</action>
<acceptance_criteria>
- integration test proves a real 0005-to-0006/head transition without touching the normal development database
- all four Phase 12 schema additions are asserted against PostgreSQL catalog/information_schema
- migration failure is observable and prevents backend startup
- RUNBOOK gives exact safe commands for current revision, upgrade, logs, restart, and recovery
- focused migration tests pass in the Compose environment
</acceptance_criteria>
<verify><automated>docker compose run --rm -e INTEGRATION=1 backend pytest -q tests/test_migration_0006.py</automated></verify>
<done>A disposable PostgreSQL test upgrades from 0005 to head and proves the column/tables/privilege boundaries; operational recovery is documented.</done>
</task>
<task type="auto">
<name>Task 3: Repair the environment, rerun blocker paths, and close protocol</name>
<files>backend/main.py, frontend/package.json, frontend/package-lock.json, AGENTS.md, README.md, RUNBOOK.md, SECURITY.md</files>
<read_first>
- .planning/phases/12-cloud-resource-foundation/12-UAT.md — two confirmed blockers and four blocked tests
- backend/api/cloud/connections.py — failing list and WebDAV/Nextcloud paths
- AGENTS.md — bug regression, documentation, version, security, commit, and push protocols
- SECURITY.md — Phase 12 threat register and gate evidence
</read_first>
<action>
Apply `alembic upgrade head` through the new migration service in the current Compose environment, verify `alembic current` reports `0006`, and restart/recreate backend plus Celery processes through the gated path. Add API regressions against PostgreSQL for `GET /api/cloud/connections` and valid mocked/controlled `POST /api/cloud/connections/webdav`, asserting neither raises UndefinedColumn and the connection response excludes credentials. Run focused cloud/migration tests, full backend/frontend suites, frontend build, Bandit, npm audit, and available pip-audit/secret checks; resolve introduced high-severity findings. Bump the patch version from 0.2.0 to 0.2.1 in tracked version sources. Update AGENTS current state to Phase 12 UAT schema-drift gap closure, README startup behavior, RUNBOOK final commands, and SECURITY evidence. Commit and push atomically under `fix(12-uat): gate startup on database migrations`.
</action>
<acceptance_criteria>
- live `alembic current` reports 0006/head before backend starts
- connection list returns successfully for an authenticated user
- Nextcloud/WebDAV connection creation reaches normal validation/persistence without schema exceptions
- focused regression, full suites, build, and mandatory security gates pass or document only pre-existing external blockers
- backend and frontend tracked versions equal 0.2.1; docs accurately describe migration-gated startup
</acceptance_criteria>
<verify><automated>docker compose run --rm migrate && docker compose run --rm backend alembic current && docker compose run --rm backend pytest -q tests/test_compose_migrations.py tests/test_migration_0006.py tests/test_cloud.py tests/test_cloud_security.py && cd frontend && npm test && npm run build</automated></verify>
<done>The running schema is at head, both UAT blocker paths work, regression/security gates pass, version/docs are 0.2.1, and Phase 12 UAT can resume at Test 1.</done>
</task>
</tasks>
<threat_model>
| Threat ID | Threat | Mitigation and evidence |
|-----------|--------|-------------------------|
| T-12-05-01 | App starts against stale schema | One-shot migrate service and successful-completion dependency gate |
| T-12-05-02 | DDL privilege escalation | Migration uses migrate URL; app retains DML-only URL and privilege regression test |
| T-12-05-03 | Concurrent migrations | Exactly one Compose migration service; replicas only wait on completion |
| T-12-05-04 | Secrets exposed in Compose/tests | Environment references only; no rendered config or logs committed with values |
| T-12-05-05 | Test destroys developer DB | Disposable database/schema/project requirement; explicit skip rather than fallback to normal DB |
| T-12-05-SC | Dependency risk | No new runtime dependencies; standard security/audit gates rerun |
</threat_model>
<verification>
1. GSD plan structure validator reports zero errors/warnings.
2. Compose config proves migration dependency ordering and hardening.
3. Disposable PostgreSQL upgrade proves 0005 -> 0006/head and schema objects.
4. Live revision and both failed API paths pass after gated restart.
5. Full tests, build, and security gates complete before UAT resumes.
</verification>
<success_criteria>
- Normal startup cannot serve application traffic with unapplied migrations.
- The exact `display_name_override` regression is caught automatically.
- Current Nextcloud and connection-list blockers are resolved at their deployment root cause.
- Documentation and versioning reflect the gap closure.
</success_criteria>
<output>Create `.planning/phases/12-cloud-resource-foundation/12-05-SUMMARY.md` when complete.</output>