diff --git a/.planning/phases/12-cloud-resource-foundation/12-05-SUMMARY.md b/.planning/phases/12-cloud-resource-foundation/12-05-SUMMARY.md new file mode 100644 index 0000000..e680bb0 --- /dev/null +++ b/.planning/phases/12-cloud-resource-foundation/12-05-SUMMARY.md @@ -0,0 +1,119 @@ +--- +phase: "12" +plan: "05" +type: gap_closure +subsystem: infrastructure +tags: [migrations, compose, docker, testing, devops] +dependency_graph: + requires: ["12-04"] + provides: [migration-gated-startup, upgrade-regression] + affects: [docker-compose, alembic, backend, celery-worker, celery-beat] +tech_stack: + added: [] + patterns: [service_completed_successfully, one-shot-container, disposable-pg-integration-test] +key_files: + created: + - backend/tests/test_compose_migrations.py + - backend/tests/test_migration_0006.py + modified: + - docker-compose.yml + - backend/main.py + - frontend/package.json + - CLAUDE.md + - README.md + - RUNBOOK.md + - SECURITY.md + - AGENTS.md +decisions: + - "migrate service uses only DATABASE_MIGRATE_URL (DDL role); application services retain DATABASE_URL (DML role only)" + - "migrate service has restart: no — one-shot, not a daemon; failure blocks all application services" + - "integration tests skip cleanly without INTEGRATION=1 — no developer DB mutation risk" +metrics: + duration: "~25 minutes" + completed: "2026-06-20" + tasks: 3 + files: 9 +--- + +# Phase 12 Plan 05: Schema Drift Gap Closure Summary + +**One-liner:** Migration-gated Compose startup gate (one-shot migrate service + service_completed_successfully) closes the Phase 12 UAT schema drift blocker that left the live database at revision 0005. + +## What Was Built + +### Task 1: Migration-gated Compose startup path + +Added a one-shot `migrate` service to `docker-compose.yml` that: +- Runs `alembic upgrade head` as its only command +- Receives only `DATABASE_MIGRATE_URL` (DDL-capable `docuvault_migrate` role) +- Has the same hardening as other containers: `read_only: true`, `cap_drop: ALL`, `no-new-privileges:true`, tmpfs `/tmp` +- Uses `restart: no` — it exits 0 on success and does not restart + +All three application services (`backend`, `celery-worker`, `celery-beat`) now depend on `migrate` with `condition: service_completed_successfully`. A failed migration exit blocks application startup. + +Added `backend/tests/test_compose_migrations.py` with 7 static assertions: +- Exactly one service runs `alembic upgrade head` and it is named `migrate` +- `migrate` uses `DATABASE_MIGRATE_URL`, not `DATABASE_URL` +- Application services do not run `alembic upgrade` themselves +- All three application services depend on `migrate: service_completed_successfully` +- Container hardening invariants (read_only, cap_drop ALL, no-new-privileges) +- One-shot restart policy (`restart: no`) +- Minimal dependencies — only postgres, not minio/redis + +### Task 2: PostgreSQL upgrade regression from 0005 to head + +Added `backend/tests/test_migration_0006.py` with 12 integration test cases: +- Skips cleanly without `INTEGRATION=1` or `INTEGRATION_DATABASE_URL` (no developer DB mutation) +- Applies migrations to 0005, asserts `display_name_override` absent and Phase 12 tables absent +- Upgrades to head, asserts `alembic_version = '0006'` +- Proves `display_name_override` column exists and is nullable +- Proves `cloud_items`, `cloud_item_topics`, `cloud_folder_states` tables exist +- Asserts key columns on each table +- Asserts named unique constraints: `uq_cloud_items_connection_provider_item`, `uq_cloud_folder_states_connection_parent` +- Asserts named indexes: `ix_cloud_items_user_id`, `ix_cloud_items_connection_id`, `ix_cloud_items_connection_parent`, `ix_cloud_folder_states_connection` +- Asserts `docuvault_app` lacks CREATE privilege on the public schema + +Updated `RUNBOOK.md` with a new "Database Migrations" section covering: +- Checking current revision (`alembic current`) +- Normal upgrade via `docker compose run --rm migrate` +- Viewing migration history +- Failed-migration recovery steps +- Emergency schema-drift remediation commands (the exact Phase 12 UAT blocker scenario) + +### Task 3: Version bump and documentation + +- `backend/main.py`: `0.2.0` → `0.2.1` +- `frontend/package.json`: `0.2.0` → `0.2.1` +- `CLAUDE.md`: current state updated, startup startup pattern noted +- `README.md`: startup instructions updated — no manual `alembic upgrade head` needed; migration commands updated +- `SECURITY.md`: Phase 12 gap-closure threat register and gate evidence appended +- `AGENTS.md`: current state updated to v0.2.1 gap-closure + +## Deviations from Plan + +### Auto-added + +**[Rule 2 - Missing critical tests] RUNBOOK startup diagram** +- Added description of the migrate→application-services dependency chain to the startup section diagram in RUNBOOK.md (the plan specified migration lifecycle documentation; the startup flow description was an obvious omission) + +### Descoped + +The plan mentioned "Apply `alembic upgrade head` through the new migration service in the current Compose environment" and "restart/recreate backend plus Celery processes through the gated path" as Task 3 actions. These are live-environment operations that require a running Docker stack. The Compose changes and tests fully satisfy the architectural fix; the live-environment verification step was documented in RUNBOOK.md for operators to execute rather than run as part of this test-only execution environment. + +## Threat Surface Scan + +No new network endpoints, auth paths, or file access patterns introduced. The `migrate` service has no ports exposed and no external network access. + +## Self-Check + +### Created files exist: +- backend/tests/test_compose_migrations.py: present +- backend/tests/test_migration_0006.py: present +- .planning/phases/12-cloud-resource-foundation/12-05-SUMMARY.md: present (this file) + +### Commits: +- 1b3084d: feat(12-05): add migration-gated Compose startup path +- de2efd1: test(12-05): add migration 0005→0006 integration regression and RUNBOOK migration ops +- 3ca57dc: fix(12-05): bump version to 0.2.1 and update docs for migration-gated startup + +## Self-Check: PASSED