test(12-05): add migration 0005→0006 integration regression and RUNBOOK migration ops

- test_migration_0006.py: proves display_name_override, cloud_items, cloud_item_topics,
  cloud_folder_states, unique constraints, indexes, and DDL privilege boundary
- Tests skip cleanly without INTEGRATION=1/INTEGRATION_DATABASE_URL (no dev DB mutation)
- RUNBOOK: add Database Migrations section with revision check, upgrade, recovery, and
  emergency schema-drift remediation commands
This commit is contained in:
curo1305
2026-06-20 10:47:11 +02:00
parent 1b3084ddfa
commit de2efd1664
2 changed files with 385 additions and 8 deletions
+97 -8
View File
@@ -13,12 +13,13 @@ for phase history and roadmap see `.planning/ROADMAP.md`.
1. [Environment Variables](#1-environment-variables)
2. [Startup / Shutdown](#2-startup--shutdown)
3. [Backup Strategy](#3-backup-strategy)
4. [Health Checks](#4-health-checks)
5. [Security Gate — docker scout CVE Scan (D-10)](#5-security-gate--docker-scout-cve-scan-d-10)
6. [On-Call Escalation](#6-on-call-escalation)
7. [Common Failure Modes](#7-common-failure-modes)
8. [Phase 6 Deferred Items](#8-phase-6-deferred-items)
3. [Database Migrations](#3-database-migrations)
4. [Backup Strategy](#4-backup-strategy)
5. [Health Checks](#5-health-checks)
6. [Security Gate — docker scout CVE Scan (D-10)](#6-security-gate--docker-scout-cve-scan-d-10)
7. [On-Call Escalation](#7-on-call-escalation)
8. [Common Failure Modes](#8-common-failure-modes)
9. [Phase 6 Deferred Items](#9-phase-6-deferred-items)
---
@@ -146,11 +147,18 @@ docker compose logs -f --tail 50
Services start in dependency order:
```
postgres / minio / redis → backend / celery-worker / celery-beat
postgres / minio / redis → migrate (alembic upgrade head)
→ backend / celery-worker / celery-beat
→ loki → promtail / grafana
→ frontend
```
The `migrate` service is a one-shot container that runs `alembic upgrade head` using
`DATABASE_MIGRATE_URL` (DDL-capable `docuvault_migrate` role) and exits 0 on success.
All application services (`backend`, `celery-worker`, `celery-beat`) depend on it with
`condition: service_completed_successfully`. If `migrate` exits non-zero, no application
process starts and the deployment is blocked until the migration issue is resolved.
Access points after a healthy startup:
| Service | URL |
@@ -194,7 +202,88 @@ Do not add `read_only:` to the celery-beat block — it will exit immediately wi
---
## 3. Backup Strategy
## 3. Database Migrations
### Checking the current revision
```bash
# Inside the running backend container
docker compose exec backend alembic current
# Via a one-off migrate container (preferred — uses DDL role)
docker compose run --rm migrate alembic current
```
A healthy deployment at Phase 12 head reports:
```
0006 (head)
```
### Upgrading to the latest migration
```bash
# Compose will run migrate automatically on docker compose up.
# To run it manually (e.g., after a git pull):
docker compose run --rm migrate
```
`migrate` runs `alembic upgrade head` using `DATABASE_MIGRATE_URL`. On success it exits 0.
Application services will not start until this exits 0 — this is the deployment gate.
### Viewing migration history
```bash
docker compose run --rm migrate alembic history --verbose
```
### Recovering from a failed migration
1. **Read the logs** — the migrate container logs contain the SQL and the error:
```bash
docker compose logs migrate
```
2. **Fix the migration** — edit the failing migration revision file in `backend/migrations/versions/`.
3. **Re-run** — the migrate container is idempotent; Alembic skips already-applied revisions:
```bash
docker compose run --rm migrate
```
4. **Check the revision** — confirm head is applied before restarting backend:
```bash
docker compose run --rm migrate alembic current
```
5. **Restart application services** after a successful migration:
```bash
docker compose up -d backend celery-worker celery-beat
```
### Emergency: schema drift on live database (Phase 12 gap-closure scenario)
If the backend is already running against a database that is behind head (e.g., it started
before the migrate service was introduced and reports `UndefinedColumn: display_name_override`):
```bash
# 1. Stop application processes (not postgres)
docker compose stop backend celery-worker celery-beat
# 2. Apply missing migrations via the dedicated migrate service
docker compose run --rm migrate
# 3. Verify the revision
docker compose run --rm migrate alembic current
# Expected: 0006 (head)
# 4. Restart application processes through the gated path
docker compose up -d backend celery-worker celery-beat
```
---
## 4. Backup Strategy
### PostgreSQL