diff --git a/.planning/phases/08-stack-upgrade-backend-decomposition/08-CONTEXT.md b/.planning/phases/08-stack-upgrade-backend-decomposition/08-CONTEXT.md new file mode 100644 index 0000000..b7f96fa --- /dev/null +++ b/.planning/phases/08-stack-upgrade-backend-decomposition/08-CONTEXT.md @@ -0,0 +1,133 @@ +# Phase 8: Stack Upgrade & Backend Decomposition — Context + +**Gathered:** 2026-06-07 +**Status:** Ready for planning + + +## Phase Boundary + +Phase 8 delivers: (1) Phase 7.1 session-revocation work completed first as Wave 1, (2) three backend router monoliths split into focused sub-packages (`api/admin/`, `api/documents/`, `api/auth/`), (3) the frontend API client decomposed into domain modules behind a re-export barrel, (4) shared Pydantic schemas extracted to `api/schemas.py`, (5) router-defined validators migrated to `services/`, (6) `requirements.txt` pinned to exact versions, and (7) PERF-01 frontend dependencies installed. Zero URL changes, zero behavior changes — invisible to consumers and tests except for Phase 7.1's new session-revocation behavior. + + + + +## Implementation Decisions + +### Phase 7.1 — Session Revocation (Wave 1, before decomposition) + +- **D-01:** Phase 7.1 is absorbed into Phase 8 as the first wave. It must complete (backend + tests + frontend) before the decomposition work begins. +- **D-02:** Phase 7.1 scope: `revoke_all_refresh_tokens()` gets a `skip_token_hash` param; the function is wired into `change_password`, `enable_totp`, and `disable_totp` with a `sessions_revoked` field in each response shape; audit log entries written for each revocation. +- **D-03:** Frontend toast implementation: create an empty `useToastStore` Pinia store stub during Phase 7.1. Phase 7.1 components (`SettingsAccountTab.vue`, `TotpEnrollment.vue`) call `toastStore.show(...)` when `sessions_revoked > 0`. Phase 10 fills in the full toast implementation — Phase 7.1 only creates the stub and wires the call sites. + +### Backend Router Decomposition + +- **D-04:** Sub-routers in all three new packages MUST have NO prefix on `APIRouter()`. The parent prefix registered in `main.py` propagates. Any sub-router prefix causes doubled URL segments (confirmed pitfall from v0.1). +- **D-05:** `api/admin/` split: `users.py`, `quotas.py`, `ai.py` — exact names already locked by ROADMAP. +- **D-06:** `api/documents/` split: 4 sub-modules matching the ROADMAP's 4 functional areas (upload flow, content proxy, document CRUD, search/listing). Exact file names are researcher/planner's choice based on reading the actual code. +- **D-07:** `api/auth/` split: 4 sub-modules (login/tokens, TOTP, password management, session management). Module names must mirror the logical groupings found in `services/auth.py` for navigability. The JTI revocation (7.2), ES256 key handling (7.3), and fgp fingerprint validation (7.4) code goes wherever the researcher determines it fits cleanest within those groupings. +- **D-08:** Re-classify endpoint (`POST /api/documents/{id}/classify`) placement within `api/documents/` is researcher/planner's choice. + +### Shared Pydantic Schemas (CODE-08) + +- **D-09:** Pydantic models shared within a single package → `shared.py` inside that package. +- **D-10:** Pydantic models referenced by 2+ packages (e.g., a `UserOut` used by both `api/admin/` and `api/auth/`) → `backend/api/schemas.py` (new top-level schemas module). This eliminates circular imports between packages. +- **D-11:** Any validators currently defined inline in router files (rather than in `services/`) must be migrated to the appropriate `services/` module during the split. This enforces the CLAUDE.md rule that the service layer raises `ValueError`, and API files never define validators. Researcher identifies which validators qualify. + +### Frontend API Client Decomposition (CODE-04) + +- **D-12:** `client.js` becomes HTTP transport layer + re-export barrel only. `request()` and `noRefreshPaths` stay in `client.js`. Zero changes to any of the 35+ consumer files. +- **D-13:** `fetchWithRetry()` (consolidating the 3 blob-download 401-retry duplicates) lives in a new `frontend/src/api/utils.js`. `client.js` re-exports it. +- **D-14:** Domain sub-modules: `documents.js`, `auth.js`, `admin.js`, `folders.js`, `shares.js`, `cloud.js`, `topics.js`. Each function lives in exactly one domain file (researcher picks the most specific domain; no duplication). Researcher maps all consumer import sites to determine grouping. + +### Frontend Dependencies (PERF-01) + +- **D-15:** Vite 5→6 is a major version bump requiring research before execution. Researcher must check the Vite 6 migration guide against the existing `frontend/vite.config.js` and identify any required config changes before the bump is applied. +- **D-16:** For the remaining PERF-01 packages (`@vueuse/core@^14`, `@vueuse/integrations@^14`, `sortablejs`, `@tailwindcss/forms`, `rollup-plugin-visualizer`, dev type packages): researcher determines which packages require config wiring in `tailwind.config.js` or `vite.config.js` vs which are install-only. + +### Backend Dependency Pinning + +- **D-17:** Backend dep version bumping (FastAPI, SQLAlchemy, etc.) is out of scope for Phase 8. However, `requirements.txt` must be converted from floating `>=` ranges to exact `==` pins for reproducible builds. Pin the currently-installed versions (no version changes). + +### Claude's Discretion + +- Exact file names for `api/documents/` sub-modules (researcher reads `api/documents.py` and chooses names that reflect the actual endpoint groupings) +- Exact file names for `api/auth/` sub-modules (researcher mirrors `services/auth.py` logical groupings) +- Re-classify endpoint placement within `api/documents/` +- Which validators in router files qualify for migration to `services/` +- Domain grouping of each `client.js` function (researcher maps all 35+ consumer import sites) +- Which PERF-01 packages need config wiring vs install-only + + + + +## Canonical References + +**Downstream agents MUST read these before planning or implementing.** + +### Phase Goals and Requirements +- `.planning/ROADMAP.md` §"Phase 8: Stack Upgrade & Backend Decomposition" — goal, implementation notes, success criteria, PITFALLS +- `.planning/REQUIREMENTS.md` §CODE-01, CODE-02, CODE-03, CODE-04, CODE-08, PERF-01 — formal requirement definitions + +### Phase 7.1 Scope (absorbed into this phase) +- `.planning/ROADMAP.md` §"Phase 7.1: Security: session revocation on privilege change (CR-01..03)" — the two plan descriptions; requirements CR-01, CR-02, CR-03 + +### Backend Architecture and Conventions +- `.planning/codebase/ARCHITECTURE.md` — component responsibilities, layer boundaries, data flow; identifies what lives in `api/` vs `services/` +- `.planning/codebase/CONVENTIONS.md` — import organization, naming patterns, service-vs-API error handling separation; the rule that validators live in `services/` +- `CLAUDE.md` §"Backend: shared module map" — `deps/utils.py`, `storage/exceptions.py`, `ai/utils.py`, `services/auth.py` shared module rules (must not be violated during split) +- `CLAUDE.md` §"Key Architectural Rules" — ownership checks, atomic quota pattern, JWT memory-only rules (must be preserved through refactor) + +### Frontend Architecture +- `.planning/codebase/ARCHITECTURE.md` §"Frontend Store Layer" — stores own data-fetching, views delegate to stores +- `CLAUDE.md` §"Frontend: shared module map" — `utils/formatters.js`, `TreeItem.vue`, `StorageBrowser.vue` rules +- `CLAUDE.md` §"Component architecture" — View → Smart component → Presentational component layering + +### Stack Details +- `.planning/codebase/STACK.md` — current package versions (for pinning to exact == in requirements.txt) + + + + +## Existing Code Insights + +### Reusable Assets +- `backend/api/admin.py` (934L), `backend/api/documents.py` (852L), `backend/api/auth.py` (825L) — the three monoliths being decomposed; researcher must read these to determine sub-module boundaries +- `frontend/src/api/client.js` (635L) — the frontend client being decomposed; researcher must read all import sites +- `backend/services/auth.py` — structure mirrors how `api/auth/` sub-modules should be named + +### Established Patterns +- `backend/api/admin.py` already contains AI config endpoints added in Phase 7 — `api/admin/ai.py` absorbs this +- `backend/main.py` router registration pattern: `app.include_router(router, prefix="/api/...")` — each new package `__init__.py` must aggregate sub-routers and export a single `router` object +- `asyncio.run()` bridge in Celery tasks — never import from router modules; this constraint applies to the new sub-packages too +- Circular import constraint: `backend/celery_app.py` intentionally avoids importing `config`; new sub-packages must not create import cycles + +### Integration Points +- `backend/main.py` — only file that registers routers; must be updated to import from new package `__init__.py` files +- `frontend/src/api/client.js` — must re-export every function from domain sub-modules; no consumer file changes +- `backend/deps/auth.py` — `get_current_user`, `get_current_admin`, `get_regular_user` imported by all router sub-modules; dependency imports must work from new sub-package paths +- `frontend/src/stores/` — all 5 stores import from `api/client.js`; barrel re-export must preserve all named exports + + + + +## Specific Ideas + +- Wave ordering: Phase 7.1 completion → backend decomposition (CODE-01/02/03 in parallel) + CODE-04 (independent) → requirements.txt pinning + PERF-01 deps +- The `useToastStore` stub created in Phase 7.1 is a deliberate forward-reference contract: Phase 10's toast implementation must honor the same `show()` API shape +- `backend/api/schemas.py` is a NEW file — it doesn't exist yet; researcher should identify exact cross-package model candidates before planning + + + + +## Deferred Ideas + +- Backend package version bumps (FastAPI to 0.136+, SQLAlchemy, PyJWT, etc.) — deferred; only exact-pin the current versions +- Composition API migration (Vue components) — explicitly out of scope for all v0.2 phases per PROJECT.md decision +- Virtual scrolling, dark mode, folder reordering, multi-select batch ops — already in REQUIREMENTS.md "Future" section + + + +--- + +*Phase: 8-Stack-Upgrade-Backend-Decomposition* +*Context gathered: 2026-06-07* diff --git a/.planning/phases/08-stack-upgrade-backend-decomposition/08-DISCUSSION-LOG.md b/.planning/phases/08-stack-upgrade-backend-decomposition/08-DISCUSSION-LOG.md new file mode 100644 index 0000000..da8953d --- /dev/null +++ b/.planning/phases/08-stack-upgrade-backend-decomposition/08-DISCUSSION-LOG.md @@ -0,0 +1,177 @@ +# Phase 8: Stack Upgrade & Backend Decomposition — Discussion Log + +> **Audit trail only.** Do not use as input to planning, research, or execution agents. +> Decisions are captured in CONTEXT.md — this log preserves the alternatives considered. + +**Date:** 2026-06-07 +**Phase:** 8-Stack-Upgrade-Backend-Decomposition +**Areas discussed:** documents.py split structure, auth.py split structure, Phase 7.1 deferred gap, Vite 5→6 migration scope, client.js domain grouping, Phase 7.1 toast implementation, Backend dep bumps, CODE-08 shared.py scope + +--- + +## documents.py split structure + +| Option | Description | Selected | +|--------|-------------|----------| +| Merge search with CRUD → 3 files | upload.py, proxy.py, documents.py (list+search merged) | | +| Keep 4 files as described | 4 sub-modules matching ROADMAP's functional areas | ✓ | + +**User's choice:** Keep 4 files as described + +| Option | Description | Selected | +|--------|-------------|----------| +| Into CRUD / documents.py | Re-classify operates on a single document by ID | | +| Into upload.py | Re-triggers the Celery pipeline, adjacent to confirm | | +| You decide | Researcher/planner places it based on code | ✓ | + +**User's choice:** Researcher decides re-classify placement + +| Option | Description | Selected | +|--------|-------------|----------| +| upload.py, proxy.py, crud.py, listing.py | Explicit names for 4 areas | | +| upload.py, content.py, documents.py, search.py | Alternative naming | | +| You decide | Researcher picks names based on actual code | ✓ | + +**User's choice:** Researcher picks file names + +--- + +## auth.py split structure + +| Option | Description | Selected | +|--------|-------------|----------| +| Keep 4 files, JTI/ES256/fgp fold into tokens.py | All token-security concerns together | | +| Keep 4 files as originally described | JTI/ES256/fgp placed by researcher | | +| You decide | Researcher reads actual code and picks cleanest split | ✓ | + +**User's choice:** Researcher decides auth split + +| Option | Description | Selected | +|--------|-------------|----------| +| Independent — router split is about HTTP grouping | api/auth/ groups by HTTP endpoint families | | +| Aligned — mirror services structure in API split | Module names match services/auth.py logical groupings | ✓ | + +**User's choice:** api/auth/ sub-module names mirror services/auth.py logical groupings + +--- + +## Phase 7.1 deferred gap + +| Option | Description | Selected | +|--------|-------------|----------| +| Intentionally deferred — keep it separate | Phase 8 proceeds as pure decomposition/upgrade | | +| Include in Phase 8 | Phase 8 absorbs the 2 Phase 7.1 plans before decomposition | ✓ | + +**User's choice:** Include Phase 7.1 in Phase 8 + +| Option | Description | Selected | +|--------|-------------|----------| +| First wave — do 7.1 before the refactor | 7.1 completes first, then decomposition follows | ✓ | +| Separate plan within Phase 8 | 7.1 as a dedicated plan that can run independently | | + +**User's choice:** Phase 7.1 as Wave 1, decomposition follows + +--- + +## Vite 5→6 migration scope + +| Option | Description | Selected | +|--------|-------------|----------| +| Assume it just works — bump and verify | Bump, run dev+build, fix anything that breaks | | +| Research first — there may be config changes | Check Vite 6 migration guide against existing vite.config.js | ✓ | + +**User's choice:** Research migration guide against vite.config.js before bumping + +| Option | Description | Selected | +|--------|-------------|----------| +| All install-only for now | No integration in Phase 8 | | +| @tailwindcss/forms needs tailwind.config.js wiring | Must register plugin even without styles | | +| You decide | Researcher determines which need config wiring | ✓ | + +**User's choice:** Researcher determines PERF-01 package integration requirements + +--- + +## client.js domain grouping + +| Option | Description | Selected | +|--------|-------------|----------| +| Most specific domain file only | Every function in exactly one file, no duplication | | +| You decide | Researcher maps all 35+ consumer imports | ✓ | + +**User's choice:** Researcher maps imports and picks grouping + +| Option | Description | Selected | +|--------|-------------|----------| +| In client.js alongside request() | Both transport helpers in same file | | +| New api/utils.js | fetchWithRetry in a separate utility module | ✓ | + +**User's choice:** fetchWithRetry → new `frontend/src/api/utils.js` + +--- + +## Phase 7.1 toast implementation + +| Option | Description | Selected | +|--------|-------------|----------| +| Simple local implementation | Local reactive message + v-if in the two components | | +| Stub the global store early | Create empty useToastStore now; Phase 10 fills it in | ✓ | +| Skip the toast for now | Backend + tests only; toast waits for Phase 10 | | + +**User's choice:** Create `useToastStore` stub in Phase 7.1; Phase 10 implements it + +**Notes:** The stub establishes the `show()` API contract that Phase 10 must honor. + +--- + +## Backend dep bumps + +| Option | Description | Selected | +|--------|-------------|----------| +| Include backend bumps in Phase 8 | Bump FastAPI, SQLAlchemy, PyJWT + pip-audit | | +| Backend deps out of scope | PERF-01 is frontend-only; backend version management is separate | ✓ | + +**User's choice:** Backend dep bumps out of scope for Phase 8 + +| Option | Description | Selected | +|--------|-------------|----------| +| Pin to exact versions | Convert >= to == for reproducible builds | ✓ | +| Leave floating for now | Only fix if bumping is in scope | | + +**User's choice:** Pin current versions to == (no version changes, just reproducibility) + +--- + +## CODE-08 shared.py scope + +| Option | Description | Selected | +|--------|-------------|----------| +| Cross-package models → top-level api/schemas.py | New backend/api/schemas.py for models used by 2+ packages | ✓ | +| Duplicate across packages OK for now | Independent definition in each package's shared.py | | +| You decide | Researcher identifies actual cross-package overlap | | + +**User's choice:** New `backend/api/schemas.py` for cross-package models; per-package `shared.py` for intra-package sharing + +| Option | Description | Selected | +|--------|-------------|----------| +| Move any router-defined validators to services/ during split | Enforce CLAUDE.md rule during the split | ✓ | +| Split only — no validator migrations | Structural changes only | | + +**User's choice:** Router-defined validators migrated to services/ during the split + +--- + +## Claude's Discretion + +- Exact file names for `api/documents/` sub-modules +- Exact file names for `api/auth/` sub-modules +- Re-classify endpoint placement within `api/documents/` +- Which validators in router files qualify for migration to `services/` +- Domain grouping of each `client.js` function (full import site mapping needed) +- Which PERF-01 packages require config wiring vs install-only + +## Deferred Ideas + +- Backend package version bumps (FastAPI to 0.136+, SQLAlchemy, PyJWT) — future phase +- Composition API migration for Vue components — explicitly out of scope for all v0.2 phases +- Virtual scrolling, dark mode, folder reordering, multi-select batch ops — already in REQUIREMENTS.md "Future"