curo/kite
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: research milestone v0.3 cloud storage integration

Browse Source
This commit is contained in:
curo1305
2026-06-17 23:19:01 +02:00
parent 0e56c85349
commit 9150d28fe1
5 changed files with 219 additions and 1430 deletions
Download Patch File Download Diff File Expand all files Collapse all files
.planning/research/ARCHITECTURE.md
+63 -480
View File
@@ -1,508 +1,91 @@
# Architecture Patterns
# v0.3 Research: Architecture
**Domain:** v0.2 UI Overhaul and Optimization — integration analysis for existing DocuVault codebase
**Researched:** 2026-06-07
**Milestone:** v0.3 Reimagining Cloud Storage integration
**Date:** 2026-06-17
---
## Target Model
## How the Layout System Currently Works
Cloud files become virtual documents: provider-owned bytes remain in the connected storage, while DocuVault stores metadata, extracted text, topics, embeddings/search data, status, and a bounded cache of temporary bytes.
App.vue is the layout switch. It checks `route.meta.layout` and renders one of two branches:
## Proposed Components
```
route.meta.layout === 'auth' → <AuthLayout /> (renders <router-view /> inside a centered card)
(anything else) → AppSidebar + <main><router-view /></main>
```
### Provider Capability Layer
There are currently only two layouts:
- **AuthLayout.vue** — centered card, no sidebar, used for login/register/password-reset
- **Implicit main layout** — App.vue's `v-else` branch: `<AppSidebar>` + scrollable main
Add a cloud file operation interface separate from the current object-storage `StorageBackend` contract:
The admin panel (`/admin`) currently falls into the `v-else` branch. AdminView.vue renders inside the main user sidebar, inheriting AppSidebar's folder tree, quota bar, and user nav — none of which are relevant to an admin.
- `list_folder`
- `get_metadata`
- `download_bytes` / streaming equivalent
- `upload_file`
- `rename_item`
- `move_item`
- `delete_item`
- `create_folder`
- `get_change_cursor` / `list_changes` where available
- `capabilities()` returning action support and limitations
---
This avoids forcing local/MinIO object semantics onto provider file-manager behavior.
## Integration Point 1: FastAPI Admin Router Decomposition
### Cloud Item Index
### Current state
Add durable tables for cloud-visible items and analysis/index state:
`backend/api/admin.py` is a single file with `router = APIRouter(prefix="/api/admin")`. It is registered in `main.py` as:
- `cloud_items`: user, connection, provider, provider item id/path, parent id/path, name, kind, MIME/content type, size, etag/version, mtime, path snapshot, last_seen_at, deleted_at.
- `cloud_item_analysis`: item, status, extracted_text, topic links, semantic index status, last_analyzed_etag, error, job id, timestamps.
- Optional `cloud_file_cache`: item, etag/version, byte location, size, last_accessed_at, expires_at, pin reason, eviction state.
```python
from api.admin import router as admin_router
app.include_router(admin_router)
```
Keep `documents` for DocuVault-owned local/imported files unless a later design intentionally merges local and cloud into one polymorphic table.
The file mixes four concern areas:
- User management (CRUD on users, status, password reset)
- Quota management (get/patch quota per user)
- AI provider configuration (system-level: get, put, test-connection, get-models)
- Audit log was already extracted to `api/audit.py` in v0.1
### API Shape
### Target structure
Split `backend/api/cloud.py` when implementation starts:
```
backend/api/admin/
__init__.py
users.py — /api/admin/users endpoints
quotas.py — /api/admin/users/{id}/quota endpoints
ai.py — /api/admin/ai-config endpoints
```
- `api/cloud/connections.py`
- `api/cloud/browse.py`
- `api/cloud/actions.py`
- `api/cloud/analysis.py`
- `api/cloud/search.py` or reuse a shared search API
- shared schemas in `api/schemas.py` or `api/cloud/schemas.py`
### How to register without changing URL prefix
Every endpoint must assert `connection.user_id == current_user.id` and `cloud_item.user_id == current_user.id`.
FastAPI supports two-level include: a parent router with the shared prefix, sub-routers without prefix. This is the correct pattern:
### Frontend Shape
```python
# backend/api/admin/__init__.py
from fastapi import APIRouter
from .users import router as users_router
from .quotas import router as quotas_router
from .ai import router as ai_router
`FileManagerView` and `CloudFolderView` remain thin data providers. `StorageBrowser.vue` becomes capability-aware:
router = APIRouter(prefix="/api/admin", tags=["admin"])
router.include_router(users_router)
router.include_router(quotas_router)
router.include_router(ai_router)
```
- accepts `capabilities` prop
- shows action buttons based on capabilities
- emits generic events for open/upload/rename/move/delete/analyze
- displays status badges for cached/indexed/analyzing/stale/error
Each sub-router declares its routes WITHOUT a prefix (no `prefix=` arg):
Cloud-specific stores own provider state and API calls; `StorageBrowser` stays store-agnostic.
```python
# backend/api/admin/users.py
router = APIRouter()
### Byte Cache and Offload
@router.get("/users") # resolves to /api/admin/users
@router.post("/users") # resolves to /api/admin/users
@router.patch("/users/{id}/status")
...
```
Cache flow:
`main.py` changes from:
1. User opens/previews/analyzes cloud item.
2. Backend verifies ownership and provider metadata.
3. Backend downloads bytes into a bounded cache or streams through memory if small.
4. Extraction/preview reads from cache.
5. Cache entry records item id, etag/version, byte size, last access, expiry.
6. Eviction removes bytes once unpinned and stale or over budget.
```python
from api.admin import router as admin_router
app.include_router(admin_router)
```
The cache is not the source of truth. Provider metadata and extracted/indexed data are.
to the identical line — because `api/admin/__init__.py` exports `router` with the same name. **No URL changes. No consumer changes.**
### Analysis Flow
### What moves where
1. User selects file/folder/provider and clicks Analyze.
2. Backend resolves selection into cloud items, stores/updates metadata rows, and enqueues Celery jobs.
3. Jobs hydrate bytes as needed, extract text, classify topics, optionally create embeddings, and persist analysis state.
4. Search reads only persisted text/vector/topic metadata; it should not download cloud bytes during query.
5. Opening a result hydrates bytes on demand if preview/download needs them.
| Endpoints | Target file | Pydantic models that move with it |
|---|---|---|
| GET/POST /users, PATCH /users/{id}/status, POST /users/{id}/password-reset, DELETE /users/{id} | `admin/users.py` | `UserCreate`, `UserStatusUpdate`, `UserDeleteConfirm`, `_user_to_dict` |
| GET/PATCH /users/{id}/quota, PATCH /users/{id}/ai-config | `admin/quotas.py` | `QuotaUpdate`, `UserAiConfigUpdate` |
| GET/PUT /ai-config, POST /ai-config/test-connection, GET /ai-config/models | `admin/ai.py` | `SystemAiConfigUpdate`, `TestConnectionRequest`, `_ai_config_to_dict` |
## Build Order
`POST /topics` (create system topic) stays in `admin/users.py` — it is short and has no unique models. Alternatively it can go in its own `admin/topics.py`; a decision either way is fine.
### Shared helpers
`_DEFAULT_QUOTA_BYTES`, `CloudConnectionOut`, and any imports shared across sub-routers can live in `backend/api/admin/shared.py` and be imported by each sub-router. Do not duplicate them.
### Files changed
| File | Change type |
|---|---|
| `backend/api/admin.py` | DELETED — replaced by package |
| `backend/api/admin/__init__.py` | CREATED — aggregating router |
| `backend/api/admin/users.py` | CREATED |
| `backend/api/admin/quotas.py` | CREATED |
| `backend/api/admin/ai.py` | CREATED |
| `backend/api/admin/shared.py` | CREATED (optional, holds shared constants/helpers) |
| `backend/main.py` | NO CHANGE — import path `api.admin.router` remains valid |
---
## Integration Point 2: Vue Admin Layout
### Current layout switch
`App.vue`:
```html
<AuthLayout v-if="route.meta.layout === 'auth'" />
<div v-else class="flex h-screen overflow-hidden">
<AppSidebar />
<main class="flex-1 overflow-y-auto">
<router-view />
</main>
</div>
```
### How to add AdminLayout without breaking anything
Extend the `v-if` chain to a three-way switch using `route.meta.layout`:
```html
<AuthLayout v-if="route.meta.layout === 'auth'" />
<AdminLayout v-else-if="route.meta.layout === 'admin'" />
<div v-else class="flex h-screen overflow-hidden">
<AppSidebar />
<main class="flex-1 overflow-y-auto">
<router-view />
</main>
</div>
```
`AdminLayout.vue` renders its own sidebar and a `<router-view />`. It is structurally identical in shape to the main layout but with admin-specific nav items.
### Router changes
Current single admin route:
```js
{ path: '/admin', component: () => import('../views/AdminView.vue'), meta: { requiresAdmin: true } }
```
Replace with a route subtree:
```js
{
path: '/admin',
component: AdminLayout,
meta: { requiresAdmin: true, layout: 'admin' },
children: [
{ path: '', redirect: '/admin/users' },
{ path: 'users', component: () => import('../views/admin/AdminUsersView.vue') },
{ path: 'quotas', component: () => import('../views/admin/AdminQuotasView.vue') },
{ path: 'ai', component: () => import('../views/admin/AdminAiView.vue') },
{ path: 'audit', component: () => import('../views/admin/AdminAuditView.vue') },
],
}
```
### Meta inheritance caveat
Vue Router 4 does NOT automatically merge `meta` from parent to children — `route.meta` on a child route only contains what is declared on that child. `App.vue` currently reads `route.meta.layout` directly. For the child routes to inherit `layout: 'admin'`, the check in App.vue must walk `route.matched`:
```js
// App.vue — layout resolution
const layout = computed(() => route.matched.find(r => r.meta.layout)?.meta.layout)
```
Likewise, the navigation guard in `router/index.js` must change from `to.meta.requiresAdmin` to `to.matched.some(r => r.meta.requiresAdmin)`. The current guard already handles `.meta.public` the same way — so this is a one-line change pattern that's already established in the router.
### AdminLayout.vue structure
```
AdminLayout.vue
└── AdminSidebar.vue (nav links: Users, Quotas, AI Config, Audit Log)
└── <router-view />
```
`AdminSidebar.vue` is NOT `AppSidebar.vue`. Do not reuse or extend `AppSidebar`. The admin sidebar is a new, independent component with admin-specific nav.
### Views created
Each current tab component becomes a thin view wrapper (following the View -> Smart -> Presentational hierarchy):
| New file | Wraps existing component |
|---|---|
| `views/admin/AdminUsersView.vue` | `AdminUsersTab.vue` (kept unchanged as smart component) |
| `views/admin/AdminQuotasView.vue` | `AdminQuotasTab.vue` |
| `views/admin/AdminAiView.vue` | `AdminAiConfigTab.vue` |
| `views/admin/AdminAuditView.vue` | `AuditLogTab.vue` |
The existing tab components (`AdminUsersTab.vue`, `AdminQuotasTab.vue`, etc.) are NOT renamed or restructured in this step. They become the smart components rendered inside the new thin view wrappers.
### Files changed
| File | Change type |
|---|---|
| `frontend/src/App.vue` | MODIFIED — add `v-else-if` for admin layout; meta layout resolution uses `route.matched` |
| `frontend/src/router/index.js` | MODIFIED — /admin replaced with children subtree; guard uses `to.matched` |
| `frontend/src/layouts/AdminLayout.vue` | CREATED |
| `frontend/src/components/layout/AdminSidebar.vue` | CREATED |
| `frontend/src/views/AdminView.vue` | DELETED — replaced by subtree |
| `frontend/src/views/admin/AdminUsersView.vue` | CREATED (thin wrapper) |
| `frontend/src/views/admin/AdminQuotasView.vue` | CREATED (thin wrapper) |
| `frontend/src/views/admin/AdminAiView.vue` | CREATED (thin wrapper) |
| `frontend/src/views/admin/AdminAuditView.vue` | CREATED (thin wrapper) |
| `frontend/src/components/admin/AdminUsersTab.vue` | KEPT (smart component, unchanged) |
| `frontend/src/components/admin/AdminQuotasTab.vue` | KEPT |
| `frontend/src/components/admin/AdminAiConfigTab.vue` | KEPT |
| `frontend/src/components/admin/AuditLogTab.vue` | KEPT |
---
## Integration Point 3: client.js Decomposition
### Current import pattern
Every consumer does one of:
```js
import * as api from '../api/client.js' // namespace import — stores, views, most components
import { specificFn } from '../api/client.js' // named import — a few components
```
35+ files import from `client.js`. A decomposition that changes the import path in all 35 files is a large, purely mechanical refactor with meaningful test/review cost for zero user-visible benefit.
### Minimal, non-breaking approach: barrel re-export
Create domain modules alongside `client.js`, then have `client.js` re-export everything so no consumer changes:
```
frontend/src/api/
client.js — request() transport function + re-exports from domain files
documents.js — document API calls (import request from './client.js')
topics.js
auth.js
admin.js
folders.js
shares.js
cloud.js
```
`client.js` becomes the HTTP transport layer and a re-export barrel:
```js
// client.js (after decomposition) — transport only + re-exports
async function request(path, options = {}) { ... }
export async function fetchDocumentContent(docId, options = {}) { ... }
export * from './documents.js'
export * from './topics.js'
export * from './auth.js'
export * from './admin.js'
export * from './folders.js'
export * from './shares.js'
export * from './cloud.js'
```
Each domain file imports `request` from `client.js`:
```js
// documents.js
import { request } from './client.js'
export function listDocuments(...) { return request('/api/documents?...') }
```
This means `import * as api from '../api/client.js'` continues to work for all 35 existing consumers. The decomposition is fully internal.
### Test mock compatibility
Existing Vitest tests mock `api/client.js` at the module level (`vi.mock('../../api/client.js')`). Because the re-exports are in `client.js`, mocking `client.js` still intercepts all domain functions. No test changes needed.
### Files changed
| File | Change type |
|---|---|
| `frontend/src/api/client.js` | MODIFIED — split out domain functions, keep re-exports |
| `frontend/src/api/documents.js` | CREATED |
| `frontend/src/api/topics.js` | CREATED |
| `frontend/src/api/auth.js` | CREATED (name does not conflict with `stores/auth.js` — different path) |
| `frontend/src/api/admin.js` | CREATED |
| `frontend/src/api/folders.js` | CREATED |
| `frontend/src/api/shares.js` | CREATED |
| `frontend/src/api/cloud.js` | CREATED |
| All 35 consumer files | NO CHANGE |
---
## Integration Point 4: Icon Strategy
### Current state
There are 58 inline `<svg>` blocks in `frontend/src/components/` and 8 more in `frontend/src/views/`. Each is a 3-6 line block with `stroke`, `viewBox`, `class`, and a `<path>`. The same path data appears multiple times for the same icon (e.g., the document icon in the sidebar, document card, and document view are separate copies of the same path string).
### Options assessed
**Option A: SVG sprite sheet**
A `<symbol>`-based sprite in `index.html` with `<use href="#icon-name" />` in components. Minimal bundle impact, no component overhead. Drawback: requires a build step or manual maintenance of the sprite file; poor DX for adding new icons during the visual redesign.
**Option B: `AppIcon.vue` component with a `name` prop**
A Vue component that maps `name` to an inline `<svg>` block. All SVG paths live in one file. Zero external dependencies. Easy to add icons.
**Option C: lucide-vue-next**
Purpose-built icon library for Tailwind projects. Tree-shakeable individual icon components: `<FileText class="w-4 h-4" />`. Maintained, typed, comprehensive. Avoids maintaining path data entirely.
**Recommendation: Option B (`AppIcon.vue`) unless a dependency is explicitly approved.**
The project already has 66 SVG paths hand-chosen to match the visual style. Switching to lucide means auditing that every lucide icon matches the expected visual. During a visual redesign milestone that may be desirable, but it is a separate decision. The minimal refactor that eliminates duplication without changing visual output is Option B.
**Named slots are wrong for this problem.** Named slots solve "inject different markup into different zones of a layout". This problem is "render the correct path data for a given icon name". A `name` prop is the correct API.
**Option B implementation:**
```
frontend/src/components/ui/AppIcon.vue
```
```html
<template>
<svg :class="$attrs.class" fill="none" stroke="currentColor" viewBox="0 0 24 24"
aria-hidden="true">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" :d="paths[name]" />
</svg>
</template>
<script>
export default {
inheritAttrs: false,
props: { name: { type: String, required: true } },
data: () => ({
paths: {
document: 'M9 12h6m-6 4h6m2 5H7a2 2 0 01-2-2V5...',
folder: 'M3 7a2 2 0 012-2h4l2 2h6a2 2 0 012 2v...',
// etc.
}
})
}
</script>
```
Usage: `<AppIcon name="document" class="w-4 h-4 mr-2 shrink-0" />`
### Files changed (Option B)
| File | Change type |
|---|---|
| `frontend/src/components/ui/AppIcon.vue` | CREATED |
| ~35 components/views with inline SVGs | MODIFIED — replace `<svg>...</svg>` with `<AppIcon name="..." class="..." />` |
---
## Suggested Build Order
Dependencies between the four changes determine order. A change that other changes depend on must land first.
```
Step 1 — Backend admin router decomposition
Step 2 — API client decomposition (client.js -> domain files)
Step 3 — Admin layout + route subtree (frontend)
Step 4 — Icon refactor
```
### Rationale
**Step 1 (backend) first:**
- No frontend dependencies. All existing tests pass with zero behavior change (same URLs, same auth, same import path in `main.py`).
- Risk: lowest. Pure internal restructuring.
**Step 2 (client.js) second:**
- No dependency on the admin layout.
- New admin view files created in Step 3 can import from `api/admin.js` specifically rather than the monolith.
- Internal-only change. All 35 consumer files unchanged. All existing mock-based tests unchanged.
- Risk: low.
**Step 3 (admin layout + route subtree) third:**
- Highest risk: modifies `App.vue` (central layout switch), `router/index.js` (all navigation), creates new layout files.
- The existing smart components (`AdminUsersTab.vue` etc.) are not touched — they are simply wrapped by new thin view files.
- Navigation guard logic for `requiresAdmin` and `layout` meta must be re-tested after this step.
- Risk: medium.
**Step 4 (icon refactor) last:**
- Pure cosmetic. Zero behavior change.
- Can be done incrementally (file by file) without blocking anything.
- Risk: lowest.
### Parallelism
Steps 1 and 2 are independent and can be executed simultaneously — they touch no shared files.
Steps 3 and 4 overlap on the admin component files (icons are replaced in Step 4, structure is set in Step 3). Complete Step 3 first to avoid merge conflicts.
---
## Component Architecture: Existing vs. Target
### Current (wrong)
```
App.vue
└── AppSidebar (always present)
└── AdminView.vue (tabs rendered in user sidebar context)
└── AdminUsersTab.vue
└── AdminQuotasTab.vue
└── AdminAiConfigTab.vue
└── AuditLogTab.vue
```
### Target (correct)
```
App.vue (layout switch)
└── AuthLayout.vue (for meta.layout === 'auth')
└── AdminLayout.vue (for meta.layout === 'admin')
└── AdminSidebar.vue
└── <router-view />
→ AdminUsersView.vue (thin — passes props to AdminUsersTab)
→ AdminQuotasView.vue
→ AdminAiView.vue
→ AdminAuditView.vue
└── Implicit main layout (v-else)
└── AppSidebar.vue
└── <router-view />
→ FileManagerView, DocumentView, TopicsView, ...
```
The smart components (`AdminUsersTab.vue` etc.) are unchanged. The view layer above them is what changes.
---
## What Is New vs. Modified: Complete Inventory
### New files
```
backend/api/admin/__init__.py
backend/api/admin/users.py
backend/api/admin/quotas.py
backend/api/admin/ai.py
backend/api/admin/shared.py (optional — shared constants/helpers)
frontend/src/layouts/AdminLayout.vue
frontend/src/components/layout/AdminSidebar.vue
frontend/src/views/admin/AdminUsersView.vue
frontend/src/views/admin/AdminQuotasView.vue
frontend/src/views/admin/AdminAiView.vue
frontend/src/views/admin/AdminAuditView.vue
frontend/src/api/documents.js
frontend/src/api/topics.js
frontend/src/api/auth.js
frontend/src/api/admin.js
frontend/src/api/folders.js
frontend/src/api/shares.js
frontend/src/api/cloud.js
frontend/src/components/ui/AppIcon.vue (Option B only)
```
### Modified files
```
backend/main.py (no functional change — import stays identical)
frontend/src/App.vue (add v-else-if; use route.matched for layout)
frontend/src/router/index.js (admin route becomes children subtree; guard uses to.matched)
frontend/src/api/client.js (gutted to transport + re-exports)
All ~35 components with inline SVGs (icons replaced with AppIcon or lucide)
```
### Deleted files
```
backend/api/admin.py (replaced by package)
frontend/src/views/AdminView.vue (replaced by route subtree)
```
---
## Minimal vs. Proper Assessment
| Change | Is the minimal solution also the proper one? |
|---|---|
| Backend decomposition | Yes. Sub-router via `__init__.py` aggregator is both minimal and the standard FastAPI pattern. |
| client.js decomposition | Yes. Re-export barrel means zero consumer churn while achieving internal separation. |
| Admin layout | Yes. Three-way `v-if` in App.vue + new layout file is 20 lines of change in existing files. |
| Icon refactor | The minimal solution (AppIcon.vue) is simpler than lucide but adds maintenance burden for the path map. lucide is more proper but adds a dependency. Call must be made by project owner. |
---
## Sources
- Inspected source: `backend/main.py`, `backend/api/admin.py`, `frontend/src/App.vue`, `frontend/src/router/index.js`, `frontend/src/layouts/AuthLayout.vue`, `frontend/src/api/client.js`, `frontend/src/views/AdminView.vue`, `frontend/src/components/layout/AppSidebar.vue`
- FastAPI sub-router `include_router` nesting: standard FastAPI router composition (HIGH confidence)
- Vue Router 4 `route.matched` for meta inheritance: Vue Router 4 documented behavior (HIGH confidence)
- lucide-vue-next: tree-shakeable Vue 3 icon library (MEDIUM confidence — verified as active, Tailwind-compatible)
1. Capability model + provider action interface + tests.
2. Cloud item metadata/index schema + browse persistence.
3. Unified browser actions for cloud files/folders.
4. Cloud analysis jobs and cache/offload service.
5. Unified smart search over local and analyzed cloud documents.
6. Provider change detection, stale markers, and polish.