curo1305andClaude Sonnet 4.6
3df62506c9
5 waves: system_settings DB + HKDF encryption (01), ProviderConfig + GenericOpenAIProvider + singleton client fix (02), Anthropic output_config + classifier wiring (03), Celery retry 30/90/270s + re-queue endpoint (04), admin AI panel + DocumentCard badge + human checkpoint (05). All 18 decisions D-01..D-18 covered. Plan checker passed after 1 revision round (4 blockers fixed: D-02/D-14 coverage, D-11 Vitest tests, Plan 05 files_modified). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
678 lines
28 KiB
Markdown
678 lines
28 KiB
Markdown
# Phase 7: Redo and Optimize LLM Integration - Pattern Map
|
||
|
||
**Mapped:** 2026-06-02
|
||
**Files analyzed:** 14 new/modified files
|
||
**Analogs found:** 13 / 14
|
||
|
||
---
|
||
|
||
## File Classification
|
||
|
||
| New/Modified File | Role | Data Flow | Closest Analog | Match Quality |
|
||
|---|---|---|---|---|
|
||
| `backend/ai/provider_config.py` | config/model | transform | `backend/api/admin.py` Pydantic models (lines 74–119) | role-match |
|
||
| `backend/ai/generic_openai_provider.py` | service | request-response | `backend/ai/openai_provider.py` | exact |
|
||
| `backend/ai/openai_provider.py` (modify) | service | request-response | self (current file) | exact |
|
||
| `backend/ai/anthropic_provider.py` (modify) | service | request-response | `backend/ai/openai_provider.py` | exact |
|
||
| `backend/ai/ollama_provider.py` (modify) | service | request-response | self + `backend/ai/ollama_provider.py` | exact |
|
||
| `backend/ai/lmstudio_provider.py` (modify) | service | request-response | `backend/ai/ollama_provider.py` | exact |
|
||
| `backend/ai/__init__.py` (refactor) | factory | request-response | self (current file) | exact |
|
||
| `backend/db/models.py` (add SystemSettings) | model | CRUD | existing models in `backend/db/models.py` (CloudConnection, lines 299–318) | exact |
|
||
| `backend/migrations/versions/0005_system_settings.py` | migration | batch | `backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py` | exact |
|
||
| `backend/services/ai_config.py` | service | CRUD + encrypt | `backend/storage/cloud_utils.py` | exact |
|
||
| `backend/services/classifier.py` (refactor) | service | request-response | self (current file) | exact |
|
||
| `backend/tasks/document_tasks.py` (refactor) | task | event-driven | self (current file) | exact |
|
||
| `frontend/src/components/admin/AdminAiConfigTab.vue` (modify) | component | request-response | self + `frontend/src/components/admin/AdminUsersTab.vue` | exact |
|
||
| `frontend/src/components/documents/DocumentCard.vue` (modify) | component | request-response | self (current file) | exact |
|
||
|
||
---
|
||
|
||
## Pattern Assignments
|
||
|
||
### `backend/ai/provider_config.py` (NEW — config model)
|
||
|
||
**Analog:** `backend/api/admin.py` Pydantic request models (lines 74–119)
|
||
|
||
**Imports pattern** (from analog lines 27–32):
|
||
```python
|
||
from __future__ import annotations
|
||
from pydantic import BaseModel, field_validator
|
||
from typing import Optional
|
||
```
|
||
|
||
**Core model pattern** (analog lines 74–119 — `AiConfigUpdate`, `QuotaUpdate`):
|
||
```python
|
||
# From backend/api/admin.py lines 102–105
|
||
class AiConfigUpdate(BaseModel):
|
||
ai_provider: Optional[str] = None
|
||
ai_model: Optional[str] = None
|
||
```
|
||
New file follows same `BaseModel` subclass pattern with typed fields and defaults. No `from_attributes` needed (not an ORM response model).
|
||
|
||
**No analog for `PROVIDER_DEFAULTS` dict** — use RESEARCH.md Pattern 2 directly (the dict of known base_urls and context_chars per provider_id).
|
||
|
||
---
|
||
|
||
### `backend/ai/generic_openai_provider.py` (NEW — service, request-response)
|
||
|
||
**Analog:** `backend/ai/openai_provider.py` (entire file — 72 lines)
|
||
|
||
**Imports pattern** (analog lines 1–4):
|
||
```python
|
||
from openai import AsyncOpenAI
|
||
from ai.base import AIProvider, ClassificationResult
|
||
from ai.utils import parse_classification, parse_suggestions
|
||
```
|
||
|
||
**Subclass pattern** (analog lines 8–16):
|
||
```python
|
||
# From backend/ai/openai_provider.py lines 8–16
|
||
class OpenAIProvider(AIProvider):
|
||
def __init__(self, api_key: str, model: str = "gpt-4o", base_url=None):
|
||
self._api_key = api_key
|
||
self._model = model
|
||
self._base_url = base_url
|
||
|
||
def _client(self) -> AsyncOpenAI:
|
||
return AsyncOpenAI(api_key=self._api_key or "placeholder", base_url=self._base_url)
|
||
```
|
||
`GenericOpenAIProvider` subclasses `OpenAIProvider`. The constructor adds `context_chars: int` and `supports_json_mode: bool = True`. The `_client()` method is replaced by `self._client = AsyncOpenAI(...)` stored in `__init__` (D-07 singleton fix). The `api_key` placeholder changes from `"placeholder"` to `"not-needed"` (openai SDK 2.34+ pitfall).
|
||
|
||
**Core classify pattern** (analog lines 17–37 — override with json_mode):
|
||
```python
|
||
# From backend/ai/openai_provider.py lines 17–37
|
||
async def classify(self, document_text, existing_topics, system_prompt):
|
||
topics_str = ", ".join(existing_topics) if existing_topics else "(none yet)"
|
||
user_msg = (
|
||
f"Existing topics: [{topics_str}]\n\n"
|
||
f"Document text:\n{document_text[:MAX_AI_CHARS]}"
|
||
)
|
||
response = await self._client().chat.completions.create(
|
||
model=self._model,
|
||
max_tokens=1024,
|
||
messages=[
|
||
{"role": "system", "content": system_prompt},
|
||
{"role": "user", "content": user_msg},
|
||
],
|
||
)
|
||
raw = response.choices[0].message.content or ""
|
||
return parse_classification(raw)
|
||
```
|
||
`GenericOpenAIProvider` overrides this to: (1) call `self._truncate(document_text)` instead of `[:MAX_AI_CHARS]`, (2) add `response_format={"type": "json_object"}` when `self.supports_json_mode` is `True`, (3) call `self._client.chat.completions.create(...)` on the stored singleton (no `()`).
|
||
|
||
**`_truncate` helper pattern** — no existing analog; copy verbatim from RESEARCH.md Pattern (smart truncation):
|
||
```python
|
||
def _truncate(self, text: str) -> str:
|
||
"""First 60% + last 40% of context window (D-13)."""
|
||
limit = self._context_chars
|
||
if len(text) <= limit:
|
||
return text
|
||
head_len = int(limit * 0.6)
|
||
tail_len = limit - head_len
|
||
return text[:head_len] + "\n[...truncated...]\n" + text[-tail_len:]
|
||
```
|
||
|
||
**health_check pattern** (analog lines 60–69):
|
||
```python
|
||
# From backend/ai/openai_provider.py lines 60–69
|
||
async def health_check(self) -> bool:
|
||
try:
|
||
await self._client().chat.completions.create(
|
||
model=self._model, max_tokens=5,
|
||
messages=[{"role": "user", "content": "ping"}],
|
||
)
|
||
return True
|
||
except Exception:
|
||
return False
|
||
```
|
||
In `GenericOpenAIProvider` call `self._client.chat.completions.create(...)` (no `()`).
|
||
|
||
---
|
||
|
||
### `backend/ai/openai_provider.py` (MODIFY — singleton + context_chars)
|
||
|
||
**Analog:** self (current file)
|
||
|
||
**Changes from current pattern:**
|
||
|
||
Current `__init__` (lines 9–15):
|
||
```python
|
||
# CURRENT — creates client per-call (anti-pattern to replace)
|
||
def __init__(self, api_key: str, model: str = "gpt-4o", base_url=None):
|
||
self._api_key = api_key
|
||
self._model = model
|
||
self._base_url = base_url
|
||
|
||
def _client(self) -> AsyncOpenAI:
|
||
return AsyncOpenAI(api_key=self._api_key or "placeholder", base_url=self._base_url)
|
||
```
|
||
|
||
Replace with (D-07 singleton pattern):
|
||
```python
|
||
# TARGET — singleton stored in __init__
|
||
def __init__(self, api_key: str, model: str, base_url: str | None, context_chars: int):
|
||
self._api_key = api_key or "not-needed" # SDK 2.34+ rejects empty string
|
||
self._model = model
|
||
self._base_url = base_url
|
||
self._context_chars = context_chars
|
||
self._client = AsyncOpenAI(api_key=self._api_key, base_url=self._base_url)
|
||
```
|
||
|
||
Remove `MAX_AI_CHARS = 8_000` constant (line 5). Replace `document_text[:MAX_AI_CHARS]` with `self._truncate(document_text)`. Add `_truncate()` method (same as GenericOpenAIProvider above). Remove `_client()` method entirely. Update all `self._client()` call-sites to `self._client` (no `()`).
|
||
|
||
---
|
||
|
||
### `backend/ai/anthropic_provider.py` (MODIFY — singleton + output_config)
|
||
|
||
**Analog:** self (current file) + `backend/ai/openai_provider.py`
|
||
|
||
**Current anti-pattern** (lines 9–14):
|
||
```python
|
||
# CURRENT — creates client per-call (anti-pattern to replace)
|
||
def __init__(self, api_key: str, model: str = "claude-sonnet-4-6"):
|
||
self._api_key = api_key
|
||
self._model = model
|
||
|
||
def _client(self):
|
||
return anthropic.AsyncAnthropic(api_key=self._api_key)
|
||
```
|
||
|
||
**Target pattern** (D-07 + D-03 output_config):
|
||
```python
|
||
# TARGET — singleton + context_chars + output_config
|
||
def __init__(self, api_key: str, model: str, context_chars: int):
|
||
self._api_key = api_key
|
||
self._model = model
|
||
self._context_chars = context_chars
|
||
self._client = anthropic.AsyncAnthropic(api_key=self._api_key)
|
||
```
|
||
|
||
Remove `MAX_AI_CHARS = 8_000` (line 5). Replace `document_text[:MAX_AI_CHARS]` with `self._truncate(document_text)`. Add `_truncate()` method. Remove `_client()` method. Replace `client = self._client()` call-sites with direct `self._client`.
|
||
|
||
**output_config pattern for classify** (from RESEARCH.md D-03):
|
||
```python
|
||
# Add to messages.create() call in classify():
|
||
output_config={
|
||
"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}
|
||
},
|
||
```
|
||
`_CLASSIFICATION_SCHEMA` defined as module-level constant. `response.content[0].text` read only when `response.stop_reason == "end_turn"`; otherwise fall back to `parse_classification("")`.
|
||
|
||
---
|
||
|
||
### `backend/ai/ollama_provider.py` / `lmstudio_provider.py` (MODIFY — context_chars)
|
||
|
||
**Analog:** `backend/ai/ollama_provider.py` (entire file — 10 lines)
|
||
|
||
**Current pattern** (lines 1–11):
|
||
```python
|
||
from ai.openai_provider import OpenAIProvider
|
||
|
||
class OllamaProvider(OpenAIProvider):
|
||
def __init__(self, base_url: str = "http://host.docker.internal:11434", model: str = "llama3.2"):
|
||
super().__init__(
|
||
api_key="ollama",
|
||
model=model,
|
||
base_url=base_url.rstrip("/") + "/v1",
|
||
)
|
||
```
|
||
These one-liner subclasses become aliases in the provider registry (both route to `GenericOpenAIProvider`). The files themselves may be reduced to empty stubs or removed if `get_provider()` registry handles instantiation directly. If kept, update `super().__init__()` to pass `context_chars=8_000` as a default.
|
||
|
||
---
|
||
|
||
### `backend/ai/__init__.py` (REFACTOR — registry pattern)
|
||
|
||
**Analog:** self (current file — lines 1–35)
|
||
|
||
**Current anti-pattern** (lines 8–35):
|
||
```python
|
||
# CURRENT — flat if/elif chain
|
||
def get_provider(settings: dict) -> AIProvider:
|
||
active = settings.get("active_provider", "lmstudio")
|
||
providers = settings.get("providers", {})
|
||
cfg = providers.get(active, {})
|
||
if active == "anthropic":
|
||
return AnthropicProvider(...)
|
||
elif active == "openai":
|
||
...
|
||
else:
|
||
raise ValueError(f"Unknown AI provider: {active}")
|
||
```
|
||
|
||
**Target registry pattern** (from RESEARCH.md get_provider() Registry Pattern):
|
||
```python
|
||
# TARGET — O(1) dict lookup, typed ProviderConfig input
|
||
from ai.provider_config import ProviderConfig
|
||
|
||
_REGISTRY: dict[str, type] = {
|
||
"openai": OpenAIProvider,
|
||
"anthropic": AnthropicProvider,
|
||
"gemini": GenericOpenAIProvider,
|
||
# ... all providers
|
||
}
|
||
|
||
def get_provider(config: ProviderConfig) -> AIProvider:
|
||
cls = _REGISTRY.get(config.provider_id)
|
||
if cls is None:
|
||
raise ValueError(f"Unknown AI provider: {config.provider_id!r}")
|
||
# AnthropicProvider does not take base_url — handle in factory
|
||
if config.provider_id == "anthropic":
|
||
return cls(api_key=config.api_key or "not-needed", model=config.model, context_chars=config.context_chars)
|
||
return cls(api_key=config.api_key or "not-needed", model=config.model,
|
||
base_url=config.base_url, context_chars=config.context_chars)
|
||
```
|
||
|
||
---
|
||
|
||
### `backend/db/models.py` — ADD `SystemSettings` model
|
||
|
||
**Analog:** `backend/db/models.py` — `CloudConnection` model (lines 299–318)
|
||
|
||
**Closest existing model pattern** (lines 299–318):
|
||
```python
|
||
# From backend/db/models.py lines 299–318
|
||
class CloudConnection(Base):
|
||
__tablename__ = "cloud_connections"
|
||
|
||
id: Mapped[uuid.UUID] = mapped_column(
|
||
UUID(as_uuid=True), primary_key=True, default=uuid.uuid4
|
||
)
|
||
user_id: Mapped[uuid.UUID] = mapped_column(
|
||
UUID(as_uuid=True), ForeignKey("users.id", ondelete="CASCADE"), nullable=False,
|
||
)
|
||
provider: Mapped[str] = mapped_column(String, nullable=False)
|
||
credentials_enc: Mapped[str] = mapped_column(Text, nullable=False)
|
||
status: Mapped[str] = mapped_column(String, nullable=False, default="ACTIVE")
|
||
connected_at: Mapped[datetime] = mapped_column(
|
||
TIMESTAMP(timezone=True), nullable=False, server_default=func.now()
|
||
)
|
||
__table_args__ = (Index("ix_cloud_connections_user", "user_id"),)
|
||
```
|
||
|
||
**Target SystemSettings model** — same `Mapped[]` typed column style, same `server_default=func.now()` for timestamps, same `UniqueConstraint` in `__table_args__`. No FK (global table, not per-user). Columns: `id` (UUID PK), `provider_id` (String UNIQUE NOT NULL), `api_key_enc` (Text nullable), `base_url` (Text nullable), `model_name` (Text NOT NULL default ""), `context_chars` (Integer NOT NULL default 8000), `is_active` (Boolean NOT NULL default False), `created_at` (TIMESTAMP), `updated_at` (TIMESTAMP).
|
||
|
||
**Import additions needed** (from top of models.py, lines 25–38):
|
||
```python
|
||
# Already imported — no new imports needed:
|
||
# Boolean, String, Text, TIMESTAMP, UniqueConstraint, Integer
|
||
# UUID, Mapped, mapped_column, func
|
||
```
|
||
|
||
---
|
||
|
||
### `backend/migrations/versions/0005_system_settings.py` (NEW — migration)
|
||
|
||
**Analog:** `backend/migrations/versions/0004_phase4_pdf_open_mode_tsvector.py` (entire file — 86 lines)
|
||
|
||
**Header pattern** (analog lines 1–36):
|
||
```python
|
||
"""Phase 4 schema additions: ..."""
|
||
from __future__ import annotations
|
||
import sqlalchemy as sa
|
||
from alembic import op
|
||
|
||
revision = "0004"
|
||
down_revision = "0003"
|
||
branch_labels = None
|
||
depends_on = None
|
||
```
|
||
New migration uses: `revision = "0005"`, `down_revision = "0004"`.
|
||
|
||
**`op.create_table()` pattern** (from analog + `backend/migrations/versions/0001_initial_schema.py` lines 38–55):
|
||
```python
|
||
# From 0001 lines 38–55 — op.create_table with postgresql.UUID
|
||
op.create_table("users",
|
||
sa.Column("id", postgresql.UUID(as_uuid=True), nullable=False),
|
||
sa.Column("handle", sa.String(), nullable=False),
|
||
...
|
||
sa.PrimaryKeyConstraint("id"),
|
||
sa.UniqueConstraint("email", name="uq_users_email"),
|
||
)
|
||
```
|
||
New migration uses `sa.dialects.postgresql.UUID(as_uuid=True)` for the `id` column, `sa.text("gen_random_uuid()")` as `server_default` for `id`, and `sa.UniqueConstraint("provider_id", name="uq_system_settings_provider_id")` in the column list.
|
||
|
||
**downgrade pattern** (analog lines 77–86):
|
||
```python
|
||
def downgrade() -> None:
|
||
op.drop_table("system_settings")
|
||
```
|
||
|
||
---
|
||
|
||
### `backend/services/ai_config.py` (NEW — service, CRUD + encrypt)
|
||
|
||
**Analog:** `backend/storage/cloud_utils.py` (entire file — 181 lines)
|
||
|
||
**Module docstring pattern** (analog lines 1–19):
|
||
```python
|
||
"""
|
||
Cloud storage shared utilities for DocuVault.
|
||
|
||
Security design:
|
||
HKDF credential encryption (D-18, CLOUD-02): _derive_fernet_key() creates a FRESH
|
||
HKDF instance on every call. The cryptography library raises AlreadyFinalized if
|
||
.derive() is called twice on the same instance.
|
||
...
|
||
"""
|
||
from __future__ import annotations
|
||
import base64
|
||
from cryptography.fernet import Fernet
|
||
from cryptography.hazmat.primitives import hashes
|
||
from cryptography.hazmat.primitives.kdf.hkdf import HKDF
|
||
```
|
||
|
||
**HKDF key derivation pattern** (analog lines 114–141 — COPY VERBATIM, change salt + info):
|
||
```python
|
||
# From backend/storage/cloud_utils.py lines 114–141
|
||
def _derive_fernet_key(master_key: bytes, user_id: str) -> Fernet:
|
||
# FRESH HKDF instance on every call — AlreadyFinalized pitfall
|
||
hkdf = HKDF(
|
||
algorithm=hashes.SHA256(),
|
||
length=32,
|
||
salt=user_id.encode("utf-8"),
|
||
info=b"cloud-credentials", # <-- change to b"ai-provider-settings" in new file
|
||
)
|
||
raw_key: bytes = hkdf.derive(master_key)
|
||
fernet_key = base64.urlsafe_b64encode(raw_key)
|
||
return Fernet(fernet_key)
|
||
```
|
||
In `ai_config.py`, function is renamed `_derive_ai_settings_key(master_key, provider_id)`. The salt is `provider_id.encode("utf-8")`. The info is `b"ai-provider-settings"` — different from `b"cloud-credentials"` (domain separation). Same AlreadyFinalized comment must be retained.
|
||
|
||
**encrypt/decrypt pattern** (analog lines 144–181):
|
||
```python
|
||
# From backend/storage/cloud_utils.py lines 144–161
|
||
def encrypt_credentials(master_key: bytes, user_id: str, credentials: dict) -> str:
|
||
f = _derive_fernet_key(master_key, user_id)
|
||
plaintext = json.dumps(credentials).encode("utf-8")
|
||
return f.encrypt(plaintext).decode("utf-8")
|
||
|
||
def decrypt_credentials(master_key: bytes, user_id: str, credentials_enc: str) -> dict:
|
||
f = _derive_fernet_key(master_key, user_id)
|
||
plaintext = f.decrypt(credentials_enc.encode("utf-8"))
|
||
return json.loads(plaintext)
|
||
```
|
||
In `ai_config.py`: `encrypt_api_key(master_key, provider_id, api_key: str) -> str` (encrypt a plain string, not a dict — use `.encode()` / `.decode()` directly without `json.dumps`). `decrypt_api_key(master_key, provider_id, api_key_enc: str) -> str`.
|
||
|
||
**Additional function `load_provider_config(session)`** — no analog; use SQLAlchemy `select()` + `where(SystemSettings.is_active == True)` to load the active row, decrypt the key, construct `ProviderConfig`. Pattern for `session.execute(select(...).where(...))` from `backend/api/admin.py` lines 163–167:
|
||
```python
|
||
# From backend/api/admin.py lines 163–167
|
||
result = await session.execute(
|
||
select(User).order_by(User.created_at.desc())
|
||
)
|
||
users = result.scalars().all()
|
||
```
|
||
|
||
---
|
||
|
||
### `backend/services/classifier.py` (REFACTOR — ProviderConfig injection)
|
||
|
||
**Analog:** self (current file — 128 lines)
|
||
|
||
**Current inline dict construction** (lines 57–64 — the anti-pattern to replace):
|
||
```python
|
||
# CURRENT — lines 57–64 in backend/services/classifier.py
|
||
_ai_provider = ai_provider or app_settings.default_ai_provider
|
||
_ai_model = ai_model or app_settings.default_ai_model
|
||
system_prompt = app_settings.system_prompt or _DEFAULT_SYSTEM_PROMPT
|
||
_settings = {
|
||
"active_provider": _ai_provider,
|
||
"providers": {_ai_provider: {"model": _ai_model}},
|
||
}
|
||
provider = get_provider(_settings)
|
||
```
|
||
|
||
**Target pattern** (replace with `load_provider_config()` call):
|
||
```python
|
||
# TARGET — ProviderConfig loaded from DB (D-06)
|
||
from services.ai_config import load_provider_config
|
||
config = await load_provider_config(session)
|
||
# Per-user override: if user has ai_provider set, build a ProviderConfig override
|
||
provider = get_provider(config)
|
||
```
|
||
|
||
Remove `MAX_AI_CHARS = 8_000` (line 28). Remove `text[:MAX_AI_CHARS]` (line 85) — truncation now lives in the provider's `_truncate()`. Remove the `ai_provider: str | None` and `ai_model: str | None` kwargs from both `classify_document()` and `suggest_topics_for_document()` signatures (or retain them as optional overrides for per-user assignment — see RESEARCH.md Runtime State Inventory note about per-user `ai_provider` column).
|
||
|
||
---
|
||
|
||
### `backend/tasks/document_tasks.py` (REFACTOR — Celery retry)
|
||
|
||
**Analog:** self (current file — 173 lines)
|
||
|
||
**Current task decorator** (lines 22–25):
|
||
```python
|
||
# CURRENT — no bind, no retry
|
||
@celery_app.task(name="tasks.document_tasks.extract_and_classify")
|
||
def extract_and_classify(document_id: str) -> dict:
|
||
return asyncio.run(_run(document_id))
|
||
```
|
||
|
||
**Target decorator** (D-09 — bind=True + max_retries):
|
||
```python
|
||
# TARGET
|
||
@celery_app.task(
|
||
name="tasks.document_tasks.extract_and_classify",
|
||
bind=True,
|
||
max_retries=3,
|
||
)
|
||
def extract_and_classify(self, document_id: str) -> dict:
|
||
try:
|
||
return asyncio.run(_run(document_id))
|
||
except _ClassificationError as exc:
|
||
countdowns = [30, 90, 270]
|
||
countdown = countdowns[min(self.request.retries, 2)]
|
||
raise self.retry(exc=exc, countdown=countdown)
|
||
```
|
||
|
||
`_ClassificationError` is a new module-level sentinel exception class (defined above the task). The existing `except Exception as e: doc.status = "classification_failed"` block in `_run()` (lines 109–124) is replaced: instead of catching and returning a dict, `_run()` raises `_ClassificationError(str(e))` for classification failures specifically (not for extract/storage failures which still `return {...}`). On `MaxRetriesExceededError`, write `classification_failed` to DB via a new `_mark_classification_failed(document_id)` async helper.
|
||
|
||
**Existing non-retryable return pattern to preserve** (lines 47–106 — storage/extract failures):
|
||
```python
|
||
# From backend/tasks/document_tasks.py lines 47–50 — keep unchanged
|
||
try:
|
||
doc_uuid = _uuid.UUID(document_id)
|
||
except ValueError:
|
||
return {"document_id": document_id, "status": "invalid_id"}
|
||
```
|
||
All `return {... "status": "extract_failed" ...}` patterns in the retrieval and extraction steps are preserved — they are not retried.
|
||
|
||
---
|
||
|
||
### `frontend/src/components/admin/AdminAiConfigTab.vue` (MODIFY — add global config section)
|
||
|
||
**Analog:** self (current file — 136 lines) + `frontend/src/components/admin/AdminUsersTab.vue`
|
||
|
||
**Current imports/setup pattern** (lines 76–94):
|
||
```javascript
|
||
// From frontend/src/components/admin/AdminAiConfigTab.vue lines 76–94
|
||
import { ref, reactive, onMounted } from 'vue'
|
||
import * as api from '../../api/client.js'
|
||
|
||
const users = ref([])
|
||
const loading = ref(false)
|
||
const loadError = ref(null)
|
||
const savingId = ref(null)
|
||
const savedId = ref(null)
|
||
const configs = reactive({})
|
||
```
|
||
|
||
**Save function pattern** (lines 96–114):
|
||
```javascript
|
||
// From frontend/src/components/admin/AdminAiConfigTab.vue lines 96–114
|
||
async function saveConfig(userId) {
|
||
savingId.value = userId
|
||
savedId.value = null
|
||
try {
|
||
await api.adminUpdateAiConfig(userId, configs[userId].provider || null, configs[userId].model || null)
|
||
savedId.value = userId
|
||
setTimeout(() => { if (savedId.value === userId) savedId.value = null }, 1500)
|
||
} catch (e) {
|
||
loadError.value = e.message
|
||
} finally {
|
||
savingId.value = null
|
||
}
|
||
}
|
||
```
|
||
New global config section adds a second `ref()` block (`systemConfig`, `savingSystem`, `savedSystem`) and a `saveSystemConfig()` function following the same try/catch/finally shape. New API functions `api.getAiConfig()` and `api.saveAiConfig(body)` are called instead of `adminUpdateAiConfig`.
|
||
|
||
**Loading state template pattern** (lines 2–9):
|
||
```html
|
||
<!-- From AdminAiConfigTab.vue lines 2–9 — reuse for global config section -->
|
||
<div v-if="loading" class="bg-white rounded-xl border border-gray-200 p-8 text-center">
|
||
<div class="flex items-center justify-center gap-2 text-gray-400 text-sm">
|
||
<span class="animate-spin rounded-full border-2 border-current border-t-transparent w-4 h-4"></span>
|
||
Loading AI config…
|
||
</div>
|
||
</div>
|
||
```
|
||
|
||
**Pitfall 6 guard:** The new "System AI Providers" section is added as a separate block *above* the existing per-user table, not replacing it. The existing `users` / `configs` reactive state and `saveConfig(userId)` function remain untouched.
|
||
|
||
---
|
||
|
||
### `frontend/src/components/documents/DocumentCard.vue` (MODIFY — classification failed badge + Re-analyze button)
|
||
|
||
**Analog:** self (current file — 142 lines)
|
||
|
||
**Existing status badge pattern** (lines 31–34 — shared indicator):
|
||
```html
|
||
<!-- From DocumentCard.vue lines 31–34 — copy badge style -->
|
||
<div v-if="doc.is_shared" class="mt-2">
|
||
<span class="bg-indigo-50 text-indigo-600 text-xs font-medium px-2 py-1 rounded-full">Shared</span>
|
||
</div>
|
||
```
|
||
|
||
**Target badge** — add alongside the shared badge block:
|
||
```html
|
||
<!-- Add after line 34 -->
|
||
<div v-if="doc.status === 'classification_failed'" class="mt-2 flex items-center gap-2">
|
||
<span class="bg-red-50 text-red-600 text-xs font-medium px-2 py-1 rounded-full">Classification failed</span>
|
||
<button
|
||
@click.stop="reanalyze"
|
||
:disabled="reanalyzing"
|
||
class="text-xs text-indigo-600 hover:text-indigo-700 font-semibold disabled:opacity-50"
|
||
>
|
||
<span v-if="reanalyzing">Re-analyzing…</span>
|
||
<span v-else>Re-analyze</span>
|
||
</button>
|
||
</div>
|
||
```
|
||
|
||
**Action button import pattern** (existing lines 97–98):
|
||
```javascript
|
||
// From DocumentCard.vue lines 97–98
|
||
import { moveDocument } from '../../api/client.js'
|
||
```
|
||
Add `classifyDocument` to the same import. Add `const reanalyzing = ref(false)` to the `ref()` block. Add `reanalyze()` async function following the same try/catch shape as `moveToFolder()` (lines 128–135):
|
||
```javascript
|
||
// From DocumentCard.vue lines 128–135 — copy shape
|
||
async function moveToFolder(folderId) {
|
||
showFolderPicker.value = false
|
||
try {
|
||
await moveDocument(props.doc.id, folderId)
|
||
} catch (e) {
|
||
console.error('Move failed:', e.message)
|
||
}
|
||
}
|
||
```
|
||
|
||
---
|
||
|
||
## Shared Patterns
|
||
|
||
### HKDF/Fernet Encryption
|
||
**Source:** `backend/storage/cloud_utils.py` lines 114–181
|
||
**Apply to:** `backend/services/ai_config.py`
|
||
```python
|
||
# Pattern: fresh HKDF instance per call — never reuse (AlreadyFinalized pitfall)
|
||
hkdf = HKDF(
|
||
algorithm=hashes.SHA256(),
|
||
length=32,
|
||
salt=<scope_id>.encode("utf-8"), # user_id for cloud, provider_id for AI settings
|
||
info=b"<domain-label>", # b"cloud-credentials" vs b"ai-provider-settings"
|
||
)
|
||
raw_key: bytes = hkdf.derive(master_key)
|
||
fernet_key = base64.urlsafe_b64encode(raw_key)
|
||
return Fernet(fernet_key)
|
||
```
|
||
|
||
### Admin Endpoint Whitelist Helper
|
||
**Source:** `backend/api/admin.py` lines 52–68 (`_user_to_dict`)
|
||
**Apply to:** New admin AI config endpoints in `backend/api/admin.py`
|
||
```python
|
||
# From backend/api/admin.py lines 52–68 — safe whitelist pattern
|
||
def _user_to_dict(user: User) -> dict:
|
||
"""Return a safe subset — never includes password_hash, credentials_enc."""
|
||
return {
|
||
"id": str(user.id),
|
||
"handle": user.handle,
|
||
# ... whitelisted fields only
|
||
}
|
||
```
|
||
New `_ai_config_to_dict(row: SystemSettings) -> dict` follows same pattern: includes `provider_id, base_url, model_name, context_chars, is_active, updated_at`, **never** `api_key_enc`.
|
||
|
||
### Admin Endpoint Auth + Audit Log
|
||
**Source:** `backend/api/admin.py` lines 153–167, 388–410
|
||
**Apply to:** New `GET /api/admin/ai-config` and `PUT /api/admin/ai-config` endpoints
|
||
```python
|
||
# From backend/api/admin.py lines 153–158
|
||
@router.get("/users")
|
||
async def list_users(
|
||
session: AsyncSession = Depends(get_db),
|
||
_admin: User = Depends(get_current_admin), # <-- mandatory on every admin endpoint
|
||
) -> dict:
|
||
```
|
||
```python
|
||
# Audit log write pattern — from lines 390–399
|
||
await write_audit_log(
|
||
session,
|
||
event_type="admin.ai_config_changed",
|
||
user_id=None, # global config — no target user
|
||
actor_id=_admin.id,
|
||
resource_id=None,
|
||
ip_address=get_client_ip(request),
|
||
metadata_={"provider_id": body.provider_id},
|
||
)
|
||
```
|
||
|
||
### Frontend API Call Pattern
|
||
**Source:** `frontend/src/api/client.js` lines 277–284
|
||
**Apply to:** New `getAiConfig()`, `saveAiConfig()` functions in `client.js`
|
||
```javascript
|
||
// From frontend/src/api/client.js lines 277–284
|
||
export function adminUpdateAiConfig(id, provider, model) {
|
||
return request(`/api/admin/users/${id}/ai-config`, {
|
||
method: 'PATCH',
|
||
headers: { 'Content-Type': 'application/json' },
|
||
body: JSON.stringify({ ai_provider: provider, ai_model: model }),
|
||
})
|
||
}
|
||
```
|
||
New functions follow same `request()` wrapper pattern with `method: 'GET'` or `method: 'PUT'` and `Content-Type: application/json`.
|
||
|
||
### Celery Task Structure (asyncio.run bridge)
|
||
**Source:** `backend/tasks/document_tasks.py` lines 22–25, 127–136
|
||
**Apply to:** Refactored `extract_and_classify` (same file)
|
||
```python
|
||
# From document_tasks.py lines 127–136 — second task shows the same pattern
|
||
@celery_app.task(name="tasks.document_tasks.cleanup_abandoned_uploads")
|
||
def cleanup_abandoned_uploads() -> dict:
|
||
return asyncio.run(_cleanup_abandoned())
|
||
```
|
||
The `asyncio.run()` bridge pattern is the standard — sync task def wraps an async `_run()`. The retry exception must be raised from the sync layer, not inside `asyncio.run()`.
|
||
|
||
---
|
||
|
||
## No Analog Found
|
||
|
||
| File | Role | Data Flow | Reason |
|
||
|---|---|---|---|
|
||
| `_truncate()` method pattern | utility | transform | No existing truncation helper in the codebase; defined in RESEARCH.md Pattern (smart truncation 60/40 split) |
|
||
|
||
---
|
||
|
||
## Metadata
|
||
|
||
**Analog search scope:** `backend/ai/`, `backend/storage/`, `backend/api/`, `backend/services/`, `backend/tasks/`, `backend/db/`, `backend/migrations/versions/`, `frontend/src/components/admin/`, `frontend/src/components/documents/`, `frontend/src/api/`
|
||
**Files scanned:** 14 source files read directly
|
||
**Pattern extraction date:** 2026-06-02
|