---
phase: 07-redo-and-optimize-llm-integration
plan: 03
type: execute
wave: 3
depends_on:
- 07-02
files_modified:
- backend/ai/anthropic_provider.py
- backend/ai/__init__.py
- backend/services/classifier.py
- backend/services/ai_config.py
- backend/tests/test_ai_providers.py
- backend/tests/test_ai_config.py
autonomous: true
requirements:
- D-03
- D-04
- D-06
- D-07
- D-12
- D-13
must_haves:
truths:
- "AnthropicProvider stores self._client = AsyncAnthropic(...) in __init__ and never recreates it"
- "AnthropicProvider.classify() passes output_config={\"format\": {\"type\": \"json_schema\", \"schema\": _CLASSIFICATION_SCHEMA}} to messages.create()"
- "AnthropicProvider.suggest_topics() passes output_config={\"format\": {\"type\": \"json_schema\", \"schema\": _SUGGESTIONS_SCHEMA}} to messages.create()"
- "AnthropicProvider has no MAX_AI_CHARS constant and uses self._truncate(document_text)"
- "classifier.classify_document calls load_provider_config(session) and passes a ProviderConfig to get_provider"
- "load_provider_config returns the real ProviderConfig from ai/provider_config.py (stub removed)"
- "Registry get_provider now passes base_url+context_chars to AnthropicProvider as well"
artifacts:
- path: "backend/ai/anthropic_provider.py"
provides: "Singleton AsyncAnthropic + output_config structured output + smart truncation"
contains: "output_config"
- path: "backend/services/classifier.py"
provides: "ProviderConfig-driven classifier (no inline dict construction)"
contains: "load_provider_config"
- path: "backend/services/ai_config.py"
provides: "ProviderConfig stub replaced by import from ai.provider_config"
contains: "from ai.provider_config import ProviderConfig"
key_links:
- from: "backend/services/classifier.py::classify_document"
to: "backend/services/ai_config.py::load_provider_config"
via: "async session.execute path"
pattern: "await load_provider_config"
- from: "backend/ai/anthropic_provider.py::classify"
to: "anthropic SDK messages.create"
via: "output_config kwarg"
pattern: "output_config"
- from: "backend/ai/__init__.py::get_provider anthropic branch"
to: "AnthropicProvider.__init__"
via: "widened ctor signature (context_chars)"
pattern: "context_chars"
---
Complete the provider refactor by giving AnthropicProvider the same singleton+context_chars+output_config treatment as the OpenAI side, and replace the classifier's inline `_settings` dict with a real ProviderConfig sourced from the database.
Purpose: Resolve D-03 (Anthropic structured output via `output_config.format.type="json_schema"`) using the constrained-decoding API that GA'd in anthropic SDK 0.95+, finish the D-07 singleton client cleanup, and close out D-06 by routing the classifier through `load_provider_config()` instead of the legacy env-var dict shim.
Output: Refactored anthropic_provider.py (output_config + singleton + truncation, MAX_AI_CHARS removed), classifier.py talking to ai_config.load_provider_config, ai_config.py stub removed in favour of the real ProviderConfig, registry updated to pass base_url/context_chars to AnthropicProvider, and the matching Anthropic + classifier integration tests promoted from xfail.
@$HOME/.claude/get-shit-done/workflows/execute-plan.md
@$HOME/.claude/get-shit-done/templates/summary.md
@.planning/PROJECT.md
@.planning/phases/07-redo-and-optimize-llm-integration/07-CONTEXT.md
@.planning/phases/07-redo-and-optimize-llm-integration/07-RESEARCH.md
@.planning/phases/07-redo-and-optimize-llm-integration/07-PATTERNS.md
@.planning/phases/07-redo-and-optimize-llm-integration/07-01-SUMMARY.md
@.planning/phases/07-redo-and-optimize-llm-integration/07-02-SUMMARY.md
@backend/ai/anthropic_provider.py
@backend/ai/openai_provider.py
@backend/ai/provider_config.py
@backend/ai/__init__.py
@backend/services/classifier.py
@backend/services/ai_config.py
From backend/ai/anthropic_provider.py (current state before this plan):
- `MAX_AI_CHARS = 8_000` module constant
- `class AnthropicProvider(AIProvider)`
- `def __init__(self, api_key: str, model: str = "claude-sonnet-4-6")` — narrow ctor
- `def _client(self)` — to be removed
- `async def classify(...)` / `async def suggest_topics(...)` / `async def health_check(...)` calling `await client.messages.create(...)`
Anthropic SDK output_config schema (07-RESEARCH.md D-03):
- `messages.create(..., output_config={"format": {"type": "json_schema", "schema": }})`
- Response read from `response.content[0].text` only when `response.stop_reason == "end_turn"`; otherwise call parse_classification("") for graceful degradation
- Required schema fields all listed in `required` array; `additionalProperties: False`
From backend/ai/provider_config.py (introduced in Plan 02):
- `ProviderConfig(BaseModel)` with provider_id/api_key/base_url/model/context_chars
- `PROVIDER_DEFAULTS["anthropic"] = {"base_url": None, "model": "claude-sonnet-4-6", "context_chars": 180000}`
From backend/services/ai_config.py (introduced in Plan 01):
- `async def load_provider_config(session) -> ProviderConfig | None` — currently imports the _ProviderConfigStub; this plan switches it to import from ai.provider_config
From backend/services/classifier.py (current — lines 57-64):
- Inline `_settings = {"active_provider": _ai_provider, "providers": {_ai_provider: {"model": _ai_model}}}` then `provider = get_provider(_settings)`
- Old signature `async def classify_document(document_id, session, ai_provider: str | None = None, ai_model: str | None = None)`
Module-level _DEFAULT_SYSTEM_PROMPT remains in classifier.py.
Task 1: AnthropicProvider — singleton, output_config, truncation
backend/ai/anthropic_provider.py
backend/ai/openai_provider.py
backend/ai/utils.py
backend/ai/provider_config.py
backend/ai/__init__.py
- AnthropicProvider.__init__(api_key, model, context_chars, base_url=None) stores self._client = AsyncAnthropic(api_key=api_key) once; base_url is accepted but ignored (anthropic SDK uses its default; signature widened so the factory can pass uniformly)
- AnthropicProvider has no MAX_AI_CHARS constant, no `_client(self)` method
- AnthropicProvider has _truncate(text) implementing the 60/40 split
- Module-level _CLASSIFICATION_SCHEMA and _SUGGESTIONS_SCHEMA dicts match the structures defined in 07-RESEARCH.md (assigned_topics + new_topic_suggestions + reasoning required for classification; suggested_topics required for suggestions; additionalProperties False)
- classify() and suggest_topics() pass output_config={"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA / _SUGGESTIONS_SCHEMA}} to messages.create
- When response.stop_reason != "end_turn" the raw text is replaced by "" so parse_classification/parse_suggestions degrade gracefully
- get_provider in ai/__init__.py now invokes AnthropicProvider with api_key+model+context_chars+base_url (uniform call signature)
- test_ai_providers.py::test_anthropic_structured_output promoted to passing (mocks anthropic.AsyncAnthropic.messages.create and asserts output_config kwarg)
Refactor backend/ai/anthropic_provider.py: delete `MAX_AI_CHARS = 8_000`. Change __init__ signature to `def __init__(self, api_key: str, model: str, context_chars: int, base_url: str | None = None)`. Set self._api_key, self._model, self._context_chars, then self._client = anthropic.AsyncAnthropic(api_key=self._api_key). Delete the `def _client(self)` method. Add `def _truncate(self, text: str) -> str` body identical to OpenAIProvider._truncate (60/40 split using self._context_chars).
Add module-level constants:
`_CLASSIFICATION_SCHEMA = {"type": "object", "properties": {"assigned_topics": {"type": "array", "items": {"type": "string"}}, "new_topic_suggestions": {"type": "array", "items": {"type": "string"}}, "reasoning": {"type": "string"}}, "required": ["assigned_topics", "new_topic_suggestions"], "additionalProperties": False}`
`_SUGGESTIONS_SCHEMA = {"type": "object", "properties": {"suggested_topics": {"type": "array", "items": {"type": "string"}}}, "required": ["suggested_topics"], "additionalProperties": False}`
Rewrite classify(): truncated = self._truncate(document_text); topics_str composed as before; user_msg as before; await self._client.messages.create(model=self._model, max_tokens=1024, system=system_prompt, messages=[{"role":"user","content":user_msg}], output_config={"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}}); raw = response.content[0].text if response.content and getattr(response, "stop_reason", "end_turn") == "end_turn" else ""; return parse_classification(raw).
Rewrite suggest_topics(): same shape but output_config uses _SUGGESTIONS_SCHEMA and returns parse_suggestions(raw).
Rewrite health_check(): await self._client.messages.create(model=self._model, max_tokens=8, messages=[{"role":"user","content":"ping"}]); return True on success, False on Exception. Do not pass output_config in health_check (the response shape doesn't matter; this just verifies credentials/connectivity).
Update backend/ai/__init__.py get_provider(): in the registry branch for "anthropic", call `cls(api_key=effective_api_key, model=effective_model, context_chars=effective_context_chars, base_url=effective_base_url)`. Remove the special-case branch from Plan 02 (the comment "AnthropicProvider does not take base_url" is no longer true after this task).
Promote test_ai_providers.py::test_anthropic_structured_output: build an AnthropicProvider(api_key="k", model="claude-sonnet-4-6", context_chars=100); patch ai.anthropic_provider.anthropic.AsyncAnthropic with a MagicMock whose messages.create is an AsyncMock returning a stub response (object with .content=[MagicMock(text='{"assigned_topics":[],"new_topic_suggestions":[]}')] and .stop_reason="end_turn"); call await provider.classify("doc", [], "sys"); assert mock_create.await_args.kwargs["output_config"] == {"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}}. Remove the @pytest.mark.xfail decorator.
cd backend && pytest tests/test_ai_providers.py::test_anthropic_structured_output -x -v && grep -c 'MAX_AI_CHARS' backend/ai/anthropic_provider.py
- Source assertion: grep -v '^#' backend/ai/anthropic_provider.py | grep -c 'MAX_AI_CHARS' returns 0
- Source assertion: backend/ai/anthropic_provider.py contains `self._client = anthropic.AsyncAnthropic(`, `output_config=`, `_CLASSIFICATION_SCHEMA`, `_SUGGESTIONS_SCHEMA`
- Source assertion: backend/ai/anthropic_provider.py does NOT contain `def _client(self)`
- Source assertion: backend/ai/__init__.py passes `context_chars=` and `base_url=` to AnthropicProvider
- Behavior: pytest backend/tests/test_ai_providers.py::test_anthropic_structured_output exits 0
- Behavior: response.stop_reason "max_tokens" or "refusal" → raw == "" → parse_classification("") returns an empty ClassificationResult (no crash)
AnthropicProvider mirrors the OpenAI-side refactor (singleton, truncation, no MAX_AI_CHARS); classify/suggest call output_config with the two schemas; registry passes uniform kwargs; test_anthropic_structured_output is green.
Task 2: Classifier wired to load_provider_config + ai_config stub removal
backend/services/classifier.py
backend/services/ai_config.py
backend/ai/provider_config.py
backend/ai/__init__.py
backend/tasks/document_tasks.py
backend/tests/test_classifier.py
backend/tests/test_ai_config.py
- backend/services/ai_config.py imports `from ai.provider_config import ProviderConfig` at module top (lazy import inside load_provider_config removed)
- The _ProviderConfigStub class defined in ai_config.py is deleted
- backend/services/classifier.py calls `config = await load_provider_config(session)` and `provider = get_provider(config)` — no inline `_settings = {...}` dict construction remains
- classify_document keeps optional ai_provider/ai_model kwargs (per-user override from ADMIN-05 still honored): when either kwarg is non-None, build a ProviderConfig override (start from PROVIDER_DEFAULTS[user provider], merge user-provided model) and skip the DB load
- When load_provider_config returns None (no active row), fall back to a ProviderConfig built from settings.default_ai_provider/default_ai_model + PROVIDER_DEFAULTS — preserves existing default-AI behaviour from STATE.md ("Default AI provider is ollama/llama3.2")
- The classifier no longer slices document_text — truncation happens inside provider._truncate()
- tests/test_classifier.py still passes (existing behavior preserved)
- tests/test_ai_config.py::test_load_provider_config and tests/test_ai_config.py::test_api_key_encrypt_decrypt promoted to passing
Edit backend/services/ai_config.py: remove the `_ProviderConfigStub` class entirely; replace the lazy import inside load_provider_config with a module-top `from ai.provider_config import ProviderConfig, PROVIDER_DEFAULTS`. load_provider_config(session) reads the row WHERE is_active = True via select(SystemSettings).where(SystemSettings.is_active.is_(True)) (use `.is_(True)` for SQLAlchemy comparison, not `==`); if no row found return None; otherwise decrypt api_key_enc using settings.cloud_creds_key when not None (else api_key=""); build and return ProviderConfig(provider_id=row.provider_id, api_key=decrypted_api_key, base_url=row.base_url, model=row.model_name, context_chars=row.context_chars). Import AsyncSession from sqlalchemy.ext.asyncio. Import select from sqlalchemy. Use base64 + bytes(settings.cloud_creds_key, "utf-8") if cloud_creds_key is a str — match the existing pattern in cloud_utils.
Edit backend/services/classifier.py: at the top of `async def classify_document`, replace the inline dict construction (lines ~57-64 per 07-PATTERNS.md) with:
1. `config = await load_provider_config(session)` (import `from services.ai_config import load_provider_config` at module top)
2. If ai_provider is not None (per-user override path): build `config = ProviderConfig(provider_id=ai_provider, model=ai_model or PROVIDER_DEFAULTS.get(ai_provider, {}).get("model", ""), api_key="", base_url=None, context_chars=PROVIDER_DEFAULTS.get(ai_provider, {}).get("context_chars", 8000))` (per-user overrides do not carry an API key in this codebase — the system_settings api_key is still authoritative; when the per-user override changes the provider away from the active system provider, api_key remains empty and get_provider falls back to "not-needed". Document this in a comment: "per-user override does not carry an api_key — admin must configure each provider's key in system_settings".)
3. If config is None: fallback ProviderConfig(provider_id=app_settings.default_ai_provider, model=app_settings.default_ai_model, base_url=None, context_chars=PROVIDER_DEFAULTS.get(app_settings.default_ai_provider, {}).get("context_chars", 8000), api_key="").
4. `provider = get_provider(config)` (unchanged call).
Apply the same load/override/fallback logic in `suggest_topics_for_document`. Remove any remaining `document_text[:MAX_AI_CHARS]` slices; pass the full document_text to provider.classify/provider.suggest_topics (the provider truncates internally).
Promote backend/tests/test_ai_config.py::test_load_provider_config and ::test_api_key_encrypt_decrypt. For test_api_key_encrypt_decrypt: a simple unit test using a 32-byte master key, encrypt_api_key + decrypt_api_key round-trip with provider_id="openai", assert plaintext recovered; assert encrypting with provider_id="anthropic" yields different ciphertext (domain salt isolation). For test_load_provider_config: requires DB — gate with `pytest.importorskip("psycopg")` and `@pytest.mark.skipif(not os.getenv("INTEGRATION"), reason="needs PostgreSQL")`; insert a SystemSettings row with is_active=True via the async session fixture; encrypt a test api_key; await load_provider_config(session); assert returned ProviderConfig has the expected fields and decrypted api_key. Remove xfail decorators from both.
cd backend && pytest tests/test_ai_config.py::test_api_key_encrypt_decrypt tests/test_classifier.py -x -v
- Source assertion: backend/services/ai_config.py contains `from ai.provider_config import ProviderConfig` at module level (grep on the top 30 lines)
- Source assertion: backend/services/ai_config.py does NOT contain `_ProviderConfigStub`
- Source assertion: backend/services/classifier.py contains `await load_provider_config(` and `get_provider(config)`
- Source assertion: backend/services/classifier.py does NOT contain `"active_provider"` or `"providers": {`
- Source assertion: grep -v '^#' backend/services/classifier.py | grep -c 'MAX_AI_CHARS' returns 0
- Behavior: pytest backend/tests/test_classifier.py exits 0
- Behavior: pytest backend/tests/test_ai_config.py::test_api_key_encrypt_decrypt exits 0
- Behavior: INTEGRATION=1 pytest backend/tests/test_ai_config.py::test_load_provider_config exits 0 when DB is available (test is skipped otherwise)
ai_config.py imports ProviderConfig directly; stub class gone; classifier uses load_provider_config + per-user/system fallback; truncation lives only in providers; two test_ai_config.py tests promoted; existing test_classifier.py still green.
## Trust Boundaries
| Boundary | Description |
|----------|-------------|
| services/ai_config.load_provider_config → providers | decrypted api_key crosses here in-memory only; never serialized |
| AnthropicProvider → anthropic.com API | api_key transits over TLS; SDK manages outbound auth |
## STRIDE Threat Register
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|-----------|----------|-----------|-------------|-----------------|
| T-07-06 | Information Disclosure | classifier per-user override path | mitigate | per-user override does not carry api_key; system_settings is the only key store; api_key never leaves backend services tier |
| T-07-07 | Tampering | Anthropic output_config grammar limit | accept | Pitfall 5 RESEARCH.md — schemas are simple (≤4 required fields, no unions); will not exceed Anthropic's 24-optional / 16-union limits |
| T-07-08 | Denial of Service | Anthropic stop_reason "refusal"/"max_tokens" | mitigate | classify() falls back to parse_classification("") which returns empty ClassificationResult; document status set by caller (Plan 04 retry logic) |
| T-07-SC | Tampering | anthropic SDK pin raised to >=0.95.0 | accept | RESEARCH.md confirms package legitimacy (official Anthropic SDK, ~3 yrs old); pip audit re-runs at phase gate |
- grep -v '^#' backend/ai/anthropic_provider.py | grep -c 'MAX_AI_CHARS' returns 0.
- grep -F 'output_config' backend/ai/anthropic_provider.py returns 2+ matches (one per classify/suggest call).
- grep -F '_ProviderConfigStub' backend/services/ai_config.py returns 0.
- pytest backend/tests/test_ai_providers.py::test_anthropic_structured_output backend/tests/test_classifier.py backend/tests/test_ai_config.py::test_api_key_encrypt_decrypt exits 0.
- pytest backend/tests/ -v shows no new failures.
- AnthropicProvider singleton + output_config + truncation + no MAX_AI_CHARS.
- classifier.classify_document driven by load_provider_config with per-user override + env fallback.
- ai_config.py stub removed; real ProviderConfig used everywhere.
- 3 previously-xfailed tests now pass (test_anthropic_structured_output, test_api_key_encrypt_decrypt, test_load_provider_config under INTEGRATION=1).
- Existing test_classifier.py still green.