kite/.planning/phases/07-redo-and-optimize-llm-integration/07-03-PLAN.md

phase, plan, type, wave, depends_on, files_modified, autonomous, requirements, must_haves

phase	plan	type	wave	depends_on	files_modified	autonomous	requirements	must_haves
07-redo-and-optimize-llm-integration	03	execute	3	07-02	backend/ai/anthropic_provider.py backend/ai/__init__.py backend/services/classifier.py backend/services/ai_config.py backend/tests/test_ai_providers.py backend/tests/test_ai_config.py	true	D-03 D-04 D-06 D-07 D-12 D-13	truths artifacts key_links AnthropicProvider stores self._client = AsyncAnthropic(...) in __init__ and never recreates it AnthropicProvider.classify() passes output_config={"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}} to messages.create() AnthropicProvider.suggest_topics() passes output_config={"format": {"type": "json_schema", "schema": _SUGGESTIONS_SCHEMA}} to messages.create() AnthropicProvider has no MAX_AI_CHARS constant and uses self._truncate(document_text) classifier.classify_document calls load_provider_config(session) and passes a ProviderConfig to get_provider load_provider_config returns the real ProviderConfig from ai/provider_config.py (stub removed) Registry get_provider now passes base_url+context_chars to AnthropicProvider as well path provides contains backend/ai/anthropic_provider.py Singleton AsyncAnthropic + output_config structured output + smart truncation output_config path provides contains backend/services/classifier.py ProviderConfig-driven classifier (no inline dict construction) load_provider_config path provides contains backend/services/ai_config.py ProviderConfig stub replaced by import from ai.provider_config from ai.provider_config import ProviderConfig from to via pattern backend/services/classifier.py::classify_document backend/services/ai_config.py::load_provider_config async session.execute path await load_provider_config from to via pattern backend/ai/anthropic_provider.py::classify anthropic SDK messages.create output_config kwarg output_config from to via pattern backend/ai/__init__.py::get_provider anthropic branch AnthropicProvider.__init__ widened ctor signature (context_chars) context_chars

Complete the provider refactor by giving AnthropicProvider the same singleton+context_chars+output_config treatment as the OpenAI side, and replace the classifier's inline `_settings` dict with a real ProviderConfig sourced from the database.

Purpose: Resolve D-03 (Anthropic structured output via output_config.format.type="json_schema") using the constrained-decoding API that GA'd in anthropic SDK 0.95+, finish the D-07 singleton client cleanup, and close out D-06 by routing the classifier through load_provider_config() instead of the legacy env-var dict shim. Output: Refactored anthropic_provider.py (output_config + singleton + truncation, MAX_AI_CHARS removed), classifier.py talking to ai_config.load_provider_config, ai_config.py stub removed in favour of the real ProviderConfig, registry updated to pass base_url/context_chars to AnthropicProvider, and the matching Anthropic + classifier integration tests promoted from xfail.

<execution_context> @$HOME/.claude/get-shit-done/workflows/execute-plan.md @$HOME/.claude/get-shit-done/templates/summary.md </execution_context>

@.planning/PROJECT.md @.planning/phases/07-redo-and-optimize-llm-integration/07-CONTEXT.md @.planning/phases/07-redo-and-optimize-llm-integration/07-RESEARCH.md @.planning/phases/07-redo-and-optimize-llm-integration/07-PATTERNS.md @.planning/phases/07-redo-and-optimize-llm-integration/07-01-SUMMARY.md @.planning/phases/07-redo-and-optimize-llm-integration/07-02-SUMMARY.md @backend/ai/anthropic_provider.py @backend/ai/openai_provider.py @backend/ai/provider_config.py @backend/ai/__init__.py @backend/services/classifier.py @backend/services/ai_config.py

From backend/ai/anthropic_provider.py (current state before this plan):

• MAX_AI_CHARS = 8_000 module constant
• class AnthropicProvider(AIProvider)
• def __init__(self, api_key: str, model: str = "claude-sonnet-4-6") — narrow ctor
• def _client(self) — to be removed
• async def classify(...) / async def suggest_topics(...) / async def health_check(...) calling await client.messages.create(...)

Anthropic SDK output_config schema (07-RESEARCH.md D-03):

• messages.create(..., output_config={"format": {"type": "json_schema", "schema": <dict>}})
• Response read from response.content[0].text only when response.stop_reason == "end_turn"; otherwise call parse_classification("") for graceful degradation
• Required schema fields all listed in required array; additionalProperties: False

From backend/ai/provider_config.py (introduced in Plan 02):

• ProviderConfig(BaseModel) with provider_id/api_key/base_url/model/context_chars
• PROVIDER_DEFAULTS["anthropic"] = {"base_url": None, "model": "claude-sonnet-4-6", "context_chars": 180000}

From backend/services/ai_config.py (introduced in Plan 01):

• async def load_provider_config(session) -> ProviderConfig | None — currently imports the _ProviderConfigStub; this plan switches it to import from ai.provider_config

From backend/services/classifier.py (current — lines 57-64):

• Inline _settings = {"active_provider": _ai_provider, "providers": {_ai_provider: {"model": _ai_model}}} then provider = get_provider(_settings)
• Old signature async def classify_document(document_id, session, ai_provider: str | None = None, ai_model: str | None = None)

Module-level _DEFAULT_SYSTEM_PROMPT remains in classifier.py.

Task 1: AnthropicProvider — singleton, output_config, truncation backend/ai/anthropic_provider.py backend/ai/openai_provider.py backend/ai/utils.py backend/ai/provider_config.py backend/ai/__init__.py - AnthropicProvider.__init__(api_key, model, context_chars, base_url=None) stores self._client = AsyncAnthropic(api_key=api_key) once; base_url is accepted but ignored (anthropic SDK uses its default; signature widened so the factory can pass uniformly) - AnthropicProvider has no MAX_AI_CHARS constant, no `_client(self)` method - AnthropicProvider has _truncate(text) implementing the 60/40 split - Module-level _CLASSIFICATION_SCHEMA and _SUGGESTIONS_SCHEMA dicts match the structures defined in 07-RESEARCH.md (assigned_topics + new_topic_suggestions + reasoning required for classification; suggested_topics required for suggestions; additionalProperties False) - classify() and suggest_topics() pass output_config={"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA / _SUGGESTIONS_SCHEMA}} to messages.create - When response.stop_reason != "end_turn" the raw text is replaced by "" so parse_classification/parse_suggestions degrade gracefully - get_provider in ai/__init__.py now invokes AnthropicProvider with api_key+model+context_chars+base_url (uniform call signature) - test_ai_providers.py::test_anthropic_structured_output promoted to passing (mocks anthropic.AsyncAnthropic.messages.create and asserts output_config kwarg) Refactor backend/ai/anthropic_provider.py: delete `MAX_AI_CHARS = 8_000`. Change __init__ signature to `def __init__(self, api_key: str, model: str, context_chars: int, base_url: str | None = None)`. Set self._api_key, self._model, self._context_chars, then self._client = anthropic.AsyncAnthropic(api_key=self._api_key). Delete the `def _client(self)` method. Add `def _truncate(self, text: str) -> str` body identical to OpenAIProvider._truncate (60/40 split using self._context_chars).

Add module-level constants:
`_CLASSIFICATION_SCHEMA = {"type": "object", "properties": {"assigned_topics": {"type": "array", "items": {"type": "string"}}, "new_topic_suggestions": {"type": "array", "items": {"type": "string"}}, "reasoning": {"type": "string"}}, "required": ["assigned_topics", "new_topic_suggestions"], "additionalProperties": False}`
`_SUGGESTIONS_SCHEMA = {"type": "object", "properties": {"suggested_topics": {"type": "array", "items": {"type": "string"}}}, "required": ["suggested_topics"], "additionalProperties": False}`

Rewrite classify(): truncated = self._truncate(document_text); topics_str composed as before; user_msg as before; await self._client.messages.create(model=self._model, max_tokens=1024, system=system_prompt, messages=[{"role":"user","content":user_msg}], output_config={"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}}); raw = response.content[0].text if response.content and getattr(response, "stop_reason", "end_turn") == "end_turn" else ""; return parse_classification(raw).

Rewrite suggest_topics(): same shape but output_config uses _SUGGESTIONS_SCHEMA and returns parse_suggestions(raw).

Rewrite health_check(): await self._client.messages.create(model=self._model, max_tokens=8, messages=[{"role":"user","content":"ping"}]); return True on success, False on Exception. Do not pass output_config in health_check (the response shape doesn't matter; this just verifies credentials/connectivity).

Update backend/ai/__init__.py get_provider(): in the registry branch for "anthropic", call `cls(api_key=effective_api_key, model=effective_model, context_chars=effective_context_chars, base_url=effective_base_url)`. Remove the special-case branch from Plan 02 (the comment "AnthropicProvider does not take base_url" is no longer true after this task).

Promote test_ai_providers.py::test_anthropic_structured_output: build an AnthropicProvider(api_key="k", model="claude-sonnet-4-6", context_chars=100); patch ai.anthropic_provider.anthropic.AsyncAnthropic with a MagicMock whose messages.create is an AsyncMock returning a stub response (object with .content=[MagicMock(text='{"assigned_topics":[],"new_topic_suggestions":[]}')] and .stop_reason="end_turn"); call await provider.classify("doc", [], "sys"); assert mock_create.await_args.kwargs["output_config"] == {"format": {"type": "json_schema", "schema": _CLASSIFICATION_SCHEMA}}. Remove the @pytest.mark.xfail decorator.

cd backend && pytest tests/test_ai_providers.py::test_anthropic_structured_output -x -v && grep -c 'MAX_AI_CHARS' backend/ai/anthropic_provider.py - Source assertion: grep -v '^#' backend/ai/anthropic_provider.py | grep -c 'MAX_AI_CHARS' returns 0 - Source assertion: backend/ai/anthropic_provider.py contains `self._client = anthropic.AsyncAnthropic(`, `output_config=`, `_CLASSIFICATION_SCHEMA`, `_SUGGESTIONS_SCHEMA` - Source assertion: backend/ai/anthropic_provider.py does NOT contain `def _client(self)` - Source assertion: backend/ai/__init__.py passes `context_chars=` and `base_url=` to AnthropicProvider - Behavior: pytest backend/tests/test_ai_providers.py::test_anthropic_structured_output exits 0 - Behavior: response.stop_reason "max_tokens" or "refusal" → raw == "" → parse_classification("") returns an empty ClassificationResult (no crash) AnthropicProvider mirrors the OpenAI-side refactor (singleton, truncation, no MAX_AI_CHARS); classify/suggest call output_config with the two schemas; registry passes uniform kwargs; test_anthropic_structured_output is green. Task 2: Classifier wired to load_provider_config + ai_config stub removal backend/services/classifier.py backend/services/ai_config.py backend/ai/provider_config.py backend/ai/__init__.py backend/tasks/document_tasks.py backend/tests/test_classifier.py backend/tests/test_ai_config.py - backend/services/ai_config.py imports `from ai.provider_config import ProviderConfig` at module top (lazy import inside load_provider_config removed) - The _ProviderConfigStub class defined in ai_config.py is deleted - backend/services/classifier.py calls `config = await load_provider_config(session)` and `provider = get_provider(config)` — no inline `_settings = {...}` dict construction remains - classify_document keeps optional ai_provider/ai_model kwargs (per-user override from ADMIN-05 still honored): when either kwarg is non-None, build a ProviderConfig override (start from PROVIDER_DEFAULTS[user provider], merge user-provided model) and skip the DB load - When load_provider_config returns None (no active row), fall back to a ProviderConfig built from settings.default_ai_provider/default_ai_model + PROVIDER_DEFAULTS — preserves existing default-AI behaviour from STATE.md ("Default AI provider is ollama/llama3.2") - The classifier no longer slices document_text — truncation happens inside provider._truncate() - tests/test_classifier.py still passes (existing behavior preserved) - tests/test_ai_config.py::test_load_provider_config and tests/test_ai_config.py::test_api_key_encrypt_decrypt promoted to passing Edit backend/services/ai_config.py: remove the `_ProviderConfigStub` class entirely; replace the lazy import inside load_provider_config with a module-top `from ai.provider_config import ProviderConfig, PROVIDER_DEFAULTS`. load_provider_config(session) reads the row WHERE is_active = True via select(SystemSettings).where(SystemSettings.is_active.is_(True)) (use `.is_(True)` for SQLAlchemy comparison, not `==`); if no row found return None; otherwise decrypt api_key_enc using settings.cloud_creds_key when not None (else api_key=""); build and return ProviderConfig(provider_id=row.provider_id, api_key=decrypted_api_key, base_url=row.base_url, model=row.model_name, context_chars=row.context_chars). Import AsyncSession from sqlalchemy.ext.asyncio. Import select from sqlalchemy. Use base64 + bytes(settings.cloud_creds_key, "utf-8") if cloud_creds_key is a str — match the existing pattern in cloud_utils.

Edit backend/services/classifier.py: at the top of `async def classify_document`, replace the inline dict construction (lines ~57-64 per 07-PATTERNS.md) with:
1. `config = await load_provider_config(session)` (import `from services.ai_config import load_provider_config` at module top)
2. If ai_provider is not None (per-user override path): build `config = ProviderConfig(provider_id=ai_provider, model=ai_model or PROVIDER_DEFAULTS.get(ai_provider, {}).get("model", ""), api_key="", base_url=None, context_chars=PROVIDER_DEFAULTS.get(ai_provider, {}).get("context_chars", 8000))` (per-user overrides do not carry an API key in this codebase — the system_settings api_key is still authoritative; when the per-user override changes the provider away from the active system provider, api_key remains empty and get_provider falls back to "not-needed". Document this in a comment: "per-user override does not carry an api_key — admin must configure each provider's key in system_settings".)
3. If config is None: fallback ProviderConfig(provider_id=app_settings.default_ai_provider, model=app_settings.default_ai_model, base_url=None, context_chars=PROVIDER_DEFAULTS.get(app_settings.default_ai_provider, {}).get("context_chars", 8000), api_key="").
4. `provider = get_provider(config)` (unchanged call).
Apply the same load/override/fallback logic in `suggest_topics_for_document`. Remove any remaining `document_text[:MAX_AI_CHARS]` slices; pass the full document_text to provider.classify/provider.suggest_topics (the provider truncates internally).

Promote backend/tests/test_ai_config.py::test_load_provider_config and ::test_api_key_encrypt_decrypt. For test_api_key_encrypt_decrypt: a simple unit test using a 32-byte master key, encrypt_api_key + decrypt_api_key round-trip with provider_id="openai", assert plaintext recovered; assert encrypting with provider_id="anthropic" yields different ciphertext (domain salt isolation). For test_load_provider_config: requires DB — gate with `pytest.importorskip("psycopg")` and `@pytest.mark.skipif(not os.getenv("INTEGRATION"), reason="needs PostgreSQL")`; insert a SystemSettings row with is_active=True via the async session fixture; encrypt a test api_key; await load_provider_config(session); assert returned ProviderConfig has the expected fields and decrypted api_key. Remove xfail decorators from both.

cd backend && pytest tests/test_ai_config.py::test_api_key_encrypt_decrypt tests/test_classifier.py -x -v - Source assertion: backend/services/ai_config.py contains `from ai.provider_config import ProviderConfig` at module level (grep on the top 30 lines) - Source assertion: backend/services/ai_config.py does NOT contain `_ProviderConfigStub` - Source assertion: backend/services/classifier.py contains `await load_provider_config(` and `get_provider(config)` - Source assertion: backend/services/classifier.py does NOT contain `"active_provider"` or `"providers": {` - Source assertion: grep -v '^#' backend/services/classifier.py | grep -c 'MAX_AI_CHARS' returns 0 - Behavior: pytest backend/tests/test_classifier.py exits 0 - Behavior: pytest backend/tests/test_ai_config.py::test_api_key_encrypt_decrypt exits 0 - Behavior: INTEGRATION=1 pytest backend/tests/test_ai_config.py::test_load_provider_config exits 0 when DB is available (test is skipped otherwise) ai_config.py imports ProviderConfig directly; stub class gone; classifier uses load_provider_config + per-user/system fallback; truncation lives only in providers; two test_ai_config.py tests promoted; existing test_classifier.py still green.

<threat_model>

Trust Boundaries

Boundary	Description
services/ai_config.load_provider_config → providers	decrypted api_key crosses here in-memory only; never serialized
AnthropicProvider → anthropic.com API	api_key transits over TLS; SDK manages outbound auth

STRIDE Threat Register

Threat ID	Category	Component	Disposition	Mitigation Plan
T-07-06	Information Disclosure	classifier per-user override path	mitigate	per-user override does not carry api_key; system_settings is the only key store; api_key never leaves backend services tier
T-07-07	Tampering	Anthropic output_config grammar limit	accept	Pitfall 5 RESEARCH.md — schemas are simple (≤4 required fields, no unions); will not exceed Anthropic's 24-optional / 16-union limits
T-07-08	Denial of Service	Anthropic stop_reason "refusal"/"max_tokens"	mitigate	classify() falls back to parse_classification("") which returns empty ClassificationResult; document status set by caller (Plan 04 retry logic)
T-07-SC	Tampering	anthropic SDK pin raised to >=0.95.0	accept	RESEARCH.md confirms package legitimacy (official Anthropic SDK, ~3 yrs old); pip audit re-runs at phase gate
</threat_model>

- grep -v '^#' backend/ai/anthropic_provider.py | grep -c 'MAX_AI_CHARS' returns 0. - grep -F 'output_config' backend/ai/anthropic_provider.py returns 2+ matches (one per classify/suggest call). - grep -F '_ProviderConfigStub' backend/services/ai_config.py returns 0. - pytest backend/tests/test_ai_providers.py::test_anthropic_structured_output backend/tests/test_classifier.py backend/tests/test_ai_config.py::test_api_key_encrypt_decrypt exits 0. - pytest backend/tests/ -v shows no new failures.

<success_criteria>

• AnthropicProvider singleton + output_config + truncation + no MAX_AI_CHARS.
• classifier.classify_document driven by load_provider_config with per-user override + env fallback.
• ai_config.py stub removed; real ProviderConfig used everywhere.
• 3 previously-xfailed tests now pass (test_anthropic_structured_output, test_api_key_encrypt_decrypt, test_load_provider_config under INTEGRATION=1).
• Existing test_classifier.py still green. </success_criteria>

Create `.planning/phases/07-redo-and-optimize-llm-integration/07-03-SUMMARY.md` when done.