ProviderConfig is a Pydantic BaseModel with provider_id, api_key, base_url, model, context_chars
GenericOpenAIProvider subclasses OpenAIProvider and passes response_format={"type":"json_object"} when supports_json_mode is True
GenericOpenAIProvider imports parse_classification + parse_suggestions from ai.utils and uses them on every classify()/suggest_topics() raw response (D-02 last-resort fallback)
OpenAIProvider stores self._client = AsyncOpenAI(...) in __init__ and never recreates it
get_provider(config: ProviderConfig) is a typed registry lookup (no if/elif chain)
MAX_AI_CHARS is absent from backend/ai/openai_provider.py and backend/services/classifier.py
PROVIDER_DEFAULTS dict covers all 10 providers listed in 07-RESEARCH.md Pattern 2
requirements.txt pins anthropic>=0.95.0
backend/ai/utils.py (parse_classification / parse_suggestions) remains intact — no edits, no deletions — and is imported by generic_openai_provider.py
path
provides
contains
backend/ai/provider_config.py
ProviderConfig Pydantic model + PROVIDER_DEFAULTS dict
backend/ai/provider_config.py::supports_json_mode flag
conditional response_format kwarg
response_format
from
to
via
pattern
backend/ai/generic_openai_provider.py
backend/ai/utils.py::parse_classification
D-02 last-resort fallback import
from ai.utils import parse_classification
from
to
via
pattern
backend/ai/openai_provider.py::__init__
AsyncOpenAI httpx connection pool
singleton storage on self._client
self._client = AsyncOpenAI
Replace the legacy AI provider plumbing with a typed Pydantic config model, a unified GenericOpenAIProvider that covers all 8 OpenAI-compatible vendors, a singleton client lifecycle, and a registry-based factory. Pin the anthropic SDK floor so Plan 03 can use output_config.
Purpose: Eliminate the per-request _client() anti-pattern (D-07) that prevents httpx connection pool reuse; consolidate Groq/xAI/DeepSeek/OpenRouter/Gemini/Mistral/Ollama/LMStudio into one class with named presets (D-16/D-17/D-18); make adding a new provider an O(1) registry edit; remove MAX_AI_CHARS in favour of per-provider context_chars (D-12) with 60/40 smart truncation (D-13); preserve parse_classification/parse_suggestions in ai/utils.py as the last-resort fallback path for providers that do not honour response_format (D-02 — Gemini preset path).
Output: provider_config.py, generic_openai_provider.py, refactored openai_provider.py, ollama_provider.py + lmstudio_provider.py with context_chars passthrough, registry-based ai/__init__.py, MAX_AI_CHARS removed from openai_provider.py and classifier.py, requirements.txt anthropic floor bumped.
class AIProvider(ABC) with abstract async methods classify(document_text, existing_topics, system_prompt) -> ClassificationResult and suggest_topics(document_text) -> list[str] and health_check() -> bool
ClassificationResult dataclass with assigned_topics: list[str], new_topic_suggestions: list[str], optional reasoning
From backend/ai/utils.py (D-02 — last-resort fallback; DO NOT modify or delete):
Each value: {"base_url": str|None, "model": str, "context_chars": int}
Add a parallel SUPPORTS_JSON_MODE dict (defaults True; "gemini" -> False per 07-RESEARCH.md JSON Compatibility Matrix; "ollama" / "lmstudio" remain True but classify() falls back to parse_classification() on any output anyway)
Task 1: ProviderConfig model + PROVIDER_DEFAULTS
backend/ai/__init__.py
backend/api/admin.py
.planning/phases/07-redo-and-optimize-llm-integration/07-RESEARCH.md
- ProviderConfig(BaseModel) accepts provider_id (str, required), api_key (str, default ""), base_url (str | None, default None), model (str, default ""), context_chars (int, default 8000)
- PROVIDER_DEFAULTS dict at module top contains exactly the 10 provider_ids listed in 07-RESEARCH.md Pattern 2 with matching base_url/model/context_chars values
- SUPPORTS_JSON_MODE dict has gemini=False and openai/anthropic/groq/xai/deepseek/openrouter/mistral/ollama/lmstudio=True
- Module imports without side effects; importing ProviderConfig does not import any provider class
- pytest backend/tests/test_ai_providers.py::test_get_provider_typed promotes from xfail to pass after Task 4
Create backend/ai/provider_config.py with: `from __future__ import annotations`; `from pydantic import BaseModel`; `class ProviderConfig(BaseModel)` with the five fields above and a class-level model_config setting extra="forbid" so unknown keys raise validation errors; `PROVIDER_DEFAULTS: dict[str, dict] = {...}` with the 10 entries from 07-RESEARCH.md Pattern 2 verbatim (do not change any base_url, model name, or context_chars value); `SUPPORTS_JSON_MODE: dict[str, bool] = {...}` with the 10 entries described above. No provider class imports — keep this file a pure data module. Add a brief docstring noting "Loaded by get_provider() in ai/__init__.py; populated by load_provider_config() in services/ai_config.py."
cd backend && python -c "from ai.provider_config import ProviderConfig, PROVIDER_DEFAULTS, SUPPORTS_JSON_MODE; c=ProviderConfig(provider_id='openai'); assert c.context_chars==8000; assert set(PROVIDER_DEFAULTS)=={'openai','anthropic','gemini','groq','xai','deepseek','openrouter','mistral','ollama','lmstudio'}; assert SUPPORTS_JSON_MODE['gemini'] is False; print('OK')"
- Source assertion: backend/ai/provider_config.py contains `class ProviderConfig(BaseModel)` and `PROVIDER_DEFAULTS` and `SUPPORTS_JSON_MODE`
- Source assertion: backend/ai/provider_config.py contains "gemini" once each in PROVIDER_DEFAULTS and SUPPORTS_JSON_MODE
- Behavior: Smoke test prints "OK"
- Behavior: ProviderConfig(provider_id="x", garbage="y") raises ValidationError (extra="forbid")
- Behavior: ProviderConfig does not import any provider class (no circular imports at module load)
provider_config.py exists, smoke test passes, ProviderConfig and the two defaults dicts are importable.
Task 2: GenericOpenAIProvider + OpenAIProvider singleton + MAX_AI_CHARS removal + ollama/lmstudio context_chars
backend/ai/openai_provider.py
backend/ai/ollama_provider.py
backend/ai/lmstudio_provider.py
backend/services/classifier.py
backend/ai/utils.py
backend/ai/base.py
- OpenAIProvider.__init__(api_key, model, base_url, context_chars) stores self._client = AsyncOpenAI(api_key=api_key or "not-needed", base_url=base_url) exactly once
- OpenAIProvider has no `_client(self)` method and no `MAX_AI_CHARS` constant
- OpenAIProvider has a _truncate(text) method that returns text unchanged when len(text) <= self._context_chars; otherwise returns text[:int(self._context_chars*0.6)] + "\n[...truncated...]\n" + text[-(self._context_chars - int(self._context_chars*0.6)):]
- OpenAIProvider.classify() and suggest_topics() use self._truncate(document_text) and call self._client.chat.completions.create(...) directly (no parentheses)
- GenericOpenAIProvider(OpenAIProvider) overrides classify() and suggest_topics() to pass response_format={"type":"json_object"} when class flag supports_json_mode is True; falls back to omitting the kwarg when False; parses with parse_classification/parse_suggestions in both branches (D-02 — last-resort fallback always wraps the raw response)
- GenericOpenAIProvider.classify() and .suggest_topics() import parse_classification and parse_suggestions from ai.utils (D-02 contract — these helpers in ai/utils.py remain the canonical fallback; no provider re-implements them)
- GenericOpenAIProvider accepts supports_json_mode as a constructor kwarg (default True) so the factory can set False for Gemini
- OllamaProvider.__init__ accepts an optional context_chars kwarg (default 8000) and passes it through super().__init__
- LMStudioProvider mirrors OllamaProvider's context_chars passthrough
- MAX_AI_CHARS is removed from backend/services/classifier.py (Plan 03 will fix the call-sites that used it; this task removes the constant)
- backend/ai/utils.py is NOT modified — parse_classification/parse_suggestions remain intact per D-02
- All existing test_classifier.py and test_lmstudio.py tests remain green
Refactor backend/ai/openai_provider.py: change __init__ signature to `def __init__(self, api_key: str, model: str, base_url: str | None, context_chars: int)`. Inside __init__ set self._api_key=api_key or "not-needed", self._model=model, self._base_url=base_url, self._context_chars=context_chars, self._client = AsyncOpenAI(api_key=self._api_key, base_url=self._base_url). Delete the existing `def _client(self):` method entirely. Delete the module-level `MAX_AI_CHARS = 8_000` line. Add `def _truncate(self, text: str) -> str:` returning the 60/40 split as described under behavior. Update classify(): replace `document_text[:MAX_AI_CHARS]` with `self._truncate(document_text)`; replace `await self._client().chat.completions.create(...)` with `await self._client.chat.completions.create(...)` (no parens). Update suggest_topics() and health_check() with the same self._client (no parens) change. Do not add response_format kwarg to OpenAIProvider — that lives in GenericOpenAIProvider (D-16). Existing OpenAIProvider semantics for OpenAI proper remain unchanged otherwise.
Create backend/ai/generic_openai_provider.py: import AsyncOpenAI from openai, OpenAIProvider from ai.openai_provider, and EXACTLY `from ai.utils import parse_classification, parse_suggestions` (D-02 contract — these are the last-resort fallback helpers; do not redefine locally, do not import-rename). Define `class GenericOpenAIProvider(OpenAIProvider)` with class attribute `supports_json_mode: bool = True` and override __init__ to accept supports_json_mode as a kwarg (after context_chars), set self.supports_json_mode = supports_json_mode, then call super().__init__(api_key=api_key, model=model, base_url=base_url, context_chars=context_chars). Override classify(self, document_text, existing_topics, system_prompt): assemble topics_str and user_msg the same way as OpenAIProvider.classify(); call self._client.chat.completions.create with model=self._model, max_tokens=1024, messages=[system, user], and conditionally add response_format={"type":"json_object"} when self.supports_json_mode is True; raw = response.choices[0].message.content or ""; return parse_classification(raw). Override suggest_topics() with the same conditional response_format pattern (omit response_format when supports_json_mode is False) and call parse_suggestions on the raw text. Do NOT override health_check — inherit from OpenAIProvider.
Update backend/ai/ollama_provider.py: change __init__ signature to `def __init__(self, base_url: str = "http://host.docker.internal:11434", model: str = "llama3.2", context_chars: int = 8000)`. Pass context_chars=context_chars to super().__init__. Same for lmstudio_provider.py (default model and base_url unchanged, add context_chars=8000 default and pass through). These files remain functional shims even though the registry (Task 3) routes "ollama" and "lmstudio" to GenericOpenAIProvider — keep them green for the existing test_lmstudio.py / test_classifier.py callers.
Update backend/services/classifier.py: delete the module-level `MAX_AI_CHARS = 8_000` constant. Replace any remaining `text[:MAX_AI_CHARS]` usage with `text` (truncation now happens inside the provider via _truncate; Plan 03 finalizes the load_provider_config wiring). Do NOT change the function signatures or remove the inline _settings dict yet — that is Plan 03's job. The only change to classifier.py in this plan is removing the MAX_AI_CHARS constant and the slice that uses it.
DO NOT modify backend/ai/utils.py — parse_classification, parse_suggestions, and strip_code_fences must remain intact (D-02 contract). They are imported by generic_openai_provider.py as the last-resort JSON fallback for non-json_mode paths.
Bump requirements.txt: locate the line beginning with `anthropic` and change the constraint to `anthropic>=0.95.0` (preserving any other version markers). If the line is `anthropic>=0.26` change to `anthropic>=0.95.0`; if it pins an exact version that is < 0.95, raise it to `>=0.95.0`. This unblocks Plan 03 output_config usage.
cd backend && python -c "from ai.openai_provider import OpenAIProvider; from ai.generic_openai_provider import GenericOpenAIProvider; p=OpenAIProvider(api_key='', model='gpt-4o', base_url=None, context_chars=100); assert hasattr(p,'_client') and not callable(p._client.chat); print('singleton:', type(p._client).__name__); g=GenericOpenAIProvider(api_key='', model='m', base_url='http://x/v1', context_chars=50, supports_json_mode=False); assert g.supports_json_mode is False; print('OK')" && grep -c 'MAX_AI_CHARS' backend/ai/openai_provider.py backend/services/classifier.py 2>/dev/null | grep -v ':0' || echo "MAX_AI_CHARS removed"
- Source assertion: grep -v '^#' backend/ai/openai_provider.py | grep -c 'MAX_AI_CHARS' returns 0
- Source assertion: grep -v '^#' backend/services/classifier.py | grep -c 'MAX_AI_CHARS' returns 0
- Source assertion: backend/ai/openai_provider.py contains `self._client = AsyncOpenAI(`
- Source assertion: backend/ai/openai_provider.py does NOT contain `def _client(self)` (regex: `def _client\(self\)`)
- Source assertion: backend/ai/generic_openai_provider.py contains `class GenericOpenAIProvider(OpenAIProvider)` and `response_format={"type": "json_object"}`
- Source assertion (D-02 enforcement): `grep "from ai.utils import parse_classification" backend/ai/generic_openai_provider.py` returns a match
- Source assertion (D-02 enforcement): `grep -c "parse_suggestions" backend/ai/generic_openai_provider.py` returns >= 1
- Source assertion (D-02 invariant): backend/ai/utils.py contains `def parse_classification` and `def parse_suggestions` (file untouched — these helpers remain the single canonical fallback)
- Source assertion: backend/ai/ollama_provider.py and backend/ai/lmstudio_provider.py both contain `context_chars`
- Source assertion: requirements.txt contains a line matching `^anthropic>=0\.9[5-9]` or `^anthropic>=0\.1` followed by `[0-9][0-9]`
- Behavior: python -c smoke prints "OK"
- Behavior: pytest backend/tests/test_classifier.py backend/tests/test_lmstudio.py -x exits 0
OpenAIProvider singleton fix landed; GenericOpenAIProvider created and imports parse_classification/parse_suggestions from ai.utils (D-02); ai/utils.py unchanged; ollama/lmstudio carry context_chars; MAX_AI_CHARS removed from openai_provider.py and classifier.py (anthropic_provider.py removal is Plan 03 because that file is refactored there); requirements.txt anthropic pin raised to >=0.95.0; existing tests still green.
Task 3: Registry-based get_provider(config: ProviderConfig)
backend/ai/__init__.py
backend/ai/provider_config.py
backend/ai/openai_provider.py
backend/ai/generic_openai_provider.py
backend/ai/anthropic_provider.py
- backend/ai/__init__.py exposes `get_provider(config: ProviderConfig) -> AIProvider` with a typed signature (not `settings: dict`)
- The flat if/elif chain is replaced by a _REGISTRY dict mapping provider_id strings to provider classes
- "openai" -> OpenAIProvider, "anthropic" -> AnthropicProvider, "gemini"/"groq"/"xai"/"deepseek"/"openrouter"/"mistral"/"ollama"/"lmstudio" -> GenericOpenAIProvider
- Unknown provider_id raises ValueError("Unknown AI provider: ...")
- For "anthropic" the factory passes api_key+model+context_chars (no base_url — AnthropicProvider has no base_url ctor arg until Plan 03 refactors it; until then continue passing only the args its current __init__ accepts)
- For GenericOpenAIProvider entries the factory reads PROVIDER_DEFAULTS to resolve base_url when config.base_url is None, and reads SUPPORTS_JSON_MODE to pass supports_json_mode kwarg
- For api_key="" the factory passes "not-needed" placeholder (OpenAI SDK 2.34+ rejects empty string)
- test_ai_providers.py::test_get_provider_typed promotes from xfail to passing
Rewrite backend/ai/__init__.py to: import AIProvider from ai.base, OpenAIProvider from ai.openai_provider, AnthropicProvider from ai.anthropic_provider, GenericOpenAIProvider from ai.generic_openai_provider, ProviderConfig + PROVIDER_DEFAULTS + SUPPORTS_JSON_MODE from ai.provider_config. Define `_REGISTRY: dict[str, type[AIProvider]]` with the 10 mappings described above. Define `def get_provider(config: ProviderConfig) -> AIProvider`: lookup cls in _REGISTRY, raise ValueError on miss; resolve effective_base_url = config.base_url or PROVIDER_DEFAULTS[config.provider_id]["base_url"]; effective_model = config.model or PROVIDER_DEFAULTS[config.provider_id]["model"]; effective_context_chars = config.context_chars or PROVIDER_DEFAULTS[config.provider_id]["context_chars"]; effective_api_key = config.api_key or "not-needed"; branch on provider_id: if "anthropic" return cls(api_key=effective_api_key, model=effective_model) — passing only the args its current ctor accepts (Plan 03 will widen this), elif cls is GenericOpenAIProvider return cls(api_key=effective_api_key, model=effective_model, base_url=effective_base_url, context_chars=effective_context_chars, supports_json_mode=SUPPORTS_JSON_MODE[config.provider_id]), else (OpenAIProvider) return cls(api_key=effective_api_key, model=effective_model, base_url=effective_base_url, context_chars=effective_context_chars). Promote the test_get_provider_typed xfail in backend/tests/test_ai_providers.py to a real test: build a ProviderConfig(provider_id="groq"), call get_provider(config), assert isinstance(result, GenericOpenAIProvider), assert result.supports_json_mode is True, assert result._context_chars == PROVIDER_DEFAULTS["groq"]["context_chars"]. Build a second ProviderConfig(provider_id="gemini"), assert result.supports_json_mode is False. Build a ProviderConfig(provider_id="bogus"), assert pytest.raises(ValueError).
cd backend && pytest tests/test_ai_providers.py::test_get_provider_typed -x -v
- Source assertion: backend/ai/__init__.py contains `_REGISTRY` and `def get_provider(config: ProviderConfig)`
- Source assertion: backend/ai/__init__.py contains all 10 provider_id keys in _REGISTRY ("openai","anthropic","gemini","groq","xai","deepseek","openrouter","mistral","ollama","lmstudio")
- Source assertion: backend/ai/__init__.py does NOT contain a sequence of `elif active ==` (no if/elif chain)
- Behavior: pytest backend/tests/test_ai_providers.py::test_get_provider_typed exits 0
- Behavior: ValueError raised when provider_id is unknown (asserted by test_get_provider_typed)
Registry get_provider function in place; test_get_provider_typed promoted and green.
Task 4: Promote singleton + JSON-mode + truncation + Gemini fallback tests
backend/tests/test_ai_providers.py
backend/ai/openai_provider.py
backend/ai/generic_openai_provider.py
backend/ai/provider_config.py
backend/ai/utils.py
- test_client_singleton: building two GenericOpenAIProvider instances and calling classify on the same instance twice exercises self._client only — the test asserts the AsyncOpenAI class is called exactly once per instance (use unittest.mock.patch on openai.AsyncOpenAI)
- test_generic_openai_json_mode: mock self._client.chat.completions.create and assert response_format={"type":"json_object"} appears in the call kwargs when supports_json_mode=True; assert response_format kwarg is absent when supports_json_mode=False (Gemini preset path)
- test_context_chars_truncation: build a provider with context_chars=100, feed an input of length 500, assert _truncate returns a string of length < 500 that contains "[...truncated...]"
- test_smart_truncation: build context_chars=1000, input of length 5000; assert the returned string starts with input[:600] and ends with input[-400:]
- test_gemini_fallback_to_parse_classification (D-02 coverage): build GenericOpenAIProvider(supports_json_mode=False) (Gemini preset path); mock self._client.chat.completions.create to return a synthetic response whose .choices[0].message.content is a valid JSON classification string; assert classify() returns a valid ClassificationResult; assert response_format kwarg is absent from mock_create.call_args.kwargs; assert ai.utils.parse_classification was the helper that produced the ClassificationResult (use unittest.mock.patch to wrap parse_classification and assert it was called once with the raw content)
- All five tests run green; xfail markers removed
Edit backend/tests/test_ai_providers.py: replace the xfail stub bodies for test_client_singleton, test_generic_openai_json_mode, test_context_chars_truncation, test_smart_truncation, and test_gemini_fallback_to_parse_classification with real implementations. Remove the @pytest.mark.xfail decorator on these five functions only (leave anthropic and others as xfail until their respective plans). Use unittest.mock.patch("ai.openai_provider.AsyncOpenAI") and AsyncMock to mock the async chat.completions.create method. For test_generic_openai_json_mode, assert response_format kwarg is in mock_create.call_args.kwargs when supports_json_mode=True and absent when supports_json_mode=False. For test_client_singleton, instantiate one OpenAIProvider, await its classify() twice (mock create returns a synthetic OpenAI response), assert AsyncOpenAI class mock was called exactly once. For truncation tests, instantiate any provider with context_chars=100 (or 1000) and call provider._truncate(text) directly — no async needed. For test_gemini_fallback_to_parse_classification (D-02): use unittest.mock.patch("ai.generic_openai_provider.parse_classification", wraps=parse_classification) so the real function still runs but the call is observable; instantiate GenericOpenAIProvider(api_key="", model="gemini-2.0-flash", base_url="https://generativelanguage.googleapis.com/v1beta/openai/", context_chars=8000, supports_json_mode=False); mock create() to return `MagicMock(choices=[MagicMock(message=MagicMock(content='{"assigned_topics":["x"],"new_topic_suggestions":[],"reasoning":"r"}'))])`; await classify("doc text", [], "sys"); assert the returned ClassificationResult.assigned_topics == ["x"]; assert mock_parse.called; assert "response_format" not in mock_create.call_args.kwargs. Imports needed: pytest, unittest.mock (AsyncMock, MagicMock, patch), ai.generic_openai_provider.GenericOpenAIProvider, ai.openai_provider.OpenAIProvider, ai.provider_config.ProviderConfig, ai.utils.parse_classification.
cd backend && pytest tests/test_ai_providers.py::test_client_singleton tests/test_ai_providers.py::test_generic_openai_json_mode tests/test_ai_providers.py::test_context_chars_truncation tests/test_ai_providers.py::test_smart_truncation tests/test_ai_providers.py::test_gemini_fallback_to_parse_classification -v
- Source assertion: backend/tests/test_ai_providers.py defines test_client_singleton/test_generic_openai_json_mode/test_context_chars_truncation/test_smart_truncation/test_gemini_fallback_to_parse_classification without @pytest.mark.xfail decorators on those five functions
- Behavior: pytest -v reports 5 passed for those five test ids
- Behavior: test_gemini_fallback_to_parse_classification asserts parse_classification was invoked AND response_format is absent (D-02 contract enforcement)
- Behavior: pytest backend/tests/ -v shows no new failures
Five Wave-2 tests green (including D-02 fallback coverage); total xfail count down by 5; full suite still passes.
<threat_model>
Trust Boundaries
Boundary
Description
Provider class → external LLM API
api_key transits over TLS; client owns the only outbound auth header
Untrusted document text → provider.classify()
content is the user's own; truncation prevents context blowout but is not a security boundary
STRIDE Threat Register
Threat ID
Category
Component
Disposition
Mitigation Plan
T-07-04
Tampering
OpenAI SDK 2.34+ empty api_key rejection
mitigate
Factory normalizes empty api_key to "not-needed" placeholder; never passes "" to AsyncOpenAI ctor (Pitfall 1 in RESEARCH.md)
T-07-05
Information Disclosure
Singleton _client retained across calls
accept
Each Celery task creates its own asyncio.run() event loop and a fresh ProviderConfig → fresh provider instance → fresh client; no cross-task sharing per RESEARCH.md D-07
T-07-SC
Tampering
No new packages
accept
RESEARCH.md Package Legitimacy Audit: zero new packages; only anthropic floor raised; pip audit re-runs at phase gate
