Model choice is not AI architecture

Choosing OpenAI, Anthropic, Gemini, Azure OpenAI, Mistral, or a local model is a component decision, not the complete AI architecture. Models differ meaningfully in quality, latency, context, policy, deployment, availability, and price. But the product still needs a job definition, authorised data, evaluation, tools, fallbacks, observability, and an interface regardless of which provider answers today.

Architecture should make model choice evidence-based and replaceable where replacement has value. That does not mean building a universal abstraction for every possible provider. It means keeping application identity, source data, evaluation cases, tool contracts, and business state outside vendor-specific prompt code.

Compare models against the task

A useful comparison runs representative cases and records properties the product cares about: factual correctness, source use, structured-output validity, refusal behaviour, tool selection, latency, and cost per completed task. Public benchmarks can inform a shortlist but cannot represent the organisation’s documents, terminology, permissions, and failure consequences.

Different stages may use different models. A smaller model can classify or route; a larger one can handle ambiguous synthesis; a local model can serve a restricted environment. Routing is justified when the quality, availability, or cost benefit exceeds the complexity it adds.

Keep durable controls outside the provider

Permissions should come from the application. Retrieval should preserve source identity. Tools should have validated contracts. Evaluation cases should live in a versioned system. Logs should record provider, model, prompt or workflow version, latency, usage, retrieved sources, tool results, and outcome without exposing unnecessary confidential content.

Fallback can mean another model, but it can also mean deterministic search, a cached result, a request for clarification, or human review. Switching providers does not help if the source is missing or the action itself is unsafe.

Accept some provider-specific engineering

Complete portability is not free. Providers expose different tool semantics, structured-output guarantees, context behaviour, safety controls, and hosting options. A thin internal boundary should preserve product concepts while allowing adapters to use capabilities that matter. Hiding every difference behind the lowest common denominator can make all models worse.

The stable architecture is the part the organisation owns: the task, data, permissions, evaluation, tool boundaries, operational history, and user experience. A model is selected inside that system. It should not become the system by accident.