CH 04 · Model-Provider Abstraction and Multi-Model Strategy · 09/15
Read as one pageReal code: LiteLLM reads the provider from the name
A registry names the provider that owns a model id, and it refuses an id it cannot place. LiteLLM does both in get_llm_provider, which completion() calls before the request. The first excerpt is the shortcut for a provider/model string. The second excerpt runs only when custom_llm_provider is still empty, after the name sets and the startswith prefix rules.
# check if llm provider part of model name
if (
model.split("/", 1)[0] in litellm.provider_list # [1]
and model.split("/", 1)[0] not in litellm.model_list_set
and len(model.split("/"))
> 1 # handle edge case where user passes in `litellm --model mistral` https://github.com/BerriAI/litellm/issues/1351
):
return _get_openai_compatible_provider_info( # [2]
model=model,
api_base=api_base,
api_key=api_key,
dynamic_api_key=dynamic_api_key,
litellm_params=litellm_params,
)
Source: BerriAI/litellm, litellm/litellm_core_utils/get_llm_provider_logic.py (MIT)
Between the excerpts, a bare name is tested against sets filled at startup from the cost map. A known id resolves there, with no prefix on the string.
# Last resort for an otherwise-unknown model: a declarative
# fallback-generalization routing rule (e.g. routes future claude-* to anthropic).
# Exact provider matches above always win; this only runs on a miss.
if not custom_llm_provider:
custom_llm_provider = match_routing_generalization(model) # [3]
if not custom_llm_provider:
# ... (8 lines: a debug print, then error_str, which tells the caller to pass provider/model)
raise litellm.exceptions.BadRequestError( # [4]
message=error_str,
model=model,
response=None,
llm_provider="",
)
Source: BerriAI/litellm, litellm/litellm_core_utils/get_llm_provider_logic.py (MIT)
What to notice
- [1]
provider_listholds the names from theLlmProvidersenum, andsplit("/", 1)cuts at the first slash only. The text before that slash has to be one of those names, and it has to be absent frommodel_list_set. That set isset(model_list), a union of many per-provider name sets. Mistral and Fireworks are registered in other sets and are not in this union. The id also has to contain a slash, so a baremistraldoes not take this shortcut. A prefix that is both a provider name and a name inmodel_list_setskips this helper, and the followingelifstill splits on the slash and returns, so the name sets never see that id. - [2]
_get_openai_compatible_provider_infosplits the same way. The prefix becomes the provider, and the remainder becomes the model id. - [3]
match_routing_generalizationreturns the provider of the first routing rule whose pattern matches. This line runs only whilecustom_llm_provideris still empty, so a name-set match or astartswithprefix above it wins. - [4] If the rule returns nothing, the function raises
BadRequestErrorand does not substitute a model. A registry that quietly fell back to a default provider would send the request somewhere the caller never chose.
Read it yourself: Trace claude-haiku-4-5 and not-a-real-model through get_llm_provider. The first hits anthropic_models. The second reaches the BadRequestError. Then read anthropic-claude-ids, the rule that still sends an unmapped Claude-shaped id to Anthropic.
Finished this idea?
Your learning progress stays in this browser.