Skip to slide
054 / 221

CH 04 · Model-Provider Abstraction and Multi-Model Strategy · 09/15

Read as one page

Real code: LiteLLM reads the provider from the name

A registry names the provider that owns a model id, and it refuses an id it cannot place. LiteLLM does both in get_llm_provider, which completion() calls before the request. The first excerpt is the shortcut for a provider/model string. The second excerpt runs only when custom_llm_provider is still empty, after the name sets and the startswith prefix rules.

# check if llm provider part of model name

if (
    model.split("/", 1)[0] in litellm.provider_list  # [1]
    and model.split("/", 1)[0] not in litellm.model_list_set
    and len(model.split("/"))
    > 1  # handle edge case where user passes in `litellm --model mistral` https://github.com/BerriAI/litellm/issues/1351
):
    return _get_openai_compatible_provider_info(  # [2]
        model=model,
        api_base=api_base,
        api_key=api_key,
        dynamic_api_key=dynamic_api_key,
        litellm_params=litellm_params,
    )

Source: BerriAI/litellm, litellm/litellm_core_utils/get_llm_provider_logic.py (MIT)

Between the excerpts, a bare name is tested against sets filled at startup from the cost map. A known id resolves there, with no prefix on the string.

# Last resort for an otherwise-unknown model: a declarative
# fallback-generalization routing rule (e.g. routes future claude-* to anthropic).
# Exact provider matches above always win; this only runs on a miss.
if not custom_llm_provider:
    custom_llm_provider = match_routing_generalization(model)  # [3]

if not custom_llm_provider:
    # ... (8 lines: a debug print, then error_str, which tells the caller to pass provider/model)
    raise litellm.exceptions.BadRequestError(  # [4]
        message=error_str,
        model=model,
        response=None,
        llm_provider="",
    )

Source: BerriAI/litellm, litellm/litellm_core_utils/get_llm_provider_logic.py (MIT)

What to notice

  1. [1] provider_list holds the names from the LlmProviders enum, and split("/", 1) cuts at the first slash only. The text before that slash has to be one of those names, and it has to be absent from model_list_set. That set is set(model_list), a union of many per-provider name sets. Mistral and Fireworks are registered in other sets and are not in this union. The id also has to contain a slash, so a bare mistral does not take this shortcut. A prefix that is both a provider name and a name in model_list_set skips this helper, and the following elif still splits on the slash and returns, so the name sets never see that id.
  2. [2] _get_openai_compatible_provider_info splits the same way. The prefix becomes the provider, and the remainder becomes the model id.
  3. [3] match_routing_generalization returns the provider of the first routing rule whose pattern matches. This line runs only while custom_llm_provider is still empty, so a name-set match or a startswith prefix above it wins.
  4. [4] If the rule returns nothing, the function raises BadRequestError and does not substitute a model. A registry that quietly fell back to a default provider would send the request somewhere the caller never chose.

Read it yourself: Trace claude-haiku-4-5 and not-a-real-model through get_llm_provider. The first hits anthropic_models. The second reaches the BadRequestError. Then read anthropic-claude-ids, the rule that still sends an unmapped Claude-shaped id to Anthropic.

Finished this idea?

Your learning progress stays in this browser.