any-llm SDK

any-llm provides two primary ways to interact with LLMs depending on your requirements:

Direct API Functions

Recommended for simple use cases. These are top-level functions:

• completion: Chat completions with any provider.
• embedding: Text embeddings.
• moderation: Content moderation.
• responses: Implementation of the OpenResponses API for agentic AI systems.

AnyLLM Class

Recommended for advanced use cases. This provides a lower-level Provider API that allows for metadata access and object reusability.

Use Direct API Functions (completion, acompletion) when:

• Making simple, one-off requests.
• Prototyping or writing quick scripts.
• You want the simplest possible interface.

Use Provider Class (AnyLLM.create) when:

• Building applications that make multiple requests with the same provider.
• You want to avoid repeated provider instantiation overhead.

To get started with any-llm, install the SDK with the specific provider extras you need (e.g., mistral and ollama), set your API key as an environment variable, and use the completion function.

Note: If you are migrating from LiteLLM, your existing environment variables will work. You only need to change your imports and use the <provider>:<model> format for the model string.

pip install 'any-llm-sdk[mistral,ollama]'

export MISTRAL_API_KEY="YOUR_KEY_HERE"
from any_llm import completion
import os

# Make sure you have the appropriate environment variable set
assert os.environ.get('MISTRAL_API_KEY')

response = completion(
    model="mistral-small-latest",
    provider="mistral",
    messages=[{"role": "user", "content": "Hello!"}]
)
print(response.choices[0].message.content)

To install the any-llm-sdk with all provider dependencies, use the following command. If you are working in a Jupyter notebook, it is also recommended to install nest-asyncio to allow the use of await directly in cells, as any-llm relies on asynchronous functions for API calls.

%pip install any-llm-sdk[all] nest-asyncio -q

import nest_asyncio
nest_asyncio.apply()

from any_llm import completion
from any_llm.exceptions import (
    AnyLLMError,
    AuthenticationError,
    InvalidRequestError,
    ModelNotFoundError,
    ProviderError,
    RateLimitError,
)

try:
    response = completion(
        model="gpt-4",
        provider="openai",
        messages=[{"role": "user", "content": "Hello!"}]
    )
except ModelNotFoundError as e:
    print(f"Model not found: {e.message}")
except RateLimitError as e:
    print(f"Rate limited: {e.message}")
except AuthenticationError as e:
    print(f"Auth failed: {e.message}")
except InvalidRequestError as e:
    print(f"Invalid request: {e.message}")
except ProviderError as e:
    print(f"Provider error: {e.message}")
except AnyLLMError as e:
    print(f"Error: {e.message}")

export ANY_LLM_UNIFIED_EXCEPTIONS=1

You can install dependencies for specific providers using extras. Examples include:

• Mistral: pip install any-llm-sdk[mistral]
• Ollama: pip install any-llm-sdk[ollama]
• Multiple providers: pip install any-llm-sdk[mistral,ollama]

You can provide API keys to any-llm in two ways:

1. Environment Variables (Recommended): Set the standard environment variable for the provider (e.g., OPENAI_API_KEY, ANTHROPIC_API_KEY, MISTRAL_API_KEY).
2. Directly in Code: Pass the api_key argument when using the AnyLLM.create method.

export OPENAI_API_KEY="your-key-here"
export ANTHROPIC_API_KEY="your-key-here"
export MISTRAL_API_KEY="your-key-here"

Install the SDK using pip. You can install support for specific providers using extras to keep your installation lightweight, or install all supported providers at once.

Requirements:

• Python 3.11 or newer
• API keys for your chosen LLM providers

Installation Commands:

• Just OpenAI: pip install 'any-llm-sdk[openai]'
• Multiple providers: pip install 'any-llm-sdk[mistral,ollama]'
• All supported providers: pip install 'any-llm-sdk[all]'

The completion function is the recommended direct API for simple chat completion use cases. It allows you to interact with different LLM providers using a unified messages format. You can switch between providers (e.g., openai, anthropic) by simply changing the provider and model arguments without modifying your core logic.

from any_llm import completion

# Using the messages format
response = completion(
    model="gpt-4o-mini",
    messages=[{"role": "user", "content": "What is Python?"}],
    provider="openai"
)
print(response)

# Switch providers without changing your code
response = completion(
    model="claude-sonnet-4-5-20250929",
    messages=[{"role": "user", "content": "What is Python?"}],
    provider="anthropic"
)
print(response)

To authenticate with a provider, you can either:

1. Set the provider's API key as an environment variable (e.g., export MISTRAL_API_KEY="your-key").
2. Pass the api_key parameter directly to the API call or class instantiation.