repository·main·Indexed 24 days ago
https://github.com/agentops-ai/tokencostA client-side library for calculating token counts and USD costs of Large Language Model (LLM) API requests. It supports a wide registry of models from providers including OpenAI, Anthropic, Azure, Mistral, and Groq. The library provides functions to calculate prompt and completion costs, as well as token counting for both raw strings and message lists (ChatML), utilizing Tiktoken for OpenAI and the beta token counting API for newer Anthropic models.
Tokenization logic depends on the model provider:
cl100k_base encoding.Install the tokencost package from PyPI using pip.
pip install tokencostThe TOKEN_COSTS dictionary is the central source of truth for model pricing in this library. It is initialized from a local model_prices.json file and can be updated via the provided API functions.
Each entry in the dictionary provides information used to calculate costs:
Note that total token limits are typically one more than the prompt token limit to allow for at least one completion token.
Use calculate_prompt_cost and calculate_completion_cost to estimate the USD cost of LLM interactions. You must provide the content (either a message list or a string) and the specific model name to ensure accurate pricing based on the provider's latest rates.
from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Hello world"}]
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")
# 0.0000135 + 0.000014 = 0.0000275from tokencost import calculate_prompt_cost
prompt_string = "Hello world"
response = "How may I assist you today?"
model= "gpt-3.5-turbo"
prompt_cost = calculate_prompt_cost(prompt_string, model)
print(f"Cost: ${prompt_cost}")
# Cost: $3e-06from tokencost import calculate_prompt_cost, calculate_completion_cost
model = "gpt-3.5-turbo"
prompt = [{ "role": "user", "content": "Hello world"}]
completion = "How may I assist you today?"
prompt_cost = calculate_prompt_cost(prompt, model)
completion_cost = calculate_completion_cost(completion, model)
print(f"{prompt_cost} + {completion_cost} = {prompt_cost + completion_cost}")If you only need the token count without the cost estimation, use the following functions:
count_message_tokens(messages, model): Counts tokens in a list of message dictionaries (e.g., OpenAI chat format). This accounts for additional tokens used for message formatting and roles.count_string_tokens(prompt, model): Counts tokens in a raw string.from tokencost import count_message_tokens, count_string_tokens
model = "gpt-3.5-turbo"
message_prompt = [{ "role": "user", "content": "Hello world"}]
# Counting tokens in message lists
print(count_message_tokens(message_prompt, model=model))
# 9
# Counting tokens in string prompts
print(count_string_tokens(prompt="Hello world", model=model))
# 2from tokencost import count_message_tokens, count_string_tokens
message_prompt = [{ "role": "user", "content": "Hello world"}]
print(count_message_tokens(message_prompt, model="gpt-3.5-turbo"))
print(count_string_tokens(prompt="Hello world", model="gpt-3.5-turbo"))The tokencost library maintains a registry of model identifiers, their input/output costs (per million tokens), and their context window limits. This data is used to estimate the cost of LLM operations. The registry includes models from providers like AWS Bedrock, Meta, Together AI, SageMaker, and Ollama. Note that some models (like those running via Ollama) are listed with $0 cost as they are typically run locally.
| model_id | input_cost | output_cost | context_window | max_output |
| :--- | :--- | :--- | :--- | :--- |
| bedrock/us-west-1/meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |
| meta.llama3-1-8b-instruct-v1:0 | $0.22 | $0.22 | 128,000 | 2048 |
| meta.llama3-1-405b-instruct-v1:0 | $5.32 | $16 | 128,000 | 4096 |
| together-ai-41.1b-80b | $0.9 | $0.9 | nan | nan |
| ollama/llama3:70b | $0 | $0 | 8,192 | 8192 |The following table provides a reference for supported models across various providers (Ollama, DeepInfra, Perplexity, Fireworks AI), including their input cost, output cost, and context window limits. Note that some models (like those from Ollama) may have $0 cost if running locally.
| ollama/mistral-7B-Instruct-v0.2 | $0 | $0 | 32,768 | 32768 |
| ollama/mixtral-8x7B-Instruct-v0.1 | $0 | $0 | 32,768 | 32768 |
| ollama/mixtral-8x22B-Instruct-v0.1 | $0 | $0 | 65,536 | 65536 |
| ollama/codellama | $0 | $0 | 4,096 | 4096 |
| ollama/orca-mini | $0 | $0 | 4,096 | 4096 |
| ollama/vicuna | $0 | $0 | 2,048 | 2048 |
| deepinfra/lizpreciatior/lzlv_70b_fp16_hf | $0.35 | $0.4 | 4,096 | 4096 |
| deepinfra/Gryphe/MythoMax-L2-13b | $0.07 | $0.07 | 4,096 | 4096 |
| deepinfra/mistralai/Mistral-7B-Instruct-v0.1 | $0.06 | $0.06 | 32,768 | 32768 |
| deepinfra/meta-llama/Llama-2-70b-chat-hf | $0.64 | $0.8 | 4,096 | 4096 |
| deepinfra/cognitivecomputations/dolphin-2.6-mixtral-8x7b | $0.24 | $0.24 | 32,768 | 32768 |
| deepinfra/codellama/CodeLlama-34b-Instruct-hf | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/deepinfra/mixtral | $0.27 | $0.27 | 32,000 | 4096 |
| deepinfra/Phind/Phind-CodeLlama-34B-v2 | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.08 | $0.24 | 32,768 | 32768 |
| deepinfra/deepinfra/airoboros-70b | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-34B-Chat | $0.6 | $0.6 | 4,096 | 4096 |
| deepinfra/01-ai/Yi-6B-200K | $0.13 | $0.13 | 200,000 | 4096 |
| deepinfra/jondurbin/airoboros-l2-70b-gpt4-1.4.1 | $0.7 | $0.9 | 4,096 | 4096 |
| deepinfra/meta-llama/Llama-2-13b-chat-hf | $0.13 | $0.13 | 4,096 | 4096 |
| deepinfra/amazon/MistralLite | $0.2 | $0.2 | 32,768 | 8191 |
| deepinfra/meta-llama/Llama-2-7b-chat-hf | $0.13 | $0.13 | 4,096 | 4096 |
| deepinfra/meta-llama/Meta-Llama-3-8B-Instruct | $0.03 | $0.06 | 8,192 | 8192 |
| deepinfra/meta-llama/Meta-Llama-3-70B-Instruct | $0.3 | $0.4 | 8,192 | 8192 |
| deepinfra/01-ai/Yi-34B-200K | $0.6 | $0.6 | 200,000 | 4096 |
| deepinfra/openchat/openchat_3.5 | $0.06 | $0.06 | 8,192 | 8192 |
| perplexity/codellama-34b-instruct | $0.35 | $1.4 | 16,384 | 16384 |
| perplexity/codellama-70b-instruct | $0.7 | $2.8 | 16,384 | 16384 |
| perplexity/llama-3.1-70b-instruct | $1 | $1 | 131,072 | 131072 |
| perplexity/llama-3.1-8b-instruct | $0.2 | $0.2 | 131,072 | 131072 |
| perplexity/llama-3.1-sonar-huge-128k-online | $5 | $5 | 127,072 | 127072 |
| perplexity/llama-3.1-sonar-large-128k-online | $1 | $1 | 127,072 | 127072 |
| perplexity/llama-3.1-sonar-large-128k-chat | $1 | $1 | 131,072 | 131072 |
| perplexity/llama-3.1-sonar-small-128k-chat | $0.2 | $0.2 | 131,072 | 131072 |
| perplexity/llama-3.1-sonar-small-128k-online | $0.2 | $0.2 | 127,072 | 127072 |
| perplexity/pplx-7b-chat | $0.07 | $0.28 | 8,192 | 8192 |
| perplexity/pplx-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/pplx-7b-online | $0 | $0.28 | 4,096 | 4096 |
| perplexity/pplx-70b-online | $0 | $2.8 | 4,096 | 4096 |
| perplexity/llama-2-70b-chat | $0.7 | $2.8 | 4,096 | 4096 |
| perplexity/mistral-7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/mixtral-8x7b-instruct | $0.07 | $0.28 | 4,096 | 4096 |
| perplexity/sonar-small-chat | $0.07 | $0.28 | 16,384 | 16384 |
| perplexity/sonar-small-online | $0 | $0.28 | 12,000 | 12000 |
| perplexity/sonar-medium-chat | $0.6 | $1.8 | 16,384 | 16384 |
| perplexity/sonar-medium-online | $0 | $1.8 | 12,000 | 12000 |
| fireworks_ai/accounts/fireworks/models/llama-v3p2-1b-instruct | $0.1 | $0.1 | 16,384 | 16384 |
| fireworks_ai/accounts/fireworks/models/llama-v3p2-3b-instruct | $0.1 | $0.1 | 16,384 | 16384 |
| fireworks_ai/accounts/fireworks/models/llama-v3p2-11b-vision-instruct | $0.2 | $0.2 | 16,384 | 16384 |
| accounts/fireworks/models/llama-v3p2-90b-vision-instruct | $0.9 | $0.9 | 16,384 | 16384 |
| fireworks_ai/accounts/fireworks/models/firefunction-v2 | $0.9 | $0.9 | 8,192 | 8192 |
| fireworks_ai/accounts/fireworks/models/mixtral-8x22b-instruct-hf | $1.2 | $1.2 | 65,536 | 65536 |
| fireworks_ai/accounts/fireworks/models/qwen2-72b-instruct | $0.9 | $0.9 | 32,768 | 32768 |
| fireworks_ai/accounts/fireworks/models/yi-large | $3 | $3 | 32,768 | 32768 |
| fireworks_ai/accounts/fireworks/models/deepseek-coder-v2-instruct | $1.2 | $1.2 | 65,536 | 65536 |
| fireworks_ai/nomic-ai/nomic-embed-text-v1.5 | $0.01 | $0 | 8,192 | nan |
| fireworks_ai/nomic-ai/nomic-embed-text-v1 | $0.01 | $0 | 8,192 | nan |
| fireworks_ai/WhereIsAI/UAE-Large-V1 | $0.02 | $0 | 512 | nan |
| fireworks_ai/thenlper/gte-large | $0.02 | $0 | 512 | nan |
| fireworks_ai/thenlper/gte-base | $0.01 | $0 | 512 | nan |
| fireworks-ai-up-to-16b | $0.2 | $0.2 | nan | nan |The tokencost library maintains a registry of model identifiers, their input/output costs, context window limits, and output token limits. This data is used to calculate cost estimates for various LLM providers including Anthropic, Cohere, Meta (Llama), and AWS Bedrock.
Common model identifier patterns include:
anthropic.claude-v2, cohere.command-r-v1:0bedrock/us-east-1/anthropic.claude-v2meta.llama3-8b-instruct-v1:0Note that some models (like those under commitment plans) may have -- listed for costs in the registry.
| anthropic.claude-v2 | $8 | $24 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2 | $8 | $24 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2 | $8 | $24 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2 | $8 | $24 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2 | $8 | $24 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2 | -- | -- | 100,000 | 8191 |
| anthropic.claude-v2:1 | $8 | $24 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-v2:1 | $8 | $24 | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-v2:1 | $8 | $24 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-v2:1 | $8 | $24 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-v2:1 | $8 | $24 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-v2:1 | -- | -- | 100,000 | 8191 |
| anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/us-east-1/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/us-east-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-east-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/us-west-2/anthropic.claude-instant-v1 | $0.8 | $2.4 | 100,000 | 8191 |
| bedrock/ap-northeast-1/anthropic.claude-instant-v1 | $2.23 | $7.55 | 100,000 | 8191 |
| bedrock/ap-northeast-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/ap-northeast-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/anthropic.claude-instant-v1 | $2.48 | $8.38 | 100,000 | 8191 |
| bedrock/eu-central-1/1-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| bedrock/eu-central-1/6-month-commitment/anthropic.claude-instant-v1 | -- | -- | 100,000 | 8191 |
| cohere.command-text-v14 | $1.5 | $2 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-light-text-v14 | $0.3 | $0.6 | 4,096 | 4096 |
| bedrock/*/1-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| bedrock/*/6-month-commitment/cohere.command-light-text-v14 | -- | -- | 4,096 | 4096 |
| cohere.command-r-plus-v1:0 | $3 | $15 | 128,000 | 4096 |
| cohere.command-r-v1:0 | $0.5 | $1.5 | 128,000 | 4096 |
| cohere.embed-english-v3 | $0.1 | $0 | 512 | nan |
| cohere.embed-multilingual-v3 | $0.1 | $0 | 512 | nan |
| meta.llama2-13b-chat-v1 | $0.75 | $1 | 4,096 | 4096 |
| meta.llama2-70b-chat-v1 | $1.95 | $2.56 | 4,096 | 4096 |
| meta.llama3-8b-instruct-v1:0 | $0.3 | $0.6 | 8,192 | 8192 |
| bedrock/us-east-1/meta.llama3-8b-instruct-v1:0 | $0.3 | $0.6 | 8,192 | 8192 |
| bedrock/us-west-1/meta.llama3-8b-instruct-v1:0 | $0.3 | $0.6 | 8,192 | 8192 |
| bedrock/ap-south-1/meta.llama3-8b-instruct-v1:0 | $0.36 | $0.72 | 8,192 | 8192 |
| bedrock/ca-central-1/meta.llama3-8b-instruct-v1:0 | $0.35 | $0.69 | 8,192 | 8192 |
| bedrock/eu-west-1/meta.llama3-8b-instruct-v1:0 | $0.32 | $0.65 | 8,192 | 8192 |
| bedrock/eu-west-2/meta.llama3-8b-instruct-v1:0 | $0.39 | $0.78 | 8,192 | 8192 |
| bedrock/sa-east-1/meta.llama3-8b-instruct-v1:0 | $0.5 | $1.01 | 8,192 | 8192 |
| meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |
| bedrock/us-east-1/meta.llama3-70b-instruct-v1:0 | $2.65 | $3.5 | 8,192 | 8192 |The tokencost library maintains a registry of model pricing and context window limits for various providers. This includes input/output costs per million tokens, maximum context window sizes, and maximum output token limits.
Commonly supported providers and model patterns include:
amazon.nova-lite-v1:0, us.amazon.nova-pro-v1:0meta.llama3-3-70b-instruct-v1:0, together_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbogemini/gemini-2.0-flash, gemini/gemini-2.0-flash-thinking-expo3-mini, azure/o1-2024-12-17deepseek/deepseek-reasoner, openrouter/deepseek/deepseek-r1voyage/voyage-3-large, voyage/rerank-2sambanova/Meta-Llama-3.1-8B-Instructxai/grok-2Note that some models may have nan values for context limits or costs if they are not explicitly defined in the registry.
| amazon.nova-lite-v1:0 | $0.06 | $0.24 | 300,000 | 10000 |
| amazon.nova-pro-v1:0 | $0.8 | $3.2 | 300,000 | 10000 |
| meta.llama3-3-70b-instruct-v1:0 | $0.72 | $0.72 | 128,000 | 4096 |
| together_ai/meta-llama/Meta-Llama-3.1-8B-Instruct-Turbo | $0.18 | $0.18 | nan | nan |
| together_ai/meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo | $0.88 | $0.88 | nan | nan |
| together_ai/meta-llama/Meta-Llama-3.1-405B-Instruct-Turbo | $3.5 | $3.5 | nan | nan |
| deepinfra/meta-llama/Meta-Llama-3.1-405B-Instruct | $0.8 | $0.8 | 32,768 | 32768 |
| fireworks_ai/accounts/fireworks/models/deepseek-v3 | $0.9 | $0.9 | 128,000 | 8192 |
| voyage/voyage-3-large | $0.18 | $0 | 32,000 | nan |
| voyage/voyage-3 | $0.06 | $0 | 32,000 | nan |
| voyage/voyage-3-lite | $0.02 | $0 | 32,000 | nan |
| voyage/voyage-code-3 | $0.18 | $0 | 32,000 | nan |
| voyage/voyage-multimodal-3 | $0.12 | $0 | 32,000 | nan |
| voyage/rerank-2 | $0.05 | $0 | 16,000 | 16000 |
| voyage/rerank-2-lite | $0.02 | $0 | 8,000 | 8000 |
| databricks/meta-llama-3.3-70b-instruct | $1 | $3 | 128,000 | 128000 |
| sambanova/Meta-Llama-3.1-8B-Instruct | $0.1 | $0.2 | 16,384 | 16384 |
| sambanova/Meta-Llama-3.1-70B-Instruct | $0.6 | $1.2 | 128,000 | 128000 |
| sambanova/Meta-Llama-3.1-405B-Instruct | $5 | $10 | 16,384 | 16384 |
| sambanova/Meta-Llama-3.2-1B-Instruct | $0.04 | $0.08 | 16,384 | 16384 |
| sambanova/Meta-Llama-3.2-3B-Instruct | $0.08 | $0.16 | 4,096 | 4096 |
| sambanova/Meta-Llama-3.3-70B-Instruct | $0.6 | $1.2 | 131,072 | 131072 |
| sambanova/Qwen2.5-Coder-32B-Instruct | $1.5 | $3 | 8,000 | 8000 |
| sambanova/Qwen2.5-72B-Instruct | $2 | $4 | 8,000 | 8000 |
| o3-mini | $1.1 | $4.4 | 200,000 | 100000 |
| o3-mini-2025-01-31 | $1.1 | $4.4 | 200,000 | 100000 |
| azure/o3-mini-2025-01-31 | $1.1 | $4.4 | 200,000 | 100000 |
| azure/o3-mini | $1.1 | $4.4 | 200,000 | 100000 |
| azure/o1-2024-12-17 | $15 | $60 | 200,000 | 100000 |
| azure_ai/deepseek-r1 | $1.35 | $5.4 | 128,000 | 8192 |
| deepseek/deepseek-reasoner | $0.55 | $2.19 | 65,536 | 8192 |
| xai/grok-2-vision-1212 | $2 | $10 | 32,768 | 32768 |
| xai/grok-2-vision-latest | $2 | $10 | 32,768 | 32768 |
| xai/grok-2-vision | $2 | $10 | 32,768 | 32768 |
| xai/grok-vision-beta | $5 | $15 | 8,192 | 8192 |
| xai/grok-2-1212 | $2 | $10 | 131,072 | 131072 |
| xai/grok-2 | $2 | $10 | 131,072 | 131072 |
| xai/grok-2-latest | $2 | $10 | 131,072 | 131072 |
| groq/deepseek-r1-distill-llama-70b | $0.75 | $0.99 | 128,000 | 128000 |
| gemini/gemini-2.0-flash | $0.1 | $0.4 | 1,048,576 | 8192 |
| gemini-2.0-flash-001 | $0.15 | $0.6 | 1,048,576 | 8192 |
| gemini-2.0-flash-thinking-exp | $0 | $0 | 1,048,576 | 8192 |
| gemini-2.0-flash-thinking-exp-01-21 | $0 | $0 | 1,048,576 | 65536 |
| gemini/gemini-2.0-flash-001 | $0.1 | $0.4 | 1,048,576 | 8192 |
| gemini/gemini-2.0-flash-lite-preview-02-05 | $0.08 | $0.3 | 1,048,576 | 8192 |
| gemini/gemini-2.0-flash-thinking-exp | $0 | $0 | 1,048,576 | 65536 |
| vertex_ai/codestral-2501 | $0.2 | $0.6 | 128,000 | 128000 |
| openrouter/deepseek/deepseek-r1 | $0.55 | $2.19 | 65,336 | 8192 |
| ai21.jamba-1-5-large-v1:0 | $2 | $8 | 256,000 | 256000 |
| ai21.jamba-1-5-mini-v1:0 | $0.2 | $0.4 | 256,000 | 256000 |
| us.amazon.nova-micro-v1:0 | $0.04 | $0.14 | 128,000 | 10000 |
| us.amazon.nova-lite-v1:0 | $0.06 | $0.24 | 300,000 | 10000 |
| us.amazon.nova-pro-v1:0 | $0.8 | $3.2 | 300,000 | 10000 |
| stability.sd3-5-large-v1:0 | -- | -- | 77 | nan |
| stability.stable-image-core-v1:0 | -- | -- | 77 | nan |
| stability.stable-image-core-v1:1 | -- | -- | 77 | nan |
| stability.stable-image-ultra-v1:1 | -- | -- | 77 | nan |
| together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo | $0.88 | $0.88 | nan | nan |
| together_ai/meta-llama/Llama-3.3-70B-Instruct-Turbo-Free | $0 | $0 | nan | nan |
| fireworks_ai/accounts/fireworks/models/llama-v3p1-8b-instruct | $0.1 | $0.1 | 16,384 | 16384 |
| assemblyai/nano | -- | -- | nan | nan |TokenCost includes pricing and context window information for a wide variety of models from providers like replicate, openrouter, anthropic, google, and others. The data includes input cost per million tokens, output cost per million tokens, and context window limits (both input and output).
| replicate/meta/llama-2-7b-chat | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/meta/llama-3-70b | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-70b-instruct | $0.65 | $2.75 | 8,192 | 8192 |
| replicate/meta/llama-3-8b | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/meta/llama-3-8b-instruct | $0.05 | $0.25 | 8,086 | 8086 |
| replicate/mistralai/mistral-7b-v0.1 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mistral-7b-instruct-v0.2 | $0.05 | $0.25 | 4,096 | 4096 |
| replicate/mistralai/mixtral-8x7b-instruct-v0.1 | $0.3 | $1 | 4,096 | 4096 |
| openrouter/deepseek/deepseek-coder | $0.14 | $0.28 | 66,000 | 4096 |
| openrouter/microsoft/wizardlm-2-8x22b:nitro | $1 | $1 | nan | nan |
| openrouter/google/gemini-pro-1.5 | $2.5 | $7.5 | 1,000,000 | 8192 |
| openrouter/mistralai/mixtral-8x22b-instruct | $0.65 | $0.65 | nan | nan |
| openrouter/cohere/command-r-plus | $3 | $15 | nan | nan |
| openrouter/databricks/dbrx-instruct | $0.6 | $0.6 | nan | nan |
| openrouter/anthropic/claude-3-haiku | $0.25 | $1.25 | nan | nan |
| openrouter/anthropic/claude-3-haiku-20240307 | $0.25 | $1.25 | 200,000 | 4096 |
| anthropic/claude-3-5-sonnet-20241022 | $3 | $15 | 200,000 | 8192 |
| anthropic/claude-3-5-sonnet-latest | $3 | $15 | 200,000 | 8192 |
| openrouter/anthropic/claude-3.5-sonnet | $3 | $15 | 200,000 | 8192 |
| openrouter/anthropic/claude-3.5-sonnet:beta | $3 | $15 | 200,000 | 8192 |
| openrouter/anthropic/claude-3-sonnet | $3 | $15 | nan | nan |
| openrouter/mistralai/mistral-large | $8 | $24 | nan | nan |
| openrouter/cognitivecomputations/dolphin-mixtral-8x7b | $0.5 | $0.5 | nan | nan |
| openrouter/google/gemini-pro-vision | $0.12 | $0.38 | nan | nan |
| openrouter/fireworks/firellava-13b | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:free | $0 | $0 | nan | nan |
| openrouter/meta-llama/llama-3-8b-instruct:extended | $0.22 | $2.25 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct:nitro | $0.9 | $0.9 | nan | nan |
| openrouter/meta-llama/llama-3-70b-instruct | $0.59 | $0.79 | nan | nan |
| openrouter/openai/o1-mini | $3 | $12 | 128,000 | 65536 |
| openrouter/openai/o1-mini-2024-09-12 | $3 | $12 | 128,000 | 65536 |
| openrouter/openai/o1-preview | $15 | $60 | 128,000 | 32768 |
| openrouter/openai/o1-preview-2024-09-12 | $15 | $60 | 128,000 | 32768 |
| openrouter/openai/gpt-4o | $2.5 | $10 | 128,000 | 4096 |
| openrouter/openai/gpt-4o-2024-05-13 | $5 | $15 | 128,000 | 4096 |
| openrouter/openai/gpt-4-vision-preview | $10 | $30 | nan | nan |
| openrouter/openai/gpt-3.5-turbo | $1.5 | $2 | nan | nan |
| openrouter/openai/gpt-3.5-turbo-16k | $3 | $4 | nan | nan |
| openrouter/openai/gpt-4 | $30 | $60 | nan | nan |
| openrouter/anthropic/claude-instant-v1 | $1.63 | $5.51 | nan | 8191 |
| openrouter/anthropic/claude-2 | $11.02 | $32.68 | nan | 8191 |
| openrouter/anthropic/claude-3-opus | $15 | $75 | 200,000 | 4096 |
| openrouter/google/palm-2-chat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/google/palm-2-codechat-bison | $0.5 | $0.5 | nan | nan |
| openrouter/meta-llama/llama-2-13b-chat | $0.2 | $0.2 | nan | nan |
| openrouter/meta-llama/llama-2-70b-chat | $1.5 | $1.5 | nan | nan |
| openrouter/meta-llama/codellama-34b-instruct | $0.5 | $0.5 | nan | nan |
| openrouter/nousresearch/nous-hermes-llama2-13b | $0.2 | $0.2 | nan | nan |
| openrouter/mancer/weaver | $5.62 | $5.62 | nan | nan |
| openrouter/gryphe/mythomax-l2-13b | $1.88 | $1.88 | nan | nan |
| openrouter/jondurbin/airoboros-l2-70b-2.1 | $13.88 | $13.88 | nan | nan |
| openrouter/undi95/remm-slerp-l2-13b | $1.88 | $1.88 | nan | nan |
| openrouter/pygmalionai/mythalion-13b | $1.88 | $1.88 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct | $0.13 | $0.13 | nan | nan |
| openrouter/mistralai/mistral-7b-instruct:free | $0 | $0 | nan | nan |
| j2-ultra | $15 | $15 | 8,192 | 8192 |
| jamba-1.5-mini@001 | $0.2 | $0.4 | 256,000 | 256000 |
| jamba-1.5-large@001 | $2 | $8 | 256,000 | 256000 |
| jamba-1.5 | $0.2 | $0.4 | 256,000 | 256000 |
| jamba-1.5-mini | $0.2 | $0.4 | 256,000 | 256000 |
| jamba-1.5-large | $2 | $8 | 256,000 | 256000 |The following table provides a reference for supported models, including their input/output costs (per million tokens) and context window limits. Note that some values may be nan if not specified for that specific model provider.
| model | input_cost | output_cost | context_window | max_output_tokens |
|---|---|---|---|---|
| fireworks-ai-16.1b-to-80b | $0.9 | $0.9 | nan | nan |
| fireworks-ai-moe-up-to-56b | $0.5 | $0.5 | nan | nan |
| fireworks-ai-56b-to-176b | $1.2 | $1.2 | nan | nan |
| fireworks-ai-default | $0 | $0 | nan | nan |
| fireworks-ai-embedding-up-to-150m | $0.01 | $0 | nan | nan |
| fireworks-ai-embedding-150m-to-350m | $0.02 | $0 | nan | nan |
| anyscale/mistralai/Mistral-7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x7B-Instruct-v0.1 | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/mistralai/Mixtral-8x22B-Instruct-v0.1 | $0.9 | $0.9 | 65,536 | 65536 |
| anyscale/HuggingFaceH4/zephyr-7b-beta | $0.15 | $0.15 | 16,384 | 16384 |
| anyscale/google/gemma-7b-it | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Llama-2-7b-chat-hf | $0.15 | $0.15 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-13b-chat-hf | $0.25 | $0.25 | 4,096 | 4096 |
| anyscale/meta-llama/Llama-2-70b-chat-hf | $1 | $1 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-34b-Instruct-hf | $1 | $1 | 4,096 | 4096 |
| anyscale/codellama/CodeLlama-70b-Instruct-hf | $1 | $1 | 4,096 | 4096 |
| anyscale/meta-llama/Meta-Llama-3-8B-Instruct | $0.15 | $0.15 | 8,192 | 8192 |
| anyscale/meta-llama/Meta-Llama-3-70B-Instruct | $1 | $1 | 8,192 | 8192 |
| cloudflare/@cf/meta/llama-2-7b-chat-fp16 | $1.92 | $1.92 | 3,072 | 3072 |
| cloudflare/@cf/meta/llama-2-7b-chat-int8 | $1.92 | $1.92 | 2,048 | 2048 |
| cloudflare/@cf/mistral/mistral-7b-instruct-v0.1 | $1.92 | $1.92 | 8,192 | 8192 |
| cloudflare/@hf/thebloke/codellama-7b-instruct-awq | $1.92 | $1.92 | 4,096 | 4096 |
| voyage/voyage-01 | $0.1 | $0 | 4,096 | nan |
| voyage/voyage-lite-01 | $0.1 | $0 | 4,096 | nan |
| voyage/voyage-large-2 | $0.12 | $0 | 16,000 | nan |
| voyage/voyage-law-2 | $0.12 | $0 | 16,000 | nan |
| voyage/voyage-code-2 | $0.12 | $0 | 16,000 | nan |
| voyage/voyage-2 | $0.1 | $0 | 4,000 | nan |
| voyage/voyage-lite-02-instruct | $0.1 | $0 | 4,000 | nan |
| voyage/voyage-finance-2 | $0.12 | $0 | 32,000 | nan |
| databricks/databricks-meta-llama-3-1-405b-instruct | $5 | $15 | 128,000 | 128000 |
| databricks/databricks-meta-llama-3-1-70b-instruct | $1 | $3 | 128,000 | 128000 |
| databricks/databricks-dbrx-instruct | $0.75 | $2.25 | 32,768 | 32768 |
| databricks/databricks-meta-llama-3-70b-instruct | $1 | $3 | 128,000 | 128000 |
| databricks/databricks-llama-2-70b-chat | $0.5 | $1.5 | 4,096 | 4096 |
| databricks/databricks-mixtral-8x7b-instruct | $0.5 | $1 | 4,096 | 4096 |
| databricks/databricks-mpt-30b-instruct | $1 | $1 | 8,192 | 8192 |
| databricks/databricks-mpt-7b-instruct | $0.5 | $0 | 8,192 | 8192 |
| databricks/databricks-bge-large-en | $0.1 | $0 | 512 | nan |
| databricks/databricks-gte-large-en | $0.13 | $0 | 8,192 | nan |
| azure/gpt-4o-mini-2024-07-18 | $0.16 | $0.66 | 128,000 | 16384 |
| amazon.titan-embed-image-v1 | $0.8 | $0 | 128 | nan |
| azure_ai/mistral-large-2407 | $2 | $6 | 128,000 | 4096 |
| azure_ai/ministral-3b | $0.04 | $0.04 | 128,000 | 4096 |
| azure_ai/Llama-3.2-11B-Vision-Instruct | $0.37 | $0.37 | 128,000 | 2048 |
| azure_ai/Llama-3.2-90B-Vision-Instruct | $2.04 | $2.04 | 128,000 | 2048 |
| azure_ai/Phi-3.5-mini-instruct | $0.13 | $0.52 | 128,000 | 4096 |
| azure_ai/Phi-3.5-vision-instruct | $0.13 | $0.52 | 128,000 | 4096 |
| azure_ai/Phi-3.5-MoE-instruct | $0.16 | $0.64 | 128,000 | 4096 |
| azure_ai/Phi-3-mini-4k-instruct | $0.13 | $0.52 | 4,096 | 4096 |
| azure_ai/Phi-3-mini-128k-instruct | $0.13 | $0.52 | 128,000 | 4096 |
| azure_ai/Phi-3-small-8k-instruct | $0.15 | $0.6 | 8,192 | 4096 |
| azure_ai/Phi-3-small-128k-instruct | $0.15 | $0.6 | 128,000 | 4096 |
| azure_ai/Phi-3-medium-4k-instruct | $0.17 | $0.68 | 4,096 | 4096 |
| azure_ai/Phi-3-medium-128k-instruct | $0.17 | $0.68 | 128,000 | 4096 |
| xai/grok-beta | $5 | $15 | 131,072 | 131072 |
| claude-3-5-haiku-20241022 | $0.8 | $4 | 200,000 | 8192 |
| vertex_ai/claude-3-5-haiku@20241022 | $1 | $5 | 200,000 | 8192 |
| openrouter/anthropic/claude-3-5-haiku | $1 | $5 | nan | nan |
| openrouter/anthropic/claude-3-5-haiku-20241022 | $1 | $5 | 200,000 | 8192 |
| anthropic.claude-3-5-haiku-20241022-v1:0 | $0.8 | $4 | 200,000 | 8192 |The following table provides a reference for supported OpenAI models, including their input/output costs (per million tokens), context window limits, and max output token limits. Use these identifiers (e.g., openai/gpt-4o-2024-11-20) when calling the library to ensure accurate cost estimation.
| openai/gpt-4-0125-preview | $10 | $30 | 128,000 | 4096 |
| openai/gpt-4-vision-preview | $10 | $30 | 128,000 | 4096 |
| openai/gpt-4-1106-vision-preview | $10 | $30 | 128,000 | 4096 |
| openai/gpt-3.5-turbo | $1.5 | $2 | 16,385 | 4096 |
| openai/gpt-3.5-turbo-0301 | $1.5 | $2 | 4,097 | 4096 |
| openai/gpt-3.5-turbo-0613 | $1.5 | $2 | 4,097 | 4096 |
| openai/gpt-3.5-turbo-1106 | $1 | $2 | 16,385 | 4096 |
| openai/gpt-3.5-turbo-0125 | $0.5 | $1.5 | 16,385 | 4096 |
| openai/gpt-3.5-turbo-16k | $3 | $4 | 16,385 | 4096 |
| openai/gpt-3.5-turbo-16k-0613 | $3 | $4 | 16,385 | 4096 |
| openai/ft:gpt-3.5-turbo | $3 | $6 | 16,385 | 4096 |
| openai/ft:gpt-3.5-turbo-0125 | $3 | $6 | 16,385 | 4096 |
| openai/ft:gpt-3.5-turbo-1106 | $3 | $6 | 16,385 | 4096 |
| openai/ft:gpt-3.5-turbo-0613 | $3 | $6 | 4,096 | 4096 |
| openai/ft:gpt-4-0613 | $30 | $60 | 8,192 | 4096 |
| openai/ft:gpt-4o-2024-08-06 | $3.75 | $15 | 128,000 | 16384 |
| openai/ft:gpt-4o-mini-2024-07-18 | $0.3 | $1.2 | 128,000 | 16384 |
| openai/text-embedding-3-large | $0.13 | $0 | 8,191 | nan |
| openai/text-embedding-3-small | $0.02 | $0 | 8,191 | nan |
| openai/text-embedding-ada-002 | $0.1 | $0 | 8,191 | nan |
| openai/text-embedding-ada-002-v2 | $0.1 | $0 | 8,191 | nan |
| openai/text-moderation-stable | $0 | $0 | 32,768 | 0 |
| openai/text-moderation-007 | $0 | $0 | 32,768 | 0 |
| openai/text-moderation-latest | $0 | $0 | 32,768 | 0 |
| openai/256-x-256/dall-e-2 | -- | -- | nan | nan |
| openai/512-x-512/dall-e-2 | -- | -- | nan | nan |
| openai/1024-x-1024/dall-e-2 | -- | -- | nan | nan |
| openai/hd/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| openai/hd/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| openai/hd/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| openai/standard/1024-x-1792/dall-e-3 | -- | -- | nan | nan |
| openai/standard/1792-x-1024/dall-e-3 | -- | -- | nan | nan |
| openai/standard/1024-x-1024/dall-e-3 | -- | -- | nan | nan |
| openai/whisper-1 | -- | -- | nan | nan |
| openai/tts-1 | -- | -- | nan | nan |
| openai/tts-1-hd | -- | -- | nan | nan |
| openai/gpt-4o-2024-11-20 | $2.5 | $10 | 128,000 | 16384 |
| openai/ft:gpt-4o-2024-11-20 | $3.75 | $15 | 128,000 | 16384 |
| openai/omni-moderation-latest | $0 | $0 | 32,768 | 0 |
| openai/omni-moderation-latest-intents | $0 | $0 | 32,768 | 0 |
| openai/omni-moderation-2024-09-26 | $0 | $0 | 32,768 | 0 |
| openai/gpt-4o-audio-preview-2024-12-17 | $2.5 | $10 | 128,000 | 16384 |
| openai/gpt-4o-mini-audio-preview-2024-12-17 | $0.15 | $0.6 | 128,000 | 16384 |
| openai/o1 | $15 | $60 | 200,000 | 100000 |
| openai/o1-2024-12-17 | $15 | $60 | 200,000 | 100000 |
| openai/gpt-4o-realtime-preview-2024-10-01 | $5 | $20 | 128,000 | 4096 |
| openai/gpt-4o-realtime-preview | $5 | $20 | 128,000 | 4096 |
| openai/gpt-4o-realtime-preview-2024-12-17 | $5 | $20 | 128,000 | 4096 |
| openai/gpt-4o-mini-realtime-preview | $0.6 | $2.4 | 128,000 | 4096 |
| openai/gpt-4o-mini-realtime-preview-2024-12-17 | $0.6 | $2.4 | 128,000 | 4096 |
| openai/o3-mini | $1.1 | $4.4 | 200,000 | 100000 |
| openai/o3-mini-2025-01-31 | $1.1 | $4.4 | 200,000 | 100000 |
| openai/gpt-4.5-preview | $75 | $150 | 128,000 | 16384 |
| openai/gpt-4.5-preview-2025-02-27 | $75 | $150 | 128,000 | 16384 |
| openai/o1-pro | $150 | $600 | 200,000 | 100000 |
| openai/o1-pro-2025-03-19 | $150 | $600 | 200,000 | 100000 |
| openai/gpt-4o-search-preview-2025-03-11 | $2.5 | $10 | 128,000 | 16384 |
| openai/gpt-4o-search-preview | $2.5 | $10 | 128,000 | 16384 |
| openai/gpt-4o-mini-search-preview-2025-03-11 | $0.15 | $0.6 | 128,000 | 16384 |
| openai/gpt-4o-mini-search-preview | $0.15 | $0.6 | 128,000 | 16384 |
| openai/gpt-4.1 | $2 | $8 | 1,047,576 | 32768 |