Catwalk Documentation

repository·main·Indexed 20 days ago

https://github.com/charmbracelet/catwalk

Catwalk is an AI Provider Database that manages model information and capabilities across various providers via JSON files and specialized CLI tools. Designed as a storage layer for Crush-compatible data structures, it includes CLI tools to generate provider configurations for AIHubMix, Atlas Cloud, Avian, Baseten, Chutes, and GitHub Copilot.

Tokens
14.5K
Snippets
30
Records
51
Agent score
73%

What's inside catwalk

  1. Update the Zai provider manually

    main

    The zai provider requires manual data extraction because specific parameters are not available via standard overview pages.

    To update zai:

    1. Retrieve the model list and capabilities from https://docs.z.ai/guides/overview/overview.
    2. For context_window and default_max_tokens, do not use the overview page. Instead, extract the exact values from ./internal/providers/configs/openrouter.json.
  2. Add a new provider command

    main

    To extend Catwalk with a new provider, follow these steps:

    1. Create a new entry point at ./cmd/{provider-name}/main.go.
    2. Determine the available models. You should attempt to use the provider's API to list models. If no API endpoint exists, look for structured text formats in their documentation.
    3. Constraint: If you cannot programmatically determine the models, do not create a command. Instead, add the provider to the MANUAL_UPDATES.md file.
    4. Register the new command in the GitHub Actions workflow at .github/workflows/update.yml to ensure it is included in automated updates.
  3. Build and run Catwalk commands

    main

    You can use the following Go commands to interact with the Catwalk project:

    • Start the HTTP server: Runs the main server on port :8080.
    • Run a provider CLI: Builds and runs a specific CLI tool to update a provider's JSON file (e.g., go run ./cmd/openai).
    • Run tests: Executes all tests in the repository.
    go run .
    go run ./cmd/{provider-name}
    go test ./...
  4. Generate xAI provider configuration via xai CLI

    main

    The xai command-line tool fetches the latest available models from the xAI API and generates a structured JSON configuration file for use within the Catwalk ecosystem.

    To use this tool, you must have the XAI_API_KEY environment variable set. The tool performs the following steps:

    1. Fetches model metadata from https://api.x.ai/v1/language-models.
    2. Maps xAI model IDs to their shortest available aliases.
    3. Calculates pricing (converting cents per hundred million to dollars per million).
    4. Determines capabilities like context window size, reasoning support, and image modality support.
    5. Writes the resulting configuration to internal/providers/configs/xai.json.

    Note: The tool automatically skips models containing multi-agent in their ID.

    # Ensure your API key is set
    export XAI_API_KEY="your_api_key_here"
    
    # Run the tool (assuming it is built and in your PATH)
    xai
    
    # The output will be written to:
    # internal/providers/configs/xai.json
  5. Generate Avian provider configuration via avian CLI

    main

    The avian command-line tool fetches the latest models from the Avian API (https://api.avian.io/v1/models) and generates a provider configuration file compatible with catwalk.

    This tool automates the creation of internal/providers/configs/avian.json, mapping Avian's specific pricing and reasoning capabilities to the catwalk.Provider and catwalk.Model types. It specifically handles:

    • Mapping reasoning capabilities to reasoning_levels (low, medium, high).
    • Sorting models by name and then by ID.
    • Setting the provider type to OpenAICompat.
    • Configuring default large and small model IDs.
  6. Generate Synthetic provider configuration

    main

    The synthetic command-line tool fetches available models from the Synthetic API and generates a synthetic.json configuration file for use within Catwalk.

    Filtering Logic applied during generation:

    • Models must have a ContextLength of at least 20,000.
    • Models must support both text input and text output modalities.
    • Models must support tools.

    Configuration Details:

    • The generated file is saved to internal/providers/configs/synthetic.json.
    • The provider uses the OpenAICompat type.
    • The API endpoint is https://api.synthetic.new/openai/v1.
    • The API key is expected to be provided via the SYNTHETIC_API_KEY environment variable.
  7. Generate OpenRouter configuration via CLI

    main

    The openrouter command-line tool fetches the latest model list and endpoint details from the OpenRouter API to generate a structured openrouter.json configuration file. This file is used by Catwalk to provide a pre-configured OpenRouter provider.

    The tool performs the following logic during generation:

    • Fetches all available models from https://openrouter.ai/api/v1/models.
    • Filters for models that meet specific criteria: non-batch models, context length $\ge$ 20,000, support for tools, and text-to-text modality.
    • For each valid model, it attempts to fetch specific endpoints via https://openrouter.ai/api/v1/models/{model_id}/endpoints.
    • Selects the 'best' endpoint based on:
      1. Tool support (tools parameter).
      2. Context length.
      3. Uptime (last 30 minutes).
    • Calculates pricing per 1 million tokens (input, output, cached input, and cached output).
    • Determines reasoning capabilities (low, medium, high levels) if the reasoning parameter is supported.
    • Saves the resulting configuration to internal/providers/configs/openrouter.json.
  8. Initialize a Catwalk Client

    main

    You can create a new Client instance using New() or NewWithURL(url string).

    • New(): Automatically uses the CATWALK_URL environment variable. If that variable is not set, it defaults to http://localhost:8080.
    • NewWithURL(url): Allows you to specify a custom base URL for the service. The client includes a default HTTP timeout of 30 seconds.
    import "github.com/charmbracelet/catwalk"
    
    // Using environment variables or default localhost:8080
    client := catwalk.New()
    
    // Using a specific URL
    client, err := catwalk.NewWithURL("https://api.example.com")
  9. Generate OpenCode Go provider configuration

    main

    The opencode-go command fetches the latest model definitions from https://models.dev/api.json, processes the pricing and reasoning capabilities, and generates a provider configuration file.

    This tool is used to keep the opencode-go.json configuration in sync with the upstream model definitions. It automatically handles:

    • Rounding costs to two decimal places.
    • Mapping reasoning levels (e.g., low, medium, high) based on the model ID.
    • Sorting models by name and then by ID.

    The resulting configuration is written to internal/providers/configs/opencode-go.json with 0600 permissions.

  10. Generate io.net provider configuration

    main

    The ionet command-line tool fetches available models from the io.net API and generates a provider configuration file (ionet.json) used by Catwalk.

    It performs the following tasks:

    1. Fetches models from https://api.intelligence.io.solutions/api/v1/models.
    2. Filters models based on suitability for coding (e.g., prioritizing deepseek-r1, llama-3.2, mistral-nemo).
    3. Calculates costs per 1M tokens from per-token prices.
    4. Detects reasoning capabilities and reasoning levels (e.g., for gpt-oss).
    5. Writes the resulting configuration to internal/providers/configs/ionet.json.

    Note: The tool expects the IONET_API_KEY environment variable to be available for the provider configuration.

  11. Generate Baseten provider configuration

    main

    The baseten command-line tool fetches available models from the Baseten Model APIs and generates a structured JSON configuration file for use within the Catwalk ecosystem.

    Prerequisites

    You must set the BASETEN_API_KEY environment variable before running the tool.

    Workflow

    1. The tool queries https://inference.baseten.co/v1/models using your API key.
    2. It filters models that support both tools and text modalities.
    3. It calculates pricing per 1M tokens and maps model-specific reasoning capabilities (e.g., reasoning_effort levels).
    4. It writes the resulting configuration to internal/providers/configs/baseten.json.

    Output Format

    The generated file is a JSON object representing a catwalk.Provider with the following fixed properties:

    • Name: Baseten
    • ID: catwalk.InferenceProviderBaseten
    • APIKey: $BASETEN_API_KEY (as a literal string for environment variable substitution)
    • APIEndpoint: https://inference.baseten.co/v1
    • Type: catwalk.TypeOpenAICompat
    export BASETEN_API_KEY="your_api_key_here"
    go run cmd/baseten/main.go