DOCSEARCHbyAlgolia
Submit Docs

chatlas

repository·main·Indexed 41 days ago

https://github.com/posit-dev/chatlas

A Python library providing a simple and consistent interface for chatting with LLMs. It supports a wide variety of providers including OpenAI, Anthropic, Google, Mistral, and Ollama, and features tools for batch and parallel chat operations, structured data output via Pydantic, multi-modal content management, and automatic provider selection through ChatAuto.

Tokens
47.7K
Snippets
169
Records
228
Agent score
51%

What's inside chatlas

  1. How VCR test recording and replay works

    main

    Chatlas uses pytest-recording (wrapping vcrpy) to manage HTTP interactions during tests.

    How it works:

    1. Recording: During a test run, VCR captures HTTP requests and responses and saves them as YAML files called "cassettes" in the tests/_vcr/ directory.
    2. Replay: On subsequent runs, VCR intercepts the HTTP calls and returns the saved responses from the cassettes instead of making live network requests.

    Benefits:

    • Fast CI: Tests run without network latency.
    • No secrets in CI: Replay mode uses dummy API keys, so real credentials aren't needed in CI environments.
    • Deterministic tests: Ensures the same response is received every time, preventing flakiness caused by changing model outputs or network issues.

  2. How the chatlas documentation is built

    main

    The documentation site is a composition of several tools:

    • quartodoc: Reads docstrings from the chatlas/ package to generate API reference pages in docs/reference/ and creates the docs/_sidebar.yml navigation file. Configuration is managed in the quartodoc: section of docs/_quarto.yml.
    • Quarto: The primary engine that renders guides (found in docs/*.qmd) and the generated API references into a static website. Site-wide configuration is located in docs/_quarto.yml.
    • interlinks: Provides cross-linking capabilities between Python, Pydantic, and chatlas API objects.
    • Code Execution: Some .qmd guides contain executable Python cells that perform real API calls (e.g., to OpenAI or Anthropic). To render these locally, you must have the relevant API keys set in your environment. In CI, these outputs are cached using Quarto's freeze mechanism located in docs/_freeze/.

  3. Verify cassettes for leaked secrets

    main

    VCR automatically filters common sensitive headers (e.g., authorization, x-api-key, user-agent). However, you should always verify your cassettes before committing them to version control.

    Manual check: Run the following command to scan all cassettes for potential secrets using Claude:

    make check-vcr-secrets

    Automated CI check: The check-vcr-secrets.yml workflow automatically scans cassettes on PRs and pushes that modify tests/_vcr/.

  4. Prerequisites for building the documentation

    main

    To build the chatlas documentation locally, you need the following:

    1. Quarto: Must be installed on your system. Refer to the Quarto install guide for installation instructions.
    2. Python Dependencies: You must install the docs optional-dependency group from pyproject.toml. This group contains quartodoc and other documentation tools.

    You can install all dependencies (including the docs extra) using:

    make setup

    Note: The make quartodoc, make docs, and make docs-preview commands automatically run uv sync --all-extras as a prerequisite, so you do not need to run make setup manually before them.

  6. Record or update VCR cassettes

    main

    To record new cassettes or overwrite existing ones, you must set your real API keys in the environment and use the --record-mode=rewrite flag with pytest.

    Record all cassettes:

    export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="..."

# Using make (if available)
make update-snaps-vcr

# Or directly with pytest
uv run pytest --record-mode=rewrite

    Record a specific test file:

    uv run pytest tests/test_provider_openai.py -v --record-mode=rewrite

    Record a single test:

    uv run pytest tests/test_provider_openai.py::test_openai_simple_request -v --record-mode=rewrite
    # Record all cassettes
export OPENAI_API_KEY="sk-..."
export ANTHROPIC_API_KEY="..."

# Record with --record-mode=rewrite (overwrites existing)
make update-snaps-vcr

# Or directly with pytest
uv run pytest --record-mode=rewrite

  7. Common commands for building and previewing documentation

    main

    Use the following make commands to manage the documentation lifecycle. All commands automatically ensure dependencies are synced via uv sync --all-extras before execution.

    • make quartodoc: Generates API reference stubs and handles interlinks. This is a prerequisite for building the full site.
    • make docs: Runs quartodoc and then quarto render docs to build the complete static website.
    • make docs-preview: Runs quartodoc and then quarto preview docs to serve the documentation locally for live viewing.
    make quartodoc
make docs
make docs-preview

  8. Quick start with chatlas

    main

    To build a basic chat application, follow these steps:

    1. Choose a provider: Select a model provider like ChatOpenAI or ChatAnthropic.
    2. Configure credentials: Ensure you have the necessary credentials set up for your chosen provider.
    3. Initialize and chat: Create a Chat client instance and use the .chat() method to send prompts.

    You can optionally provide a model name and a system_prompt during initialization, and register Python functions as tools using .register_tool().

    from chatlas import ChatOpenAI

# Initialize the client with a specific model and system prompt
chat = ChatOpenAI(
    model="gpt-4.1-mini",
    system_prompt="You are a helpful assistant.",
)

# Register a function as a tool for the LLM to use
def get_current_weather(lat: float, lng: float):
    """Get the current weather for a given location."""
    return "sunny"

chat.register_tool(get_current_weather)

# Send a prompt to the model
chat.chat("How's the weather in San Francisco?")

  10. How parallel chat execution handles tools

    main

    The parallel_chat family of functions implements a multi-phase execution model to support tool calling across multiple concurrent conversations:

    1. Phase 1 (Parallel Submission): All initial prompts are submitted to the LLM in parallel, respecting max_active and rpm limits.
    2. Phase 2 (Sequential Tool Processing): If any conversations request tool calls, those tool requests are processed sequentially in their original submission order to manage complexity and state.
    3. Phase 3 (Parallel Result Submission): Once tool results are obtained, they are submitted back to the LLM in parallel.
    4. Repeat: This cycle repeats until all conversations have reached a terminal state (no more tool calls pending).

    This ensures that while the LLM requests are parallelized, the execution of local tools remains controlled.

  11. How Bedrock APIs and endpoints work

    main

    Chatlas manages different endpoints based on the selected api:

    1. Converse API (api="converse"): Uses the bedrock-runtime endpoint. It is the standard path for most models.
    2. Mantle APIs (api="messages" or api="responses"): Uses the bedrock-mantle endpoint.
      • "messages" uses the /anthropic path.
      • "responses" uses the /openai/v1 path.

    Endpoint Overrides: You can override these endpoints using standard AWS environment variables:

    • AWS_ENDPOINT_URL_BEDROCK_RUNTIME for the Converse API.
    • AWS_ENDPOINT_URL_BEDROCK_MANTLE for the Mantle APIs (messages and responses).

  12. How Tool parameter schemas are generated

    main

    Chatlas uses Pydantic and Python type hints to build the JSON Schema required by LLM providers (like OpenAI).

    • Inference: If no model is provided to Tool.from_func, Chatlas inspects the function signature. It supports Annotated types to extract FieldInfo (for descriptions, defaults, etc.).
    • Validation: If you provide a Pydantic model, Chatlas validates that the model's fields match the function's parameters exactly (including default values).
    • Cleaning: Chatlas automatically removes title fields from the generated JSON Schema to reduce noise in the prompt sent to the LLM.