Stirrup Documentation

repository·main·Indexed 19 days ago

https://github.com/artificialanalysis/stirrup

A lightweight Python framework for building autonomous agents that allows models to drive workflows. Stirrup provides built-in support for context management, tool execution, and multimodal capabilities. It includes clients for OpenAI-compatible APIs, LiteLLM, and the OpenAI Responses API, as well as integrations for Slack and browser automation.

Tokens
71.8K
Snippets
187
Records
259
Agent score
67%

What's inside stirrup

  1. Overview of Code Execution Backends

    main

    Stirrup provides multiple backends for executing code in isolated environments via the CodeExecToolProvider interface. Each provider offers a code_exec tool for shell commands, file upload/download capabilities, and an isolated environment.

    BackendIsolationUse Case
    LocalCodeExecToolProviderTemp directoryDevelopment, trusted code
    DockerCodeExecToolProviderContainerProduction, semi-trusted code
    E2BCodeExecToolProviderCloud sandboxProduction, untrusted code
  2. What is included in a Stirrup cache?

    main

    When a task is cached, the following components are preserved:

    • Conversation messages and history: The full dialogue between the user and the agent.
    • Tool metadata: Metadata for tools, keyed by the accepted assistant turn.
    • Execution environment: All files present in the agent's execution environment.

    Note: The cache key is derived from the initial prompt. Using the exact same prompt will trigger the resumption of the existing cache.

  3. What are Skills in Stirrup

    main

    Skills are modular packages used to extend agent capabilities with domain-specific expertise. They provide a structured way to give agents instructions, scripts, and reference resources for specific tasks (e.g., data analysis, report writing).

    A skill is defined as a directory containing:

    • SKILL.md: The main instruction file containing YAML frontmatter (name, description) and detailed guidance.
    • reference/ (optional): A subdirectory for focused reference documentation.
    • scripts/ (optional): A subdirectory containing ready-to-use Python scripts for the agent to execute.

    When a skill is loaded, the agent receives a list of available skills in its system prompt, access to the skill files in its execution environment, and instructions on how to use them.

  4. What are ToolProviders and when to use them

    main

    A ToolProvider is an async context manager used to manage the lifecycle of resources required by tools. Unlike regular tools, providers allow you to handle setup and teardown logic for shared or temporary resources.

    Use a ToolProvider when your tools require:

    • Connections: HTTP clients, database connections, websockets.
    • Temporary resources: Temp directories, sandboxes, processes.
    • Cleanup logic: Releasing resources or closing connections.
    • Shared state: State that must be shared across multiple tool calls.
  5. How browser automation workflows work

    main

    The typical workflow for automating a browser session follows these steps:

    1. Navigate: Use browser_navigate or browser_search to reach a target page.
    2. Snapshot: Use browser_snapshot to retrieve the accessibility tree. This tree contains interactive elements paired with numerical indices.
    3. Interact: Use the indices from the snapshot to perform actions like browser_click or browser_input_text on specific elements.
    4. Repeat: Continue the snapshot-and-interact cycle until the task is complete.
  6. How Stirrup works: Core Abstractions

    main

    Stirrup is built around several key abstractions that manage the agent lifecycle and capabilities:

    • Agent: The central component that configures and runs the agent loop. It continues execution until a 'finish' tool is called or the maximum number of turns is reached.
    • session(): A context manager used to set up tools, manage files, and handle cleanup during an agent's run.
    • Tool: The mechanism for defining capabilities using Pydantic parameters.
    • ToolProvider: Manages tools that require a lifecycle, such as maintaining connections or managing temporary directories.
    • default_tools(): A collection of standard tools provided out-of-the-box.
  7. Handle Context Overflow and Recovery

    main

    By default, Stirrup attempts to recover from ContextOverflowError by shortening the conversation (removing the latest completed assistant turn) and retrying.

    Important distinctions:

    • Context Overflow: Recoverable via summarization/turn removal.
    • Output Token Limit: If the model hits its max_tokens limit, it raises OutputTokenLimitError and aborts the run. It does not retry.
    • Summarization Failure: If the summarization model itself hits an output limit, the run is not recovered.

    To disable automatic recovery and fail immediately on overflow:

    agent = Agent(client=client, name="my_agent", recover_from_context_overflow=False)
  8. Understand the Stirrup project structure for customization

    main

    When customizing the framework, use the following directory mapping to locate the relevant logic:

    DirectoryPurpose
    src/stirrup/clients/LLM client implementations (use this for custom LLM providers or API modifications)
    src/stirrup/core/Agent class, models, and exceptions (modify core/agent.py to change Agent loop behavior)
    src/stirrup/tools/Tool implementations (use this for new tools or modifying existing ones)
    src/stirrup/tools/code_backends/Code execution backends (use this for custom execution environments)
    src/stirrup/utils/Logging and text utilities
    src/stirrup/prompts/System prompt templates
  9. Perform Group By operations

    main

    Group By operations collapse the dataset into groups based on one or more columns. Use .group_by() followed by .agg() to apply aggregation functions to each group.

    Basic Group By

    You can group by a single column or a list of columns.

    Common Aggregation Functions

    • Numeric: sum(), mean(), median(), std(), var(), min(), max().
    • Counting: pl.len(), n_unique().
    • First/Last: first(), last(), min(), max().
    • Quantiles: quantile(q).

    Computed Aggregations

    You can perform arithmetic within .agg() to calculate ratios, weighted averages, ranges, or coefficients of variation.

    Collect Into Lists

    Aggregations can return lists of values per group using col("column") or concatenate strings using .str.concat().

    # Single grouping column
    result = df.group_by("category").agg(
        col("value").sum().alias("total"),
        col("value").mean().alias("avg"),
        pl.len().alias("count"),
    )
    
    # Multiple grouping columns
    result = df.group_by(["region", "category"]).agg(
        col("revenue").sum().alias("total_revenue"),
        col("quantity").sum().alias("total_quantity"),
    )
    
    # Computed Aggregations
    df.group_by("category").agg([
        (col("successes").sum() / col("attempts").sum()).alias("success_rate"),
        (col("value") * col("weight")).sum() / col("weight").sum().alias("weighted_avg"),
    ])
    
    # Collect Into Lists
    df.group_by("user_id").agg([
        col("product").alias("products"),
        col("product").unique().alias("unique_products"),
    ])
  10. Integration contract for custom clients

    main

    When implementing a client, be aware of the following guarantees provided by the Stirrup framework:

    • Stable identity: AssistantMessage.id is assigned once at construction and remains stable through serialization (model_dump/model_validate) cycles.
    • Same object in history: The exact object returned by generate is what is appended to history. The framework does not copy or rebuild messages unless performing summarization or context-overflow unwinding.
    • Subclasses are preserved: You can return an AssistantMessage subclass to carry integration-specific state. To prevent this state from being serialized into history, mark the extra fields with exclude=True.
    • metadata is opaque: The framework does not interact with the metadata field. You should namespace your keys (e.g., "myco/...") to avoid collisions with other users.
    • Summaries carry lineage: SummaryMessage objects record the IDs of the assistant messages they replaced via replaced_ids, allowing for reconstruction of history lineage.
  11. Use Window Functions with .over()

    main

    Window functions compute values across related rows without collapsing the dataset. Use the .over() method to define the partition (the group) over which the function operates.

    Ranking

    Use .rank(method=...) to assign ranks within a group. Supported methods:

    • "ordinal": 1, 2, 3, 4 (unique ranks).
    • "dense": 1, 2, 2, 3 (no gaps for ties).
    • "min": 1, 2, 2, 4 (min rank for ties).
    • "max": 1, 3, 3, 4 (max rank for ties).
    • "average": 1, 2.5, 2.5, 4 (average rank for ties).

    Lag and Lead

    Use .shift(n) to access values from other rows within a group:

    • shift(1): Previous value (Lag).
    • shift(-1): Next value (Lead).

    Cumulative Functions

    • cum_sum(): Running total.
    • cum_mean(): Running average.
    • cum_count(): Running count.
    • cum_min() / cum_max(): Running minimum/maximum.
    # Sum per group (added to each row)
    df = df.with_columns(
        col("value").sum().over("category").alias("category_total")
    )
    
    # Ranking
    df = df.with_columns(
        col("score").rank(method="dense", descending=True).over("department").alias("dense_rank")
    )
    
    # Lag/Lead
    df = df.with_columns(
        col("value").shift(1).over("user_id").alias("prev_value")
    )
    
    # Cumulative
    df = df.with_columns(
        col("value").cum_sum().over("user_id").alias("running_total")
    )
  12. Understanding Agent output from session.run()

    main

    The run() method returns a tuple of three values: (finish_params, history, metadata).

    finish_params

    Contains the agent's final response when it calls the finish tool. It includes:

    • reason: A string explanation of what was accomplished.
    • paths: A list of file paths created or modified during execution.

    history

    A list of message groups (SystemMessage, UserMessage, AssistantMessage, ToolMessage).

    AssistantMessage objects use a blocks list to preserve the model's emission order (e.g., reasoning $\rightarrow$ text $\rightarrow$ tool call). Block types include text, reasoning, tool_call, and various media kinds.

    Note: Accessing deprecated attributes like .content or .tool_calls on an AssistantMessage will emit a DeprecationWarning. Use convenience accessors like joined_text, final_text, tool_call_blocks, or reasoning_blocks instead.

    metadata

    A dictionary containing metadata from tool executions and token usage. You can use aggregate_metadata to combine metadata across all tool calls.

    # Example of unpacking run results
    finish_params, history, metadata = await session.run("Your task")
    
    # Example of accessing finish_params
    print(finish_params["reason"])
    print(finish_params["paths"]) # e.g., ["output.png"]
    
    # Example of aggregating metadata
    from stirrup import aggregate_metadata
    aggregated = aggregate_metadata(metadata)
    print(f"Total tokens: {aggregated['token_usage'].total}")