AISDK Rust Library

The React + Axum Agent example implements several advanced AI patterns:

• Tool Calling with #[tool]: Uses the aisdk.rs macro to define callable functions with structured inputs and outputs.
• Streaming + Tool Progress: Streams model responses while simultaneously tracking and emitting tool execution events in real time.
• Axum SSE Integration: Utilizes AxumSseResponse to manage state and stream responses via Server-Sent Events (SSE).
• Vercel AI SDK Integration: Uses the useChat hook from the Vercel AI SDK on the frontend to handle text and tool-call streaming seamlessly.

AISDK includes a file-based prompt template system powered by the Tera template engine. This allows for reusable templates with variable substitution, conditionals, loops, and inclusion.

To use this feature, enable it via cargo:

cargo add aisdk --features prompt

To run the frontend application for the AI Agent demo, follow these steps:

1. Install dependencies using pnpm install.
2. Start the development server with pnpm dev.
3. Access the application at http://localhost:5173 in your browser.

Note: This frontend requires a running backend to function.

The frontend requires the backend service to be running. To start the backend from the frontend directory, navigate to the backend folder and use cargo run:

cd ../backend && cargo run

To add AISDK to your Rust project, use cargo add. You can also enable specific provider features (like OpenAI) during installation.

Basic installation:

cargo add aisdk

Installation with OpenAI provider support:

cargo add aisdk --features openai

The generator is a single Python executable named provider-codegen.py.

Prerequisites

• Python 3.13+
• uv package manager

Standard Installation/Execution

You can run the generator directly using uv without a manual install:

uv run --python 3.13 --with typer --with requests provider-codegen.py [command]

Recommended Setup (Shell Alias)

For a better developer experience, add the following alias to your ~/.bashrc, ~/.zshrc, or equivalent shell configuration file:

alias provider-codegen='uv run --python 3.13 --with typer --with requests ./scripts/provider-codegen.py'

After adding the alias, reload your shell (e.g., source ~/.bashrc) to use the simplified provider-codegen command.

Use the openai-compatible command to generate provider modules using macros. This is useful for adding support for new models that follow the OpenAI API specification.

Usage Patterns

• Generate all providers: provider-codegen openai-compatible
• Generate a specific provider: provider-codegen openai-compatible <provider_name> (e.g., deepseek)
• Include capabilities: Use the --with-capabilities flag or the -c shorthand to also generate the capabilities.rs file.

Output

• Creates src/providers/{provider}/mod.rs
• Optionally creates src/providers/{provider}/capabilities.rs if requested.

# All OpenAI-compatible providers
provider-codegen openai-compatible

# Single provider
provider-codegen openai-compatible deepseek

# With capabilities
provider-codegen openai-compatible --with-capabilities
provider-codegen openai-compatible deepseek -c

Use the capabilities command to generate the capabilities.rs file for one or more providers. This file defines what specific features a provider supports.

Usage Patterns

• Generate for all providers: provider-codegen capabilities
• Generate for a single provider: provider-codegen capabilities <provider_name> (e.g., openai)
• Generate for multiple providers: provider-codegen capabilities <provider1> <provider2> (e.g., anthropic)

Output

• Creates src/providers/{provider}/capabilities.rs

# All provider capabilities
provider-codegen capabilities

# Single provider
provider-codegen capabilities openai

# Multiple providers
provider-codegen capabilities anthropic

If you need to use a model that is not explicitly defined in the SDK's type-safe constants, use the DynamicModel marker. This allows you to pass any OpenAI model identifier as a string.

Note: Using DynamicModel bypasses compile-time checks for model capabilities. You must ensure the model you specify supports the features (like tool calling) you intend to use in your application logic.

// Using DynamicModel to specify a model by string
let provider = OpenAI::<DynamicModel>::builder()
    .model_name("gpt-3.5-turbo")
    .api_key("sk-...)
    .build()
    .unwrap();

The client includes built-in resilience via a retry mechanism. When a request fails due to a retryable error, the client waits before attempting again.

Retryable Conditions:

• HTTP Status Codes: 429 (Too Many Requests), 502 (Bad Gateway), 503 (Service Unavailable), and 504 (Gateway Timeout).
• Transport Errors: Connection failures and timeouts.

Backoff Strategy:

• Exponential Backoff: The wait time increases exponentially based on the retry count (initial_wait * 2^retry_count).
• Retry-After Header: If the server provides a Retry-After header, the client respects this duration (capped by max_wait).
• Jitter: If the jitter feature is enabled, a random variation of ±10% is added to the backoff time to prevent the "thundering herd" problem.
• Limits: Retries are capped by max_retries and the wait time is capped by max_wait.