fantasy

repository·main·Indexed 21 days ago

https://github.com/charmbracelet/fantasy

A Go library for building AI agents with support for multiple providers including Google Gemini (AI Studio and Vertex AI), Azure AI, AWS Bedrock, and Kronk for local inference. It provides a high-level API for agent orchestration via NewAgent, featuring streaming responses, tool registration (AgentTool and ProviderTool), custom stop conditions, and execution loop interception through PrepareStepFunction.

Tokens
14.2K
Snippets
48
Records
68
Agent score
75%

What's inside fantasy

  1. Use the Kronk provider for local inference

    main

    The Kronk provider allows you to perform hardware-accelerated local inference within Fantasy. It uses yzma and llama.cpp under the hood to provide a high-level API that mimics OpenAI-compatible interfaces.

    When configuring the Kronk provider, you only need to specify the desired model name. The provider will automatically handle downloading the model to your machine and using it for inference.

    Note: The Kronk provider is currently considered experimental. The API and behavior may change before reaching stability.

  2. Authenticate and configure Vertex AI

    main

    To use Vertex AI, follow these steps to authenticate your account, create/set a project, and enable the necessary APIs:

    1. Authenticate your account: Run gcloud auth login.
    2. Create a project: Use the Google Cloud Console or the gcloud projects create command.
    3. Set your active project: Use gcloud config set project {YOUR_PROJECT_ID}.
    4. Enable Vertex AI: Enable the aiplatform.googleapis.com service.
    5. Setup Application Default Credentials (ADC): Run gcloud auth application-default login so that libraries can locate your credentials.
    # Authenticate user account
    gcloud auth login
    
    # Create a new project
    gcloud projects create {YOUR_PROJECT_ID} --name="{YOUR_PROJECT_NAME}"
    
    # Set the active project
    gcloud config set project {YOUR_PROJECT_ID}
    
    # Enable the Vertex AI API
    gcloud services enable aiplatform.googleapis.com
    
    # Setup Application Default Credentials
    gcloud auth application-default login
  3. Register and manage tools for an Agent

    main

    Fantasy distinguishes between two types of tools:

    1. AgentTool: Client-side tools that the agent orchestrates. These are registered using WithTools. They have a defined schema (parameters and required fields) that the agent validates before execution.
    2. ProviderTool: Tools defined by the AI provider (e.g., built-in web search). These are registered using WithProviderDefinedTools.

    If a ProviderTool also implements the ExecutableProviderTool interface, it is treated as a tool that can be executed locally by the client while still being passed to the provider for wire formatting.

  4. Implement custom provider options with ProviderOptionsData

    main

    When building a custom provider, you must implement the ProviderOptionsData interface to handle provider-specific configuration. This interface requires implementing Options(), json.Marshaler, and json.Unmarshaler.

    To ensure proper serialization within the provider registry, use the recommended pattern of defining a type constant and registering the type in an init() function using fantasy.RegisterProviderType. Use the provided generic helpers fantasy.MarshalProviderType and fantasy.UnmarshalProviderType inside your JSON methods to handle the type-routing logic.

    // 1. Define type constant
    const TypeMyProviderOptions = "myprovider.options"
    
    type MyProviderOptions struct {
        Field string `json:"field"`
    }
    
    // 2. Register in init()
    func init() {
        fantasy.RegisterProviderType(TypeMyProviderOptions, func(data []byte) (fantasy.ProviderOptionsData, error) {
            var opts MyProviderOptions
            if err := json.Unmarshal(data, &opts); err != nil {
                return nil, err
            }
            return &opts, nil
        })
    }
    
    // 3. Implement interface methods
    func (*MyProviderOptions) Options() {}
    
    func (m MyProviderOptions) MarshalJSON() ([]byte, error) {
        type plain MyProviderOptions
        return fantasy.MarshalProviderType(TypeMyProviderOptions, plain(m))
    }
    
    func (m *MyProviderOptions) UnmarshalJSON(data []byte) error {
        type plain MyProviderOptions
        var p plain
        if err := fantasy.UnmarshalProviderType(data, &p); err != nil {
            return err
        }
        *m = MyProviderOptions(p)
        return nil
    }
  5. JSON Structure of Content Types

    main

    The project uses a tagged union pattern for JSON serialization. Most content types follow a structure where a top-level type field determines the schema of the data field.

    Content JSON Schema

    {
      "type": "text | reasoning | file | source | tool_call | tool_result",
      "data": { ... type-specific fields ... }
    }

    Message JSON Schema

    Messages include a role and an array of content parts, along with optional provider-specific options.

    {
      "role": "user | assistant | system | ...",
      "content": [ ... message parts ... ],
      "provider_options": { "key": "value" }
    }
  6. Use StopConditions to control agent execution

    main

    Agents execute in multiple steps (e.g., model generates a tool call, tool is executed, model processes result). You can use StopCondition functions in AgentCall.StopWhen to decide when the agent should stop looping.

    Available built-in conditions:

    • StepCountIs(count int): Stops after a specific number of steps.
    • HasToolCall(toolName string): Stops if the last step contains a call to the specified tool.
    • HasContent(contentType ContentType): Stops if the last step contains a specific content type (e.g., ContentTypeText).
    • FinishReasonIs(reason FinishReason): Stops when the model provides a specific finish reason.
    • MaxTokensUsed(maxTokens int64): Stops when the cumulative token usage across all steps exceeds the limit.
    // Stop after 5 steps
    call := fantasy.AgentCall{
        StopWhen: []fantasy.StopCondition{fantasy.StepCountIs(5)},
    }
    
    // Stop when a specific tool is called
    call := fantasy.AgentCall{
        StopWhen: []fantasy.StopCondition{fantasy.HasToolCall("search")},
    }
  7. Handle authentication refreshes with OnAuthRefresh

    main

    If your provider errors with an authentication failure (e.g., an expired SSO session), you can provide an OnAuthRefresh hook in your RetryOptions.

    When RetryWithExponentialBackoffRespectingRetryHeaders encounters an authentication error, it calls this hook exactly once.

    • If the hook returns nil: The library assumes credentials were successfully refreshed and restarts the entire retry sequence from the beginning with a fresh retry budget.
    • If the hook returns an error: The library stops retrying and returns the original authentication error.

    This is designed as a one-shot, potentially human-in-the-loop step to prevent infinite loops with invalid credentials.

    options := fantasy.RetryOptions{
        OnAuthRefresh: func(ctx context.Context, err *fantasy.ProviderError) error {
            // Logic to refresh tokens or re-authenticate
            return refreshMyToken(ctx)
        },
    }