GoAI SDK

GoAI SDK provides a unified API for over 25 LLM providers. While the core functions like GenerateText, StreamText, GenerateObject[T], StreamObject[T], Embed, EmbedMany, GenerateImage, and GenerateVideo share a single interface, specific capabilities (like image generation or tool support) vary by provider.

Providers are categorized into tiers based on their implementation depth:

• Tier 1: Dedicated implementations with extended API support (e.g., OpenAI, Anthropic, Google, Bedrock, Azure, Vertex AI).
• Tier 2: Standard implementations (e.g., Cohere, Mistral, xAI, Groq, DeepSeek, MiniMax).
• Tier 3: Providers that typically use the internal/openaicompat codec (e.g., Fireworks, Together, OpenRouter, Perplexity, Cloudflare).
• Local / Custom: Local runners or generic compatible endpoints (e.g., Ollama, llama.cpp, vLLM, or any OpenAI-compatible endpoint).

GoAI provides a unified API for 25+ LLM providers with the following core capabilities:

• Text Generation: GenerateText (non-streaming) and StreamText (real-time streaming via Go channels).
• Structured Output: GenerateObject[T] (type-safe using generics) and StreamObject[T] (partial object streaming).
• Embeddings: Embed and EmbedMany with automatic chunking.
• Image Generation: GenerateImage (supports OpenAI, Google, Azure, Vertex AI).
• Tool Calling: Custom tools with automatic tool loops controlled by MaxSteps.
• MCP Support: Connect to Model Context Protocol (MCP) servers and automatically convert their tools for use in GoAI.
• Prompt Caching: Automatic cache control for supported providers like Anthropic, Bedrock, and MiniMax.
• Observability: Built-in integrations for Langfuse and OpenTelemetry.

The repository is organized into core logic, provider implementations, and shared internal utilities:

• goai/: Core SDK entrypoints (e.g., generate.go for GenerateText, object.go for GenerateObject, embed.go for embeddings).
• goai/provider/: Defines the interfaces (LanguageModel, EmbeddingModel, etc.) and contains specific provider implementations (OpenAI, Anthropic, Google, Bedrock, etc.).
• goai/internal/: Shared logic used across providers, such as openaicompat for shared codecs, sse for parsing Server-Sent Events, and httpc for HTTP helpers.
• goai/examples/: Contains 33 runnable examples, including MCP (Model Context Protocol) implementations.

The TokenSource interface abstracts LLM authentication, allowing providers to work with both static API keys and expiring credentials (like Azure Managed Identity, Google service accounts, or OAuth tokens).

Most providers accept a TokenSource via the WithTokenSource option. The provider calls the Token(ctx context.Context) method before every API request to retrieve the current credential.

Interface Definition:

type TokenSource interface {
    Token(ctx context.Context) (string, error)
}

import "github.com/zendev-sh/goai/provider"

// Example usage with a provider
model := openai.Chat("gpt-4o",
    openai.WithTokenSource(provider.StaticToken("sk-...")),
)

GoAI is built on several core principles to ensure a high-quality developer experience:

1. Go-native API: Uses functional options, interfaces, and composition rather than translating TypeScript patterns.
2. Minimal required dependencies: The core library depends only on the standard library and golang.org/x/oauth2. Optional features like OpenTelemetry are in separate submodules.
3. Provider-agnostic: The same code works across all providers; switching models is a single-line change.
4. Consumer flexibility: Consumers can manage their own auth, proxies, and custom endpoints using WithHTTPClient and TokenSource.
5. No middleware, no registry: Relies on Go's interface composition rather than adding unnecessary abstraction layers.

GoAI provides lifecycle hooks that allow you to intercept tool execution, transform outputs, and control the agent loop. These are useful for implementing permission gates, secret scanning, or custom termination logic.

Available Hooks:

• OnBeforeToolExecute: Use this as a permission gate before a tool runs.
• OnAfterToolExecute: Use this for post-execution tasks like secret scanning.
• OnBeforeStep: Use this for custom loop control logic.
• OnFinish: Triggered when the loop ends (e.g., when max_steps is reached).

export GEMINI_API_KEY=...
go run examples/hooks/main.go

Anthropic's support for structured output varies by model. GoAI handles this in two ways:

1. Synthetic Tool Injection (Default): For models without native support, GoAI injects a hidden tool containing the requested schema. The model is then forced to call this tool to provide the structured data.
2. Native output_format: For newer models (e.g., claude-opus-4-5, claude-sonnet-4-5, claude-haiku-4-5, and claude-5 series), you can use native support by setting structuredOutputMode: "outputFormat" in the provider options.

Use structuredOutputMode: "outputFormat" whenever the model version supports it to improve reliability and reduce latency.

Requesty models must be specified using a provider/model prefix format. Examples include:

• openai/gpt-4o-mini
• anthropic/claude-sonnet-4-5
• google/gemini-2.5-flash
• deepseek/deepseek-chat

The google.Tools object provides access to five built-in tools:

1. GoogleSearch(): Grounding with Google Search (Server-side). Supports WithWebSearch(), WithImageSearch(), and WithTimeRange().
2. URLContext(): Fetches and processes content from URLs provided in the prompt (Server-side).
3. CodeExecution(): Executes Python code in a sandbox (Server-side).
4. ComputerUse(opts...): Client-side computer use with configurable environment/excluded functions.
5. FileSearch(opts...): Grounding on Vertex AI Search or uploaded files (Server-side). Supports WithDynamicThreshold(t) and WithCorpus(id).

GoAI manages the tool loop lifecycle using two primary composable primitives:

1. AgentState + WithStateRef: Provides a race-free, observable handle into the current phase of the loop. This allows external goroutines (pollers) to monitor progress or decide when to inject work/signals.
2. WithStopWhen: A stop predicate that is evaluated after a step's LLM call and tool executions are complete, but before the next LLM call begins. This allows for custom exit logic.

Note on Scope: These primitives are fully supported for GenerateText and StreamText. They are ignored (with a stderr warning) for GenerateObject and StreamObject, which use fixed exit conditions tied to structured output parsing.

goai.GenerateText(ctx, model, 
    goai.WithStateRef(&state), 
    goai.WithStopWhen(predicate), 
    ...
)

Panics occurring within lifecycle hooks (OnRequest, OnResponse, OnStepFinish, OnFinish, OnBeforeStep) or the StopWhen predicate are no longer swallowed. They are now surfaced as a *PanicError.

• For Text Generation: GenerateText and GenerateObject return the error.
• For Streaming: Use stream.Err() for StreamText and StreamObject.
• Observability: You can register a new WithOnPanic hook to observe these panics.

Note: The tool execution path remains resilient and converts panics into tool errors.