AnyLanguageModel

AnyLanguageModel uses Swift 6.1 traits to conditionally include heavy dependencies. By default, no traits are enabled. You must specify the traits you need to enable support for specific backends, which helps reduce binary size and build times.

Available traits:

• CoreML: Enables Core ML model support (requires huggingface/swift-transformers)
• MLX: Enables MLX model support (requires ml-explore/mlx-swift-lm)
• Llama: Enables llama.cpp support (requires mattt/llama.swift)

Workaround for SPM dependency resolution errors: If you encounter the error "exhausted attempts to resolve the dependencies graph" when using traits, add the underlying dependencies for each trait directly to your Package.swift.

// In your Package.swift
dependencies: [
    .package(
        url: "https://github.com/huggingface/AnyLanguageModel.git",
        from: "0.8.0",
        traits: ["CoreML", "MLX"]
    ),
    .package(url: "https://github.com/huggingface/swift-transformers", from: "1.0.0"), // CoreML
    .package(url: "https://github.com/ml-explore/mlx-swift-lm", from: "2.25.5"),       // MLX
    .package(url: "https://github.com/mattt/llama.swift", from: "2.0.0"),              // Llama
]

Uses the Anthropic Messages API. Supports tool use and multimodal (image) inputs.

Custom Options:

• thinking: Configure extended thinking with budgetTokens.
• toolChoice: Control tool selection.
• serviceTier: e.g., .priority.

let model = AnthropicLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["ANTHROPIC_API_KEY"]!,
    model: "claude-sonnet-4-5-20250929"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])

// Multimodal example
let response = try await session.respond(
    to: "Explain the key parts of this diagram",
    image: .init(
        data: try Data(contentsOf: URL(fileURLWithPath: "/path/to/diagram.png")),
        mimeType: "image/png"
    )
)

// Custom options
var options = GenerationOptions(temperature: 0.7)
options[custom: AnthropicLanguageModel.self] = .init(
    topP: 0.9,
    topK: 40,
    stopSequences: ["END", "STOP"],
    thinking: .init(budgetTokens: 4096),
    toolChoice: .auto,
    serviceTier: .priority
)

Connect to a local Ollama instance via its HTTP API. By default, it connects to http://localhost:11434.

Multimodal usage: Ensure you use a vision-capable model (e.g., a -vl variant) to pass images in your prompts.

// Default connection
let model = OllamaLanguageModel(model: "qwen3")

// Custom endpoint
let model = OllamaLanguageModel(
    endpoint: URL(string: "http://remote-server:11434")!,
    model: "llama3.2"
)

let session = LanguageModelSession(model: model)

// Multimodal example
let response = try await session.respond(
    to: "Compare these posters and summarize their differences",
    images: [
        .init(url: URL(string: "https://example.com/poster1.jpg")!),
        .init(url: URL(fileURLWithPath: "/path/to/poster2.jpg"))
    ]
)

// Custom parameters
var options = GenerationOptions(temperature: 0.8)
options[custom: OllamaLanguageModel.self] = [
    "seed": .int(42),
    "repeat_penalty": .double(1.2),
    "num_ctx": .int(4096),
    "stop": .array([.string("###")])
]

Instead of parsing raw strings, you can request strongly typed Swift data structures using guided generation. This is supported by all on-device models (Apple Foundation Models, Core ML, MLX, llama.cpp) and cloud providers (OpenAI, Open Responses, Anthropic, and Gemini).

To use it, define a struct conforming to @Generable and use the @Guide property wrapper to provide descriptions or constraints (like .range) for specific fields. Pass the struct type to the generating: parameter of the session.respond method.

@Generable(description: "Basic profile information about a cat")
struct CatProfile {
    var name: String

    @Guide(description: "The age of the cat", .range(0...20))
    var age: Int

    @Guide(description: "A one sentence profile about the cat's personality")
    var profile: String
}

let session = LanguageModelSession(model: model)
let response = try await session.respond(
    to: "Generate a cute rescue cat",
    generating: CatProfile.self
)
print(response.content)

Run GGUF quantized models via llama.cpp. This requires the Llama trait.

Package.swift setup:

.package(
    url: "https://github.com/huggingface/AnyLanguageModel.git",
    from: "0.8.0",
    traits: ["Llama"]
)

Llama Configuration: Use LlamaLanguageModel custom options to control runtime parameters like contextSize, batchSize, threads, seed, and sampling parameters (temperature, topK, topP, repeatPenalty, etc.).

let model = LlamaLanguageModel(modelPath: "/path/to/model.gguf")

let session = LanguageModelSession(model: model)
let response = try await session.respond {
    Prompt("Translate 'hello world' to Spanish")
}

// Example with custom options
var options = GenerationOptions(temperature: 0.8)
options[custom: LlamaLanguageModel.self] = .init(
    contextSize: 4096,
    batchSize: 512,
    threads: 8,
    seed: 42,
    temperature: 0.7,
    topK: 40,
    topP: 0.95,
    repeatPenalty: 1.2,
    repeatLastN: 128,
    frequencyPenalty: 0.1,
    presencePenalty: 0.1,
    mirostat: .v2(tau: 5.0, eta: 0.1)
)

let response = try await session.respond(
    to: "Write a story",
    options: options
)

Tool calling is supported by all providers except llama.cpp. To implement tools, create a type that conforms to the Tool protocol.

1. Define a name and description for the tool.
2. Define an internal Arguments struct marked with @Generable to specify the schema for the tool's input.
3. Implement the call(arguments:) method to execute the tool logic.
4. Pass the tools to the LanguageModelSession during initialization using the tools: parameter.

struct WeatherTool: Tool {
    let name = "getWeather"
    let description = "Retrieve the latest weather information for a city"

    @Generable
    struct Arguments {
        @Guide(description: "The city to fetch the weather for")
        var city: String
    }

    func call(arguments: Arguments) async throws -> String {
        "The weather in \(arguments.city) is sunny and 72°F / 23°C"
    }
}

let session = LanguageModelSession(model: model, tools: [WeatherTool()])

let response = try await session.respond { 
    Prompt("How's the weather in Cupertino?") 
}
print(response.content)

Uses the Gemini API. Supports tool use, multimodal inputs, and a specialized "thinking process".

Thinking Mode: Configure via GeminiLanguageModel.CustomGenerationOptions.thinking:

• .dynamic: Dynamic budget allocation.
• .budget(Int): Explicit token budget.
• .disabled: Default behavior.

Server-side Tools: Gemini can use tools that execute on Google's infrastructure. These are configured via serverTools in custom options:

• .googleSearch: Real-time web information.
• .googleMaps(latitude:longitude:): Location-aware responses.
• .codeExecution: Python code execution.
• .urlContext: Content analysis from URLs.

let model = GeminiLanguageModel(
    apiKey: ProcessInfo.processInfo.environment["GEMINI_API_KEY"]!,
    model: "gemini-2.5-flash"
)

let session = LanguageModelSession(model: model, tools: [WeatherTool()])

// Multimodal example
let response = try await session.respond(
    to: "Identify the plants in this photo",
    image: .init(url: URL(string: "https://example.com/garden.jpg")!)
)

// Thinking mode configuration
var options = GenerationOptions()
options[custom: GeminiLanguageModel.self] = .init(thinking: .dynamic)

// Server-side tools configuration
var toolOptions = GenerationOptions()
toolOptions[custom: GeminiLanguageModel.self] = .init(
    serverTools: [
        .googleSearch,
        .googleMaps(latitude: 35.6580, longitude: 139.7016)
    ]
)

let response = try await session.respond(to: "What coffee shops are nearby?", options: toolOptions)