Ollama Swift Client

Models like deepseek-r1 support a "thinking" mode where they output their reasoning process. Enable this by setting think: true in generate or chat calls. You can access the reasoning via the thinking property on the response.

// Generate with thinking enabled
let response = try await client.generate(
    model: "deepseek-r1:8b",
    prompt: "What is 17 * 23? Show your work.",
    think: true
)

print("Thinking: \(response.thinking ?? "None")")
print("Answer: \(response.response)")

To add ollama-swift to your project, add the following dependency to your Package.swift file. This library requires Swift 5.7+, macOS 13+, and a running instance of Ollama.

.package(url: "https://github.com/mattt/ollama-swift.git", from: "1.8.0")

You can use a default client configured for http://localhost:11434 or create a custom client by specifying a host URL and a user agent string.

import Ollama

// Use the default client (http://localhost:11434)
let client = Client.default

// Or create a custom client
let customClient = Client(host: URL(string: "http://your-ollama-host:11434")!, userAgent: "MyApp/1.0")

Before using this library, ensure your environment meets the following requirements:

• Swift: version 5.7 or higher
• macOS: version 13 or higher
• Ollama: Must be installed and running on your system.

The keepAlive parameter controls how long a model remains loaded in the Ollama server's memory. This helps balance between immediate response times and system resource usage.

Available options:

• .default: Use server default.
• .none: Unload model immediately after the request.
• .seconds(Int): Keep loaded for N seconds.
• .minutes(Int): Keep loaded for N minutes.
• .hours(Int): Keep loaded for N hours.
• .forever: Keep loaded indefinitely.

Note: Zero durations are treated as .none; negative durations are treated as .forever.

// Keep model loaded for 10 minutes
let response = try await client.generate(
    model: "llama3.2",
    prompt: "Hello!",
    keepAlive: .minutes(10)
)

To receive responses in real-time, use generateStream. This returns an async sequence of chunks that you can iterate over to process the response as it arrives.

do {
    let stream = try await client.generateStream(
        model: "llama3.2",
        prompt: "Tell me a joke about Swift programming.",
        options: [
            "temperature": 0.7,
            "max_tokens": 100
        ]
    )

    var fullResponse = ""
    for try await chunk in stream {
        // Process each chunk of the response as it arrives
        print(chunk.response, terminator: "")
        fullResponse += chunk.response
    }
    print("\nFull response: \(fullResponse)")
} catch {
    print("Error: \(error)")
}

The client provides methods to manage the local Ollama model library:

• listModels(): Returns a list of available models.
• showModel(name): Returns detailed information (Modelfile, parameters, template) for a specific model.
• pullModel(name): Downloads a model from the Ollama library.
• pushModel(name): Uploads a model to a namespace.

// List models
let models = try await client.listModels()

// Pull a model
let success = try await client.pullModel("llama3.2")

Use chatStream to get real-time partial completions for chat conversations. This is useful for UI elements that show text being typed out. You can also check for toolCalls or the done flag within the stream chunks.

do {
    let stream = try await client.chatStream(
        model: "llama3.2",
        messages: [
            .system("You are a helpful assistant."),
            .user("Write a short poem about Swift programming.")
        ]
    )

    var fullContent = ""
    for try await chunk in stream {
        // Process each chunk of the message as it arrives
        if let content = chunk.message.content {
            print(content, terminator: "")
            fullContent += content
        }
    }
    print("\nComplete poem: \(fullContent)")
} catch {
    print("Error: \(error)")
}

Use the generate method to produce a single text response from a model using a prompt. You can provide optional parameters like options (e.g., temperature, max_tokens) and keepAlive to manage model memory.

do {
    let response = try await client.generate(
        model: "llama3.2",
        prompt: "Tell me a joke about Swift programming.",
        options: [
            "temperature": 0.7,
            "max_tokens": 100
        ],
        keepAlive: .minutes(10)  // Keep model loaded for 10 minutes
    )
    print(response.response)
} catch {
    print("Error: \(error)")
}

Use the chat method to generate completions based on a conversation history. Messages are provided as an array of Chat.Message objects (e.g., .system, .user, .assistant).

do {
    let response = try await client.chat(
        model: "llama3.2",
        messages: [
            .system("You are a helpful assistant."),
            .user("In which city is Apple Inc. located?")
        ],
        keepAlive: .minutes(10)  // Keep model loaded for 10 minutes
    )
    print(response.message.content)
} catch {
    print("Error: \(error)")
}

Use embed() to generate vector embeddings for text. You can pass a single string via input or an array of strings via inputs for batch processing.

do {
    let response = try await client.embed(
        model: "llama3.2",
        input: "Here is an article about llamas..."
    )
    print("Embeddings: \(response.embeddings)")
} catch {
    print("Error: \(error)")
}