DOCSEARCHbyAlgolia
Submit Docs

swift-transformers

repository·main·Indexed 23 days ago

https://github.com/huggingface/swift-transformers

A collection of Swift utilities for integrating language models into Swift applications. It provides idiomatic APIs for tokenization, chat templating, tool calling, and downloading models from the Hugging Face Hub, with specialized support for CoreML. Features include the AutoTokenizer class, Hub module for model snapshots, and a transformers-cli tool for text generation using exported .mlpackage models.

Tokens
1.9K
Snippets
7
Records
8
Agent score
30%

What's inside swift-transformers

  1. Load offline CoreML models and tokenizers

    main

    To avoid network requests when bundling models with your app, you can load a compiled CoreML model and a tokenizer from local files. Use AutoTokenizer.from(modelFolder:) to initialize a tokenizer from a local directory containing tokenizer_config.json and tokenizer.json, then pass it to LanguageModel.loadCompiled(url:tokenizer:).

    To download only the necessary tokenizer files via CLI for local bundling:

    huggingface-cli download \
  mistralai/Mistral-7B-Instruct-v0.3 \
  tokenizer.json tokenizer_config.json \
  --local-dir Examples/Mistral7B/local-tokenizer
    let compiledURL: URL = ... // path to .mlmodelc
let tokenizerFolder: URL = ... // folder containing tokenizer_config.json and tokenizer.json

// Construct the tokenizer from local files (inside an async context)
let tokenizer = try await AutoTokenizer.from(modelFolder: tokenizerFolder)
let model = try LanguageModel.loadCompiled(
    url: compiledURL,
    tokenizer: tokenizer
)

  2. Install swift-transformers via SwiftPM

    main

    Add swift-transformers to your Package.swift dependencies and include the Transformers product in your target dependencies.

    dependencies: [
    .package(url: "https://github.com/huggingface/swift-transformers", from: "1.3.0")
]

targets: [
    .target(
        name: "YourTargetName",
        dependencies: [
            .product(name: "Transformers", package: "swift-transformers")
        ]
    )
]

  3. Download models from the Hugging Face Hub

    main

    Use the Hub module to download model snapshots to a device. The Hub.snapshot method allows you to specify a Hub.Repo, filter for specific files using a matching array (supporting wildcards), and provides a progressHandler to monitor download progress.

    let repo = Hub.Repo(id: "mlx-community/Qwen2.5-0.5B-Instruct-2bit-mlx")
let modelDirectory: URL = try await Hub.snapshot(
    from: repo,
    matching: ["config.json", "*.safetensors"],
    progressHandler: { progress in
        print("Download progress: \(progress.fractionCompleted * 100)%")
    }
)
print("Files downloaded to: \(modelDirectory.path)")

  4. Enable the Xet trait for fast downloads

    main

    On Swift 6.1+, you can enable the Xet package trait to use swift-xet for fast, parallel downloads from the Hugging Face Hub. This is opt-in because it introduces additional transitive dependencies like AsyncHTTPClient.

    Note: Xcode does not yet support declaring package traits directly. To use Xet in an Xcode project, create an internal Swift package that re-exports swift-transformers with the Xet trait enabled, then add that local package to your project.

    SwiftPM Configuration:

    dependencies: [
    .package(
        url: "https://github.com/huggingface/swift-transformers",
        from: "1.3.0",
        traits: ["Xet"]
    )
]

    CLI Usage:

    swift build --traits Xet
swift test --traits Xet

  5. Tokenize text and apply chat templates

    main

    Use the AutoTokenizer class to load a tokenizer from a Hugging Face model ID. You can then use applyChatTemplate to format a list of messages (including roles like user) into tokens, and decode to convert tokens back into text. This API is designed to be idiomatic for Swift developers while remaining familiar to those used to the Python transformers library.

    let tokenizer = try await AutoTokenizer.from(pretrained: "deepseek-ai/DeepSeek-R1-Distill-Qwen-7B")
let messages = [["role": "user", "content": "Describe the Swift programming language."]]
let encoded = try tokenizer.applyChatTemplate(messages: messages)
let decoded = tokenizer.decode(tokens: encoded)

  6. Export Mistral 7B Instruct v0.3 for CoreML

    main

    To prepare the Mistral 7B Instruct v0.3 model for use with swift-transformers, you must first export it using the provided Python export script. This process converts the PyTorch weights into MIL (Model Intermediate Language) operations and ultimately into an .mlpackage format suitable for CoreML. Use uv to run the export script.

    uv run export.py

  7. Format inputs for tool calling

    main

    swift-transformers supports tool calling by allowing you to pass a tools array to applyChatTemplate. Each tool is defined as a dictionary containing the function's type, name, description, and parameters.

    let tokenizer = try await AutoTokenizer.from(pretrained: "mlx-community/Qwen2.5-7B-Instruct-4bit")

let weatherTool = [
    "type": "function",
    "function": [
        "name": "get_current_weather",
        "description": "Get the current weather in a given location",
        "parameters": [
            "type": "object",
            "properties": ["location": ["type": "string", "description": "City and state"]],
            "required": ["location"]
        ]
    ]
]

let tokens = try tokenizer.applyChatTemplate(
    messages: [["role": "user", "content": "What's the weather in Paris?"]],
    tools: [weatherTool]
)

  8. Generate text using transformers-cli

    main

    Once the model is exported as an .mlpackage, you can use the transformers-cli tool to perform text generation. The CLI accepts a prompt as a positional argument and requires the path to the exported model package. You can also specify the --max-length flag to control the generation length.

    Usage Pattern: swift run transformers-cli "<prompt>" --max-length <length> <model_path>

    swift run transformers-cli "Best recommendations for a place to visit in Paris in August 2024:" --max-length 128 StatefulMistral7BInstructInt4.mlpackage