Browser AI

The @browser-ai/web-llm package provides a Vercel AI SDK v7 compatible provider that allows you to run WebLLM compiled models directly in the browser. This enables client-side text generation without relying on external server-side APIs for model inference.

Key Capabilities:

• Language Models: Support for text generation using models like Llama, Qwen, and Phi.
• Tool Calling: Support for function calling using JSON format.
• Structured Output: Ability to generate validated JSON objects.
• Agents: Support for multi-step autonomous workflows.
• Web Worker Support: Executes models off the main thread to ensure a smooth user experience.

The @browser-ai/web-llm package allows you to run models compiled via WebLLM directly in the browser. It provides a seamless integration with the Vercel AI SDK v6, enabling client-side inference without a dedicated backend server for the model execution.

Key Capabilities:

• Language Models: Supports text generation using models like Llama, Qwen, and Phi.
• Tool Calling: Supports function calling with JSON format support.
• Structured Output: Capability to generate validated JSON objects.
• Agents: Supports multi-step autonomous workflows.
• Web Worker Support: Enables off-main-thread execution to ensure the UI remains responsive during inference.

The @browser-ai/transformers-js package is a provider for the Vercel AI SDK v6 that enables running inference with Hugging Face Transformers models directly in the browser. It abstracts away the complexity of Web Worker management and manual streaming, providing a seamless integration with the Vercel AI SDK ecosystem.

Supported capabilities include:

• Language Models: Text generation (e.g., SmolLM, Qwen, Llama).
• Text Embeddings: Embedding models (e.g., GTE, MiniLM).
• Vision Models: Multimodal models (e.g., SmolVLM).
• Transcription: Speech-to-text using Whisper models.
• Tool Calling: Support for function calling.
• Agents: Support for multi-step autonomous workflows.

The @browser-ai/core package provides a Vercel AI SDK v7 compatible provider that leverages native AI models built into Chrome and Edge browsers. This allows you to run inference locally on the user's device.

Key Capabilities

• Local Inference: Models run directly in the browser, providing zero-latency and offline capabilities once models are downloaded.
• Privacy: Data remains on the user's device, as it never leaves the browser.
• Text Generation: Supports Gemini Nano (Chrome) and Phi Mini (Edge).
• Embeddings: Supports generating text embeddings using browser-native models.
• Multimodal & Tools: Supports image and audio input (Chrome only), tool calling (function calling with JSON support), and structured output (validated JSON objects).
• Agents: Enables building multi-step autonomous workflows using the Vercel AI SDK agent patterns.

The @browser-ai/core package provides a model provider for the Vercel AI SDK v6 that leverages native AI models built into Chrome and Edge browsers. This allows you to run inference locally on the user's device.

Key Capabilities

• Local Inference: Uses Gemini Nano (Chrome) or Phi Mini (Edge) for text generation.
• Privacy & Offline Support: Models run locally, meaning data stays on the device and the application can work offline after the initial model download.
• Advanced AI Features:
  • Text Embeddings: Native browser-based embedding generation.
  • Multimodal Support: Supports image and audio input (Chrome only).
  • Tool Calling: Supports function calling with JSON format.
  • Structured Output: Capability to generate validated JSON objects.
  • Agents: Support for building multi-step autonomous workflows.

The Next.js Hybrid Chat example demonstrates several advanced patterns for building browser-integrated AI applications:

• Hybrid AI Architecture: Automatically falls back from browser-based AI (like Chrome/Edge Prompt API) to server-side AI models if the browser capability is unavailable.
• Download Progress Streaming: Provides real-time progress bars to the user during the downloading of browser AI models.
• Multimodal Support: Handles image and audio file uploads and processing within the chat interface.
• Type-Safe Streaming: Uses TypeScript to support custom data parts during streaming.
• Error Handling: Implements graceful error states and retry mechanisms for model availability issues.

The @browser-ai/core package provides access to the experimental Prompt API built into Chrome and Edge. The browser handles model orchestration automatically:

• Edge: Uses Phi4-mini.
• Chrome: Uses Gemini Nano.

Capabilities:

• Text generation (Language Models)
• Text Embeddings
• Multimodal Support (Image and audio input)
• Structured Data generation
• Tool Calling (JSON format support)

The @browser-ai/web-llm package uses WebLLM with WebGPU to provide high-performance in-browser LLM inference for popular open-source models like Llama3 and Qwen3.

Capabilities:

• Open-source Language Models: Llama, Qwen, Phi, and more
• Download Progress: Real-time tracking of model downloads
• Tool Calling: Function calling support
• Web Worker Support: Efficient background processing

The @browser-ai ecosystem bridges several distinct in-browser LLM implementations into a single developer experience:

• Chrome/Edge Prompt API: Uses native browser integration via the JavaScript LanguageModel namespace (utilizing models like Gemini Nano and Phi4-mini).
• WebLLM: Provides an OpenAI-style API leveraging MLCEngine, WebGPU, and WebAssembly for model execution.
• Transformers.js: Uses the pipeline API to support various NLP, Computer Vision, Audio, and Multimodal tasks via ONNX Runtime.

By using the @browser-ai providers, you can abstract away the API fragmentation between these technologies.

Using @huggingface/transformers directly in the browser requires significant boilerplate to manage Web Workers, progress tracking, streaming tokens, interrupt handling, and complex state management between the worker and the UI.

@browser-ai/transformers-js abstracts this complexity by providing:

• A pre-built TransformersJSWorkerHandler for the worker side.
• A declarative transformersJS model definition.
• Seamless integration with the Vercel AI SDK via TransformersChatTransport and the useChat hook.
• Automatic handling of loading states, progress updates, and streaming.