Web AI Demos

The Built-in AI Extension provides polyfills for emerging built-in AI APIs such as LanguageModel, Summarizer, and Writer.

Key Technical Features:

• Early Injection: Polyfills are injected at document_start within the MAIN world, ensuring they are available before other scripts run.
• Force Injection: You can configure the extension to overwrite native APIs even if they are already present in the browser.
• Configurable Backends: Supports local (Transformers.js) and cloud-based (Gemini, OpenAI, Firebase) processing.

This repository provides a collection of demonstrations for implementing AI directly in the browser. The demos are categorized into two main approaches:

1. Chrome Built-in AI: Utilizing experimental APIs provided directly by the Chrome browser (e.g., Prompt API, Summarization API, SemanticEmbedder API).
2. Generic Client-side AI: Using external libraries and models that run in the browser, such as Transformers.js or Google's Gemma model via MediaPipe.

Developers can use these demos to understand how to implement features like text summarization, semantic search, sentiment analysis, and speech recognition without relying on heavy server-side infrastructure.

This demo implements client-side AI (Web AI) to detect toxic comments in real-time.

Key technical characteristics:

• Engine: Uses Transformers.js for text classification.
• Execution: Runs entirely in the browser (client-side).
• Concurrency: Uses a Web Worker to run the model, ensuring the main thread remains responsive.
• Communication: Implements message passing between the worker and the main thread to provide model readiness status updates to the user.

The evaluation tool consists of the following key files:

• compare_judge.js: The primary execution script. It orchestrates the evaluation by invoking llama-cli to run the local LLM judge.
• reference-dataset.jsonc: Contains the ground truth or reference data used for comparison. In this implementation, the reference evaluations are generated by Gemini Flash (cloud).
• few-shots.jsonc: Contains 3 few-shot examples that are injected into the prompt templates to guide the local LLM judge.

The Invoice Engine is a zero-cloud, multimodal vision parser. It uses Chrome's built-in Gemini Nano via the Prompt API to process images locally.

Workflow: Image $\rightarrow$ Gemini Nano (Multimodal vision) $\rightarrow$ Structured Invoice Fields

This process allows for automatic extraction of sender, receiver, dates, line items, and VAT without any data leaving the user's machine or requiring external API keys.

The window.ai object is deprecated. Chrome Built-in AI APIs are now accessed via separate top-level objects. When writing code, ensure you use the new global identifiers instead of the old window.ai namespace.

Deprecated vs. New Mappings:

• window.ai.languageModel $\rightarrow$ window.LanguageModel
• window.ai.summarizer $\rightarrow$ window.Summarizer
• window.ai.writer $\rightarrow$ window.Writer
• window.ai.Rewriter $\rightarrow$ window.Rewriter
• window.ai.translator $\rightarrow$ window.Translator
• window.ai.languageDetector $\rightarrow$ window.LanguageDetector

Additionally, the capabilities() function is deprecated; use availability() instead.

Before attempting to create a model or skill, check its availability using the availability() method. This prevents errors when models are not yet downloaded or supported.

Possible Availability values:

• "unavailable": The feature/model is not supported.
• "downloadable": The model is supported but needs to be downloaded.
• "downloading": The model is currently being downloaded.
• "available": The model is ready to use.

const status = await LanguageModel.availability();
if (status === 'available') {
  const model = await LanguageModel.create();
} else if (status === 'downloadable') {
  // Handle download logic
}

WebMCP Evals is designed to automate the comparison of actual tool calls against expected tool calls.

Core Components:

• Evaluation Loop: The engine that executes test cases against a model and verifies the output.
• Backends: Implementations for different LLM providers. Currently supports Google GenAI (Gemini) and experimental Ollama support.
• Schemas: Uses JSON schemas to define tools and evaluation test cases.

Project Structure:

• src/bin/runevals.ts: The main entry point for the evaluation runner.
• src/backend/: Contains backend implementations like googleai.ts and ollama.ts.
• src/types/: Contains TypeScript definitions for tools, messages, and evaluations.
• examples/: Contains reference implementations, such as the travel/ directory which includes tools_schema.json and evals.json.

The demo is built using client-side AI (in-browser) with the following architecture:

• Model: Uses the Gemma 2B model.
• Inference Engine: Uses the MediaPipe LLM Inference API via the @mediapipe/tasks-genai npm package.
• Threading: Runs the model in a Web Worker to prevent blocking the main thread.
• Communication: Implements message passing between the worker and the main thread to provide real-time model readiness status updates to the user.

The polyfill selects a backend based on which global configuration object is set on the window object before the polyfill is imported. If no configuration is provided, it defaults to Transformers.js.

Supported Backends