DOCSEARCHbyAlgolia
Submit Docs

React Native AI

repository·main·Indexed 23 days ago

https://github.com/callstackincubator/ai

A set of on-device AI primitives for React Native that enable privacy-preserving, low-latency local inference. It provides Vercel AI SDK compatibility and includes providers for Google ADK (supporting Gemini Nano and Cloud Gemini) and Apple Foundation Models, as well as development tools for inspecting AI SDK requests via Rozenite.

Tokens
53.1K
Snippets
151
Records
295
Agent score
79%

What's inside @react-native-ai/monorepo

  1. Overview of React Native AI

    main

    React Native AI is a collection of on-device AI primitives for React Native that provides first-class support for the Vercel AI SDK. It allows you to run AI models directly on user devices, ensuring privacy (data stays local), low latency, and zero server costs.

    Key features include:

    • Instant AI: Use built-in system models without manual downloads.
    • Privacy-first: All processing is local to the device.
    • Vercel AI SDK Compatibility: Works as a drop-in replacement for remote AI providers.
    • Complete Toolkit: Supports text generation, embeddings, transcription, and speech synthesis.

  2. Features of the MLC Provider

    main

    The MLC provider offers the following capabilities for React Native developers:

    • On-device text generation: Run LLMs locally without requiring a cloud backend.
    • Multiple model support: Compatible with various model families including Llama, Phi, Qwen, and more.
    • Model management: Built-in support for downloading and preparing model assets.
    • Streaming: Supports streaming responses for a better user experience.
    • Hardware acceleration: Utilizes hardware-accelerated inference for improved performance on-device.

  3. Features of @react-native-ai/llama

    main

    The Llama provider supports the following capabilities:

    • Text generation: Using GGUF models.
    • Streaming: Support for streaming responses.
    • On-device inference: Runs locally on iOS and Android.
    • Model management: Built-in model download management.
    • Hardware acceleration: Support for GPU acceleration.

  4. Features of the Apple Provider

    main

    The Apple Provider supports the following capabilities via the Vercel AI SDK:

    • Text generation: Generate text using Apple Foundation Models.
    • Structured outputs: Receive responses in specific formats.
    • Tool calling: Enable models to interact with external tools.
    • Streaming: Stream responses for a better user experience.

  6. Handle reasoning models (e.g., DeepSeek-R1)

    main

    Reasoning models automatically handle <think> tags. The Llama provider separates reasoning content from the main response.

    • Non-streaming: Access reasoning via result.reasoning.
    • Streaming: Reasoning tokens are emitted via reasoning-start, reasoning-delta, and reasoning-end events.
    import { llama, downloadModel } from '@react-native-ai/llama'
import { generateText } from 'ai'

const modelPath = await downloadModel('owner/repo/deepseek-r1.gguf')

const model = llama.languageModel(modelPath)
await model.prepare()

const result = await generateText({
  model,
  prompt: 'Solve this math problem step by step: 2x + 5 = 13',
})

// Access main response
console.log(result.text)

// Access reasoning content (if present)
console.log(result.reasoning)

  7. How to check and prepare Gemini Nano on-device models

    main

    When using genai-nano models, you must manage the device's capability and runtime readiness. Gemini Nano has two distinct checks:

    1. adk.isNanoSupported(): Returns whether the device hardware/software is capable of running Nano.
    2. adk.isAvailable('genai-nano'): Returns whether the model is ready to use (e.g., it might be downloading in the background).

    Recommended Implementation Flow

    import { createAdkProvider } from '@react-native-ai/adk'

const adk = createAdkProvider({
  modelType: 'genai-nano',
  modelName: 'gemini-nano',
})

// 1. Check hardware capability
const supported = await adk.isNanoSupported()
if (!supported) {
  // Device lacks Gemini Nano / AICore support
  return
}

// 2. Check runtime readiness (e.g. is it downloading?)
const ready = await adk.isAvailable('genai-nano')
if (!ready) {
  // Device supports Nano but ML Kit is not ready yet
  return
}

// 3. Prepare and use
await adk.prepareNano()
const model = adk()

    Note: While model.prepare(), generateText(), and streamText() will automatically call prepareNano() for genai-nano models, they only gate on isNanoSupported(). For a good user experience, always use isAvailable('genai-nano') to handle the 'downloading' state in your UI.

    ML Kit Status Mapping

    StatusisNanoSupported()isAvailable('genai-nano')Suggested UX
    0falsefalseHide or disable — not supported
    1truetrueReady — call prepareNano()
    3truetrueReady to download — call prepareNano()
    Other non-zerotruefalseShow disabled — not ready yet
    import { createAdkProvider } from '@react-native-ai/adk'

const adk = createAdkProvider({
  modelType: 'genai-nano',
  modelName: 'gemini-nano',
})

const supported = await adk.isNanoSupported()
if (!supported) {
  return
}

const ready = await adk.isAvailable('genai-nano')
if (!ready) {
  return
}

await adk.prepareNano()
const model = adk()

  8. Check Gemini Nano availability and readiness

    main

    Gemini Nano requires two distinct checks to ensure a smooth user experience:

    1. adk.isNanoSupported(): Checks device capability. Determines if the device can ever run Nano (based on AICore support). If false, the feature is unavailable.
    2. adk.isAvailable('genai-nano'): Checks runtime readiness. Determines if the model is currently ready to be used (e.g., it might be downloading in the background). If false, you should show a 'disabled' state or wait.

    Both methods are cheap native calls and are safe to cache at app startup.

  9. How @react-native-ai/json-ui works

    main

    The @react-native-ai/json-ui package is a lightweight tooling suite designed for small language models (e.g., ~3B parameters) running locally or on-device.

    The Workflow:

    1. The model calls tools (like add node or set props) to build or update a UI specification.
    2. The createGenUITools utility handles the mutation of this specification.
    3. The GenerativeUIView component receives the resulting specification and renders it using GenUINode components, applying styles validated by GEN_UI_STYLES.

  10. How tool calling works with ADK

    main

    ADK enables agents to call JavaScript tools. Unlike standard AI SDK usage, ADK orchestrates the agent loop natively on Android.

    Key Behaviors:

    • Pre-registration required: You must pass tool executors to createAdkProvider via availableTools so the native side can invoke them.
    • Native Loop: ADK runs the multi-turn tool loop internally; the AI SDK maxSteps option does not control ADK's internal iterations.
    • Stream markers: Streamed tool calls are marked with providerExecuted: true and emit tool-input-start, tool-input-delta, tool-input-end, and tool-call parts.
    import { createAdkProvider } from '@react-native-ai/adk'
import { generateText, tool } from 'ai'
import { z } from 'zod'

const getCurrentTime = tool({
  description: 'Get the current time for a city',
  inputSchema: z.object({
    city: z.string(),
  }),
  execute: async ({ city }) => ({
    city,
    time: new Date().toLocaleTimeString(),
  }),
})

const adk = createAdkProvider({
  availableTools: { getCurrentTime },
})

await adk.prepareNano()

const { text } = await generateText({
  model: adk(),
  tools: { getCurrentTime },
  prompt: 'What time is it in Warsaw?',
})

  11. Select the correct React Native AI provider

    main

    Before integrating, classify your use case to choose the appropriate provider from the @react-native-ai ecosystem:

    • Apple: Use for Apple Intelligence (iOS 26+), Apple Foundation Models, transcription, speech synthesis, or embeddings on Apple devices. Supports tool calling.
    • Llama: Use for GGUF models, llama.rn, HuggingFace models (like SmolLM), embedding models, reranking, or speech models.
    • MLC: Use for MLC-LLM models that require custom models and build-time model optimizations.
    • NCNN: Use for low-level inference on bare metal tensors or custom models (e.g., convolutional networks, multi-layer perceptrons). Do not use NCNN if you only need LLMs; use Llama or MLC instead.

  12. How tool calling works with the Apple Provider

    main

    Tool calling with Apple Foundation Models has specific behaviors that differ from other providers because tools are executed by Apple, not the Vercel AI SDK:

    1. No AI SDK callbacks: Lifecycle callbacks like maxSteps, onStepStart, and onStepFinish will not be executed.
    2. Pre-registration required: You must pass all tools to createAppleProvider upfront. You cannot simply pass them to generateText if they haven't been registered with the provider.
    3. Empty toolCallId: Apple does not provide tool call IDs, so these will be returned as empty strings.

    Registering Tools

    Use createAppleProvider to register tools. You can update tools at runtime using model.updateTools().

    import { createAppleProvider } from '@react-native-ai/apple';
import { generateText, tool } from 'ai';
import { z } from 'zod';

const getWeather = tool({
  description: 'Get current weather information',
  inputSchema: z.object({
    city: z.string()
  }),
  execute: async ({ city }) => {
    return `Weather in ${city}: Sunny, 25°C`;
  }
});

// Initial registration
const apple = createAppleProvider({
  availableTools: {
    getWeather
  }
});

// Updating tools at runtime
const model = apple();
model.updateTools({
  getWeather,
  getDate
});