expo-speech-recognition

The library emits several events during the speech recognition lifecycle. The flow differs slightly depending on whether you are using non-continuous or continuous recognition.

Non-continuous recognition flow

1. Initialization: start(options) is called, triggering start and audiostart events.
2. Recognition Loop:
   • volumechange events are emitted (only if the user has opted in).
   • result events with {isFinal: false} are emitted for partial results (only if interimResults is enabled).
3. Completion:
   • A final result event is emitted with {isFinal: true}.
   • audioend and end events are emitted to signal the process is finished.

Continuous recognition flow

In continuous mode, the engine processes multiple segments:

1. Interim Results: Volatile result events (isFinal: false) are emitted for new segments. You may need to concatenate these with previous final results.
2. Final Results: result events with {isFinal: true} are emitted for completed utterances/segments.
3. Completion: The process concludes with audioend and end events.

You can save the audio captured during speech recognition by enabling recordingOptions.persist in the start() method. When enabled, the module emits { uri: string } in the audiostart and audioend events, providing the local file path.

Important Requirements:

• Available on Android 13+ and iOS.
• Always call supportsRecording() to verify availability before attempting to use this feature.
• Do not attempt to access the file until the audioend event has been emitted.

Default Output Formats:

• Android: Linear PCM (16000 Hz, mono).
• iOS: 32-bit Float PCM (44100/48000 Hz, mono). You can customize this using recordingOptions.outputSampleRate and recordingOptions.outputEncoding.

import { useState } from "react";
import {
  ExpoSpeechRecognitionModule,
  useSpeechRecognitionEvent,
} from "expo-speech-recognition";

function RecordAudio() {
  const [recording, setRecording] = useState(false);
  const [recordingUri, setRecordingUri] = useState<string | null>(null);

  const handleStart = () => {
    setRecording(true);
    ExpoSpeechRecognitionModule.start({
      lang: "en-US",
      recordingOptions: {
        persist: true,
        outputDirectory: "/path/to/directory",
        outputFileName: "recording.wav",
        outputSampleRate: 16000, // iOS only
        outputEncoding: "pcmFormatInt16", // iOS only
      },
    });
  };

  useSpeechRecognitionEvent("audiostart", (event) => {
    console.log("Recording started for file:", event.uri);
  });

  useSpeechRecognitionEvent("audioend", (event) => {
    console.log("Local file path:", event.uri);
    setRecordingUri(event.uri);
  });

  // ... render logic
}

To run the example application, you must install dependencies for both the example folder and the root repository, prepare the library, and then build the native platforms. Follow these steps in order:

1. Install dependencies: Run npm install in the example directory.
2. Install root dependencies: Navigate to the root folder and run npm install.
3. Prepare the library: Run npm run prepare from the root to build the expo-speech-recognition library.
4. Build and run: Navigate back to the example folder to build for iOS (npm run ios), Android (npm run android), or start the Metro bundler (npm start).

# Install dependencies
npm install

# Also install dependencies of the root folder
cd ../
npm install

# Build the expo-speech-recognition library
npm run prepare

# Go back to the example folder and build the app
cd example

# Build the iOS app
npm run ios
# Build the Android app
npm run android

# Run the Metro JS bundling server
npm start

Instead of using the microphone, you can transcribe pre-recorded audio files by providing a URI in the audioSource.uri option within start().

Key Considerations:

• Availability: Requires Android 13+ or iOS. If unsupported, an error event with code audio-capture is emitted.
• On-Device Recognition: For long-form audio, use requiresOnDeviceRecognition: true to avoid network latency/costs. On Android, check getSupportedLocales() first to ensure the speech model is installed.
• Android Configuration: You must specify audioChannels, audioEncoding (using AudioEncodingAndroid), and sampleRate for the source file.
• Chunk Delay: Use chunkDelayMillis to manage the flow of audio chunks to the service. Default is 50ms for network-based and 15ms for on-device recognition.

import { Platform } from "react-native";
import {
  ExpoSpeechRecognitionModule,
  useSpeechRecognitionEvent,
  AudioEncodingAndroid,
} from "expo-speech-recognition";

function TranscribeAudioFile() {
  const handleTranscribe = () => {
    ExpoSpeechRecognitionModule.start({
      lang: "en-US",
      interimResults: true,
      requiresOnDeviceRecognition: Platform.OS === "ios",
      audioSource: {
        uri: "file:///path/to/audio.wav",
        audioChannels: 1,
        audioEncoding: AudioEncodingAndroid.ENCODING_PCM_16BIT,
        sampleRate: 16000,
        chunkDelayMillis: undefined,
      },
    });
  };

  useSpeechRecognitionEvent("result", (ev) => {
    // Note: multiple final results may be returned; concatenate as needed
    console.log(ev.results[0]?.transcript);
  });
}

Language detection is available on Android 14+ using the com.google.android.as service package. To use it, you must enable EXTRA_ENABLE_LANGUAGE_DETECTION in androidIntentOptions and select the correct service package.

Requirements:

• Android 14+ only.
• Use androidRecognitionServicePackage: "com.google.android.as" (or set requiresOnDeviceRecognition: true).
• Enable EXTRA_ENABLE_LANGUAGE_DETECTION in androidIntentOptions.
• (Optional) Enable EXTRA_ENABLE_LANGUAGE_SWITCH to allow language switching, but ensure the required language model is downloaded via androidTriggerOfflineModelDownload().

import { useSpeechRecognitionEvent, ExpoSpeechRecognitionModule } from "expo-speech-recognition";

useSpeechRecognitionEvent("languagedetection", (event) => {
  console.log("Language detected:", event.detectedLanguage); // e.g. "en-us"
  console.log("Confidence:", event.confidence); // A value between 0.0 and 1.0
  console.log("Top locale alternatives:", event.topLocaleAlternatives); // e.g. ["en-au", "en-gb"]
});

// Start recognition
ExpoSpeechRecognitionModule.start({
  androidIntentOptions: {
    EXTRA_ENABLE_LANGUAGE_DETECTION: true,
    EXTRA_ENABLE_LANGUAGE_SWITCH: true,
  },
  androidRecognitionServicePackage: "com.google.android.as", // or set "requiresOnDeviceRecognition" to true
});