DOCSEARCHbyAlgolia
Submit Docs

Cactus React Native

repository·main·Indexed 41 days ago

https://github.com/cactus-compute/cactus-react-native

A library for running AI models locally on mobile devices using React Native. Version 1.13.1 supports text completion, vision, tool calling, and Retrieval Augmented Generation (RAG) via the CactusLM class and useCactusLM hook. It includes specialized tools for speech-to-text (CactusSTT), audio analysis (CactusAudio), and vector storage (CactusIndex) for managing embeddings and similarity searches.

Tokens
23.6K
Snippets
111
Records
117
Agent score
51%

What's inside Cactus

  1. Implement RAG (Retrieval Augmented Generation)

    main

    RAG allows the model to reference a local corpus of documents.

    1. Setup: Initialize CactusLM with a corpusDir pointing to a directory containing .txt files.
    2. Generation: Use complete({ messages }) to ask questions based on the corpus.
    3. Direct Query: Use ragQuery({ query, topK }) to search the RAG index directly. It returns an array of chunks, where each chunk contains a score, source filename, and the content.

    ragQuery parameters:

    • query: The search terms.
    • topK: The number of chunks to retrieve.
    import { CactusLM, type CactusLMMessage } from 'cactus-react-native';

const cactusLM = new CactusLM({
  corpusDir: 'path/to/your/corpus', // Directory containing .txt files
});

const messages: CactusLMMessage[] = [
  { role: 'user', content: 'What information is in the documents?' },
];

const result = await cactusLM.complete({ messages });
console.log(result.response);

// Or query the RAG index directly
const ragResult = await cactusLM.ragQuery({ query: 'search terms', topK: 5 });
console.log('Chunks:', ragResult.chunks);
// [{ score: 0.85, source: 'doc.txt', content: '...' }, ...]

  3. Build and run the iOS app

    main

    To run the iOS app, you must first ensure CocoaPods dependencies are installed.

    1. If it is your first time or you have updated native dependencies, run bundle install followed by bundle exec pod install.
    2. Once dependencies are installed, run the build command.
    # Install CocoaPods dependencies
bundle install
bundle exec pod install

# Build and run
# Using npm
npm run ios

# OR using Yarn
yarn ios

  4. Quick Start with CactusLM class

    main

    You can interact with the language model directly by instantiating the CactusLM class. The workflow involves creating an instance, downloading the model (with optional progress tracking), generating completions using a list of CactusLMMessage objects, and finally destroying the instance to clean up resources.

    import { CactusLM, type CactusLMMessage } from 'cactus-react-native';

// Create a new instance
const cactusLM = new CactusLM();

// Download the model
await cactusLM.download({
  onProgress: (progress) => console.log(`Download: ${Math.round(progress * 100)}%`)
});

// Generate a completion
const messages: CactusLMMessage[] = [
  { role: 'user', content: 'What is the capital of France?' }
];

const result = await cactusLM.complete({ messages });
console.log(result.response); // "The capital of France is Paris."

// Clean up resources
await cactusLM.destroy();

  5. Reload the application

    main

    If you need to perform a full reload to reset the app state, use the following platform-specific shortcuts:

    • Android: Press the <kbd>R</kbd> key twice or open the Dev Menu via <kbd>Ctrl</kbd> + <kbd>M</kbd> (Windows/Linux) or <kbd>Cmd ⌘</kbd> + <kbd>M</kbd> (macOS) and select "Reload".
    • iOS: Press <kbd>R</kbd> in the iOS Simulator.

  6. Optimize performance for mobile devices

    main

    Follow these best practices to ensure smooth performance on React Native:

    • Model Selection: Choose smaller models to ensure faster inference on mobile hardware.
    • Memory Management: Always call .destroy() on model instances when they are no longer needed to free up system resources.
    • VAD (Voice Activity Detection): When transcribing audio with silence, set useVad: true (default) to strip non-speech regions, which speeds up transcription.

  9. Configure CactusLM model options

    main

    When initializing CactusLM, you can specify model quantization and NPU acceleration for Pro models.

    • quantization: Choose between 'int8' (default, better accuracy) or 'int4'.
    • pro: Set to true to use Pro models for NPU acceleration.
    import { CactusLM } from 'cactus-react-native';

// Use int8 for better accuracy (default)
const cactusLM = new CactusLM({
  model: 'lfm2-vl-450m',
  options: {
    quantization: 'int8', // 'int4' or 'int8'
    pro: false
  }
});

// Use pro models for NPU acceleration
const cactusPro = new CactusLM({
  model: 'lfm2-vl-450m',
  options: {
    quantization: 'int8',
    pro: true
  }
});

  10. Initialize the Cactus engine

    main

    Before using any Cactus features, you must initialize the engine with a model path. You can optionally provide a corpus directory for RAG (Retrieval Augmented Generation) and enable cache indexing to improve performance.

    const cactus = new Cactus();
await cactus.init(
  '/path/to/your/model',
  '/path/to/your/corpus',
  true // cacheIndex
);

  11. Initialize and use CactusAudio for audio processing

    main

    CactusAudio is the primary class for performing audio tasks such as Voice Activity Detection (VAD), speaker diarization, and speaker embedding.

    Lifecycle

    1. Instantiate: Create an instance with an optional model and options (quantization and pro status).
    2. Download: If using a remote model, call .download() to fetch the model files to the device.
    3. Initialize: Call .init() to prepare the underlying native engine with the model path. Note that most processing methods (like .vad()) call .init() automatically if it hasn't been called yet.
    4. Process: Execute audio tasks.
    5. Destroy: Call .destroy() to clean up native resources.

    Configuration Options

    • model: The name of the model to use (defaults to 'silero-vad'). If the string starts with file:// or /, it is treated as a local file path.
    • options.quantization: Supports 'int4' or 'int8' (defaults to 'int8').
    • options.pro: A boolean flag for pro-tier models (defaults to false).
    import { CactusAudio } from 'cactus-react-native';

const audio = new CactusAudio({
  model: 'silero-vad',
  options: {
    quantization: 'int8',
    pro: false
  }
});

// 1. Download the model
await audio.download({
  onProgress: (progress) => console.log(`Download: ${progress * 100}%`)
});

// 2. Run Voice Activity Detection (VAD)
// Note: .init() is called automatically inside .vad()
const vadResult = await audio.vad({
  audio: myAudioBuffer,
  options: { /* VAD specific options */ }
});

// 3. Cleanup
await audio.destroy();

  12. Use streaming transcription with streamTranscribeStart()

    main

    For real-time audio processing, use the streaming transcription lifecycle:

    1. streamTranscribeStart(options) to initialize the stream.
    2. streamTranscribeProcess(audioChunks) repeatedly to feed audio data.
    3. streamTranscribeStop() to finalize the transcription.
    // 1. Start
await cactus.streamTranscribeStart({ language: 'en' });

// 2. Process chunks
const result = await cactus.streamTranscribeProcess(audioBuffer);
if (result.confirmed) {
  console.log(result.confirmed);
}

// 3. Stop
await cactus.streamTranscribeStop();