repository·main·Indexed 21 days ago
https://github.com/software-mansion/react-native-audio-apiA high-performance audio engine for React Native that implements the Web Audio API specification. It enables web-standard audio manipulation techniques in mobile environments and includes a dual-graph architecture to separate high-level graph manipulation from low-level audio processing.
react-native-audio-api is a high-performance audio engine for React Native that implements the Web Audio API specification. It allows developers to generate, manipulate, and control audio in a React Native environment using the same patterns and logic used in web browsers.React Native Audio API is a high-performance, imperative API designed for processing and synthesizing audio within React Native applications. It is built to bridge the gap in the React Native ecosystem for high-performance audio tasks like creating audio, applying effects, or controlling individual audio parameters.
Key features include:
react-native-web).AVFoundation and CoreAudio on iOS, and Oboe on Android.WorkletAudioContext is a specialized audio context that runs an audio-processing graph on a dedicated background thread. Unlike a standard AudioContext, it does not route audio to the device speakers; instead, it uses an internal sink where output is discarded.
WorkletNode while actual playback happens in a separate AudioContext.| Context | Output | Render Driver | Typical Use |
|---|---|---|---|
AudioContext | Device speakers / headphones | System audio callback | Playback, recording, effects |
WorkletAudioContext | Internal sink (discarded) | Software timer on a worker thread | Worklet visualization, isolated processing |
OfflineAudioContext | AudioBuffer | Renders as fast as possible | Offline bounce / export |
Note: WorkletAudioContext advances in real time at the context sample rate, unlike OfflineAudioContext.
An AudioParam is an interface used to control audio node properties (like volume in a GainNode, pan, or frequency) over time. This allows for smooth transitions rather than abrupt changes.
Key methods for automation include:
setValueAtTime(value, time): Sets the parameter to a specific value at a specific time.exponentialRampToValueAtTime(value, time): Smoothly ramps the parameter exponentially toward a target value.An envelope describes how a sound's amplitude (volume) changes over time. The standard model is ADSR:
A WorkletAudioContext is a dedicated context for worklet graphs that do not require speaker output. It renders on a background thread and discards audio at the destination.
This allows you to keep visualization or analysis graphs (like RMS meters) separate from your main AudioContext playback session, preventing analysis logic from interfering with actual audio playback.
Use these nodes when you need to perform actual audio processing on the audio path. They invoke callbacks synchronously on a dedicated audio worklet runtime every render quantum (128 frames).
Node Types:
WorkletSourceNode: Use for building custom synthesizers or procedural generators.WorkletProcessingNode: Use for implementing in-graph JavaScript effects, filters, or dynamics processors.Critical Performance Warning: These callbacks run on the audio thread. You must keep them short and allocation-free. At 44.1 kHz, you have approximately 2.9 ms to complete the entire callback. If the worklet exceeds this time budget, you will experience audio dropouts or glitches.
The AudioListener interface represents the position and orientation of the listener in a 3D audio scene. It is used by PannerNode objects to spatialize sound relative to the listener's location and direction.
Key Concepts:
AudioListener does not have inputs, outputs, or connect()/disconnect() methods. It only exposes nine AudioParam properties.AudioListener yourself. Instead, every BaseAudioContext exposes a single, read-only listener property.positionX, positionY, positionZ: The 3D Cartesian coordinates of the listener.forwardX, forwardY, forwardZ: A direction vector representing where the listener is facing (the "nose").upX, upY, upZ: A direction vector representing the top of the listener's head.forward and up vectors must be linearly independent.Note on Implementation: As of the current version, PannerNode is not yet implemented, so changes to the AudioListener will not have an audible effect. The interface is provided for future compatibility.
import { AudioContext } from 'react-native-audio-api';
const ctx = new AudioContext();
const listener = ctx.listener;
// Move the listener 3 meters along the x axis.
listener.positionX.value = 3;
// Or automate the movement over time.
listener.positionX.linearRampToValueAtTime(10, ctx.currentTime + 2);Extension packages can integrate with react-native-audio-api in C++ using a single public header. To ensure stability and avoid internal dependency issues, you must only include the StableAPI.h header. All necessary types and helpers for extensions are transitively included via this header. Including any other audioapi/... headers is explicitly unsupported and may lead to build failures or instability.
The only permitted include is:
#include <audioapi/compatibility/StableAPI.h>#include <audioapi/compatibility/StableAPI.h>The audio engine uses a dual-graph architecture to separate high-level graph manipulation from low-level audio processing. This ensures that the UI/Main thread can manage the graph structure without blocking the high-priority audio thread.
Graph.hpp): The primary entry point and orchestrator. It provides the high-level public API used to create and manipulate the graph. It manages internal structures including AudioGraph, HostGraph, SPSC, Disposer, and pool capacity.HostGraph.hpp): Manages the graph structure on the main/control thread. It tracks nodes, edges, connections, and cycles. It provides APIs for adding, removing, and connecting nodes.HostGraph returns an AGEvent (a closure). This closure is sent asynchronously to the audio thread to ensure the AudioGraph eventually reaches the same state as the HostGraph.HostGraph checks for potential cycles before adding edges. It uses NodeHandle to verify if nodes are still active in the AudioGraph to prevent adding edges to orphaned nodes.AudioGraph.hpp): Responsible for the actual audio processing on the audio thread. It maintains a topological order of nodes and edges and applies AGEvent closures received from the HostGraph to update its state.InputPool.hpp): An efficiency-focused component used by AudioGraph to manage edges via a free list. It minimizes memory fragmentation and is designed for high-speed allocation/deallocation on the audio thread. Note: It is not thread-safe and must be used within AudioGraph or its events.NodeHandle.hpp): A lightweight 32-bit index used to reference nodes within the AudioGraph. These handles should only be used within AGEvent closures to safely reference nodes on the audio thread.Disposer.hpp): Manages the destruction of nodes and edges. It uses a worker thread and an SPSC (Single Producer Single Consumer) queue to offload heavy destruction tasks, ensuring the audio thread remains performant.The ChannelCountMode type determines how the number of input channels affects the number of output channels in an audio node. You can choose between three modes to control channel up-mixing and clamping:
max: The number of channels is equal to the maximum number of channels of all connections. In this mode, channelCount is ignored and only up-mixing occurs.clamped-max: The number of channels is equal to the maximum number of channels of all connections, but it is clamped to the value specified by channelCount (which acts as the maximum permissible value).explicit: The number of channels is strictly defined by the value of channelCount.The ConvolverNode interface implements a linear convolution effect. It is primarily used to apply echo or reverb effects to an audio signal by using an impulse response provided via an AudioBuffer.
Note on Tail-Time: A ConvolverNode has 'tail-time', meaning it will continue to output non-silent audio even after the input signal has become silent, for the duration of the impulse response buffer.
Performance Warning: Linear convolution is computationally expensive. If you encounter audio artifacts, consider decreasing the duration of your impulse response buffer.
// Example conceptual usage
const convolver = context.createConvolver();
convolver.buffer = myImpulseResponseBuffer;
// Connect to audio graph
source.connect(convolver);
convolver.connect(context.destination);