DOCSEARCHbyAlgolia
Submit Docs

Elementary Audio Documentation

repository·main·Indexed 19 days ago

https://github.com/elemaudio/elementary

A declarative JavaScript library for digital audio signal processing that decouples audio logic from the underlying engine. It provides a portable framework for building audio applications across web, native, and embedded platforms using a standard library of audio primitives (@elemaudio/core) and environment-specific renderers including @elemaudio/web-renderer for browsers and @elemaudio/offline-renderer for Node.js/WASM file processing.

Tokens
13.7K
Snippets
55
Records
71
Agent score
68%

What's inside Elementary

  1. Overview of Elementary Audio

    main

    Elementary is a JavaScript library for digital audio signal processing designed around three core principles:

    • Declarative: You describe your audio process as a function of your application state. Elementary efficiently updates the underlying audio engine as state changes.
    • Dynamic: The framework is built to handle changing audio requirements throughout a user journey, rather than just static processes.
    • Portable: The JavaScript API is decoupled from the underlying audio engine. This allows the same JavaScript layer to run in various environments, including browsers, audio plugins, or embedded devices.

    To build with Elementary, you will primarily use @elemaudio/core to define processes, combined with a renderer specific to your target environment.

  2. Understand Elementary release and installation channels

    main

    Elementary follows semver conventions and provides two main installation tracks depending on your stability requirements:

    Stable (Recommended)

    For most projects, use the main branch of the repository and the latest tagged packages on npm.

    Bleeding Edge (Canary)

    If you need the newest features and can tolerate potential instability, use the develop branch (via git tags) and the next tagged packages on npm.

    Release TypeGit Branchnpm TagStatus
    CanarydevelopnextExperimental/Validation
    StablemainlatestProduction Ready

  3. Bundle JavaScript examples for the CLI

    main

    The CLI runs JavaScript within a QuickJS environment. Because QuickJS cannot resolve standard module imports at runtime in this setup, you must bundle your JavaScript files into a single file using esbuild before running them.

    Navigate to the cli/examples/ directory to install dependencies and run the build script, which outputs bundled files to examples/dist/.

    cd cli/examples/
npm install
npm run build

  4. Build the Elementary CLI (Native)

    main

    To build the native CLI tool, use CMake from the top-level repository directory. This process requires cloning the repository with submodules included to ensure all dependencies are present.

    1. Clone the repository with submodules: git clone https://github.com/elemaudio/elementary.git --recurse-submodules
    2. Create and enter a build/ directory.
    3. Initialize CMake (example uses Xcode generator for macOS).
    4. Build the binaries.
    # Get Elementary
git clone https://github.com/elemaudio/elementary.git --recurse-submodules
cd elementary

# Initialize the CMake directory
mkdir build/
cd build/

# Choose your favorite CMake generator here
cmake -G Xcode -DCMAKE_OSX_DEPLOYMENT_TARGET=10.15 ../

# Build the binaries
cmake --build .

  5. Use the OfflineRenderer for file processing

    main

    The @elemaudio/offline-renderer package provides a Renderer implementation designed for offline tasks (like file processing) where no real-time audio driver is present. It works by taking an Elementary processing graph and rendering it into provided buffers.

    To use it, you must:

    1. Instantiate OfflineRenderer.
    2. Call .initialize() with your desired audio configuration (channels and sample rate).
    3. Call .render(graph) to prepare the processing graph using @elemaudio/core primitives.
    4. Call .process(inps, outs) to execute the rendering. The results will be written directly into the Float32Array buffers provided in the outs array.
    import { el } from '@elemaudio/core';
import OfflineRenderer from '@elemaudio/offline-renderer';

(async function main() {
  let core = new OfflineRenderer();

  await core.initialize({
    numInputChannels: 0,
    numOutputChannels: 1,
    sampleRate: 44100,
  });

  // Our sample data for processing: an empty input and a silent 10s of
  // output data to be written into.
  let inps = [];
  let outs = [new Float32Array(44100 * 10)]; // 10 seconds of output data

  // Render our processing graph
  core.render(el.cycle(440));

  // Pushing samples through the graph. After this call, the buffer in `outs` will
  // have the desired output data which can then be saved to file if you like.
  core.process(inps, outs);
})();

  8. Use @elemaudio/core to render audio

    main

    The @elemaudio/core package provides the el namespace for composing audio nodes and a Renderer class to handle the audio graph.

    To use the Renderer, you must provide a callback function that receives instruction batches. It is your responsibility to send these batches to an underlying audio engine or message-passing channel. Once initialized, you can call .render() with audio nodes (created via el) to generate audio instructions.

    import { el, Renderer } from '@elemaudio/core';

// Initialize the Renderer with a callback to handle instruction batches
let core = new Renderer((batch) => {
  // Send the instruction batch to your audio engine
  console.log(batch);
});

// Render audio nodes (e.g., two detuned sine tones)
core.render(el.cycle(440), el.cycle(441));

  9. Choose an integration workflow for Elementary

    main

    Elementary requires a renderer to connect your declarative audio definitions to an actual audio output. Choose a package based on your target environment:

    • Web Applications: Use @elemaudio/web-renderer to integrate with frontend UI libraries (like React) to create interactive audio experiences in the browser.
    • Static File Processing (Node.js): Use @elemaudio/offline-renderer for processing audio files or performing batch tasks in a Node.js environment.
    • Native/Embedded/Plugins: For native environments, you can embed the C++ engine directly. Refer to the Native Integrations guide for embedding the engine in your own C++ code, or use the SRVB plugin template to create audio effects plugins.

  10. Use @elemaudio/web-renderer to run audio in the browser

    main

    The @elemaudio/web-renderer package provides a WebRenderer class that runs Elementary applications using WASM and the Web Audio API.

    To use it:

    1. Create an AudioContext.
    2. Instantiate WebRenderer.
    3. Call core.initialize(ctx, options) to get a Web Audio node. This returns a Promise.
    4. Connect the resulting node to ctx.destination.
    5. Use core.render(graph, ...inputs) to send audio graphs to the renderer.

    Note: Most browsers require a user gesture (like a click) to resume an AudioContext before audio will play. In production, ensure your initialization logic is triggered by a user interaction.

    import { el } from '@elemaudio/core';
import WebRenderer from '@elemaudio/web-renderer';

const ctx = new AudioContext();
const core = new WebRenderer();

(async function main() {
  // Initialize the renderer with audio configuration
  let node = await core.initialize(ctx, {
    numberOfInputs: 0,
    numberOfOutputs: 1,
    outputChannelCount: [2],
  });

  // Connect the renderer node to the AudioContext destination
  node.connect(ctx.destination);

  // Render an audio graph (e.g., a 440Hz sine wave)
  core.render(el.cycle(440), el.cycle(441));
})();

  12. Understand the Delegate abstraction

    main

    A Delegate is responsible for recording instructions during a render pass. It tracks changes such as nodes added, nodes removed, edges added, and properties written.

    When a reconciliation pass occurs, the Delegate accumulates instructions in a batch (e.g., CREATE_NODE, APPEND_CHILD, SET_PROPERTY, ACTIVATE_ROOTS, COMMIT_UPDATES). These can be retrieved as a packed array using getPackedInstructions() to be sent to the audio engine.

    Developers can implement their own Delegate if they need fine-grained control over how instructions are captured or if they are building a custom Renderer.