flutter_soloud

repository·main·Indexed 19 days ago

https://github.com/alnitak/flutter_soloud

A high-performance, low-latency audio plugin for Flutter designed for games and immersive applications. It provides advanced features including 3D positional audio, mixing buses, sample-accurate scheduling, and real-time audio visualization via waveforms and FFT data. The plugin supports Android, iOS, macOS, Windows, Linux, and Web, offering capabilities such as pitch shifting, dynamic range compression, and pull-buffer streaming for PCM, MP3, WAV, and Ogg formats.

Tokens
11.8K
Snippets
26
Records
41
Agent score
64%

What's inside flutter_soloud

  1. Overview of SoLoud audio engine

    main
    SoLoud is a free, portable, and easy-to-use C/C++ audio engine designed specifically for games. It is licensed under Zlib/LibPng, making it suitable for various project types. While flutter_soloud provides the Flutter plugin interface, the underlying engine is SoLoud.
  2. Key features of flutter_soloud

    main

    The plugin is a high-performance, low-latency audio engine suitable for games and immersive apps. Key capabilities include:

    • Precise Scheduling: playClocked for sub-millisecond spacing and playScheduled for batch scheduling (ideal for rhythm games/sequencers).
    • 3D Audio: Positional audio with Doppler effect.
    • Advanced Mixing: Mixing buses to group voices (e.g., Music, SFX) with independent volume and filters.
    • Streaming: Support for PCM, MP3, WAV, and Ogg (Opus, Vorbis, FLAC) with pull-buffer streaming for custom/network sources.
    • Real-time Data: Access to audio waveforms and FFT data for visualization.
    • Effects & Generation: Reverb, echo, limiter, EQ, pitch shift, and real-time waveform generation (sine, square, saw, triangle, etc.).
    • Recording: Capture the master mixer output in various formats (PCM, Opus, Vorbis, FLAC, WAV).
  3. Explore flutter_soloud feature examples

    main

    The example directory contains several implementations demonstrating the capabilities of flutter_soloud. Use these as templates for your own implementation:

    Basic Usage

    • Basic Setup: lib/main.dart shows the fundamental initialization and usage.
    • Output Devices: lib/output_device/output_device.dart demonstrates how to list and select available audio output devices.

    Audio Visualization

    • Audio Data: lib/audio_data/audio_data.dart shows how to use AudioData for visualization.
    • Waveform Data: lib/wave_data/wave_data.dart demonstrates reading and displaying audio samples from files.

    Advanced Features

    • Audio Generation: Real-time waveform control (lib/waveform/waveform.dart) and precise metronome creation (lib/metronome/metronome.dart).
    • Streaming: Generating PCM audio in an Isolate (lib/buffer_stream/generate.dart) or streaming PCM/Opus audio via WebSockets (lib/buffer_stream/websocket.dart).

    Audio Effects

    • Compressor: Dynamic range compression (lib/filters/compressor.dart).
    • Limiter: Peak limiting and volume control (lib/filters/limiter.dart).
    • Pitch Shifting: Real-time pitch shifting (lib/filters/pitchshift.dart).
  4. Handle Latency and Automation

    main

    The library reports latency in two parts: .inputLatency() and .outputLatency().

    • Input Latency: The number of samples you should supply ahead of the processing time.
    • Output Latency: The number of samples you will receive behind the processing time.

    Automation Tip: To follow pitch/time automation accurately, provide automation values from the current processing time (.outputLatency() samples ahead of the output) and feed input from .inputLatency() samples ahead of the current processing time.

  5. Seeking, Starting, and Ending playback

    main

    Managing the lifecycle of a sound (especially fixed-length sounds):

    • Starting/Seeking: Use .seek(inputBuffers, inputSamples, playbackRateHint) to move through audio. For the very first block (or after a .reset()), it is recommended to call .seek() with inputSamples = stretch.inputLatency() to align processing time with the start of the input.
    • Ending (Fixed-length sounds): When input runs out, you may still have pending output.
      1. Pass an additional .inputLatency() samples of silence to .process() to ensure the processing time reaches the end.
      2. Use .flush(outputBuffers, outputSamples) to read the final remaining output. It is recommended to read at least .outputLatency() samples.
    // Seeking
    stretch.seek(inputBuffers, inputSamples, playbackRateHint);
    
    // Ending a fixed-length sound
    // 1. Feed silence to reach the end
    float **silence = ...;
    stretch.process(silence, stretch.inputLatency(), outputBuffers, outputSamples);
    
    // 2. Flush remaining output
    stretch.flush(outputBuffers, outputSamples);
  6. Configure Signalsmith Stretch

    main

    To use the library, include signalsmith-stretch.h and instantiate signalsmith::stretch::SignalsmithStretch<T>.

    Configuration can be done via presets or manual configuration:

    • Presets: Use .presetDefault(channels, sampleRate) for standard settings or .presetCheaper(channels, sampleRate) for lower computational cost.
    • Manual: Use .configure(channels, blockSamples, intervalSamples) to specify custom block and interval sizes. You can query the current settings using .blockSamples() and .intervalSamples().

    Both preset and configure methods accept an optional splitComputation flag. When enabled, this spreads computation out more evenly by introducing one extra interval of output latency, which can help in strict real-time environments.

    #include "signalsmith-stretch.h"
    
    signalsmith::stretch::SignalsmithStretch<float> stretch;
    
    // Using presets
    stretch.presetDefault(2, 44100);
    
    // Manual configuration
    stretch.configure(2, 512, 128);
  7. Build Signalsmith Linear with g++

    main

    If not using CMake, you must manually define preprocessor flags and link frameworks to enable hardware acceleration.

    Accelerate (macOS)

    Link the Accelerate framework and define SIGNALSMITH_USE_ACCELERATE:

    g++ -framework Accelerate -DSIGNALSMITH_USE_ACCELERATE

    IPP (Intel)

    Define SIGNALSMITH_USE_IPP and link to IPP::ippcore and IPP::ipps.

    PFFFT

    Use SIGNALSMITH_USE_PFFFT or SIGNALSMITH_USE_PFFFT_DOUBLE depending on whether you require double-precision support.

    g++ -framework Accelerate -DSIGNALSMITH_USE_ACCELERATE
  8. Initialize and use flutter_soloud

    main

    To use flutter_soloud, you must first access the singleton instance via SoLoud.instance and call init() before performing any audio operations. When finished with the audio engine, call deinit() to release resources.

    You can play sounds in two ways:

    1. Directly from an asset: Use playSource(asset: ...) for quick playback.
    2. Pre-loading a sound: Use loadAsset(...) to get a Sound object, then use play(sound) to get a SoundHandle. This is more efficient for sounds that will be played multiple times.
    import 'package:flutter_soloud/flutter_soloud.dart';
    
    void example() async {
      final soloud = SoLoud.instance;
      await soloud.init();
    
      // Option 1: Play directly from asset
      await soloud.playSource(asset: 'assets/sound.mp3');
    
      // Option 2: Pre-load and play
      final sound = await soloud.loadAsset('assets/sound.mp3');
      final handle = soloud.play(sound);
      
      // ... perform audio operations ...
    
      soloud.deinit();
    }
  9. Run flutter_soloud examples

    main

    To run the sample projects provided in the repository, follow these steps:

    1. Clone the repository.
    2. Navigate to the example directory.
    3. Install the necessary Flutter dependencies.
    4. Run the specific target file using flutter run -t <path_to_file>.
    git clone https://github.com/alnitak/flutter_soloud.git
    cd flutter_soloud/example
    flutter pub get
    
    # Run the basic example
    flutter run -t lib/main.dart
    
    # Run a specific feature example
    flutter run -t lib/waveform/waveform.dart
  10. Customize iOS launch screen assets

    main

    To change the launch screen image for the iOS version of your Flutter app, you can use one of two methods:

    Method 1: Direct File Replacement

    Replace the existing image files located in the example/ios/Runner/Assets.xcassets/LaunchImage.imageset/ directory with your own assets.

    Method 2: Using Xcode

    1. Open the iOS project in Xcode by running open ios/Runner.xcworkspace from your terminal.
    2. In the Xcode Project Navigator, navigate to Runner/Assets.xcassets.
    3. Drag and drop your desired images into the asset catalog to replace the existing launch images.
    open ios/Runner.xcworkspace
  11. Compile Signalsmith Stretch

    main

    The library has been tested primarily with AppleClang (Mac) and MSVC (Windows).

    Important Notes:

    • Fast Math: Enabling -ffast-math (or equivalent) is supported, except on Apple Clang 16.0.0, which generates incorrect SIMD code with this flag.
    • Optimization: The algorithm is computationally intensive. Debug builds can be up to 10x slower. It is recommended to enable optimizations specifically for the Stretch module even in Debug builds.
    • Dependencies: Requires the Signalsmith Linear library for FFTs. You can use CMake options to enable faster FFT implementations like SIGNALSMITH_USE_ACCELERATE, SIGNALSMITH_USE_IPP, SIGNALSMITH_USE_PFFFT, or SIGNALSMITH_USE_PFFFT_DOUBLE.