SoundFlow .NET Audio Framework

repository·master·Indexed 19 days ago

https://github.com/lsxprime/soundflow

A comprehensive .NET audio framework for high-fidelity synthesis, real-time processing, and secure audio distribution. It features a core engine with modular extensions, including SoundFlow.Codecs.FFMpeg for wide format support (MP3, AAC, FLAC, etc.), SoundFlow.Midi.PortMidi for hardware MIDI I/O and clock synchronization, and SoundFlow.Extensions.WebRtc.Apm for voice processing features such as Acoustic Echo Cancellation (AEC), Noise Suppression (NS), and Automatic Gain Control (AGC).

Tokens
4.2K
Snippets
15
Records
19
Agent score
67%

What's inside SoundFlow

  1. Overview of SoundFlow features

    master

    SoundFlow is a cross-platform .NET audio engine designed for high-performance, real-time audio processing. It uses a modular component architecture where you build pipelines by connecting sources, modifiers, mixers, and analyzers.

    Key capabilities include:

    • Audio I/O: Multi-device management, on-the-fly device switching, and advanced control (WASAPI, CoreAudio, ALSA).
    • Processing: Mixing, effects (reverb, delay, EQ), and surround sound support.
    • Synthesis & MIDI: Polyphonic synthesis, SoundFont (.sf2) support, MPE support, and graph-based MIDI routing.
    • Analysis: FFT-based spectrum analysis, level metering, and voice activity detection.
    • Security: AES-256-CTR stream encryption, ECDSA digital signatures, audio watermarking (DSSS/LSB), and acoustic fingerprinting.
  2. Configure MIDI Synchronization (Master and Slave modes)

    master

    The PortMidi backend supports advanced MIDI synchronization via the ConfigureSync method on the backend instance.

    Master Mode

    Act as a master clock source. You must provide a renderer (typically a Composition.Renderer) to link the MIDI clock, Start, Stop, and Continue messages to the transport of your composition.

    Slave Mode

    Synchronize the SoundFlow transport to an external device. This supports:

    • MIDI Clock: Includes automatic BPM detection.
    • MIDI Time Code (MTC): For frame-accurate sync with video or professional equipment.

    Parameters for ConfigureSync:

    • mode: SyncMode.Master or SyncMode.Slave.
    • source: The sync source (not used in Master mode).
    • inputDeviceInfo: The input device for Slave mode.
    • outputDeviceInfo: The output device for Master mode.
    • renderer: The transport renderer to synchronize.
    // Example: Master Mode configuration
    midiBackend.ConfigureSync(
        mode: SyncMode.Master,
        source: SyncSource.Internal,
        inputDeviceInfo: null,
        outputDeviceInfo: syncOutputDevice,
        renderer: composition.Renderer
    );
  3. Manage and route MIDI devices

    master

    You can enumerate available MIDI hardware by calling engine.UpdateMidiDevicesInfo(). Once updated, you can access engine.MidiInputDevices and engine.MidiOutputDevices to find specific hardware.

    To connect an input to an output (or an input to a synthesizer), use engine.MidiManager.CreateRoute(input, output).

    Note: Always clean up routes using engine.MidiManager.RemoveRoute(route) when they are no longer needed.

    // Refresh device list
    engine.UpdateMidiDevicesInfo();
    
    // Create a route from an input device to an output device
    MidiRoute route = engine.MidiManager.CreateRoute(firstInput, firstOutput);
    
    // ... use route ...
    
    // Clean up
    engine.MidiManager.RemoveRoute(route);
  4. Install SoundFlow.Midi.PortMidi extension

    master

    To enable hardware MIDI I/O (connecting to physical keyboards, synthesizers, and controllers) on Windows, macOS, or Linux, install the PortMidi backend extension. This also provides high-precision clock synchronization for MIDI Clock Master/Slave functionality.

    dotnet add package SoundFlow.Midi.PortMidi
  5. Use NoiseSuppressor for offline/batch processing

    master

    For processing existing audio files or data streams without a real-time graph, use the NoiseSuppressor component. It works with any ISoundDataProvider.

    There are two ways to process data:

    1. ProcessAll(): Returns the entire processed signal as a float[]. Best for smaller files.
    2. ProcessChunks(): Processes the source in chunks. This is the recommended method for large files to avoid high memory usage. You can subscribe to the OnAudioChunkProcessed event to handle each processed ReadOnlyMemory<float> chunk (e.g., encoding it to a file).
    using SoundFlow.Extensions.WebRtc.Apm.Components;
    
    // ... setup sourceForOffline ...
    
    // Process chunk-by-chunk for large files
    using var offlineSuppressor = new NoiseSuppressor(sourceForOffline, 48000, 1, NoiseSuppressionLevel.VeryHigh); 
    
    offlineSuppressor.OnAudioChunkProcessed += (chunk) =>
    {
        // 'chunk' is ReadOnlyMemory<float>
        encoder.Encode(chunk.ToArray()); 
    };
    
    offlineSuppressor.ProcessChunks(); // Blocks until completion
  6. Install SoundFlow.Codecs.FFMpeg via NuGet

    master

    To add FFmpeg codec support to your SoundFlow project, install the SoundFlow.Codecs.FFMpeg package using either the NuGet Package Manager or the .NET CLI. This package requires the core SoundFlow library to be present in your project.

    # NuGet Package Manager
    Install-Package SoundFlow.Codecs.FFMpeg
    
    # .NET CLI
    dotnet add package SoundFlow.Codecs.FFMpeg
  7. Enable FFmpeg codec support in SoundFlow

    master

    To enable support for a wide range of audio formats (such as MP3, AAC, OGG, Opus, FLAC, etc.), you must register the FFmpegCodecFactory with your AudioEngine instance during initialization. Once registered, the engine will automatically use FFmpeg to decode or encode supported file types without requiring changes to your existing playback or recording logic.

    using SoundFlow.Abstracts;
    using SoundFlow.Backends.MiniAudio;
    using SoundFlow.Codecs.FFMpeg; // 1. Import the FFmpeg codec namespace
    using SoundFlow.Components;
    using SoundFlow.Providers;
    using SoundFlow.Structs;
    
    // 2. Initialize the Audio Engine.
    using var engine = new MiniAudioEngine();
    
    // 3. Register the FFmpeg Codec Factory.
    // This single line enables support for all FFmpeg formats.
    engine.RegisterCodecFactory(new FFmpegCodecFactory());
    
    // From here, the usage is standard SoundFlow.
    // The engine will now automatically use FFmpeg when it encounters an MP3 file.
    
    // Initialize a playback device.
    using var device = engine.InitializePlaybackDevice(null, AudioFormat.DvdHq);
    
    // Create a SoundPlayer with a StreamDataProvider for an MP3 file.
    var player = new SoundPlayer(engine, device.Format,
        new StreamDataProvider(engine, File.OpenRead("path/to/your/audio.mp3")));
    
    // Add the player to the device's MasterMixer.
    device.MasterMixer.AddComponent(player);
    
    // Start playback.
    device.Start();
    player.Play();
    
    Console.WriteLine("Playing MP3 file using FFmpeg... Press any key to stop.");
    Console.ReadKey();
    
    // Clean up.
    player.Stop();
    device.Stop();
  8. Install SoundFlow.Codecs.FFMpeg extension

    master

    The core engine handles common formats, but you can unlock support for virtually any audio format (including AAC, OGG Vorbis, Opus, ALAC, and more) by installing the FFmpeg extension. Once registered via its factory, the engine will automatically detect and play these formats.

    dotnet add package SoundFlow.Codecs.FFMpeg