RustySynth SoundFont MIDI Synthesizer

repository·main·Indexed 42 days ago

https://github.com/sinshu/rustysynth

A pure Rust SoundFont MIDI synthesizer ported from MeltySynth, supporting real-time audio playback and offline waveform rendering. It provides a Synthesizer API for manual note control and a MidiFileSequencer for processing MIDI files, including support for various loop extensions like RPG Maker and Final Fantasy.

Tokens
7.6K
Snippets
29
Records
38
Agent score
53%

What's inside RustySynth

  1. Understand the MidiFile structure

    main

    A MidiFile consists of two primary vectors that represent the sequence of events:

    • messages: A Vec<Message> containing the MIDI events (e.g., Normal notes, TempoChange, LoopStart, EndOfTrack).
    • times: A Vec<f64> containing the absolute time in seconds for each corresponding message in the messages vector.

    Note that MidiFile is non_exhaustive, meaning new fields may be added in future versions without breaking code, but you should access existing fields via their public names.

  2. Configure MIDI file loop types with MidiFileLoopType

    main

    When playing back MIDI files in RustySynth, you can specify how the playback should loop using the MidiFileLoopType enum. This allows you to define loop points either via explicit tick values or by following specific MIDI controller (CC) conventions used by various game engines or software styles.

    Available loop modes:

    • LoopPoint(usize): Uses a specific MIDI tick value as the loop start point.
    • RpgMaker: Uses MIDI CC #111 to define the loop start point.
    • IncredibleMachine: Uses MIDI CC #110 for the loop start point and CC #111 for the loop end point.
    • FinalFantasy: Uses MIDI CC #116 for the loop start point and CC #117 for the loop end point.
    // Example of using a specific tick value for looping
    let loop_type = MidiFileLoopType::LoopPoint(480);
    
    // Example of using the RPG Maker style (CC #111)
    let loop_type = MidiFileLoopType::RpgMaker;
  3. Synthesize a MIDI file offline

    main

    To render a MIDI file to audio buffers (offline synthesis), use the MidiFileSequencer. This wraps a Synthesizer and allows you to play a MidiFile before calling render to generate the waveform based on the MIDI file's duration.

    // Load the SoundFont.
    let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
    let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());
    
    // Load the MIDI file.
    let mut mid = File::open("flourish.mid").unwrap();
    let midi_file = Arc::new(MidiFile::new(&mut mid).unwrap());
    
    // Create the MIDI file sequencer.
    let settings = SynthesizerSettings::new(44100);
    let synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();
    let mut sequencer = MidiFileSequencer::new(synthesizer);
    
    // Play the MIDI file.
    sequencer.play(&midi_file, false);
    
    // The output buffer.
    let sample_count = (settings.sample_rate as f64 * midi_file.get_length()) as usize;
    let mut left: Vec<f32> = vec![0_f32; sample_count];
    let mut right: Vec<f32> = vec![0_f32; sample_count];
    
    // Render the waveform.
    sequencer.render(&mut left[..], &mut right[..]);
  4. Synthesize a simple chord manually

    main

    To synthesize specific notes without a MIDI file, you can use the Synthesizer directly. You must load a SoundFont, initialize SynthesizerSettings with a sample rate, and then call note_on for each note. Finally, use render to fill stereo output buffers.

    // Load the SoundFont.
    let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
    let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());
    
    // Create the synthesizer.
    let settings = SynthesizerSettings::new(44100);
    let mut synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();
    
    // Play some notes (middle C, E, G).
    synthesizer.note_on(0, 60, 100);
    synthesizer.note_on(0, 64, 100);
    synthesizer.note_on(0, 67, 100);
    
    // The output buffer (3 seconds).
    let sample_count = (3 * settings.sample_rate) as usize;
    let mut left: Vec<f32> = vec![0_f32; sample_count];
    let mut right: Vec<f32> = vec![0_f32; sample_count];
    
    // Render the waveform.
    synthesizer.render(&mut left[..], &mut right[..]);
  5. Synthesize a MIDI file in real-time

    main

    For real-time audio playback, integrate MidiFileSequencer with an audio output device (e.g., using the TinyAudio crate). In the audio callback, call sequencer.render to fill the output buffers and interleave the left and right channels for the device.

    // Setup the audio output.
    let params = OutputDeviceParameters {
        channels_count: 2,
        sample_rate: 44100,
        channel_sample_count: 4410,
    };
    
    // Buffer for the audio output.
    let mut left: Vec<f32> = vec![0_f32; params.channel_sample_count];
    let mut right: Vec<f32> = vec![0_f32; params.channel_sample_count];
    
    // Load the SoundFont.
    let mut sf2 = File::open("TimGM6mb.sf2").unwrap();
    let sound_font = Arc::new(SoundFont::new(&mut sf2).unwrap());
    
    // Load the MIDI file.
    let mut mid = File::open("flourish.mid").unwrap();
    let midi_file = Arc::new(MidiFile::new(&mut mid).unwrap());
    
    // Create the MIDI file sequencer.
    let settings = SynthesizerSettings::new(params.sample_rate as i32);
    let synthesizer = Synthesizer::new(&sound_font, &settings).unwrap();
    let mut sequencer = MidiFileSequencer::new(synthesizer);
    
    // Play the MIDI file.
    sequencer.play(&midi_file, false);
    
    // Start the audio output.
    let _device = run_output_device(params, {
        move |data| {
            sequencer.render(&mut left[..], &mut right[..]);
            for (i, value) in left.iter().interleave(right.iter()).enumerate() {
                data[i] = *value;
            }
        }
    })
    .unwrap();
    
    // Wait for 10 seconds.
    std::thread::sleep(std::time::Duration::from_secs(10));
  6. Use the PresetRegion type to access instrument parameters

    main

    A PresetRegion represents a specific range of keys and velocities within a SoundFont preset, defining how instrument parameters (like filter cutoff, pan, or LFO settings) should be modified. You can use the PresetRegion methods to retrieve processed parameter values (e.g., as f32 or i32) that are ready for use in a synthesizer engine.

    Common parameter accessors include:

    • get_pan(): Returns the pan as an f32.
    • get_initial_filter_cutoff_frequency(): Returns the filter cutoff as an f32.
    • get_key_range_start() / get_key_range_end(): Returns the key range boundaries.
    • get_velocity_range_start() / get_velocity_range_end(): Returns the velocity range boundaries.
    • get_instrument_id(): Returns the associated instrument index.
  7. Process MIDI messages

    main

    The process_midi_message method is the primary way to interact with the synthesizer using MIDI data. It supports standard MIDI commands including:

    • Note On (0x90): Starts a note.
    • Note Off (0x80): Stops a note.
    • Program Change (0xC0): Changes the instrument patch.
    • Controller Messages (0xB0): Supports various controllers such as Bank Selection (0x00), Modulation (0x01, 0x21), Volume (0x07, 0x27), Pan (0x0A, 0x2A), Expression (0x0B, 0x2B), Hold Pedal (0x40), Reverb Send (0x5B), and Chorus Send (0x5D).
    • Pitch Bend (0xE0): Adjusts pitch.
    • All Notes Off/Reset (0x78, 0x79, 0x7B): Resets or stops notes on a specific channel.
    synthesizer.process_midi_message(channel, command, data1, data2);
  8. Initialize a MidiFileSequencer

    main

    To play MIDI files through the synthesizer, create a new MidiFileSequencer instance by passing in a Synthesizer. This sequencer manages the playback state, including playback speed, current time, and looping logic.

    use rustysynth::midifile_sequencer::MidiFileSequencer;
    use rustysynth::synthesizer::Synthesizer;
    
    let synthesizer = Synthesizer::new(); // Assuming a standard constructor
    let mut sequencer = MidiFileSequencer::new(synthesizer);
    let mut sequencer = MidiFileSequencer::new(synthesizer);
  9. Use the Preset struct to access SoundFont instrument data

    main

    The Preset struct represents a single instrument preset within a SoundFont. It contains metadata such as the preset name, patch number, bank number, and other classification info (library, genre, morphology), as well as a collection of PresetRegion objects that define how the preset sounds.

    You can access the metadata using the following getter methods:

    • get_name(&self) -> &str: Returns the name of the preset.
    • get_patch_number(&self) -> i32: Returns the patch number.
    • get_bank_number(&self) -> i32: Returns the bank number.
    • get_library(&self) -> i32: Returns the library information.
    • get_genre(&self) -> i32: Returns the genre information.
    • get_morphology(&self) -> i32: Returns the morphology information.
    • get_regions(&self) -> &[PresetRegion]: Returns a slice of the regions associated with this preset.
    // Example of accessing Preset metadata
    let name = preset.get_name();
    let patch = preset.get_patch_number();
    let regions = preset.get_regions();
  10. Handle MIDI loop extensions with new_with_loop_type

    main

    The new_with_loop_type method allows you to parse MIDI files that use specific Control Change (CC) patterns to define loop points. This is useful for compatibility with legacy game engines.

    Supported MidiFileLoopType variants:

    • LoopPoint(usize): Specifies the loop start point by a specific tick value. A Message::LoopStart is automatically inserted at this tick.
    • RpgMaker: Uses CC #111 as the loop start point.
    • IncredibleMachine: Uses CC #110 as the loop start point and CC #111 as the loop end point.
    • FinalFantasy: Uses CC #116 as the loop start point and CC #117 as the loop end point.