SoulX-Singer Documentation

The MIDI Editor is a specialized web tool designed for singing voice preprocessing. Key capabilities include:

• Piano Roll Editing: Visual editing from C1 to C8 with drag-to-move, drag-to-resize, and pitch nudging (Ctrl/Cmd + Up/Down).
• Lyric Editing: Supports inline editing, batch filling, and precise editing of PITCH, START, and END fields.
• Audio Alignment: Syncs audio waveforms (MP3, WAV, etc.) with the MIDI timeline for precise alignment.
• Overlap Detection: Automatically detects overlapping notes (highlighted in red) with a one-click fix option.
• Import/Export: Handles standard MIDI files with automatic lyric metadata extraction and encoding.

SoulX-Singer is a high-fidelity, zero-shot singing voice synthesis (SVS) and conversion (SVC) framework.

SoulX-Singer (SVS)

Generates realistic singing voices for unseen singers using melody (F0 contour) or score (MIDI notes) conditioning. It supports cross-lingual synthesis and timbre cloning.

SoulX-Singer-SVC (SVC)

Transforms a source singing recording into a target singer's voice. It is transcription-free, meaning it can convert target singing directly from raw audio without requiring lyric or MIDI transcriptions.

To ensure high-quality Singing Voice Synthesis (SVS), follow this manual correction workflow:

1. Export: Convert existing metadata to MIDI using python -m preprocess.tools.midi_parser --meta2midi.
2. Edit: Open the resulting .mid file in the MIDI Editor to correct lyrics, phoneme alignment, note pitches, or durations. Save as vocal_edited.mid.
3. Import: Convert the edited MIDI back to metadata using python -m preprocess.tools.midi_parser --midi2meta.
4. Inference: Use the newly generated edit_metadata.json and the audio clips in edit_cut_wavs for your SVS pipeline.

Overlap Detection

• Visual Warning: Notes that overlap in time are highlighted in red and will flash.
• Auto-fix: Use the 'Eliminate Overlaps' button to automatically fix all overlapping notes.

Audio Alignment

• Waveform Support: Supports MP3, WAV, OGG, FLAC, M4A, and AAC.
• Synchronization: The audio waveform scrolls in sync with the MIDI. Audio and MIDI can be played simultaneously with independent volume controls.
• Navigation: Click on the waveform or the timeline to quickly jump to a specific playback position.

The MIDI Editor is a web-based tool for singing voice preprocessing. To run it locally, ensure you have Node.js 18+ and npm or yarn installed.

Installation Steps

1. Install dependencies:

   npm install

2. Start the development server:

   npm run dev

3. To expose the server to your local network (LAN):

   npm run dev -- --host 0.0.0.0

Production Build

To create a production-ready build:

npm run build

To preview the production build locally:

npm run preview

# Install dependencies
npm install

# Start dev server
npm run dev

# Expose to LAN
npm run dev -- --host 0.0.0.0

The Piano Roll allows for visual MIDI editing with the following interactions:

Mouse Operations

• Move Note: Drag the note block to change position and pitch.
• Adjust Start Time: Drag the left edge of a note.
• Adjust End Time: Drag the right edge of a note.
• Add Note: Double-click on an empty area in the Piano Roll.
• Preview Pitch: Click on the piano keys on the left side to hear the pitch.

Keyboard Shortcuts

• Adjust Pitch: Command/Ctrl + Up/Down Arrow keys.
• Delete Note: Backspace or Delete.
• Horizontal Zoom: Ctrl(Command) + Mouse Wheel.
• Vertical Zoom: Ctrl(Command) + Shift(Option) + Mouse Wheel.

The pipeline transforms raw audio into structured singing data (metadata) through a multi-step process: vocal separation/dereverberation, F0 extraction/VAD, lyrics transcription, and note transcription.

Prerequisites:

1. Prepare Prompt audio (for timbre/style) and Target audio (the audio to process).
2. Configure input/output paths and module switches in example/preprocess.sh.

Execution: Run the following command to start the pipeline:

bash example/preprocess.sh

Outputs:

• Metadata (JSON): Saved in the same directory as your input audio with the same filename (e.g., audio.mp3 $\rightarrow$ audio.json).
• Intermediate Results: Saved in the configured save_dir (includes separated vocals, accompaniment, F0, and VAD outputs).

You must download the SVS/SVC models and the preprocessing models using huggingface_hub. Ensure the models are placed in the pretrained_models/ directory structure as specified.

pip install -U huggingface_hub

# Download the SoulX-Singer SVS and SVC model
hf download Soul-AILab/SoulX-Singer --local-dir pretrained_models/SoulX-Singer

# Download models required for preprocessing
hf download Soul-AILab/SoulX-Singer-Preprocess --local-dir pretrained_models/SoulX-Singer-Preprocess

Lyrics and precise note timing can be managed via the lyric table/list:

• Real-time Editing: Edit lyrics directly in the list on the right side of the interface.
• Batch Filling: Input a string of lyrics to automatically fill notes in sequence. You can choose to start the batch fill from the currently selected note.
• Precise Value Adjustment: Manually edit the following fields for each note:
  • PITCH: Pitch value.
  • START: Start time.
  • END: End time.
• Confirmation: After modifying numerical values, press Enter or click the checkmark (✓) to confirm. Press Escape to cancel.

SoulX-Singer provides interactive web interfaces for both SVS and SVC tasks.

• For SVS (synthesizing from lyrics and MIDI): python webui.py
• For SVC (audio-to-audio conversion): python webui_svc.py

# SVS WebUI
python webui.py

# SVC WebUI
python webui_svc.py

To set up SoulX-Singer, clone the repository, create a Conda environment with Python 3.10, and install the required dependencies via pip.

# 1. Clone Repository
git clone https://github.com/Soul-AILab/SoulX-Singer.git
cd SoulX-Singer

# 2. Set Up Environment
conda create -n soulxsinger -y python=3.10
conda activate soulxsinger
pip install -r requirements.txt

To use the transcription and editing toolkit, you must set up a Conda environment with Python 3.10 and install the required dependencies from the preprocess directory.

1. Create and activate a new conda environment named soulxsinger:

   conda create -n soulxsinger -y python=3.10
   conda activate soulxsinger

2. Install dependencies:

   cd preprocess
   pip install -r requirements.txt