DOCSEARCHbyAlgolia
Submit Docs

OpenUtau Documentation

repository·master·Indexed 24 days ago

https://github.com/openutau/openutau

A free, open-source, cross-platform MIDI editor for the UTAU community. It features extensible phonetic editing, advanced tuning with curve expressions, and support for ENUNU AI singers. The documentation covers building the C++ core library and executable via Bazelisk, implementing Phonemizer and Batch Edit Macro plugins, training G2P models, and using the Core API for manipulating notes, pitch points, and vibrato settings.

Tokens
6.4K
Snippets
15
Records
53
Agent score
82%

What's inside OpenUtau

  1. Overview of OpenUtau Features

    master

    OpenUtau is a modern, open-source MIDI editor designed for the UTAU community. Key capabilities include:

    • MIDI Editing: Feature-rich editor with support for importing VSQX (Vocaloid 4) tracks.
    • Phonetic Editing: Extensible real-time phonetic editing with support for various systems (VCV, CVVC, Arpasing, etc.) and languages (English, Japanese, Chinese, Korean, Russian, etc.).
    • Tuning & Expressions: Uses expressions instead of standard UTAU flags. The built-in WORLDLINE-R resampler supports curve tuning.
    • Rendering: Supports pre-rendering for quick previews and is compatible with most UTAU resamplers.
    • AI Support: Supports ENUNU AI singers.
    • Cross-Platform: Runs on Windows (32/64 bit), macOS, and Linux.
    • Internationalization: Full UI translation and file system encoding support without requiring system locale changes.

  2. Train a custom G2P model

    master

    To train a G2P model for a new language:

    1. Directory Setup: Create a new folder adjacent to the en_us folder and place your language's dictionary inside it.
    2. Configuration: Copy cfg.yaml into your new folder. Modify the graphemes and phonemes fields to match your language.
      • Note: The first 4 graphemes and phonemes must be 𒂙, 𒁎, , and " respectively.
    3. Code Modification: Modify train.py to load your specific configuration and dictionary.
    4. Data Formatting: Ensure your dictionary is compatible with SphinxDataset. If not, you may need to reformat the dictionary or implement a custom dataset class.
    5. Hyperparameter Tuning: Adjust batch size, epochs, and other parameters to optimize results.

  3. Build the C++ core executable

    master

    You can build an executable version of the core, though please note that curve expressions will not be available in this version.

    Note for Windows users: Omit the // prefix in the Bazel commands.

    bazelisk build //worldline:main

  4. Build the C++ core library (DLL, dylib, or SO)

    master

    To build the core library (producing a .dll on Windows, .dylib on macOS, or .so on Linux), follow these steps:

    1. Install Bazelisk.
    2. Navigate to the cpp directory.
    3. Execute the build command using Bazelisk.

    Note for Windows users: Omit the // prefix in the Bazel commands.

    bazelisk build //worldline

  5. Train an en_us arpabet G2P model

    master

    To train a G2P (Grapheme-to-Phoneme) model for en_us using the arpabet dictionary, follow these steps:

    1. Place the cmudict-0.7b file in the en_us folder.
    2. Navigate to the py directory.
    3. Install dependencies and run the training script.

    Ensure you have the necessary Python environment set up before running the commands.

    cd py
pip install -r requirements.txt
python g2p/train.py

  6. Implement a Phonemizer plugin

    master

    To create a custom phonemizer in OpenUtau, you must implement the Phonemizer class. The core logic resides in the Process method, which converts lyrics into phonemes or diphones.

    Requirements for a complete Phonemizer:

    1. Produce phonemes (or diphones) based on the lyric and the presence of previous/next notes.
    2. Distribute phonemes to positions relative to the first note in the group.
    3. Support phonetic hinting (e.g., lyrics formatted as read[r iy d] or [r iy d]).
    4. Support extender note alignments (e.g., +n notes) for multisyllabic languages.

    Implementation Tips:

    • Resource Loading: Implement resource loading within SetSinger() and use singer.Location to locate singer-specific files.
    • Performance: If using expensive resources, load them lazily when the phonemizer is first created to manage lifetime efficiently.
    public abstract Phoneme[] Process(Note[] notes, Note? prevNeighbour, Note? nextNeighbour);

  7. Download and Install OpenUtau

    master

    OpenUtau is available for Windows, macOS, and Linux. You can download the latest releases from the official GitHub releases page.

    Available Platforms:

    • Windows: x64 and x86 versions available.
    • macOS: x64 version available.
    • Linux: x64 version available.
    https://github.com/stakira/OpenUtau/releases/latest

  8. Implement Batch Edit Macros

    master

    When developing batch edit macros, follow these requirements to ensure proper undo/redo support and performance:

    1. Undo Grouping: Wrap the entire operation between docManager.StartUndoGroup(); and docManager.EndUndoGroup();.
    2. Command Execution: Perform all modifications by calling docManager.ExecuteCmd(); using command objects. Do not modify the document state directly.
    3. Batch Optimization: For commands that support both single and multiple item variations (e.g., AddNoteCommand), always use the multiple-item variation (e.g., passing a List<UNote> instead of a single UNote) to improve efficiency during batch operations.
    4. Localization: Ensure the macro has a localized name for user visibility.

  10. Explore OpenUtau Plugin Development APIs

    master

    Developers looking to extend OpenUtau can contribute plugins using the following API documentation:

    • Editing Macros API: For creating macros that automate editing tasks.
    • Phonemizers API: For implementing new phonetic systems (e.g., VCV, CVVC, Arpasing) across different languages.

  11. Structure of a G2pPack zip file

    master

    A G2pPack zip file must contain the following three files to function correctly:

    • dict.txt: A dictionary file used for direct lookups.
    • g2p.onnx: The trained ONNX model used to generate phonemes if a lookup in dict.txt fails.
    • phones.txt: A file defining phoneme properties, allowing phonemizers to identify which phonemes are vowels and which should be stretched.
    dict.txt
g2p.onnx
phones.txt