DOCSEARCHbyAlgolia
Submit Docs

w-okada Voice Changer

repository·master·Indexed 10 days ago

https://github.com/w-okada/voice-changer

A real-time voice conversion tool supporting AI models such as RVC, MMVC, and so-vits-svc. It features a server-client architecture for cross-platform use on Windows, Mac, Linux, and Colab, including Docker-based training environments for MMVC and a specialized AudioWorkletNode for real-time browser-based processing.

Tokens
41.8K
Snippets
77
Records
219
Agent score
95%

What's inside Voice Changer

  1. Overview of VCClient

    master

    VCClient is software designed for real-time voice conversion using AI. It supports multiple AI models, various platforms, and can be used in two primary configurations:

    1. Standalone: Voice conversion is performed locally on your PC.
    2. Network-based: Voice conversion is offloaded to an external machine via a network. This is useful for reducing the computational load on your local machine when running high-demand applications like games simultaneously.

  2. What is VC Client

    master

    VC Client is client software designed for real-time voice conversion using various AI models. It supports a server-client architecture, allowing you to run the voice conversion process on a separate high-performance PC to minimize the impact on other resource-intensive tasks like streaming.

    Supported AI Models:

    • RVC (Retrieval-based Voice Conversion)
    • Beatrice v2 (v2 only)
    • Beatrice JVS Corpus Edition (Experimental, Windows only, depends on CPU, v1 only)
    • MMVC (v1 only)
    • so-vits-svc (v1 only)
    • DDSP-SVC (v1 only)

    Platform Support:

    • Windows
    • Mac (including Apple Silicon M1)
    • Linux
    • Google Colaboratory

  3. Licensing and Usage Restrictions for Beatrice API

    master

    The Beatrice API (1.1.0 JVS Corpus Edition) is subject to strict usage restrictions:

    • Commercial Use Prohibited: Commercial use is strictly forbidden. The criteria for what constitutes commercial use follow the standards set by the JVS Corpus.
    • Redistribution: If you intend to use the Beatrice API in ways that exceed the scope permitted by copyright law (such as redistribution), you must obtain prior permission from Project Beatrice.

    Always ensure your use case complies with these terms before integration.

  5. Understand the difference between Client Device Mode and Server Device Mode

    master

    Since version 1.5.2.9, the voice changer supports two distinct modes for handling audio input (microphone) and output (speakers). Choosing the right mode depends on whether you prioritize low latency or browser-based audio processing features.

    Client Device Mode

    In this mode, the browser (e.g., Chrome) manages the microphone and speaker hardware.

    • Pros:
      • Leverages Chrome's built-in audio handling.
      • Allows use of Chrome's native audio features like noise cancellation and web conferencing optimizations.
    • Cons:
      • May introduce higher latency (delay) in the voice conversion process.

    Server Device Mode

    In this mode, the VC Client directly controls the microphone and speaker hardware connected to the PC.

    • Pros:
      • Significantly lower latency because the client bypasses browser-level audio processing.
    • Cons:
      • Some microphone or speaker hardware may not be compatible with direct control.
      • You cannot use Chrome's built-in audio enhancement features like noise cancellation.

  6. Understand Realtime Conversion Latency and Controls

    master

    The total latency from sound input to output is calculated as buf + res seconds. When adjusting settings, ensure that buf is larger than res to avoid issues.

    Latency Components

    • buf: The length (ms) of the audio segment being sliced. Reducing CHUNK will decrease this value.
    • res: The time required to process the data (the sum of CHUNK and EXTRA). Reducing either CHUNK or EXTRA will decrease this value.

    Note: If using 'server device' mode, buf is not displayed; refer to the CHUNK value instead.

    Audio Controls

    • vol: The volume of the voice after conversion.
    • GAIN (in): Adjusts the volume of the audio being fed into the model.
    • GAIN (out): Adjusts the volume of the converted audio output.
    • pass through: Outputs the original audio without any conversion.
    • TUNE: Adjusts the pitch of the voice.
    • INDEX (RVC only): Specifies the ratio of features used from training. 0 uses raw HuBERT output; 1 uses all features. Higher ratios may increase search time.

  7. Understand Realtime Conversion Latency and Tuning

    master

    The latency (lag) from speaking to hearing the converted voice is calculated as buf + res seconds.

    To optimize performance:

    • Adjust buf so that its value is longer than res.
    • If using server device mode, buf is not displayed; use the CHUNK value as a guide for adjustments.
    • To reduce res, decrease either CHUNK or EXTRA.
    • To reduce buf, decrease CHUNK.

  8. Terms of Use for Tsukuyomi-chan Voice Model

    master

    When using the Tsukuyomi-chan real-time voice changer, you must adhere to the Tsukuyomi-chan corpus terms of use. The following uses are strictly prohibited:

    • Using the voice changer to criticize or attack individuals (definitions of 'criticism/attack' follow the Tsukuyomi-chan character license).
    • Using the voice changer to call for support or opposition to specific political positions, religions, or ideologies.
    • Publishing highly stimulating/explicit expressions without proper zoning/filtering.
    • Publishing content in a way that permits others to perform secondary use (e.g., using the output as material for others to use).

    Note: Distributing or selling works intended for appreciation/viewing is permitted.

  9. Configure audio processing and latency settings

    master

    Adjust these settings to balance audio quality, noise reduction, and processing latency:

    • NOISE: Toggles noise reduction. Only applicable in client Device mode. Includes Echo (echo cancellation) and Sup1/Sup2 (noise suppression).
    • S. Thresh (Noise Gate): Sets the RMS threshold for voice conversion. If the input volume is below this threshold, the system treats it as silence and skips conversion to reduce load.
    • CHUNK (Input Chunk Num): Determines the length of audio segments processed at once. Larger values increase conversion efficiency but increase the maximum delay before conversion starts (indicated by the buff: value).
    • EXTRA (Extra Data Length): Determines how much historical audio data is used during conversion. Longer history improves accuracy but increases computation time (latency/res) significantly, likely due to Transformer complexity.

  10. Understand the GUI Interface Components

    master

    The GUI is organized into several functional areas:

    Title Area

    Contains links to external resources:

    • Octocat icon: Link to the GitHub repository.
    • Question Mark icon: Link to the manual.
    • Wrench icon: Link to various utility tools.
    • Coffee icon: Link to donate to the developer.
    • clear setting: Resets all current settings to default.

    Model Selection Area

    Used to select the AI model for conversion.

    • Clicking a model loads it and displays the character image.
    • Clicking the Edit button allows you to manage the list of available model slots.

  11. Terms of Use for Amitaro Voice Model

    master

    When using the Amitaro real-time voice changer, you must follow the Amitaro Voice Material Laboratory terms.

    Permitted Uses:

    • Creating voice models using Amitaro's voice materials or corpus.
    • Using voice changers or voice conversion to transform your own voice into Amitaro's voice.

    Mandatory Requirements:

    • You must explicitly state that you are using voice conversion to sound like Amitaro (or Koharane Ami).
    • You must ensure it is clear to anyone that it is not actually Amitaro (or Koharane Ami) speaking.
    • Content spoken using Amitaro's voice must remain within the scope of the voice material usage terms; sensitive or inappropriate statements are prohibited.