TangoFlux Documentation

Install the TangoFlux package directly from the GitHub repository using pip:

pip install git+https://github.com/declare-lab/TangoFlux

The CRPO (Clap-Ranked Preference Optimization) dataset can be generated using two scripts:

1. tangoflux/generate_crpo.py: Generates the initial dataset using a prompt bank and model weights. You can specify sample size and samples per prompt.
2. tangoflux/label_crpo.py: Labels the generated audio and constructs preference pairs, producing a train.json file suitable for train_dpo.py.

You can iterate this process by repeating the steps with updated model weights.

TangoFlux nodes are located in the "TangoFlux" category within the ComfyUI node menu. The core nodes are:

• TangoFluxLoader: Loads the model weights and configuration.
• TangoFluxSampler: Performs the audio generation process.
• TangoFluxVAEDecodeAndPlay: Decodes the latent representation into playable audio.

Speed up inference with TeaCache

You can speed up TangoFlux by up to 2x using TeaCache without significant audio quality degradation. This is a training-free method. On an A800 GPU, inference latency can be reduced from ~4.08s to ~1.95s using TeaCache (0.4).

TangoFlux models can be acquired in three ways:

1. Automatic Download: The TangoFluxLoader node will automatically download all required models upon its first use.
2. Scripted Download: Run the install.py script to download the models.
3. Manual Download: If downloading manually, ensure you place files in the exact directory structure shown below without renaming anything:
   • TangoFlux weights: Download from HuggingFace into models/tangoflux.
   • Text Encoders: Download flan-t5-large from HuggingFace into models/text_encoders/google-flan-t5-large.

To use TangoFlux within ComfyUI, you must install the custom nodes into your custom_nodes directory and install the necessary Python dependencies.

Manual Installation

1. Navigate to your ComfyUI custom_nodes directory.
2. Clone the repository as ComfyUI-TangoFlux.
3. Run the install.py script located in the comfyui subdirectory to install requirements.

Alternatively, you can install the nodes using the ComfyUI Manager.

cd ComfyUI/custom_nodes
git clone https://github.com/declare-lab/TangoFlux ComfyUI-TangoFlux
cd ComfyUI-TangoFlux/comfyui
python install.py

TangoFlux can be used as a custom node within ComfyUI for advanced graph-based audio generation pipelines.

Refer to the ComfyUI-TangoFlux repository to install the necessary custom nodes.

To start a local web-based demonstration interface, run the following command:

tangoflux-demo

To install TangoFlux directly from the GitHub repository, use the following command:

!pip install git+https://github.com/declare-lab/TangoFlux.git

Use the TangoFluxInference class to generate audio from text prompts.

Important: You must specify a duration between 1 and 30 seconds when calling model.generate.

import torchaudio
from tangoflux import TangoFluxInference

model = TangoFluxInference(name='declare-lab/TangoFlux')
# duration must be between 1 and 30
audio = model.generate('Hammer slowly hitting the wooden table', steps=50, duration=10)

torchaudio.save('output.wav', audio, 44100)

To save the generated audio using torchaudio, ensure the tensor is in the correct 2D format [channels, samples]. If the output is a 1D tensor, use .unsqueeze(0) to add the channel dimension.

import torchaudio

# Ensure audio is in the correct format (2D Tensor: [channels, samples])
if len(audio.shape) == 1:
    audio_tensor = audio.unsqueeze(0)
elif len(audio.shape) == 2:
    audio_tensor = audio
else:
    raise ValueError(f"Unexpected audio tensor shape: {audio.shape}")

# Save the audio as a .wav file
torchaudio.save('generated_audio.wav', audio_tensor, sample_rate=44100)

Use the tangoflux command to generate audio files from text prompts via the terminal. Use the --duration flag to specify length in seconds and --steps to control generation quality.

tangoflux "Hammer slowly hitting the wooden table" output.wav --duration 10 --steps 50