Moshi Documentation

If you want to use the moshi-client with a specific standalone worker instance instead of the default queue, append the worker_addr query parameter to the URL in your browser. The format is /?worker_addr={WORKER_ADDR}, where {WORKER_ADDR} is the address of your worker instance.

Example: https://localhost:5173/?worker_addr=0.0.0.0:8088

To build the web client locally, you must have npm installed. The web UI provides additional echo cancellation which improves model quality. Once built, the files are located in the client/dist directory.

cd client
npm install
npm run build

The rustymimi package provides a standalone Rust-based implementation of Mimi with Python bindings. It is used by moshi_mlx and can be installed independently.

To install the pre-built package via pip:

pip install rustymimi

To compile the package from the repository root:

maturin dev -r -m rust/mimi-pyo3/Cargo.toml

If you are developing Moshi or its MLX implementation, install the packages in editable mode with development dependencies from the root of the repository clone.

# From the root of the clone of the repo
pip install -e 'moshi[dev]'
pip install -e 'moshi_mlx[dev]'
pre-commit install

To run Moshi via Docker Compose (CUDA only), ensure you have the NVIDIA Container Toolkit installed.

docker compose up

For on-device inference on Mac (optimized for Apple Silicon), use the moshi_mlx package. You can run quantized versions (4-bit or 8-bit) or use the web UI.

Important: When using the --hf-repo flag, ensure the repository matches the quantization level specified by -q.

The moshi-client is the frontend for the Moshi demo. To set it up, ensure you have Node.js installed (using nvm use is recommended to match the project's required version).

Follow these steps to prepare the environment:

1. Generate a public/private key pair named cert.pem and key.pem and place them in the root of the moshi-client package.
2. Create an env.local file and define the VITE_QUEUE_API_PATH environment variable (the default value is /api).
3. Install dependencies using npm install.

To start the development server, use npm run dev. To create a production build, use npm run build.

To run Moshi in interactive mode using PyTorch, start the server. You can then access the web UI at http://localhost:8998.

If you need to access the server from a remote machine, use the --gradio-tunnel flag or use SSH port forwarding (ssh -L 8998:localhost:8998). Use --hf-repo to specify a different pretrained model from Hugging Face.

python -m moshi.server [--gradio-tunnel] [--hf-repo kyutai/moshika-pytorch-bf16]

Install the MLX implementation of Moshi via PyPI. Python 3.12 is recommended. If you are not using Python 3.12, you may need to install the Rust toolchain to handle dependencies like rustymimi.

To install the bleeding edge version directly from the git repository, use the editable install command provided below.

You can install the stable version of Moshi from PyPI or the bleeding edge version directly from GitHub.

Requirements:

• Python 3.10 or higher.
• PyTorch 2.2 or 2.4 (ensure PyTorch is installed correctly if using specific CUDA versions).
• For the PyTorch version, a GPU with significant memory (approx. 24GB) is recommended as quantization is not currently supported.

Note for Python 3.14 users: If using Python 3.14, you must disable torch.compile() by setting the environment variable NO_TORCH_COMPILE=1.

To install the repository in editable mode for development, including development dependencies, run:

pip install -e '.[dev]'
pre-commit install

Testing:

• To test Mimi streaming (from the repository root): python scripts/mimi_streaming_test.py
• To test Moshi (requires GPU): python scripts/moshi_benchmark.py