Real-Time AI Voice Chat

The core codebase of this project is licensed under the MIT License.

Important: This project integrates external components such as specific TTS engines (e.g., Coqui XTTSv2) and LLM providers. These components are subject to their own respective licensing terms. Users are responsible for ensuring compliance with the licenses of all integrated third-party components used in their implementation.

After installation, follow these steps to start the server and access the web interface.

If using Docker

The application starts automatically with docker compose up -d. Use docker compose logs -f app to monitor logs.

If using Manual/Script Installation

1. Activate your virtual environment.
2. Navigate to the code directory.
3. Run python server.py.

Accessing the UI

1. Open http://localhost:8000 in your browser.
2. Grant microphone permissions.
3. Click "Start" to begin.

A pre-built DeepSpeed wheel is available for Windows users to avoid manual compilation. This specific wheel was built on Windows 11 with the following environment requirements:

• PyTorch: 2.5.1
• CUDA: 12.1
• Python: 3.10.9
• Enabled DeepSpeed Options: CUTLASS_OPS, SPARSE_ATTN, INFERENCE_CORE_OPS

If your environment matches these specifications, you can install the wheel directly using pip.

pip install .\code\deepspeed_wheel\deepspeed-0.16.1+unknown-cp310-cp310-win_amd64.whl

The Docker installation is the recommended method for Linux users with GPUs, as it bundles the application, dependencies, and Ollama into manageable containers.

Note: If you need to customize models or settings in code/*.py, do so before building the images.

# 1. Clone the repository
git clone https://github.com/KoljaB/RealtimeVoiceChat.git
cd RealtimeVoiceChat

# 2. Build the Docker images
docker compose build

# 3. Start the services (App & Ollama)
docker compose up -d

# 4. Pull the desired Ollama model into the running container
docker compose exec ollama ollama pull hf.co/bartowski/huihui-ai_Mistral-Small-24B-Instruct-2501-abliterated-GGUF:Q4_K_M

# (Optional) Verify the model is available
docker compose exec ollama ollama list

To enable SSL in server.py:

1. Set USE_SSL = True.
2. Provide paths to your certificate via SSL_CERT_PATH and SSL_KEY_PATH.

Docker Users: You must adjust docker-compose.yml to map the SSL port (e.g., 443) and mount your certificate files as volumes.

You can customize the Text-to-Speech engine and specific voice settings by modifying the Python files in the code/ directory.

• Engine Selection: In server.py, change START_ENGINE to one of: "coqui", "kokoro", or "orpheus".
• Voice Settings: Adjust engine-specific settings (e.g., voice model path for Coqui, speaker ID for Orpheus, or speed) within the AudioProcessor.__init__ method in audio_module.py.

Fine-tune how the system listens and detects speech:

• STT Settings: In transcribe.py, modify DEFAULT_RECORDER_CONFIG to change the Whisper model, language, or silence thresholds like silence_limit_seconds.
• Turn Detection: In turndetect.py, adjust the pause duration constants within the TurnDetector.update_settings method to change how sensitive the system is to pauses in conversation.

Modify the AI's 'brain' by editing server.py and llm_module.py:

• Provider: Set LLM_START_PROVIDER to either "ollama" or "openai" in server.py.
• Model: Set LLM_START_MODEL in server.py (e.g., a GGUF path for Ollama or a model name for OpenAI).
• Personality: Customize the AI's behavior by editing system_prompt.txt.

Note for Docker users: If using Ollama, ensure you pull the model into the container after startup using docker compose exec ollama ollama pull <model_name>.

The server can be started using uvicorn. The behavior depends on the USE_SSL configuration variable.

Without SSL: Starts the server on 0.0.0.0:8000.

With SSL: Requires the following certificate files to exist in the root directory:

• 127.0.0.1+1.pem
• 127.0.0.1+1-key.pem

If these files are missing, the server will exit with an error. You can generate them using mkcert.

if __name__ == "__main__":
    if not USE_SSL:
        uvicorn.run("server:app", host="0.0.0.0", port=8000, log_config=None)
    else:
        uvicorn.run(
            "server:app",
            host="0.0.0.0",
            port=8000,
            log_config=None,
            ssl_certfile=cert_file,
            ssl_keyfile=key_file,
        )

The server.py entrypoint uses the MAX_AUDIO_QUEUE_SIZE environment variable to control the maximum number of audio chunks allowed in the incoming queue before dropping packets to prevent lag. If not provided, it defaults to 50.

To set this, export the variable before running the server:

export MAX_AUDIO_QUEUE_SIZE=100
python server.py