jetson-containers

Llamaspeak allows you to talk live with Large Language Models (LLMs) using NVIDIA Riva's Automatic Speech Recognition (ASR) and Text-to-Speech (TTS).

Requirements:

• riva-server must be running.
• text-generation-webui must be running.

Note: For version 2 with multimodal support, use the local_llm container instead.

The repository provides modular containers for a wide range of Edge AI and Robotics tasks. Key categories include:

• ML: pytorch, tensorflow, jax, onnxruntime, deepstream, holoscan
• LLM: SGLang, vLLM, MLC, transformers, ollama, llama.cpp
• VLM (Vision Language Models): llava, VILA, NanoLLM
• VIT (Vision Transformers): NanoOWL, NanoSAM, segment-anything
• RAG & Vector DB: llama-index, langchain, NanoDB, FAISS
• L4T (JetPack specific): l4t-pytorch, l4t-tensorflow, l4t-diffusion
• Robotics & Simulation: ROS, LeRobot, Isaac Sim, MuJoCo
• Speech: whisper, faster-whisper, piper, xtts
• Graphics: ComfyUI, Stable Diffusion WebUI, nerfstudio

For a complete list, browse the packages/ directory.

Speaches is a package that provides support for Voice Activity Detection (VAD), Speech-to-Text (STT), and Text-to-Speech (TTS).

Voice Activity Detection (VAD) is used to identify the presence or absence of human speech in audio streams, which helps reduce computational overhead by preventing the processing of silence or background noise.

The whisper_trt container provides a TensorRT optimized version of Whisper Automatic Speech Recognition (ASR), based on the implementation from https://github.com/NVIDIA-AI-IOT/whisper_trt.

Requirements:

• L4T version >=36

Key Dependencies:

• cuda:12.6
• cudnn:9.3
• tensorrt
• pytorch:2.8
• onnxruntime
• torch2trt
• python, numpy, cmake, onnx

The wyoming-assist-microphone is a Home Assistant add-on designed for NVIDIA Jetson devices. It uses the wyoming-satellite project to enable remote voice satellites via the wyoming protocol. It integrates with other Wyoming containers to provide a full voice pipeline:

• Wake Word Detection: Uses wyoming-openwakeword.
• Speech-to-Text (STT): Uses wyoming-whisper.
• Text-to-Speech (TTS): Uses wyoming-piper.

Requirement: Home Assistant 2023.9 or later.

Tools for knowledge retrieval and RAG workflows:

• graphiti
• jetson-copilot
• langchain
• llama-index
• memvid
• n8n

Speaches provides a suite for speech processing including:

• VAD (Voice Activity Detection): Detects human speech to prevent processing silence or background noise.
• STT (Speech-to-Text)
• TTS (Text-to-Speech)

TTS Implementation Note: The current TTS backend uses onnxruntime and kokoro-onnx. Note that this implementation is currently slower than the kokoro-tts:fastapi backend (which also implements the OpenAI protocol and can be used as a separate service).