LiveKit Agents Python and Node.js Examples

repository·main·Indexed 43 days ago

https://github.com/livekit-examples/python-agents-examples

A collection of runnable examples for building production-ready voice, video, and telephony agents using the LiveKit Agents framework. Includes a healthcare intake demo featuring an Anam lip-synced avatar, starter templates for Python and Node.js, and a React-based frontend using Shadcn/ui.

Tokens
154.1K
Snippets
413
Records
538
Agent score
55%

What's inside LiveKit Agents Examples

  1. Overview of the Agent Starter React frontend

    main

    This is a Next.js frontend designed for AI voice assistants. It utilizes the LiveKit JavaScript SDK and React Components to implement core features including:

    • Voice: Real-time audio interaction.
    • Transcription: Text-based representation of audio.
    • Virtual Avatars: Visual representations for agents.

    The application includes its own token server to manage session authentication.

  2. Overview of the Model Battleground Demo

    main
    The Model Battleground is a multi-agent LiveKit experience designed to compare three different voice stacks side-by-side. It uses a single Next.js dashboard to visualize real-time performance metrics (STT, LLM, and TTS latency), transcripts, and connection states for multiple agents. Users can interact with one agent and then use the dashboard to dispatch or disconnect other agents in real time to compare their performance.
  3. Overview of the Web Voice Assistant template

    main

    The Web Voice Assistant is a Next.js starter template for LiveKit Agents. It provides a pre-built voice interface using the LiveKit JavaScript SDK and includes support for:

    • Voice: Real-time voice interaction.
    • Transcriptions: Text representations of the audio stream.
    • Virtual Avatars: Integration for visual avatars.

    This template is designed to be used as a frontend for LiveKit Agents.

  4. Overview of RAG Voice Agent components

    main

    This project implements a Retrieval-Augmented Generation (RAG) system for a voice agent. It uses the following technologies and patterns:

    • Vector Search: Uses an Annoy vector index for semantic search.
    • Embeddings: Uses OpenAI embeddings to convert text into vectors.
    • Context Retrieval: Implements paragraph-based context retrieval and result deduplication using a 'seen' tracking mechanism.
    • Function Tooling: Provides the assistant with a function tool to query the document search on demand.
    • Noise Cancellation: Utilizes BVC for audio processing.

    Project Structure:

    • main.py: The core agent implementation.
    • scrape_docs.py: Scraper for the target documentation site.
    • build_rag_data.py: Orchestrates building the RAG database.
    • rag_db_builder.py: Implementation of the database builder.
    • rag_handler.py: Logic for processing RAG queries.
    • data/: Storage for the vector database files.
  5. Overview of xAI Patient Intake Agent

    main

    The xAI Patient Intake agent is a family-medicine front desk automation designed to handle complex medical office tasks in a single conversation. It can identify callers, book/move appointments, answer policy questions, collect clinical intake, and escalate emergencies.

    Speech Pipeline (xAI end-to-end):

    • Speech to text: xai/stt-1
    • Reasoning: xai/grok-4.3 (with reasoning_effort="none")
    • Text to speech: xai/tts-1 (voice carina)

    Note: The clinic uses an in-memory fake database. It is not a medical device and does not provide medical advice.

  6. Overview of the Next.js AI Voice Assistant Frontend

    main

    This application is a Next.js-based frontend designed for AI voice assistants. It utilizes the LiveKit JavaScript SDK and React Components to provide a complete user interface. Key features include:

    • Voice Interaction: Support for real-time voice AI.
    • Transcription: Displaying text transcriptions of audio.
    • Virtual Avatars: Integration with LiveKit's avatar capabilities.
    • Self-contained Token Server: The application implements its own server to handle LiveKit authentication tokens.
  7. Overview of the LiveKit Drive-Thru Demo

    main

    The LiveKit Drive-Thru Demo is a full-stack voice agent experience consisting of a Python-based voice agent and a Next.js dashboard. The agent acts as a drive-thru attendant, taking orders via voice, while the dashboard provides a real-time visual representation of the user's cart.

    Repository Layout:

    • drive-thru-agent: A Python worker running the 'Kelly' persona. It manages ordering tools and sends checkout RPCs (Remote Procedure Calls) when the user completes their order.
    • frontend: A Next.js 15 application that joins a LiveKit room, streams browser audio, and renders an animated order board and controls.
  8. Project structure and business logic location

    main

    The project is organized as follows:

    • app/: Next.js application routes and API.
    • components/agents-ui/: Core Agents UI components.
    • components/ai-elements/: AI-specific UI elements.
    • components/app/: Business logic location. This folder manages application state, behavior, and the composition of Shadcn UI components.
    • components/ui/: Primitive shadcn/ui components.

    Key Application Components in components/app:

    • session-view.tsx: Initializes the LiveKit session and renders the main UI (chat, media tiles, control bar).
    • view-controller.tsx: Manages transitions between the welcome view and the active session view.
    • welcome-view.tsx: The UI shown when not connected to a session.
    • chat-transcript.tsx: Manages chat transcript transitions.
    • tile-layout.tsx: Manages media tile layouts and transitions.
  9. Understand LemonSlice agent features and modes

    main

    The LemonSlice agent provides an interactive salary negotiation experience with the following capabilities:

    Boss Personalities

    Users can select from three distinct personalities, each with unique voices, avatars, and negotiation styles:

    • Easy (The Encourager): Supportive, warm, and open to discussion.
    • Medium (The Skeptic): Professional, measured, and prone to questioning numbers.
    • Hard (The Busy Executive): Impatient, direct, and dismissive.

    Dual-mode Operation

    • Roleplay mode: The primary mode for negotiating with the selected boss.
    • Coaching mode: Triggered by asking "How am I doing?", this mode provides real-time feedback on performance.

    Session Management

    • Sessions are limited to 3 minutes with an automatic timer.
    • Sessions conclude with a summary once the time expires.
    • The agent tracks coaching requests and negotiation phases.
  10. Understand the Role-Playing Frontend folder structure

    main

    The project follows a standard Next.js App Router structure:

    • app/: Contains layouts, pages, and the API route for connection details.
    • components/: Contains React components like CharacterPortrait, GameStatus, and SessionView.
    • hooks/: Contains custom hooks for connection management, transcription, and debugging.
    • lib/: Contains shared utilities and TypeScript types.
    • public/portraits/: Stores portrait images for NPCs and agents.
    • public/fonts/, postcss.config.js, tailwind.config.ts: Styling and theming configuration.
  11. Model Battleground Repository Layout

    main

    The project is organized into the following directories:

    DirectoryDescription
    battleground-agentAgent 1 (Baseline): Uses AssemblyAI + GPT-4.1-mini + Inworld TTS. Contains the dispatch RPC used to spawn other agents.
    battleground-agent-2Agent 2 (Alt Stack A): Uses Deepgram + Gemini + Cartesia.
    battleground-agent-3Agent 3 (Alt Stack B): Uses Deepgram + Moonshot + Rime.
    frontendNext.js dashboard for real-time metrics, transcripts, and dispatch controls.
  12. Understand the project structure

    main

    The project is organized to separate UI primitives, AI-specific elements, and application business logic:

    • app/: Next.js application routes and API endpoints.
    • components/agents-ui/: Core Agents UI components.
    • components/ai-elements/: Specialized AI components.
    • components/app/: Business logic layer. This folder manages application state, behavior, and the composition of Shadcn UI components. Key files include:
      • session-view.tsx: Initializes the LiveKit session and renders the main UI (chat, media tiles, control bar).
      • view-controller.tsx: Manages transitions between the welcome screen and the active session.
      • welcome-view.tsx: The UI shown before a session is connected.
      • chat-transcript.tsx: Handles chat transcript transitions.
      • tile-layout.tsx: Manages media tile layouts and transitions.