DOCSEARCHbyAlgolia
Submit Docs

LiveKit Agents for Node.js

repository·main·Indexed 21 days ago

https://github.com/livekit/agents-js

A framework for building real-time, programmable, multi-modal voice and visual agents on servers. It enables the integration of various STT, LLM, and TTS providers to create conversational AI participants. The framework includes plugins for Anthropic, AssemblyAI, Azure, Baseten, and Anam, and provides examples for implementing complex workflows such as drive-thru ordering, appointment scheduling, hotel reception, and survey agents.

Tokens
36K
Snippets
131
Records
202
Agent score
74%

What's inside livekit-agents-js

  2. Overview of the Hotel Receptionist Example

    main

    The Hotel Receptionist is a phone-oriented agent designed for a hotel front-desk scenario. It demonstrates complex agentic workflows including:

    • Reservations: Room booking, restaurant reservations, and booking verification/changes.
    • Payments: Card collection and updates via the GetCardTask workflow.
    • Information Retrieval: Policy lookups and concierge requests.
    • Guest Services: Privacy-safe guest messaging, wake-up calls, and group inquiries.
    • Error Recovery: Handling overbooking scenarios.

    The example is structured into three main components:

    1. hotel_db.ts: An in-memory inventory and booking store seeded with demo data.
    2. policies.ts: Implements the lookupPolicy tool by processing hotel policy text.
    3. hotel_receptionist.ts: Defines the HotelReceptionistAgent and the GetCardTask card-capture workflow.

    For general setup instructions, refer to the main repository README.

  3. Use the ElevenLabs plugin for voice synthesis

    main
    The @livekit/agents-plugin-elevenlabs package provides integration for ElevenLabs, allowing you to use their high-quality voice synthesis capabilities within a LiveKit Agent. This enables your conversational agents to speak with realistic, expressive voices during real-time interactions.

  5. Use the Rime plugin for text-to-speech (TTS)

    main
    The Rime plugin provides high-quality text-to-speech (TTS) capabilities for voice synthesis within the LiveKit Agents framework. It allows you to create conversational, multi-modal voice agents that can synthesize speech in real-time.

  6. Use the Silero plugin for Voice Activity Detection

    main
    The Silero plugin is a specialized plugin for the LiveKit Agents Framework that provides Voice Activity Detection (VAD). It allows your agents to detect when a user is speaking and when they have finished speaking, which is essential for building responsive conversational voice agents.

  7. Use the Deepgram plugin for speech recognition

    main

    The @livekit/agents-plugin-deepgram package provides speech recognition capabilities for LiveKit Agents. It allows your programmable participants to hear and transcribe audio in real-time, enabling the creation of conversational voice agents.

    For detailed implementation guides, refer to the LiveKit Agents documentation. For specific technical details on classes and methods, consult the Deepgram plugin API reference.

  8. Use the Resemble plugin for voice synthesis

    main
    The Resemble plugin for LiveKit Agents enables voice synthesis capabilities for your agents. It allows your programmable participants to generate speech using Resemble's voice models, making them suitable for conversational, multi-modal voice agents that can hear and speak in real-time.

  10. Use the xAI plugin for LiveKit Agents

    main
    The xAI plugin provides access to xAI's Grok models via the Realtime API within the LiveKit Agents framework. It allows you to build realtime, programmable participants that can participate in conversational, multi-modal voice agent workflows. To use this plugin, you must integrate it into a LiveKit Agent implementation as described in the LiveKit Agents documentation.

  11. Use the Fish Audio plugin for voice synthesis

    main

    The Fish Audio plugin for LiveKit Agents enables voice synthesis capabilities for your agents. It supports two modes of operation:

    1. Streaming Synthesis: Uses the Fish Audio Live TTS WebSocket API for real-time, low-latency voice generation.
    2. One-shot Synthesis: Uses the HTTP /v1/tts endpoint for generating complete audio clips in a single request.

    Output Format: The plugin delivers audio as raw 16-bit little-endian PCM. The default sample rate is 24 kHz.