DOCSEARCHbyAlgolia
Submit Docs

ElevenLabs UI Documentation

repository·main·Indexed 25 days ago

https://github.com/elevenlabs/ui

A React component library built on shadcn/ui for developers creating audio-centric and agentic applications. It provides specialized UI elements including waveforms, orbs, voice agents, and audio players. Components can be installed via the @elevenlabs/cli or the shadcn/ui CLI.

Tokens
26.7K
Snippets
83
Records
143
Agent score
80%

What's inside ElevenLabs UI

  1. Overview of ElevenLabs UI

    main

    ElevenLabs UI is a component library built on top of shadcn/ui specifically designed for building audio and agentic applications. It provides pre-built, customizable React components such as:

    • Orbs
    • Waveforms
    • Voice agents
    • Audio players

  2. Important considerations for Speech Input

    main

    When implementing the SpeechInput component, keep the following requirements and behaviors in mind:

    • API Key Security: You must use an ElevenLabs API key to generate Scribe tokens. Crucially, token generation must happen on the server-side to prevent exposing your API key to the client.
    • Microphone Permissions: The component manages microphone access requests automatically.
    • Real-time Communication: It utilizes WebSockets to communicate with the ElevenLabs Scribe API.
    • Voice Activity Detection (VAD): In VAD mode, the component automatically commits transcripts when it detects pauses in speech.
    • UI Behavior:
      • The preview component employs a gradient mask to handle text overflow.
      • The layout is responsive and adjusts based on whether the record button is positioned at the start or the end of the input field.

  3. How the Audio Player works with AudioPlayerProvider

    main

    The AudioPlayerProvider is the central state manager for the audio player. It manages playback state, current tracks, duration, and errors. All other audio player components (buttons, progress bars, etc.) must be wrapped within an AudioPlayerProvider to function.

    Basic Implementation

    <AudioPlayerProvider>
  <div className="flex items-center gap-4">
    <AudioPlayerButton />
    <AudioPlayerProgress className="flex-1" />
    <AudioPlayerTime />
    <span>/</span>
    <AudioPlayerDuration />
  </div>
</AudioPlayerProvider>

  4. Use the Orb component

    main

    The Orb component is a 3D animated orb built with Three.js. It can be used as a simple visual element or enhanced with audio reactivity and agent state visualization.

    Basic usage:

    import { Orb } from "@/components/ui/orb"

export function MyComponent() {
  return <Orb />
}
    import { Orb } from "@/components/ui/orb"

<Orb />

  5. Use the Message component

    main

    The Message component is a composable set of components used to display chat messages. It automatically handles alignment (user messages to the right, assistant messages to the left) and styling based on the from prop.

    It is composed of three main parts:

    1. Message: The layout container.
    2. MessageAvatar: The profile image/fallback.
    3. MessageContent: The text or content container.

    Basic Usage

    import { Message, MessageAvatar, MessageContent } from "@/components/ui/message"

// User message
<Message from="user">
  <MessageAvatar src="/user-avatar.jpg" name="John" />
  <MessageContent>Hello, how can I help you?</MessageContent>
</Message>

// Assistant message
<Message from="assistant">
  <MessageAvatar src="/assistant-avatar.jpg" name="AI" />
  <MessageContent>I'm here to assist you with any questions!</MessageContent>
</Message>
    import { Message, MessageAvatar, MessageContent } from "@/components/ui/message"

<Message from="user">
  <MessageAvatar src="/user-avatar.jpg" name="John" />
  <MessageContent>Hello, how can I help you?</MessageContent>
</Message>

<Message from="assistant">
  <MessageAvatar src="/assistant-avatar.jpg" name="AI" />
  <MessageContent>I'm here to assist you with any questions!</MessageContent>
</Message>

  6. Install the Bar Visualizer component

    main

    You can install the BarVisualizer component using the ElevenLabs CLI or by manually copying the source code into your project.

    Using CLI (Recommended)

    Run the following command to add the component to your project:

    npx @elevenlabs/cli@latest components add bar-visualizer

    Manual Installation

    1. Copy the source code from components/ui/bar-visualizer.tsx.
    2. Paste it into your project's component directory.
    3. Update any import paths to match your project's structure.

  7. Install the Mic Selector component

    main

    You can install the MicSelector component using the ElevenLabs CLI or by manually copying the source code into your project.

    Using CLI (Recommended)

    Run the following command to add the component to your project:

    npx @elevenlabs/cli@latest components add mic-selector

    Manual Installation

    1. Copy the source code from components/ui/mic-selector.tsx into your project.
    2. Update the import paths within the file to match your project's directory structure.

  8. Use the Matrix component

    main

    The Matrix component is a retro dot-matrix display that supports static patterns, animations, and VU meter modes.

    Importing components:

    import { digits, loader, Matrix, vu, wave } from "@/components/ui/matrix"

    Static Pattern (e.g., using digits):

    <Matrix rows={7} cols={5} pattern={digits[5]} ariaLabel="Number five" />

    Animated Display (e.g., using wave):

    <Matrix
  rows={7}
  cols={7}
  frames={wave}
  fps={20}
  loop
  ariaLabel="Wave animation"
/>

    VU Meter (Audio Visualization):

    <Matrix
  rows={7}
  cols={12}
  mode="vu"
  levels={[0.1, 0.6, 0.9, 0.7, 0.4, 0.8, 0.5, 0.3, 0.6, 0.9, 0.5, 0.2]}
/>
    import { digits, loader, Matrix, vu, wave } from "@/components/ui/matrix"

// Static Pattern
<Matrix rows={7} cols={5} pattern={digits[5]} ariaLabel="Number five" />

// Animated Display
<Matrix
  rows={7}
  cols={7}
  frames={wave}
  fps={20}
  loop
  ariaLabel="Wave animation"
/>

// VU Meter
<Matrix
  rows={7}
  cols={12}
  mode="vu"
  levels={[0.1, 0.6, 0.9, 0.7, 0.4, 0.8, 0.5, 0.3, 0.6, 0.9, 0.5, 0.2]}
/>

  9. Install the Message component

    main

    You can install the Message component using the ElevenLabs CLI or by manually copying the source code.

    Using CLI:

    npx @elevenlabs/cli@latest components add message

    Manual Installation:

    1. Copy the source code from components/ui/message.tsx into your project.
    2. Update the import paths to match your project's directory structure.

  10. Install the Response component

    main

    You can install the Response component using the ElevenLabs CLI or manually.

    Using CLI (Recommended)

    Run the following command to add the component to your project:

    npx @elevenlabs/cli@latest components add response

    Manual Installation

    1. Install the streamdown dependency:
      npm install streamdown
    2. Copy the component source code into your project (e.g., components/ui/response.tsx).
    3. Update the import paths within the component to match your project's directory structure.

  11. Use the VoiceButton component

    main

    The VoiceButton is an interactive component designed for voice recording workflows. It supports five distinct states: idle, recording, processing, success, and error. It features live waveform visualization during the recording state and automatic feedback transitions for success and error states.

    Basic Usage

    To use the button, manage its state using a local state hook and pass the state and an onPress handler to the component.

    import { useState } from "react"
import { VoiceButton } from "@/components/ui/voice-button"

const [state, setState] = useState<"idle" | "recording" | "processing">("idle")

<VoiceButton
  state={state}
  onPress={() => {
    if (state === "idle") {
      setState("recording")
    } else {
      setState("processing")
    }
  }}
/>

    Icon Button Mode

    To create an icon-only button, set the size prop to "icon" and provide an icon prop.

    import { MicIcon } from "lucide-react"
import { VoiceButton } from "@/components/ui/voice-button"

<VoiceButton state="idle" size="icon" icon={<MicIcon />} />

    Customizing Layout and Styling

    You can add labels, keyboard shortcuts (via the trailing prop), and custom CSS classes.

    <VoiceButton
  state="idle"
  label="Press to speak"
  trailing="⌥Space"
  onPress={() => console.log("Button pressed")}
  variant="outline"
  size="lg"
  className="w-full"
  waveformClassName="bg-primary/10"
/>
    import { VoiceButton } from "@/components/ui/voice-button"

// Basic usage example
const [state, setState] = useState<"idle" | "recording" | "processing">("idle")

<VoiceButton
  state={state}
  onPress={() => {
    if (state === "idle") {
      setState("recording")
    } else {
      setState("processing")
    }
  }}
/>

  12. Install the Audio Player component

    main

    You can install the Audio Player component using the ElevenLabs CLI or by manual installation.

    Using CLI (Recommended)

    npx @elevenlabs/cli@latest components add audio-player

    Manual Installation

    1. Install the required dependencies:
    npm install @radix-ui/react-slider lucide-react
    1. Copy the component source code into your project (e.g., components/ui/audio-player.tsx).
    2. Update the import paths to match your project structure.