ElevenLabs JS Library

The Speech Engine allows you to build voice-powered AI agents. The ElevenLabs API connects to your server via WebSocket, where each connection represents one conversation. You provide the LLM responses, and ElevenLabs handles the speech generation.

There are two ways to host a Speech Engine server:

1. Attach to an existing HTTP server: Use elevenlabs.speechEngine.attach() to handle connections on a specific path within your existing web server (e.g., Express, Next.js).
2. Standalone server: Use SpeechEngine.Server to run a dedicated WebSocket server on a specific port.

Install the official Node.js library for ElevenLabs using npm or yarn.

npm install @elevenlabs/elevenlabs-js
# or
yarn add @elevenlabs/elevenlabs-js

When initializing or calling methods on the ToolConfigsClient, you can provide configuration at two levels:

1. ToolConfigsClient.Options: Passed to the constructor to set global client settings like apiKey, baseUrl, environment, timeoutInSeconds, and fetcher.
2. ToolConfigsClient.RequestOptions: Passed to individual method calls to override settings for a specific request, such as apiKey, headers, queryParams, timeoutInSeconds, maxRetries, or an abortSignal.

Note: The client automatically attempts to use process.env.ELEVENLABS_API_KEY if no API key is provided in the options.

By default, SpeechEngine.Server and speechEngine.attach() verify the X-Elevenlabs-Speech-Engine-Authorization header using the provided apiKey.

Disabling authentication: If you are running behind an infrastructure layer that restricts traffic to ElevenLabs (e.g., via IP allowlisting), you can skip JWT verification by setting disableAuth: true.

⚠️ Warning: Only use disableAuth: true if you have network-level restrictions in place. Otherwise, anyone can access your server and consume your quota.

// Standalone — no apiKey required when disableAuth is true
new SpeechEngine.Server({ port: 3001, disableAuth: true, onTranscript }).start();

// Or on attach
elevenlabs.speechEngine.attach("seng_123", httpServer, "/api/speech-engine/ws", {
    disableAuth: true,
    onTranscript,
});

The AudioNativeClient can be initialized with AudioNativeClient.Options. These options are used to configure the client's behavior, such as the apiKey, baseUrl, or environment.

When making requests, you can also provide AudioNativeClient.RequestOptions to override configuration for a specific call (e.g., providing a different apiKey or timeoutInSeconds).

The KnowledgeBaseClient provides access to manage and interact with knowledge base resources for Conversational AI. It acts as a gateway to three specialized sub-clients:

• documents: For managing collections of documents.
• crawlJobs: For managing web crawling tasks.
• document: For interacting with individual documents.

You can access these via the documents, crawlJobs, and document properties on an instance of KnowledgeBaseClient.

import { ElevenLabs } from "@elevenlabs/elevenlabs-js";

const client = new ElevenLabs.conversationalAi.knowledgeBase();

// Access sub-clients
const docsClient = client.documents;
const crawlClient = client.crawlJobs;
const singleDocClient = client.document;

When interacting with the SettingsClient, you can provide configuration at two levels:

1. Client Options: Set during the instantiation of the SettingsClient. These apply to all requests made by this client instance (e.g., apiKey, baseUrl, environment, timeoutInSeconds).
2. Request Options: Passed directly to get() or update() methods. These allow for per-request overrides of apiKey, headers, queryParams, timeoutInSeconds, maxRetries, and abortSignal.

The simulateConversation and simulateConversationStream methods are deprecated.

To create and run simulations, use the /v1/convai/agent-testing/create and /v1/convai/agents/:agent_id/run-tests endpoints instead.

// Deprecated usage example
await client.conversationalAi.agents.simulateConversation("agent_3701k3ttaq12ewp8b7qv5rfyszkz", {
    simulationSpecification: {
        simulatedUserConfig: {
            firstMessage: "Hello, how can I help you today?",
            language: "en",
            disableFirstMessageInterruptions: false
        }
    }
});

Use elevenlabs.textToSpeech.convert() to generate audio from text. You must provide a voiceId and an options object containing text and a modelId.

Note: To use the play() utility for local audio playback, you must have MPV and ffmpeg installed on your system.

import { ElevenLabsClient, play } from "@elevenlabs/elevenlabs-js";

const elevenlabs = new ElevenLabsClient({
    apiKey: "YOUR_API_KEY", // Defaults to process.env.ELEVENLABS_API_KEY
});

const audio = await elevenlabs.textToSpeech.convert("Xb7hH8MSUJpSbSDYk0k2", {
    text: "Hello! 你好! Hola! नमस्ते! Bonjour! こんにちは! مرحبا! 안녕하세요! Ciao! Cześć! Привіт! வணக்கம்!",
    modelId: "eleven_multilingual_v2",
});

await play(audio);

Use client.conversationalAi.agents.procedures.remove to remove a procedure from the agent's current draft working set on a specific branch.

Parameters:

• agent_id (string): The ID of the agent.
• branch_id (string): The ID of the branch.
• procedure_id (string): The ID of the procedure to remove.

await client.conversationalAi.agents.procedures.remove("agent_3701k3ttaq12ewp8b7qv5rfyszkz", "agtbranch_0901k4aafjxxfxt93gd841r7tv5t", "agtprc_6qbpwdq8n01bxhk44bgjy6f10ck3");

Use client.textToVoice.create() to save a voice permanently. This method must be called after obtaining a generated_voice_id from either client.textToVoice.design() or client.textToVoice.remix().

await client.textToVoice.create({
    voiceName: "Sassy squeaky mouse",
    voiceDescription: "A sassy squeaky mouse",
    generatedVoiceId: "37HceQefKmEi3bGovXjL"
});