VoiceMode Documentation

skip_back acts as 'CD transport', allowing you to jump back through previously spoken utterances using the server's in-memory history buffer.

The History Buffer

• Contents: Stores the last N rendered utterances, including decoded PCM audio, text, and metadata (sample_rate, channels, timestamp, voice).
• Configuration: The buffer size is controlled by VOICEMODE_HISTORY_BUFFER_SIZE (default: 8, min: 1).
• Lifecycle: The buffer is process-wide and lives for the life of the server; it is not persisted across restarts.
• Capture Logic: Utterances are captured in full. If pause or skip_back is called mid-playback, the server continues draining the provider to ensure the full utterance is buffered.

Skip-back Semantics

• First press: Restarts the current/most-recent utterance from the beginning.
• Subsequent presses: Steps back one utterance at a time through the buffer.
• Clamping: Once the oldest record is reached, further presses continue replaying that oldest record.

Note: skip_back is a playback-layer operation. It does not trigger new STT, model calls, or agent turns.

{"command": "skip_back"}

The control channel operates as a side channel into the running VoiceMode server using a Unix domain socket (AF_UNIX).

Key Mechanics:

• Triggering: An external process writes a newline-delimited JSON command to the socket.
• Latency: The TTS playback loop polls the control state every audio chunk (~85 ms), ensuring commands like stop land in under ~200 ms.
• Transport Styles:
  • Cassette transport: Handles pause, resume, stop, and skip_forward. skip_forward acts as a universal advance: it cuts the current utterance and hands control to your next record turn.
  • CD transport: Handles skip_back. This uses a process-wide history buffer (of size defined by VOICEMODE_HISTORY_BUFFER_SIZE) to replay previously spoken audio. Replaying cached audio does not start a new agent turn.
• Security: The channel is local-only. The server performs a peer-credential check to ensure only processes with the same UID can connect to the socket.

VoiceMode uses an automatic discovery mechanism to find tools on the filesystem, applies filters, and dynamically imports them at startup. This allows for zero-configuration registration and selective loading to optimize token usage in LLM contexts (like Claude Code).

Discovery Logic

1. Regular Tools: Files located at voice_mode/tools/{tool_name}.py are loaded as {tool_name}.
2. Service Tools: Files located at voice_mode/tools/services/{service}/{tool}.py are loaded using a flattened namespace: {service}_{tool} (e.g., whisper_install).

Filtering and Exclusions

To prevent accidental loading of internal logic, the following patterns are automatically excluded:

• __init__.py
• _*.py (private modules)
• *_helpers.py (utility modules)
• types.py (type definitions)

Token Optimization

Loading all tools can consume ~25,000 tokens. By using selective loading via environment variables, you can reduce this to ~5,000 tokens (e.g., loading only converse), saving 20,000 tokens in the context window.

Voices are managed as directories within your VOICEMODE_VOICES_DIR (defaults to ~/.voicemode/voices/).

Directory Structure

To register a voice, create a directory named after the voice and include a default.wav file (5-9 seconds of clean audio).

~/.voicemode/voices/
└── <voice_name>/
    ├── default.wav        # Required: 5-9s clean reference audio
    ├── description.txt    # Optional: one-line human description
    └── persona.md         # Optional: structured character notes for LLM steering

Using multiple samples

You can store multiple WAV files in a voice directory to act as a sample bin. To activate a specific sample, create a symlink named default.wav pointing to your preferred file:

ln -sfn samantha-2024-loud.wav ~/.voicemode/voices/samantha/default.wav

Warning: A directory containing multiple WAVs but no default.wav symlink will be ignored by VoiceMode.

When a Claude Code event fires, VoiceMode searches for audio files (.mp3 or .wav) using a specific priority order, moving from most specific to least specific.

Lookup Priority Table

Numbered Variants

To add variety to repeated operations, you can place numbered files (e.g., 01.mp3, 02.mp3, ..., 99.mp3) alongside a default.mp3 in any tool directory. The receiver will randomly select one of the numbered files.

When running multiple voice agents (e.g., in different tmux panes), VoiceMode uses a "conch" mechanism to serialize speech so only one agent talks at a time. This prevents audio overlap.

Key Concepts

• The Conch: A synchronization mechanism that manages the "floor" for speaking.
• Queueing: If the conch is busy, agents can join a queue.
  • wait_for_conch: If set to true (or a duration), the agent joins the queue.
  • VOICEMODE_CONCH_MODE: Determines how queued agents are served. wait blocks until the turn is granted; callback returns immediately with a queue position.
• Fairness: The floor is handed out in FIFO (First-In-First-Out) order via a grant hint.
• Operator Overrides: Use the CLI to bypass FIFO: voicemode conch give <session> (jump the line) or voicemode conch bump (drop current holder).

Remote Agents (MCP conch tool)

Remote agents on a streamable-HTTP server use the MCP conch tool to interact with the queue.

• Required: session_id must be provided for registration and heartbeats.
• Liveness: Remote agents must send a heartbeat (approx. every 30s) to maintain their place. If the VOICEMODE_CONCH_REMOTE_TTL (default 90s) expires, the agent is pruned.
• Tool Actions:
  • status: View holder and queue.
  • callback: Recommended for joining when busy (returns position immediately).
  • wait: Block until turn (capped by VOICEMODE_CONCH_MCP_WAIT_CAP).
  • heartbeat: Refresh liveness.
  • leave: Give up place.
  • give/bump/release: Operator overrides.

# Auto-focus tmux pane when an agent starts speaking (default: false)
VOICEMODE_AUTO_FOCUS_PANE=false

# Override the default focus-hold duration if the sentinel file has no
# explicit value (default: 30 seconds)
VOICEMODE_FOCUS_HOLD_SECONDS=30

# Conch coordination (serialises speech across agents)
VOICEMODE_CONCH_ENABLED=true
VOICEMODE_CONCH_TIMEOUT=60           # Seconds to wait for the conch
VOICEMODE_CONCH_CHECK_INTERVAL=0.5   # Polling interval
VOICEMODE_CONCH_LOCK_EXPIRY=300      # Stale-lock expiry (0 disables)
VOICEMODE_CONCH_MODE=wait            # Default mode when a busy converse() queues:
                                     #   wait     = block until your turn
                                     #   callback = return now with your position
VOICEMODE_CONCH_REMOTE_TTL=90        # Heartbeat TTL (s) for a REMOTE MCP waiter
VOICEMODE_CONCH_MCP_WAIT_CAP=25      # Hard cap (s) on a blocking MCP conch wait