DOCSEARCHbyAlgolia
Submit Docs

Agent CLI

repository·main·Indexed 42 days ago

https://github.com/basnijholt/agent-cli

A suite of local-first, AI-powered command-line agents for text correction, audio transcription, and voice assistance. It features tools for autocorrection, voice-driven clipboard editing, and a wake word-based assistant, supporting local LLM, ASR, and TTS services like Ollama and Whisper. Includes a native macOS menu bar app with global hotkeys and a Claude Code plugin for parallel agent execution in git worktrees.

Tokens
90.1K
Snippets
304
Records
436
Agent score
50%

What's inside Agent CLI

  1. Overview of Agent CLI features

    main

    Agent CLI is a collection of local-first, AI-powered command-line agents. Key features include:

    • autocorrect: Correct grammar and spelling using a local LLM.
    • transcribe: Transcribe microphone audio to the clipboard.
    • speak: Text-to-speech using a local TTS engine.
    • voice-edit: Edit clipboard text using voice commands.
    • assistant: Wake word-based voice assistant.
    • chat: Conversational AI with tool-calling.
    • memory: Long-term memory system (memory proxy, memory add).
    • rag-proxy: RAG proxy server for chatting with documents.
    • dev: AI coding agents and git worktree management.
    • server: Local ASR and TTS servers (supports Whisper via MLX/Faster Whisper and TTS via Kokoro/Piper).
    • transcribe-live: Continuous background transcription with VAD (requires uv tool install "agent-cli[vad]").

  2. Overview of agent-cli Docker services

    main

    The Docker deployment provides a suite of microservices for an agentic workflow:

    • whisper: Speech-to-text using Faster Whisper.
    • tts: Text-to-speech using Kokoro or Piper.
    • diarization: (Optional) Speaker timing and labeled transcripts using local pyannote inference.
    • transcribe-proxy: ASR proxy for iOS or external applications.
    • rag-proxy: Document-aware chat (Retrieval-Augmented Generation).
    • memory-proxy: Long-term memory chat.
    • ollama: LLM server.
    • openwakeword: Wake word detection.

  3. Manage agent-cli servers as background daemons

    main

    The agent-cli daemon command allows you to install, uninstall, and monitor agent-cli servers as background system services. These services run via uv tool run and are configured to start automatically at login.

    Platform Support:

    • macOS: Uses launchd services located in ~/Library/LaunchAgents/.
    • Linux: Uses systemd user services located in ~/.config/systemd/user/.

    Available Daemons:

    DaemonDescriptionPorts
    whisperSpeech-to-text ASR server10300/10301
    tts-kokoroText-to-speech with Kokoro (GPU)10200/10201
    tts-piperText-to-speech with Piper (CPU)10200/10201
    transcription-proxyASR provider proxy61337
    memoryLong-term memory proxy for LLMs8100
    ragDocument retrieval proxy for LLMs8000
    agent-cli daemon [COMMAND] [OPTIONS]

  4. Manage persistent speaker identities with `speakers` command

    main

    The speakers command allows you to manage persistent diarization speaker identities. Speaker profiles are stored voice embeddings created by transcribe --diarize or diarize-live-session when using the --remember-unknown-speakers or --enroll-speakers flags.

    Key capabilities include:

    • Listing profiles: View stable profile IDs (e.g., UNKNOWN_001). Note that Pyannote labels like SPEAKER_00 are local to a single run and are not stable, whereas UNKNOWN_xxx IDs are persistent.
    • Renaming: Assign human names to unknown profiles.
    • Merging: Combine duplicate profiles for the same person.
    • Reviewing: Interactively listen to audio snippets to decide if unknown voices should be named or merged into existing profiles.
    agent-cli speakers COMMAND [OPTIONS]

  5. What is the `dev` command?

    main
    The dev command is a parallel development environment manager that uses git worktrees to create isolated working directories. It allows you to work on multiple features simultaneously by giving each feature its own branch and directory. This enables switching between features without stashing changes, running multiple AI coding agents in parallel, and keeping your main branch clean during experimentation.

  6. Overview of installed services and ports

    main

    All installation methods set up the following core services. If you are configuring external tools to interact with the agent, use these default ports:

    Services

    • Ollama: LLM server (defaults to gemma3:4b model)
    • Wyoming Whisper: Speech-to-text (uses faster-whisper on Linux/Intel or MLX Whisper on Apple Silicon)
    • Wyoming Piper: Text-to-speech
    • Wyoming OpenWakeWord: Wake word detection

    Service Ports

    • Ollama (LLM): 11434
    • Whisper (ASR): 10300
    • Piper (TTS): 10200
    • OpenWakeWord: 10400

  7. Available Tools in Chat Mode

    main

    The chat agent can interact with your system using the following tools:

    • read_file: Read file contents.
    • execute_code: Run a single command (no shell features like pipes or redirects).
    • duckduckgo_search: Search the web via DuckDuckGo.
    • add_memory: Store information for future conversations.
    • search_memory: Search stored memories.
    • update_memory: Update existing memories.
    • list_all_memories: List all stored memories.
    • list_memory_categories: Show memory category summary.

    Note: Memory tools use a built-in JSON storage system. For vector-backed memory, use the memory command.

  8. How TTL-based memory management works

    main
    To allow voice services to run 24/7 without permanently consuming RAM or VRAM, agent-cli servers implement TTL (Time-To-Live) based memory management. Models are loaded on-demand and automatically unload after a period of inactivity (defaulting to 5 minutes). This is particularly useful for memory-constrained systems.

  9. Automatic Project Setup in Dev Environments

    main

    When creating a new dev environment using agent-cli dev new, the tool automatically detects the project type and runs the appropriate setup command to prepare the worktree. This ensures dependencies are ready before the agent starts working.

    Project TypeDetectionSetup Command
    Python (uv)pyproject.toml + uv.lockuv sync --all-extras
    Python (unidep)requirements.yaml or [tool.unidep]unidep install -e . -n <env> *
    Python (unidep monorepo)requirements.yaml in subdirsunidep install-all -e -n <env> *
    Python (poetry)pyproject.toml + poetry.lockpoetry install
    Python (pip)requirements.txtpip install -r requirements.txt
    Node.js (pnpm)pnpm-lock.yamlpnpm install
    Node.js (yarn)yarn.lockyarn install
    Node.js (npm)package-lock.jsonnpm install
    RustCargo.tomlcargo build
    Gogo.modgo mod download
    RubyGemfile or Gemfile.lockbundle install
    Pixipixi.toml or pixi.lockpixi install
    • For unidep projects, if conda-lock.yml is present, -f conda-lock.yml is also added to the command.

  10. How the Memory Ingestion Pipeline works

    main

    The memory system uses an ingestion pipeline (executed via _postprocess_after_turn) to extract and reconcile facts from conversations. The process follows these steps:

    1. Streaming vs. Non-Streaming:
      • In Streaming mode, user turns are persisted immediately, and assistant tokens are buffered until the stream completes.
      • In Non-Streaming mode, both turns are persisted sequentially after the full response is received.
    2. Fact Extraction: The system uses FACT_SYSTEM_PROMPT to extract 0–3 atomic, standalone statements from the latest user message into a JSON list of strings.
    3. Reconciliation (Search-Decide-Update): To prevent contradictions, the system performs a local search of existing role="memory" entries and uses an LLM (UPDATE_MEMORY_PROMPT) to decide whether to ADD, UPDATE, DELETE, or do NONE.
    4. Summarization: If enabled, the system uses SUMMARY_PROMPT to update a running summary stored in summaries/summary.md for the conversation.
    5. Eviction: To stay under max_entries (default 500), the system deletes the oldest facts or turns based on created_at. Summaries are exempt from eviction.
    6. Versioning: If enable_git_versioning is active, an asynchronous git commit snapshots the memory store state at the end of the pipeline.

  11. Configure Local Setup Files and Git Assets

    main

    When creating a new dev environment, agent-cli automatically performs several synchronization tasks:

    Git Submodules and LFS

    • Submodules: Automatically initialized and updated via git submodule update --init --recursive.
    • Git LFS: If .gitattributes contains filter=lfs, files are automatically pulled via git lfs pull.

    Local Environment Files

    The following files are automatically copied to the new environment:

    • .env, .env.local, .env.example, .envrc
    • AGENTS.local.md, CLAUDE.local.md

    Note: If the target branch already contains a local agent instruction file, the checked-out version is preserved.

    To skip copying environment files, use the --no-copy-env flag.

  12. Configure command-specific settings

    main

    Each agent-cli command can have its own unique settings defined in a dedicated section in config.toml. The section name matches the command name. For subcommands, use dot notation.

    Supported command sections:

    • [transcribe]
    • [voice-edit]
    • [transcribe-live]
    • [diarize-live-session]
    • [memory.proxy]
    • [chat]
    • [autocorrect]
    • [speak]
    • [assistant]