vibe-tools

repository·main·Indexed 26 days ago

https://github.com/eastlondoner/vibe-tools

A CLI toolkit for AI coding agents (such as Cursor, Claude Code, and Windsurf) that provides advanced capabilities including web research via Perplexity, deep repository context and reasoning via Gemini 2.0, and browser automation via Stagehand. It includes specialized commands for codebase analysis (repo), task planning (plan), browser interaction (browser), and integrations with GitHub, Linear, and YouTube.

Tokens
37.2K
Snippets
55
Records
220
Agent score
86%

What's inside vibe-tools

  1. Overview of vibe-tools

    main
    vibe-tools is a CLI designed to expand the capabilities of AI agents (like Cursor Composer, Claude Code, Windsurf, etc.) by providing them with a suite of advanced tools and an 'AI team'. It allows agents to perform tasks such as web research via Perplexity, deep codebase reasoning via Gemini 2.0, browser automation via Stagehand, and integration with GitHub, Linear, and YouTube.
  2. Overview of vibe-tools feature behavior testing

    main

    vibe-tools uses an AI-driven regression testing approach. Developers create feature behavior files (Markdown) that describe desired behaviors. AI agents then interpret these descriptions to:

    1. Determine how to test the behaviors using vibe-tools commands.
    2. Generate detailed reports.
    3. Produce simple PASS/FAIL results for CI/CD automation.
  3. Infrastructure components for telemetry

    main

    The telemetry pipeline consists of the following Cloudflare-based components:

    1. Cloudflare Worker (vibe-tools-infra): Acts as the ingestion endpoint. It handles requests at /api/pipeline using a Nuxt server route (infra/server/api/pipeline.post.ts).
    2. Cloudflare Pipeline (vibe-tools-telemetry): Receives data from the Worker and batches it based on size (max 10MB), time (max 5s), or row count (max 100 rows).
    3. Cloudflare R2 Bucket (vibe-tools-telemetry): The final destination where batched JSON data is stored.
    4. API Token (telemetry-pipeline-r2-access-token): An account-level token with Workers R2 Storage Bucket Item Write permissions used by the Pipeline to write to R2.
  4. Understand collected telemetry data

    main

    Vibe Tools collects anonymous usage data to improve the tool.

    Collected data includes:

    • command: The executed command (e.g., repo, web, plan).
    • startTime: Command start timestamp.
    • Token counts: tokenCount, promptTokens, and completionTokens.
    • AI Model details: provider (e.g., gemini, openai) and model name.
    • Plan command specifics: fileProvider, fileModel, thinkingProvider, and thinkingModel.
    • options: Sanitized command-line options (e.g., --debug).
    • error: Error type and message if a command fails.

    Data NOT collected:

    • User queries or prompts
    • File contents or code
    • Personal data
    • API keys
  5. The vibe-tools AI Team components

    main

    vibe-tools provides access to several specialized 'team members' for your agent:

    • Perplexity: Used for web search and performing deep research.
    • Gemini 2.0: Used for large whole-codebase context windows, search grounding, and reasoning.
    • Stagehand: Used for browser operations to test and debug web applications (supports Anthropic, OpenAI, Gemini, or OpenRouter models).
    • OpenRouter: Provides access to a variety of models through a unified API, primarily used for MCP commands.
  6. Install vibe-tools globally

    main

    vibe-tools is a Node package that should be installed globally. Running the installation command configures instruction files tailored to your specific development environment (e.g., Cursor, Claude Code, Windsurf, etc.).

    vibe-tools install
  7. Configure Authentication and API Keys for vibe-tools

    main

    vibe-tools requires API keys for Perplexity AI and Google Gemini. It also supports optional keys for OpenAI, Anthropic, OpenRouter, xAI, Groq, GitHub, and Linear.

    There are two primary ways to configure these keys:

    1. Interactive Setup: Run vibe-tools install and follow the command-line prompts.
    2. Manual Setup: Create a .env file in one of the following locations:
      • ~/.vibe-tools/.env (Home directory)
      • .vibe-tools.env (Project root)

    Note for MCP users: To use mcp commands, you must provide at least one of ANTHROPIC_API_KEY or OPENROUTER_API_KEY.

    CI/CD Environments: In non-interactive mode (automatically detected in CI), vibe-tools uses environment variables exclusively and will not write to the filesystem.

    PERPLEXITY_API_KEY="your-perplexity-api-key"
    GEMINI_API_KEY="your-gemini-api-key"
    OPENAI_API_KEY="your-openai-api-key"  # Optional, for Stagehand
    ANTHROPIC_API_KEY="your-anthropic-api-key" # Optional, for Stagehand and MCP
    OPENROUTER_API_KEY="your-openrouter-api-key" # Optional, for MCP
    XAI_API_KEY="your-xai-api-key" # Optional, for xAI Grok models
    GROQ_API_KEY="your-groq-api-key" # Optional, for Groq models
    GITHUB_TOKEN="your-github-token"  # Optional, for enhanced GitHub access
    LINEAR_API_KEY="your-linear-api-key" # Optional, for Linear integration
  8. Use Linear Integration to manage issues

    main

    Access Linear issues with full context including priority, status, assignees, comments, and attachments. Supports both Linear identifiers (e.g., ITE-123) and UUIDs.

    Authentication Setup:

    1. Interactive Setup (Recommended): Run vibe-tools linear connect to choose between Personal API Key or OAuth2 with PKCE.
    2. Environment Variable: Set LINEAR_API_KEY="your-linear-api-key" in your .vibe-tools.env file. Do not include a "Bearer" prefix.

    Authentication Modes:

    • Personal API Key: Best for individual use; supports local or global storage.
    • OAuth2 with PKCE: Best for organizational use; provides secure browser-based authentication and automatic token management.
    # Set up authentication (interactive prompts)
    vibe-tools linear connect
    
    # View specific issue with full details
    vibe-tools linear get-issue ITE-123
    vibe-tools linear issue ABC-456
  9. Generate documentation using Gemini 2.0

    main

    Generate comprehensive documentation for local or remote GitHub repositories. You can include external web documentation as additional context to improve accuracy.

    Key Flags:

    • --save-to=<file>: Save the generated documentation to a specific file.
    • --from-github=<user/repo>[@branch]: Specify the target GitHub repository and optional branch.
    • --with-doc=<url>: Add a web URL as additional context for the documentation generation.
    • --hint="<text>": Provide a hint to guide the documentation generation.
    • --quiet: Run without verbose output.
    # Document local repository and save to file
    vibe-tools doc --save-to=docs.md
    
    # Document remote GitHub repository
    vibe-tools doc --from-github=username/repo-name@branch
    
    # Document with additional web documentation as context
    vibe-tools doc --from-github=reactjs/react-redux --with-doc=https://redux.js.org/tutorials/fundamentals/part-5-ui-and-react --save-to=docs/REACT_REDUX.md
  10. Run vibe-tools tests using the test command

    main

    Use the pnpm dev test command to execute test scenarios defined in Markdown feature behavior files. You can run a single file, multiple files using glob patterns, or run tests in parallel.

    Common usage patterns:

    • Single file: pnpm dev test <path-to-file>
    • Glob pattern: pnpm dev test "<glob-pattern>" (ensure you use quotes to prevent shell expansion)
    • Parallel execution: Use the --parallel flag to specify the number of concurrent scenarios.
    # Run a specific test file
    pnpm dev test tests/feature-behaviors/web/web-command.md
    
    # Run multiple test files using a glob pattern
    pnpm dev test "tests/feature-behaviors/web/*.md"
    
    # Run tests in parallel
    pnpm dev test tests/feature-behaviors/test/test-command-parallel-example.md --parallel 4
  11. Write feature behavior files for testing

    main

    Feature behavior files are written in Markdown and define the test scenarios for vibe-tools commands. They must follow this structure:

    1. Feature Behavior: The name of the feature.
    2. Description: A summary of what the test verifies.
    3. Test Scenarios: A list of specific test cases, each containing:
      • Scenario: The name of the test case.
      • Task Description: Instructions for what the AI agent should do.
      • Expected Behavior: A description of the expected outcome.
      • Success Criteria: Specific, measurable criteria for a successful test.
    # Feature Behavior: Web Command
    
    ## Description
    
    This test verifies that the web command can query online information correctly.
    
    ## Test Scenarios
    
    ### Scenario 1: Basic Web Query
    
    **Task Description:**
    Use vibe-tools to search for information about climate change.
    
    **Expected Behavior:**
    
    - The AI agent should use the web command
    - Response should include information about climate change
    - The command should complete successfully
    
    **Success Criteria:**
    
    - AI agent correctly uses the vibe-tools web command
    - Response contains relevant information about climate change
    - Command completes without errors
  12. Implement Scenario-Level Isolation in executeScenario

    main

    To implement scenario-level isolation, wrap the executeScenario logic with TestEnvironmentManager. This ensures each scenario runs in its own temporary directory with its required assets copied in, preventing cross-test contamination.

    Workflow:

    1. Call TestEnvironmentManager.createTempDirectory(scenarioId).
    2. Call TestEnvironmentManager.copyAssets(scenario, tempDir) to get a modifiedTaskDescription.
    3. Initialize tools (like createCommandExecutionTool) using the tempDir as the cwd.
    4. Execute the scenario using the modifiedScenario.
    5. Use a finally block to call TestEnvironmentManager.cleanup(tempDir).
    // Inside executeScenario implementation
    const tempDir = await TestEnvironmentManager.createTempDirectory(scenarioId);
    const modifiedTaskDescription = await TestEnvironmentManager.copyAssets(scenario, tempDir);
    
    const modifiedScenario = {
      ...scenario,
      taskDescription: modifiedTaskDescription
    };
    
    try {
      const tools = [
        createCommandExecutionTool({
          debug,
          cwd: tempDir,
          scenarioId,
          appendToBuffer
        })
      ];
      // ... execution logic
    } finally {
      await TestEnvironmentManager.cleanup(tempDir);
    }