repository·main·Indexed 23 days ago
https://github.com/langwatch/better-agentsA CLI tool and set of standards for building production-ready AI agent projects using LangWatch best practices. It enforces a standardized project structure featuring versioned prompts, scenario-based testing, observability instrumentation, and Model Context Protocol (MCP) configurations to optimize coding assistants like Cursor and Claude Code.
Better Agents integrates with LangWatch to provide the following capabilities for agent development:
The Tool Setup module is responsible for configuring external tools and services for an agent project. Its primary tasks include:
A standard Better Agent project is organized to separate agent logic, testing, and prompt management. Follow this directory structure to ensure compatibility with development guidelines and AI coding assistants:
app/ (or src/): The core agent implementation code.tests/evaluations/: Jupyter notebooks containing datasets and notebooks for evaluating specific pipeline pieces (like RAG or classification).tests/scenarios/: End-to-end scenario tests (e.g., .py or .ts files) that simulate agent conversations.prompts/: Versioned prompt files stored in .yaml format.prompts.json: The prompt registry that syncs and controls the versioned prompts..mcp.json: MCP server configuration that enables AI coding assistants to understand your framework and write Scenario tests..cursor/mcp.json: A symlink to .mcp.json specifically for Cursor users.AGENTS.md: Development guidelines for the project.CLAUDE.md: Configuration for Claude Code, which references AGENTS.md.my-agent-project/
├── app/ (or src/) # The actual agent code, structured according to the chosen framework
├── tests/
│ ├── evaluations/ # Jupyter notebooks for evaluations
│ │ └── example_eval.ipynb
│ └── scenarios/ # End-to-end scenario tests
│ └── example_scenario.test.{py,ts}
├── prompts/
│ └── sample_prompt.yaml
├── prompts.json # Prompt registry
├── .mcp.json # MCP server configuration (universal)
├── .cursor/mcp.json # Symlink to .mcp.json for Cursor
├── AGENTS.md # Development guidelines
├── CLAUDE.md # References AGENTS.md for Claude Code
├── .env # Environment variables
└── .gitignoreThe Better Agent structure is designed for production-ready agents by enforcing best practices for testing, evaluation, and observability. A standard project includes:
app/ or src/: The core agent implementation.tests/scenarios/: End-to-end tests simulating agent conversations.tests/evaluations/: Jupyter notebooks for measuring specific pipeline performance (e.g., RAG or classification).prompts/: Versioned YAML prompt files.prompts.json: The prompt registry used for team collaboration and playground integration..mcp.json: Configuration for Model Context Protocol (MCP) servers to enhance AI coding assistants.AGENTS.md & CLAUDE.md: Development guidelines and AI coding editor compatibility files.my-agent-project/
├── app/ (or src/) # The actual agent code
├── tests/
│ ├── evaluations/ # Jupyter notebooks for evaluations
│ │ └── example_eval.ipynb
│ └── scenarios/ # End-to-end scenario tests
│ └── example_scenario.test.{py,ts}
├── prompts/ # Versioned prompt files
│ └── sample_prompt.yaml
├── prompts.json # Prompt registry
├── .mcp.json # MCP server configuration
├── .cursor/
│ └── mcp.json # Symlink to ../.mcp.json
├── AGENTS.md # Development guidelines
├── CLAUDE.md # Claude Code compatibility
├── .env # Environment variables
└── .gitignoreBetter Agents follows the Agent Testing Pyramid to ensure reliability in AI agent development. The approach consists of three layers:
scenarios to simulate full conversations and ensure the agent behaves as expected.To make your coding assistant (like Claude Code or Cursor) an expert in your agent's framework and provide it with additional tools, use the .mcp.json file. This file contains the Model Context Protocol (MCP) server configuration.
For Cursor IDE compatibility, ensure there is a symlink at .cursor/mcp.json pointing to the root .mcp.json file.
Builders are the core components responsible for creating the artifacts that constitute a Better Agents project. They work in conjunction with providers to generate configurations that are specific to the chosen framework or AI coding editor.
Key responsibilities include:
AGENTS.md).The scaffolding system is organized into two main parts:
create-directory-structure.ts: Contains the core logic for creating the directory hierarchy.file-generators/: A collection of individual template generators responsible for creating specific files (e.g., .env, main entry, etc.).Instead of hardcoding strings, store prompts as versioned YAML files within the prompts/ directory. These files are managed via a prompts.json registry. This approach enables:
Better Agents enforces a three-tier testing methodology to ensure agents are production-ready:
After the project setup is complete, Better Agents automatically launches your selected coding assistant. The launch includes a customized initial prompt containing:
The CLI detects which coding assistants are installed on your system and prioritizes them in the selection menu. If an assistant is not installed, it will be marked as (not installed) but can still be selected. You can also select None - I will prompt it myself to handle the assistant manually.
The Better Agent structure enforces industry best practices for production-ready agents by organizing code, tests, prompts, and configuration into a standardized layout:
app/ or src/: The core agent implementation based on your chosen framework.tests/evaluations/: Jupyter notebooks for evaluating specific pipeline components like RAG or classification.tests/scenarios/: End-to-end scenario tests (using scenario) that simulate conversations to ensure expected behavior.prompts/: Versioned prompt files in YAML format.prompts.json: The registry that controls and syncs the versioned prompts..mcp.json: Model Context Protocol configuration that enables your coding assistant to become an expert in your framework and testing.AGENTS.md: Development guidelines for the project..env: Environment variables for API keys and configuration.