Claude Code ROI Measurement and Monitoring Guide
repository·main·Indexed 43 days ago
https://github.com/anthropics/claude-code-monitoring-guideA framework for measuring the return on investment (ROI) of Claude Code. Includes instructions for setting up OpenTelemetry and Prometheus to track cost metrics, token usage, and developer productivity metrics such as PR count and commit frequency. Covers pricing model comparisons, telemetry verification, and integration with Linear via Model Context Protocol (MCP) for automated productivity reporting.
What's inside Claude Code Monitoring Guide
- The Claude Code ROI Measurement Guide provides a framework for measuring the return on investment (ROI) of Claude Code within a development organization. It includes tools and methodologies for setting up telemetry, analyzing costs, tracking developer productivity, and calculating overall ROI. This is applicable for both individual developers and engineering managers.
Understand Claude Code Productivity Report Metrics
mainThe Claude Code Productivity Report provides a multi-dimensional view of how the tool is being used and its impact on development. The report typically includes the following metric categories:
Usage Metrics
Tracks how often specific tools are used and their effectiveness:
- Tool Usage Distribution: Frequency of tools like
Edit,MultiEdit,Write,Read, andBash. - Tool Acceptance Rate: The percentage of tool suggestions accepted by the user (e.g.,
Edit,MultiEdit,Write). High acceptance rates (e.g., >75%) are correlated with faster time-to-merge. - Session Metrics: Total
Session Count,Average Session Duration, and distribution of session lengths.
Linear Integration Metrics
Analyzes productivity through the lens of your issue tracking system (via Linear MCP):
- Issue Completion: Tracks
Issues Completed,Issues Created,Development Issues in Progress, and theCompletion Ratio. - Active Development Tickets: Categorizes current work by priority (e.g., Urgent, High, Low).
- Team Velocity: Compares implementation time (days) before and after adopting Claude Code.
- Productivity Comparison: Benchmarks metrics like
Commits/Week,Review Cycles,Merge Time, andPR Size(LOC).
Cost Analysis
Provides visibility into the financial impact of using Claude Code:
- Token Usage: Breakdown of
Input TokensvsOutput Tokens. - Total Cost: The aggregate cost for the reporting period.
- Cost per Issue: The average spend per Linear issue.
- Tool Usage Distribution: Frequency of tools like
Key metrics for Claude Code ROI tracking
mainTo measure the impact of Claude Code, the guide tracks several categories of metrics:
- Cost Metrics: Total spend, cost per session, and cost broken down by model.
- Token Usage: Input/output tokens and cache efficiency.
- Productivity: PR (Pull Request) count, commit frequency, and session duration.
- Team Analytics: Usage patterns by individual developer and overall adoption rates.
Getting started with Claude Code ROI measurement
mainTo begin implementing the measurement framework, refer to the complete implementation guide inclaude_code_roi_full.md. This guide contains the detailed setup instructions, real-world examples, and actionable insights required for your organization.Generate a Claude Code ROI Report
mainYou can generate a comprehensive productivity report by passing a structured prompt to the
claudeCLI. This prompt should include a JSON object containing Claude Code metrics and instructions to use the Linear MCP to correlate these metrics with team velocity and issue completion.Required Metrics JSON Structure: To get accurate results, provide a JSON object containing keys such as:
claude_code_sessions: Total number of sessions.total_cost: Total expenditure.pull_requests: Number of PRs generated.avg_session_duration: Average time per session.top_users: List of active users.cost_per_pr: Cost efficiency metric.token_usage: Object containinginput_tokens,output_tokens,input_cost, andoutput_cost.tool_usage: Counts for tools likeEdit,MultiEdit,Write,Read, andBash.tool_acceptance_rates: Success rates for specific tools (e.g.,Edit,MultiEdit).
Example Command:
claude -p "Using the Linear MCP, analyze our team's velocity for the last sprint and combine with these Claude Code metrics: { "claude_code_sessions": 42, "total_cost": 103.45, "pull_requests": 23, "avg_session_duration": "28.5 minutes", "top_users": ["alice@company.com", "bob@company.com"], "cost_per_pr": 4.50, "token_usage": { "input_tokens": 3245670, "output_tokens": 2156780, "input_cost": 32.46, "output_cost": 70.99 }, "tool_usage": { "Edit": 356, "MultiEdit": 128, "Write": 73, "Read": 892, "Bash": 204 }, "tool_acceptance_rates": { "Edit": 0.81, "MultiEdit": 0.92, "Write": 0.65 } } Generate a comprehensive productivity report that includes: 1. Executive summary with velocity improvements 2. Usage patterns and engagement metrics 3. Linear issue completion metrics (use actual Linear data) 4. Cost analysis with visualizations 5. Actionable insights based on tool usage patterns 6. Productivity comparison (before/after Claude Code) 7. Recommendations for optimization Use Mermaid diagrams for visualizations. Reference specific Linear ticket IDs where relevant."Verify Claude Code telemetry with console output
mainBefore setting up a full Prometheus infrastructure, you can verify that Claude Code is correctly emitting OpenTelemetry metrics by directing them to the console. This is useful for debugging and ensuring your environment variables are correctly configured.
Set
CLAUDE_CODE_ENABLE_TELEMETRYto1, set theOTEL_METRICS_EXPORTERtoconsole, and provide a shortOTEL_METRIC_EXPORT_INTERVALto see immediate results.export CLAUDE_CODE_ENABLE_TELEMETRY=1 export OTEL_METRICS_EXPORTER=console export OTEL_METRIC_EXPORT_INTERVAL=1000 claude -p "hello world"Set up Prometheus for Claude Code ROI measurement
mainTo move beyond console debugging and achieve historical data and dashboards, you must configure Claude Code to export metrics via OTLP to a Prometheus instance.
1. Prepare Infrastructure
Clone the configuration repository and start the services using Docker Compose:
git clone https://github.com/katchu11/claude-code-guide docker-compose up -d2. Configure Claude Code Environment
Set the following environment variables to enable telemetry and point the OTLP exporter to your Prometheus/Collector endpoint:
CLAUDE_CODE_ENABLE_TELEMETRY=1: Enables telemetry.OTEL_METRICS_EXPORTER=otlp: Sets the exporter to OTLP.OTEL_EXPORTER_OTLP_PROTOCOL=grpc: Sets the protocol to gRPC.OTEL_EXPORTER_OTLP_ENDPOINT: The URL of your collector (e.g.,http://localhost:4317).OTEL_EXPORTER_OTLP_HEADERS: (Optional) Used for authentication, e.g.,Authorization=Bearer your-token.
Common Endpoints
- Local:
http://localhost:4317 - Kubernetes:
http://otel-collector.monitoring.svc.cluster.local:4317 - DataDog:
https://otlp.datadoghq.com - Honeycomb:
https://api.honeycomb.io:443
# Enable telemetry export CLAUDE_CODE_ENABLE_TELEMETRY=1 # Configure OTLP exporter export OTEL_METRICS_EXPORTER=otlp export OTEL_EXPORTER_OTLP_PROTOCOL=grpc export OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4317 # Optional: Set authentication if required # export OTEL_EXPORTER_OTLP_HEADERS="Authorization=Bearer your-token" # Run Claude Code claudeGenerate a Claude Code Productivity Report
mainTo generate a productivity report similar to the sample, you must use a specific prompt template. This report aggregates data from Claude Code telemetry metrics and your Linear MCP integration to provide insights on usage, cost, and team velocity.
Steps to generate:
- Ensure you have the Linear MCP integration configured to allow Claude Code to access issue data.
- Use the prompt template located in
report-generation-prompt.mdto instruct Claude Code to analyze your metrics and generate the report.
# Use the prompt template in report-generation-prompt.md to generate your own report.Set up Linear MCP integration for Claude Code
mainYou can integrate Linear with Claude Code using the Model Context Protocol (MCP) to allow Claude to analyze project management data alongside telemetry.
Using the CLI
Run the following command to add the Linear MCP server:
claude mcp add linear -s user -- npx -y mcp-remote https://mcp.linear.app/sseAfter adding, verify the installation with
claude mcp listand restart Claude Code to activate the integration.Using .claude.json
Alternatively, manually configure the server in your
.claude.jsonfile:{ "mcpServers": { "linear": { "type": "stdio", "command": "npx", "args": ["-y", "mcp-remote", "https://mcp.linear.app/sse"] } } }# Add Linear MCP server to Claude Code claude mcp add linear -s user -- npx -y mcp-remote https://mcp.linear.app/sse # Verify it's working claude mcp listManage High Memory Usage and Performance
mainTo prevent Claude Code from consuming excessive resources in large codebases:
- Reduce Context: Use the
/compactcommand regularly. - Lifecycle Management: Close and restart the tool between major tasks.
- Ignore Build Artifacts: Ensure large directories are in your
.gitignoreto prevent them from being indexed/processed.
# Add to .gitignore node_modules/ dist/ build/ .next/- Reduce Context: Use the
Fix Permission Errors on Linux during npm installation
mainIf
npm install -gfails due to permission errors, configure a user-writable npm prefix instead of using sudo. This also resolves auto-update failures.# Save existing global packages npm list -g --depth=0 > ~/npm-global-packages.txt # Create user directory for npm mkdir -p ~/.npm-global npm config set prefix ~/.npm-global # Add to PATH (use ~/.zshrc for zsh) echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrc source ~/.bashrc # Install Claude Code npm install -g @anthropic-ai/claude-codeConfigure Linear MCP for Claude Code
mainTo generate productivity reports that combine Claude Code telemetry with Linear issue data, you must first configure the Linear MCP (Model Context Protocol) integration in Claude Code.
Run the following command to add the Linear MCP using the remote SSE endpoint:
claude mcp add linear -s user -- npx -y mcp-remote https://mcp.linear.app/sse