goose

Goose is a load testing framework designed for simulating complex, stateful user behaviors rather than just high-volume HTTP requests. It is particularly effective for:

• Complex User Workflows: Testing multi-step processes like checkout flows, user registration, or content management.
• API Load Testing: Validating REST APIs, GraphQL endpoints, or microservice interactions.
• Performance Regression Testing: Integrating into CI/CD pipelines to catch regressions.
• Capacity Planning: Identifying infrastructure scaling limits and bottlenecks.
• Coordinated Omission Detection: Identifying when server slowdowns affect more users than simple metrics suggest.

Key differentiators include support for Stateful Testing (maintaining sessions, cookies, and authentication), Realistic Load Patterns (simulating actual user behavior), and high Flexibility through custom logic and data-driven tests.

Goose is a load testing framework that supports the following core functionalities:

Load Testing & User Simulation

• Basic Load Testing: Creating and executing simple load tests.
• User Simulation: Spawning and managing simulated users.
• Scenarios & Transactions: Defining and executing user workflows.
• Weighted Selection: Controlling the frequency of scenarios and transactions.
• Transaction Sequencing: Ordered execution of transactions.
• Request Throttling: Limiting request rates to prevent overwhelming targets.

Protocol & Request Management

• HTTP Requests: Support for GET, POST, HEAD, DELETE, and other HTTP methods.
• Session Management: Maintaining state across requests.
• Cookie Management: Automatic cookie handling across requests.
• Header Management: Custom HTTP headers for requests.
• TLS Support: Support for rustls.
• Request Customization: Configuration for timeouts, redirects, and other options.

Metrics & Reporting

• Metrics Collection: Gathering and reporting performance data.
• Coordinated Omission Mitigation: Statistical corrections for accurate metrics.
• HTML Reports: Graphical representation of test results.
• Test Plans: Complex load patterns with controlled scaling.

When performing load tests on GraphQL endpoints, Goose provides specialized reporting that helps distinguish between different GraphQL operations even though they all use the same HTTP method and endpoint.

Key aspects of the GraphQL metrics output include:

• Named Operations: Instead of seeing generic POST requests, the report displays custom names for each GraphQL operation (e.g., get all users, create user).
• Request Grouping: All GraphQL requests are identified as POST requests to the configured GraphQL endpoint.
• Transaction Timing: Metrics show the duration of complete GraphQL transactions, which may involve multiple steps.
• Weighted Distribution: If you have configured weights for your transactions (e.g., a 3:2:1:1 ratio), the # times run and trans/s metrics will reflect this distribution.
• Status Codes: Note that GraphQL often returns an HTTP 200 status code even when the response body contains GraphQL-level errors. Goose reports the HTTP status code (e.g., [200]) in the final summary.

To control API usage costs, implement the following strategies:

• Usage Monitoring: Track token usage per PR and set daily/monthly limits.
• Optimization: Chunk large PRs, filter out generated code, and cache LLM responses.
• Budget Controls: Implement token caps per PR and prioritize reviews based on PR importance.

Implementation Pattern: Use a TokenUsageTracker to check if a review can proceed based on an estimated token count against a defined budget limit.

class TokenUsageTracker:
    def __init__(self, budget_limit):
        self.budget_limit = budget_limit
        self.current_usage = 0
        
    def can_process(self, estimated_tokens):
        return self.current_usage + estimated_tokens <= self.budget_limit
        
    def record_usage(self, prompt_tokens, completion_tokens):
        usage = prompt_tokens + completion_tokens
        self.current_usage += usage
        return self.current_usage

Goose prevents misleading performance data by implementing three specific mitigation strategies:

1. Detects Missing Requests: When Goose encounters an abnormally long response time, it calculates how many requests should have been initiated during that period based on the expected load.
2. Synthetic Request Injection: Goose injects "synthetic requests" into the dataset. These represent the requests that would have been made if the server had not frozen, ensuring the total request count reflects the true intended load.
3. Clear Reporting: Goose provides specific metrics to distinguish between actual and synthetic requests, allowing you to see the true percentage of traffic affected by a freeze.

Coordinated Omission Metrics Example:

=== COORDINATED OMISSION METRICS ===
Total CO Events: 1
Actual requests: 4  
Synthetic requests: 29 (87.9%)
Severity: 1 Critical event detected

In this example, the report accurately shows that 87.9% of expected traffic was affected by the server problem, rather than just reporting on the 4 actual requests.

When running tests with CO mitigation enabled, Goose outputs a === COORDINATED OMISSION METRICS === section. Use these metrics to assess system health:

Key Metrics

• Events per minute: The rate of CO events.
• Synthetic requests %: The percentage of total requests that were synthetic (injected to detect omission). A low percentage (e.g., < 1%) indicates the system is meeting timing expectations.
• Severity Distribution: Categorizes events into Minor, Moderate, Severe, and Critical.

Health Thresholds

Common Scenarios

• Microservice SLA: High synthetic % or frequent CO events indicate the service is struggling to meet latency requirements.
• Connection Pool Exhaustion: A sudden spike in CO events and a high synthetic % (e.g., > 9%) often indicates database connection pool exhaustion.
• CDN Issues: High moderate/severe events can indicate CDN latency or unavailability affecting user experience.