gobenchdata

To view your continuous benchmark data, you can use the built-in web server or generate a static web application.

Local Preview

1. Install the CLI: go install go.bobheadxi.dev/gobenchdata@latest.
2. Generate the web config: gobenchdata web generate --web.config-only ..
3. Start the server: gobenchdata web serve (this opens the visualization in your browser).

Generate Static Web App

To publish a static version of the dashboard (e.g., to GitHub Pages), run: gobenchdata web generate ./app

The web application is configured via gobenchdata-web.yml, where you define chartGroups using regular expressions to match packages and benchmark names.

You can use gobenchdata to fail Pull Requests if performance regressions are detected.

1. Install the CLI: go install go.bobheadxi.dev/gobenchdata@latest.
2. Generate a configuration file: gobenchdata checks generate.
3. Edit the generated gobenchdata-checks.yml to define your thresholds and diff functions.
4. Enable checks in your workflow by setting CHECKS: true.

Example gobenchdata-checks.yml configuration:

checks:
- name: My Check
  description: |
    Define a check here - in this example, we caculate % difference for NsPerOp in the diff function.
    diff is a function where you receive two parameters, current and base, and in general this function
    should return a negative value for an improvement and a positive value for a regression.
  package: .
  benchmarks: []
  diff: (current.NsPerOp - base.NsPerOp) / base.NsPerOp * 100
  thresholds:
    max: 10

The easiest way to use gobenchdata is via its official Docker container action. This method ties the benchmark execution to the Go version included in the action image.

In your GitHub workflow file (e.g., .github/workflows/push.yml), use uses: bobheadxi/gobenchdata@v1. You can configure the action using with: for inputs and env: for environment variables like GITHUB_TOKEN.

name: gobenchdata publish
on: push
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - name: gobenchdata publish
      uses: bobheadxi/gobenchdata@v1
      with:
        PRUNE_COUNT: 30
        GO_TEST_FLAGS: -cpu 1,2
        PUBLISH: true
        PUBLISH_BRANCH: gh-pages
      env:
        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}

Install the gobenchdata tool using Go. Ensure you have Go installed on your system before running the installation command.

go get -u go.bobheadxi.dev/gobenchdata

If you need to run benchmarks against a specific Go version, do not use the Docker container action. Instead, use actions/setup-go to install your desired version and then run the gobenchdata action command directly using go run.

When running via go run, you must provide configuration using environment variables prefixed with INPUT_ instead of the with: syntax.

name: gobenchdata publish
on: push
jobs:
  publish:
    runs-on: ubuntu-latest
    steps:
    - name: checkout
      uses: actions/checkout@v2
    - uses: actions/setup-go@v2
      with: { go-version: "1.18" }
    - name: gobenchdata publish
      run: go run go.bobheadxi.dev/gobenchdata@v1 action
      env:
        GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
        INPUT_PRUNE_COUNT: 30
        INPUT_GO_TEST_FLAGS: -cpu 1,2
        INPUT_PUBLISH: true
        INPUT_PUBLISH_BRANCH: gh-pages

The CLI can execute checks to ensure performance regressions do not occur. This involves two steps: generating the check configuration and then evaluating the benchmarks.

1. Generate checks: gobenchdata checks generate
2. Evaluate benchmarks: gobenchdata checks eval ${base benchmarks} ${current benchmarks}

Example workflow:

gobenchdata checks generate
gobenchdata checks eval base.json current.json

Use the checks command to evaluate differences between benchmark datasets. This process involves two steps:

1. Generate a config file: Use checks generate to create a configuration file that defines what metrics to check.
2. Evaluate benchmarks: Use checks eval to compare a base benchmark file against a current benchmark file based on the generated configuration.

gobenchdata checks generate
gobenchdata checks eval base-benchmarks.json current-benchmarks.json

You can install the gobenchdata CLI tool using go install. This tool is used to manage benchmark data, generate reports, and run performance checks.

go install go.bobheadxi.dev/gobenchdata@latest

The primary way to use gobenchdata is to pipe the output of Go benchmark tests directly into the tool. This allows you to inspect and format benchmark results.

Usage Pattern: go test -bench . -benchmem ./... | gobenchdata [flags]

After generating a JSON report, you can view it using the built-in web application by running the web serve command.

gobenchdata web serve

The easiest way to use the CLI is by piping the output of go test -bench directly into gobenchdata. It will consume the benchmark output and generate a JSON report file.

go test -bench . -benchmem ./... | gobenchdata --json bench.json