hdrhistogram-go

To update the implementation to the latest version, use the -u flag with go get.

go get -u github.com/HdrHistogram/hdrhistogram-go

Use go get to retrieve the implementation and add it to your GOPATH workspace or your project's Go module dependencies.

go get github.com/HdrHistogram/hdrhistogram-go

When using Go modules, you can specify a particular release version using the @<tag> syntax, or use @latest to pull the latest changes from the master branch.

# Install a specific version
go get github.com/HdrHistogram/hdrhistogram-go@v0.9.0

# Install the latest master branch changes
go get github.com/HdrHistogram/hdrhistogram-go@latest

The HistogramLogReader expects a specific log format consisting of metadata lines and data lines:

1. Metadata Lines: Lines starting with # used for configuration.
   • #[StartTime: <seconds_since_epoch>] sets the start time.
   • #[BaseTime: <seconds_since_epoch>] sets the base time.
2. Tag Lines: Optional lines starting with Tag= used to assign a string label to the subsequent histogram.
   • Example: Tag=A,0.127,1.007,2.769,HIST...
3. Data Lines: Comma-separated values representing the interval.
   • Format: startTimestamp, intervalLength, maxTime, histogramPayload
   • The histogramPayload is a byte sequence that is decoded into a Histogram object using the Decode function.

The HdrHistogram V2 format is designed for high compactness, allowing a typical histogram to fit within a single MTU-sized packet (~1500 bytes).

Key Mechanisms:

• Modified ZigZag LEB128: Uses a modified encoding scheme for counts.
• Zero-Run Compression: Consecutive zero counters are represented as a negative integer indicating the run length of zeros. Non-zero counters are represented as positive integers. This significantly reduces the size of sparse histograms.
• V2 Compressed Format: The implementation uses zlib compression on the encoded counts and wraps the result in a Base64-encoded string for easy transmission or storage.

A WindowedHistogram combines multiple Histogram instances to provide statistics over a sliding window of time or events. It manages a set of underlying histograms and rotates through them, allowing you to maintain a rolling view of data by resetting the oldest histogram in the cycle.

To use it, initialize it with NewWindowed, record data into the Current histogram, and call Rotate() periodically to move to the next window. To get the aggregate statistics across all windows, call Merge().

// Create a windowed histogram with 3 underlying histograms
wh := hdrhistogram.NewWindowed(3, 1, 1000000, 3)

// Record data into the current window
wh.Current.RecordValue(100)

// Rotate to the next window (resets the oldest one)
wh.Rotate()

// Get a single histogram representing the merged data of all windows
merged := wh.Merge()

To write histogram data to a log file, use NewHistogramLogWriter by providing an io.Writer (such as an os.File or os.Stdout). The writer supports logging multiple histograms, metadata like start/base times, and comments into a single file using the HISTOGRAM_LOG_FORMAT_VERSION (currently 1.3).

package main

import (
	"os"
	"hdrhistogram"
)

func main() {
	// Create a writer that outputs to stdout
	writer := hdrhistogram.NewHistogramLogWriter(os.Stdout)
	_ = writer
}

The repository was transferred from github.com/codahale/hdrhistogram to github.com/HdrHistogram/hdrhistogram-go. To prevent breaking changes in applications that depend on the old URL, you can use go mod edit -replace to point the old dependency to the new repository. Using the @v0.9.0 tag ensures you are using the exact code that was frozen at the time of the transfer.

go mod edit -replace github.com/codahale/hdrhistogram=github.com/HdrHistogram/hdrhistogram-go@v0.9.0

When using OutputIntervalHistogramWithLogOptions, you can provide a HistogramLogOptions struct to override default logging behavior. This is useful if you need to specify custom timestamps or change how the maximum value is scaled.

Fields in HistogramLogOptions:

• startTimeStampSec: Overrides the histogram's start timestamp (in seconds).
• endTimeStampSec: Overrides the histogram's end timestamp (in seconds).
• maxValueUnitRatio: The ratio used to scale the maximum value. The default is MsToNsRatio (1,000,000.0), which scales milliseconds to nanoseconds.

options := &hdrhistogram.HistogramLogOptions{
	startTimeStampSec: 1625097600.0,
	endTimeStampSec:   1625097660.0,
	maxValueUnitRatio: 1.0,
}

// Use these options when outputting the histogram
err := writer.OutputIntervalHistogramWithLogOptions(myHistogram, options)

The HistogramLogWriter provides several methods to add metadata and structure to the log file. These lines are often used for human readability or to signal the start of a log session.

• OutputLogFormatVersion(): Writes the current log format version (e.g., [Histogram log format version 1.3]).
• OutputStartTime(msec int64): Logs a start time in seconds since epoch (with 3 decimal places) and an ISO-8601 UTC timestamp.
• OutputBaseTime(msec int64): Logs a base time in seconds since epoch.
• OutputLegend(): Writes human-readable column headers: "StartTimestamp","Interval_Length","Interval_Max","Interval_Compressed_Histogram".
• OutputComment(comment string): Writes a custom comment line starting with #.

Use NewHistogramLogReader to create a reader for consuming histogram log files. It accepts any io.Reader (such as an open file or a network stream) and parses the log format containing metadata (StartTime, BaseTime) and interval-based histogram data.

import "github.com/hdrhistogram/hdrhistogram-go"

// Assuming 'file' is an *os.File containing histogram logs
reader := hdrhistogram.NewHistogramLogReader(file)