repository·main·Indexed 18 days ago
https://github.com/hdrhistogram/hdrhistogram_rustA Rust port of Gil Tene's HdrHistogram designed for high-performance recording and analysis of sampled data, such as latency, across large configurable value ranges with configurable precision. Version 7.6.0 provides tools for creating histograms with specific bounds and significant figures, recording samples, and traversing data using various iterators including linear, logarithmic, quantile, and recorded value types.
To use HdrHistogram in your Rust project, add the following dependency to your Cargo.toml:
[dependencies]
hdrhistogram = "7"Then, in your crate root, include:
extern crate hdrhistogram;[dependencies]
hdrhistogram = "7"extern crate hdrhistogram;A SyncHistogram allows multiple threads to record samples concurrently. Instead of sharing a single histogram with a lock, each thread uses its own Recorder. The SyncHistogram acts as the central aggregator that merges these distributed samples.
Workflow:
SyncHistogram from an existing Histogram..recorder() to obtain a Recorder handle for each thread.Recorder to record values (e.g., via .record(value) or += value)..refresh() or .refresh_timeout(duration) on the SyncHistogram to block until all active Recorder instances have synchronized their data into the main histogram.// Assuming a Histogram is already initialized
let mut sync_hist = SyncHistogram::from(histogram);
// In a thread:
let mut recorder = sync_hist.recorder();
recorder.record(100).unwrap();
// In the main thread/reader:
sync_hist.refresh(); // All samples from all recorders are now in sync_histThe Iter iterator uses a specific stepping logic to ensure quantile steps are easy for humans to read. Instead of a constant increment across the entire 0.0-1.0 range, it uses a scale that changes as it approaches 100%.
ticks_per_half_distance.This ensures that even as you approach the tail of the distribution, the steps remain granular and comprehensible.
The hdrhistogram crate provides several specialized iterators for traversing histogram data depending on your needs:
quantile: Iterates over specific quantiles.linear: Iterates linearly over histogram values.log: Iterates logarithmically over histogram values.recorded: Iterates only over values that have actually been recorded.all: Iterates over all values in the histogram.These iterators yield IterationValue<T> items, which contain the value, its quantile, and the associated counts.
The Counter trait defines the necessary operations for a type to be used as a counter within a histogram. To implement or use a type as a Counter, it must support basic numeric operations, conversion to/from primitives, and saturating/checked arithmetic. This is primarily used to facilitate floating-point operations for calculating quantiles and converting counts back to integers.
Supported primitive types that implement Counter include:
u8u16u32u64Because HdrHistogram uses buckets to maintain high precision with low memory, multiple distinct input values may be mapped to the same internal representation. These are considered equivalent.
equivalent(value1: u64, value2: u64): Returns true if both values fall into the same resolution bucket.lowest_equivalent(value: u64): Returns the smallest value that is considered equivalent to the input.highest_equivalent(value: u64): Returns the largest value that is considered equivalent to the input.median_equivalent(value: u64): Returns the middle value of the range of equivalent values.equivalent_range(value: u64): Returns the size (in value units) of the range of equivalent values.next_non_equivalent(value: u64): Returns the first value that is not equivalent to the input.The library distinguishes between safe Result-returning methods and ergonomic panicking methods. Use the safe versions in production code.
| Functionality | Safe Method (Returns Result) | Panicking Method (via AddAssign/SubAssign) |
|---|---|---|
| Increment count | h.record(v) | h += v |
| Add histograms | h.add(h2) | h += h2 |
| Subtract histograms | h.subtract(h2) | h -= h2 |
Common Errors:
CreationError::LowIsZero: Provided low value was less than 1.CreationError::HighLessThanTwiceLow: Provided high was less than 2 * low.AdditionError::OtherAddendValueExceedsRange: Attempted to add a value larger than the current range when auto_resize is disabled.SubtractionError::SubtrahendCountExceedsMinuendCount: Subtraction would result in a negative count.To iterate only over the bins in a Histogram that contain at least one sample (skipping empty bins), use the iter_recorded method. This returns a HistogramIterator using the Iter implementation, which yields only non-empty bins.
// Assuming hist is a Histogram instance
for (index, count) in hist.iter_recorded() {
// index is the bin index, count is the number of samples in that bin
println!("Bin {}: {}", index, count);
}A Recorder is a write-only handle to a SyncHistogram. It provides wait-free recording for high-performance concurrent environments. Writes are scalable except during a phase shift (when the SyncHistogram initiates a refresh).
Key Methods:
record(value: u64): Records a single value.record_n(value: u64, count: C): Records a value multiple times.add_assign(value: u64): Ergonomic syntax for recorder += value;.saturating_record(value: u64): Records a value without erroring on overflow.add(source: B): Merges another histogram into this recorder.When a Recorder is dropped, its remaining samples are automatically made visible to the next SyncHistogram::refresh() call.
let mut recorder = sync_hist.recorder();
// Direct recording
recorder.record(42).unwrap();
// Ergonomic addition
recorder += 10;
// Recording multiple counts
recorder.record_n(5, count).unwrap();The library provides several ways to iterate over the recorded data, yielding IterationValue structs:
iter_quantiles(ticks_per_half_distance: u32): Iterates by quantile levels (e.g., 0.1, 0.2...). The step size halves every ticks_per_half_distance iterations.iter_linear(step: u64): Iterates using fixed linear steps of size step.iter_log(start: u64, exp: f64): Iterates using logarithmically increasing steps starting at start with an exponential factor exp.iter_recorded(): Iterates only through the values that actually have non-zero counts.iter_all(): Iterates through all possible unit value levels supported by the histogram's resolution, regardless of whether they have recorded values.Note on IterationValue: Because of how buckets work, a value's cumulative count might reach a quantile (like 1.0) before the iterator reaches the actual value. IterationValue provides both quantile() and quantile_iterated_to() to handle this.
// Example: Iterating through recorded values
for val in hist.iter_recorded() {
println!("Value: {}, Quantile: {}", val.value_iterated_to(), val.quantile());
}The V2DeflateSerializer is used to serialize a Histogram into a compressed binary format. While named 'deflate' to maintain consistency with the Java implementation, it actually uses the zlib wrapper format around plain DEFLATE.
To use it, implement the Serializer trait. The serialize method takes a Histogram and a writer, and returns the number of bytes written.
Serialization can fail with a V2DeflateSerializeError, which wraps:
InternalSerializationError(V2SerializeError): Errors occurring during the underlying V2 serialization process.IoError(io::Error): Standard I/O errors encountered during writing.use hdrhistogram::Histogram;
use hdrhistogram::serialization::Serializer;
use hdrhistogram::serialization::v2_deflate_serializer::V2DeflateSerializer;
// Assuming 'h' is an existing Histogram and 'writer' is a type implementing std::io::Write
let mut serializer = V2DeflateSerializer::new();
let bytes_written = serializer.serialize(&h, &mut writer)?;Use Deserializer::new() to create a fresh instance. The Deserializer maintains an internal buffer (payload_buf) to reuse memory during the deserialization process, which can improve performance when deserializing multiple histograms.
use hdrhistogram::Deserializer;
let mut deserializer = Deserializer::new();