HdrHistogram Python

For most users, the preferred method is a binary installation via PyPI. This installs prebuilt wheels for Linux, macOS, and Windows, supporting CPython 3.10 through 3.14. If no matching wheel is available, pip will attempt to build from source, which requires a C compiler for the low-level encoding/decoding C extension.

pip install hdrhistogram

To profile the performance of encoding and decoding, you must clone the GitHub repository and run pytest with the --runperf flag. The profiling functions will execute the encoding or decoding of a typical histogram 1000 times and provide a detailed breakdown of function calls and execution time.

To profile encoding performance:

pytest -s -k test_cod_perf --runperf

To profile decoding performance:

pytest -s -k test_dec_perf --runperf

# To profile encoding
pytest -s -k test_cod_perf --runperf

# To profile decoding
pytest -s -k test_dec_perf --runperf

If you are developing the library or want to run the test suite, install tox and clone the repository to build from source.

1. Install tox via pip.
2. Clone the repository.
3. Run tox to execute the unit test suite across all supported Python versions (3.10-3.14) and run linting.

pip install tox
git clone https://github.com/HdrHistogram/HdrHistogram_py.git
cd HdrHistogram_py
tox

When using hdrhistogram_py, be aware of the following limitations compared to the original Java/C implementations:

Missing Features

The following APIs/features from the original HDR Histogram library are not implemented in this Python port:

• Concurrency support (AtomicHistogram, ConcurrentHistogram, etc.)
• DoubleHistogram
• Histogram auto-resize
• recorder function

Platform and Runtime Issues

• PyPy: There are byte endianness encoding issues when using PyPy due to PyPy-specific limitations.
• Windows 32-bit: The library may crash during the decoding process on 32-bit Windows systems.

Create a histogram using the HdrHistogram class. You must specify the minimum trackable value, the maximum trackable value, and the number of precision digits desired.

Example: Creating a histogram for latencies in the range [1ms to 1 hour] with 2 digits of precision (1% precision).

from hdrhistogram import HdrHistogram

# range [1..3600000], 2 digits of precision
histogram = HdrHistogram(1, 60 * 60 * 1000, 2)

A 'histoblob' is a base64 encoded/compressed representation of a histogram. This allows for lossless serialization that can be stored in JSON, XML, or CSV.

• Use encode() to generate a histoblob string.
• Use HdrHistogram.decode(histoblob) to recreate a histogram instance.
• Use decode_and_add(histoblob) to aggregate a histoblob into an existing histogram instance (useful for distributed aggregation).

# Serialize
histoblob = histogram.encode()

# Deserialize into a new instance
decoded_histogram = HdrHistogram.decode(histoblob)

# Aggregate into an existing instance
aggregation_histogram.decode_and_add(histoblob)

The library provides several methods to retrieve statistical properties of the recorded data:

• get_total_count(): Returns the total number of values recorded.
• get_value_at_percentile(percentile): Returns the value at the specified percentile (e.g., 99.9).
• get_min_value(): Returns the minimum value recorded.
• get_max_value(): Returns the maximum value recorded.
• get_mean(): Returns the mean value.
• get_standard_deviation(): Returns the standard deviation.

count = histogram.get_total_count()
value = histogram.get_value_at_percentile(99.9)

Use record_value(value) to add a single observation to the histogram. If your data generation is subject to Coordinated Omission, use record_corrected_value(value, expected_interval) to adjust the recording based on the expected interval between observations.

# Standard recording
histogram.record_value(latency)

# Recording with correction for Coordinated Omission (e.g., 10ms expected interval)
histogram.record_corrected_value(latency, 10)

To output the histogram in a tabular format suitable for plotting, use output_percentile_distribution(file, scaling_ratio).

# file: destination file, scaling_ratio: ratio for the output
histogram.output_percentile_distribution(file, scaling_ratio)

You can iterate over the recorded values using get_recorded_iterator(). Each item in the iterator provides the value, the count added in that step, and the percentile.

for item in histogram.get_recorded_iterator():
    print('value=%f count=%d percentile=%f' % (
        item.value_iterated_to,
        item.count_added_in_this_iter_step,
        item.percentile
    ))