bitstring

bitstring is a Python library designed for the efficient creation and analysis of bit-level binary data.

Key capabilities include:

• Creation: Generate bitstrings from hex, octal, binary, files, formatted strings, bytes, integers, and floats with various endiannesses.
• Manipulation: Perform bit-level slicing, joining, searching, replacing, and concatenation.
• Packing/Unpacking: Powerful functions for binary packing and unpacking.
• Sequential Reading: Use the Reader class to interpret bitstrings sequentially.
• Fixed-length Arrays: Create and manipulate arrays of fixed-length bitstrings using the Array class.

A bitstring.Array is a contiguously allocated sequence of bitstrings of the same type (dtype). It is similar to Python's array.array but more flexible. You can initialize an Array with an iterable, from raw bytes, or with zeroed items. Both the dtype and the underlying BitArray data can be modified after creation.

Note on operations:

• Element-wise arithmetic/shifts: Performed on the interpreted data (e.g., +, -, *, >>). Shift operations do not work on floating-point formats.
• Bit-wise logical operations (&, |, ^): Performed on each element using a Bits object of the same length as the array elements.

from bitstring import Array

# Initialization examples
Array('>H', [1, 10, 20])          # From an iterable
Array.from_bytes('i4', data)      # From raw binary data
Array.from_zeros('u8', 100)       # Full of zeroed items

When performing operations between two Array objects, the resulting Array's dtype is determined by specific promotion rules. For comparison operators, the result is always a bool dtype. For other operations, the following rules are applied in order to select the winning dtype:

1. Floating point types win against integer types.
2. Signed integer types win against unsigned integer types.
3. Longer types win against shorter types.
4. In a tie, the first type wins.

Note: Some operations like bitwise shifts (<<, >>) require integer dtypes only.

# Example Rule 1: Floating point wins
# 'i32' + 'f16' -> 'f16'

# Example Rule 2: Signed wins
# 'u20' // 'i10' -> 'i10'

# Example Rule 3: Longer wins
# 'i8' * 'i16' -> 'i16'

The bitstring module is built around four primary classes:

• Bits: An immutable container of bits.
• BitArray: A mutable container that adds mutating methods to Bits.
• Reader: A wrapper for Bits or BitArray that maintains a bit position, allowing for sequential reading, searching, and navigation (similar to a file stream).
• Array: An efficient array of bitstrings that all share the same fixed-length format.

There are two ways to compare Arrays, which behave differently:

1. Array.equals(other): Performs a structural equality test. Returns True only if the dtypes are equivalent AND the underlying bit data is identical.
2. == operator: Performs an element-wise equality check. This returns a new Array of dtype 'bool' containing the results of the comparison for each element.

To compare only the values (ignoring dtype/structure), convert both to lists first: a.to_list() == b.to_list().

>>> a = Array('u8', [1, 2, 3, 2, 1])
>>> b = Array('i8', [1, 2, 3, 2, 1])

# Structural equality (False because dtypes differ)
>>> a.equals(b)
False

# Element-wise comparison (Returns a boolean Array)
>>> a == b
Array('bool', [True, True, True, True, True])

# Value-only comparison
>>> a.to_list() == b.to_list()
True

Bitstring implements several specialized float formats in Python (via fp8.py and mxfp.py). These formats currently do not benefit from the Array bulk path and instead use a per-element Python loop.

Supported formats include:

• bfloat, bfloatbe, bfloatle
• e2m1mxfp, e2m3mxfp, e3m2mxfp, e4m3mxfp_saturate, e4m3mxfp_overflow, e5m2mxfp_saturate, e5m2mxfp_overflow, e8m0mxfp
• mxint, p3binary, p4binary

Because these use a per-element loop, they are roughly 20x slower than bulk-capable dtypes. If you require high-performance bulk operations on these types, they are currently a performance bottleneck.

The bitstring module provides two primary bit container classes. The choice between them depends on whether you need to modify the data after creation:

• Bits: An efficient, immutable container. Once created, its value cannot be changed.
• BitArray: A mutable container. It behaves like Bits but allows for in-place modifications (appending, inserting, deleting, etc.).

Both classes can be wrapped in a Reader for sequential reading.

Interleaved codes are a variation of exponential-Golomb codes used in standards like Dirac. They are accessed via specific attributes and format strings.

Unsigned Interleaved (uie)

• Property: Bits.uie
• Format String: 'uie'

Signed Interleaved (sie)

• Property: Bits.sie
• Format String: 'sie'

# Example of how you would typically use these with unpack
# (Note: exact bit patterns are defined by the Dirac standard)
# bits.unpack('5*uie')

The bitstring module provides two primary container classes for bits. Choosing the right one depends on your use case:

• Bits: An immutable container. Use this if:
  • You need to use the bitstring as a key in a dict or as a member of a set.
  • You are reading extremely large files (it does not read the whole file into memory).
  • You want the fastest and most memory-efficient option and do not need to modify the contents.
• BitArray: A mutable container. Use this if:
  • You need to change the contents (e.g., truncating, replacing, inserting, or appending bits).

Note that BitArray inherits from Bits, so isinstance(obj, Bits) will return True for both classes.

from bitstring import Bits, BitArray

# Immutable
b = Bits('0b1101')

# Mutable
a = BitArray('0b1101')
a.append('0b10') # Works
# b.append('0b10') # Would raise an error