hyperscan Python Bindings

You can install hyperscan directly using pip. Because the library is statically linked, you do not need to manually install libhs (Hyperscan or Vectorscan) on your system before installing the Python package.

pip install hyperscan

You can install the hyperscan package directly from PyPI. The package provides pre-built binary wheels that include the scanning engine (Vectorscan on Unix, Hyperscan on Windows), PCRE, Ragel, and Boost, so no system-level libraries are required.

Supported Platforms & Versions:

• Python: CPython 3.9–3.14
• Linux: manylinux2014, manylinux_2_28, and musllinux (x86_64 & aarch64)
• macOS: 11+ universal2
• Windows: AMD64

pip install hyperscan

Build from source if you need to patch the extension, experiment with upstream engine versions, or produce architecture-native binaries.

Prerequisites:

• Python 3.9–3.14
• CMake 3.31+
• C/C++ toolchain (GCC/Clang on POSIX, Visual Studio 2022 on Windows)
• Ragel 6.9+ (can be fetched automatically if not on PATH)

Standard Build Workflow:

git clone https://github.com/darvid/python-hyperscan.git
cd python-hyperscan
python -m venv .venv
source .venv/bin/activate  # .\.venv\Scripts\activate on Windows
pip install --upgrade pip build[uv]
pip install .

Chimera mode mixes PCRE literals with the Hyperscan engine. To use it, instantiate the database with chimera=True.

Note: When using Chimera, it is recommended to reuse a Scratch object per thread to avoid reallocations caused by the larger databases.

chimera_db = hyperscan.Database(chimera=True)
chimera_db.compile(expressions=[br'(foo)+', br'b(ar|az)'])
chimera_db.scan(b'foobaz', match_event_handler=on_match)

Streaming mode allows scanning data in chunks. To use it, initialize the Database with mode=hyperscan.HS_MODE_STREAM.

Use the Database.stream context manager to manage the stream lifecycle. The match_event_handler and context provided to db.stream will be used for all subsequent stream.scan calls unless explicitly overridden in a specific call.

db = hyperscan.Database(mode=hyperscan.HS_MODE_STREAM)

with db.stream(match_event_handler=on_match, context=2345) as stream:
    stream.scan(b'foobar')
    # Override context only for one chunk
    stream.scan(b'barfoofoobarbarfoobar', context=1234)
    # Override match handler only for one chunk
    stream.scan(b'qux', match_event_handler=on_qux_match)

Vectored mode allows scanning multiple buffers in a single call. Pass a list of buffers (e.g., bytearray objects) to the scan method.

db = hyperscan.Database(mode=hyperscan.HS_MODE_VECTORED)
buffers = [
    bytearray(b'xxxfooxxx'),
    bytearray(b'xxfoxbarx'),
    bytearray(b'barxxxxxx'),
]
db.scan(buffers, match_event_handler=on_match)

Block mode is used for scanning a single contiguous buffer. You can pass an optional context object to the scan method, which will be passed to your match handler.

db.scan(b'foobar', match_event_handler=on_match)
# Or, to provide a context object:
db.scan(b'foobar', match_event_handler=on_match, context='foo')

To confirm that hyperscan is installed correctly and to see the version of the bundled engine, run the following command in your terminal. It uses the hyperscan.Database().info() method to return engine details.

python - <<'PY'
import hyperscan
print(hyperscan.Database().info())
PY

Chimera runtimes are included in every build. To use them, you must explicitly enable them when initializing a hyperscan.Database object by setting the chimera parameter to True.

import hyperscan
# Enable Chimera support
db = hyperscan.Database(chimera=True)

For advanced pattern matching, you can use the ext parameter in db.compile. This requires a list of hyperscan.ExpressionExt objects. You only need to provide the fields relevant to your specific flags; all other fields default to 0.

db.compile(
    expressions=[b'foobar'],
    flags=hyperscan.HS_FLAG_SOM_LEFTMOST,
    ext=[
        hyperscan.ExpressionExt(
            flags=hyperscan.HS_EXT_FLAG_MIN_OFFSET, min_offset=12
        )
    ],
)
# Matches the second `foobar` in a stream
db.scan(b'foobarfoobar', match_event_handler=callback)