The performance suite is orchestrated by a Python script that runs Criterion benchmarks, normalizes results, and can compare zmq.rs against libzmq or OMQ implementations.
Basic Smoke Profile
Run a smoke profile comparing zmqrs and libzmq over TCP:
python3 scripts/run_perf_suite.py --profile smoke --impl zmqrs,libzmq --transport tcp
Comparing with OMQ
To include OMQ in the comparison, the script will clone the pinned revision from perf-suite.json into target/perf-deps/omq.rs. You can specify a custom revision using --omq-rev <sha> or a specific toolchain using --toolchain <version>:
python3 scripts/run_perf_suite.py --profile smoke --impl omq --transport tcp --toolchain 1.94.1
Comprehensive Local Run
For a full decision-making run covering multiple implementations and transports:
python3 scripts/run_perf_suite.py --profile standard --impl zmqrs,libzmq,omq --transport tcp,ipc
Viewing Results
After a run completes, results (including manifest.json, results.jsonl, summary.md, and summary.html) are stored in target/perf-runs/<run-id>/. You can generate a report using:
python3 scripts/report_perf_suite.py target/perf-runs/<run-id>
python3 scripts/run_perf_suite.py --profile smoke --impl zmqrs,libzmq --transport tcp