pyzmq Documentation

repository·main·Indexed 26 days ago

https://github.com/zeromq/pyzmq

Python bindings for the ZeroMQ (ØMQ) messaging library. pyzmq provides a lightweight and fast messaging implementation, including asynchronous support via zmq.asyncio and zmq.eventloop.future, authentication tools in zmq.auth, and device/proxy management in zmq.devices. It supports Python ≥ 3.9 and libzmq ≥ 3.2.2, and allows high-performance extensions by accessing libzmq functions directly via Cython.

Tokens
11.6K
Snippets
26
Records
93
Agent score
88%

What's inside pyzmq

  1. Check PyZMQ compatibility and supported libzmq versions

    main

    PyZMQ supports stable versions of libzmq (≥ 3.2.2 and ≥ 4.0.1). Note that libzmq 3.0 and 3.1 are not supported.

    When installing via binary distributions (wheels on PyPI), PyZMQ ships with a stable version of libzmq built with default configuration, including CURVE support provided by libsodium.

  2. Explore pyzmq examples and integrations

    main

    The examples/ directory in this repository contains usage patterns for pyzmq, specifically focusing on features that extend beyond standard libzmq bindings. These include:

    • Integrations with event loops
    • Security features
    • Serialization features
  3. PyZMQ Compatibility Requirements

    main

    PyZMQ has the following compatibility requirements:

    • Python: $\ge$ 3.9 (also supports PyPy).
    • libzmq: $\ge$ 3.2.2 (including 4.x).
    • API Support: Fully supports stable 3.x and 4.x APIs of libzmq.
  4. Use Context Managers for Sockets and Contexts

    main

    PyZMQ supports Pythonic resource management using with statements.

    • Using a zmq.Context as a context manager terminates the context upon exit.
    • Using a zmq.Socket as a context manager closes the socket upon exit.
    • Using socket.connect(url) or socket.bind(url) as a context manager automatically calls socket.disconnect(url) upon exit.
    import zmq
    
    # Managing Context and Socket
    with zmq.Context() as ctx:
        with ctx.socket(zmq.PUSH) as s:
            s.connect(url)
            s.send_multipart([b"message"])
        # exiting Socket context closes socket
    # exiting Context context terminates context
    
    # Managing connection lifecycle
    with socket.connect(url):
        s.send_multipart([b"message"])
    # exiting connect context calls socket.disconnect(url)
  5. Configure Cython build to find zmq definitions

    main
    When building a Cython extension that uses pyzmq exports, you must ensure the compiler can find the zmq definitions. In your setup.py, include the directories provided by zmq.get_includes() in your include_dirs configuration.
  6. Access libzmq via Cython in pyzmq

    main

    Since pyzmq 19, you can use Cython to access the underlying libzmq functions and type definitions directly. This allows for high-performance extensions by bypassing some of the Python overhead.

    To import the zmq definitions in a Cython file, use:

    cimport zmq

    To access the underlying wrapped libzmq specifically, use:

    from zmq cimport libzmq
    cimport zmq
    # or
    from zmq cimport libzmq
  7. Install PyZMQ from source

    main

    If you want to force PyZMQ to be compiled from source (which is often preferable if you already have a specific libzmq installed and configured), use the --no-binary flag. Note that building from the GitHub repository requires a recent version of Cython installed.

    pip install --no-binary=pyzmq pyzmq
  8. Cross-compile pyzmq

    main

    Cross-compiling Python extensions requires specific environment configurations. To cross-compile pyzmq, you generally need:

    • A Python installation built for the 'build' machine.
    • A Python installation built for the 'host' machine (must be the identical version).
    • A cross-compiling toolchain (e.g., aarch64-linux-gnu-gcc).
    • A Python setup configured for cross-compilation (tools like crossenv are recommended).

    Best Practice: It is highly recommended to build libzmq and libsodium separately and link them using the ZMQ_PREFIX environment variable. Cross-compiling the bundled versions of these libraries is not guaranteed to work.

  9. Manage thread safety in PyZMQ

    main
    In PyZMQ, zmq.Context objects are thread-safe and can be shared across your entire multithreaded application (e.g., via zmq.Context.instance()). However, Sockets are NOT thread-safe. You should create sockets on a per-thread basis. Sharing sockets across threads without a threading.Lock can lead to uncatchable C-level application crashes.
  10. Tunnel PyZMQ connections with SSH using zmq.ssh.tunnel

    main

    You can tunnel ØMQ socket connections across machines or untrusted networks using the zmq.ssh.tunnel module. This module allows you to connect to a remote ØMQ endpoint by routing the traffic through an SSH server.

    By default, PyZMQ uses the system's ssh command via pexpect, but it also supports paramiko, making it compatible with Windows.

    An SSH tunnel consists of:

    • server: The SSH server used to create the tunnel (can be a hostname or user@server:port).
    • remote ip: The IP of the remote machine as seen from the server.
    • remote port: The port on the remote machine you want to connect to.
    • local ip: The local interface to use (defaults to 127.0.0.1).
    • local port: The local port to forward to (defaults to a high random port).

    Once established, connections to the local ip:local port are transparently forwarded to remote ip:remote port.