pynng

repository·master·Indexed 43 days ago

https://github.com/codypiersall/pynng

Ergonomic Python bindings for the nng (nanomsg next generation) library, providing process-to-process and network communication. It supports various scalability protocols including Pair, Request/Reply, Publish/Subscribe, Pipeline, Survey, and Bus. pynng offers first-class support for synchronous usage as well as asyncio and Trio for asynchronous I/O, and supports transports such as TCP, TLS, IPC, and Linux-specific abstract sockets.

Tokens
12.5K
Snippets
30
Records
58
Agent score
55%

What's inside pynng

  1. Available pynng usage examples

    master

    The examples/ directory provides implementations of various NNG communication patterns using pynng. These examples serve as templates for different messaging topologies:

    • Pair (Two-Way Radio): Bi-directional connection between two endpoints (pair0.py).
    • Polyamorous Pair: Multiple connections using the pair pattern, including async implementations for trio (pair1_async.py) and curio (pair1_async_curio.py).
    • Pipeline (One-Way Pipe): Push/Pull pattern for unidirectional data flow (pipeline.py).
    • Request/Reply (I ask, you answer): Rep0/Req0 pattern for synchronous-style request/response cycles (reqprep.py).
    • Pub/Sub (Topics & Broadcast): Pub0/Sub0 pattern for broadcasting messages to multiple subscribers (pubsub.py).
    • Survey (Everybody Votes): Surveyor0/Respondent0 pattern for collecting responses from multiple participants (survey.py).
    • Bus (Routing): Bus0 pattern for complex routing and many-to-many communication (bus.py).
  2. Overview of pynng protocols and transports

    master

    pynng provides Pythonic bindings to the nng (Nanomsg Next Generation) library, implementing Scalability Protocols. These protocols allow for structured communication patterns that are language-agnostic.

    Available Protocols

    • pair: Simple one-to-one communication (use pynng.Pair0 or pynng.Pair1).
    • request/response: A request-response pattern (use pynng.Req0 or pynng.Rep0).
    • pub/sub: Publish/subscribe pattern where subscribers are notified of specific topics (use pynng.Pub0 or pynng.Sub0).
    • pipeline (push/pull): Used for load balancing (use pynng.Push0 or pynng.Pull0).
    • survey: Query the state of multiple applications (use pynng.Surveyor0 or pynng.Respondent0).
    • bus: Messages are sent to all connected sockets (use pynng.Bus0).

    Available Transports

    Transports define how data moves between endpoints:

    • inproc: Communication within a single process.
    • ipc: Communication across processes on a single machine.
    • tcp: Communication over networks via TCP.
    • ws: Communication over networks via WebSockets.
    • tls+tcp: Encrypted TLS communication over networks.
    • carrier pigeons: High-latency communication via avian carriers.
  3. Use Linux-specific Abstract Sockets

    master

    Abstract sockets are a Linux-only feature that allows socket communication without creating files in the filesystem. They use the abstract:// URL scheme.

    Key Characteristics:

    • They do not exist in the filesystem.
    • They are automatically freed by the system when no longer in use.
    • They ignore socket permissions.
    • They support arbitrary values in the path, including embedded NUL bytes (which must be URI-encoded).

    URI Encoding: To include a NUL byte in a name, use %00. For example, the name a\0b is represented as abstract://a%00b.

    Auto-bind: Using an empty name abstract:// with a listener allows the system to automatically assign a free name. You can retrieve this assigned name using the NNG_OPT_LOCADDR option.

    import pynng
    import platform
    
    # Abstract sockets are only supported on Linux
    if platform.system() != "Linux":
        raise RuntimeError("Abstract sockets are only supported on Linux")
    
    with pynng.Pair0() as sock:
        # Listen on an abstract socket
        listener = sock.listen("abstract://my_test_socket")
    
        # Get the assigned local address
        local_addr = listener.local_address
        print(f"Address name: {local_addr.name}")
    
        # To connect from another process:
        # dialer = sock.dial("abstract://my_test_socket")
  4. Use protocol subclasses instead of instantiating Socket directly

    master

    The pynng.Socket class is the base class for all sockets in pynng. You should never instantiate pynng.Socket directly. Instead, you must instantiate one of the specific protocol subclasses provided by the library. Each subclass implements a specific communication pattern (e.g., Request/Reply, Publish/Subscribe).

    import pynng
    
    # Correct: Use a protocol subclass
    with pynng.Req0() as sock:
        pass
    
    # Incorrect: Do not instantiate Socket directly
    # sock = pynng.Socket()
  5. Exception hierarchy in pynng

    master

    pynng translates all NNG error codes into Python exceptions. The base class for all exceptions in this library is NNGException, which inherits from the built-in Exception. You can catch all pynng-related errors by catching NNGException.

    from pynng.exceptions import NNGException
    
    try:
        # pynng operation
        pass
    except NNGException:
        # handles any error originating from NNG
        pass
  6. Configure TLS transport for sockets

    master

    You can use TLS on top of TCP by specifying a URL scheme starting with tls+. This is configured via the address string passed to listen() or dial().

    Valid TLS address examples:

    • tls+tcp:127.0.0.1:1313: Listen on TCP port 1313 on localhost.
    • tls+tcp4:127.0.0.1:1313: Explicitly request IPv4 for TCP port 1313 on localhost.
    • tls+tcp6://[::1]:4433: Explicitly request IPv6 for IPv6 localhost on port 4433.
  7. Install pynng

    master

    You can install pynng using pip. On Windows, MacOS, and Linux, the standard installation command is usually sufficient.

    Note for 32-bit Linux and macOS: Binary distributions are not available for these platforms. You must have CMake installed to build from source.

    To build directly from the GitHub repository, use:

    git clone https://github.com/codypiersall/pynng
    cd pynng
    pip3 install -e .
    pip3 install pynng
  8. Use Surveyor0 and Respondent0 for the survey pattern

    master

    The survey pattern allows a Surveyor0 socket to send a message and give all connected Respondent0 sockets a chance to respond.

    Key details:

    • Surveyor0 sockets use the survey_time attribute to define how long a survey is valid (in milliseconds).
    • Respondent0 sockets are the counterparts that receive the survey and chime in.

    Example usage (conceptual):

    # Surveyor0 setup
    with Surveyor0(survey_time=1000) as surveyor:
        # ... survey logic ...
    
    # Respondent0 setup
    with Respondent0(listen='tcp://127.0.0.1:12345') as respondent:
        # ... respond to surveys ...
    # See snippets/surveyor_sync.py in the repository for a full implementation example
  9. Configure and retrieve socket options in pynng

    master

    Options can be set or retrieved on various pynng objects (Socket, Dialer, Listener, Pipe, or Context). The library provides specialized internal handlers for different data types to ensure correct C-level type mapping.

    When setting integer-based options (like int, size, or ms), the library accepts floats as long as they are exactly equal to their integer representation (e.g., 1.0 is accepted as 1).

    Common option types supported:

    • Boolean: Use for flags.
    • Integer/Size/MS: Use for counts, sizes, or durations (milliseconds).
    • String: Use for text-based configuration.
    • Sockaddr: Use for address-related configurations.
    • Pointer (TLS): Specifically used for passing pynng.tls.TLSConfig objects.
  10. How pynng handles asynchronous backends (asyncio and trio)

    master

    pynng uses an internal AIOHelper class to bridge NNG's asynchronous I/O with Python's asynchronous frameworks. It detects the running event loop using sniffio and applies specific machinery to ensure compatibility:

    • asyncio: Uses asyncio_helper to create futures and reschedule the event loop via loop.call_soon_threadsafe when NNG callbacks trigger.
    • trio: Uses trio_helper to manage Trio's low-level task rescheduling and cancellation mechanics, ensuring that Trio cancellations are correctly propagated to NNG via nng_aio_cancel.