DOCSEARCHbyAlgolia
Submit Docs

Pyro5 Documentation

repository·master·Indexed 18 days ago

https://github.com/irmen/pyro5

A remote objects communication library for Python that allows objects to interact over a network using standard method calls. Pyro5 provides features for distributed systems including a Name Server for service discovery, support for multiple serializers (serpent, json, marshal, msgpack), a compatibility layer for Pyro4, and tools for managing remote proxies, one-way calls, and batched requests.

Tokens
43.7K
Snippets
111
Records
206
Agent score
60%

What's inside Pyro5

  1. What is Pyro5?

    master

    Pyro5 (Python Remote Objects) is a pure Python library that enables building distributed applications where objects can communicate over a network with minimal effort. It allows you to use standard Python method calls to interact with objects running on different machines.

    Key characteristics:

    • Network Transparency: Call remote objects as if they were local.
    • Pure Python: Runs on many platforms and Python versions.
    • Modern Version: Pyro5 is the current, actively improved version. New projects should use Pyro5 instead of the predecessor, Pyro4.

  2. Overview of Pyro5 features

    master

    Pyro5 is a pure Python library designed to enable distributed object communication with minimal effort. Key capabilities include:

    • Portability: Runs on Python 3.x and Pypy3 across different architectures and operating systems.
    • Serialization: Defaults to the safe serpent serializer, with support for json, marshal, and msgpack.
    • Networking: Supports IPv4, IPv6, and Unix domain sockets, with optional SSL/TLS encryption and 2-way certificate validation.
    • Object Management:
      • A Name Server tracks object locations, allowing for transparent movement of objects.
      • Yellow-pages lookups based on metadata tags.
      • Automatic proxying: Remote objects can be returned as if they were local objects.
    • Performance: Supports one-way invocations, batched invocations, and remote iterators for on-demand item streaming.
    • Robustness: Supports automatic reconnection to servers and provides detailed remote tracebacks when exceptions occur in the caller.

  3. List Pyro5 command line tools

    master

    Pyro5 provides several executable tools that are installed alongside the package. You can run them directly as commands or invoke them as modules using python -m <module_name>.

    Available tools:

    • pyro5-ns: The Name Server.
    • pyro5-nsc: Name Server Client tool.
    • pyro5-echoserver: A test echo server for connectivity testing.
    • pyro5-check-config: Prints current configuration and version info.
    • pyro5-httpgateway: An HTTP gateway server.

  4. Use Pyrolite to interface Java or .NET with Python

    master

    Pyrolite is a client library that enables Java and .NET applications to communicate with Python programs using the Pyro protocol. It allows remote method calls on Python objects as if they were local to the Java or .NET environment.

    Compatibility Note:

    • Use Pyrolite 5.x when working with Pyro5.
    • Use Pyrolite 4.x when working with Pyro4.

  5. Key features of Pyro5

    master

    Pyro5 provides several advanced features for distributed computing:

    • Portability: 100% Python, supporting CPython 3 and Pypy 3 across different architectures and operating systems.
    • Interoperability: Transparent communication between different Python versions.
    • Serialization: Defaults to the safe serpent serializer, but also supports json, marshal, and msgpack.
    • Networking: Supports IPv4, IPv6, and Unix domain sockets, with optional SSL/TLS for secure connections (encryption, authentication, and integrity).
    • Object Management: Includes a name server for tracking object locations and supports "yellow-pages" lookups via metadata tags.
    • Performance: Supports one-way invocations, batched invocations, and remote iterators for on-demand item streaming.
    • Reliability: Automatic reconnection to servers, configurable timeouts, and remote exceptions that are raised in the caller with detailed tracebacks.
    • Advanced Patterns: Supports three instance modes (singleton, one per session, one per call), message annotations, correlation IDs for tracing, and SerializedBlob for large data transfers.

  6. Understand the 'push' model distributed computing example

    master

    This example demonstrates a distributed word-counting system using a 'push' model.

    Workflow:

    1. Counters: Multiple word-counter instances are started. Each instance registers itself in the Pyro5 Name Server using a common name prefix.
    2. Dispatcher: A dispatcher component uses that common name prefix to look up all available counters in the Name Server.
    3. Client: The client reads a text file (e.g., Alice in Wonderland) and sends it to the counters.

    Execution Modes:

    • Single Counter: The client sends the full text to one counter.
    • Distributed (Dispatcher): The client sends the text to the dispatcher, which chunks the text and distributes the work across all available counters in parallel.

    Prerequisite: A Pyro5 Name Server must be running before starting any part of this example.

  7. What is the Pyro Name Server?

    master
    The Pyro Name Server acts as a phone-book-like registry that maps logical object names to their corresponding URIs. Instead of requiring clients to know the exact, often volatile URI (e.g., PYRO:obj_dcf713ac20ce4fb2a6e72acaeba57dfd@localhost:51850), clients can simply request an object by a logical name (e.g., Department.ArchiveServer). This allows you to move servers or change object locations without updating client code, as long as the server updates its registration in the name server.

  8. Access Pyro5 configuration via Pyro5.config

    master
    Configuration settings for Pyro5 are managed through the Pyro5.config object. This object is an instance of the Pyro5.configure.Configuration class and is automatically constructed when you import the Pyro5 package. The initial state of the configuration is determined by a combination of built-in defaults and settings provided via environment variables.

  9. Access attributes added to Pyro objects

    master

    When an object is registered as a Pyro object, Pyro adds two specific attributes to it. These can be used by the object itself or by the developer, but they should not be modified:

    • _pyroId: A str representing the unique ID of the object.
    • _pyroDaemon: A reference to the Pyro5.server.Daemon object that contains this object.

    The _pyroDaemon attribute is particularly useful for registering newly created objects with the existing daemon without needing to maintain a global daemon reference.

  10. Note on parallel execution and the Python GIL

    master

    In this specific example, distributed computing is simulated using threads to make concurrent Pyro calls. Due to Python's Global Interpreter Lock (GIL), these threads will not achieve true CPU parallelism unless they are performing I/O or waiting on signals.

    To demonstrate speedup in this demo, an artificial timer delay is used to prevent the calls from being purely CPU-bound. For actual high-performance distributed parallel calculations, use other distributed computing examples in the repository that do not rely on thread-based concurrency for the compute logic.

  11. Use Metadata tags for Yellow-pages object lookup

    master

    The Name Server supports "Yellow-pages" style lookups using metadata tags. Instead of looking up an object by its exact unique name, you can query for objects belonging to a specific category (tag).

    Key Concepts:

    • Metadata Tags: Simple, case-sensitive strings associated with object registrations.
    • Registration: When registering an object, you can provide a set of strings as metadata.
    • Lookup Modes: You can query for objects that match all provided tags or any of the provided tags.

    API Methods:

    • register(name, uri, metadata=None): Associates metadata with a name.
    • lookup(name, return_metadata=False): Returns the URI (or a (uri, metadata) tuple if return_metadata=True).
    • list(return_metadata=False): Returns a dictionary of names to URIs (or metadata tuples).
    • yplookup(meta_all=None, meta_any=None): Performs a Yellow-pages lookup based on tag sets.
    # Registering with metadata
ns.register("printer.secondfloor", "PYRO:printer1@host:1234", metadata={"printer"})

# Querying for all objects with the 'printer' tag
results = ns.yplookup(meta_any={"printer"})
# returns: {'printer.secondfloor': 'PYRO:printer1@host:1234'}

  12. Understand the Pyro5 exception hierarchy

    master

    Pyro5 uses a specific hierarchy of exception classes to categorize errors occurring during remote procedure calls, daemon management, or security enforcement. When writing error handling logic, you can catch specific exceptions or use the base PyroError to catch any Pyro-related issue.

    Common error categories include:

    • NamingError: Issues with the Name Server.
    • DaemonError: Issues with the Pyro daemon.
    • SecurityError: Issues related to security/authentication.
    • CommunicationError: Network or protocol-level issues, including ConnectionClosedError, TimeoutError, and ProtocolError (which may include MessageTooLargeError or SerializeError).
    # Example of catching a broad Pyro error
try:
    proxy.some_remote_method()
except Pyro5.errors.PyroError as e:
    print(f"A Pyro error occurred: {e}")

# Example of catching a specific communication error
try:
    proxy.some_remote_method()
except Pyro5.errors.ConnectionClosedError:
    print("The connection was closed unexpectedly.")