repository·master·Indexed 23 days ago
https://github.com/tomerfiliba-org/rpycA transparent and symmetric distributed computing library for Python. RPyC enables bidirectional communication between clients and servers, supporting synchronous and asynchronous remote procedure calls. Key features include the ability to subclass rpyc.Service for custom APIs, the use of rpyc.async_ for non-blocking calls, and various server implementations such as ThreadedServer and OneShotServer. It supports complex patterns like server-side callbacks and session management via token objects.
OneShotServer, the server process will automatically shut down once the client has finished its interaction.services, a client in classic mode can connect to a server and manipulate it without predefined restrictions. In current versions of RPyC, classic mode is implemented as a rpyc.core.service.SlaveService. It is particularly useful for testing environments where unrestricted access is desired.RPyC (Remote Python Call) is a transparent, symmetric library for remote procedure calls, clustering, and distributed computing.
It uses object-proxying to overcome physical boundaries between processes and computers. This allows remote objects to be manipulated as if they were local, meaning existing code can often work seamlessly with both local and remote objects without modification.
Key characteristics include:
RPyC is a service-oriented library where a service is defined as a class that exposes a specific set of remote functions and objects.
To implement a service, create a class containing the methods you wish to expose to clients. RPyC uses a capability-based security model. Instead of granting broad permissions (like file system access), you can pass specific objects (like an open file handle) to a client. This allows the client to perform operations on that specific object (e.g., read(), write()) without having access to the rest of the file system.
By default, RPyC prevents the use of getattr on remote objects except for "allowed attributes."
RPyC is designed with a capability-based security model, but improper usage can create back-doors.
sys), a client might traverse that reference to gain access to all imported modules.allow_public_attrs if untrusted: Enabling allow_public_attrs can allow clients to bypass intended restrictions and reach dangerous objects.SlaveService (Classic Mode) is intentionally insecure and exposes everything to the client. Use it only for testing in isolated environments.Because of the Global Interpreter Lock (GIL) in CPython, CPU-bound programs require multiple processes rather than threads to utilize multicore CPUs.
RPyC simplifies multiprocessing by allowing you to treat RPyC-connected processes as if they were part of "one big process." A common pattern is to have a "master" process spawn multiple worker processes and distribute the workload between them using RPyC connections.
RPyC provides the underlying mechanism for distributed computing and clustering. It is architecture-agnostic, supports both synchronous and asynchronous invocation, and treats clients and servers symmetrically.
While RPyC is not a full-featured distributed computing framework itself, it can be used to build one. A framework built on RPyC would typically handle:
Clients connect to services using various tools:
timed, async_, buffiter, and BgServingThread.Zero-deploy leverages SSH for both authentication and transport security:
When calling server.close(timeout=...), you can specify a timeout in seconds.
TimeoutExpired exception is raised.None (infinite).RPyC uses two primary serialization mechanisms for data transfer:
Boxing is the serialization mechanism RPyC uses to transfer objects across a connection. It uses two distinct strategies depending on the object type:
str, int, tuple). The actual value is copied to the other side. Since the value cannot change, it is safe to duplicate.On the receiving end, unboxing occurs: by-value data is deserialized into local objects, while by-reference data is converted into object proxies (also called netrefs).
When using ThreadedServer, multiple clients can interact with the server simultaneously. Because ThreadedServer provides a concurrency illusion, any data shared between clients must be thread-safe to prevent data races and maintain invariants.
In the sharing/server.py implementation, a constant named THREAD_SAFE is used to toggle between thread-safe and unsafe function calls. When developing shared-state applications, ensure your logic accounts for this concurrency.