Zero Python RPC Framework

repository·main·Indexed 20 days ago

https://github.com/ananto30/zero

A high-performance Python RPC framework for microservices and distributed servers. Zero supports both synchronous and asynchronous patterns using ZeroMQ or raw TCP for communication. It features a code generation tool (zero.generate_client) for typed client wrappers, supports msgspec and Pydantic for serialization, and utilizes multiprocessing to leverage all CPU cores via ZeroServer.

Tokens
25.7K
Snippets
101
Records
123
Agent score
70%

What's inside Zero

  1. Overview of Zero RPC Framework

    main

    Zero is a high-performance Python RPC framework designed for building microservices and distributed servers. It abstracts the complexity of messaging patterns, allowing developers to expose functions as RPC endpoints with minimal boilerplate.

    Key Capabilities

    • High Performance: Utilizes ZeroMQ or raw TCP for fast inter-service communication. Supports multi-core utilization by default.
    • Flexible Communication: Supports both synchronous and asynchronous patterns.
    • Schema Support: Built-in support for Msgspec and Pydantic for structured data.
    • Code Generation: Supports client code generation based on schemas.
  2. Configure Serialization with Msgspec or Pydantic

    main

    Zero uses msgspec as the default serializer, supporting msgspec.Struct, dataclass, and other supported types.

    Pydantic Support

    If you have installed zeroapi[pydantic], you can use pydantic.BaseModel directly as argument or return types.

    Specifying Return Types on Client

    You can force the client to convert the response to a specific type using the return_type parameter in the .call() method.

    from dataclasses import dataclass
    from zero import ZeroClient
    
    @dataclass
    class Order:
        id: int
        amount: float
    
    # Use return_type to cast the response
    def get_order(id: str) -> Order:
        return zero_client.call("get_order", id, return_type=Order)
  3. Manage TCP connections and reuse clients

    main

    TCP connections in Zero are persistent and designed for reuse. To maximize performance and avoid the overhead of establishing new connections, create a single client instance and reuse it for multiple calls. The connection is managed and reused automatically by the client.

    Best Practices:

    • DO: Keep connections alive for repeated calls.
    • DO: Use connection pooling for multiple clients by utilizing the pool_size parameter.
    • DON'T: Create a new client instance for every individual call.
    • DON'T: Use the TCP client in synchronous code.
    from zero import ZeroClient
    from zero.protocols.tcp import TCPClient
    
    client = ZeroClient("localhost", 5559, protocol=TCPClient)
    
    # First call - establishes connection
    result1 = client.call("echo", "first")
    
    # Second call - reuses connection automatically
    result2 = client.call("echo", "second")
  4. Multiprocessing considerations for ZeroServer

    main

    Since ZeroServer utilizes multiprocessing, keep the following in mind:

    • Entry Point: The server must be started under if __name__ == "__main__":.
    • State Management: Global state is instantiated separately in each worker process.
    • Database Connections: Do not share database connections across processes. Connections should be created per-worker.
    • Mutability: Avoid using global mutable state, as changes in one worker will not be reflected in others.
  5. Quick Start: Create a Server and Client

    main

    Zero allows you to spin up an RPC server by registering functions with the ZeroServer instance using the @app.register_rpc decorator. Clients can then connect to the server using ZeroClient and invoke registered functions via the .call(method_name, *args) method.

    Server Implementation

    from zero import ZeroServer
    
    app = ZeroServer(port=5559)
    
    @app.register_rpc
    def echo(msg: str) -> str:
        return msg
    
    if __name__ == "__main__":
        app.run()

    Client Implementation

    from zero import ZeroClient
    
    client = ZeroClient("localhost", 5559)
    print(client.call("echo", "Hello World!"))
    from zero import ZeroServer
    
    app = ZeroServer(port=5559)
    
    @app.register_rpc
    def echo(msg: str) -> str:
        return msg
    
    if __name__ == "__main__":
        app.run()
  6. Rebuild and restart a specific microservice

    main

    If you make changes to the source code of a service, you must rebuild its Docker image. You can rebuild and restart a specific service (e.g., auth, gateway, order, or user) without affecting other services using the --no-deps and --build flags.

    docker-compose up -d --no-deps --build <auth/gateway/order/user>
  7. Set up a local development environment for Zero

    main

    To contribute to Zero, you need to set up a local development environment using a virtual environment and install the package in editable mode along with its dependencies.

    1. Create and activate a virtual environment
    2. Install the package in development mode using pip install -e ".[all]" to ensure changes to the source code are reflected immediately.
    3. Install linting dependencies using the provided requirements file.
    # Create virtual environment
    python -m venv venv
    source venv/bin/activate  # On Windows: venv\Scripts\activate
    
    # Install in development mode
    pip install -e ".[all]"
    
    # Install dev dependencies
    pip install -r requirements-lint.txt
  8. Best practices for using ZeroClient

    main

    To ensure optimal performance and stability when using ZeroClient:

    Recommended:

    • Reuse the client instance: Connections are persistent and reused automatically. Do not create a new client for every call.
    • Use type hints: Utilize the return_type parameter to simplify data handling.
    • Handle errors: Gracefully manage connection errors.
    • Use TCP for performance: If raw speed is the priority, switch from the default ZeroMQ to TCP.

    Avoid:

    • Creating new clients per call: This defeats the purpose of the connection pool.
    • Using ZeroClient in async code: ZeroClient is synchronous. If you are working in an asynchronous environment, use AsyncZeroClient instead.
  9. Follow the Zero Python code style guide

    main

    Zero adheres to PEP 8 and uses Black for formatting, Flake8 for linting, and Mypy for type checking. When writing code, follow these conventions:

    Naming Conventions

    • Classes: PascalCase (e.g., ZeroServer)
    • Functions/methods: snake_case (e.g., register_rpc)
    • Constants: UPPER_CASE (e.g., MAX_MESSAGE_SIZE)
    • Private members: Prefix with an underscore (e.g., _internal_method)

    Type Hints and Docstrings

    • Type Hints: Always include type hints for all function signatures.
    • Docstrings: Use docstrings for all public functions, following the parameter, returns, and raises structure.
    # Naming and Type Hinting Example
    class ZeroServer:
        MAX_MESSAGE_SIZE = 1024
    
        def call(self, method_name: str, data: Any) -> Any:
            """
            Register an RPC function.
    
            Parameters
            ----------
            method_name: str
                The name of the method.
            data: Any
                The data to pass.
    
            Returns
            -------
            Any
                The result of the call.
            """
            pass
    
        def _internal_method(self):
            pass
  10. Authenticate and access the Order Management API

    main

    The API uses JWT authentication. First, obtain a token by sending a POST request to the /api/v1/login endpoint with your credentials. Then, include this token in the Authorization header as a Bearer token for subsequent requests to protected endpoints like /profile and /orders.

    # 1. Login to get a JWT
    curl -X POST -H "Content-Type: application/json" -d '{"username":"user1","password":"password1"}' http://localhost:8000/api/v1/login
    
    # 2. Use the returned token to access protected routes
    curl -X GET -H "Authorization: Bearer <YOUR_TOKEN>" http://localhost:8000/api/v1/profile