picows

repository·master·Indexed 43 days ago

https://github.com/tarasko/picows

An ultra-fast, lightweight Python WebSockets client and server library for asyncio. It features a high-performance C implementation and a zero-copy interface, offering two API layers: a high-level websockets API designed as a drop-in replacement for the websockets library, and a low-level Core API for maximum throughput and minimum latency using WSListener and WSFrame.

Tokens
17.1K
Snippets
42
Records
61
Agent score
54%

What's inside picows

  1. Use picows.websockets for websockets compatibility

    master
    The picows.websockets package reimplements the websockets.asyncio interface on top of the picows core API. It is designed to allow straightforward migration from the original websockets library for typical client and server applications. Note that picows.websockets focuses specifically on the asyncio API and does not support the threading, Sans-I/O, or legacy APIs found in the original package.
  2. How to implement an async interface on top of non-async callbacks

    master

    The on_ws_* methods in WSListener are non-async to maximize performance (avoiding event loop overhead and enabling zero-copy). If you need async capabilities, you have two main patterns:

    1. Eager Tasks (Recommended for Python 3.13+): Use asyncio.task.eager_task_factory. This allows non-async callbacks to execute async logic immediately without waiting for the next event loop cycle, minimizing latency.
    2. Asyncio Queue: If you require an async receive_message() style interface (similar to aiohttp), use an asyncio.Queue. Note that this introduces higher latency because coroutines are only woken up on the next event loop cycle and payloads must be copied.

    It is often best to keep the data path non-async and only transition to async when an actual async operation is required.

    # Example approach using eager tasks (Python 3.13+)
    # This is more efficient than using a Queue for turning callbacks async
    import asyncio
    
    # (Implementation details would follow the pattern in echo_client_async_callbacks.py)
  3. How the two picows APIs differ

    master

    picows provides two distinct API layers depending on your performance and ease-of-use requirements:

    1. websockets API: A high-level, asyncio-based interface designed as a drop-in replacement for the popular websockets library. It uses async iterators and high-level message abstractions. Use this for most standard applications where ease of development is a priority.

    2. Core API: A low-level, high-performance API designed for maximum throughput and minimum latency. It uses a non-async data path (similar to asyncio transports/protocols) and provides raw WSFrame objects instead of complete messages. This allows for zero-copy operations and manual control over frame concatenation. Use this for performance-critical applications like algorithmic trading.

  4. Handle backpressure with pause_writing and resume_writing

    master
    To prevent a fast sender from overwhelming a slow consumer, you can implement backpressure-aware producers by overriding WSListener.pause_writing and WSListener.resume_writing. These methods are triggered when the transport hits high watermark limits, allowing you to pause message generation until the transport drains.
  5. Use picows with free-threaded Python

    master

    picows is fully compatible with free-threaded Python. Transports do not require the GIL and can run in parallel in different threads.

    • Multithreaded Server: Start one thread per event loop and call ws_create_server in each thread using reuse_port=True. This allows multiple threads to bind to the same host/port, letting the OS distribute connections.
    • Multithreaded Client: Create one event loop per thread and call ws_connect within those threads.
    • Thread Safety: Transport methods must be called from the thread that owns the transport, otherwise an exception will be raised. To wait for a signal from another thread within an event loop, use asyncio.to_thread with a thread-safe primitive like threading.Event.
  6. Handle WebSocket message fragmentation

    master

    In picows, frames are not automatically concatenated into messages. You must handle reassembly manually if the remote peer sends fragmented messages.

    • Unfragmented message: A single WSFrame where fin=True.
    • Fragmented message: A sequence of frames where the first frames have fin=False and msg_type=WSMsgType.CONTINUATION, ending with a final frame where fin=True.

    Note: Control messages (PING, PONG, CLOSE) are never fragmented. In many cases, Frame == Message if the peer does not fragment.

    # Fragmented message structure:
    # 1. WSFrame(msg_type=WSMsgType.<actual type>, fin=False)
    # 2. WSFrame(msg_type=WSMsgType.CONTINUATION, fin=False)
    # ...
    # 3. WSFrame(msg_type=WSMsgType.CONTINUATION, fin=True)
  7. Handle WebSocket frame fragmentation with WSFrame

    master

    WebSocket messages can consist of multiple frames. Use the fin attribute of a WSFrame to determine if it is the last frame of a message.

    • Unfragmented message: WSFrame(msg_type=WSMsgType.<type>, fin=True)
    • Fragmented message:
      1. First frame: WSFrame(msg_type=WSMsgType.<type>, fin=False)
      2. Intermediate frames: WSFrame(msg_type=WSMsgType.CONTINUATION, fin=False)
      3. Last frame: WSFrame(msg_type=WSMsgType.CONTINUATION, fin=True)
    # Example of identifying the last frame in a fragmented message
    if frame.fin:
        # This is the last frame
        pass
  8. Enable debug logs

    master

    picows uses Python's standard logging module under the picows.* logger. To enable debug logging, set the log level to picows.PICOWS_DEBUG_LL (which is 9).

    import logging
    import picows
    
    # Either set global log level
    logging.basicConfig(level=picows.PICOWS_DEBUG_LL)
    
    # Or set picows logger log level only
    logging.basicConfig(level=logging.INFO)
    logging.getLogger("picows").setLevel(picows.PICOWS_DEBUG_LL)
  9. Use the Core API (Client and Server)

    master

    The Core API uses a listener-based pattern (WSListener) and a non-async data path for maximum efficiency. Instead of receiving full messages, the user handles WSFrame objects. This allows for zero-copy data handling but requires the user to manage frame concatenation if a message spans multiple frames.

    Key components:

    • WSListener: A class you subclass to implement event handlers like on_ws_connected and on_ws_frame.
    • WSTransport: The object used to interact with the connection (e.g., transport.send(), transport.disconnect()).
    • WSFrame: Represents a single WebSocket frame, providing access to the payload and parser state.
    • WSMsgType: Enum for message types (e.g., TEXT, CLOSE).
    • WSUpgradeRequest: Used in servers to facilitate routing via a listener factory.
    # Core Client Example
    import asyncio
    from picows import ws_connect, WSFrame, WSTransport, WSListener, WSMsgType, WSCloseCode
    
    class ClientListener(WSListener):
        def on_ws_connected(self, transport: WSTransport):
            transport.send(WSMsgType.TEXT, b"Hello world")
    
        def on_ws_frame(self, transport: WSTransport, frame: WSFrame):
            print(f"Echo reply: {frame.get_payload_as_ascii_text()}")
            transport.send_close(WSCloseCode.OK)
            transport.disconnect()
    
    async def main():
        transport, client = await ws_connect(ClientListener, "ws://127.0.0.1:9001")
        await transport.wait_disconnected()
    
    if __name__ == "__main__":
        asyncio.run(main())
    
    # Core Server Example
    import asyncio
    from picows import ws_create_server, WSFrame, WSTransport, WSListener, WSMsgType, WSUpgradeRequest
    
    class ServerClientListener(WSListener):
        def on_ws_connected(self, transport: WSTransport):
            print("New client connected")
    
        def on_ws_frame(self, transport: WSTransport, frame: WSFrame):
            if frame.msg_type == WSMsgType.CLOSE:
                transport.send_close(frame.get_close_code(), frame.get_close_message())
                transport.disconnect()
            else:
                transport.send(frame.msg_type, frame.get_payload_as_memoryview())
    
    async def main():
        def listener_factory(r: WSUpgradeRequest):
            # Routing can be implemented here by analyzing request content
            return ServerClientListener()
    
        server: asyncio.Server = await ws_create_server(listener_factory, "127.0.0.1", 9001)
        for s in server.sockets:
            print(f"Server started on {s.getsockname()}")
    
        await server.serve_forever()
    
    if __name__ == "__main__":
        asyncio.run(main())
  10. Use the websockets API (Client and Server)

    master

    The websockets API is a drop-in replacement for the websockets library. You only need to change your imports to picows.websockets.asyncio.client or picows.websockets.asyncio.server.

    Note: Some features from the original websockets library are not yet supported. Check the official documentation for the full compatibility list.

    # Client Example
    from picows.websockets.asyncio.client import connect
    import asyncio
    
    async def hello():
        async with connect("ws://localhost:8765") as websocket:
            await websocket.send("Hello world!")
            message = await websocket.recv()
            print(message)
    
    if __name__ == "__main__":
        asyncio.run(hello())
    
    # Server Example
    from picows.websockets.asyncio.server import serve
    import asyncio
    
    async def echo(websocket):
        async for message in websocket:
            await websocket.send(message)
    
    async def main():
        async with serve(echo, "localhost", 8765) as server:
            await server.serve_forever()
    
    if __name__ == "__main__":
        asyncio.run(main())
  11. Connect to WebSockets via HTTP, SOCKS4, or SOCKS5 proxies

    master

    The ws_connect function supports proxies via the proxy argument using a proxy URL.

    • HTTP Proxies: Use https://user:password@host:port. When using https://, TLS is established with the proxy first. Use proxy_ssl_context to customize certificate verification.
    • SOCKS Proxies: Supports SOCKS4 and SOCKS5. Note that for SOCKS4, hostname resolution is performed by the client, whereas for other protocols, it generally happens at the proxy.
    • WSS through Proxy: For wss:// URLs, picows establishes a tunnel through the proxy before performing the TLS handshake with the WebSocket server.
    • System Proxies: picows does not use system proxy settings automatically. To use them, retrieve them via urllib.request.getproxies() and pass the result to the proxy argument.
    import ssl
    from picows import ws_connect
    
    transport, listener = await ws_connect(
        ClientListener,
        "ws://127.0.0.1:9000/",
        proxy_ssl_context=ssl.create_default_context(),
        proxy="https://user:password@127.0.0.1:1080",
    )