wsproto

repository·main·Indexed 43 days ago

https://github.com/python-hyper/wsproto

A pure-Python, state-machine-based implementation of the WebSocket protocol (RFC6455 and RFC7692). Built using a Sans-I/O model, wsproto handles protocol logic and state transitions without managing networking or concurrency, allowing it to be embedded into existing network applications. It supports WebSocket handshakes, extensions, and integration with HTTP/2 via the Connection and WSConnection classes.

Tokens
10.1K
Snippets
30
Records
54
Agent score
53%

What's inside wsproto

  1. How wsproto works: The Sans-I/O model

    main

    wsproto is a pure state-machine WebSocket implementation that does not perform any network I/O. To use it, you must implement two layers of "glue" code:

    1. Network Glue: You are responsible for reading bytes from a socket and passing them to the connection via WSConnection.receive_data(data). Conversely, when you want to send data, you call WSConnection.send(event), which returns the raw bytes that you must then write to the socket.
    2. Application Glue: You must iterate over the WSConnection.events() generator to process high-level WebSocket events (like TextMessage or Ping) yielded by the library.

    Important: If a network socket returns zero bytes (indicating a drop), call ws.receive_data(None) to update the internal state to indicate the connection has closed.

    # Network Glue: Sending
    bytes_to_send = ws.send(event)
    sock.send(bytes_to_send)
    
    # Network Glue: Receiving
    data = sock.recv(4096)
    ws.receive_data(data)
    
    # Application Glue: Processing
    for event in ws.events():
        if isinstance(event, TextMessage):
            print(event.data)
  2. Understand the connection lifecycle and state transitions

    main

    wsproto uses a state-machine approach where incoming bytes are parsed via receive_data(), but the resulting state changes are only applied when you consume the yielded events via events().

    For example, if a remote peer sends a close frame, the connection state will not transition to REMOTE_CLOSING when the bytes are passed to receive_data(); instead, the transition occurs when the corresponding CloseConnection event is yielded from the events() generator.

  3. Handle back-pressure in WebSocket implementations

    main

    Because wsproto is a Sans-IO library, it does not manage network-level concerns like back-pressure. Every call to receive_data(some_byte_string_of_data) appends data to an internal buffer. If you receive data faster than you can process it, memory usage will grow indefinitely.

    To implement back-pressure over TCP, follow these two rules:

    1. Control Inbound Data: Do not call sock.recv() (or your network equivalent) faster than your code can process the resulting messages. Your processing logic should signal the receiving loop when it is ready for more data.
    2. Bound Outbound Data: Do not store outgoing messages in an unbounded collection. Send messages to the OS as soon as possible. If you must buffer messages in memory, ensure the buffer is bounded to prevent infinite growth.
  4. Implement WebSockets over HTTP/2

    main

    When using WebSockets over HTTP/2, the WebSocket exists within a single HTTP/2 stream. wsproto does not manage the HTTP/2 connection itself; you should use a library like h2 to manage the HTTP/2 layer.

    Workflow:

    1. Use the HTTP/2 connection to handle the initial handshake and extension agreement.
    2. Once extensions are agreed upon, initialize a wsproto.connection.Connection instance.
    3. Data Flow:
      • Inbound: Pass bytes received from the HTTP/2 stream into the connection.receive_data() method.
      • Outbound: Take the bytes returned by connection.send() and wrap them in an HTTP/2 data frame to be sent over the stream.
  5. Understand the wsproto architecture and I/O model

    main

    wsproto is a pure Python WebSocket protocol stack designed for maximum flexibility. It is implemented as a state machine and performs no I/O of its own.

    To use wsproto, you must provide a bridge between the protocol stack and your chosen I/O mechanism (such as asyncio, multi-threading, or single-threaded loops). This decoupling allows it to be integrated into various concurrency models.

    Key features include:

    • RFC 6455 Compliance: Aiming for 100% compliance with the WebSocket protocol.
    • Extensibility: Supports extensions like per-message compression (RFC 7692) via provided mechanisms.
  6. How wsproto works: The state-machine model

    main

    wsproto is a pure-Python, in-memory WebSocket protocol stack implementation based on a state machine. It does not provide a network layer, a parsing layer, or concurrency management.

    To use it, you must integrate it with your own network socket and concurrency model by following this lifecycle:

    1. Initialize: Create a WSConnection as either a CLIENT or SERVER.
    2. Send Data: When you call methods like send() or when the state machine generates responses, call ws.bytes_to_send() to get the bytes that need to be transmitted over your socket.
    3. Receive Data: Feed incoming bytes from your socket into the connection using ws.receive_data(data).
    4. Handle Events: Iterate over ws.events() to react to WebSocket messages (like TextMessage or BytesMessage) or state changes (like CloseConnection).
  7. Use Connection for post-handshake WebSocket management

    main

    If you have already performed the WebSocket handshake outside of wsproto (for example, using a separate HTTP handler or a dual-stack setup), you can use the wsproto.connection.Connection class directly. This allows you to manage the WebSocket protocol state without dealing with handshake events.

    connection = Connection(extensions)  # Agreed extensions
    sock.send(connection.send(Message(data=b"Hi")))
    
    connection.receive_data(sock.recv(4096))
    
    for event in connection.events():
        # As with WSConnection, only without any handshake events
  8. Handle Closing Handshakes

    main

    WebSocket connections require a handshake to close properly. Both clients and servers must reply to a CloseConnection event initiated by the remote peer.

    When you receive a CloseConnection event from ws.events():

    1. Use event.response() to generate the appropriate reply event.
    2. Pass that reply to ws.send() to get the bytes.
    3. Send those bytes over the network.

    Example:

    if isinstance(event, CloseConnection):
        sock.send(ws.send(event.response()))
  9. Implement a WebSocket Server

    main

    To act as a server, instantiate WSConnection with ConnectionType.SERVER.

    Unlike a client, a server must explicitly respond to a Request event by either accepting or rejecting the connection:

    • To Accept: Send an AcceptConnection() event via ws.send().
    • To Reject: Send a RejectConnection() event via ws.send().

    After the handshake, the server is responsible for tearing down the underlying network connection once a CloseConnection event is generated.

    from wsproto import ConnectionType, WSConnection
    from wsproto.events import Request, AcceptConnection
    
    ws = WSConnection(ConnectionType.SERVER)
    
    # Inside your network loop:
    for event in ws.events():
        if isinstance(event, Request):
            print('Accepting connection request')
            sock.send(ws.send(AcceptConnection()))
  10. Implement a WebSocket Client

    main

    To act as a client, instantiate WSConnection with ConnectionType.CLIENT. You must first send a Request event specifying the host and target.

    Steps:

    1. Create WSConnection(ConnectionType.CLIENT).
    2. Create a Request(host="...", target="...").
    3. Call ws.send(request) to get the initial handshake bytes and send them via your socket.
    4. Loop through ws.events() to handle incoming messages, pings, or connection status.
    from wsproto import ConnectionType, WSConnection
    from wsproto.events import Request, TextMessage
    import socket
    
    ws = WSConnection(ConnectionType.CLIENT)
    request = Request(host="example.com", target='foo')
    data = ws.send(request)
    
    # Setup socket
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    sock.connect(("example.com", 80))
    sock.send(data)
    
    # Event loop
    data = sock.recv(4096)
    ws.receive_data(data)
    for event in ws.events():
        if isinstance(event, TextMessage):
            print(f'Received: {event.data}')