WatsonTcp Documentation

repository·main·Indexed 20 days ago

https://github.com/dotnet/watsontcp

A high-performance C# library for building TCP-based clients and servers. It features integrated message framing, reliable transmission, and support for byte-array and stream-based message handling. Key capabilities include SSL/TLS configuration, connection authorization via AuthorizeConnectionAsync, custom handshake state machines, and metadata attachment to messages. The library supports .NET Core, .NET Framework, and Mono environments.

Tokens
8.9K
Snippets
19
Records
37
Agent score
72%

What's inside WatsonTcp

  1. How synchronous request/response works

    main

    WatsonTcp provides a mechanism for synchronous request/response patterns using ConversationGuid to match requests with responses.

    The Workflow:

    1. Sender: Creates a TaskCompletionSource<SyncResponse>, stores it in a dictionary keyed by the message's ConversationGuid, and sends the message with SyncRequest = true and an ExpirationUtc.
    2. Receiver: Detects msg.SyncRequest == true, triggers the SyncRequestReceived callback, and sends a response with SyncResponse = true using the same ConversationGuid.
    3. Original Sender: The receiver's DataReceiver detects the SyncResponse, finds the matching ConversationGuid in the dictionary, and completes the TaskCompletionSource.
    4. Timeout: If the response does not arrive before ExpirationUtc, a TimeoutException is thrown.

    Note on Clock Skew: The library uses WatsonCommon.GetExpirationTimestamp() to adjust for differences between the sender's and receiver's system clocks.

  2. Choose between Message and Stream receive modes

    main

    WatsonTcp provides three ways to receive data. The receive-mode precedence is: Events.MessageReceived > Callbacks.StreamReceivedAsync > Events.StreamReceived.

    ModeTypeDescription
    Events.MessageReceivedBuffered byte[]The message payload is read from the stream and sent to your application as a byte array.
    Callbacks.StreamReceivedAsyncAwaited StreamRecommended for new work. Provides ownership of a stream (either a MemoryStream or a live proxied stream).
    Events.StreamReceivedLegacy Sync StreamThe legacy synchronous stream event. Large proxied streams are still synchronous and should be fully consumed before the handler returns.

    If you configure conflicting modes, warnings are emitted through Settings.Logger.

  3. Understand WatsonTcp Framing

    main

    WatsonTcp uses a hybrid framing model to solve the problem of message demarcation in a bidirectional TCP stream. It combines length prefixing and delimiters to ensure both the sender and receiver know exactly where a message starts and ends.

    The Framing Model

    WatsonTcp's framing is loosely inspired by HTTP. It consists of two parts:

    1. Metadata (Header): A JSON object containing information about the message (e.g., length, status, metadata).
    2. Data (Payload): The actual application-layer message.

    Structure

    The header and the data are separated by a specific delimiter sequence: `[

    ]` (two carriage return/line feed sequences).

    Example Structure:

    {"len":1234,"status":"Normal",...other fields...}[
    ]
    [
    ]
    [data]
    • The len field in the JSON header specifies the exact number of bytes in the [data] payload.
    • The `[

    ]` sequence acts as the boundary between the header and the payload.

  4. How WatsonTcp handles message framing

    main

    WatsonTcp provides reliable message-level delivery over TCP by implementing a custom framing protocol. Because TCP is a bidirectional byte stream without inherent message boundaries, WatsonTcp prepends each message with a JSON header that declares the payload length, followed by a \r\n\r\n delimiter, and then the raw data bytes.

    Wire Format Structure:

    1. JSON header: UTF-8 encoded JSON containing metadata (e.g., len, status, syncreq).
    2. Delimiter: The exact bytes \r\n\r\n (hex: 13, 10, 13, 10).
    3. Raw data: Exactly N bytes as specified in the header's len field.

    Both endpoints must use WatsonTcp or implement this specific framing to communicate successfully.

    +--------------------------------------------------+
    | JSON header (UTF-8 encoded, no pretty printing)  |
    | {"len":N,"status":"Normal","syncreq":false,...} |
    +--------------------------------------------------+
    | \r\n\r\n  (bytes: 13, 10, 13, 10)                |
    +--------------------------------------------------+
    | Raw data bytes (exactly N bytes)                |
    +--------------------------------------------------+
  5. Configure listener IP addresses for local vs external connections

    main

    When initializing WatsonTcpServer, the IP address provided determines which connections are accepted:

    • Localhost only: Use 127.0.0.1.
    • External/All interfaces: Use null, *, +, or 0.0.0.0 (Note: 0.0.0.0 is supported on Windows, but on Mac and Linux you must specify a specific interface IP or use 127.0.0.1).

    Requirements for external connections:

    • Using null, *, +, or 0.0.0.0 may require admin privileges.
    • If using a port number under 1024, admin privileges are required.
    • You must create a permit rule on your firewall to allow inbound connections on the chosen port.
  6. Understand the WatsonTcpClient connection lifecycle

    main

    A WatsonTcpClient follows a specific lifecycle from instantiation to disconnection.

    1. Initialization: Create the client using new WatsonTcpClient(ip, port) (or the SSL variant).
    2. Connection: Call Connect() or ConnectAsync(). This establishes the TCP connection, handles SSL/TLS negotiation if configured, sets up keepalives, and sends an initial RegisterClient message.
    3. Active State: Once connected, the ServerConnected event fires, and a background DataReceiver task begins listening for incoming messages.
    4. Disconnection: Call Disconnect() or Dispose(). This optionally sends a Shutdown message, cancels background tasks, closes the underlying streams, and sets Connected = false.
    // Example lifecycle
    var client = new WatsonTcpClient("127.0.0.1", 5000);
    client.ServerConnected += (s, e) => Console.WriteLine("Connected!");
    
    await client.ConnectAsync();
    
    // ... use client ...
    
    client.Disconnect();
  7. Understand WatsonTcp framing requirements

    main

    WatsonTcp uses integrated framing to ensure message-level delivery. Because of this framing, you must follow one of these two rules:

    1. Use WatsonTcp for both the server and the client.
    2. If using a different library for one side, ensure that the client/server exchanges messages using WatsonTcp's specific framing protocol.

    For details on the message structure, refer to FRAMING.md.

  8. Understand the WatsonTcpServer listener lifecycle

    main

    A WatsonTcpServer manages incoming connections through a structured lifecycle:

    1. Startup: Call Start(). This initializes the TcpListener, starts the _AcceptConnections task (which listens for new clients), and starts the _MonitorClients task (which handles idle timeouts).
    2. Connection Acceptance: For every incoming connection, the server:
      • Validates the IP against PermittedIPs or BlockedIPs.
      • Checks MaxConnections (if EnforceMaxConnections is true, the listener pauses when capacity is reached).
      • Performs SSL/TLS negotiation if configured.
      • Handles Preshared Key (PSK) authentication if configured.
    3. Active Client Management: Each client gets its own DataReceiver task to process messages and updates its ClientsLastSeen timestamp.
    4. Shutdown: When the server is stopped, all client connections are terminated and background tasks are cancelled.
    var server = new WatsonTcpServer("127.0.0.1", 5000);
    server.ServerStarted += (s, e) => Console.WriteLine("Server started!");
    
    server.Start();
    
    // ... server runs ...
    
    server.Stop();
  9. Understand the server-side Admission Pipeline

    main

    As of v6.2.0, WatsonTcp distinguishes between pending connections and active clients. A connection is considered "pending" during the following flow:

    1. TCP accept
    2. SSL/TLS establishment (if enabled)
    3. AuthorizeConnectionAsync execution
    4. Preshared-key flow (if configured)
    5. HandshakeAsync (if configured)
    6. RegisterClient execution

    Important: During steps 1 through 6, the connection is tracked as pending and will not appear in the results of ListClients().

  10. Enable SSL/TLS for WatsonTcp

    main

    To use SSL, provide a PFX certificate file and password to the constructor of both WatsonTcpServer and WatsonTcpClient. Ensure the certificate is in the execution directory.

    Common settings for SSL:

    • AcceptInvalidCertificates: Set to true if using self-signed certificates.
    • MutuallyAuthenticate: Set to true for mutual TLS.
    // Server with SSL
    WatsonTcpServer server = new WatsonTcpServer("127.0.0.1", 9000, "test.pfx", "password"); 
    server.Settings.AcceptInvalidCertificates = true;
    server.Settings.MutuallyAuthenticate = true;
    server.Start();
    
    // Client with SSL
    WatsonTcpClient client = new WatsonTcpClient("127.0.0.1", 9000, "test.pfx", "password"); 
    client.Settings.AcceptInvalidCertificates = true;
    client.Settings.MutuallyAuthenticate = true;
    client.Connect();
  11. Run WatsonTcp under Mono

    main

    While .NET Core is preferred for multi-platform deployments, WatsonTcp supports Mono environments. For best results:

    1. Execute the containing EXE using the --server flag.
    2. Use the Mono Ahead-of-Time (AOT) compiler.

    Note: TLS 1.2 is hard-coded in WatsonTcp, which may require downgrading to TLS in certain Mono environments.

    # Using AOT
    mono --aot=nrgctx-trampolines=8096,nimt-trampolines=8096,ntrampolines=4048 --server myapp.exe
    
    # Standard server execution
    mono --server myapp.exe