nghttp3 Documentation

repository·main·Indexed 22 days ago

https://github.com/ngtcp2/nghttp3

A C implementation of the HTTP/3 protocol (RFC 9114) mapping over QUIC, including support for QPACK (RFC 9204). Designed to be transport-agnostic, nghttp3 provides APIs for managing HTTP/3 connections, handling stream lifecycles, and implementing QPACK encoding and decoding. It supports the Extensible Prioritization Scheme for HTTP (RFC 9218) and Bootstrapping WebSockets with HTTP/3 (RFC 9220), though it does not support server push.

Tokens
3.2K
Snippets
3
Records
16
Agent score
78%

What's inside nghttp3

  1. Overview of nghttp3

    main

    nghttp3 is a C implementation of HTTP/3 (RFC 9114) mapping over QUIC and QPACK (RFC 9204).

    Key characteristics:

    • Transport Agnostic: It does not depend on any specific QUIC transport implementation.
    • HTTP/3 Support: Implements RFC 9114 but does not support server push.
    • QPACK Support: Implements RFC 9204 and supports the dynamic table.
    • Extensions:
      • Extensible Prioritization Scheme for HTTP (RFC 9218)
      • Bootstrapping WebSockets with HTTP/3 (RFC 9220)
      • The ORIGIN Extension in HTTP/3 (RFC 9412)
      • HTTP Datagrams and the Capsule Protocol (via SETTINGS_H3_DATAGRAM)
  2. Build nghttp3 from git

    main

    To build nghttp3 from the source repository, ensure you have a C11 compatible compiler (e.g., clang >= 19, gcc >= 15, or MSVC 2022 (1944)). Follow these steps to clone, initialize submodules, and build the project:

    1. Clone the repository.
    2. Initialize submodules.
    3. Run autoreconf to prepare the build system.
    4. Configure and make.
    $ git clone https://github.com/ngtcp2/nghttp3
    $ cd nghttp3
    $ git submodule update --init
    $ autoreconf -i
    $ ./configure
    $ make -j$(nproc) check
  3. Gracefully close an HTTP/3 connection

    main

    To shut down a connection without abruptly cutting off active streams, follow this sequence:

    1. Signal Shutdown: Call nghttp3_conn_submit_shutdown_notice. This notifies the remote endpoint that the connection is going down. The receiver should stop sending new requests after seeing this.
    2. Wait: Allow a couple of RTTs for the notice to propagate.
    3. Start Shutdown: Call nghttp3_conn_submit_shutdown. This causes the local endpoint to start rejecting new incoming streams while allowing existing streams to process normally.
    4. Finalize:
      • Clients: Naturally know when their requests are complete.
      • Servers: Monitor nghttp3_conn_is_drained. When this returns a non-zero value, all existing streams have been processed, and the connection can be safely closed.
  4. Using the QPACK decoder

    main

    To decode QPACK encoded data, follow these steps:

    1. Initialize the decoder: Call nghttp3_qpack_decoder_new. It requires hard_max_dtable_size and max_blocked parameters (e.g., 4096 and 0 respectively).
    2. Process encoder stream: Call nghttp3_qpack_decoder_read_encoder to read the encoder stream. This may update the dynamic table but does not emit header fields.
    3. Process request stream: Call nghttp3_qpack_decoder_read_request to emit header fields. This requires a per-stream decoder state (sctx) created via nghttp3_qpack_stream_context_new.
      • The fin parameter must be non-zero if and only if the provided data contains the last part of the encoded header block.
      • If the function returns, you may need to call it repeatedly by adjusting the data pointer and length until all data is consumed or the NGHTTP3_QPACK_DECODE_FLAG_BLOCKED flag is set.
    4. Manage stream context: The nghttp3_qpack_stream_context is scoped to a single header block. You can reuse it for another header block in the same stream by calling nghttp3_qpack_stream_context_reset, typically when the NGHTTP3_QPACK_DECODE_FLAG_FINAL flag is detected.
    5. Handle emitted headers: If *pflags has the NGHTTP3_QPACK_DECODE_FLAG_EMIT flag set, header fields are stored in the buffer pointed to by nv.
      • Memory Management: If nv is assigned, the nv->name and nv->value members are reference counted. Once finished processing them, you must call nghttp3_rcbuf_decref(nv->name) and nghttp3_rcbuf_decref(nv->value) to prevent leaks.
    6. Cancel streams: If you do not need to decode headers for a specific stream, call nghttp3_qpack_decoder_cancel_stream.
    7. Provide feedback to encoder: Call nghttp3_qpack_decoder_write_decoder to write the decoder stream, informing the encoder of the decoding state.
    8. Cleanup: Call nghttp3_qpack_decoder_del to free the decoder's memory and nghttp3_qpack_stream_context_del to free individual stream contexts.
  5. Using the QPACK encoder

    main

    To encode HTTP header fields using QPACK, follow these steps:

    1. Initialize the encoder: Call nghttp3_qpack_encoder_new. You must provide a hard_max_dtable_size parameter (e.g., 4096).
    2. Configure capacity (Optional):
      • Use nghttp3_qpack_encoder_set_max_dtable_capacity to set the maximum size of the dynamic table.
      • Use nghttp3_qpack_encoder_set_max_blocked_streams to set the maximum number of streams that can be blocked.
    3. Encode headers: Prepare an array of nghttp3_nv structures containing the header fields. Call nghttp3_qpack_encoder_encode. This function populates three buffers:
      • pbuf: Header block prefix.
      • rbuf: Request stream data.
      • ebuf: Encoder stream data.
      • Note: You must send the header block prefix (pbuf) and the request stream (rbuf) in that order to the stream_id provided. The encoder stream data (ebuf) must be sent to the dedicated encoder stream you have established.
    4. Handle decoder feedback: Call nghttp3_qpack_encoder_read_decoder to read data from the decoder stream.
    5. Cleanup: Call nghttp3_qpack_encoder_del to free the encoder's memory.
  6. Initialize an nghttp3 connection

    main

    An nghttp3_conn object is the core building block for an HTTP/3 connection. You must create one per connection using either nghttp3_conn_client_new (for clients) or nghttp3_conn_server_new (for servers).

    Both functions require:

    1. nghttp3_callbacks: A structure of callback functions. While optional, most transactions require several callbacks to function correctly.
    2. nghttp3_settings: Options to tweak HTTP/3 connection settings. Use nghttp3_settings_default to populate these with default values.
    3. user_data: An opaque pointer passed to all callback functions, allowing you to associate application-specific state with the connection.
    /* Example initialization concept */
    nghttp3_conn *conn;
    nghttp3_callbacks callbacks = { ... };
    nghttp3_settings settings;
    nghttp3_settings_default(&settings);
    
    if (is_client) {
        conn = nghttp3_conn_client_new(&callbacks, &settings, user_data);
    } else {
        conn = nghttp3_conn_server_new(&callbacks, &settings, user_data);
    }
  7. Optimize nghttp3 with AVX2

    main

    nghttp3 can optionally use AVX2 instructions to improve performance if your hardware supports it. To enable this, you must pass -mavx2 to your CFLAGS during the configuration step.

    Note: Since the default CFLAGS is -g -O2, you must include those flags when specifying your own to ensure optimization and debugging symbols are preserved (e.g., -g -O2 -mavx2).

    # Example of configuring with AVX2 optimization
    $ ./configure CFLAGS="-g -O2 -mavx2"
  8. View nghttp3 examples

    main

    The following implementation examples are available in the repository to help you understand how to use the library in client, server, and curl contexts:

    • Client example: examples/client.cc
    • Server example: examples/server.cc
    • curl integration: curl/lib/vquic/curl_ngtcp2.c
  9. Required nghttp3_callbacks for HTTP/3 transactions

    main

    To perform standard HTTP/3 transactions, you should implement the following callbacks in nghttp3_callbacks:

    CallbackPurpose
    acked_stream_dataTells you the largest offset of HTTP payload acknowledged by the remote endpoint. Use this to know when it is safe to free/reuse HTTP payload data.
    stream_close2Called when a stream is closed. Use this to free resources allocated for that specific stream.
    recv_dataCalled when HTTP payload (request/response body) is received.
    recv_headerCalled when an HTTP header field is received.
    deferred_consumeCalled when nghttp3_conn consumes HTTP stream data that was previously blocked for synchronization. You must notify the QUIC stack of the bytes consumed to update flow control.
    send_stop_sendingCalled when a QUIC STOP_SENDING frame must be sent. The application must tell the QUIC stack to send it.
    reset_streamCalled when a QUIC RESET_STREAM frame must be sent. The application must tell the QUIC stack to send it.
    randUsed for generating unpredictable data (e.g., for map key hashing). Strongly recommended for servers to harden against suspicious activity.
  10. Bind HTTP/3 control and QPACK streams

    main

    HTTP/3 requires specific unidirectional streams for management. Use these functions to bind stream IDs provided by your QUIC stack to their required roles:

    • nghttp3_conn_bind_control_stream: Binds a stream ID to the HTTP control stream.
    • nghttp3_conn_bind_qpack_streams: Binds two stream IDs to the QPACK encoder and decoder streams respectively.
  11. Read HTTP stream data from QUIC

    main

    Use nghttp3_conn_read_stream2 to read HTTP stream data from a specific stream.

    Key behaviors:

    • Consumption: The function returns the number of bytes "consumed". Consumed bytes are those completely processed, allowing the QUIC stack to increase flow control credit for both the stream and the connection.
    • Payload vs. Consumption: The HTTP payload notified via recv_data is not included in the return value of read_stream2, as the library does not know when your application has finished processing the payload.
    • Deferred Consumption: If data is blocked due to stream synchronization, consumption is handled via the deferred_consume callback.
    • Flow Control: You must notify your QUIC stack of the number of bytes consumed to ensure flow control limits are extended.
  12. Write HTTP stream data to QUIC

    main

    Use nghttp3_conn_writev_stream to generate HTTP stream data for a particular stream.

    Workflow:

    1. Call nghttp3_conn_writev_stream. It populates *pstream_id with the stream ID, vec with the data pointers, and *pfin with a non-zero value if it is the final part of the stream.
    2. Pass the produced data to your QUIC stack.
    3. Call nghttp3_conn_add_write_offset with the number of bytes accepted by the QUIC stack. Note: You must call this even if 0 bytes were written but *pfin was non-zero (signaling the end of the stream).

    Handling Flow Control and Stream State:

    • Stream Blocked: If the QUIC stack indicates a stream-level flow control limit, call nghttp3_conn_block_stream. Resume by calling nghttp3_conn_unblock_stream when the limit increases.
    • Stream Closed: If the QUIC stack indicates the write side of a stream is closed, call nghttp3_conn_shutdown_stream_write instead of blocking it. This ensures the stream is never scheduled again.