DOCSEARCHbyAlgolia
Submit Docs

kcp-go Documentation

repository·master·Indexed 26 days ago

https://github.com/xtaci/kcp-go

A high-performance, Reliable-UDP library for Go designed for latency-sensitive applications such as gaming and live broadcasting. It implements net.Conn and net.Listener interfaces for compatibility with Go's standard networking, featuring Forward Error Correction (FEC) using Reed-Solomon Codes and various packet-level encryption options including AES, Salsa20, and AEAD. The library is optimized for scalability, supporting over 5,000 concurrent connections per server.

Tokens
6.8K
Snippets
5
Records
57
Agent score
88%

What's inside kcp-go

  1. Overview of kcp-go

    master
    kcp-go is a Reliable-UDP library for Go that provides smooth, resilient, ordered, error-checked, and anonymous stream delivery over UDP packets. It is designed for latency-sensitive scenarios like online games, live broadcasting, and network acceleration. It is highly optimized for performance, handling over 5,000 concurrent connections on a single commodity server, and is compatible with Go's net.Conn and net.Listener interfaces, making it a drop-in replacement for net.TCPConn.

  2. Key Features of kcp-go

    master

    kcp-go includes the following features:

    • Latency-Sensitive Design: Optimized for low-latency applications.
    • High Performance: Cache-friendly and memory-optimized core.
    • Scalability: Supports >5K concurrent connections per server.
    • Standard Library Compatibility: Implements net.Conn and net.Listener.
    • FEC Support: Forward Error Correction using Reed-Solomon Codes.
    • Encryption:
      • Packet-level encryption (AES, TEA, 3DES, Blowfish, Cast5, Salsa20) in CFB mode.
      • AEAD packet encryption support.
      • Encrypts headers, checksums, and payload for complete anonymity.
    • Resource Efficiency: Uses a fixed number of goroutines to minimize context switching.
    • Platform Optimizations: Supports sendmmsg and recvmmsg on Linux.

  3. Install the KCP dissector for Wireshark on macOS

    master
    To enable KCP protocol dissection in Wireshark on macOS, you must manually install the Lua dissector script. Copy the kcp_dissector.lua file into the Wireshark PlugIns directory within the application bundle.

  4. When to enable FEC and Encryption

    master

    Forward Error Correction (FEC)

    Enable FEC for long-distance transmissions. In complex routing networks, packet loss incurs significant time penalties. Because RTT samples can deviate significantly over long distances, typical RTT estimators may result in a larger RTO (Retransmission TimeOut), slowing down transmission. FEC helps mitigate this.

    Encryption

    It is recommended to enable encryption for protocol security, even if your upper-layer application already implements encryption.

  5. Security Considerations for kcp-go

    master

    When using kcp-go, keep the following security aspects in mind:

    • Encryption: Using encryption (like AES-128) makes the entire packet (including headers, FEC, and checksums) anonymous. If encryption is disabled, the header is sent in plaintext and is susceptible to tampering (e.g., jamming sliding window size, RTT, or checksums).
    • Traffic Analysis: While encryption hides content, data flow patterns may still be visible. Using smux to mix data streams and introduce noise can help mitigate this.
    • Replay Attacks: kcp-go does not implement asymmetric encryption. An attacker could capture and replay packets on a different machine. While they cannot decrypt the session, they could potentially replay messages. It is recommended to use an upper-layer asymmetric encryption system (like HTTPS/OpenSSL) to ensure message authenticity and prevent replay attacks.

  6. Handle connection termination in KCP

    master

    KCP does not define control messages like TCP's SYN/FIN/RST. To detect or manage connection termination, you must implement a keepalive/heartbeat mechanism at the application level.

    A recommended approach is to use a multiplexing protocol over the KCP session, such as smux, which includes an embedded keepalive mechanism.

  7. Establish a KCP listener

    master

    Use Listen or ListenWithOptions to start a KCP server that listens for incoming UDP packets. You can specify encryption and Forward Error Correction (FEC) parameters.

    • Listen(laddr string): Listens on the provided address without encryption or FEC.
    • ListenWithOptions(laddr string, block BlockCrypt, dataShards, parityShards int): Listens with optional encryption (block) and FEC (dataShards and parityShards).

    For FEC details, refer to the reedsolomon implementation.

  8. Optimize CPU utilization for high connection counts

    master

    If you are handling a large number of connections (>5K) and experiencing high CPU utilization, consider the following:

    1. Use a dedicated agent/gate server: Running KCP on a standalone server can reduce CPU load and improve RTT measurement precision.
    2. Adjust SetNoDelay interval: Increasing the update interval via SetNoDelay can dramatically reduce system load, though it may impact performance. For example, using conn.SetNoDelay(1, 40, 1, 1) increases the interval.

  9. Run performance benchmarks for kcp-go

    master

    You can run the built-in benchmarks to evaluate the performance of various encryption algorithms (e.g., AES, Salsa20, SM4), FEC (Forward Error Correction) encoding/decoding, and throughput speeds (Echo/Sink) on your specific hardware. Use the standard Go testing tool with the -bench flag.

    $ go test -v -run=^$ -bench .

  10. KCP Packet Specification and Frame Format

    master

    The KCP protocol uses a specific frame format for transmission. The frame includes a nonce, CRC32 checksum, FEC information, and the KCP header.

    Frame Structure

    Nonce: 16 bytes of cryptographically secure random numbers (changes every packet). CRC32: CRC-32 checksum of data using the IEEE polynomial. FEC TYPE:

    • typeData = 0xF1
    • typeParity = 0xF2 FEC SEQID: Monotonically increasing range: [0, (0xffffffff/shardSize) * shardSize - 1] SIZE: The size of the KCP frame plus 2.

    KCP Header Format

    FieldType
    convu32
    cmdu8
    fragu8
    wndu16
    tsu32
    snu32
    unau32
    databytes
    KCP Header
+------------------------------+
|           conv (u32)         |
+-------+-------+--------------+
|  cmd  |  frag |     wnd      |
|  u8   |  u8   |     u16      |
+-------+-------+--------------+
|           ts   (u32)         |
+------------------------------+
|           sn   (u32)         |
+------------------------------+
|           una  (u32)         |
+------------------------------+
|           data (bytes)       |
+------------------------------+

  11. Find C++ clients and related libraries

    master

    If you require a C++ implementation or related tools:

    • C++ Client (iOS/Android): libkcp is an FEC-enhanced KCP session library.
    • Stream Multiplexing: smux is a memory-efficient stream multiplexing library for Go.
    • Erasure Coding: reedsolomon provides Reed-Solomon erasure coding in Go.

  12. Initialize a new KCP state machine

    master

    Use NewKCP to create a new KCP protocol instance. The conv (conversation ID) must be identical for both peers in a connection, otherwise data will be rejected. You must provide an output_callback function which the KCP engine will call whenever it has data to be sent over the network.

    output_callback signature: func(buf []byte, size int)