DOCSEARCHbyAlgolia
Submit Docs

cloudwego/netpoll

repository·main·Indexed 26 days ago

https://github.com/cloudwego/netpoll

A high-performance, non-blocking I/O networking framework developed by ByteDance and optimized for RPC scenarios. Designed to reduce context-switching overhead compared to the standard Go 'one goroutine per connection' model, it serves as the underlying engine for Kitex and Hertz. It features a nocopy API (LinkBuffer), high-performance goroutine pools (gopool), and efficient memory reuse (mcache). Supports TCP and Unix Domain Sockets on Linux and macOS.

Tokens
14.5K
Snippets
18
Records
109
Agent score
85%

What's inside netpoll

  1. Overview of Netpoll

    main
    Netpoll is a high-performance, non-blocking I/O networking framework developed by ByteDance, specifically optimized for RPC (Remote Procedure Call) scenarios. Unlike the Go standard net library which uses a blocking I/O design (often leading to a 'one connection, one goroutine' model and high context-switching costs), Netpoll is designed to handle high concurrency efficiently. It is used as the underlying networking engine for frameworks like Kitex (RPC) and Hertz (HTTP).

  2. Overview of CloudWeGo-Netpoll

    main
    Netpoll is a high-performance NIO (Non-blocking I/O) network library developed by ByteDance, specifically optimized for RPC (Remote Procedure Call) scenarios. Unlike Go's standard net library which uses a BIO (Blocking I/O) model that requires one goroutine per connection, Netpoll uses an event-driven design inspired by evio and netty to reduce scheduling overhead in high-concurrency microservices. It is the underlying network engine for the Kitex RPC framework and Hertz HTTP framework.

  3. Implement a Netpoll Server

    main

    To run a Netpoll server, follow these steps:

    1. Create a Listener: Use either net.Listen or netpoll.CreateListener.
    2. Create an EventLoop: Initialize an EventLoop using netpoll.NewEventLoop. You must provide a handler (implementing the OnRequest interface) and can pass configuration via Option functions like netpoll.WithOnPrepare or netpoll.WithReadTimeout.
    3. Run the Server: Call eventLoop.Serve(listener) to start the listening loop. This call blocks until an error occurs or Shutdown is called.
    4. Shutdown Gracefully: Use eventLoop.Shutdown(ctx) with a context to stop the server gracefully.
    package main

import (
	"context"
	"time"
	"github.com/cloudwego/netpoll"
)

func main() {
	// 1. Create Listener
	listener, err := netpoll.CreateListener("tcp", ":8080")
	if err != nil {
		panic(err)
	}

	// 2. New EventLoop
	eventLoop, _ := netpoll.NewEventLoop(
		handler, // must implement OnRequest
		netpoll.WithReadTimeout(time.Second),
	)

	// 3. Run Server (blocks)
	go func() {
		eventLoop.Serve(listener)
	}()

	// 4. Shutdown Server
	time.Sleep(time.Second * 10)
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()
	eventLoop.Shutdown(ctx)
}

  4. Use Netpoll Dialer as a Client

    main

    Netpoll provides two ways to establish client connections:

    1. Quick Methods: Use direct public methods for immediate connection:

      • DialConnection(network, address string, timeout time.Duration) (connection Connection, err error)
      • DialTCP(ctx context.Context, network string, laddr, raddr *TCPAddr) (*TCPConnection, error)
      • DialUnix(network string, laddr, raddr *UnixAddr) (*UnixConnection, error)

    2. Using Dialer Interface: For more structured usage, create a netpoll.NewDialer() and call dialer.DialConnection(network, address, timeout).

    package main

import (
	"time"
	"github.com/cloudwego/netpoll"
)

func main() {
	dialer := netpoll.NewDialer()
	conn, err := dialer.DialConnection("tcp", "127.0.0.1:8080", 5*time.Second)
	if err != nil {
		panic(err)
	}
	_ = conn
}

  5. Set up a Netpoll Server

    main

    To run a Netpoll server, follow these steps:

    1. Create a Listener: You can use either a standard net.Listener or a netpoll.Listener via netpoll.CreateListener(network, address).
    2. Create an EventLoop: Initialize an EventLoop using netpoll.NewEventLoop. You must provide an OnRequest handler for business logic. You can also pass configuration options like netpoll.WithOnPrepare or netpoll.WithReadTimeout.
    3. Run the Server: Call eventLoop.Serve(listener). This is a blocking call that runs until a panic occurs or Shutdown is called.
    4. Shutdown Gracefully: Use eventLoop.Shutdown(ctx) with a context to stop the server gracefully.
    package main

import (
	"context"
	"time"
	"github.com/cloudwego/netpoll"
)

func main() {
	// 1. Create Listener
	listener, _ := netpoll.CreateListener("tcp", ":8080")

	// 2. Create EventLoop
	handler := func(ctx context.Context, conn netpoll.Connection) error {
		return nil
	}
	eventLoop, _ := netpoll.NewEventLoop(
		handler,
		netpoll.WithReadTimeout(time.Second),
	)

	// 3. Run Server (blocking)
	go func() {
		eventLoop.Serve(listener)
	}()

	// 4. Shutdown
	time.Sleep(time.Second)
	ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
	defer cancel()
	eventLoop.Shutdown(ctx)
}

  6. Configure Netpoll Poller settings

    main

    You can tune the underlying poller behavior using global configuration functions:

    • Set Number of Loops: Use netpoll.SetNumLoops(n) to set the number of epoll instances. This defaults to runtime.GOMAXPROCS(0). Use this if your service has extremely high I/O requirements.
    • Set Load Balancing Strategy: Use netpoll.SetLoadBalance(strategy) to choose how new connections are distributed across pollers. Supported strategies:
      • netpoll.Random: Assigns new connections to a random poller.
      • netpoll.RoundRobin: Assigns new connections sequentially (Default).
    package main

import (
	"runtime"
	"github.com/cloudwego/netpoll"
)

func init() {
	// Set number of loops
	netpoll.SetNumLoops(runtime.GOMAXPROCS(0))

	// Set load balancing to Random
	netpoll.SetLoadBalance(netpoll.Random)
}

  7. Configure Connection Timeouts

    main

    Netpoll supports two types of timeouts:

    1. Read Timeout: Controls how long a blocking read operation waits.

      • Can be set per connection: conn.SetReadTimeout(timeout).
      • Can be set globally via EventLoop options: netpoll.WithReadTimeout(timeout).
      • Default is infinite (waits forever).

    2. Idle Timeout: Uses TCP KeepAlive to close connections that have been inactive for a long time to prevent dead connections.

      • Can be set per connection: conn.SetIdleTimeout(timeout).
      • Can be set globally via EventLoop options: netpoll.WithIdleTimeout(timeout).
      • Default minimum value is 10min.

  8. Configure Netpoll Runtime Settings

    main

    Adjust the following global settings to optimize Netpoll performance:

    • Number of Pollers: Use netpoll.SetNumLoops(n) to set the number of epoll instances. By default, it scales with runtime.GOMAXPROCS(0). Use this if your service has heavy I/O.
    • Connection Load Balancing: Use netpoll.SetLoadBalance(strategy) to choose how connections are assigned to pollers. Supported strategies:
      • netpoll.Random: New connections are assigned to a randomly picked poller.
      • netpoll.RoundRobin: (Default) New connections are assigned to pollers in order.
    • Goroutine Pool: Netpoll uses gopool by default to optimize stack growth. If you do not have stack growth issues, you can disable it using netpoll.DisableGopool().
    package main

import (
	"runtime"
	"github.com/cloudwego/netpoll"
)

func init() {
	// Configure number of loops
	netpoll.SetNumLoops(runtime.GOMAXPROCS(0))

	// Configure load balancing
	netpoll.SetLoadBalance(netpoll.Random)

	// Disable gopool if not needed
	netpoll.DisableGopool()
}

  9. Optimize performance by setting NumLoops

    main

    If your server is running on a physical machine but does not utilize all available CPU cores, having too many pollers can cause performance degradation. To prevent this, you can limit the number of pollers using netpoll.SetNumLoops(num_you_want).

    Alternatively, you can use system-level tools like taskset or Go's runtime.GOMAXPROCS to manage CPU affinity and process threads.

    package main

import (
	"github.com/cloudwego/netpoll"
)

func init() {
	// Actively set the number of pollers
	netpoll.SetNumLoops(num_you_want)
}

  10. Understand Netpoll's data race detection behavior

    main

    Netpoll uses conditional compilation with //+build !race and //+build race tags to manage data race detection.

    In certain performance-critical sections, Netpoll uses unsafe.Pointer to access struct pointers via epoll. While this is a deliberate optimization to improve performance, it can trigger false positives in the Go race detector. These detections are not actual code bugs but are artifacts of the race detector's inability to track these specific unsafe.Pointer operations.

  11. Optimize performance by setting the number of poller loops

    main

    If your server runs on a physical machine, the number of Go P's (processors) defaults to the number of CPU cores. If the Netpoll Server does not utilize all cores, having too many pollers can degrade performance. To optimize, you can limit the number of loops using one of the following methods:

    1. System level: Use the taskset command to restrict CPU cores for the process.
    2. Go Runtime level: Set runtime.GOMAXPROCS to a specific value.
    3. Netpoll level: Use netpoll.SetNumLoops to explicitly set the number of poller loops.

  12. Configure connection read event callback (OnRequest)

    main

    The OnRequest callback is triggered when a read event occurs on a connection.

    • Server side: Register the callback when creating an EventLoop using netpoll.NewEventLoop. It will be triggered whenever connection data arrives.
    • Client side: By default, there is no OnRequest on the client side, but you can enable it by calling conn.SetOnRequest(handler) on a netpoll.Connection instance.
    package main

import (
	"context"
	"github.com/cloudwego/netpoll"
)

func main() {
	var onRequest netpoll.OnRequest = handler
	
	// 1. on server side
	evl, _ := netpoll.NewEventLoop(onRequest, opts...)
	...
	
	// 2. on client side
	conn, _ := netpoll.DialConnection(network, address, timeout)
	conn.SetOnRequest(handler)
	...
}

func handler(ctx context.Context, connection netpoll.Connection) (err error) {
	// ... handling ...
	return nil
}