fasthttp

repository·master·Indexed 12 days ago

https://github.com/valyala/fasthttp

A high-performance HTTP implementation for Go optimized for extreme throughput and low latency. It provides a faster alternative to net/http for servers and clients handling thousands of requests per second, featuring minimal memory allocation and a specialized prefork package for multi-core scaling.

Tokens
36.8K
Snippets
151
Records
196
Agent score
98%

What's inside fasthttp

  1. Performance comparison: fasthttp vs net/http

    master

    fasthttp is optimized for speed and minimal memory allocation.

    • HTTP Server: The fasthttp server is up to 6 times faster than net/http. Benchmarks show significantly lower operations per second (ns/op) and often zero bytes per operation (0 B/op) and zero allocations (0 allocs/op) compared to net/http.
    • HTTP Client: The fasthttp client is up to 4 times faster than net/http.
  2. Use TCPListen for high-performance TCP listeners

    master

    The tcplisten package provides a customizable net.Listener designed for high-performance scenarios. It allows you to enable specific socket options that can improve scaling and latency on multi-CPU servers.

    Key performance features include:

    • SO_REUSEPORT: Enables linear scaling of server performance on multi-CPU systems by allowing multiple processes or threads to bind to the same port.
    • TCP_DEFER_ACCEPT: Optimizes performance by waiting for the server to read from an accepted connection before notifying the kernel, reducing unnecessary wakeups.
    • TCP_FASTOPEN: Enables TCP Fast Open (TFO) to reduce handshake latency.

    This package is a derivative of go_reuseport.

  3. How prefork improves performance

    master

    Preforking works by splitting the master process into several child processes. This architecture improves throughput and latency by ensuring that Go does not have to share and manage memory between multiple CPU cores within a single process.

    In benchmarks simulating heavy workloads (e.g., a 100ms sleep per request), the prefork implementation showed higher Requests/sec and lower average latency compared to a standard non-prefork fasthttp.Server.

  4. Understand the design trade-offs between fasthttp and net/http

    master

    fasthttp is designed for high performance by minimizing memory allocations, which leads to several key differences from the standard net/http package:

    • Object Reuse: Unlike net/http, which creates new request/response objects per request, fasthttp reuses existing objects to reduce GC pressure.
    • Data Types: fasthttp prefers []byte over string to avoid the memory allocation and copying required for []byte to string conversion. If you need strings, you can wrap results in string(), but be aware of the overhead.
    • Header Storage: fasthttp avoids the map[string][]string structure used by net/http to prevent unnecessary parsing and allocations.
    • Body Handling: fasthttp buffers bodies by default. To prevent untrusted clients from consuming too much memory, use MaxResponseBodySize to bound response sizes. Note that net/http supports streaming by default, whereas fasthttp requires explicit configuration for streaming.
    • API Stability: net/http has a stable API and handles more HTTP corner cases, while fasthttp is optimized for speed and its API evolves more frequently.
  5. Avoid data races with RequestCtx references

    master

    CRITICAL: Do not hold references to RequestCtx

    fasthttp disallows holding references to *fasthttp.RequestCtx or any of its members after the RequestHandler has returned. Doing so will cause inevitable data races because the context is reused for subsequent requests.

    Mitigation Strategies

    1. Use TimeoutError: If you must retain references to RequestCtx or its members (e.g., for asynchronous processing), call ctx.TimeoutError() before returning from the RequestHandler.
    2. Use TimeoutHandler: Wrap your RequestHandler in fasthttp.TimeoutHandler to manage request lifecycles safely.
    3. Race Detector: Always use the Go race detector (go test -race or go run -race) to identify potential leaks or improper usage of the context.
    func main() {
      fasthttp.ListenAndServe(":8080", fasthttp.TimeoutHandler(func(ctx *fasthttp.RequestCtx) {
        select {
        case <-ctx.Done():
          // ctx.Done() is only closed when the server is shutting down.
          log.Println("context cancelled")
          return
        case <-time.After(10 * time.Second):
          log.Println("process finished ok")
        }
      }, time.Second*2, "timeout"))
    }
  6. Choose between Client and HostClient

    master

    When deciding which client abstraction to use, consider your target endpoints:

    1. fasthttp.Client: Use this when working with multiple different hostnames. It manages the lifecycle of multiple HostClient instances for you.
    2. fasthttp.HostClient: Use this when you have a single, heavily loaded API endpoint. A HostClient is more efficient for a single destination as it avoids the overhead of the Client managing multiple host pools.
  7. When to use fasthttp vs net/http

    master

    fasthttp is a high-performance HTTP implementation for Go designed for specific high-performance edge cases.

    Use fasthttp if:

    • Your server or client needs to handle thousands of small to medium requests per second.
    • You require consistent low millisecond response times.
    • You are operating at extreme scale (e.g., serving up to 200K rps with millions of concurrent keep-alive connections).

    Avoid fasthttp if:

    • You want the ease of use and broad compatibility of the standard library.
    • Your application does not have extreme throughput requirements. For most standard use cases, net/http is recommended as it is easier to use and handles more edge cases effectively.
  8. Efficiently work with []byte buffers

    master

    Follow these patterns to simplify code and improve performance when handling byte slices:

    • Omit Nil Checks: Standard Go functions like append, copy, len, and range work correctly with nil buffers. You can safely remove redundant if buf != nil checks.
    • Append Strings to Bytes: You can append a string directly to a []byte using append(dst, "string"...).
    • Extend Capacity: You can extend a slice up to its existing capacity using slicing (e.g., buf[:100] if cap(buf) is 100).
    • Nil-friendly API: Most fasthttp functions (like fasthttp.Get or fasthttp.AppendUint) accept nil as a buffer argument.
    // Example: Safe use of nil buffers
    var src []byte
    srcLen := len(src) // Works even if src is nil
    
    // Example: Appending string to byte slice
    dst = append(dst, "foobar"...)
    
    // Example: fasthttp accepting nil
    statusCode, body, err := fasthttp.Get(nil, "http://google.com/")
  9. Use third-party routers and frameworks with fasthttp

    master

    The core fasthttp package does not include a request router. To implement routing, use one of the following compatible third-party libraries:

  10. Optimize fasthttp for multi-core systems

    master

    To achieve maximum performance on multi-core hardware, follow these optimization strategies:

    • Use reuseport listener: Utilize the reuseport package for better listener handling.
    • Per-core server instances: Run a separate server instance for each CPU core by setting GOMAXPROCS=1.
    • CPU Pinning: Use taskset to pin each server instance to a specific CPU core.
    • Network Interrupt Distribution: Ensure multiqueue network card interrupts are evenly distributed across CPU cores to prevent bottlenecks.
    • Go Version: Always use the latest version of Go to benefit from continuous runtime performance improvements.
  11. Switching from net/http to fasthttp

    master

    fasthttp does not provide an API identical to Go's standard net/http. While a fasthttpadaptor exists to convert net/http handlers, it is recommended to write native fasthttp handlers manually to fully leverage the library's performance advantages.

    Key Differences

    • Handler Type: fasthttp uses RequestHandler functions instead of the http.Handler interface. You can pass bound struct methods to fasthttp.ListenAndServe to maintain state.
    • Request Context: The RequestHandler accepts a single argument, *fasthttp.RequestCtx, which encapsulates all request and response functionality.
    • Response Ordering: Unlike net/http, fasthttp allows you to set headers, status codes, and write the body in any order. The response is not sent to the wire until the RequestHandler returns.
    • Routing: fasthttp does not include a built-in ServeMux. Instead, developers typically use third-party routers like fasthttp-routing, router, or high-level frameworks like Fiber or atreugo.
    type MyHandler struct {
    	foobar string
    }
    
    // request handler in net/http style, i.e. method bound to MyHandler struct.
    func (h *MyHandler) HandleFastHTTP(ctx *fasthttp.RequestCtx) {
    	fmt.Fprintf(ctx, "Hello, world! Requested path is %q. Foobar is %q",
    		ctx.Path(), h.foobar)
    }
    
    myHandler := &MyHandler{
    	foobar: "foobar",
    }
    fasthttp.ListenAndServe(":8080", myHandler.HandleFastHTTP)