DOCSEARCHbyAlgolia
Submit Docs

ulule/limiter

repository·master·Indexed 25 days ago

https://github.com/ulule/limiter

A Go library providing rate-limiting capabilities designed for use as middleware in web frameworks such as Gin, Echo, Chi, Beego, fasthttp, and the standard library. It supports both Redis and In-Memory storage backends and includes features for handling client IPs behind reverse proxies via custom headers and IP masking.

Tokens
3.7K
Snippets
11
Records
25
Agent score
77%

What's inside ulule-limiter

  1. Limiter storage mechanisms: Redis vs In-Memory

    master

    Limiter supports two primary storage drivers:

    • Redis: Relies on Redis TTL (Time To Live) and increments the rate limit on each request. This is ideal for distributed systems where multiple application instances need to share the same rate limit state.
    • In-Memory: Uses a local cache with a background goroutine to clear expired keys. This is faster and simpler but only works for a single application instance (local to that process).

  2. Handling Client IPs behind a reverse proxy

    master

    When running behind a reverse proxy (like AWS ALB), the request IP might be the proxy's IP rather than the actual client's. To fix this, you must explicitly tell the limiter which header to trust.

    Using X-Forwarded-For

    If your reverse proxy is configured to overwrite/remove existing X-Forwarded-For or X-Real-IP headers from the incoming request (to prevent spoofing), you can enable TrustForwardHeader in your limiter options.

    Warning: If your proxy merely appends to these headers, they are untrustworthy and you should keep TrustForwardHeader disabled.

    Using Custom Headers

    Many CDNs provide a specific header for the real client IP. You can use these by passing the header name to limiter.WithClientIPHeader.

    Common examples:

    • Fastly-Client-IP (Fastly)
    • CF-Connecting-IP (Cloudflare)
    • X-Azure-ClientIP (Azure)

    Custom Key Logic

    If none of the above methods work, you can implement a custom KeyGetter within your middleware to define your own strategy for identifying clients.

  4. View middleware examples for different frameworks

    master

    The official examples for integrating the limiter middleware with various Go web frameworks have been moved to a dedicated repository. You can find implementation examples for the following frameworks there:

    • HTTP (Standard library)
    • Gin
    • Beego
    • Chi
    • Echo
    • fasthttp

  5. How to use Limiter in your application

    master

    To implement rate limiting, follow these five steps:

    1. Create a limiter.Rate instance: Define the number of requests allowed per period.
    2. Create a limiter.Store instance: Choose a backend like Redis or In-Memory.
    3. Create a limiter.Limiter instance: Combine the store and rate.
    4. Create a middleware instance: Select the middleware compatible with your web framework (e.g., stdlib, gin, fasthttp).
    5. Initialize the middleware: Pass the limiter instance to the middleware initializer.

    When the limit is reached, the middleware returns a 429 HTTP status code.

    // 1. Create a rate
rate := limiter.Rate{
    Period: 1 * time.Hour,
    Limit:  1000,
}

// Alternatively, use a formatted string like "1000-H"
rate, err := limiter.NewRateFromFormatted("1000-H")
if err != nil {
    panic(err)
}

// 2. Create a store (e.g., Redis)
store, err := redis.NewStore(client)
if err != nil {
    panic(err)
}

// Or an in-memory store
// store := memory.NewStore()

// 3. Create the limiter instance
instance := limiter.New(store, rate)

// 4 & 5. Create and use the middleware
middleware := stdlib.NewMiddleware(instance)

  6. Handle IP addresses behind a reverse proxy

    master

    When running behind a reverse proxy, CDN, or Cloud provider, the direct connection IP might be the proxy's IP rather than the client's. You can configure the limiter to extract the real client IP using headers.

    Options

    • WithTrustForwardHeader(enable bool): When true, the limiter parses X-Real-IP and X-Forwarded-For headers.
    • WithClientIPHeader(header string): Use this to specify a specific custom header provided by your infrastructure. This takes precedence over WithTrustForwardHeader.

    Security Warning: Using these options can allow users to spoof their IP address if your reverse proxy is not configured to strip these headers from incoming client requests before adding its own.

  7. Configure the In-Memory Store

    master

    The In-Memory store uses a fork of go-cache and runs a background goroutine to clear expired keys at a default interval. This is suitable for single-instance applications where distributed state is not required.

    import "github.com/ulule/limiter/v3/drivers/store/memory"

store := memory.NewStore()

  8. Configure the Redis Store

    master

    The Redis store is a built-in driver for limiter.Store. By default, it uses limiter as the Redis key prefix and allows up to 3 retries under race conditions.

    You can customize the store using redis.NewStoreWithOptions to provide a custom Prefix via limiter.StoreOptions.

    import "github.com/ulule/limiter/v3/drivers/store/redis"

// Basic initialization
store, err := redis.NewStore(client)
if err != nil {
    panic(err)
}

// Initialization with options
store, err := redis.NewStoreWithOptions(pool, limiter.StoreOptions{
    Prefix: "your_own_prefix",
})
if err != nil {
    panic(err)
}

  9. Configure Rate using formatted strings

    master

    You can create a limiter.Rate using the limiter.NewRateFromFormatted function with a simplified string format: <limit>-<period>.

    Supported period suffixes:

    • S: second
    • M: minute
    • H: hour
    • D: day

    Examples:

    • 5-S (5 requests per second)
    • 10-M (10 requests per minute)
    • 1000-H (1000 requests per hour)
    • 2000-D (2000 requests per day)
    rate, err := limiter.NewRateFromFormatted("1000-H")
if err != nil {
    panic(err)
}

  10. Configure Limiter options

    master

    When creating a limiter.Limiter instance with limiter.New, you can pass functional options to customize behavior:

    • limiter.WithClientIPHeader(string): Specifies a custom header to use for identifying the client IP (e.g., for CDNs).
    • limiter.WithIPv6Mask(mask): Applies an IPv6 mask to the client IP.
    instance := limiter.New(store, rate, limiter.WithClientIPHeader("True-Client-IP"), limiter.WithIPv6Mask(mask))

  11. Configure StoreOptions

    master

    When initializing a store, you can use StoreOptions to customize its behavior.

    • Prefix: A string prepended to all keys used by the store.
    • CleanUpInterval: Specifically for the memory store, this defines how often garbage collection runs on stale entries.
      • Low value: Optimizes memory consumption but may reduce performance and increase lock contention.
      • High value: Maximizes throughput but increases the memory footprint.
    type StoreOptions struct {
	// Prefix is the prefix to use for the key.
	Prefix string

	// MaxRetry is the maximum number of retry under race conditions on redis store.
	// Deprecated: this option is no longer required since all operations are atomic now.
	MaxRetry int

	// CleanUpInterval is the interval for cleanup (run garbage collection) on stale entries on memory store.
	CleanUpInterval time.Duration
}

  12. Configure limiter options using functional options

    master

    The limiter package uses the functional options pattern to configure its behavior. You can pass one or more Option functions to the limiter constructor (typically New or similar, though not shown in this file) to customize IP mask handling and header trust settings.

    Available configuration options include:

    • WithIPv4Mask(mask net.IPMask): Sets the mask used to obtain an IPv4 address.
    • WithIPv6Mask(mask net.IPMask): Sets the mask used to obtain an IPv6 address.
    • WithTrustForwardHeader(enable bool): Enables parsing of X-Real-IP and X-Forwarded-For headers to obtain the user's IP. Warning: This can be insecure if your reverse proxy is not configured to prevent IP spoofing.
    • WithClientIPHeader(header string): Specifies a custom header (e.g., from a CDN or Cloud provider) to obtain the user's IP. This overrides WithTrustForwardHeader if both are set.