lua-resty-limit-traffic

The resty.limit.traffic module allows you to aggregate multiple different limiter instances (such as request rate limiters and connection concurrency limiters) into a single check. This is useful when you want to enforce multiple constraints simultaneously (e.g., limiting by both Host IP and Client IP) without adding significant CPU overhead.

How it works

1. Aggregation: You provide an array of limiter objects and a corresponding array of keys.
2. Delay Calculation: The module calculates the maximum delay required across all limiters. The caller is responsible for calling ngx.sleep(delay).
3. Atomic Rejection: If any single limiter in the set rejects the request, combine ensures the request is not committed in any of the other limiters in the set. It returns nil and the error string "rejected".
4. State Tracking: You can optionally provide a states table to capture metadata from the limiters (e.g., current concurrency levels or excess request rates).

Requirements for Limiter Objects

To be compatible with combine, a limiter object must implement:

• incoming(key, commit): Returns delay, state_or_error. If delay is nil, the second return value is an error string. If delay is not nil, the second value is an opaque state value.
• uncommit(key): Used to undo a committed incoming call if the overall combination fails or needs adjustment.

http {
    lua_shared_dict my_req_store 100m;
    lua_shared_dict my_conn_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_conn = require "resty.limit.conn"
                local limit_req = require "resty.limit.req"
                local limit_traffic = require "resty.limit.traffic"

                -- Initialize limiters
                local lim1, err = limit_req.new("my_req_store", 300, 200)
                assert(lim1, err)
                local lim2, err = limit_req.new("my_req_store", 200, 100)
                assert(lim2, err)
                local lim3, err = limit_conn.new("my_conn_store", 1000, 1000, 0.5)
                assert(lim3, err)

                local limiters = {lim1, lim2, lim3}
                local host = ngx.var.host
                local client = ngx.var.binary_remote_addr
                local keys = {host, client, client}
                local states = {}

                -- Combine limiters
                local delay, err = limit_traffic.combine(limiters, keys, states)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit traffic: ", err)
                    return ngx.exit(500)
                end

                -- Handle connection limiting cleanup in log_by_lua
                if lim3:is_committed() then
                    local ctx = ngx.ctx
                    ctx.limit_conn = lim3
                    ctx.limit_conn_key = keys[3]
                end

                if delay >= 0.001 then
                    ngx.sleep(delay)
                end
            }

            log_by_lua_block {
                local ctx = ngx.ctx
                local lim = ctx.limit_conn
                if lim then
                    local latency = tonumber(ngx.var.request_time)
                    local key = ctx.limit_conn_key
                    assert(key)
                    local conn, err = lim:leaving(key, latency)
                    if not conn then
                        ngx.log(ngx.ERR, "failed to record the connection leaving ", "request: ", err)
                        return
                    end
                end
            }
         }
    }
}

The resty.limit.conn module allows you to limit the number of concurrent requests (or connections) in OpenResty. Unlike the standard NGINX ngx_limit_conn module, this Lua module supports connection delaying: instead of immediately rejecting all requests that exceed the limit, it can delay requests that fall within a defined burst range to smooth out traffic spikes.

Implementation Pattern

To use this module correctly, you must follow a specific lifecycle across different NGINX phases:

1. access_by_lua phase: Call incoming(key, true) to record the connection. If a delay is returned, use ngx.sleep(delay) to pause the request. If the error is "rejected", return a 503 status.
2. log_by_lua phase: Call leaving(key, latency) to decrement the concurrency counter. This ensures the counter is updated even if the request finishes normally or with an error.

Note: Always check is_committed() after incoming. Only call leaving if is_committed() returned true.

http {
    lua_shared_dict my_limit_conn_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_conn = require "resty.limit.conn"
                -- conn=200, burst=100, default_delay=0.5s
                local lim, err = limit_conn.new("my_limit_conn_store", 200, 100, 0.5)
                if not lim then
                    ngx.log(ngx.ERR, "failed to instantiate: ", err)
                    return ngx.exit(500)
                end

                local key = ngx.var.binary_remote_addr
                local delay, err = lim:incoming(key, true)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit req: ", err)
                    return ngx.exit(500)
                end

                if lim:is_committed() then
                    local ctx = ngx.ctx
                    ctx.limit_conn = lim
                    ctx.limit_conn_key = key
                    ctx.limit_conn_delay = delay
                end

                if delay >= 0.001 then
                    ngx.sleep(delay)
                end
            }

            log_by_lua_block {
                local ctx = ngx.ctx
                local lim = ctx.limit_conn
                if lim then
                    local latency = tonumber(ngx.var.request_time) - ctx.limit_conn_delay
                    local key = ctx.limit_conn_key
                    assert(key)
                    local conn, err = lim:leaving(key, latency)
                    if not conn then
                        ngx.log(ngx.ERR, "failed to record leaving: ", err)
                        return
                    end
                end
            }
        }
    }
}

The resty.limit.req module allows you to limit the request rate using a "leaky bucket" algorithm. You can configure a base rate (requests per second) and a burst capacity. Requests exceeding the rate but within the burst limit are delayed, while requests exceeding rate + burst are rejected.

To use it, you must define a lua_shared_dict in your NGINX configuration to store the limiting state. In your Lua code, you instantiate the limiter and call :incoming(key, commit) for every request. If a delay is returned, you must manually call ngx.sleep(delay) to enforce the limit.

http {
    lua_shared_dict my_limit_req_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_req = require "resty.limit.req"

                -- rate: 200 req/sec, burst: 100 req/sec
                local lim, err = limit_req.new("my_limit_req_store", 200, 100)
                if not lim then
                    ngx.log(ngx.ERR, "failed to instantiate: ", err)
                    return ngx.exit(500)
                end

                local key = ngx.var.binary_remote_addr
                local delay, err = lim:incoming(key, true)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit req: ", err)
                    return ngx.exit(500)
                end

                if delay > 0 then
                    ngx.sleep(delay)
                end
            }
        }
    }
}

The resty.limit.count module allows you to limit the number of requests allowed within a fixed time window (similar to the GitHub API rate limiting model). It uses an NGINX lua_shared_dict to share state across all NGINX worker processes on a single server instance.

To use it, you must:

1. Define a lua_shared_dict in your NGINX configuration.
2. Enable lua-resty-core in the init_by_lua_block.
3. Instantiate the limiter using new().
4. Call incoming() with a unique key (e.g., an IP address or an Authorization header) to check and record the request.

http {
    lua_shared_dict my_limit_count_store 100m;

    init_by_lua_block {
        require "resty.core"
    }

    server {
        location / {
            access_by_lua_block {
                local limit_count = require "resty.limit.count"

                -- rate: 5000 requests per 3600s
                local lim, err = limit_count.new("my_limit_count_store", 5000, 3600)
                if not lim then
                    ngx.log(ngx.ERR, "failed to instantiate a resty.limit.count object: ", err)
                    return ngx.exit(500)
                end

                -- use the Authorization header as the limiting key
                local key = ngx.req.get_headers()["Authorization"] or "public"
                local delay, err = lim:incoming(key, true)

                if not delay then
                    if err == "rejected" then
                        ngx.header["X-RateLimit-Limit"] = "5000"
                        ngx.header["X-RateLimit-Remaining"] = 0
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit count: ", err)
                    return ngx.exit(500)
                end

                -- the 2nd return value holds the current remaining number
                local remaining = err

                ngx.header["X-RateLimit-Limit"] = "5000"
                ngx.header["X-RateLimit-Remaining"] = remaining
            }
        }
    }
}

The resty.limit.traffic module allows you to aggregate multiple limiters (from req, count, or conn modules) into a single traffic control mechanism.

To use it:

1. Create an array of limiter instances.
2. Create an array of keys (one key per limiter instance).
3. Create an empty states table.
4. Call limit_traffic.combine(limiters, keys, states). This returns a delay and an err.
5. If err == "rejected", return a 503 status code.
6. If a delay is returned, use ngx.sleep(delay) to throttle the request.
7. If using a concurrency limiter (conn) within the combined set, ensure you still handle the is_committed() check and the leaving() call in log_by_lua to release the connection.

http {
    lua_shared_dict my_req_store 100m;
    lua_shared_dict my_conn_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_conn = require "resty.limit.conn"
                local limit_req = require "resty.limit.req"
                local limit_traffic = require "resty.limit.traffic"

                local lim1 = limit_req.new("my_req_store", 300, 200)
                local lim2 = limit_req.new("my_req_store", 200, 100)
                local lim3 = limit_conn.new("my_conn_store", 1000, 1000, 0.5)

                local limiters = {lim1, lim2, lim3}
                local host = ngx.var.host
                local client = ngx.var.binary_remote_addr
                local keys = {host, client, client}
                local states = {}

                local delay, err = limit_traffic.combine(limiters, keys, states)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit traffic: ", err)
                    return ngx.exit(500)
                end

                if lim3:is_committed() then
                    local ctx = ngx.ctx
                    ctx.limit_conn = lim3
                    ctx.limit_conn_key = keys[3]
                end

                if delay >= 0.001 then
                    ngx.sleep(delay)
                end
            }

            log_by_lua_block {
                local ctx = ngx.ctx
                local lim = ctx.limit_conn
                if lim then
                    local latency = tonumber(ngx.var.request_time)
                    local key = ctx.limit_conn_key
                    assert(key)
                    local conn, err = lim:leaving(key, latency)
                    if not conn then
                        ngx.log(ngx.ERR, "failed to record leaving: ", err)
                        return
                    end
                end
            }
        }
    }
}

The resty.limit.req module implements rate limiting using a leaky bucket algorithm.

To use it:

1. Define a lua_shared_dict in your NGINX configuration to store the state.
2. Instantiate the limiter using limit_req.new(store, rate, burst).
3. Call lim:incoming(key, true) per request.
   • If it returns a delay, use ngx.sleep(delay) to throttle the request.
   • If it returns an error "rejected", return a status code like 503 to reject the request.
   • The second return value from incoming is the err variable, which contains the number of excess requests per second if a delay is applied.

http {
    lua_shared_dict my_limit_req_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_req = require "resty.limit.req"
                -- rate: 200 req/sec, burst: 100 req/sec
                local lim, err = limit_req.new("my_limit_req_store", 200, 100)
                if not lim then
                    ngx.log(ngx.ERR, "failed to instantiate: ", err)
                    return ngx.exit(500)
                end

                local key = ngx.var.binary_remote_addr
                local delay, err = lim:incoming(key, true)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit req: ", err)
                    return ngx.exit(500)
                end

                if delay >= 0.001 then
                    ngx.sleep(delay)
                end
            }
        }
    }
}

The resty.limit.conn module limits the number of concurrent requests.

To use it:

1. Define a lua_shared_dict in your NGINX configuration.
2. Instantiate the limiter using limit_conn.new(store, capacity, burst, default_request_time).
3. In the access_by_lua phase, call lim:incoming(key, true). If lim:is_committed() is true, store the limiter and key in ngx.ctx so you can clean up in the log_by_lua phase.
4. In the log_by_lua phase, calculate the latency (e.g., ngx.var.request_time minus the delay recorded in access_by_lua) and call lim:leaving(key, latency) to release the connection slot.

http {
    lua_shared_dict my_limit_conn_store 100m;

    server {
        location / {
            access_by_lua_block {
                local limit_conn = require "resty.limit.conn"
                -- capacity: 200, burst: 100, default_request_time: 0.5
                local lim, err = limit_conn.new("my_limit_conn_store", 200, 100, 0.5)
                if not lim then
                    ngx.log(ngx.ERR, "failed to instantiate: ", err)
                    return ngx.exit(500)
                end

                local key = ngx.var.binary_remote_addr
                local delay, err = lim:incoming(key, true)
                if not delay then
                    if err == "rejected" then
                        return ngx.exit(503)
                    end
                    ngx.log(ngx.ERR, "failed to limit req: ", err)
                    return ngx.exit(500)
                end

                if lim:is_committed() then
                    local ctx = ngx.ctx
                    ctx.limit_conn = lim
                    ctx.limit_conn_key = key
                    ctx.limit_conn_delay = delay
                end

                if delay >= 0.001 then
                    ngx.sleep(delay)
                end
            }

            log_by_lua_block {
                local ctx = ngx.ctx
                local lim = ctx.limit_conn
                if lim then
                    local latency = tonumber(ngx.var.request_time) - ctx.limit_conn_delay
                    local key = ctx.limit_conn_key
                    assert(key)
                    local conn, err = lim:leaving(key, latency)
                    if not conn then
                        ngx.log(ngx.ERR, "failed to record leaving: ", err)
                        return
                    end
                end
            }
        }
    }
}