lua-nginx-module (ngx_http_lua_module)

The ngx_http_lua_module embeds LuaJIT 2.0/2.1 into Nginx HTTP servers. It is a core component of OpenResty.

Key Features

• Non-blocking I/O: By leveraging Nginx's subrequests and Lua coroutines, Lua code can be 100% non-blocking on network traffic when using the provided Nginx API for Lua to handle upstream services (MySQL, PostgreSQL, Redis, etc.).
• Performance: Provides performance levels comparable to native C programs due to LuaJIT.
• Memory Efficiency: The Lua interpreter (Lua State) is shared across all requests in a single Nginx worker process, and request contexts are segregated using lightweight coroutines.

Scope

This module is plugged into the Nginx http subsystem. It handles downstream communication protocols in the HTTP family (HTTP 0.9/1.0/1.1/2.0, WebSockets, etc.). For generic TCP communications, use the ngx_stream_lua module instead.

The ngx.thread.spawn(func, arg1, arg2, ...) function creates a "light thread" (a specialized Lua coroutine) that is scheduled by the ngx_lua module. This allows for concurrent execution of Lua functions within Nginx.

Key Characteristics

• Scheduling: Light threads are not preemptive. A thread runs until it hits a non-blocking I/O operation that cannot be completed, calls coroutine.yield(), or is aborted by an error/exit command.
• Lifecycle: When ngx.thread.spawn returns, the thread runs asynchronously. The Nginx handler will not terminate until the "entry thread" and all spawned "light threads" have terminated.
• Error Handling: If a user light thread terminates with a Lua error, it does not abort the entry thread or other running light threads.
• Waiting: A parent coroutine can use ngx.thread.wait(co) to wait for a child light thread to terminate.
• Zombie State: A thread enters a "zombie" state if it has terminated but its parent is still alive and not waiting on it.

Use Case: Concurrent Upstream Requests

Light threads are ideal for making multiple concurrent requests (e.g., to MySQL, Memcached, and an HTTP service) simultaneously to reduce total latency.

Contexts: rewrite_by_lua*, access_by_lua*, content_by_lua*, ngx.timer.*, ssl_certificate_by_lua*.

-- Example: Running multiple tasks concurrently
local function task1()
    -- do something
end

local function task2()
    -- do something else
end

ngx.thread.spawn(task1)
ngx.thread.spawn(task2)

When issuing subrequests, you can control how Nginx variables are handled using three primary options:

1. copy_all_vars: Creates a copy of the parent request's variables for the subrequest. Modifications in the subrequest do not affect the parent.
2. vars: A more efficient way to set specific variables for the subrequest. These are applied after sharing/copying logic.
3. share_all_vars: Shares the exact same variable scope. Modifications in the subrequest will affect the parent request. Warning: This is considered harmful and can lead to hard-to-debug side effects.

Note: If both share_all_vars and copy_all_vars are set to true, share_all_vars takes precedence.

# Example: Using share_all_vars
location /lua {
    content_by_lua_block {
        local res = ngx.location.capture("/other", { share_all_vars = true })
        ngx.print(res.body)
        ngx.say(ngx.var.uri, ": ", ngx.var.dog)
    }
}

# Example: Using vars for specific values
location /lua {
    content_by_lua_block {
        local res = ngx.location.capture("/other", { vars = { dog = "hello", cat = 32 }})
        ngx.print(res.body)
    }
}

The ngx.socket.tcp() cosocket object is designed to be compatible with the LuaSocket TCP API but is 100% non-blocking.

Lifetime and Ownership

• Ownership: A cosocket object has the same lifetime as the Lua handler that created it.
• Restriction: Never pass a cosocket object to another Lua handler (including ngx.timer callbacks) and never share it between different Nginx requests.
• Automatic Closure: If you do not explicitly call close() or setkeepalive(), the connection is automatically closed when:
  1. The current request handler completes.
  2. The Lua cosocket object is collected by the Lua Garbage Collector (GC).

Concurrency Model

• Full-Duplex: Since version 0.9.9, cosockets are full-duplex. A reader "light thread" and a writer "light thread" can operate on a single cosocket simultaneously, provided both belong to the same Lua handler.
• Concurrency Limits: You cannot have two light threads both performing the same operation (e.g., two readers, two writers, or two connectors) on the same cosocket. Doing so results in a "socket busy reading" (or writing) error.

Timers in OpenResty run in a "fake request" context (a detached request context). This has several implications for developers:

1. Memory Management: Because Nginx releases memory based on connection closure, running APIs that allocate memory (like tcpsock:connect) inside a timer can cause memory accumulation. It is recommended to create a new timer after running several times to allow memory to be released.
2. Context Isolation: The timer handler has its own copy of the ngx.ctx magic table and does not share the ngx.ctx of the Lua handler that created it. Use the extra parameters of ngx.timer.at() to pass data to the handler.
3. Forbidden Objects: You cannot pass thread objects (from coroutine.create or ngx.thread.spawn) or cosocket objects (from ngx.socket.tcp, ngx.socket.udp, or ngx.req.socket) into a timer callback. These objects are bound to the request context that created them. Attempting to share them across the boundary will result in:
   • "no co ctx found" error for threads.
   • "bad request" error for cosockets.

Best Practice: Create all thread and cosocket objects inside the timer callback itself.

To share data globally among all requests handled by the same Nginx worker process, encapsulate the data within a Lua module and use the require builtin. Because required modules are loaded only once, all coroutines within that worker will share the same module instance and its data.

Important Constraints:

• Per-Worker Scope: Data sharing is limited to the individual worker process. It cannot cross process boundaries to other workers.
• Race Conditions: Avoid using global Lua variables. For changeable data, ensure there are no non-blocking I/O operations (including ngx.sleep) during calculations. If you do not yield control back to the Nginx event loop, race conditions are prevented.
• Recommendation: Use this pattern primarily for read-only data.

Server-wide Data Sharing Alternatives:

If you need to share data across all workers/processes, use:

1. ngx.shared.DICT API.
2. A single Nginx worker/server configuration (not recommended for multi-core systems).
3. External storage like memcached, redis, MySQL, or PostgreSQL.

-- mydata.lua
local _M = {}

local data = {
     dog = 3,
     cat = 4,
     pig = 5,
 }

function _M.get_age(name)
     return data[name]
end

return _M

location /lua {
     content_by_lua_block {
         local mydata = require "mydata"
         ngx.say(mydata.get_age("dog"))
     }
}

Developers can run the test suite locally using the provided utility script.

Quickstart Setup

git clone https://github.com/openresty/lua-nginx-module.git
cd lua-nginx-module
bash util/run-ci.sh

Manual Testing with prove

To run the full test suite in default mode, ensure your Nginx sbin is in your PATH and use prove with the test-nginx library:

cd /path/to/lua-nginx-module
export PATH=/path/to/your/nginx/sbin:$PATH
prove -I/path/to/test-nginx/lib -r t

To run specific test files:

prove -I/path/to/test-nginx/lib t/002-content.t t/003-errors.t

Test Dependencies

Running the suite requires:

• Nginx version >= 1.4.2
• Perl module Test::Nginx
• Various Nginx modules (e.g., ngx_devel_kit, ngx_echo, ngx_headers_more, ngx_drizzle, etc.)
• 3rd-party Lua libraries: lua-cjson
• Applications: mysql (db ngx_test), memcached (port 11211), and redis (port 6379).