DOCSEARCHbyAlgolia
Submit Docs

LuaSocket

repository·master·Indexed 24 days ago

https://github.com/lunarmodules/luasocket

A Lua extension library for network programming providing low-level TCP/UDP transport support via C modules and high-level Internet protocol support (HTTP, FTP, SMTP) via Lua modules. It features the LTN12 functional data processing model using Sources, Sinks, Filters, and Pumps for memory-efficient, chunked data transformations.

Tokens
3.1K
Snippets
10
Records
19
Agent score
84%

What's inside LuaSocket

  1. Overview of LuaSocket

    master

    LuaSocket is a Lua extension library designed for network programming. It is composed of two primary layers:

    1. C Modules: Provide low-level support for the TCP and UDP transport layers.
    2. Lua Modules: Provide high-level functions commonly used by applications interacting with the Internet (such as HTTP, FTP, or SMTP support).

  3. Use Sources to provide data

    master

    A source is a node that provides data to a network of transformations. A source returns the next chunk of data each time it is called. When no more data is available, it returns nil. If an error occurs, it returns nil followed by an error message.

    Sources are compatible with Lua iterators and can be used in for loops. Note that when using a source in a loop, you may need a final call outside the loop to process the last chunk of data produced when the source returns nil.

    local process = normalize("\r\n")
for chunk in source.file(io.stdin) do
    io.write(process(chunk))
end
io.write(process(nil))

  4. How to handle filters that return multiple output chunks

    master

    In the LTN12 design, some filters (like decompression filters) can produce more output data than the size of the input chunk provided. To handle this, you must implement a specific looping pattern:

    1. Processing input chunks: After passing an input chunk to a filter and receiving the first output chunk, you must repeatedly call the filter with an empty string ("") to retrieve any remaining output chunks. When the filter returns an empty string, you have exhausted the current input's output.
    2. Handling end-of-input: When you have no more input data, pass nil to the filter. Because a filter might still have buffered data to output after receiving nil, you must continue calling the filter with nil in a loop until the filter itself returns nil to signal it is completely finished.

    This ensures that even if a filter 'explodes' a small input into a large output, all data is captured.

  5. Understand the LTN12 data processing abstractions

    master

    LTN12 provides a functional framework for data processing using four primary abstractions:

    • Sources: Abstractions for data acquisition (providing data).
    • Sinks: Abstractions for final data destinations (receiving data).
    • Filters: Abstractions for data transformations (taking input and producing output).
    • Pumps: The mechanism that puts the machinery to work by driving the flow of data through the chain.

    These components can be chained together to create complex data transformation pipelines from simple, reusable parts.

  6. Understand the Filter, Source, Sink, and Pump model

    master

    LuaSocket uses a functional approach to data processing based on four core abstractions that allow for memory-efficient, chunked data transformations:

    • Filters: Functions that accept successive chunks of input and produce successive chunks of output. They are used for transformations like Base64 encoding, line normalization, or text breaking. Filters can be chained to create complex composite operations.
    • Sources: Functions that produce new data chunk by chunk (e.g., reading from a file or stdin).
    • Sinks: Functions that act as the final destination for data chunks (e.g., writing to a file or stdout).
    • Pumps: The driving force that moves data through the network, pulling from a Source and pushing it through any intermediate Filters into a Sink.

    This model allows processing data that is too large to fit in memory by handling it in small, manageable pieces.

  7. Use Sinks to consume data

    master

    A sink is a node that acts as the final destination for data. Sinks receive consecutive chunks of data until a nil chunk is received.

    • Error handling: An error is signaled by an extra argument (the error message) following the nil chunk.
    • Flow control: If a sink detects an error or wishes to stop receiving data, it should return nil (optionally followed by an error message). A return value that is not nil indicates the sink will accept more data.
    • Replacement: Sinks can also choose to be replaced by another sink using a similar interface to sources.
    local store, t = sink.table()
while 1 do
    local chunk = load()
    store(chunk)
    if not chunk then break end
end
print(table.concat(t))

  8. How finalized exceptions work in LuaSocket

    master

    LuaSocket utilizes a pattern of "finalized exceptions" to keep code clean and maintainable. This pattern relies on two primary abstractions:

    1. protect(f): A factory that wraps a function f. It catches any errors raised within f (typically via assert) and converts them into the standard Lua return pattern: return nil, error_message. This is used at the top-level of modules to shield users from internal crashes.
    2. newtry(finalizer): A factory that returns a function similar to assert. If the call passed to it fails (returns nil), it executes the provided finalizer function before propagating the error. This is used for mid-level logic where resources like sockets must be closed if a subsequent operation fails.

    By combining these, developers can write linear, readable code that looks like a sequence of successful steps, while the underlying factories handle error propagation and resource management automatically.

  9. Chain multiple filters together

    master

    You can compose multiple primitive filters into a single composite filter using filter.chain(...). This is useful when data must undergo multiple transformations (e.g., normalizing text before encoding it). The resulting composite filter can be used anywhere a single filter is expected.

    When chaining, the system ensures that the 'final chunk' signal (when chunk is nil) is correctly propagated through all filters in the chain to allow them to flush any remaining buffered data.

    local chain = filter.chain(normalize("\r\n"), encode("quoted-printable"))

while 1 do
    local chunk = io.read(2048)
    io.write(chain(chunk))
    if not chunk then break end
end

  10. Use an identity filter for side effects like progress tracking

    master
    An identity filter is a filter that returns received data unaltered. You can use it to inject side effects (like updating a progress bar or counter) into a data pipeline without modifying the original sink. By chaining a sink with an identity filter, the filter can inspect the data chunks as they pass through to provide feedback to the user.

  11. Chain sources and filters for data transformation

    master

    You can combine a source with one or more filters using a chain operation to create a single input stream that performs transformations on the fly.

    Example workflow:

    1. Define a Source (e.g., source.file(io.stdin)).
    2. Chain it with a Filter (e.g., normalize(" ")).
    3. Use a Pump to move data from the resulting chained input to a Sink (e.g., sink.file(io.stdout)).
    input = source.chain(source.file(io.stdin), normalize("\r\n"))
output = sink.file(io.stdout)
pump(input, output)

  12. Chain a sink with a filter

    master

    You can combine a sink and a filter into a single new sink using sink.chain(f, snk). This new sink passes all incoming data through the filter f before handing it to the original sink snk.

    local store = sink.chain(
    wrap(76),
    sink.file(io.open("output.b64", "w"),)
)