Glommio Documentation

repository·master·Indexed 25 days ago

https://github.com/datadog/glommio

Glommio is a high-performance, thread-per-core asynchronous runtime for Rust on Linux, powered by io_uring. It minimizes context switching and thread overhead by avoiding helper threads. The library provides asynchronous file I/O via BufferedFile and DmaFile, as well as specialized stream readers and writers for linear I/O. It requires Rust 1.70+, Linux kernel 5.8+, and a minimum of 512 KiB of locked memory.

Tokens
12.1K
Snippets
34
Records
63
Agent score
86%

What's inside glommio

  1. Overview of Glommio

    master
    Glommio is a Cooperative Thread-per-Core crate for Rust and Linux built on io_uring. It allows writing asynchronous code using Rust's async/await syntax without using helper threads, optimizing performance by leveraging the thread-per-core model.
  2. Run a basic Glommio async application

    master

    To use Glommio, import the glommio::prelude::* and use LocalExecutorBuilder to spawn a local executor. The executor runs your asynchronous code on the current thread.

    use glommio::prelude::*;
    
    LocalExecutorBuilder::default().spawn(|| async move {
        /// your async code here
    })
    .expect("failed to spawn local executor")
    .join();
  3. Configure system requirements for Glommio

    master

    Glommio requires specific system configurations to function correctly:

    Rust Version

    • Minimum supported version: 1.70

    Linux Kernel

    • Requires io_uring support. Minimum kernel version: 5.8

    Memory Limits

    Glommio requires at least 512 KiB of locked memory for io_uring to work. This is the minimum required to spawn a single executor; spawning multiple executors may require a higher limit.

  4. Increase memlock resource limit (rlimit)

    master

    To ensure io_uring works with Glommio, you must increase the memlock resource limit.

    1. Edit /etc/security/limits.conf and add the following lines:
    *    hard    memlock        512
    *    soft    memlock        512
    1. Log out and log back in for changes to take effect.

    2. Verify the limit by running:

    $ ulimit -l
    $ vi /etc/security/limits.conf
    *    hard    memlock        512
    *    soft    memlock        512
    
    # After re-logging, verify:
    $ ulimit -l
  5. Use BufferedFile for asynchronous file I/O

    master
    The BufferedFile struct provides an asynchronously accessed file backed by the OS page cache. All operations, including opening and closing, are asynchronous and use buffered I/O for efficiency. This is suitable for general-purpose file operations where alignment is not a strict requirement (unlike DmaFile).
  6. Configure system memlock limits for io_uring

    master

    Glommio requires at least 512 KiB of locked memory to function correctly with io_uring. You must adjust the memlock resource limit in /etc/security/limits.conf.

    # Add to /etc/security/limits.conf
    *    hard    memlock        512
    *    soft    memlock        512

    After logging in again, verify the limit with ulimit -l.

    $ vi /etc/security/limits.conf
    *    hard    memlock        512
    *    soft    memlock        512
    
    $ ulimit -l
    512
  7. Create and use an ImmutableFile

    master

    An ImmutableFile provides Direct I/O enabled, read-only access to a file. This allows Glommio to apply optimizations like caching and request coalescing. You can create an ImmutableFile in two ways:

    1. From a new file (Write then Read): Use ImmutableFileBuilder::new(path).build_sink() to get an ImmutableFilePreSealSink. Write data to it, then call .seal() to convert it into an ImmutableFile for reading.
    2. From an existing file (Read only): Use ImmutableFileBuilder::new(path).build_existing() to open an existing file directly as an ImmutableFile.

    Note: Glommio cannot guarantee the file isn't modified by external processes; external modifications lead to undefined behavior.

  8. Manage task scheduling with Task Queues and Shares

    master

    Glommio allows fine-grained scheduling within a single thread using task queues. You can create task queues with specific Shares to define how much CPU time they receive relative to other queues.

    • Shares::Static(n): Assigns a fixed number of shares to a queue.
    • Latency: Defines if a queue is latency-sensitive (Latency::Matters(Duration)) or not (Latency::NotImportant).

    Use glommio::spawn_local_into to run a task within a specific queue.

    use glommio::{executor, Latency, LocalExecutorBuilder, Placement, Shares};
    
    LocalExecutorBuilder::new(Placement::Fixed(0))
        .spawn(|| async move {
            let tq1 = 
                executor().create_task_queue(Shares::Static(2), Latency::NotImportant, "test1");
            let tq2 = 
                executor().create_task_queue(Shares::Static(1), Latency::NotImportant, "test2");
            
            let t1 = glommio::spawn_local_into(
                async move {
                    // your code here
                },
                tq1,
            )
            .unwrap();
            let t2 = glommio::spawn_local_into(
                async move {
                    // your code here
                },
                tq2,
            )
            .unwrap();
    
            t1.await;
            t2.await;
        })
        .unwrap();
  9. Load balance connections across executors with shared_accept

    master

    If you want to perform manual load balancing instead of relying on the OS ReusePort behavior, use TcpListener::shared_accept. This returns an AcceptedTcpStream, which implements Send and can be passed to a different executor via a channel. Once received by the target executor, call bind_to_executor() to convert it into a standard TcpStream.

    use glommio::{
        channels::shared_channel,
        net::TcpListener,
        LocalExecutor,
        LocalExecutorBuilder,
    };
    
    let ex = LocalExecutor::default();
    ex.run(async move {
        let (sender, receiver) = shared_channel::new_bounded(1);
        let sender = sender.connect().await;
    
        let listener = TcpListener::bind("127.0.0.1:0").unwrap();
    
        // Accept connection and send it to another executor
        let accepted = listener.shared_accept().await.unwrap();
        sender.try_send(accepted).unwrap();
    
        let ex1 = LocalExecutorBuilder::default()
            .spawn(move || async move {
                let receiver = receiver.connect().await;
                let accepted = receiver.recv().await.unwrap();
                // Bind the stream to this new executor
                let _stream = accepted.bind_to_executor();
            })
            .unwrap();
    
        ex1.join().unwrap();
    });
  10. Pin an executor to a specific CPU

    master

    To implement a thread-per-core architecture, you can bind an executor to a specific CPU core using LocalExecutorBuilder::new(Placement::Fixed(N)), where N is the CPU index. This minimizes context switching and increases efficiency.

    /// This will now never leave CPU 0
    use glommio::{LocalExecutorBuilder, Placement};
    
    LocalExecutorBuilder::new(Placement::Fixed(0))
        .spawn(|| async move {
            // your code here
        })
        .unwrap();
  11. Load balance Unix connections across executors using AcceptedUnixStream

    master

    If you want to perform custom load balancing instead of relying on OS-level ReusePort, use UnixListener::shared_accept(). This returns an AcceptedUnixStream which can be sent across threads/executors. To use the connection on a new executor, call bind_to_executor() on the received object.

    use glommio::{
        channels::shared_channel,
        net::UnixListener,
        LocalExecutor,
        LocalExecutorBuilder,
    };
    
    let ex = LocalExecutor::default();
    ex.run(async move {
        let (sender, receiver) = shared_channel::new_bounded(1);
        let sender = sender.connect().await;
    
        let listener = UnixListener::bind("/tmp/named").unwrap();
    
        // Accept connection and send it to another executor
        let accepted = listener.shared_accept().await.unwrap();
        sender.try_send(accepted).unwrap();
    
        let ex1 = LocalExecutorBuilder::default()
            .spawn(move || async move {
                let receiver = receiver.connect().await;
                let accepted = receiver.recv().await.unwrap();
                // Bind the connection to the new executor
                let _stream = accepted.bind_to_executor();
            })
            .unwrap();
    
        ex1.join().unwrap();
    });
  12. Initialize a Glommio LocalExecutor

    master

    To run asynchronous code in Glommio, use LocalExecutorBuilder. By default, it creates an executor that runs on the current thread. You can use .spawn() to run an async block.

    Note: Glommio is Linux-only and requires a kernel version 5.8 or newer. It also requires at least 512 KiB of locked memory (memlock).

    use glommio::LocalExecutorBuilder;
    
    LocalExecutorBuilder::default()
        .spawn(|| async move {
            // your code here
        })
        .unwrap();