RedBPF Toolchain

repository·main·Indexed 23 days ago

https://github.com/foniod/redbpf

A Rust-based toolchain for developing eBPF programs, enabling the creation of both kernel-space BPF programs and userspace management programs using idiomatic Rust. The toolchain includes the redbpf userspace library, redbpf-probes for kernel-context programs, redbpf-macros for defining maps and probes (kprobes, uprobes, tracepoints, XDP, and socket filters), and the cargo-bpf subcommand for building, debugging, and loading eBPF programs.

Tokens
21.5K
Snippets
46
Records
122
Agent score
82%

What's inside RedBPF

  1. Overview of the RedBPF toolchain

    main

    RedBPF is a Rust-based eBPF toolchain designed to allow developers to write both BPF programs (kernel context) and userspace programs entirely in Rust. The project consists of several key components:

    • redbpf: A userspace library for loading eBPF programs and accessing eBPF maps.
    • redbpf-probes: An idiomatic Rust API for writing eBPF programs that run in the Linux kernel.
    • redbpf-macros: A companion crate for redbpf-probes providing procedural macros like #[map] for defining maps and #[kprobe] for attaching programs to kernel functions.
    • cargo-bpf: A Cargo subcommand used to create, build, and debug eBPF programs.
  2. Core Concepts of RedBPF

    main

    RedBPF operates using two distinct components:

    • BPF Program: A single Rust function defined using redbpf-macros. It is attached to instrumentation points (e.g., kprobe, xdp, tracepoint) and executes in the kernel context. Because it runs in the kernel, it must use #![no_std] and #![no_main].
    • BPF Maps: Used for communication between BPF programs and userspace programs. Common types include HashMap, Array, and PerfMap (used for delivering events to userspace).
    • Userspace Program: A standard Rust program (using the redbpf crate) that loads BPF programs and maps into the kernel and communicates with them via maps.

    Key Crates:

    • redbpf-macros: Provides attribute macros (#[kprobe], #[map], program!) for defining BPF programs and maps.
    • redbpf-probes: Provides the API for BPF programs executing in the kernel context.
    • redbpf: Provides the API for userspace programs to load and interact with BPF programs.
  3. Understand LLVM version compatibility in RedBPF

    main

    Compiling BPF programs with RedBPF involves two different LLVM versions, and their relationship is critical to avoid errors:

    1. LLVM (1): The version statically linked into cargo-bpf when it was built. This version is used to parse LLVM bitcode and convert it to BPF bytecode.
    2. LLVM (2): The version linked to rustc (the version used by your Rust compiler). This version is used to emit LLVM bitcode from your Rust code.

    Compatibility Rule: LLVM (1) must be greater than or equal to LLVM (2).

    If LLVM (1) is older than LLVM (2), cargo-bpf may not be able to handle the bitcode emitted by the newer rustc. If LLVM (1) is newer than LLVM (2), it typically maintains backward compatibility for the intermediate representation.

    Rust versionLLVM version of the rustcValid LLVM version of system
    1.56 ~LLVM 13LLVM 13 and newer
  4. Build RedBPF inside Docker containers

    main

    When building RedBPF inside a Docker container, you must specify the location of kernel headers using environment variables.

    If using Linux kernel headers, use KERNEL_VERSION:

    # KERNEL_VERSION=5.11.0-25-generic cargo build --examples

    If the container has vmlinux (the Linux kernel image containing the .BTF section), use REDBPF_VMLINUX:

    # REDBPF_VMLINUX=/boot/vmlinux cargo build --examples
  5. Build LLVM from source for RedBPF

    main

    If your distribution does not provide LLVM 13 via pre-built packages, you can build it from source. Use the following steps to install it to a custom directory:

    $ tar -xaf llvm-13.0.0.src.tar.xz
    $ mkdir -p llvm-13.0.0.src/build
    $ cd llvm-13.0.0.src/build
    $ cmake .. -DCMAKE_INSTALL_PREFIX=$HOME/llvm-13-release -DCMAKE_BUILD_TYPE=Release -DLLVM_BUILD_LLVM_DYLIB=1
    $ cmake --build . --target install

    When installing cargo-bpf or building RedBPF, point to this custom installation using the LLVM_SYS_130_PREFIX environment variable:

    $ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo install cargo-bpf
    $ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo build

    Note: Ensure -DCMAKE_BUILD_TYPE is set to Release. Debug is not recommended unless you are debugging LLVM itself.

    $ LLVM_SYS_130_PREFIX=$HOME/llvm-13-release/ cargo install cargo-bpf
  6. Build RedBPF from source

    main

    To build the RedBPF repository from source (e.g., for development or bug fixing):

    $ git clone https://github.com/foniod/redbpf.git
    $ cd redbpf
    $ git submodule sync
    $ git submodule update --init
    $ cargo build
    $ cargo build --examples
  7. Compile BPF programs using cargo-bpf

    main

    To compile your BPF programs into ELF relocatable files, run cargo bpf build from within the probes directory. It is recommended to use the --target-dir flag to point to the userspace project's target directory so the userspace loader can easily find the .elf file.

    $ cd probes
    $ cargo bpf build --target-dir=../target

    This produces an ELF file (e.g., ../target/bpf/programs/<name>/<name>.elf) which is used by the userspace program to load the BPF logic into the kernel.

    $ cd probes
    $ cargo bpf build --target-dir=../target
  8. Initialize a RedBPF project scaffold

    main

    RedBPF projects typically consist of two parts: a userspace cargo project and a probes sub-project for BPF programs. Use cargo-bpf to manage this structure.

    1. Install the tool:
    $ cargo install cargo-bpf
    1. Create the userspace project:
    $ cargo new redbpf-tutorial
    $ cd redbpf-tutorial
    1. Create the BPF probes directory:
    $ cargo bpf new probes
    1. Add a new BPF program template within the probes directory:
    $ cd probes
    $ cargo bpf add <program_name>
    $ cargo install cargo-bpf
    $ cargo new redbpf-tutorial
    $ cd redbpf-tutorial
    $ cargo bpf new probes
    $ cd probes
    $ cargo bpf add openmonitor
  9. Install RedBPF requirements and LLVM 13

    main

    RedBPF requires LLVM 13 to compile BPF bytecode. Additionally, you must have access to one of the following to generate Rust bindings for Linux kernel data structures:

    1. Linux kernel headers
    2. vmlinux (the Linux kernel image with a .BTF section)
    3. Raw BTF data (e.g., from /sys/kernel/btf/vmlinux)

    Ubuntu 20.04 LTS

    # apt-get update \ 
      && apt-get -y install \ 
           wget \ 
           build-essential \ 
           software-properties-common \ 
           lsb-release \ 
           libelf-dev \ 
           linux-headers-generic \ 
           pkg-config \ 
      && wget https://apt.llvm.org/llvm.sh && chmod +x llvm.sh && ./llvm.sh 13 && rm -f ./llvm.sh
    # llvm-config-13 --version | grep 13

    Fedora 35

    # dnf install -y \ 
        clang-13.0.0 \ 
    \tllvm-13.0.0 \ 
    \tllvm-libs-13.0.0 \ 
    \tllvm-devel-13.0.0 \ 
    \tllvm-static-13.0.0 \ 
    \tkernel \ 
    \tkernel-devel \ 
    \telfutils-libelf-devel \ 
    \tmake \ 
        pkg-config \ 
        zstd
    # llvm-config --version | grep 13

    Arch Linux

    # pacman --noconfirm -Syu \ 
      && pacman -S --noconfirm \ 
           llvm \ 
           llvm-libs \ 
           libffi \ 
           clang \ 
           make \ 
           pkg-config \ 
           linux-headers \ 
           linux
    # llvm-config --version | grep -q '^13'
  10. Define a BPF program with redbpf-macros

    main

    When writing a BPF program in src/probes/<name>/main.rs, you must follow these requirements:

    • Use #![no_std] and #![no_main].
    • Use the program! macro to set the version and license (license must be GPL compatible).
    • Use the #[map] attribute to define BPF maps.
    • Use the #[kprobe] (or similar) attribute to define the entry point function.

    Example structure:

    #![no_std]
    #![no_main]
    
    use redbpf_probes::kprobe::prelude::*;
    
    program!(0xFFFFFFFE, "GPL");
    
    #[map]
    static mut MY_MAP: PerfMap<MyStruct> = PerfMap::with_max_entries(1024);
    
    #[kprobe]
    fn my_kprobe_func(regs: Registers) {
        // Logic here
    }
    #![no_std]
    #![no_main]
    
    use redbpf_probes::kprobe::prelude::*;
    
    program!(0xFFFFFFFE, "GPL");
    
    #[map]
    static mut OPEN_PATHS: PerfMap<OpenPath> = PerfMap::with_max_entries(1024);
    
    #[kprobe]
    fn do_sys_open(regs: Registers) {
        let mut path = OpenPath::default();
        unsafe {
            let filename = regs.parm2() as *const u8;
            if bpf_probe_read_user_str(
                path.filename.as_mut_ptr() as *mut _,
                path.filename.len() as u32,
                filename as *const _,
            ) <= 0
            {
                bpf_trace_printk(b"error on bpf_probe_read_user_str\0");
                return;
            }
            OPEN_PATHS.insert(regs.ctx, &path);
        }
    }
  11. Configure TcHashMap pinning with TcMapPinning

    main

    When creating a TcHashMap using with_max_entries, you can specify a TcMapPinning strategy to determine how the tc utility handles the map's lifecycle and visibility.

    • TcMapPinning::None: No map pinning. A new map instance is created with every tc invocation.
    • TcMapPinning::ObjectNamespace: The map is private to the ELF object and shared among various program sections within that object. It is pinned to /sys/fs/bpf/tc/<some object id>/<map name>.
    • TcMapPinning::GlobalNamespace: The map is placed in a global namespace, allowing it to be shared across different object files. It is pinned to /sys/fs/bpf/tc/globals/<map name>. This is the required setting if you want to access the map from a redbpf userspace program.