Rust FFI Omnibus

repository·master·Indexed 19 days ago

https://github.com/shepmaster/rust-ffi-omnibus

A collection of examples and a specialized toolkit for creating Foreign Function Interfaces (FFI) in Rust. It provides standardized implementations for common data patterns—including integers, objects, slices, strings, tuples, and vectors—to facilitate safe data exchange between Rust and other languages such as C, Ruby, Python, Haskell, Node.js, C#, and Julia.

Tokens
8.4K
Snippets
45
Records
47
Agent score
67%

What's inside rust-ffi-omnibus

  1. Overview of the Rust FFI Omnibus

    master
    The Rust FFI Omnibus is a collection of tools and patterns designed to facilitate Foreign Function Interface (FFI) operations between Rust and other languages. It provides standardized ways to handle common data types and return patterns across the FFI boundary.
  2. How to return allocated strings from Rust via FFI

    master

    Returning an allocated string from Rust to another language is complex because the Rust allocator may differ from the caller's allocator. To safely return a string, you must transfer ownership to the caller via a raw pointer, and the caller must eventually return that pointer to Rust to ensure the memory is properly deallocated.

    In Rust, use CString::into_raw to convert a CString into a raw pointer for FFI. To reclaim ownership and deallocate the memory later, the caller must pass that pointer back to a Rust function that uses CString::from_raw.

    // Rust side logic pattern
    // 1. Create CString
    // 2. Call .into_raw() to pass ownership to caller
    // 3. Caller must eventually pass pointer back to Rust to call CString::from_raw()
  3. How opaque objects work across FFI

    master

    An opaque object (or opaque pointer) is a pattern used to pass complex Rust structures to other languages without exposing their internal memory layout. Instead of sharing the struct definition, you pass a raw pointer to the memory address where the object lives on the heap.

    Core Principles:

    • Stability: The Rust object must be placed on the heap (e.g., using Box::into_raw) to ensure it has a stable memory address that remains valid across FFI calls.
    • Namespacing: Since C-style FFI lacks native namespacing, functions should be prefixed with a package or type name (e.g., zip_code_database_...) to avoid collisions.
    • Ownership & Deallocation: Memory allocated by Rust must be deallocated by Rust. You must provide a specific deallocation function that uses Box::from_raw to reclaim the pointer and allow Rust to drop the object.
    • Safety: The client language is responsible for ensuring pointers are not NULL before use and for calling the deallocation function exactly once to prevent memory leaks or double-frees.
    // Rust side pattern
    #[no_mangle]
    pub extern "C" fn zip_code_database_new() -> *mut ZipCodeDatabase {
        Box::into_raw(Box::new(ZipCodeDatabase::new()))
    }
    
    #[no_mangle]
    pub extern "C" fn zip_code_database_free(ptr: *mut ZipCodeDatabase) {
        if !ptr.is_null() {
            unsafe { Box::from_raw(ptr); }
        }
    }
  4. Use allocated strings in C#

    master

    In C#, wrap the Rust string in a subclass of SafeHandle. Use a wrapper class to ensure the handle is disposed of properly.

    Note: Since C# does not have a native way to read a pointer directly as a UTF-8 string (it supports ANSI and UCS-2/Unicode), you must implement a custom method to read the pointer as UTF-8.

    // C# pattern
    // Wrap in SafeHandle and use a wrapper to ensure disposal
  5. Locate Rust dynamic libraries when running examples

    master

    When calling Rust FFI functions from other languages, the system must be able to locate the compiled dynamic library. The method for doing this depends on your operating system:

    macOS and Linux

    Prefix your execution command with LD_LIBRARY_PATH pointing to the directory containing your compiled library (typically target/debug).

    Note: On macOS, System Integrity Protection (SIP) may prevent setting LD_LIBRARY_PATH for system-provided binaries. You may need to use a different binary or disable SIP.

    Windows

    Copy the compiled .dll file directly into the current working directory before running your application.

    # macOS/Linux example
    LD_LIBRARY_PATH=target/debug python src/main.py
  6. Use allocated strings in Julia

    master

    In Julia, use the Cstring data type to represent the NUL-terminated string. To avoid managing the Rust-allocated memory directly in Julia, use unsafe_string to construct a copy of the string that is managed by Julia's garbage collector, then transfer the original Rust pointer back to Rust to be freed.

    # Julia pattern
    ptr = get_string_ffi()
    if ptr != C_NULL
        str = unsafe_string(ptr)
        free_rust_string(ptr)
    end
  7. Use allocated strings in Haskell

    master

    When calling the FFI method in Haskell, first check if the returned pointer is NULL. If it is not NULL, use peekCString to convert the pointer into a Haskell String, and then immediately call the Rust function to free the string.

    -- Haskell pattern
    ptr <- get_string_ffi
    if ptr == nullPtr
        then return ""
        else do
            str <- peekCString ptr
            free_rust_string ptr
            return str
  8. Implement opaque objects in Node.js

    master

    In Node.js, wrap the FFI functions in a class to provide an idiomatic JavaScript API (e.g., camelCase) and manage the pointer lifecycle.

    Implementation Steps:

    1. Declare the pointer type for returned or accepted arguments in your FFI definitions.
    2. Create a wrapper class that maintains the pointer.
    3. Use a try...finally block in your application code to ensure the deallocation method is called in the finally block, preventing leaks.
    const db = new ZipCodeDatabase();
    try {
      const pop = db.getPopulation(zipCode);
    } finally {
      db.free();
    }
  9. Implement opaque objects in C# using SafeHandle

    master

    For robust memory management in C#, inherit from System.Runtime.InteropServices.SafeHandle. This integrates the Rust pointer into the .NET lifecycle and ensures it is released correctly.

    Implementation Steps:

    1. Create a Native class to hold the FFI function definitions.
    2. Create a subclass of SafeHandle (e.g., ZipCodeDatabaseHandle).
    3. Implement IsInvalid (returning false if your Rust function accepts NULL) and ReleaseHandle (calling the Rust deallocation function).
    4. Implement the IDisposable pattern on your high-level wrapper to forward disposal to the SafeHandle.
    class ZipCodeDatabaseHandle : SafeHandle {
        public override bool IsInvalid => IsNull;
        protected override bool ReleaseHandle() {
            Native.zip_code_database_free(handle);
            return true;
        }
    }
  10. Implement opaque objects in Julia

    master

    In Julia, hide the handler pointer behind a new data type. You can manage the lifecycle using either a mapping constructor (for do blocks) or a manual close method.

    Implementation Steps:

    1. Define a type that wraps the pointer.
    2. Option 1 (Automatic): Implement a mapping constructor ZipCodeDatabase(f) that handles both allocation and deallocation. This allows usage with the do syntax, similar to Python's with.
    3. Option 2 (Manual): Implement a close method to manually free the object when it is no longer needed.
    # Option 1: Using 'do' syntax (Automatic)
    zip_code_database(zip) do db
        pop = get_population(db, zip)
    end
    
    # Option 2: Manual closing
    db = ZipCodeDatabase()
    # ... use db ...
    close(db)
  11. Implement Rust functions that accept string arguments

    master

    When accepting strings from C-compatible interfaces, you must bridge the gap between C's NUL-terminated char* pointers and Rust's UTF-8 guaranteed &str slices.

    To safely convert a C pointer to a Rust string slice (&str), follow these steps:

    1. Null Check: Ensure the incoming C pointer is not NULL, as Rust references cannot be null.
    2. Wrap with CStr: Use std::ffi::CStr to wrap the raw pointer. This computes the length based on the NUL terminator. This step requires an unsafe block because you are dereferencing a raw pointer.
    3. UTF-8 Validation: Convert the CStr to a Rust string slice by ensuring it is valid UTF-8.
    4. Usage: Use the resulting &str within your logic.

    Note on Ownership and Lifetimes: The Rust code does not own the string slice. The compiler ensures the slice lives only as long as the CStr instance. The caller is responsible for ensuring the memory remains valid for the duration of the function call.

    use std::ffi::CStr;
    use std::os::raw::c_char;
    
    #[no_mangle]
    pub unsafe extern "C" fn print_string(ptr: *const c_char) {
        if ptr.is_null() {
            return;
        }
    
        let c_str = CStr::from_ptr(ptr);
        if let Ok(s) = c_str.to_str() {
            println!("Rust received: {}", s);
        } else {
            // Handle invalid UTF-8
        }
    }