Rust2Go Documentation

repository·master·Indexed 19 days ago

https://github.com/ihciah/rust2go

A high-performance FFI framework enabling bidirectional calling between Rust and Golang. Rust2Go focuses on low-latency communication using direct memory references and assembly-based callbacks via the asmcall package, reducing overhead compared to standard CGO or socket-based IPC. It provides a CLI tool (rust2go-cli) and macros like #[derive(rust2go::R2G)] and #[rust2go::r2g] to generate bindings, supporting both synchronous and asynchronous calls across the language boundary.

Tokens
17.6K
Snippets
62
Records
80
Agent score
64%

What's inside Rust2Go

  1. Overview of Rust2Go examples

    master

    The rust2go repository provides several example implementations demonstrating different call directions, runtimes, and backend technologies for interop between Rust and Go. Use these examples to understand how to implement specific communication patterns like unidirectional calls, bidirectional calls, or shared-memory-based communication.

    | Name                   | Call Direction | Runtime | Backend Technology |
    |------------------------|----------------|---------|--------------------|
    | example-monoio         | Rust -> Go     | Monoio  | CGO                |
    | example-tokio          | Rust -> Go     | Tokio   | CGO                |
    | example-monoio-mem     | Rust -> Go     | Monoio  | Shared Memory Lockless Queue |
    | example-tokio-mem      | Rust -> Go     | Tokio   | Shared Memory Lockless Queue |
    | example-bidirectional  | Rust -> Go & Go -> Rust  | N/A | CGO          |
    | example-go2rust        | Go -> Rust     | N/A     | CGO                |
  2. What is Mem Ring

    master
    Mem Ring is a shared-memory-based ring buffer designed to bridge Rust and Go. It enables bidirectional communication, allowing both Rust and Go to initiate calls to each other. It is compatible with both the tokio and monoio runtimes.
  3. Understand Rust2Go memory safety and data handling

    master

    Rust2Go uses FFI to pass data by reference to minimize memory operations. This introduces specific safety responsibilities:

    Golang Side

    Data received from Rust is referenced. The Golang handler can implement logic arbitrarily, but if you need to use that data outside the function's lifecycle, you must manually perform a deep copy.

    Rust Side

    You must ensure that the slot pointer of the callback FFI operation and user parameters remain valid when the Future is dropped. To assist with this, Rust2Go provides:

    • An atomic slot structure to manage callback state.
    • The [drop_safe] attribute, which requires user-passed parameters to have ownership to ensure they aren't dropped prematurely.
  4. How ASM CALL in Go works

    master

    The asmcall package provides a high-performance alternative to CGO for executing external functions by using hand-written assembly instead of the standard CGO mechanism. This avoids the overhead of extensive checks, scheduling, and GC synchronization.

    Core Mechanism

    1. ABI Conversion: Converts the calling convention from Go ABI0 to the System V AMD64 ABI or Microsoft x64 Calling Convention.
    2. Stack Switching: Saves the current Stack Pointer (SP) and switches the SP to the g0 stack (switching the g to g0).
    3. Execution: Performs the CALL to the external function.
    4. Restoration: Switches the SP and g back to their original states.

    Platform Support

    • AMD64 and ARM64: Uses optimized assembly for stack switching and ABI conversion.
    • Other Platforms: Falls back to the standard CGO implementation.

    Performance Trade-offs

    • Pros: Significantly faster for simple C/Rust functions. Benchmarks show a reduction from ~29ns (CGO) to ~2.3ns (ASM).
    • Cons: Because it bypasses standard Go scheduling, long-running external functions called via ASM prevent Go from performing asynchronous preemption. This can increase scheduling latency for other goroutines on the same thread. Use ASM for short, fast functions.
  5. Configure linking: Static vs Dynamic

    master

    You can choose between static and dynamic linking when connecting Go to your Rust library.

    Static Linking

    • Rust side: Use crate-type = ["staticlib"].
    • Go side: Use #cgo LDFLAGS: ./path/to/librust_lib.a.
    • Benefit: The resulting Go binary is self-contained regarding the Rust logic.

    Dynamic Linking

    • Rust side: Use crate-type = ["cdylib"].
    • Go side: Use #cgo LDFLAGS: -L. -lrust_lib.
    • Runtime: You must distribute the shared library (.so, .dylib, or .dll) alongside your executable. You may need to set LD_LIBRARY_PATH so the OS can find it:
      export LD_LIBRARY_PATH=$(pwd):$LD_LIBRARY_PATH
  6. Implement Bidirectional Calling (Rust <-> Go)

    master

    Bidirectional calling allows Rust to call Go functions and Go to call Rust functions using CGO as the backend. This is achieved by defining a trait in Rust, generating Go interface implementations, and using a build script to link the two.

    // In Rust, define your data and trait
    #[derive(rust2go::R2G)]
    pub struct DemoRequest {
        pub name: String,
        pub age: u8,
    }
    
    #[derive(rust2go::R2G)]
    pub struct DemoResponse {
        pub pass: bool,
    }
    
    #[rust2go::r2g]
    pub trait DemoCall {
        fn demo_oneway(req: &DemoRequest);
        fn demo_check(req: DemoRequest) -> DemoResponse;
    }
  7. Generate Go code from Rust definitions

    master

    Use the rust2go-cli to generate the Go interface implementation based on your Rust trait definitions.

    Run the following command: rust2go-cli --src <path_to_rust_file> --dst <path_to_go_destination>

    After generation, you must create a Go file (e.g., impl.go) that defines a struct implementing the generated {$trait} interface. Assign this struct to the {$trait}Impl type to allow Rust to access it.

    rust2go-cli --src src/user.rs --dst go/gen.go
  8. Setup Rust2Go with Tokio and Shared Memory

    master

    To use Rust2Go with the Tokio runtime and a Shared Memory Lockless Queue backend, follow these setup steps:

    1. Configure Dependencies

    Add rust2go to your Cargo.toml. You must include it in both dependencies and build-dependencies (with the build feature enabled).

    2. Install CLI Tool

    Install the required command-line tool for code generation:

    cargo install --force rust2go-cli

    3. Configure Build Script

    Create a build.rs file in your Rust project root to automate the Go source integration:

    fn main() {
        rust2go::Builder::new().with_go_src("./go").build();
    }
    [dependencies]
    rust2go = { version = "0.4.0" }
    
    [build-dependencies]
    rust2go = { version = "0.4.0", features = ["build"] }
  9. Setup Rust2Go for Tokio with Shared Memory

    master

    To use Rust2Go with the Tokio runtime and a Shared Memory Lockless Queue backend, follow these setup steps:

    1. Update Cargo.toml: Add rust2go to both dependencies and build-dependencies (with the build feature enabled).
    2. Install CLI: Install the required command-line tool via cargo.
    3. Configure Build Script: Create a build.rs to automate the Go source integration.
    4. Initialize Go Project: Create a directory for your Go code and initialize a module.
    [dependencies]
    rust2go = { version = "0.4.0" }
    
    [build-dependencies]
    rust2go = { version = "0.4.0", features = ["build"] }
    cargo install --force rust2go-cli

    In build.rs

    fn main() { rust2go::Builder::new().with_go_src("./go").build(); }

  10. Generate and call Go code from Rust

    master

    After defining your Rust interface, use the rust2go-cli to generate the Go bindings.

    1. Generate bindings: Run the CLI pointing to your Rust source file:

      rust2go-cli --src rust-lib/src/user.rs --dst gen.go --without-main
    2. Call from Go: Import the generated code and use the {$trait}Impl struct to invoke the methods. You must also include the appropriate cgo LDFLAGS to link the Rust library.

    /*
    // For statically link:
    #cgo LDFLAGS: ./librust_lib.a
    
    // For dynamically link:
    #cgo LDFLAGS: -L. -lrust_lib
    */
    import "C"
    
    func main() {
        user := DemoUser{
            name: "chihai",
            age:  28,
        }
        // Use the Impl struct to call methods
        G2RCallImpl{}.demo_log(&user.name, &user.age)
        new_name := G2RCallImpl{}.demo_convert_name(&user)
        fmt.Printf("new name: %s", new_name)
    }
  11. Control code generation using trait attributes

    master

    When defining traits for rust2go, you can use specific attributes on async functions to control how the code is generated for both Rust and Go. These attributes allow you to manage thread safety, memory management, and performance optimizations.

    Available Attributes

    AttributeEffect on Generated Code
    #[send]Generates the function as impl Future<Output=..> + Send + Sync. Use this when thread safety is required.
    #[drop_safe]Makes the function safe by requiring all parameters to pass ownership. Use this when you cannot guarantee the future can be safely cancelled.
    #[drop_safe_ret]Similar to #[drop_safe], but allows the caller to receive ownership of the parameters back after the call. Use this when cancellation safety is needed and you want to reclaim ownership.
    #[mem] or #[shm]Implements the function using shared memory for high performance. Note: This currently requires Unix and should only be used if you identify a significant performance bottleneck.
    #[go_pass_struct]Forces the generated Go code to use pointers instead of values for parameters. This is useful for large structures but is generally not recommended unless explicitly needed.
    #[cgo_callback]Forces the generated Go code to use CGO-based methods instead of ASM. Use this only if you encounter failures caused by ASMCALL.

    Automatic Lifetime Management

    If all parameters in an async function are passed by ownership, rust2go automatically adds a 'static lifetime to the generated impl Future. This is particularly useful for spawning tasks.

    #[rust2go::r2g]
    pub trait DemoCall {
        #[send]
        fn demo_check_async(
            req: &DemoComplicatedRequest,
        ) -> impl std::future::Future<Output = DemoResponse>;
    
        #[drop_safe]
        fn demo_check_async_safe(
            req: DemoComplicatedRequest,
        ) -> impl std::future::Future<Output = DemoResponse>;
    
        #[drop_safe_ret]
        fn demo_check_async_safe_with_ret(
            req: DemoComplicatedRequest,
        ) -> impl std::future::Future<Output = DemoResponse>;
    }
  12. Setup Rust2Go for Monoio with Shared Memory

    master

    To use Rust2Go with the Monoio runtime and shared memory backend, follow these setup steps:

    1. Install the CLI tool:

      cargo install --force rust2go-cli
    2. Configure Cargo.toml: Add rust2go to both dependencies and build-dependencies (with the build feature enabled).

    3. Initialize the Go project: Create a directory for your Go code and initialize a module:

      mkdir go && cd go && go mod init r2gexample
    4. Configure build.rs: Create a build.rs in your Rust project root to automate Go code generation.

    5. Include bindings: In your Rust source file (e.g., user.rs), use the r2g_include_binding! macro to link the generated code.

    [dependencies]
    rust2go = { version = "0.4.0" }
    
    [build-dependencies]
    rust2go = { version = "0.4.0", features = ["build"] }
    // build.rs
    fn main() {
        rust2go::Builder::new().with_go_src("./go").build();
    }
    // user.rs
    pub mod binding {
        rust2go::r2g_include_binding!();
    }