Rustler Precompiled

repository·main·Indexed 41 days ago

https://github.com/philss/rustler_precompiled

Facilitates the use of precompiled NIFs in Elixir projects to reduce the friction of managing Rust-based native extensions. It provides tools for downloading pre-built binaries, generating checksums for Hex releases, and configuring target triples and NIF versions (2.14 through 2.18) to avoid the need for local Rust compilation during the Elixir build process.

Tokens
7.1K
Snippets
15
Records
19
Agent score
52%

What's inside Rustler Precompiled

  1. Overview of Rustler Precompiled

    main
    Rustler Precompiled is designed to simplify the use of precompiled NIFs (Native Implemented Functions) for Elixir projects that utilize Rustler. It helps avoid the need to compile Rust code during the Elixir build process by providing access to pre-built binaries.
  2. Configure NIF versions for Erlang/OTP compatibility

    main

    Rustler defaults to NIF version 2.15 (targeting Erlang/OTP 22+). If your project requires support for newer or older OTP versions, you must disable default features in your Cargo.toml and explicitly enable the desired NIF version feature.

    Available NIF versions:

    • 2.14: OTP 21+
    • 2.15: OTP 22+
    • 2.16: OTP 24+
    • 2.17: OTP 26+
    • 2.18: OTP 29+

    Note: When using rustler-precompiled-action, ensure you follow the nif_version_MAJOR_MINOR format (e.g., nif_version_2_16) to allow the GitHub Action to activate the feature automatically.

    [dependencies]
    rustler = { version = "0.37", default-features = false, features = ["nif_version_2_16"] }
  3. Install rustler_precompiled via mix.exs

    main

    To use precompiled NIFs in your Elixir project, add rustler_precompiled to your deps list in mix.exs. It is recommended to keep rustler as an optional dependency, as it is typically only required during development.

    def deps do
      [
        {:rustler_precompiled, "~> 0.9"}
      ]
    end
  4. Configure build targets in GitHub Actions

    main

    When setting up a build matrix for precompiled NIFs, you should target common operating systems (Linux, Windows, macOS) and architectures (x86_64, aarch64). Use the philss/rustler-precompiled-action to handle the installation of cross and the naming of artifacts.

    Note: For targets requiring cross, set use-cross: true in your matrix configuration.

    matrix:
      nif: ["2.15"]
      job:
        - { target: aarch64-unknown-linux-gnu   , os: ubuntu-22.04 , use-cross: true }
        - { target: aarch64-apple-darwin        , os: macos-15      }
        - { target: x86_64-apple-darwin         , os: macos-15-intel }
        - { target: x86_64-unknown-linux-gnu    , os: ubuntu-22.04  }
        - { target: x86_64-unknown-linux-musl   , os: ubuntu-22.04 , use-cross: true }
        - { target: x86_64-pc-windows-gnu       , os: windows-2022  }
        - { target: x86_64-pc-windows-msvc      , os: windows-2022  }
  5. Configure GitHub Actions permissions for Rustler Precompiled

    main

    To allow your CI workflow to create releases and use security attestations, you must grant specific write permissions to the GITHUB_TOKEN. Add the permissions: key to your job section in your workflow YAML file.

    permissions:
      # For creating a new release.
      contents: write
      # The following are needed for the "actions/attest" GH Action.
      id-token: write
      attestations: write
      artifact-metadata: write
  6. Generate the mandatory checksum file for Hex releases

    main

    When releasing a Hex package, you must include a checksum file. This file ensures the integrity of the precompiled NIFs and prevents supply chain attacks.

    1. Build your artifacts in CI.
    2. Run the mix rustler_precompiled.download task with the --all and --print flags.
    3. Include the resulting checksum-*.exs file in your mix.exs files: list.
    $ mix rustler_precompiled.download YourRustlerModule --all --print

    Example mix.exs configuration:

    defp package do
      [
        files: [
          "lib",
          "native/my_nif/.cargo",
          "native/my_nif/src",
          "native/my_nif/Cargo*",
          "checksum-*.exs",
          "mix.exs"
        ]
      ]
    end
  7. Recommended release flow for Rustler Precompiled

    main

    Follow these steps to ensure a successful release:

    1. Release a new tag in your repository.
    2. Push the code and tags: git push origin main --tags.
    3. Wait for CI to complete all NIF builds.
    4. Run mix rustler_precompiled.download YourModule --all to generate the checksum file.
    5. Release the package to Hex.pm, ensuring the checksum-*.exs file is included in the package contents.
  8. Configure .cargo/config.toml for specific targets

    main

    Certain targets require specific Rust flags. For example, x86_64-unknown-linux-musl requires disabling crt-static to work correctly. You can also configure the release profile for optimizations like LTO (Link Time Optimization).

    # This is needed for "musl". See https://github.com/rust-lang/rust/issues/59302
    [target.x86_64-unknown-linux-musl]
    rustflags = [
      "-C", "target-feature=-crt-static"
    ]
    
    # Provides a small build size for the "release" profile, but takes more time to build.
    [profile.release]
    lto = true
  9. Configure variant detection via `:variants`

    main

    The :variants option allows you to specify logic for selecting specific NIF variants based on the target. It is a map where keys are target triples and values are keyword lists of {variant_name, detection_function}.

    The detection_function must be a function that accepts either 0 or 1 argument and returns a boolean indicating if the variant should be activated.

    Note: All targets specified in the :variants map must also be present in the :targets list.

    # Example of defining variants for specific targets
    variants: %{
      "x86_64-unknown-linux-gnu" => [
        {:musl, fn -> true end},
        {:gnu, fn _ -> false end}
      ]
    }
  10. Support "musl" ABI targets (e.g., Alpine Linux)

    main

    When targeting systems using the musl ABI, such as Alpine Linux, you must disable static linking. This is achieved by modifying your .cargo/cargo.toml file to include specific rustflags for the target. This is common for x86_64-unknown-linux-musl and aarch64-unknown-linux-musl (often used in containers on Apple Silicon/ARM64 machines).

    # This is needed for "musl". See https://github.com/rust-lang/rust/issues/59302
    [target.x86_64-unknown-linux-musl]
    rustflags = [
      "-C", "target-feature=-crt-static"
    ]
  11. Link libatomic for 32-bit ARM builds

    main

    When targeting 32-bit platforms like ARM (e.g., arm-unknown-linux-gnueabihf), you may need to explicitly link libatomic. To do this, modify your .cargo/config.yml file to add the dylib=atomic flag for the specific target.

    # Libatomic is needed for 32 bits ARM.
    # See: https://github.com/philss/rustler_precompiled/issues/53
    [target.arm-unknown-linux-gnueabihf]
    rustflags = [
      "-l", "dylib=atomic"
    ]
  12. Use the RustlerPrecompiled macro in Elixir

    main

    To enable precompiled NIF downloading, use the RustlerPrecompiled macro in your native module. You must provide the otp_app, the crate name, and a base_url pointing to your release artifacts (e.g., GitHub Releases).

    To allow users to force a local build if needed, you can use an environment variable to toggle force_build. To ensure the project can fall back to a local build, declare {:rustler, ">= 0.0.0", optional: true} in your dependencies.

    defmodule RustlerPrecompilationExample.Native do
      version = Mix.Project.config()[:version]
    
      use RustlerPrecompiled,
        otp_app: :rustler_precompilation_example,
        crate: "example",
        base_url:
          "https://github.com/philss/rustler_precompilation_example/releases/download/v#{version}",
        force_build: System.get_env("RUSTLER_PRECOMPILATION_EXAMPLE_BUILD") in ["1", "true"],
        version: version
    
      # When your NIF is loaded, it will override this function.
      def add(_a, _b), do: :erlang.nif_error(:nif_not_loaded)
    end