DOCSEARCHbyAlgolia
Submit Docs

Zigler

repository·main·Indexed 21 days ago

https://github.com/e-xyza/zigler

A tool for creating Zig Native Implemented Functions (NIFs) for Elixir and Erlang. It enables the integration of high-performance Zig code into BEAM applications with automatic type marshalling for scalars, arrays, slices, and structs. Zigler supports binding to C libraries, provides a standard BEAM allocator, and includes a dedicated installer via `mix zig.get` to manage the Zig compiler toolchain.

Tokens
20.6K
Snippets
73
Records
98
Agent score
72%

What's inside Zigler

  1. Manage resource reference counts with release and keep

    main

    Resources use reference counting to prevent the BEAM GC from destroying memory while a NIF is still using it. The resource type provides .release() and .keep() methods.

    Key Behaviors

    • release: Decrements the reference count.
      • By default, a resource created via StructResource.create(...) is released on creation. This can be disabled with .release = false in the options.
      • If a function takes a beam.Resource(...) type as an argument, it is released at the end of the call unless the :noclean flag is set in the function argument options.
    • keep: Increments the reference count.
      • beam.get keeps the resource by default. This can be disabled with .keep = false in the options.

    Warning: For wrapped datatypes that require cleanup (like pointers), do not use beam.get with .keep = false. This can cause a race condition where the pointer is dereferenced after another thread has performed cleanup.

    Example:

    pub fn release(resource: StructResource) void {
    resource.release();
}

pub fn keep(resource: StructResource) void {
    resource.keep();
}

  2. Use Enums as Elixir Atoms

    main

    Zigler allows using Zig enums to represent Elixir atoms.

    • Passing Atoms: You can pass an Elixir atom to a Zig function by using the corresponding enum variant name.
    • Returning Atoms: A Zig function returning an enum type will return the corresponding Elixir atom.
    • Passing Integers: You can pass the underlying integer value of an enum in place of the atom.
    • Creating Literals: Use beam.make to convert enum literals into atoms. This is useful for returning :ok or :error tuples.

    Note: Since error is a reserved word in Zig, use the builtin syntax .@"error" to create an error atom.

    ~Z"""
const EnumType = enum(u8) {
  foo,
  bar = 47
};

pub fn flip(input: EnumType) EnumType {
  return switch (input) {
    .foo => .bar,
    .bar => .foo
  };
}
"""

test "flipping enums" do
  assert :bar = flip(:foo)
  assert :foo = flip(:bar)
end

  3. Use Structs as atom-keyed maps

    main

    In Zigler, most Zig structs are interpreted as Elixir maps with atom keys.

    • Requirements: For a struct type to be used in parameters or returns, it must be exported as pub in the module interface.
    • Anonymous Structs: You can return anonymous structs using beam.make.
    • Tuples: Since Zig tuples are structs with integer keys, you can construct Elixir tuples by passing a Zig tuple to beam.make.
    • Packed/Extern Structs: These can be passed as Elixir maps OR as binaries. Be mindful of endianness when using them as binaries.
    • Pointers to Structs: You can pass pointers to structs (*Struct) to allow Zig to mutate the data in a way that reflects in Elixir.
    ~Z"""
pub const Point2D = struct{ x: i32, y: i32 };

pub fn reflect(input: Point2D) Point2D {
  return .{.x = input.y, .y = input.x};
}
"""

test "structs" do
  assert %{x: 48, y: 47} == reflect(%{x: 47, y: 48})
end

  4. Build composable allocators using Zig's allocator interface

    main

    Because Zigler's beam allocators conform to the standard Zig allocator interface, you can wrap them in composable allocators from the Zig standard library, such as std.heap.ArenaAllocator.

    This is useful for managing the lifecycle of a group of allocations together.

    pub fn with_arena() !beam.term {
    var arena = std.heap.ArenaAllocator.init(beam.allocator);
    defer arena.deinit();

    const allocator = arena.allocator();

    const slice = try allocator.alloc(u16, 4);
    defer allocator.free(slice);

    for (slice, 0..) |*item, index| {
        item.* = @intCast(index);
    }

    return beam.make(slice, .{});
}

  5. Choose a NIF concurrency strategy

    main

    When executing Zig code via a NIF, the BEAM VM loses control over the execution flow. Because the VM cannot tolerate native code running for longer than approximately 1ms, you must choose an appropriate concurrency strategy based on your workload:

    1. Synchronous (Default): Use only if your code is guaranteed to run in under 1ms.
    2. Dirty CPU: Use for long-running CPU-intensive tasks. This uses the VM's Dirty CPU schedulers. Tag with :dirty_cpu.
    3. Dirty IO: Use for blocking IO operations. Tag with :dirty_io.
    4. Threaded: Use when your OS supports spawning threads. This runs the code in a separate OS thread. Tag with :threaded.

    Warning: Consuming all Dirty CPU schedulers can cause subsequent :dirty_cpu calls to block, increasing latency.

    use Zig, 
  otp_app: :zigler,
  nifs: [
    long_running_cpu: [:dirty_cpu],
    long_running_io: [:dirty_io],
    long_running_threaded: [:threaded]
  ]

  6. Use Array-of-Structs (AoS) and Struct-of-Arrays (SoA) patterns

    main

    Zigler supports both common memory layouts for collections of data:

    1. Array-of-Structs (AoS): Passing a list of maps in Elixir to a Zig function expecting a slice of structs (e.g., []Point2D).
    2. Struct-of-Arrays (SoA): Passing a single map in Elixir where each key contains a list of values (e.g., %{x: [1, 2], y: [3, 4]}) to a Zig function expecting a struct containing slices (e.g., struct { x: []u16, y: []u16 }).
    # Array of Structs (AoS) example
~Z"""
pub fn sum_points(points: []Point2D) Point2D {
    var result: Point2D = .{.x = 0, .y = 0};
    for (points) |point| {
        result.x += point.x;
        result.y += point.y;
    }
    return result;
}
"""

test "array of struct" do
  assert %{x: 9, y: 12} = sum_points([%{x: 1, y: 2}, %{x: 3, y: 4}, %{x: 5, y: 6}])
end

# Struct of Arrays (SoA) example
~Z"""
pub const PointSOA = struct{
  x: []u16,
  y: []u16
};

pub fn sum_point_soa(points: PointSOA) Point2D {
    var result: Point2D = .{.x = 0, .y = 0};
    for (points.x) |x| {
        result.x += x;
    }
    for (points.y) |y| {
        result.y += y;
    }
    return result;
}
"""

test "struct of array" do
  assert %{x: 9, y: 12} = sum_point_soa(%{x: [1, 3, 5], y: [2, 4, 6]})
end

  7. Use raw nifs in Zigler

    main

    While Zigler typically generates adapter functions with automatic term marshalling, you can define 'raw nifs' to bypass this process. Raw nifs allow you to write Zig functions that match the standard C header for a BEAM nif:

    static ERL_NIF_TERM hello(ErlNifEnv* env, int argc, const ERL_NIF_TERM argv[])

    This is useful when you want full manual control over the environment and arguments without the overhead of Zigler's automatic marshalling.

  8. How the callback context works

    main

    When a Zig callback is executed, Zigler provides a context containing the following fields, allowing you to use beam.get, beam.make, or beam.send without extra configuration:

    • env: The e.ErlNifEnv value.
    • mode: Set to .callback.
    • allocator: The beam.allocator.

    This context is automatically available within the callback scope.

  10. How resources work in Zigler

    main

    Resources are datatypes managed by the BEAM reference-counted garbage collector. Instead of passing raw pointers or global variables between function calls, you should use resources.

    In the Elixir environment, resources are passed as t:reference/0 tokens. When a resource is created, the BEAM allocates memory for it, and this memory is cleared when the garbage collector is triggered. You can attach custom cleanup logic (destructors) to these GC events.

    Note: Passing references between different modules is currently not supported by Zigler, though it is planned.

  11. Setup Zigler in an Elixir module

    main

    To use Zigler, add the use Zig, otp_app: :your_app directive to your module. The otp_app option is required so Zigler can locate the directory for compilation artifacts (libraries), which defaults to /priv/lib and allows them to be included in releases.

    defmodule MyNifModule do
  use Zig, otp_app: :my_app
end
    defmodule NifGuideTest do
  use Zig, otp_app: :zigler
  use ExUnit.Case, async: true

  12. Use web-hosted precompiled modules with Zigler

    main

    You can host precompiled assets on the web by providing the triple {:web, address, shasum} to the :precompiled option.

    Single File

    For a single file, provide the direct URL and a base-16 encoded SHA256 string.

    Multiplatform Files

    For multiplatform support, provide an address template and a keyword list of hashes. Zigler will interpolate the following placeholders in the address template:

    • #VERSION: Your project's version number.
    • #TRIPLE: The architecture-os-abi triple (e.g., x86_64-linux-gnu). This must match the keys in your hash list.
    • #EXT: The OS-based file extension (.dll for Windows, .so for others).

    Note: Precompiled modules are an experimental feature.

    # Single file example
use Zig, 
  otp_app: :zigler, 
  precompiled: {:web, "https://address-for-archive/precompiled-artifact.so", "935f9829d4c0058acba4118c9dc1a98dbdda5c4035e16a7893c33a3aff2caee8"}, 
  ...

# Multiplatform example
@lib_address "https://address-for-archived-libraries/MyModuleName.#VERSION.#TRIPLE.#EXT"
@shasum [
  "aarch64-linux-gnu": "17546c34adf8b6a14dd38ebb9d5485610348e5afff72e88b991f18d6b818197f",
  "x86_64-windows-gnu": "3ee18aec252eca92f19d920f6de143e59700e0f038916fb3717aa9869b2c5102"
]

use Zig, 
  otp_app: :zigler, 
  precompiled: {:web, @lib_address, @shasum}