wit-bindgen

Cancellation in MoonBit components involves three distinct layers that must be managed carefully:

1. MoonBit Task Cancellation: Cooperatively stops local work.
2. Endpoint Copy Cancellation: Recovers an in-flight operation buffer.
3. Peer Drop: Reports loss-of-interest through a read or write result.

Cancellation Behaviors

• Incoming Reads: When a MoonBit read is cancelled, the system issues a concrete endpoint cancel operation, waits for the terminal event, reclaims the buffer, and only then drops the readable end.
• Outgoing Streams: Writes and close operations are serialized using a Mutex. When MoonBit task cancellation occurs on an outgoing stream, it issues stream.cancel-write, waits for buffer ownership to return, commits any transferred prefix, rejects the staged suffix, and then drops the writable end.
• Outgoing Futures: An exposed outgoing future writer is a 'settlement obligation'. The producer is shielded from ordinary task/subtask cancellation until it either writes a real value or a write reports a reader drop. If a local future never produces a value, the binding will not fabricate a default; the writer may remain pending until instance teardown.

When working with MoonBit and the Component Model, it is critical to distinguish between Component Model (CM) types and local MoonBit types to avoid confusion regarding how data and control flow across boundaries.

Naming Convention

• Component Model types: Use lowercase future and stream (e.g., future<T>, stream<T>). These represent WIT types. Their canonical value transfers a readable endpoint, not a MoonBit computation or buffer.
• Local MoonBit types: Use uppercase Future[T], Stream[T], Promise[T], and Sink[T]. These are local language constructs used for intra-component concurrency.

Key Concepts

• Readable/Writable Ends: Component futures and streams are pairs of endpoints. Generated future.new and stream.new calls create the pair; the WIT value itself carries the readable end.
• Async Task Scope: This is the component task context that owns a waitable set and manages suspended bridge work. It is a local context, not a global event loop.
• Background Group: The generated background_group : @async-core.TaskGroup[Unit] export parameter. Work spawned into this group can continue after the component result is published, but it cannot modify that result.

When your component implements an exported function (making it available to other components), follow these rules:

• Arguments: Your component receives ownership of all arguments passed to the exported function and is responsible for freeing them.
• Return values: Your component is responsible for allocating the memory for return values. You do not free them; wit-bindgen generates *_post_return functions that the caller will use to clean up.

Warning: Because *_post_return assumes strings and lists are dynamically allocated, you must always dynamically allocate the contents of your return values. If you use *_string_set with a string literal, the generated cleanup will attempt to free() that literal, causing undefined behavior. You can override the default *_post_return by defining your own version as a non-weak symbol.

bool exports_cat_registry_cat_registry_api_get_cat_by_name(
    cat_registry_string_t *name,
    exports_cat_registry_cat_registry_api_cat_t *ret) {
  bool found = strncmp((const char *)name->ptr, "Poptart", name->len) == 0;

  if (found) {
    // MUST dynamically allocate return values
    cat_registry_string_dup(&ret->name, "Poptart");
    ret->nicknames.ptr = (cat_registry_string_t *)malloc(2 * sizeof(cat_registry_string_t));
    ret->nicknames.len = 2;
    cat_registry_string_dup(&ret->nicknames.ptr[0], "Poppy");
    cat_registry_string_dup(&ret->nicknames.ptr[1], "Popster");
  }

  // We own the input parameter, so we must free it
  cat_registry_string_free(name);
  return found;
}

In MoonBit, there is a strict distinction between local coordination types and Component Model endpoints. This separation ensures that local async operations do not accidentally leak component handles or vtables into the MoonBit runtime.

Local Coordination Types

Future[T], Promise[T], Stream[T], and Sink[T] are local types used for coordination within MoonBit. They never contain component handles or operation tables.

Component Endpoints

Component Model future<T> or stream<T> types are transferable readable endpoints. Their operations and payload representations are tied to specific WIT function positions.

Conversion Logic

• Outgoing values: When a local value crosses the boundary, generated FFI code creates a new component endpoint pair to bridge the local value to the component endpoint.
• Incoming values: Incoming endpoints are treated as lazy generated sources.
• Nested endpoints: These are converted one layer at a time as the containing value is read at the boundary.

Requirements and Constraints

• Task Scopes: Producing a component future or stream requires an active component async task scope. You cannot perform scope-free synchronous lowering.
• Lifecycle Invariants: Once a component future readable end is exposed, the writer must eventually write a real value or observe that the reader was dropped. The binding cannot fabricate a default value or close the endpoint without a value.

The MoonBit component-model async design provides bindings for async, future<T>, and stream<T>. It uses a three-layer architecture to bridge MoonBit's local async types with the Component Model's canonical ABI:

1. Local Runtime (@async-core): Manages MoonBit-native concepts like Future[T], Promise[T], Stream[T], Sink[T], Task[T], TaskGroup[T], and synchronization primitives (Semaphore, Mutex, CondVar).
2. Recursive Generator Plan: A generator-side data structure that maps WIT function positions to specific async intrinsics, handling payload lifting, lowering, and cleanup.
3. Generated Site Helpers (ffi.mbt): The actual code that manages raw handles, ABI buffers, and converts between local MoonBit values and component endpoints.

Key Constraints:

• A component future<T> or stream<T> is a readable end; its writable counterpart is created via generated future.new or stream.new calls.
• Incoming conversions (imports) are lazy and do not create endpoint pairs.
• Outgoing conversions (exports) create a pair where the producer task owns the writable end and settlement logic.

When passing async types across the Component Model boundary, the following lifecycle and conversion rules apply:

Commit and Reject

• Commit: Transfers the accepted payload and initiates the producer's work.
• Reject: Cleans up values and untransferred streams. If a future is rejected, the paired writer is still driven until a write operation observes that the reader has been dropped.

Stream Operations

• Accepted Stream Prefix: The count of values successfully transferred to the peer.
• Staged Suffix: Values that were staged but not yet committed are cleaned up exactly once upon rejection.
• Unstaged Values: Values that have not been staged remain with the local stream.
• Staged Write Window: The portion of a local stream chunk lowered for a single component write. For nested future payloads, this is a one-element window in the MVP.

Settlement and Cancellation

• Future Writer Settlement Obligation: After calling future.new, the writable end must only be dropped after either a successful write or a write that reports the readable end was dropped. Cancellation alone does not settle this obligation.
• Endpoint Copy Cancellation: This is the cancellation of a specific component read or write to recover its operation buffer. It is distinct from cancelling the MoonBit coroutine itself.
• Peer Loss of Interest: Occurs when the opposite endpoint is dropped. Operations will report dropped. This is not equivalent to a hard cancellation of the producing coroutine.

MoonBit provides several primitives for managing asynchronous work within a single component boundary.

Local Future and Promise

• Future[T]: A consuming one-shot local value. It can hold a ready value or a lazy source. It does not contain component endpoints.
• Promise[T]: The producer paired with a Future[T]. Use it to complete, fail, or close local waiters. Note that a Promise is not a component future writer.

Local Stream and Sink

• Stream[T] / Sink[T]: A bounded local pipe used for streaming data.
  • Consumers: Pull owned FixedArray[T] chunks.
  • Producers: Push immutable ArrayView[T] values.
  • Capacity: A capacity of zero results in strict rendezvous behavior.
• Sink::close(): Marks a graceful EOF (End Of File) while ensuring buffered values are preserved for readers.
• Stream::drop(): Records a loss of interest, wakes writers, and triggers the configured payload cleanup operation for unread values.

The representation of result<T, E> and option<T> depends on whether the --no-sig-flattening flag is used.

Without --no-sig-flattening (Flattened)

Functions return a boolean indicating success/presence, and use out-parameters for the values.

Option Example: extern bool my_example_string_getter_get_string_by_index(uint32_t index, string_getter_user_option_string_t *ret);

Result Example: extern bool my_example_string_getter_get_string_by_index(uint32_t index, string_getter_user_string_t *ret, my_example_string_getter_error_t *err);

With --no-sig-flattening

Functions use a single out-parameter containing a struct with an is_err or is_some discriminator.

Option Example: extern void my_example_string_getter_get_string_by_index(uint32_t index, string_getter_user_option_string_t *ret);

Result Example: extern void my_example_string_getter_get_string_by_index(uint32_t index, my_example_string_getter_result_string_error_t *ret);