Maelstrom Documentation
repository·main·Indexed 20 days ago
https://github.com/maelstrom-software/maelstromA suite of tools for running tests in isolated, lightweight, rootless micro-containers. Maelstrom provides drop-in replacements for standard test runners including cargo-maelstrom (Rust), maelstrom-go-test (Go), and maelstrom-pytest (Python), supporting scaling from local standalone execution to large distributed clusters. The ecosystem includes core components such as the Maelstrom Broker, Maelstrom Worker, and Maelstrom Admin, as well as specialized libraries for WASM compatibility (maelstrom-base), OCI image management (maelstrom-container), and simulation exploration (SIMEX).
What's inside Maelstrom
- The Maelstrom WASM Web Client is a web-based interface for interacting with Maelstrom clusters. It is implemented in Rust, compiled to WebAssembly (WASM), and is served directly by the Maelstrom broker.
Overview of the Maelstrom Container Library
mainThemaelstrom-containerlibrary provides functionality for Maelstrom to interact with OCI (Open Container Initiative) images. Its primary responsibilities are downloading images from registries (such as Docker Hub) and managing them within a local storage system.Overview of the maelstrom-macro library
mainThemaelstrom-macrocrate is a dedicated macro library designed to be used by other Maelstrom binaries and libraries. It provides procedural macros to assist in the development and implementation of Maelstrom-compatible components.Overview of Maelstrom capabilities
mainMaelstrom is a suite of tools designed for running tests in isolated micro-containers. It can run locally on a single machine or be distributed across large clusters.
Key features include:
- Test Runners: Provides drop-in replacements for
cargo test,go test, andpytest. - Isolation: Runs every test in its own lightweight, rootless container to eliminate inter-test dependencies.
- Scalability: Supports cluster mode to increase test throughput by adding worker machines.
- Low Overhead: Uses a custom Rust-based rootless container implementation (not relying on Docker or RunC).
- General Purpose Execution: Beyond testing, it provides a CLI utility for arbitrary commands, a gRPC-based API, and Rust bindings for programmatic control.
Platform Support: Currently Linux-only (x86 and ARM) due to its reliance on Linux namespaces for container implementation.
- Test Runners: Provides drop-in replacements for
Overview of Maelstrom Linux Syscalls
mainThe
maelstrom-linuxlibrary provides simple, non-allocating wrappers around essential Linux syscalls. It is designed specifically for use bymaelstrom-workerandmaelstrom-worker-child.Key characteristics:
no_stdcompatibility: The library does not rely on the Rust standard library, making it suitable for low-level environments.- Zero-allocation: It is designed to be non-allocating to ensure predictable behavior in worker processes.
- Architecture emulation: For syscalls not natively available on all architectures, the library provides emulation similar to
libc.
Overview of the GitHub Client Library
mainThemaelstrom-githubcrate provides client code specifically designed for communicating with GitHub APIs. It is a component used by the broader Maelstrom project to facilitate GitHub-related interactions.Overview of Maelstrom
mainMaelstrom is a suite of tools designed for running tests in isolated micro-containers. It can run tests locally on a single machine or distributed across large clusters.
Key features include:
- Test Runner Replacements: Provides drop-in replacements for
cargo test,go test, andpytest. - Isolation: Runs every test in its own lightweight, rootless container to eliminate inter-test dependencies.
- Scalability: Supports cluster mode to increase test throughput by adding worker machines.
- Low Overhead: Uses a custom Rust-based rootless container implementation that does not rely on Docker or RunC.
- General Purpose Execution: Beyond testing, it includes a CLI for running arbitrary commands and provides a gRPC-based API and Rust bindings for programmatic control.
Platform Support: Currently Linux-only (x86 and ARM) due to its reliance on Linux namespaces for container implementation.
- Test Runner Replacements: Provides drop-in replacements for
Overview of the Maelstrom Plot Library
mainThe Maelstrom Plot Library is a specialized fork of theegui_plotcrate. It is designed specifically for use within the Maelstrom ecosystem, providing enhanced functionality for stacked line plots. To maintain focus on these specific use cases, support for other plot types found in the originalegui_plothas been removed.Overview of Maelstrom Client Library Base
mainThe
maelstrom-client-baselibrary provides the shared core logic used by both the high-level Maelstrom client library and the Maelstrom client process. It serves as the foundational layer for client-side interactions within the Maelstrom ecosystem.For detailed API documentation and usage instructions, refer to the official
maelstrom-clientdocumentation.Overview of Maelstrom FUSE
mainMaelstrom FUSE is a library that provides a FUSE (Filesystem in Userspace) implementation specifically designed for use with the Maelstrom project. It allows Maelstrom-related data or operations to be exposed via a filesystem interface.What is Maelstrom?
mainMaelstrom is a suite of tools designed to run tests in isolated micro-containers. It can run tests locally on a single machine or distribute them across large clusters.
Key features include:
- Drop-in Replacements: Provides replacements for
cargo test,go test, andpytestwith minimal configuration. - Isolation: Every test runs in its own lightweight, rootless container to prevent inter-test dependencies.
- Scalability: Supports cluster mode to increase test throughput by adding worker machines.
- Low Overhead: Uses a custom Rust-based rootless container implementation (not relying on Docker or RunC) optimized for speed.
- General Purpose: Beyond testing, it includes a CLI for arbitrary commands, a gRPC-based API, and Rust bindings for programmatic control.
Platform Support: Currently Linux-only (x86 and ARM) due to its reliance on Linux namespaces for container implementation.
- Drop-in Replacements: Provides replacements for
What is the Maelstrom Worker Binary?
mainThe Maelstrom Worker Binary is a component of the Maelstrom system designed to run on worker machines. Its primary responsibility is the actual execution of jobs. For a Maelstrom cluster to perform any work, it must have at least one worker connected to it.