repository·main·Indexed 12 days ago
https://github.com/uutils/coreutilsA cross-platform reimplementation of GNU coreutils written in Rust. It aims to be a drop-in replacement for GNU utilities, providing identical output and error codes with improved performance and internationalization support. Includes a differential fuzzing tool called uufuzz for comparing Rust implementations against reference GNU commands.
uu_split tool is a rudimentary implementation of the split command. It is currently in development and lacks several standard features found in the GNU coreutils version.The uu_cp utility (a Rust implementation of the cp command) tracks its feature parity with GNU cp through a list of completed and pending flags.
Completed features include:
archive, attributes-only, backup, dereference, interactive, link, no-clobber, no-dereference, no-dereference-preserve-links, no-preserve, no-target-directory, one-file-system, parents, paths, preserve, preserve-default-attributes, recursive, reflink, remove-destination, strip-trailing-slashes, suffix, symbolic-link, target-directory, update, verbose.force (Note: Not implemented on Windows), remove-destination (Note: On Windows, only works for writeable files).version.Planned/To Do features:
cli-symbolic-linkscontextcopy-contentssparseuutils' basenc is designed for streaming encoding and decoding, which ensures constant maximum memory usage regardless of input size.
Base64 operations utilize SIMD acceleration via the base64-simd crate, automatically detecting the best available CPU instructions (e.g., SSE2, SSSE3, SSE4.1, AVX2).
Performance gains from SIMD:
base64.The yes utility prints a provided string followed by a newline continuously.
To achieve high throughput, yes avoids a simple println! loop, which is slow due to frequent write syscalls. Instead, it prints an extended string of several bytes per loop iteration to minimize the number of syscalls.
On Linux, the implementation can further optimize performance by using tee() and splice() for non-pipe output. This avoids the overhead of copying content from RAM during read() and write() syscalls, allowing the utility to approach RAM's bandwidth limits.
The dd utility operates in a simple loop: it reads blocksize bytes from an input, optionally performs a conversion, and writes blocksize bytes to an output.
To achieve maximum throughput when copying files or writing to devices (like .iso files to drives), you should optimize the blocksize. Devices typically have an optimal block size; dd performs best when the blocksize is set to that optimal size or a multiple of it.
The wc implementation uses different strategies depending on the requested flags to avoid unnecessary work:
-c)wc attempts to read the file size from the filesystem without inspecting content.splice(): On Linux, splice() can be used to get the input's length while discarding it directly. To test this, pipe uucat into wc: uucat somefile | wc -c.-l) and UTF-8 characters (-m)-clm, the input does not need to be decoded. The input is read in chunks, and the bytecount crate is used to count newlines and/or UTF-8 characters.-w), characters (-m), lines (-l), and maximum line length (-L). Individual steps are toggled based on the flags provided.-wcl.uutils uses a two-tier system to define platform support and testing guarantees. This helps you understand the reliability of the utilities on your target system:
The WASI specification imposes specific constraints on data encoding and I/O operations that differ from standard Linux environments:
argv entries and filenames must be valid UTF-8. Tests using non-UTF-8 bytes in arguments or filenames are incompatible with WASI.mkfifo is skipped.SIGPIPE) and pipe creation. Tests relying on broken pipe detection or pipe-based I/O are skipped.stdin is a seekable file, wasmtime does not preserve the file position between the host and the guest. This affects tests validating stdin offset behavior (e.g., after a head command read).The uu_seq implementation employs several optimization strategies to achieve performance parity with GNU seq:
stdout in a BufWriter prevents excessive system calls caused by unbuffered writes.stdout.write_all(separator.as_bytes())? is faster than using the write! macro with formatting for simple separators.uu_seq uses a custom fast path that performs arithmetic directly on u8 arrays (strings) instead of calling the formatting engine. This provides a 10-20x performance gain and supports large increments and equal width.Because real-world hardware measurements are difficult to reproduce in CI environments, there is a proposal to use CPU simulation for performance regression testing.
Instead of measuring wall-clock time, the goal is to use tools like [cachegrind] to measure execution "time" in a simulated model. In the Rust ecosystem, [iai] is the recommended implementation for this approach.
When adding new microbenchmarks to factor, follow these specific design principles based on Daniel Lemire's methodology:
gcd, ~10µs for factor::table) to maximize sample counts and minimize variability.criterion: Utilize the [criterion] framework and criterion::black_box rather than ad-hoc measurement solutions.The location of Fluent (.ftl) files depends on your build mode:
debug_assertions enabled)Paths are resolved relative to the crate source:
$CARGO_MANIFEST_DIR/../uu/<utility>/locales/
Paths are resolved relative to the executable or standard system paths:
<executable_dir>/locales/<utility>/<prefix>/share/locales/<utility>/~/.local/share/coreutils/locales/<utility>/~/.cargo/share/coreutils/locales/<utility>//usr/share/coreutils/locales/<utility>/If external files are not found in these locations, the system falls back to the embedded English locales.