lz4_flex

lz4_flex supports no_std environments, but currently only for the Block Format, as the Frame format requires std::io::Write.

To use it in an environment without an allocator, you must:

1. Disable the alloc feature.
2. Use the _into variants (e.g., compress_into, decompress_into) which operate on user-provided slices.

In these modes, the compression hash table is either placed on the stack (8-16KB depending on input size) or can be provided by the caller via compress_into_with_table.

You can include lz4_flex in your Rust project using Cargo.toml. By default, the library uses safe encoding and decoding via the safe-encode and safe-decode feature flags. For maximum performance, you can disable default features and enable only what you need (e.g., alloc).

# For safe usage (default)
lz4_flex = { version = "0.12" }

# For maximum performance
lz4_flex = { version = "0.12", default-features = false, features = ["alloc"] }

lz4_flex provides feature flags to toggle between safe and unsafe code usage.

• Safe Mode (Default): Uses safe-encode and safe-decode features. This is the recommended setting for most users.
• Unsafe Mode (High Performance): Disable default features to bypass safety checks. This is useful for performance-critical applications where the input is trusted or handled elsewhere.

The CompressTable enum manages the internal hash tables used during compression.

• Small: Uses 16-bit entries. It is more memory-efficient but can only be used for inputs where input.len() < 65535 bytes.
• Large: Uses 32-bit entries and works for any input size.

If you use a Small table with an input that is too large, compress_into_with_table will transparently upgrade it to a Large table. However, it will not automatically downgrade. For performance-critical loops with large inputs, use CompressTable::large() upfront.

The Sink trait defines the interface for writing compressed or decompressed data. Depending on the enabled features, the implementation details change to balance safety and performance:

1. Safe Mode (safe-encode and safe-decode enabled): Uses SliceSink and provides methods like push(byte) and extend_with_fill to ensure memory safety and proper initialization.
2. Unsafe/Fast Mode (Safe features disabled): Uses PtrSink or SliceSink with raw pointer access (pos_mut_ptr) to allow writing directly to uninitialized memory, maximizing throughput.

Common Sink Operations:

• pos(): Returns the current write position.
• capacity(): Returns the total available capacity.
• extend_from_slice(data): Appends a slice of data to the sink.
• push(byte): (Available with safe-encode) Appends a single byte.

If you want to ensure the LZ4 stream is finalized even if you forget to call finish(), use the auto_finish() method. This wraps the FrameEncoder in an AutoFinishEncoder which calls try_finish() during its Drop implementation.

Warning: Because Drop cannot return a Result, any errors encountered during the finalization process (like failing to write the end marker) will be silently ignored. For critical applications where stream integrity must be verified, use finish() or try_finish() instead.

let file = std::fs::File::create("datafile").unwrap();
let compressor = lz4_flex::frame::FrameEncoder::new(file).auto_finish();
// The stream will be finalized when `compressor` goes out of scope.

To compress data using a streaming interface, wrap a std::io::Write implementation in a lz4_flex::frame::FrameEncoder. You must call .finish() on the encoder to ensure all data is flushed and the frame is properly closed.

Example: Compressing data from stdin to stdout:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
let mut rdr = stdin.lock();
// Wrap the stdout writer in a LZ4 Frame writer.
let mut wtr = lz4_flex::frame::FrameEncoder::new(stdout.lock());
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");
wtr.finish().unwrap();

To decompress data using a streaming interface, wrap a std::io::Read implementation in a lz4_flex::frame::FrameDecoder. This decoder implements std::io::Read.

Example: Decompressing data from stdin to stdout:

use std::io;
let stdin = io::stdin();
let stdout = io::stdout();
// Wrap the stdin reader in a LZ4 FrameDecoder.
let mut rdr = lz4_flex::frame::FrameDecoder::new(stdin.lock());
let mut wtr = stdout.lock();
io::copy(&mut rdr, &mut wtr).expect("I/O operation failed");

The Block Format is intended for smaller data chunks because it performs de/compression entirely in memory. For larger datasets, use the Frame format instead.

Use compress_prepend_size to compress data with the size prepended, and decompress_size_prepended to decompress it.

use lz4_flex::block::{compress_prepend_size, decompress_size_prepended};

fn main(){
    let input: &[u8] = b"Hello people, what's up?";
    let compressed = compress_prepend_size(input);
    let uncompressed = decompress_size_prepended(&compressed).unwrap();
    assert_eq!(input, uncompressed);
}