Add lz4 to your Rust project
masterTo use the lz4 bindings in your Rust project, add the following dependency to your Cargo.toml file:
[dependencies]
lz4 = "1.23.1"repository·master·Indexed 41 days ago
https://github.com/bozaro/lz4-rsLow-level FFI bindings to the LZ4 C library for Rust, providing support for simple block compression and the LZ4 Frame API for streaming. Version 1.23.1 includes high-level wrappers like EncoderBuilder and Decoder, as well as direct access to LZ4F functions for fine-grained control over block size, checksums, and compression levels.
To use the lz4 bindings in your Rust project, add the following dependency to your Cargo.toml file:
[dependencies]
lz4 = "1.23.1"The lz4 crate provides high-level wrappers around the LZ4 compression algorithm. It exposes Encoder for compression and Decoder for decompression. You can use EncoderBuilder to configure compression settings before creating an encoder.
use lz4::Encoder;
use lz4::Decoder;
// Example usage pattern (conceptual):
// let mut encoder = Encoder::new(writer, block_size, block_mode, content_checksum)?;
// let mut decoder = Decoder::new(reader)?;The CompressionMode enum allows you to select the compression strategy:
HIGHCOMPRESSION(i32): Uses high compression settings with a specified compression parameter.FAST(i32): Uses fast compression with a specified acceleration parameter.DEFAULT: Uses the default LZ4 compression settings.use lz4::block::CompressionMode;
let mode = CompressionMode::HIGHCOMPRESSION(12);
let fast_mode = CompressionMode::FAST(1);
let default_mode = CompressionMode::DEFAULT;When calling compress with prepend_size: true, the resulting Vec<u8> contains a 4-byte u32 header representing the original uncompressed size, followed by the compressed data.
This pattern allows for simpler decompression: when calling decompress, passing None for the uncompressed_size parameter tells the function to parse this 4-byte prefix to determine how much memory to allocate for the output buffer.
// Compression with prefix
let compressed = compress(&data, None, true).unwrap();
// Decompression without needing to know the size beforehand
let original = decompress(&compressed, None).unwrap();To decompress LZ4 frames, use the LZ4F_decompressionContext.
Workflow:
LZ4F_createDecompressionContext with LZ4F_VERSION.LZ4F_getFrameInfo to decode header information (like block size) without starting decompression.LZ4F_decompress. This function returns a hint of how many bytes are expected in the next call. When it returns 0, the frame is fully decoded.LZ4F_freeDecompressionContext.// 1. Create context
let mut ctx = LZ4FDecompressionContext(std::ptr::null_mut());
unsafe { LZ4F_createDecompressionContext(&mut ctx, LZ4F_VERSION) };
// 2. Decompress
// dst_size and src_size are updated by the function
let result = unsafe { LZ4F_decompress(ctx, dst_buffer, &mut dst_size, src_buffer, &mut src_size, &options) };
if result == 0 { /* Frame fully decoded */ }
// 3. Cleanup
unsafe { LZ4F_freeDecompressionContext(ctx) };The Frame API is used for compressing data into a formal LZ4 frame format, which includes headers and checksums. This is suitable for streaming data or creating files.
Workflow:
LZ4F_createCompressionContext with LZ4F_VERSION.LZ4F_compressBegin (writes the header).LZ4F_compressUpdate (can be called repeatedly).LZ4F_flush to compress buffered data without finishing the frame.LZ4F_compressEnd (writes end mark and checksum).LZ4F_freeCompressionContext.// 1. Create context
let mut ctx = LZ4FCompressionContext(std::ptr::null_mut());
let res = unsafe { LZ4F_createCompressionContext(&mut ctx, LZ4F_VERSION) };
// 2. Begin frame (writes header to dst_buffer)
let header_size = unsafe { LZ4F_compressBegin(ctx, dst_buffer, dst_max_size, &preferences) };
// 3. Update (compress data blocks)
let bytes_written = unsafe { LZ4F_compressUpdate(ctx, dst_buffer, dst_max_size, src_buffer, src_size, &options) };
// 4. End frame (finalizes frame, flushes, and writes end mark)
let end_size = unsafe { LZ4F_compressEnd(ctx, dst_buffer, dst_max_size, &options) };
// 5. Cleanup
unsafe { LZ4F_freeCompressionContext(ctx) };You can perform lossless compression using EncoderBuilder and decompression using Decoder.
Use EncoderBuilder::new() to configure compression settings (such as compression level) before calling .build(destination) to create an encoder. After writing data to the encoder, call .finish() to finalize the compression process and retrieve the result.
Use Decoder::new(source) to create a decoder from a source (such as a File). The decoder implements Read, allowing you to use standard I/O utilities like io::copy to decompress data into a destination.
extern crate lz4;
use std::fs::File;
use std::io::{self, Result};
use std::path::{Path};
use lz4::{Decoder, EncoderBuilder};
fn compress(source: &Path, destination: &Path) -> Result<()> {
let mut input_file = File::open(source)?;
let output_file = File::create(destination)?;
// Build an encoder with a specific compression level
let mut encoder = EncoderBuilder::new()
.level(4)
.build(output_file)?;
io::copy(&mut input_file, &mut encoder)?;
// Finalize the compression
let (_output, result) = encoder.finish();
result
}
fn decompress(source: &Path, destination: &Path) -> Result<()> {
let input_file = File::open(source)?;
// Create a decoder from the input file
let mut decoder = Decoder::new(input_file)?;
let mut output_file = File::create(destination)?;
io::copy(&mut decoder, &mut output_file)?;
Ok(())
}You can retrieve the version of the underlying LZ4 library using the lz4::version() function.
```rust
println!(The LZ4FPreferences struct allows you to tune the compression behavior of the Frame API.
| Field | Type | Description |
|---|---|---|
frame_info.block_size_id | BlockSize | Controls the size of compressed blocks (e.g., BlockSize::Default, BlockSize::Max4MB). |
frame_info.block_mode | BlockMode | Linked (default) or Independent. |
frame_info.content_checksum_flag | ContentChecksum | NoChecksum or ChecksumEnabled. |
compression_level | c_uint | 0 is default (fast). Values above 16 are treated as 16. |
auto_flush | c_uint | 1 to always flush, reducing the need for temporary buffers. |
The Decoder<R> struct provides a way to decompress LZ4 data streams. It implements the std::io::Read trait, allowing it to be used with any standard Rust reading utilities (like read_to_end).
To use it, wrap an existing reader R (which must implement std::io::Read) in a new Decoder instance using Decoder::new(r).
use lz4::Decoder;
use std::io::{Read, Cursor};
let compressed_data = vec![/* ... LZ4 compressed bytes ... */];
let mut decoder = Decoder::new(Cursor::new(compressed_data)).unwrap();
let mut decompressed_data = Vec::new();
decoder.read_to_end(&mut decompressed_data).unwrap();If you need an immutable reference to the underlying reader R without consuming the Decoder, you can use the reader() method.
let decoder = Decoder::new(my_reader).unwrap();
let underlying_reader = decoder.reader();For simple, one-shot compression and decompression of data blocks without frame headers, use the following functions. These are ideal for small buffers where the overhead of a full frame is not desired.
// Compress using default settings
let compressed_size = unsafe { LZ4_compress_default(src_ptr, dst_ptr, src_size, max_dst_size) };
// Compress with fast mode and acceleration
let compressed_size = unsafe { LZ4_compress_fast(src_ptr, dst_ptr, src_size, max_dst_size, acceleration) };
// Compress with High Compression (HC) level
let compressed_size = unsafe { LZ4_compress_HC(src_ptr, dst_ptr, src_size, dst_capacity, compression_level) };
// Safe decompression
let decompressed_size = unsafe { LZ4_decompress_safe(src_ptr, dst_ptr, compressed_size, max_dst_size) };