repository·master·Indexed 43 days ago
https://github.com/skbkontur/zstdnetA .NET wrapper for the Zstandard (zstd) native compression library. It provides high-performance, lossless compression and decompression for byte arrays, spans, and streams, including support for dictionary-based compression via DictBuilder. Supports netstandard2.0 and netstandard2.1 with pre-built native libraries for Windows, Linux, and macOS.
ZstdNet provides high-performance compression and decompression capabilities using the Zstandard (zstd) algorithm. Key features include:
byte[]) and Span<byte>.ZstdNet is a .NET wrapper for the Zstd native library, supporting netstandard2.{0|1}. It includes pre-built native shared libraries for Windows (win), Linux (linux), and macOS (osx).
x86, x64, or ARM64).If the native library is not automatically found, you can resolve it manually:
NativeLibrary.SetDllImportResolver.SetDllDirectory from kernel32.dll.The wrapper throws ZstdException if there is malformed data or an error within the underlying libzstd.
A common error is ZSTD_error_dstSize_tooSmall, which occurs if the provided destination buffer is too small for the operation.
The CompressionOptions class stores compression settings and dictionary information.
Important Notes:
IDisposable. Call Dispose() to avoid finalizer thread loading.Configuration Options:
byte[] dict: A compression dictionary (can be loaded from a file or generated via DictBuilder).int compressionLevel: Range from CompressionOptions.MinCompressionLevel to CompressionOptions.MaxCompressionLevel (currently 22). Default is 3.IReadOnlyDictionary<ZSTD_cParameter, int> advancedParams: Allows setting specific parameters like multi-threading or long-distance matching mode.using var options = new CompressionOptions(dict, compressionLevel: 5);
using var compressor = new Compressor(options);
var compressedData = compressor.Wrap(sourceData);The DictBuilder static class allows you to train a compression dictionary from a collection of data samples.
Method:
TrainFromBuffer(IEnumerable<byte[]> samples, int dictCapacity = DefaultDictCapacity): Generates a dictionary from the provided samples.Options:
int dictCapacity: The maximum dictionary size in bytes. Defaults to DictBuilder.DefaultDictCapacity (currently 110 KiB).The DecompressionOptions class stores decompression settings and dictionary information.
Important Notes:
IDisposable.Configuration Options:
byte[] dict: A compression dictionary.IReadOnlyDictionary<ZSTD_dParameter, int> advancedParams: Advanced parameters such as maximum memory usage.using var options = new DecompressionOptions(dict);
using var decompressor = new Decompressor(options);
var decompressedData = decompressor.Unwrap(compressedData);The Compressor class provides block-based compression.
Important Notes:
ThreadStatic or a pool of compressors for bulk processing.IDisposable. Always use using or call Dispose() to avoid pressure on the finalizer thread.Wrap is only reliable if GetCompressBoundLong(size) <= 0x7FFFFFC7. For larger data, use the CompressionStream API.Methods:
Wrap(...): Compresses data into a new buffer or an existing one. Supports byte[], ArraySegment<byte>, ReadOnlySpan<byte>, and Span<byte>.GetCompressBound(int size): Returns the required destination buffer size for a given source size.GetCompressBoundLong(ulong size): Returns the required destination buffer size for a given source size (ulong).using var compressor = new Compressor();
var compressedData = compressor.Wrap(sourceData);The DecompressionStream class implements read-only streaming decompression. It uses ArrayPool<byte>.Shared.Rent() for its internal buffer.
Lifecycle: Implements IDisposable and IAsyncDisposable. Use await using for asynchronous workflows.
Constructor Parameters:
Stream stream: The input stream for reading raw data.DecompressionOptions options: Decompression settings (defaults to null).int bufferSize: The size of the decompression buffer. Defaults to the result of ZSTD_DStreamInSize (~128KB).await using var decompressionStream = new DecompressionStream(inputStream, zstdBufferSize);
await decompressionStream.CopyToAsync(outputStream, copyBufferSize);The CompressionStream class implements write-only streaming compression. It uses ArrayPool<byte>.Shared.Rent() for its internal buffer.
Lifecycle: Implements IDisposable and IAsyncDisposable. Use await using for asynchronous workflows.
Constructor Parameters:
Stream stream: The output stream for writing compressed data.CompressionOptions options: Compression settings (defaults to CompressionOptions.Default).int bufferSize: The size of the compression buffer. Defaults to the result of ZSTD_CStreamOutSize (~128KB), which ensures at least one complete compressed block is flushed.await inputStream.CopyToAsync(compressionStream, copyBufferSize);The Decompressor class provides block-based decompression.
Important Notes:
ThreadStatic or a pool for bulk processing.IDisposable.Unwrap with a new buffer, data can only be saved if a field with the decompressed size is present in the compressed data. You can limit the size using maxDecompressedSize to protect against untrusted data.bufferSizePrecheck is true, the destination buffer size is checked against the decompressed field length before decompression begins.Methods:
Unwrap(...): Decompresses data into a new or existing buffer. Supports byte[], ArraySegment<byte>, ReadOnlySpan<byte>, and Span<byte>.GetDecompressedSize(...): Reads the decompressed data size field from the compressed source.using var decompressor = new Decompressor();
var decompressedData = decompressor.Unwrap(compressedData);