DOCSEARCHbyAlgolia
Submit Docs

pixo

repository·main·Indexed 19 days ago

https://github.com/leerob/pixo

A high-performance, minimal-dependency image compression library written in Rust supporting PNG and JPEG formats. Designed for a small footprint and high portability, it is suitable for Rust applications, CLI tools, and WebAssembly-based environments (browser or Node.js). Features include lossless and lossy PNG compression via palette quantization, JPEG encoding with 4:2:0 subsampling, and a set of tuned presets (Fast, Balanced, Max) to balance speed and compression ratio.

Tokens
53K
Snippets
141
Records
227
Agent score
63%

What's inside pixo

  1. Overview of pixo image compression

    main

    pixo is a high-performance, minimal-dependency image compression library written entirely in Rust. It is designed to be lightweight, featuring a small WASM binary (approx. 159 KB) and zero runtime dependencies because all encoding algorithms are implemented from scratch.

    Key capabilities include:

    • PNG Support: Lossless PNG and lossy PNG (via quantization).
    • JPEG Support: Lossy JPEG encoding.
    • Portability: Can be used in Rust projects, via WebAssembly (WASM) in browsers or Node.js, or through a Command Line Interface (CLI).

  2. pixo core features and capabilities

    main

    The pixo library is a compact, well-tested image compression library written in Rust. Its core capabilities include:

    • Format Support: Full encoding and decoding for both PNG and JPEG (baseline).
    • Image Resizing: Built-in support for Nearest, Bilinear, and Lanczos3 algorithms.
    • Compression Engines: Includes DEFLATE, INFLATE, and LZ77 implementations.
    • Performance Optimizations: Supports SIMD (x86_64 and ARM/aarch64) and is optimized for WASM.
    • Deployment Options: Provides a CLI tool and is highly compatible with WebAssembly (WASM).
    • Dependency Profile: Zero external dependencies.

  3. Compare pixo features and format support

    main
    pixo is a zero-dependency image compression library that provides full support for encoding, decoding, and resizing for both PNG and JPEG formats. Unlike many other libraries that delegate to specialized codecs, pixo implements its own decoders (including PNG with INFLATE and JPEG baseline DCT) and encoders from scratch.

  4. What is JPEG Quantization and how does it work?

    main

    Quantization is the lossy step in the JPEG process where information is permanently discarded to reduce file size. It works by dividing each Discrete Cosine Transform (DCT) coefficient by a corresponding value from a quantization table and rounding the result to the nearest integer.

    The mathematical operation: Quantized[i] = round(DCT[i] / Q[i])

    This process is irreversible. During decoding, the value is reconstructed by multiplying the quantized result back by the quantization value (Reconstructed[i] = Quantized[i] × Q[i]), which introduces a permanent error (the difference between the original DCT and the reconstructed value).

    DCT coefficient: 47
Quantization value: 10
Quantized result: round(47/10) = round(4.7) = 5

During decoding:
Reconstructed: 5 × 10 = 50
Error: |47 - 50| = 3

  5. What is a codec?

    main

    A codec (coder-decoder) is a complete system consisting of an Encoder and a Decoder:

    • Encoder: Compresses the original data.
    • Decoder: Decompresses the data back into its original form (for lossless) or an approximation (for lossy).
    ┌─────────────┐         compressed         ┌─────────────┐
│   Encoder   │ ──────────────────────────▶│   Decoder   │
│  (compress) │          data              │(decompress) │
└─────────────┘                            └─────────────┘
      ▲                                           │
      │         original data                     │
      └───────────────────────────────────────────┘
                    (lossless: identical)
                    (lossy: approximation)
    ┌─────────────┐         compressed         ┌─────────────┐
│   Encoder   │ ──────────────────────────▶│   Decoder   │
│  (compress) │          data              │(decompress) │
└─────────────┘                            └─────────────┘
      ▲                                           │
      │         original data                     │
      └───────────────────────────────────────────┘
                    (lossless: identical)
                    (lossy: approximation)

  6. Understand Rust's Ownership and Borrowing

    main

    Rust uses an ownership system to manage memory without a garbage collector. This ensures memory safety and high performance, which is critical for compression tasks.

    Ownership Rules

    1. Each value has exactly one owner.
    2. When the owner goes out of scope, the value is dropped.
    3. Ownership can be transferred (moved) or borrowed.

    Borrowing Rules

    You can use references to access data without taking ownership. At any given time, you can have EITHER:

    • One mutable reference (&mut T)
    • Any number of immutable references (&T)

    But you can never have both simultaneously. This prevents data races at compile time.

    // Example of ownership move
let tokens = compressor.compress(data);
let oops = tokens;     // Ownership moves to 'oops'
// println!("{:?}", tokens); // This would cause a compile error

// Example of borrowing
pub fn compress_into(&mut self, data: &[u8], tokens: &mut Vec<Token>) {
    // data is borrowed immutably (&[u8])
    // tokens is borrowed mutably (&mut Vec<Token>)
    // self is borrowed mutably (&mut self)
}

  7. The three pillars of compression improvement

    main

    Compression advancements generally fall into one of three conceptual categories:

    1. Better Representation: Transforming data into a form that is easier to compress. An example is the Discrete Cosine Transform (DCT), which converts spatial pixel data into frequency data, concentrating most 'energy' into a few coefficients.
    2. Smarter Encoding: Using variable-length codes to represent data. Huffman coding assigns shorter bit-sequences to frequent symbols and longer ones to rare symbols. Arithmetic coding improves on this by treating the entire message as a single number between 0 and 1.
    3. Rate-Distortion Optimization (R-D Optimization): Deciding which data to discard to save bits. This involves a tradeoff between Rate (R) (bit cost) and Distortion (D) (quality loss). The goal is to minimize the function R + λD, where λ is a parameter controlling the tradeoff (higher λ results in smaller files but more quality loss).

  8. Understand bit-level reading conventions in PNG and JPEG

    main

    When implementing or debugging decoders, note that PNG/DEFLATE and JPEG use different bit-ordering conventions:

    • PNG/DEFLATE: Uses LSB-first (Least Significant Bit) order. New bytes fill the upper bits of the buffer.
    • JPEG: Uses MSB-first (Most Significant Bit) order and requires handling byte stuffing (where 0xFF 0x00 is treated as 0xFF).
    // DEFLATE: LSB-first, new bytes fill upper bits
self.bit_buf |= (self.data[self.pos] as u64) << self.bits_in_buf;

// JPEG: MSB-first, new bytes shift left
self.bit_buf = (self.bit_buf << 8) | (byte as u32);

  9. Manage module visibility and structure

    main

    Rust code is organized into modules. By default, all items are private. Use visibility modifiers to control access:

    • pub: Visible everywhere.
    • pub(crate): Visible only within the current crate.
    • pub(super): Visible to the parent module.
    • (nothing): Private to the current module.

    You can also use pub use to re-export items from submodules for a cleaner public API. Conditional compilation via #[cfg(feature = "...")] allows modules to be included only when specific features are enabled.

    pub mod bits;        // Public module
pub mod color;

#[cfg(feature = "simd")]
pub mod simd;        // Only compiled when "simd" feature is enabled

pub use color::ColorType; // Re-exporting for convenience

  10. Understand the hierarchy of performance optimization impact

    main

    When optimizing code, prioritize improvements based on their potential impact. The hierarchy of optimization impact in pixo is:

    1. Algorithm Choice: Can provide 10x-1000x improvement.
    2. Data Structure Selection: Can provide 2x-100x improvement.
    3. Memory Access Patterns: Can provide 2x-10x improvement.
    4. Low-Level Optimizations: Can provide 1.1x-2x improvement.

    Always start with the highest impact area (Algorithm Choice) before moving to low-level micro-optimizations.

  11. How the Huffman coding algorithm works

    main

    Huffman coding is an optimal algorithm for assigning variable-length binary codes to symbols based on their frequency. Common symbols receive shorter codes, while rare symbols receive longer codes, minimizing the total number of bits required for a message.

    The Huffman Algorithm Process

    1. Initialize: Create a leaf node for each symbol with its frequency.
    2. Queue: Insert all nodes into a priority queue (min-heap) ordered by frequency.
    3. Iterate: While more than one node remains in the queue:
      • Remove the two nodes with the lowest frequencies.
      • Create a new internal node with these two as children.
      • Set the new node's frequency to the sum of the children's frequencies.
      • Insert the new node back into the queue.
    4. Root: The final remaining node is the root of the Huffman tree.

    Prefix-Free Property

    To ensure unambiguous decoding, Huffman codes must be prefix-free, meaning no code is a prefix of another. This is naturally guaranteed by the tree structure: symbols are only located at leaf nodes, and paths from the root to a leaf define the code (e.g., left=0, right=1).

    /* Example Huffman Tree Structure */
        (root)
        /    \
       0      1
      /        \
    [A]        ( )
              /   \
             0     1
            /       \
          [B]       ( )
                   /   \
                  0     1
                 /       \
               [C]       [D]

Codes:
  A = 0
  B = 10
  C = 110
  D = 111

  12. Understand LZ77 Compression tokens

    main

    LZ77 compression works by replacing repeated sequences of data with back-references to earlier occurrences. The output is a stream of Tokens. There are two types of tokens:

    1. Literal: A single byte (u8) that could not be matched with previous data.
    2. Match: A back-reference consisting of a length and a distance.

    Match constraints:

    • length: 3 to 258 bytes.
    • distance: 1 to 32,768 bytes back (the sliding window size).

    Minimum match length is set to 3 because encoding a match (length and distance) requires more bits than simply sending the literal bytes for sequences shorter than 3.

    pub enum Token {
    /// A literal byte that couldn't be compressed.
    Literal(u8),
    /// A back-reference: (length, distance).
    Match {
        length: u16,   // 3-258 bytes
        distance: u16, // 1-32768 bytes back
    },
}