PyTurboJPEG

repository·master·Indexed 43 days ago

https://github.com/lilohuang/pyturbojpeg

A high-performance Python wrapper for libjpeg-turbo designed for efficient JPEG decoding and encoding. It supports standard 8-bit JPEG as well as high-precision 12-bit and 16-bit lossless compression (requiring libjpeg-turbo 3.0+). Key features include fast decoding, rescaling, YUV plane support, lossless cropping, and Huffman table optimization.

Tokens
4.5K
Snippets
16
Records
17
Agent score
5%

What's inside PyTurboJPEG

  1. High-precision JPEG requirements and modes

    master

    High-precision support (12-bit and 16-bit) requires libjpeg-turbo 3.0 or later.

    Precision Modes:

    • 12-bit JPEG: Supports both lossy and lossless compression.
    • 16-bit JPEG: Only supports lossless compression due to JPEG standard limitations.
  2. Run PyTurboJPEG tests

    master

    You can run the tests using pytest. The suite includes unit tests for core functions, integration tests, and regression tests.

    Run all tests in the main test file:

    pytest tests/test_turbojpeg.py -v

    Run all tests in the tests directory:

    pytest tests/ -v

    Run specific test classes:

    • Decode tests: pytest tests/test_turbojpeg.py::TestDecode -v
    • Encode tests: pytest tests/test_turbojpeg.py::TestEncode -v
    • Integration tests: pytest tests/test_turbojpeg.py::TestIntegration -v
    • Regression tests (e.g., Buffer Handling): pytest tests/test_turbojpeg.py::TestBufferHandlingRobustness -v

    Run a single specific test:

    pytest tests/test_turbojpeg.py::TestDecode::test_decode_basic -v
    pytest tests/test_turbojpeg.py -v
  3. Generate a coverage report

    master

    To see how much of the turbojpeg package is covered by the tests, use the --cov flag with pytest to generate an HTML report:

    pytest tests/test_turbojpeg.py --cov=turbojpeg --cov-report=html
    pytest tests/test_turbojpeg.py --cov=turbojpeg --cov-report=html
  4. Install the PyTurboJPEG test toolchain

    master

    To run the full test suite, you must first install the system-level TurboJPEG library and then install the project with its test extra to include all testing dependencies (like pytest and pytest-memray).

    System Dependencies:

    • Ubuntu/Debian: sudo apt-get install libturbojpeg
    • macOS: brew install jpeg-turbo

    Python Dependencies: Install the project in editable mode with the test extras:

    python -m pip install -e ".[test]"
    python -m pip install -e ".[test]"
  5. Run memory leak detection and profiling

    master

    The test suite uses pytest-memray to detect memory leaks during repeated function execution. The TestMemoryManagement class includes tests decorated with @pytest.mark.limit_memory() that fail if memory usage exceeds specific thresholds.

    Run memory management tests:

    pytest tests/test_turbojpeg.py::TestMemoryManagement -v

    Generate a memory flamegraph for a specific test:

    pytest --memray tests/test_turbojpeg.py::TestMemoryManagement::test_encode_decode_stress_1000_cycles
    pytest --memray tests/test_turbojpeg.py::TestMemoryManagement::test_encode_decode_stress_1000_cycles
  6. Run performance regression benchmarks

    master

    Performance benchmarks are opt-in and compare the current working tree against the latest release tag. To run them, set the PYTURBOJPEG_RUN_BENCHMARKS environment variable to 1.

    Basic benchmark run:

    PYTURBOJPEG_RUN_BENCHMARKS=1 pytest benchmarks/test_performance.py -v -s

    Benchmark with custom parameters: You can override the baseline version, number of rounds, tolerance percentage, and absolute microsecond allowance:

    PYTURBOJPEG_RUN_BENCHMARKS=1 \
    PYTURBOJPEG_BENCHMARK_BASELINE=v2.4.0 \
    PYTURBOJPEG_BENCHMARK_ROUNDS=15 \
    PYTURBOJPEG_BENCHMARK_MAX_PERCENT=3 \
    PYTURBOJPEG_BENCHMARK_MAX_US=0.15 \
    pytest benchmarks/test_performance.py -v -s
    PYTURBOJPEG_RUN_BENCHMARKS=1 pytest benchmarks/test_performance.py -v -s
  7. Install PyTurboJPEG

    master

    PyTurboJPEG requires libjpeg-turbo 3.0 or later (for PyTurboJPEG 2.0+) and numpy.

    Note: If you need compatibility with libjpeg-turbo 2.x, you must use PyTurboJPEG 1.x.

    macOS

    brew install jpeg-turbo
    pip install -U git+https://github.com/lilohuang/PyTurboJPEG.git

    Windows

    1. Download and install the official libjpeg-turbo installer.
    2. Install PyTurboJPEG:
    pip install -U git+https://github.com/lilohuang/PyTurboJPEG.git

    Linux

    1. Download and install the official libjpeg-turbo installer.
    2. Install PyTurboJPEG:
    pip install -U git+https://github.com/lilohuang/PyTurboJPEG.git
    pip install -U git+https://github.com/lilohuang/PyTurboJPEG.git
  8. Encode medical and scientific images with high precision

    master

    When working with medical images (like DICOM), use encode_12bit with specific pixel formats and subsampling to ensure maximum quality. You can specify pixel_format (e.g., TJPF_GRAY) and jpeg_subsample (e.g., TJSAMP_GRAY) to match the source data.

    import numpy as np
    from turbojpeg import TurboJPEG, TJPF_GRAY, TJSAMP_GRAY
    
    jpeg = TurboJPEG()
    
    # Create a 12-bit medical image (e.g., DICOM format)
    medical_img = np.random.randint(0, 4096, (512, 512, 1), dtype=np.uint16)
    
    # Encode with the highest quality for medical applications
    jpeg_medical = jpeg.encode_12bit(
        medical_img,
        pixel_format=TJPF_GRAY,
        jpeg_subsample=TJSAMP_GRAY,
        quality=100
    )
    
    # Decode for analysis
    decoded_medical = jpeg.decode_12bit(jpeg_medical, pixel_format=TJPF_GRAY)
  9. Use 12-bit JPEG encoding and decoding

    master

    PyTurboJPEG 2.0+ supports 12-bit JPEG, which allows for higher precision than standard 8-bit JPEG. It supports both lossy and lossless compression. Use encode_12bit() for encoding and decode_12bit() for decoding. For 12-bit images, input values should range from 0 to 4095 using np.uint16.

    import numpy as np
    from turbojpeg import TurboJPEG
    
    jpeg = TurboJPEG()
    
    # Create a 12-bit image (values range from 0 to 4095)
    img_12bit = np.random.randint(0, 4096, (480, 640, 3), dtype=np.uint16)
    
    # Encode to 12-bit lossy JPEG
    jpeg_data = jpeg.encode_12bit(img_12bit, quality=95)
    
    # Decode from 12-bit JPEG
    decoded_img = jpeg.decode_12bit(jpeg_data)
  10. Use 12-bit lossless JPEG encoding and decoding

    master

    To achieve perfect reconstruction with 12-bit precision, use encode_12bit() with the lossless=True parameter. This is useful for scientific or medical applications where data integrity is critical. Use decode_12bit() to retrieve the original data.

    import numpy as np
    from turbojpeg import TurboJPEG
    
    jpeg = TurboJPEG()
    
    # Create a 12-bit image
    img_12bit = np.random.randint(0, 4096, (480, 640, 3), dtype=np.uint16)
    
    # Encode to 12-bit lossless JPEG using encode_12bit() with lossless=True
    jpeg_data = jpeg.encode_12bit(img_12bit, lossless=True)
    
    # Decode using decode_12bit()
    decoded_img = jpeg.decode_12bit(jpeg_data)
    
    # Perfect reconstruction
    assert np.array_equal(img_12bit, decoded_img)
  11. Decode JPEG header information

    master

    You can extract image properties without performing a full pixel decode using decode_header(). This is useful for inspecting dimensions, subsampling, or color space before deciding how to decode.

    To select the correct decoding function for high-bit-depth images (12-bit or 16-bit), use return_precision=True to get the bit precision.

    from turbojpeg import TurboJPEG
    
    jpeg = TurboJPEG()
    with open('input.jpg', 'rb') as f:
        jpeg_data = f.read()
        
        # Basic header info
        width, height, jpeg_subsample, jpeg_colorspace = jpeg.decode_header(jpeg_data)
        
        # Get precision for bit-depth selection
        width, height, jpeg_subsample, jpeg_colorspace, precision = jpeg.decode_header(jpeg_data, return_precision=True)
        
        if precision == 8:
            img = jpeg.decode(jpeg_data)
        elif precision == 12:
            img = jpeg.decode_12bit(jpeg_data)
        elif precision == 16:
            img = jpeg.decode_16bit(jpeg_data)
  12. Perform advanced JPEG operations

    master

    PyTurboJPEG provides high-performance lossless and near-lossless operations:

    • scale_with_quality: Scales an image without full color conversion, allowing you to specify a quality factor.
    • crop: Performs a lossless crop. Note that origins must be aligned to the JPEG iMCU grid.
      • preserve=False (default): Rounds origin down and expands the result to keep the requested area.
      • preserve=True: Rounds origin up so the result stays within the requested area.
    • optimize: Performs lossless Huffman table optimization. This re-encodes the image with optimal tables, which typically results in a smaller file size without changing pixels.
    from turbojpeg import TurboJPEG
    
    jpeg = TurboJPEG()
    with open('input.jpg', 'rb') as f:
        jpeg_data = f.read()
    
    # Scale with quality
    scaled_data = jpeg.scale_with_quality(jpeg_data, scaling_factor=(1, 4), quality=70)
    
    # Lossless crop
    cropped_data = jpeg.crop(jpeg_data, 8, 8, 320, 240)
    
    # Huffman optimization
    optimized_data = jpeg.optimize(jpeg_data)