go-pdfium

repository·main·Indexed 43 days ago

https://github.com/klippa-app/go-pdfium

A Go binding for Google's PDFium C++ library providing a thread-safe interface for PDF manipulation, rendering, and text extraction. It supports three threading implementations: single-threaded CGO, multi-threaded CGO via gRPC workers, and a portable WebAssembly implementation using the Wazero runtime.

Tokens
13.5K
Snippets
15
Records
48
Agent score
57%

What's inside go-pdfium

  1. Overview of go-pdfium

    main

    go-pdfium is a fast, multi-threaded, and easy-to-use PDF library for Go applications. It acts as a binding for Google's PDFium C++ library, providing a high-level Go interface to handle complex PDF operations without requiring direct CGO or WebAssembly knowledge.

    Key capabilities include:

    • Document Management: Loading (from bytes, paths, or io.ReadSeeker), saving (to bytes, paths, or io.Writer), and metadata extraction.
    • Rendering: Rendering pages to Go image.Image (RGBA or Grayscale) using DPI or pixel size, and exporting directly to JPEG or PNG.
    • Text & Content: Extracting plain or structured text (including position, size, and font info), searching text, and handling bookmarks, links, and annotations.
    • Editing: Rotating pages, importing pages, flattening, and form filling.
    • Advanced Features: Handling XFA packets, attachments, JavaScript actions, and transformations.
  2. Understand io.ReadSeeker and io.Writer behavior across implementations

    main

    The library supports loading documents via io.ReadSeeker and saving them via io.Writer. However, the efficiency and availability of these interfaces depend on your chosen implementation:

    • Single-threaded or WebAssembly implementations: These work efficiently as they reside in the same process. io.ReadSeeker and io.Writer are fully supported.
    • Multi-threaded (gRPC) implementation:
      • io.ReadSeeker: The library will load the complete file into memory and pass the bytes through the gRPC interface, which is less efficient than direct access.
      • io.Writer: Saving via io.Writer is not possible because io.Writer cannot be encoded with gRPC or shared between processes.
  3. Understand the API stability and Go support policy

    main

    API Stability

    go-pdfium follows semantic versioning (SemVer):

    • Major version: Incremented for breaking changes to exported function signatures.
    • Minor version: Incremented for new features, behaviors, or upgrades to the supported PDFium version.
    • Patch version: Incremented for bug fixes or internal changes.

    Go Version Support

    go-pdfium supports the last two versions of Go. For example, if Go 1.24 is the latest release, the library will support Go 1.23 and 1.24. Changes to supported Go versions are communicated via minor version upgrades.

  4. Verify correctness and bit-identity in go-pdfium

    main

    The go-pdfium implementation ensures correctness across both Native and WebAssembly (Wasm) runtimes through several validation layers:

    • Bit-identical Rendering: The Wasm SIMD kernels are bit-identical to PDFium's scalar paths by construction. Portable patches and exact-match caches (memos) also maintain bit-identity.
    • Test Suite Validation: Every configuration passes the full go-pdfium test suite and renders hash-identically against the golden corpus.
    • PDFium Test Suite: The project runs PDFium's own internal test suite for both native and Wasm configurations. Results are compared as differential sets of failing test names rather than simple counts to ensure identical behavior.
    • Validation Methodology: Patches are validated against a 'null control' (a byte-identical baseline build) to ensure no unintended side effects, and against 'poison builds' (builds with deliberately incorrect constants) to verify that new code paths are actually being executed.
  5. Handle WebAssembly filesystem paths

    main

    In WebAssembly mode, you have control over the filesystem via Wazero mounts.

    • Path Style: All paths must be POSIX style and absolute (e.g., /home/user/file.pdf). Do not use backslashes or volume names.
    • Default Behavior: On non-Windows, it mounts the full root disk. On Windows, it mounts the current working directory volume.
    • Customization: You can define custom mounts by overwriting FSConfig in the webassembly.Config during pool setup.
  6. Choose a threading implementation for go-pdfium

    main

    Because PDFium is not natively multithreaded, go-pdfium provides three distinct implementations that share the same interface. You can switch between them without changing your application logic.

    ImplementationMechanismRequirements
    single_threadedCalls PDFium from the same process via CGO.Requires PDFium installed on the machine and CGO enabled.
    multi_threadedUses multiple workers via CGO (implemented using go-plugin).Requires PDFium installed on the machine and CGO enabled.
    webassemblyUses the Wazero runtime.No external dependencies or CGO required. Best on amd64 and arm64 architectures.

    Important Considerations:

    • Thread Safety: All implementations are thread/subroutine safe. The library uses locking to ensure that PDFium operations are protected.
    • Memory Usage: PDFium can be memory-intensive depending on PDF size and request type. Monitor the number of workers configured in multi_threaded mode.
    • Wazero Performance: On architectures other than amd64 or arm64, Wazero will use an interpreter, which is significantly slower.
  7. Compare Native (CGO) vs WebAssembly performance

    main

    The go-pdfium project supports two backends: a native CGO backend and a WebAssembly (WASM) backend. When choosing between them, consider the following performance characteristics:

    Performance Gap

    • Current State (Scenario B): Against stock (unpatched) native PDFium, the WASM backend is approximately 1.83× slower (geomean).
    • Future State (Scenario A): Once the portable PDFium patches are upstreamed to the official PDFium project, the gap is expected to become a near-uniform ~2.04× slower (geomean) across all document types.

    Why the gap exists

    The performance difference is primarily attributed to runtime overhead rather than algorithmic differences. Key factors include:

    • wazero's single-pass JIT code quality.
    • WASM-level memory bounds checking.
    • 128-bit-only SIMD limitations.

    When to use which

    • Native (CGO): Use when maximum raw performance is required and your environment supports CGO. It is currently faster than WASM because it uses stock PDFium which lacks the specific optimizations applied to the WASM build.
    • WebAssembly (WASM): Use for portability or environments where CGO is not feasible. The performance penalty is predictable (roughly 2×) and consistent across different document types (images, text, CAD-style, etc.).
  8. Understand the PDFium patch inventory and performance effects

    main

    The go-pdfium project utilizes various patches to optimize performance across both Native (CGO) and WebAssembly (WASM) runtimes. These patches are categorized by their portability and target runtime:

    1. Portable patches under upstream review: These are designed to work on both Native and WASM and are currently being reviewed for inclusion in the main PDFium project. They include optimizations for png_predictor_perf, compositor_perf, and stretch_engine_perf.
    2. Portable patches not yet upstreamed: These are validated for both backends but currently live only in the project's fork. Examples include jpeg_decode_bgr, swap_translate_perf, lcms_translate_memo, and t4_psfunc_memo.
    3. WebAssembly-specific patches: These exploit wasm_simd128 or work around Emscripten platform gaps. They are compiled only for the WASM target and include optimizations for the stretch engine, compositor, and glyph blending.

    Performance effects are measured using a 5000-document corpus benchmark. Note that some patches (like fillrect_memset_perf) may actually regress performance on Native runtimes due to how libc memset behaves on certain architectures.

  9. Use Multi-threaded CGO implementation

    main

    Multi-threading in CGO is implemented using HashiCorp's Go Plugin System. It launches separate PDFium worker processes and routes requests via gRPC. This provides better safety, as segfaults in a worker are less likely to corrupt the main Go application.

    1. Create a Worker Package

    The worker package must be named main to be compiled as a binary.

    package main
    
    import (
    	"github.com/klippa-app/go-pdfium/multi_threaded/worker"
    )
    
    func main() {
    	worker.StartWorker(nil)
    }

    2. Configure the Worker Pool

    Initialize the pool in your application (e.g., in an init() function) using multi_threaded.Init.

    package renderer
    
    import (
    	"log"
    	"time"
    
    	"github.com/klippa-app/go-pdfium"
    	"github.com/klippa-app/go-pdfium/multi_threaded"
    )
    
    var pool pdfium.Pool
    var instance pdfium.Pdfium
    
    func init() {
    	pool = multi_threaded.Init(multi_threaded.Config{
    		MinIdle:  1,
    		MaxIdle:  1,
    		MaxTotal: 1,
    		Command: multi_threaded.Command{
    			BinPath: "go", // Use actual binary path in production
    			Args:    []string{"run", "pdfium/worker/main.go"},
    		},
    	})
    
    	var err error
    	instance, err = pool.GetInstance(time.Second * 30)
    	if err != nil {
    		log.Fatal(err)
    	}
    }
  10. Configure CGO implementation prerequisites

    main

    To use the CGO implementation, you must have the PDFium library available via pkgconfig.

    1. Obtain PDFium: Download pre-compiled binaries (e.g., from bblanchon/pdfium-binaries) and extract them to a location like /opt/pdfium.
    2. Configure pkg-config: Create or edit /usr/lib/pkgconfig/pdfium.pc with the following structure, replacing {path} with your extraction path:
    prefix={path}
    libdir={path}/lib
    includedir={path}/include
    
    Name: PDFium
    Description: PDFium
    Version: 8044
    Requires:
    
    Libs: -L${libdir} -lpdfium
    Cflags: -I${includedir}
    1. Set Library Path: Ensure your library path is extended by setting the LD_LIBRARY_PATH environment variable:
    export LD_LIBRARY_PATH={path}/lib
    prefix={path}
    libdir={path}/lib
    includedir={path}/include
    
    Name: PDFium
    Description: PDFium
    Version: 8044
    Requires:
    
    Libs: -L${libdir} -lpdfium
    Cflags: -I${includedir}
  11. Improve JPEG rendering speed with libjpeg-turbo

    main

    By default, go-pdfium uses the standard Go image/jpeg package for simplicity. To achieve significantly faster rendering (up to 3x faster), you can enable libjpeg-turbo using a build tag.

    Requirements:

    • Build time: You must have libturbojpeg-dev installed.
    • Runtime: You must have the libturbojpeg package installed.

    This optimization is supported in both the CGO and WebAssembly implementations (though the WebAssembly implementation currently uses CGO for libjpeg-turbo).

    go build -tags pdfium_use_turbojpeg .
  12. Enable experimental PDFium APIs (CGO)

    main

    By default, experimental PDFium APIs are not compiled to ensure compatibility with most PDFium versions. Calling them will result in an error. To enable them, use the pdfium_experimental build tag during compilation.

    go run -tags pdfium_experimental {package}/{file.go}
    # or
    go build -tags pdfium_experimental {package}/{file.go}