DOCSEARCHbyAlgolia
Submit Docs

pdf_oxide

repository·main·Indexed 21 days ago

https://github.com/yfedoseev/pdf_oxide

A high-performance PDF toolkit with a Rust core and bindings for 20 languages, including Clojure, C++, .NET, and Dart. It provides capabilities for text, image, and markdown extraction, PDF creation and editing, and rendering pages as images. Features include Markdown/HTML conversion, form field filling, metadata editing, and OCR support via ONNX Runtime in 'Auto Mode'.

Tokens
407.1K
Snippets
958
Records
1.4K
Agent score
74%

What's inside pdf_oxide

  1. Overview of PDF Interactive Features

    main

    The PDF specification defines several interactive features that allow users to engage with a document via mouse, keyboard, or other pointing devices. These features include:

    • Navigation: Moving through the document at both the document level and page level.
    • Annotations: Adding ancillary information like text notes, sounds, or movies.
    • Actions: Triggering specific behaviors based on user events.
    • Interactive Forms: Gathering information from the user via input fields.
    • Digital Signatures: Authenticating user identity and document validity.
    • Measurement Properties: Displaying real-world units for objects on a page.
    • Viewer Preferences: Controlling the visual presentation of the document.

  2. Python Example Capabilities

    main

    The PDF Oxide Python package supports the following core operations through its example scripts:

    • Text Extraction: Open a PDF, retrieve the page count, and extract text on a per-page basis.
    • Format Conversion: Convert PDF pages into Markdown, HTML, or plain text.
    • PDF Creation: Generate new PDF documents from Markdown, HTML, or plain text sources.
    • Text Search: Perform full-text searches across all pages of a document.
    • Structured Extraction: Extract high-fidelity data including words with bounding boxes, text lines, and tables.
    • Document Editing: Modify PDF metadata, delete specific pages, or merge multiple PDFs.
    • Forms & Annotations: Extract form fields and annotations from documents.
    • Batch Processing: Perform concurrent PDF processing using ProcessPoolExecutor for high performance.

  3. Use Cases for PDF Oxide

    main

    PDF Oxide is designed for high-performance PDF manipulation and extraction. Key use cases include:

    • RAG / LLM pipelines: Converting PDFs to clean Markdown for retrieval-augmented generation.
    • Document processing at scale: Extracting text, images, and metadata from large volumes of PDFs.
    • Data extraction: Pulling structured data from forms, tables, and layouts.
    • PDF generation: Programmatically creating invoices, reports, certificates, and templated documents.
    • PyMuPDF alternative: A faster (5×), MIT-licensed alternative that avoids AGPL restrictions and CPython dependencies.

  4. What is a Linearized PDF and when to use it

    main

    Linearization is an optional PDF feature (available since PDF 1.2) designed to enable efficient incremental access of files over a network (e.g., via HTTP).

    Key Benefits

    • Fast First Page Display: The first page (or an arbitrary requested page) is displayed as quickly as possible.
    • Incremental Loading: Pages are delivered and displayed incrementally as data arrives over slow channels.
    • Early Interaction: Users can interact with elements like links before the entire page has been fully received.

    Best Practices

    • Read-Only Optimization: Linearized PDFs are optimized for read-only documents. They should be generated once and read many times.
    • Avoid Incremental Updates: Performing an incremental update on a linearized PDF will result in a standard (non-linearized) PDF. To restore linearization, the entire file must be reprocessed.

  5. What is a Tagged PDF and why use it?

    main

    A Tagged PDF (defined in PDF 1.4) is a stylized PDF that uses a logical structure framework to allow page content (text, graphics, images) to be extracted and reused.

    Key Use Cases

    • Text/Graphics Extraction: Simple extraction for pasting into other applications.
    • Reflow: Automatic reflow of text and graphics to fit different page sizes.
    • Text Processing: Searching, indexing, and spell-checking.
    • Format Conversion: Converting to HTML, XML, or RTF while preserving structure and basic styling.
    • Accessibility: Making content accessible to users with visual impairments.

    Requirements for Conformance

    A Tagged PDF must adhere to rules regarding:

    1. Page Content: Text must be convertible to Unicode, word breaks must be explicit, and layout artifacts must be distinguished from actual content.
    2. Basic Layout Model: Rules for describing the arrangement of structure elements.
    3. Structure Types: Use of standard types (e.g., paragraphs, headings).
    4. Structure Attributes: Preservation of styling information.
    5. Mark Information: The document's mark information dictionary must have the /Marked entry set to true.

  6. Handle Reverse-Order Show Strings for RTL languages

    main

    For right-to-left (RTL) writing systems like Arabic or Hebrew, text may be represented using show strings where character codes are provided in reverse order.

    To inform a reader that a sequence contains reversed characters, use the ReversedChars marked-content tag. When this tag is used:

    1. The characters within each individual show string are reversed.
    2. The strings themselves remain in their natural reading order.
    3. Show strings may contain a SPACE (U+0020) at the beginning or end to indicate a word break, but must not contain interior spaces.
    /ReversedChars
BMC
( olleH ) Tj
−200 0 Td
( . dlrow ) Tj
EMC

  7. Understand Uncoloured Tiling Patterns

    main

    An uncoloured tiling pattern (Pattern type 1, Paint type 2) is a pattern that has no inherent color. Instead, the color is specified separately whenever the pattern is used. This allows the same pattern shape to be tiled across different regions of a page using different colors.

    To use an uncoloured tiling pattern:

    1. Define the Pattern Color Space: The pattern color space must include a parameter identifying the underlying color space (e.g., DeviceRGB) where the actual color will be specified.
      • Example: [ /Pattern /DeviceRGB ] defines a pattern color space using DeviceRGB.
    2. Set the Color and Pattern: Use the scn (or SCN) operator. The operands must include the color components for the underlying color space followed by the name of the pattern object.

    This mechanism allows you to reuse a single pattern resource while changing its color for different shapes (e.g., painting a circle blue and a triangle red using the same pattern definition).

    % Define a pattern color space with DeviceRGB as underlying
[ /Pattern /DeviceRGB ]

% Set the nonstroking color and pattern (P2) using RGB components
0.77 0.20 0.00 /P2 scn

% Subsequent painting operators like 'f' (fill) will use this color/pattern combo
f

  8. Understand Radio Button field structures

    main

    A radio button field (type Btn) is a collection of related buttons where typically only one button can be in the 'on' state at a time. Selecting one button automatically deselects others.

    Key Concepts:

    • Field Type: Btn.
    • Flags (Ff): Must have the Radio flag set and the Pushbutton flag clear.
    • NoToggleToOff Flag: If set, exactly one button must be selected at all times (clicking the active button does nothing). If clear, clicking the active button deselects it, leaving no button selected.
    • RadiosInUnison Flag: An exception where multiple buttons can share the same 'on' state; turning one on turns them all on.
    • Structure: The parent field's Kids entry contains an array of widget annotations for individual buttons. The parent's V entry holds the name of the currently 'on' child field.
    10 0 obj % Radio button field
<< /FT /Btn
/Ff … % … Radio flag = 1, Pushbutton = 0 …
/T ( Credit card )
/V /cardbrand1
/Kids [ 11 0 R
12 0 R
]
>>
endobj

  9. Implement stencil masking with ImageMask

    main

    A stencil mask (an image XObject with ImageMask set to true) is a monochrome image used to define which areas of the page should be painted with the current color and which should remain unchanged.

    Key characteristics of an ImageMask:

    • The dictionary must not contain a ColorSpace entry.
    • BitsPerComponent must be 1.
    • The Decode array determines behavior:
      • [ 0 1 ] (Default): 0 marks the page with the current color; 1 leaves the previous contents unchanged.
      • [ 1 0 ]: Reverses the meaning (1 marks, 0 leaves unchanged).

    When Interpolate is used with a stencil mask, it smooths the edges of the mask rather than interpolating color values.

    /* Example of an ImageMask dictionary */
<< /Type /XObject
/Subtype /Image
/Width 100
/Height 100
/BitsPerComponent 1
/ImageMask true
/Decode [ 0 1 ]
>>

  10. Color space requirements for transparency groups

    main

    Transparency groups manage color through specific inheritance and conversion rules:

    • Color Space (CS) Specification: A group can specify its own color space in its attributes dictionary or inherit it from the parent group/page.
    • Isolation Rules: Only isolated groups may have an explicitly declared color space. Non-isolated groups must inherit their color space from the parent to avoid complex and potentially impossible color conversions from the backdrop.
    • Conversion: All source colors within a group must be converted to the group's color space before compositing. The resulting color is then interpreted in that color space when composited with the backdrop.
    • Best Practices:
      • For device-independent compositing, use CIE-based color spaces.
      • To avoid unexpected visual results, choose a color space with a linear gamma function. Using nonlinear spaces like sRGB may lead to results that do not match user expectations.
      • Implementations should use high precision for intermediate compositing results to minimize roundoff errors.

  11. Identify types of Artifacts

    main

    Artifacts are categorized by their Type property into four main classes:

    • Pagination artifacts: Ancillary features like running heads and folios (page numbers).
    • Layout artifacts: Cosmetic typographical or design elements (e.g., footnote rules or background screens).
    • Page artifacts: Production aids extraneous to the document (e.g., cut marks, color bars).
    • Background artifacts: Images, patterns, or colored blocks that enhance the visual experience (e.g., a colored background under text). Unlike a Figure, removing a background artifact should not detract from the contextual understanding of the content.

  12. JPXDecode Filter (JPEG2000)

    main

    The JPXDecode filter (introduced in PDF 1.5) decodes data using the JPEG2000 compression method (ISO standard).

    Usage Constraints:

    • It must only be applied to image XObjects (not inline images).
    • It supports images with a single colour component or multiple components.
    • It supports varying bits per sample, with values from 1 to 38 allowed.

    Key Features:

    • Progressions: A single JPEG2000 stream can contain multiple versions of an image (progressions) across sampling resolution, colour depth, band, and location. This allows decoders to extract low-resolution thumbnails or specific tiles for performance benefits.
    • Packaging: Supports both JP2 and JPX formats. The filter expects a full JPX file structure (internal or external).
    • Color Spaces: Supports predefined enumerated colour spaces, restricted ICC profiles, any ICC-1 input profile, and vendor-defined colour spaces. It also supports CMYK (enumerated colour space 12) and CIEJab (enumerated colour space 19).