DOCSEARCHbyAlgolia
Submit Docs

pdfplumber Documentation

repository·stable·Indexed 25 days ago

https://github.com/jsvine/pdfplumber

A tool built on top of pdfminer.six for extracting detailed information from PDFs, including text characters, rectangles, lines, and tables. It provides a Python API and a CLI to export data into CSV, JSON, or text formats, with specialized features for machine-generated PDFs, layout analysis, and visual debugging via PageImage.

Tokens
5.7K
Snippets
10
Records
30
Agent score
95%

What's inside pdfplumber

  1. Understand pdfplumber capabilities and limitations

    stable

    Capabilities

    pdfplumber is designed for detailed PDF information extraction and features:

    • Detailed access to individual PDF objects.
    • High-level, customizable methods for extracting text and tables.
    • Integrated visual debugging tools.
    • Utility functions like filtering objects via a crop-box.

    Limitations

    pdfplumber does not provide:

    • PDF generation or modification.
    • Optical Character Recognition (OCR).
    • Strong support for extracting tables from OCR'ed documents.

  3. Extract form field values from a PDF

    stable

    While pdfplumber does not have a high-level API for form data, you can access AcroForm fields using its internal wrappers around pdfminer. This allows you to retrieve field names (T), alternate human-readable names (TU), and current values (V).

    import pdfplumber
from pdfplumber.utils.pdfinternals import resolve_and_decode, resolve

pdf = pdfplumber.open("document_with_form.pdf")

def parse_field_helper(form_data, field, prefix=None):
    resolved_field = field.resolve()
    field_name = '.'.join(filter(lambda x: x, [prefix, resolve_and_decode(resolved_field.get("T"))]))
    if "Kids" in resolved_field:
        for kid_field in resolved_field["Kids"]:
            parse_field_helper(form_data, kid_field, prefix=field_name)
    if "T" in resolved_field or "TU" in resolved_field:
        alternate_field_name  = resolve_and_decode(resolved_field.get("TU")) if resolved_field.get("TU") else None
        field_value = resolve_and_decode(resolved_field["V"]) if 'V' in resolved_field else None
        form_data.append([field_name, alternate_field_name, field_value])

form_data = []
fields = resolve(resolve(pdf.doc.catalog["AcroForm")]["Fields"])
for field in fields:
    parse_field_helper(form_data, field)

  4. Convert PDF BBox coordinates to pdfplumber coordinates

    stable

    The BBox attribute found within the attributes field of a structure element uses PDF coordinate space, where the origin is at the bottom-left of the page. To use these coordinates in pdfplumber's coordinate system (where the origin is at the top-left), you must transform them using the page height and initial_doctop.

    x0, y0, x1, y1 = element['attributes']['BBox']
top = page.height - y1
bottom = page.height - y0
doctop = page.initial_doctop + top
bbox = (x0, top, x1, bottom)

  5. Repair malformed PDFs using pdfplumber

    stable

    If you encounter parsing issues due to malformed PDFs, pdfplumber can automatically run repairs using Ghostscript. You can repair PDFs in three ways:

    1. On-the-fly repair: Pass repair=True to pdfplumber.open() to repair the PDF during the session without saving a file to disk.
    2. In-memory repair: Use pdfplumber.repair(path_to_pdf) to get a BytesIO object containing the repaired bytes.
    3. Save to disk: Use pdfplumber.repair(path_to_pdf, outfile="path/to/repaired.pdf") to write the repaired version directly to a file.

  7. Configure table extraction settings

    stable

    The table_settings dictionary allows you to customize how pdfplumber identifies table borders and cells.

    Strategies

    Both vertical_strategy and horizontal_strategy accept:

    • "lines": Uses graphical lines and rectangle edges.
    • "lines_strict": Uses graphical lines but ignores rectangle edges.
    • "text": Deduces lines based on word alignment (left, right, center, or top).
    • "explicit": Uses only the lines provided in explicit_vertical_lines or explicit_horizontal_lines.

    Key Configuration Keys

    KeyDescription
    vertical_strategyStrategy for vertical cell separators.
    horizontal_strategyStrategy for horizontal cell separators.
    explicit_vertical_linesList of x-coordinates or line/rect/curve objects to use as vertical separators.
    explicit_horizontal_linesList of y-coordinates or line/rect/curve objects to use as horizontal separators.
    snap_toleranceTolerance for snapping parallel lines together.
    join_toleranceTolerance for joining line segments on the same infinite line.
    edge_min_lengthMinimum length for an edge to be considered.
    text_*Settings passed to Page.extract_text() for text extraction within cells.

    Note: It is often helpful to use Page.crop(bounding_box) to isolate a specific area of a page before calling table extraction methods.

    {
    "vertical_strategy": "lines", 
    "horizontal_strategy": "lines",
    "explicit_vertical_lines": [],
    "explicit_horizontal_lines": [],
    "snap_tolerance": 3,
    "snap_x_tolerance": 3,
    "snap_y_tolerance": 3,
    "join_tolerance": 3,
    "join_x_tolerance": 3,
    "join_y_tolerance": 3,
    "edge_min_length": 3,
    "edge_min_length_prefilter": 1,
    "min_words_vertical": 3,
    "min_words_horizontal": 1,
    "intersection_tolerance": 3,
    "intersection_x_tolerance": 3,
    "intersection_y_tolerance": 3,
    "text_tolerance": 3,
    "text_x_tolerance": 3,
    "text_y_tolerance": 3,
}

  8. Inspect character properties and rotation

    stable

    Each character in .chars is a dictionary containing spatial and stylistic data. To handle character rotation, use the pdfplumber.ctm submodule to process the matrix property.

    Key Character Properties:

    • text: The character string.
    • fontname, size: Font details.
    • x0, x1, y0, y1, top, bottom: Spatial coordinates.
    • matrix: The current transformation matrix (CTM).
    • stroking_color, non_stroking_color: Color information.

    Example: Calculating Character Rotation

    from pdfplumber.ctm import CTM

# Assuming 'pdf' is an open pdfplumber.PDF instance
my_char = pdf.pages[0].chars[3]
my_char_ctm = CTM(*my_char["matrix"])
my_char_rotation = my_char_ctm.skew_x
    from pdfplumber.ctm import CTM
my_char = pdf.pages[0].chars[3]
my_char_ctm = CTM(*my_char["matrix"])
my_char_rotation = my_char_ctm.skew_x

  9. Search for text patterns on a page

    stable

    The .search(pattern, ...) method (experimental) allows you to find specific text or regex matches on a page. It returns a list of dictionaries containing the matching text, regex groups, bounding box, and character objects.

    Arguments:

    • pattern: A compiled regex, uncompiled regex, or a non-regex string.
    • regex (bool): If False, the pattern is treated as a literal string. Default: True.
    • case (bool): If False, performs case-insensitive search. Default: True.
    • main_group (int): Restricts results to a specific regex group. Default: 0 (entire match).
    • return_groups (bool): Whether to include regex groups in the result. Default: True.
    • return_chars (bool): Whether to include character objects in the result. Default: True.
    • layout (bool): Operates like the layout parameter in .extract_text().

    Note: Zero-width and all-whitespace matches are discarded.

  10. Extract text from a PDF page

    stable

    Use the following methods on a Page object to extract text content:

    • .extract_text(x_tolerance=3, x_tolerance_ratio=None, y_tolerance=3, layout=False, x_density=7.25, y_density=13, line_dir_render=None, char_dir_render=None, **kwargs): Collates characters into a single string.
      • If layout=False: Uses x_tolerance and y_tolerance to manage spaces and newlines.
      • If layout=True (experimental): Attempts to mimic the visual structural layout using x_density and y_density.
    • .extract_text_simple(x_tolerance=3, y_tolerance=3): A faster, less flexible version of .extract_text().
    • .extract_text_lines(layout=False, strip=True, return_chars=True, **kwargs) (experimental): Returns a list of dictionaries representing lines of text. Use strip=True to remove surrounding whitespace from the text attribute.

  11. Load a PDF with pdfplumber.open()

    stable

    Use pdfplumber.open(x) to create a pdfplumber.PDF instance. The argument x can be a path to a PDF file, a file object loaded as bytes, or a file-like object loaded as bytes.

    Common Configuration Options:

    • password: Pass a string to load password-protected PDFs.
    • laparams: Pass a dictionary to set layout analysis parameters for the pdfminer.six engine (e.g., {"line_overlap": 0.7}).
    • unicode_norm: Pre-normalize Unicode text using one of the four forms: "NFC", "NFD", "NFKC", or "NFKD".
    • strict_metadata: If True, pdfplumber.open will raise an exception if it cannot parse the metadata (defaults to treating invalid metadata as a warning).

  12. Use PageImage methods to manage and save images

    stable

    Once you have a PageImage object (im), you can manipulate or save it using the following methods:

    • im.reset(): Clears all drawings made on the image.
    • im.copy(): Creates a copy of the PageImage.
    • im.show(): Opens the image in your local system's image viewer.
    • im.save(path_or_fileobject, format="PNG", quantize=True, colors=256, bits=8): Saves the image as a PNG. By default, it uses 8-bit color depth with 256 colors (quantized). You can disable this with quantize=False or adjust colors=N.