Camelot PDF Table Extraction

Camelot plans to use mypyc to ship compiled hot-path modules (specifically camelot/utils.py) in pre-built wheels to improve performance.

Key details of the implementation model:

• Default Behavior: When installing via pip install camelot-py on supported platforms, you will receive a pre-built wheel containing the compiled utils.py (as .so or .pyd files) with no extra build dependencies.
• Fallback: On exotic platforms where a pre-built wheel is unavailable, pip will install from the source distribution (sdist), which remains pure-Python and requires no build dependencies.
• Performance Impact: Compiling camelot/utils.py is expected to provide a 2-3× speedup on stream/network parse paths compared to the pure-Python version.
• Limitations: Core classes (like Table) and parsers are not compiled due to dynamic attribute usage and **kwargs constraints in mypyc.

Use the following comparison to decide if Camelot is the right tool for your PDF corpus based on specific requirements:

Note: ✓ = supported out of the box, ✗ = not supported, ◐ = partial/workaround required.

Camelot provides several parsing methods (flavors) depending on the structure of your PDF tables:

• Stream: Best for tables that use whitespace between cells to simulate structure (no visible lines).
• Lattice: Best for tables with demarcated lines (borders) between cells. It is deterministic and can automatically parse multiple tables on a single page.
• Network: A text-based parser that identifies patterns in text bounding boxes to find table structures. Useful for complex text alignments.
• Hybrid: A combination of Network and Lattice. It uses Network to identify cells and Lattice to refine the precision of row/column boundaries using solid lines.
• ML (Table Transformer): An optional neural backend designed for dense borderless tables or scanned/image-only PDFs. It uses a Table Transformer (TATR) model to detect structure while relying on the PDF's text layer (or OCR) for content to prevent hallucinations.

When processing long PDFs, RAM usage grows with the number of pages held in memory. To mitigate this, process the document in page-range chunks and export each chunk to disk. This allows Python to release the intermediate state (text objects, parser caches, and TableList) between calls to read_pdf.

import camelot


def extract_in_chunks(
    filepath,
    total_pages,
    chunk_size=50,
    export_dir=".",
    **read_pdf_kwargs,
):
    """Extract tables a chunk of pages at a time, freeing RAM between chunks.

    Parameters
    ----------
    filepath : str
        Path to the PDF file.
    total_pages : int
        Total page count of the PDF. Get it from any PDF tool, e.g. 
        ``len(playa.parse(open(filepath, "rb").read()).pages)``.
    chunk_size : int, optional (default: 50)
        How many pages to process per ``read_pdf`` call.
    export_dir : str, optional (default: ".")
        Directory in which per-chunk CSVs are written.
    **read_pdf_kwargs
        Any other keyword arguments are forwarded to
        :meth:`camelot.read_pdf` (e.g. ``flavor="stream"``, 
        ``table_areas=...``).
    """
    for start in range(1, total_pages + 1, chunk_size):
        end = min(start + chunk_size - 1, total_pages)
        tables = camelot.read_pdf(
            filepath, pages=f"{start}-{end}", **read_pdf_kwargs
        )
        tables.export(f"{export_dir}/tables_{start}-{end}.csv")