Camelot Documentation

repository·master·Indexed 26 days ago

https://github.com/atlanhq/camelot

A Python library for extracting tables from text-based PDF files. It features two parsing strategies: Lattice for tables with grid lines and Stream for tables relying on whitespace. Camelot integrates with pandas for data analysis and supports exporting to CSV, JSON, Excel, HTML, and SQLite. It provides a CLI and a Python API via `camelot.read_pdf()`, including tools for quality assessment through parsing metrics and visual debugging using matplotlib.

Tokens
6.6K
Snippets
26
Records
45
Agent score
86%

What's inside Camelot

  1. Overview of Camelot PDF Table Extraction

    master
    Camelot is a Python library designed for extracting tables from PDF files. Unlike standard PDF viewers that treat tables as simulated grids of characters on a plane, Camelot provides users with complete control over the extraction process. It is designed to handle the 'fuzzy' nature of real-world PDF tables by allowing users to tweak settings when default extraction fails, offering more flexibility than other open-source or closed-source tools.
  2. Choose a table parsing method: Stream vs Lattice

    master

    Camelot provides two distinct parsing methods for extracting tables from PDF files. Choosing the correct method depends on the visual structure of your PDF tables:

    • Stream: Best for tables that do not have visible grid lines. It relies on whitespace between cells to identify table structures. It works by grouping text into rows based on y axis overlaps and guessing column x ranges based on word distribution.
    • Lattice: Best for tables that have clearly demarcated lines (grid lines) between cells. It is more deterministic and uses computer vision (OpenCV and Ghostscript) to detect line segments, intersections, and table boundaries. It can automatically handle multiple tables on a single page.
  3. Quickstart: Extract tables from a PDF with Camelot

    master

    You can use Camelot to extract tables from text-based PDF files and export them to various formats like CSV, JSON, Excel, or HTML. Each extracted table is available as a pandas DataFrame for easy integration into data analysis workflows.

    Note: Camelot only works with text-based PDFs. If you cannot click and drag to select text in your PDF viewer, it is likely a scanned document and Camelot will not work.

  4. Specify table regions for approximate areas

    master

    If a table's position varies slightly, use table_regions to tell Camelot to look for tables within approximate regions. This is less restrictive than table_areas.

    In the CLI, use the -R flag.

    >>> tables = camelot.read_pdf('table_regions.pdf', table_regions=['170,370,560,270'])
    >>> tables[0].df
    $ camelot lattice -R 170,370,560,270 table_regions.pdf
  5. Specify exact table areas

    master

    To force Camelot to analyze specific boundaries, use the table_areas argument in read_pdf(). This accepts a list of comma-separated strings in the format x1,y1,x2,y2, where (x1, y1) is the top-left and (x2, y2) is the bottom-right in PDF coordinate space (origin is bottom-left).

    In the CLI, use the -T flag.

    >>> tables = camelot.read_pdf('table_areas.pdf', flavor='stream', table_areas=['316,499,566,337'])
    >>> tables[0].df
    $ camelot stream -T 316,499,566,337 table_areas.pdf
  6. Set up a Camelot development environment

    master

    To set up a development environment, clone the repository and install the development dependencies using the [dev] extra.

    $ git clone https://www.github.com/camelot-dev/camelot
    $ pip install camelot-py[dev]
  7. Split text along separators

    master

    If PDFMiner merges strings that should be in separate cells, use split_text=True in read_pdf() to split strings that lie in different cells but were assigned to a single one.

    In the CLI, use the -split flag.

    >>> tables = camelot.read_pdf('column_separators.pdf', flavor='stream', columns=['72,95,209,327,442,529,566,606,683'], split_text=True)
    >>> tables[0].df
    $ camelot -split stream -C 72,95,209,327,442,529,566,606,683 column_separators.pdf
  8. Specify column separators

    master

    When text is too close together and Camelot fails to guess column boundaries correctly, you can explicitly provide the x coordinates for column separators using the columns argument in read_pdf().

    Important Rules:

    • If no table_areas is specified, separators apply to the whole page.
    • If table_areas is provided, the length of the columns list must match the length of the table_areas list. Use an empty string '' in the columns list for any table area where you do not want to specify custom separators.

    In the CLI, use the -C flag.

    >>> tables = camelot.read_pdf('column_separators.pdf', flavor='stream', columns=['72,95,209,327,442,529,566,606,683'])
    >>> tables[0].df
    $ camelot stream -C 72,95,209,327,442,529,566,606,683 column_separators.pdf
  9. Install Camelot using conda

    master

    The easiest way to install Camelot is via conda using the conda-forge channel. Camelot is compatible with Python 2.7, 3.5, and 3.6 on Linux, macOS, and Windows.

    Note for Windows users: You must install ghostscript separately from the Ghostscript downloads page.

    $ conda install -c conda-forge camelot-py
  10. Export extracted tables to various formats

    master

    You can export individual Table objects or an entire TableList to different file formats.

    For a single table: Use .to_csv(), .to_json(), .to_excel(), .to_html(), or .to_sqlite().

    For all tables in a list: Use tables.export(path, f='format').

    • Supported formats for f: 'csv', 'json', 'excel', 'html', 'sqlite'.
    • Note: export() creates files with a page-*-table-* suffix. To bundle all exported files into a single ZIP file, use compress=True.
  11. Read encrypted PDFs

    master

    To extract tables from encrypted PDFs, provide the password using the password argument in read_pdf() or the --password flag in the CLI.

    Limitations:

    • Currently only supports PDFs encrypted with ASCII passwords and algorithm code 1 or 2.
    • If the encryption is unsupported, an exception will be thrown. In such cases, use a tool like QPDF to decrypt the file before processing with Camelot.
    # Python API
    tables = camelot.read_pdf('foo.pdf', password='userpass')
    
    # CLI
    $ camelot --password userpass lattice foo.pdf