DOCSEARCHbyAlgolia
Submit Docs

PDFFigures 2.0 Documentation

repository·master·Indexed 20 days ago

https://github.com/allenai/pdffigures2

A Scala-based tool designed to extract figures, captions, tables, and section titles from scholarly PDF documents, specifically optimized for computer science papers. It includes CLI tools for bulk processing and debugging, a programmatic Scala interface, and a comprehensive evaluation suite for scoring extractors against datasets using metrics like Intersection over Union (IoU).

Tokens
4.9K
Snippets
17
Records
28
Agent score
72%

What's inside PDFFigures 2.0

  1. How figure and caption extraction is scored

    master

    Extractors must return a figure region, a caption region, the page number, a figure identifier (e.g., "Figure 1"), and optionally the caption text.

    An extraction is considered correct if it meets all these criteria:

    1. The page number matches the ground truth.
    2. The identifier matches the ground truth.
    3. The bounding box overlap score (Intersection over Union / Area of Overlap / Area of Union) is $\ge 0.8$.
    4. For captions: The bounding box meets the IoU criteria, and the caption is also considered correct if its text matches the ground truth text.

    Errors:

    • False Positives (FPs): Extracted figures with page numbers/identifiers not in the gold standard.
    • False Negatives (FNs): Gold standard figures with page numbers/identifiers not found in the extraction.

  2. Understand the Dataset file format and structure

    master

    A dataset is a collection of PDFs and their corresponding annotations. While any format conforming to the Dataset API is acceptable, the standard implementation follows this directory structure:

    • <dataset_directory>/: The root directory for a specific dataset.
    • <dataset_directory>/pdfs/: Contains all PDF files named as <document-id>.pdf.
    • <dataset_directory>/annotations.json: A dictionary where keys are document-ids. Each key maps to a dictionary of Figure objects (using figure names as keys).
    • <dataset_directory>/page_images_color/ and <dataset_directory>/page_images_gray/: Directories storing rasterized images in <document-id>-page-<page#>.{jpg,pgm} format.
    • <dataset_directory>/pages_annotated.json (Optional): Records which specific pages of each PDF are/should be annotated.
    • <dataset_directory>/non_standard_pdfs.txt (Optional): A text file listing document-ids of non-standard PDFs (e.g., OCRed PDFs), optionally followed by an explanation.

    Configuration for rendering DPI, dataset versioning, and page sampling logic (like MAX_PAGES_TO_ANNOTATE or PAGE_SAMPLE_PERCENT) is defined within the objects in datasets.py.

  3. Understand the Figure object output format

    master

    When PDFFigures 2.0 extracts a figure, it produces a 'Figure' object containing the following information:

    1. Page: The 0-based index of the page where the figure occurs.
    2. Bounding Box: Pixel coordinates of the figure within the page (0,0 is top-left of the cropbox, assuming 72 DPI).
    3. Text: Any text found inside the figure.
    4. Caption: The text of the figure's caption.
    5. Caption Bounding Box: The bounding box of the caption.
    6. Name: The deduced name of the figure (e.g., "1" for "Figure 1").
    7. Type: Whether the element was identified as a 'Table' or a 'Figure' based on the caption.

  6. Download PDF datasets for evaluation

    master

    The evaluation datasets consist of PDFs and annotations. Because PDFs are distributed via URLs to avoid copyright issues, you must download them manually before running evaluations. Use the download_from_urls.py script with the -g flag to fetch the required papers.

    python download_from_urls.py -g

  7. Evaluate figure and caption extractors

    master

    To evaluate an extractor (a program that extracts figure/caption bounding regions) against a dataset, use the following scripts:

    • build_evaluation.py: Scores an extractor against a dataset. Takes the dataset name and extractor name as input. Results can be saved to a pickled file using the -o flag.
    • parse_evaluation.py: Reads a pickled evaluation file to print results or provide visualization of ground truth vs. extractor output.
    • compare_evaluation.py: Compares two pickled evaluation files and identifies differences in PDFs and Figures.
    • time_extractor.py: Measures the processing time of an extractor on a corpus without performing scoring.
    # Example: Evaluate the 'pdffigures2' extractor against the 'conference' dataset
python build_evaluation.py conference pdffigures2 -o new_evaluation.pkl

# Example: Compare two evaluation runs
python compare_evaluation.py new_evaluation.pkl old_evaluation.pkl

# Example: Parse and visualize results
python parse_evaluation.py new_evaluation.pkl

  8. Requirements and dependencies for evaluation

    master

    To run the evaluation suite, ensure the following dependencies are installed:

    Python Packages:

    • python3
    • Pillow (required for all evaluations)
    • requests (required if downloading PDFs from URLs)

    System Utilities:

    • poppler-utility (specifically the pdftoppm command) is required to rasterize PDF pages for bounding box comparison and debugging.

  9. Install PDFFigures 2.0

    master

    To install PDFFigures 2.0, clone the repository and run it using sbt.

    Note on Image Formats: Due to licensing, the project does not include all image format libraries by default. If you need to process PDFs containing specific image formats, you should add the following dependencies to your project:

    • com.github.jai-imageio:jai-imageio-core:1.2.1
    • com.github.jai-imageio:jai-imageio-jpeg2000:1.3.0 (for JPEG2000)
    • com.levigo.jbig2:levigo-jbig2-imageio:1.6.5 (for JBIG2)
      "com.github.jai-imageio" % "jai-imageio-core" % "1.2.1",
  "com.github.jai-imageio" % "jai-imageio-jpeg2000" % "1.3.0",
  "com.levigo.jbig2" % "levigo-jbig2-imageio" % "1.6.5"

  11. Use RasterizedFigure for rendered images

    master

    A RasterizedFigure represents a Figure that has been converted into a BufferedImage.

    It includes:

    • figure: The original Figure metadata.
    • imageRegion: A Box specifying the exact, inclusive, integer pixel coordinates the bufferedImage occupies within the page (at the specified dpi). Note that imageRegion might not perfectly correspond to a simple rescaling of figure.regionBoundary due to rounding or post-rasterization cleanup.
    • bufferedImage: The actual java.awt.image.BufferedImage.
    • dpi: The resolution used for rendering.

  12. Understand the Figure and Caption data models

    master

    PDFFigures 2.0 uses several case classes to represent extracted content from PDF documents.

    • Figure: Represents an extracted figure or table. It includes the name, figType (Table or Figure), page number, the associated caption text, imageText (text found within the image), and the bounding boxes for both the caption (captionBoundary) and the figure region (regionBoundary).
    • Caption: Represents the text associated with a figure, including its name, figType, page, text, and boundary.
    • FiguresInDocument: A container for all extracted Figure objects and any failedCaptions in a document.
    • RasterizedFiguresInDocument: A container for RasterizedFigure objects and any failedCaptions in a document.