DOCSEARCHbyAlgolia
Submit Docs

Dolphin Document Image Parsing

repository·master·Indexed 27 days ago

https://github.com/bytedance/dolphin

Dolphin is a document image parsing model by ByteDance that uses a two-stage architecture for classification, layout analysis, and hybrid parsing of digital and photographed documents. It supports page-level and element-level parsing (tables, formulas, text, and code) with structured output in JSON and Markdown formats. The current version includes Dolphin-v2 pre-trained models.

Tokens
790
Snippets
2
Records
6
Agent score
44%

What's inside Dolphin

  1. Perform Element-level Parsing

    master

    Element-level parsing extracts specific document elements such as tables, formulas, text, or code from a cropped image of that element.

    Use demo_element.py and specify the --element_type.

    Supported element types: table, formula, text, or code.

    python demo_element.py --model_path ./hf_model --save_dir ./results --input_path <PATH_TO_ELEMENT_IMAGE> --element_type [table|formula|text|code]

  2. Perform Page-level Parsing

    master

    Page-level parsing converts an entire document page into structured JSON and Markdown formats. You can process single images, single PDFs, or entire directories.

    Use demo_page.py with the following command patterns:

    Process a single image:

    python demo_page.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs/page_1.png

    Process a single PDF:

    python demo_page.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs/page_6.pdf

    Process all documents in a directory:

    python demo_page.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs

    Process with a custom batch size for parallel element decoding:

    python demo_page.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs --max_batch_size 8

  3. Perform Layout and Reading Order Parsing

    master

    Use demo_layout.py to identify element locations and generate the natural reading order sequence for a document.

    Process a single image:

    python demo_layout.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs/page_1.png

    Process a single PDF:

    python demo_layout.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs/page_6.pdf

    Process all documents in a directory:

    python demo_layout.py --model_path ./hf_model --save_dir ./results --input_path ./demo/page_imgs

  4. Install Dolphin

    master

    To use Dolphin, clone the repository, install the required dependencies, and download the pre-trained model.

    1. Clone the repository:
    git clone https://github.com/ByteDance/Dolphin.git
cd Dolphin
    1. Install dependencies:
    pip install -r requirements.txt
    1. Download the Dolphin-v2 pre-trained model using one of the following methods:

    Option A: Using Git LFS

    git lfs install
git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model

    Option B: Using Hugging Face CLI

    pip install huggingface_hub
huggingface-cli download ByteDance/Dolphin-v2 --local-dir ./hf_model
    git clone https://github.com/ByteDance/Dolphin.git
cd Dolphin
pip install -r requirements.txt

  5. Download Dolphin-v2 models

    master

    You can download the pre-trained Dolphin-v2 models from Hugging Face using either git lfs or the huggingface-cli.

    Using Git LFS:

    git lfs install
git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model

    Using Hugging Face CLI:

    pip install huggingface_hub
huggingface-cli download ByteDance/Dolphin-v2 --local-dir ./hf_model
    git lfs install
git clone https://huggingface.co/ByteDance/Dolphin-v2 ./hf_model