Donut (Document Understanding Transformer)

To install the official implementation from the repository, clone the repo and use conda to create a dedicated environment. This is useful if you want to modify the source or ensure specific dependency versions.

git clone https://github.com/clovaai/donut.git
cd donut/
conda create -n donut_official python=3.7
conda activate donut_official
pip install .

Use test.py to evaluate a trained model on a dataset to obtain inference results and accuracy scores (TED and F1).

Arguments:

• --dataset_name_or_path: Target dataset name (Hugging Face) or local path.
• --pretrained_model_name_or_path: Path to the trained model weights.
• --save_path: File path where predictions and scores will be saved.

python test.py --dataset_name_or_path naver-clova-ix/cord-v2 --pretrained_model_name_or_path ./result/train_cord/test_experiment --save_path ./result/output.json

The easiest way to install the Donut library is via PyPI using pip.

pip install donut-python

Run the train.py script to start training. You can specify a configuration file, a pretrained model (from Hugging Face or local), and the dataset path(s).

Arguments:

• --config: Path to the YAML config file for training.
• --pretrained_model_name_or_path: Hugging Face model name or local path to a pretrained model.
• --dataset_name_or_paths: A JSON-dumped list of dataset names (Hugging Face) or local paths.
• --result_path: Directory where model outputs and artifacts will be saved.
• --exp_version: Version string for the experiment; outputs are stored in {result_path}/{exp_version}/*.

python train.py --config config/train_cord.yaml \
                --pretrained_model_name_or_path "naver-clova-ix/donut-base" \
                --dataset_name_or_paths '["naver-clova-ix/cord-v2"]' \
                --exp_version "test_experiment"

To use SynthDoG, you must have Python 3.6 or higher installed and the synthtiger package. Install synthtiger via pip.

pip install synthtiger

You can interact with the models via Gradio web demos or Google Colab notebooks. To run a local web demo, use the provided app.py file in the repository.

./app.py

Use the synthtiger command to invoke the SynthDoG generator. You need to provide a template script, the generator name (SynthDoG), and a configuration YAML file.

synthtiger -o ./outputs/SynthDoG_en -c 50 -w 4 -v template.py SynthDoG config_en.yaml

SynthDoG supports English (en), Chinese (zh), Japanese (ja), and Korean (ko) through specific configuration files. Replace the placeholders with your desired paths and counts.

# english
synthtiger -o {dataset_path} -c {num_of_data} -w {num_of_workers} -v template.py SynthDoG config_en.yaml

# chinese
synthtiger -o {dataset_path} -c {num_of_data} -w {num_of_workers} -v template.py SynthDoG config_zh.yaml

# japanese
synthtiger -o {dataset_path} -c {num_of_data} -w {num_of_workers} -v template.py SynthDoG config_ja.yaml

# korean
synthtiger -o {dataset_path} -c {num_of_data} -w {num_of_workers} -v template.py SynthDoG config_ko.yaml

If you are running SynthDoG on macOS, you must set the OBJC_DISABLE_INITIALIZE_FORK_SAFETY environment variable to YES to avoid issues during execution.

export OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES

You can launch a Gradio web interface to interact with the Donut model for tasks like Document Visual Question Answering (DocVQA) or other document processing tasks. The demo supports specifying the task type, the pretrained model path, and the network port/URL.

CLI Arguments:

• --task: The task name (e.g., docvqa, rvlcdip, cord). Defaults to docvqa.
• --pretrained_path: The path to the pretrained model. Defaults to naver-clova-ix/donut-base-finetuned-docvqa.
• --port: The port to host the Gradio interface on.
• --url: The server name/URL to host the Gradio interface on.
• --sample_img_path: An optional path to an image to use as an example in the UI.

For DocVQA, use the gt_parses field (a list of dictionaries) to support multiple answers. Each dictionary must contain a question and an answer key.

Example: [{"question" : "what is the model name?", "answer" : "donut"}, {"question" : "what is the model name?", "answer" : "document understanding transformer"}]