docTR (Document Text Recognition)

docTR (Document Text Recognition) is a state-of-the-art Optical Character Recognition (OCR) library powered by PyTorch. It is designed for both automation (parsing textual information for NLU tasks) and research (comparing architectures against state-of-the-art models).

Key capabilities include:

• 2-stage OCR predictors: Robust detection and recognition processes using pretrained parameters.
• Layout analysis: Predictors for detecting document regions such as tables, figures, and headers.
• High performance: Optimized for inference speed on both CPU and GPU, with performance comparable to Google Vision and AWS Textract.
• Ease of use: Minimal dependencies and simple API for loading documents and extracting text.

OnnxTR is an ONNX-based backend for docTR models that enables fast, cross-platform inference using ONNX Runtime. It is designed to be a lightweight alternative that does not require PyTorch or TensorFlow.

Key Features:

• Minimal Dependencies: No PyTorch or TensorFlow required.
• Performance: Optimized for CPU, GPU, and accelerators like OpenVINO via ONNX Runtime.
• Quantization: Supports model quantization for reduced memory usage.
• Deployment: Docker-ready and optimized for servers (with OpenCV headless options).
• API: Provides a familiar inference API similar to docTR.

docling-OCR-OnnxTR is a plugin that integrates the OnnxTR OCR engine into the Docling document parsing framework. It serves as a high-performance, drop-in replacement for traditional OCR engines within Docling.

Configuration Options:

• Integration: Use OnnxtrOcrOptions to connect with Docling pipelines.
• Model Control: Select specific detection and recognition models.
• Tuning: Adjust batch size, confidence thresholds, and multi-language settings.
• Optimization: Supports orientation correction and 8-bit model loading.

A FastAPI template is provided for integrating docTR into an API. To deploy it locally:

1. Install dependencies using Poetry and pip.
2. Run the server using Uvicorn or Docker Compose.

# Setup dependencies
cd api/
pip install poetry
make lock
pip install -r requirements.txt

# Run with Uvicorn
uvicorn --reload --workers 1 --host 0.0.0.0 --port=8002 --app-dir api/ app.main:app

# OR run with Docker Compose
PORT=8002 docker-compose up -d --build

Use doctr.io.DocumentFile to load various document formats into a format compatible with docTR models.

• PDFs: Use from_pdf(path).
• Images: Use from_images(path) for a single image or from_images([path1, path2]) for multiple images. Images should be file paths or NumPy uint8 arrays shaped (H, W, C) in RGB order. Grayscale arrays must be converted to 3-channel before use.
• URLs: Use from_url(url) to load web pages. This requires the html extra: pip install "python-doctr[html]".

from doctr.io import DocumentFile

# From a PDF
doc = DocumentFile.from_pdf("path/to/your/doc.pdf")
# From one or more images
doc = DocumentFile.from_images("path/to/your/img.jpg")
doc = DocumentFile.from_images(["path/to/page1.jpg", "path/to/page2.jpg"])
# From a URL (requires the html extra)
doc = DocumentFile.from_url("https://www.example.com")

The doctr-cli tool allows you to perform full Optical Character Recognition (OCR) on images and PDF files directly from the command line. It exports the extracted text, bounding boxes, and confidence scores into a structured JSON file without requiring Python code.

doctr-cli --input_path path/to/your/document.pdf --output results.json

To share your trained models, you can use login_to_hub and push_to_hf_hub.

Prerequisites:

• A Hugging Face account.
• Git LFS installed on your system.

When pushing, you must specify the task (one of: classification, detection, recognition, or obj_detection), a model_name, and the arch (architecture name). Note that existing repositories will not be overwritten.

from doctr.models import recognition, login_to_hub, push_to_hf_hub

login_to_hub()
my_awesome_model = recognition.crnn_mobilenet_v3_large(pretrained=True)
push_to_hf_hub(
    my_awesome_model, 
    model_name='doctr-crnn-mobilenet-v3-large-french-v1', 
    task='recognition', 
    arch='crnn_mobilenet_v3_large'
)

docTR provides different Predictor types depending on your specific OCR or document analysis task. Each predictor consists of a PreProcessor and a Model (with a post-processor).

To prepare the environment for training layout detection models, install doctr in editable mode and install the required dependencies from the reference requirements file.

pip install -e . --upgrade
pip install -r references/requirements.txt

You can train text recognition models in PyTorch using the references/recognition/train.py script. You can provide local paths to your training and validation datasets or use built-in datasets which are downloaded automatically.

Built-in datasets: CORD, FUNSD, IC03, IIIT5K, SVHN, SVT, and SynthText.

Note: For each split (train/val), you must use either the local path (--train_path / --val_path) or the built-in datasets (--train_datasets / --val_datasets), but not both. If neither is provided, the script falls back to synthetic data generated via WordGenerator.