DOCSEARCHbyAlgolia
Submit Docs

PaddleOCR2Pytorch

repository·main·Indexed 22 days ago

https://github.com/frotms/paddleocr2pytorch

A project that ports high-performance OCR models from the PaddlePaddle ecosystem to PyTorch, enabling the use of PP-OCRv6 and PP-StructureV3 without PaddlePaddle dependencies. It includes tools for converting .pdparams weights to .pth, supporting layout detection, table recognition, formula recognition (LaTeX), and seal detection. The library supports multiple model tiers (Tiny, Small, Medium) and various languages, providing both a Python API and CLI for document parsing and inference.

Tokens
21.2K
Snippets
51
Records
79
Agent score
77%

What's inside paddleocr2pytorch

  1. Overview of PaddleOCR2Pytorch

    main

    PaddleOCR2Pytorch is a project designed to port PaddleOCR models to the PyTorch framework. It serves as a learning resource for PaddleOCR, a tool to enable the use of PaddleOCR-trained models within PyTorch environments, and a reference for converting models from Paddle to PyTorch.

    Key Features:

    • High-quality inference models with accurate recognition.
    • Support for ultra-lightweight PP-OCR series (Detection + Direction Classifier + Recognition).
    • Support for ptocr_mobile (mobile) and ptocr_server (server) series.
    • Support for Chinese-English-Number combinations, vertical text, and long text recognition.
    • Multi-language support including Korean, Japanese, German, and French.

  2. Available PTOCR Model Series

    main

    The repository provides several series of high-quality pre-trained PyTorch models:

    • Ultra lightweight PP-OCR series: Includes detection, direction classifier, and recognition models.
    • ptocr_mobile series: Ultra-lightweight models optimized for mobile/edge devices.
    • ptocr_server series: General-purpose models optimized for server-side accuracy.

    Supported capabilities:

    • Chinese, English, and digit recognition.
    • Vertical text recognition.
    • Long text recognition.
    • Multi-language recognition (e.g., Korean, Japanese, German, French).

  3. Understand model type differences in PaddleOCR

    main

    When selecting models, distinguish between the following formats provided by PaddleOCR:

    Model TypeFormatDescription
    Inference Modelinference.pdmodel, inference.pdiparamsUsed for prediction in the Python inference engine.
    Training/Pre-trained Model*.pdparams, *.pdopt, *.statesContains model parameters, optimizer states, and training info; used for evaluation or resuming training.

    Important: For this repository, ensure you are using the .pth (PyTorch) versions of the models.

  4. PP-StructureV3 Document Structure Analysis System

    main

    PP-StructureV3 is a complete document structure analysis pipeline ported to PyTorch. It includes:

    1. Layout Detection: Identifies 23 categories (title, text, table, figure, formula, seal, etc.).
    2. Table Recognition: Uses SLANeXt ViT+GRU architecture with wired/wireless HTML output.
    3. Formula Recognition: Uses PP-FormulaNet (PPHGNetV2+MBart) with LaTeX output.
    4. Seal Detection: Combines DB seal detector + OCR with global OCR fallback.
    5. Doc Preprocessing: Includes orientation detection (doc_ori), unwarping (UVDoc), and textline correction (textline_ori).
    6. Global OCR: Single-pass full-image detection and recognition.
    7. Output Formats: Markdown, JSON, or visualization.

    For detailed implementation details, refer to the PP-StructureV3 Porting Guide.

  5. Understand PP-StructureV3 Pipeline Architecture

    main

    The PP-StructureV3 port follows a specific execution flow to process BGR images into structured formats (Markdown/JSON).

    Execution Flow

    1. Preprocessing (Optional): doc_ori (Orientation) $\rightarrow$ UVDoc (Unwarping).
    2. Layout Detection: PPDocLayout identifies 23 types of regions.
    3. Global OCR: Instead of cropping every region for OCR, the system performs one Global Text Detection (DB) and one Global Text Recognition (SVTR/CRNN) across the whole image. This is more efficient and matches the PaddleX implementation.
    4. Regional Processing: Specific modules handle identified regions:
      • table $\rightarrow$ SLANeXt (outputs HTML).
      • formula $\rightarrow$ PP-FormulaNet (outputs LaTeX).
      • seal $\rightarrow$ DB(seal) + OCR (outputs text).
      • other $\rightarrow$ Text is extracted by finding the intersection between the global OCR bounding boxes and the layout bounding boxes.
    5. Post-processing: Reading order is restored using XY-Cut to generate Markdown, JSON, or visualizations.

    Key Design Note: Layout detection and Global OCR run in parallel to optimize speed.

  6. Understand the PP-OCR Pipeline

    main

    The PP-OCR system is an ultra-lightweight OCR system typically composed of three main stages:

    1. Text Detection: Uses algorithms like DB (Differentiable Binarization) to locate text regions.
    2. Detection Frame Correction: Corrects the orientation/geometry of the detected text boxes.
    3. Text Recognition: Uses algorithms like CRNN to convert the image patches into text strings.

    PP-OCRv2 further optimizes this via CML (Collaborative Mutual Learning) knowledge distillation for detection and LCNet lightweight backbones with U-DML distillation for recognition.

  7. Run Full OCR System (Detection + Classification + Recognition)

    main

    Use predict_system.py to chain detection, angle classification, and recognition into a single pipeline. This is the standard way to perform end-to-end OCR.

    • With Angle Classifier: Add --use_angle_cls and provide --cls_model_path to correct text orientation before recognition.
    • Without Angle Classifier: Simply provide detection and recognition model paths.
    • Configuration: For specific model versions (v3, v4, v5, v6), you must provide the corresponding --det_yaml_path and --rec_yaml_path.
    # Full system with angle classification
python3 ./tools/infer/predict_system.py --image_dir ./doc/imgs --det_model_path your_det_pth_path.pth --rec_model_path your_rec_pth_path.pth --use_angle_cls --cls_model_path your_cls_pth_path.pth

# PP-OCRv6 full pipeline (Medium version)
python ./tools/infer/predict_system.py --use_gpu false --det_algorithm DB --det_yaml_path configs/det/PP-OCRv6/PP-OCRv6_medium_det.yml --det_model_path ./models/v6/ptocr_v6_det_PP-OCRv6_medium_det_pretrained.pth --rec_algorithm CRNN --rec_yaml_path configs/rec/PP-OCRv6/PP-OCRv6_medium_rec.yml --rec_model_path ./models/v6/ptocr_v6_rec_PP-OCRv6_medium_rec_pretrained.pth --rec_image_shape='3,48,320' --rec_char_dict_path ./pytorchocr/utils/dict/ppocrv6_dict.txt --image_dir ./doc/imgs/1.jpg

  8. Compile the PSE post-processing module

    main

    The PSE (Progressive Scale Extraction) post-processing module contains C++ extensions that must be compiled before use. This code is based on the implementation from the PSENet repository. To compile the extensions and place the resulting build files in the current directory, run the setup.py script with the build_ext --inplace flags.

    python3 setup.py build_ext --inplace

  9. Convert Chinese and English General OCR Models to PyTorch

    main

    Use the specific converter scripts in the ./converter/ directory to transform trained PaddleOCR models (v2.0, v3, and v4) into PyTorch format. Most converters require the --src_model_path pointing to the PaddleOCR training directory. Some v4 and v5/v6 models also require a --yaml_path pointing to the corresponding configuration file.

    # PP-OCRv2 Detection
python3 ./converter/ch_ppocr_v2_det_converter.py --src_model_path ./paddle_ch_PP-OCRv2_det_distill_train_dir

# PP-OCRv3 Detection
python ./converter/ch_ppocr_v3_det_converter.py --src_model_path paddle_ch_PP-OCRv3_det_train_dir

# PP-OCRv4 Detection (Student)
python ./converter/ch_ppocr_v4_det_converter.py --yaml_path ./configs/det/ch_PP-OCRv4/ch_PP-OCRv4_det_student.yml --src_model_path ch_PP-OCRv4_det_train_dir

# PP-OCRv5 Mobile Detection
python ./converter/ppocr_v5_det_converter.py --yaml_path configs/det/PP-OCRv5/PP-OCRv5_mobile_det.yml --src_model_path PP-OCRv5_mobile_det_pretrained.pdparams

  10. Convert PaddlePaddle models to PyTorch

    main

    Use the provided converter scripts to transform .pdparams weights from PaddlePaddle into .pth weights for PyTorch.

    Layout Detection

    python converter/ppstructure_layout_converter.py \
    --src_model_path=./models/PP-DocLayout-M_pretrained.pdparams \
    --dst_model_path=./models/ptocr_ppdoclayout_m.pth --variant=M

    Table Structure Recognition

    python converter/ppstructure_slanext_converter.py \
    --src_model_path=./models/SLANeXt_wired_pretrained.pdparams \
    --dst_model_path=./models/ptocr_slanext_wired.pth

    Formula Recognition

    python converter/ppstructure_formula_converter.py \
    --src_model_path=PP-FormulaNet_plus-M_pretrained.pdparams \
    --dst_model_path=ptocr_formulanet_m.pth --variant=M

    Seal Detection

    python converter/ppstructure_seal_converter.py \
    --src_model_path=PP-OCRv4_mobile_seal_det_pretrained.pdparams \
    --dst_model_path=ptocr_seal_det.pth

  11. Convert PaddleOCR weights to PyTorch

    main

    To use models from the original PaddleOCR repository, you must convert them using the provided converter scripts. Each module has a specific converter.

    Layout Detection Conversion

    python converter/ppstructure_layout_converter.py \
    --src_model_path=PP-DocLayout-M_pretrained.pdparams \
    --dst_model_path=ptocr_ppdoclayout_m.pth --variant=M

    Table Recognition Conversion

    python converter/ppstructure_slanext_converter.py \
    --src_model_path=SLANeXt_wired_pretrained.pdparams \
    --dst_model_path=ptocr_slanext_wired.pth

    Formula Recognition Conversion

    python converter/ppstructure_formula_converter.py \
    --src_model_path=PP-FormulaNet_plus-M_pretrained.pdparams \
    --dst_model_path=ptocr_formulanet_m.pth --variant=M

    Seal Detection Conversion

    python converter/ppstructure_seal_converter.py \
    --src_model_path=PP-OCRv4_mobile_seal_det_pretrained.pdparams \
    --dst_model_path=ptocr_seal_det.pth

    Classification (Orientation) Conversion

    Uses the pplcnet_cls_converter.py:

    python converter/pplcnet_cls_converter.py \
    --yaml_path=configs/cls/doc_ori/PP-LCNet_x1_0_doc_ori.yml \
    --src_model_path=PP-LCNet_x1_0_doc_ori_pretrained.pdparams \
    --output_path=ptocr_doc_ori.pth

    Unwarping (UVDoc) Conversion

    python converter/uvdoc_converter.py \
    --src_model_path=UVDoc_pretrained.pdparams \
    --output_path=ptocr_uvdoc.pth

  12. PP-StructureV3 Porting Guide

    main

    PP-StructureV3 is a document structure analysis system ported to PyTorch. It provides full functionality for document parsing with zero Paddle dependencies.

    Core Capabilities:

    1. Layout Detection: Identifies 23 types of document regions (titles, body text, tables, images, formulas, seals, etc.).
    2. Table Structure Recognition: Uses SLANeXt ViT+GRU architecture; outputs HTML for wired/wireless tables.
    3. Formula Recognition: Uses PP-FormulaNet (PPHGNetV2+MBart) with S/M dual models; outputs LaTeX.
    4. Seal Detection: DB seal text detection + OCR, with a fallback to global OCR.
    5. Document Preprocessing: Includes doc_ori (direction classification), UVDoc (unwarping), and textline_ori (text line direction).
    6. Global OCR: Single-pass detection and recognition for the entire image, aligned with PaddleX architecture.
    7. Multi-format Output: Supports Markdown, JSON, and visualization.

    For detailed instructions, see the PP-StructureV3 Porting Guide.