DOCSEARCHbyAlgolia
Submit Docs

DeepSeek-OCR-2 Documentation

repository·main·Indexed 25 days ago

https://github.com/deepseek-ai/deepseek-ocr-2

A visual causal flow model for high-performance optical character recognition (OCR) and document conversion to markdown. It supports dynamic resolution, layout grounding, and provides inference options via vLLM and the Transformers library.

Tokens
1.2K
Snippets
3
Records
4
Agent score
36%

What's inside DeepSeek-OCR-2

  1. Run Inference using vLLM

    main

    To use vLLM for high-speed inference, navigate to the vLLM directory and configure your paths in DeepSeek-OCR2-master/DeepSeek-OCR2-vllm/config.py (specifically INPUT_PATH and OUTPUT_PATH).

    Available scripts:

    • Image streaming output: python run_dpsk_ocr2_image.py
    • PDF concurrency: python run_dpsk_ocr2_pdf.py (provides speed comparable to DeepSeek-OCR)
    • Batch evaluation: python run_dpsk_ocr2_eval_batch.py (for benchmarks like OmniDocBench v1.5)

  2. Run Inference using Transformers

    main

    You can perform OCR using the transformers library. Ensure CUDA_VISIBLE_DEVICES is set and use trust_remote_code=True when loading the model and tokenizer.

    Alternatively, you can use the provided script in the Hugging Face directory:

    cd DeepSeek-OCR2-master/DeepSeek-OCR2-hf
python run_dpsk_ocr2.py
    from transformers import AutoModel, AutoTokenizer
import torch
import os

os.environ["CUDA_VISIBLE_DEVICES"] = '0'
model_name = 'deepseek-ai/DeepSeek-OCR-2'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(model_name, _attn_implementation='flash_attention_2', trust_remote_code=True, use_safetensors=True)
model = model.eval().cuda().to(torch.bfloat16)

# Prompt options:
# document mode: "<image>\n<|grounding|>Convert the document to markdown. "
# without layouts: "<image>\nFree OCR. "

prompt = "<image>\n<|grounding|>Convert the document to markdown. "
image_file = 'your_image.jpg'
output_path = 'your/output/dir'

res = model.infer(
    tokenizer, 
    prompt=prompt, 
    image_file=image_file, 
    output_path=output_path, 
    base_size=1024, 
    image_size=768, 
    crop_mode=True, 
    save_results=True
)

  3. Install DeepSeek-OCR-2

    main

    To install DeepSeek-OCR-2, use a CUDA 11.8+ and PyTorch 2.6.0 environment. Follow these steps to set up the repository and dependencies:

    1. Clone the repository:
      git clone https://github.com/deepseek-ai/DeepSeek-OCR-2.git
cd DeepSeek-OCR-2
    2. Create and activate a Conda environment:
      conda create -n deepseek-ocr2 python=3.12.9 -y
conda activate deepseek-ocr2
    3. Install required packages (including a specific vLLM wheel):
      pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu118
pip install vllm-0.8.5+cu118-cp38-abi3-manylinux1_x86_64.whl
pip install -r requirements.txt
pip install flash-attn==2.7.3 --no-build-isolation

    Note: If you intend to run both vLLM and Transformers code in the same environment, you may encounter a dependency warning (e.g., vllm 0.8.5+cu118 requires transformers>=4.51.1), but it is noted that you do not need to worry about this for functionality.

    git clone https://github.com/deepseek-ai/DeepSeek-OCR-2.git
conda create -n deepseek-ocr2 python=3.12.9 -y
conda activate deepseek-ocr2
pip install torch==2.6.0 torchvision==0.21.0 torchaudio==2.6.0 --index-url https://download.pytorch.org/whl/cu118
pip install vllm-0.8.5+cu118-cp38-abi3-manylinux1_x86_64.whl
pip install -r requirements.txt
pip install flash-attn==2.7.3 --no-build-isolation

  4. Use Main Prompts for OCR modes

    main

    DeepSeek-OCR-2 supports different prompting styles depending on the desired output:

    • Document mode (with layouts/grounding): Use this to convert a document to markdown with grounding information. prompt = "<image>\n<|grounding|>Convert the document to markdown. "

    • Standard OCR mode (without layouts): Use this for simple OCR tasks. prompt = "<image>\nFree OCR. "

    # document mode
prompt = "<image>\n<|grounding|>Convert the document to markdown. "

# without layouts
prompt = "<image>\nFree OCR. "