Unlimited-OCR

To use Unlimited-OCR with SGLang, set up a Python 3.12 environment using uv, install the specific SGLang wheel and dependencies, and launch the server. The server provides an OpenAI-compatible API.

Server Launch Command:

python -m sglang.launch_server \
    --model baidu/Unlimited-OCR \
    --served-model-name Unlimited-OCR \
    --attention-backend fa3 \
    --page-size 1 \
    --mem-fraction-static 0.8 \
    --context-length 32768 \
    --enable-custom-logit-processor \
    --disable-overlap-schedule \
    --skip-server-warmup \
    --host 0.0.0.0 \
    --port 10000

API Usage Pattern: When sending requests to the /v1/chat/completions endpoint, you must include:

• images_config: An object containing image_mode (either gundam or base).
• custom_logit_processor: The string representation of DeepseekOCRNoRepeatNGramLogitProcessor.to_str().
• custom_params: An object containing ngram_size and window_size.

# Example request payload structure for SGLang
payload = {
    "model": "Unlimited-OCR",
    "messages": [{"role": "user", "content": build_content(prompt, image_paths)}],
    "temperature": 0,
    "skip_special_tokens": False,
    "images_config": {"image_mode": "gundam"},
    "custom_logit_processor": "DeepseekOCRNoRepeatNGramLogitProcessor", # Simplified
    "custom_params": {
        "ngram_size": 35,
        "window_size": 128,
    },
    "stream": True,
}

You can perform OCR inference on NVIDIA GPUs using the transformers library. The model supports two main configurations for single images:

1. gundam: Optimized for single images.
   • base_size=1024, image_size=640, crop_mode=True
2. base: Used for multi-page or PDF parsing.
   • base_size=1024, image_size=1024, crop_mode=False

Requirements (tested on Python 3.12.3 + CUDA 12.9):

torch==2.10.0
torchvision==0.25.0
transformers==4.57.1
Pillow==12.1.1
matplotlib==3.10.8
einops==0.8.2
addict==2.4.0
easydict==1.13
pymupdf==1.27.2.2
psutil==7.2.2

import os
import torch
from transformers import AutoModel, AutoTokenizer

model_name = 'baidu/Unlimited-OCR'

tokenizer = AutoTokenizer.from_pretrained(model_name, trust_remote_code=True)
model = AutoModel.from_pretrained(
    model_name,
    trust_remote_code=True,
    use_safetensors=True,
    torch_dtype=torch.bfloat16,
)
model = model.eval().cuda()

# Single image (gundam config)
model.infer(
    tokenizer,
    prompt='<image>document parsing.',
    image_file='your_image.jpg',
    output_path='your/output/dir',
    base_size=1024, image_size=640, crop_mode=True,
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=128,
    save_results=True,
)

# Multi page / PDF (base config)
model.infer_multi(
    tokenizer,
    prompt='<image>Multi page parsing.',
    image_files=['page1.png', 'page2.png', 'page3.png'],
    output_path='your/output/dir',
    image_size=1024,
    max_length=32768,
    no_repeat_ngram_size=35, ngram_window=1024,
    save_results=True,
)

When evaluating with OmniDocBench, you must strip the <|det|> markers from the raw output. The markers follow the format <|det|>type [bbox]<|/det|>content. The post-processing logic removes these markers, groups lines belonging to the same block with a single newline, and separates different blocks with double newlines.

import re

DET_RE = re.compile(r'<\|det\|>([^<\s]+)(?:\s*\[[^\]]*\])?\s*<\|/det\|>(.*)', re.DOTALL)

def remove_det(raw: str) -> str:
    """
    Strip <|det|>type [bbox]<|/det|> markers, group lines belonging to the
    same block with \n, and separate different blocks with \n\n.
    """
    blocks = []
    cur = None
    for line in raw.splitlines():
        line = line.rstrip()
        if not line:
            continue
        m = DET_RE.match(line)
        if m:
            category, content = m.group(1).strip(), m.group(2).strip()
            if category == 'image':
                continue
            if cur is not None:
                blocks.append(cur)
            cur = [content] if content else []
            continue
        if cur is None:
            cur = []
        cur.append(line)
    if cur is not None:
        blocks.append(cur)
    text = '\n\n'.join('\n'.join(b) for b in blocks).strip()
    return text