repository·master·Indexed 11 days ago
https://github.com/datalab-to/chandraA document intelligence model (version 0.2.0) that converts images and PDFs into structured Markdown, HTML, or JSON. It preserves complex layouts, math, tables, and handwriting, supporting 90 languages. Features include a CLI for batch processing, a vLLM server for production throughput, and an interactive Streamlit app.
chandra-ocr[app] extra, you can launch an interactive Streamlit web application for single-page processing via the chandra_app command.chandra_appTo reproduce the Chandra OCR 2 benchmark score on the upstream olmOCR-bench, follow these steps:
vLLM to serve the datalab-to/chandra-ocr-2 model on port :8000.olmocr[bench] package and the Playwright dependencies required for KaTeX math rendering.Reference Result: ~85.8% overall.
# 1. Clone + install Chandra
git clone https://github.com/datalab-to/chandra.git
cd chandra
pip install -e .
pip install huggingface_hub unicodeit
# 2. Serve the model with vLLM
chandra_vllm
# 3. Install the upstream olmOCR bench
pip install "olmocr[bench]"
playwright install-deps && playwright install chromium
# 4. Run the benchmark
python -m chandra.scripts.olmocr_bench --bench-dir ./olmOCR-bench/bench_dataFor production deployments or high-volume batch processing, you can launch a vLLM server using the chandra_vllm command. This launches a Docker container with optimized inference settings.
Configure the server using the following environment variables:
VLLM_API_BASE: Server URL (default: http://localhost:8000/v1)VLLM_MODEL_NAME: Model name for the server (default: chandra)VLLM_GPUS: GPU device IDs (default: 0)chandra_vllmThe chandra CLI tool allows you to process single files or entire directories. You can specify the inference method using the --method flag.
Inference Methods:
vllm: Uses a vLLM server (default).hf: Uses the local HuggingFace implementation (requires chandra-ocr[hf]).Output Structure: Each processed file generates a subdirectory containing:
<filename>.md: Markdown output<filename>.html: HTML output<filename>_metadata.json: Metadata (page info, token count, etc.)# Single file, with vllm server
chandra input.pdf ./output --method vllm
# Process all files in a directory with local model
chandra ./documents ./output --method hfYou can install Chandra OCR using pip. Depending on your preferred inference backend, choose one of the following installation methods:
vLLM backend (recommended for lightweight installs).torch and transformers. For better performance, it is recommended to also install flash attention.# Base install (for vLLM backend)
pip install chandra-ocr
# With HuggingFace backend (includes torch, transformers)
pip install chandra-ocr[hf]
# With all extras
pip install chandra-ocr[all]Chandra settings can be configured using environment variables or a local.env file. This is useful for managing model checkpoints and vLLM server connections.
# Model settings
MODEL_CHECKPOINT=datalab-to/chandra-ocr-2
MAX_OUTPUT_TOKENS=12384
# vLLM settings
VLLM_API_BASE=http://localhost:8000/v1
VLLM_MODEL_NAME=chandra
VLLM_GPUS=0The behavior of the CLI changes based on the selected --method:
vllm (Default):
--batch-size is not provided, it defaults to 28.--max-workers and --max-retries to manage parallel inference and reliability.hf (HuggingFace):
--batch-size is not provided, it defaults to 1 to avoid memory issues.--max-workers or --max-retries (these are specific to the vLLM implementation in the CLI).The screenshot_app.py script provides a Flask-based web server designed to generate screenshot-ready OCR visualizations. It displays the original image with layout overlays on one side and the extracted markdown on the other.
To run the application, execute the script directly. By default, it listens on 0.0.0.0:8503.
python chandra/scripts/screenshot_app.pyThe chandra CLI supports the following options for fine-tuning document processing:
| Option | Description |
|---|---|
--method [hf|vllm] | Inference method (default: vllm) |
--page-range TEXT | Page range for PDFs (e.g., "1-5,7,9-12") |
--max-output-tokens INTEGER | Max tokens per page |
--max-workers INTEGER | Parallel workers for vLLM |
--include-images/--no-images | Extract and save images (default: include) |
--include-headers-footers/--no-headers-footers | Include page headers/footers (default: exclude) |
--batch-size INTEGER | Pages per batch (default: 28 for vllm, 1 for hf) |
Throughput benchmarks were conducted using vLLM on a single NVIDIA H100 80GB GPU with the olmOCR benchmark set (math, tables, scans, multi-column layouts).
Performance Metrics (96 concurrent sequences):
Note: Real-world usage is estimated to be approximately 2 pages/s.
The chandra.scripts.olmocr_bench module accepts several flags to control the benchmarking process, including concurrency, API endpoints, and processing modes.
| Flag | Description |
|---|---|
--image-dpi <int> | Set the image DPI (default: 300). |
--workers <int> | Number of concurrent vLLM requests. |
--vllm-api-base <url> | Override the vLLM server URL (default: http://localhost:8000/v1). |
--skip-inference | Reuse existing candidate markdown; only perform re-scoring. |
--skip-scoring | Only produce candidate markdown (prints the olmocr command). |
--stock | Use stock settings without Chandra format correction. |
# Example: Running with 32 workers and a custom API base
python -m chandra.scripts.olmocr_bench --bench-dir ./olmOCR-bench/bench_data --workers 32 --vllm-api-base http://localhost:8000/v1