pdf-craft Documentation

repository·main·Indexed 26 days ago

https://github.com/oomol-lab/pdf-craft

A high-performance tool for converting PDF files, specifically scanned books, into Markdown or EPUB formats. It utilizes DeepSeek OCR for high-fidelity recognition of tables, formulas, and footnotes, supporting local GPU-accelerated processing via CUDA. The library provides configurable options for OCR model sizes (tiny to gundam), table and formula rendering methods, and LLM-powered Table of Contents extraction.

Tokens
11.8K
Snippets
38
Records
81
Agent score
86%

What's inside pdf-craft

  1. Verify pdf-craft Installation

    main

    After installation, verify that both CUDA and Poppler are correctly configured.

    Verify CUDA:

    python -c "import torch; print('CUDA available:', torch.cuda.is_available())"

    Expected output: CUDA available: True

    Verify Poppler:

    pdfinfo -v

    Expected output: Poppler version information.

    python -c "import torch; print('CUDA available:', torch.cuda.is_available())"
    pdfinfo -v
  2. Verify PyTorch and CUDA installation

    main

    Run the following command to ensure PyTorch is correctly installed and to check if CUDA is accessible by the environment.

    python -c "import torch; print(f'PyTorch version: {torch.__version__}'); print(f'CUDA available: {torch.cuda.is_available()}')"
  3. Install Poppler for PDF parsing

    main

    pdf-craft uses Poppler (via pdf2image) for PDF parsing and rendering. You must install it separately based on your operating system:

    • Ubuntu/Debian: sudo apt-get install poppler-utils
    • macOS: brew install poppler
    • Windows: Download the latest Poppler binaries from oschwartz10612/poppler-windows and add the bin/ directory to your system PATH. Alternatively, you can specify the path using the pdf_handler parameter during usage.
    sudo apt-get install poppler-utils
    # or
    brew install poppler
  4. Migrate from v1.0.2 to v1.0.3

    main

    If you are upgrading from version 1.0.2 to 1.0.3, you must perform the following steps due to breaking changes:

    1. Install Poppler on your operating system.
    2. Rename parameters: Change ignore_fitz_errors to ignore_pdf_errors in calls to transform_markdown() and transform_epub().
    3. Update exception handling: Replace FitzError with PDFError in your import and exception catching logic.
    # Before (v1.0.2)
    transform_markdown(..., ignore_fitz_errors=True)
    from pdf_craft import FitzError
    
    # After (v1.0.3)
    transform_markdown(..., ignore_pdf_errors=True)
    from pdf_craft import PDFError
  5. Migrate Table of Contents (TOC) API from v1.0.9 to v1.0.10+

    main

    In version 1.0.10, the toc_mode parameter (which used the TocExtractionMode enum) was replaced by a boolean flag toc_assumed.

    Migration Mapping:

    • Use toc_assumed=False if you previously used TocExtractionMode.NO_TOC_PAGE.
    • Use toc_assumed=True if you previously used TocExtractionMode.AUTO_DETECT or TocExtractionMode.LLM_ENHANCED.

    Note that toc_assumed defaults to False for transform_markdown() and True for transform_epub().

    # For Markdown conversion (assumes no TOC pages by default)
    transform_markdown(
        pdf_path="input.pdf",
        markdown_path="output.md",
        toc_assumed=False,  # New boolean parameter
    )
    
    # For EPUB conversion (assumes TOC pages exist)
    transform_epub(
        pdf_path="input.pdf",
        epub_path="output.epub",
        toc_assumed=True,  # New boolean parameter
    )
  6. Install pdf-craft with CUDA (Recommended)

    main

    To use the full OCR capabilities of pdf-craft, you must install it in a CUDA environment. This requires an NVIDIA GPU with 16 GB+ VRAM (24 GB+ recommended) and Poppler installed.

    1. Verify CUDA

    Check your NVIDIA driver and CUDA version:

    nvidia-smi

    2. Install PyTorch with CUDA support

    Visit the PyTorch official installation page to find the command for your specific CUDA version.

    Example for CUDA 12.1:

    pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121

    3. Install pdf-craft

    pip install pdf-craft

    4. Install Poppler

    pdf-craft requires Poppler for PDF parsing and rendering.

    • Ubuntu/Debian: sudo apt-get install poppler-utils
    • macOS: brew install poppler
    • Windows: Download the latest Poppler binary from oschwartz10612/poppler-windows and add the bin/ directory to your system PATH.
    nvidia-smi
    pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
    pip install pdf-craft
  7. Install dependencies for CPU environments

    main

    For quick development on macOS or Linux without a GPU, install the CPU version of PyTorch along with the project dependencies using poetry run pip.

    poetry run pip install torch torchvision --index-url https://download.pytorch.org/whl/cpu
  8. Manage DeepSeek OCR models

    main

    pdf-craft uses DeepSeek OCR models which are downloaded from Hugging Face on the first run. You can manage these models using predownload_models or by specifying a models_cache_path and using local_only=True for offline mode.

    Pre-download models

    from pdf_craft import predownload_models
    
    predownload_models(
        models_cache_path="models",
        revision=None,
    )

    Offline mode

    To ensure the process only uses local models without attempting network downloads:

    from pdf_craft import transform_markdown
    
    transform_markdown(
        pdf_path="input.pdf",
        markdown_path="output.md",
        models_cache_path="./my_models",
        local_only=True,
    )
  9. Install pdf-craft with CUDA support (Recommended)

    main

    To use pdf-craft for actual PDF conversion and OCR, you must install it in a CUDA-enabled environment. This requires an NVIDIA GPU with at least 16 GB of VRAM (24 GB+ recommended) and CUDA 11.8 or 12.1.

    Prerequisites

    • Python >= 3.10, < 3.14 (3.11.16 recommended)
    • NVIDIA drivers and CUDA installed
    • Poppler installed (see Poppler Installation)

    Installation Steps

    1. Verify CUDA: Run nvidia-smi to check your driver and CUDA version.
    2. Install PyTorch: Install the version matching your CUDA version. For CUDA 12.1:
      pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
    3. Install pdf-craft:
      pip install pdf-craft
    4. Install Poppler: (See Poppler Installation)
    5. Verify Installation:
      • Check CUDA: python -c "import torch; print('CUDA 可用:', torch.cuda.is_available())" (Should output True)
      • Check Poppler: pdfinfo -v (Should output version info)
    nvidia-smi
    pip install torch torchvision --index-url https://download.pytorch.org/whl/cu121
    pip install pdf-craft