DOCSEARCHbyAlgolia
Submit Docs

docstrange Documentation

repository·main·Indexed 23 days ago

https://github.com/nanonets/docstrange

A Python library (v1.1.8) for extracting and converting PDF, Word, PowerPoint, Excel, images, and URLs into LLM-optimized Markdown, structured JSON, HTML, and CSV. It features high-accuracy OCR, layout detection, and a Model Context Protocol (MCP) server for integration with AI applications like Claude Desktop. Supports cloud processing, local GPU processing via CUDA, and a self-hosted web interface.

Tokens
10.7K
Snippets
24
Records
70
Agent score
81%

What's inside docstrange

  1. Understand DocStrange model hosting infrastructure

    main

    DocStrange uses a dual-layer hosting system for its models to ensure high availability and speed:

    1. Primary (S3): Models are hosted in the Nanonets S3 bucket (public-vlms). This is the preferred method because it is faster and requires no authentication.
    2. Fallback (Hugging Face Hub): If the S3 download fails, the system automatically falls back to the original source on the Hugging Face Hub. Note that some models may require authentication when accessed via Hugging Face.

    The ModelDownloader class manages this logic automatically, providing graceful degradation if a model cannot be reached via either source.

  2. How to choose a document processing strategy

    main

    Use get_document_info to determine the best way to process a document based on its token count. The server uses the cl100k_base tokenizer.

    Strategy Guide

    Document SizeRecommended StrategyTools to Use
    Small (< 8k tokens)Process entire documentget_full_content
    Medium (8k - 32k tokens)Full content OR Chunkingget_full_content or get_section_chunks
    Large (32k - 128k tokens)Chunked processingget_section_chunks + get_chunk_content
    Very Large (> 128k tokens)Hierarchical navigationget_hierarchical_structure, get_headers, search_document, or get_section

  3. Install and run the DocStrange Local Web Interface

    main

    DocStrange includes a self-hosted, drag-and-drop web UI for private, offline document conversion. It supports PDF, DOCX, XLSX, PPTX, images, and more. The interface automatically downloads required models on its first run.

    Installation

    Install with web dependencies using:

    pip install "docstrange[web]"

    Running the Interface

    You can start the web interface using one of three methods:

    1. CLI command: docstrange web
    2. Python module: python -m docstrange.web_app
    3. Direct Python import: python -c "from docstrange.web_app import run_web_app; run_web_app()"

    Once running, navigate to http://localhost:8000 (or the port indicated in your terminal).

    Advanced Configuration

    • Custom Port: Use --port <port> via CLI or run_web_app(port=8080) in Python.
    • Development Mode: Enable debug mode using run_web_app(debug=True).
    • Custom Host: To make the UI accessible on your local network, use run_web_app(host='0.0.0.0').

  4. Prepare and upload models to S3

    main

    To manage model hosting on Nanonets S3, use the provided scripts to package models and then upload them using the AWS CLI.

    Note: This is typically a one-time setup process.

    1. Prepare models: Run the preparation script to download models from Hugging Face and package them for S3.
    2. Upload to S3: Use the aws s3 cp command to upload the generated .tar.gz files to the public-vlms bucket with public-read permissions.
    # 1. Prepare models
python scripts/prepare_s3_models.py

# 2. Upload to S3
aws s3 cp dist/layout-model-v2.2.0.tar.gz s3://public-vlms/docstrange/ --acl public-read
aws s3 cp dist/tableformer-model-v2.2.0.tar.gz s3://public-vlms/docstrange/ --acl public-read

  6. Configure Claude Desktop with Docstrange

    main

    To use the Docstrange MCP server with Claude Desktop, add it to your claude_desktop_config.json file. Ensure the path to server.py is an absolute path.

    File Locations:

    • macOS: ~/Library/Application Support/Claude/claude_desktop_config.json
    • Windows: %APPDATA%\Claude\claude_desktop_config.json
    {
  "mcpServers": {
    "docstrange": {
      "command": "python3",
      "args": ["/path/to/docstrange/mcp_server_module/server.py"]
    }
  }
}

  7. Use the DocStrange Web Interface for document extraction

    main

    The web interface allows you to upload documents and convert them into various structured formats.

    Supported File Formats

    • Documents: PDF, Word (.docx, .doc), Excel (.xlsx, .xls), PowerPoint (.pptx, .ppt)
    • Web/Data: HTML, CSV, Text
    • Images: PNG, JPG, TIFF, BMP

    Output Formats

    You can select the following output formats for your extracted content:

    • markdown: Clean, structured markdown text
    • html: Formatted HTML with styling
    • json: Structured JSON data
    • csv: Table data in CSV format
    • flat-json: Simplified JSON structure

    Workflow

    1. Upload: Drag & drop a file or click the upload area to browse.
    2. Process: The system performs real-time extraction with progress indicators.
    3. Review: Use the Preview tab for formatted content or the Raw Output tab for raw text.
    4. Download: Save your results as text or JSON files.

  9. Use the DocumentExtractor Python library

    main

    The DocumentExtractor class is the primary interface for converting documents into various formats like Markdown, HTML, CSV, and JSON. It supports a wide range of inputs including local files (PDF, DOCX, XLSX, PPTX, images) and URLs.

    Basic Extraction Examples

    from docstrange import DocumentExtractor

extractor = DocumentExtractor()

# PDF to Markdown
print(extractor.extract("report.pdf").extract_markdown())

# Word to Data
print(extractor.extract("document.docx").extract_data())

# Excel to CSV
print(extractor.extract("data.xlsx").extract_csv())

# PowerPoint to HTML
print(extractor.extract("slides.pptx").extract_html())

# Image to Text
print(extractor.extract("screenshot.png").extract_text())

# URL to Markdown
print(extractor.extract("https://example.com").extract_markdown())
    from docstrange import DocumentExtractor

extractor = DocumentExtractor()

# PDF document
pdf_result = extractor.extract("report.pdf")
print(pdf_result.extract_markdown())

  11. Set up the DocStrange MCP Server for Claude Desktop

    main

    The DocStrange MCP (Model Context Protocol) server enables intelligent document processing within Claude Desktop, featuring smart token counting, hierarchical navigation, and intelligent chunking. This server is intended for local development and is not included in the PyPI package; you must clone the repository to use it.

    Setup Steps:

    1. Clone the repository and install in development mode: pip install -e ".[dev]".
    2. Add the server to your Claude Desktop configuration file (~/Library/Application Support/Claude/claude_desktop_config.json).
    3. Restart Claude Desktop.
    {
  "mcpServers": {
    "docstrange": {
      "command": "python3",
      "args": ["/path/to/docstrange/mcp_server_module/server.py"]
    }
  }
}

  12. Install the Docstrange MCP Server

    main

    The Docstrange MCP server is designed for local development and must be cloned from the repository. It requires Python 3.10 or higher.

    1. Clone and Install

    git clone https://github.com/nanonets/docstrange.git
cd docstrange
pip install -e ".[dev]"

    2. Install System Dependencies

    You must install poppler for PDF processing based on your OS: