DOCSEARCHbyAlgolia
Submit Docs

MarkItDown

repository·main·Indexed 13 days ago

https://github.com/microsoft/markitdown

A lightweight Python utility to convert various file formats—including PDF, Office documents, images, and audio—into Markdown. Optimized for LLM-based text analysis pipelines, it preserves document structure like headings, tables, and lists. It includes a Model Context Protocol (MCP) server via the markitdown-mcp package and an OCR plugin for extracting text from images in PDF, DOCX, PPTX, and XLSX files using OpenAI-compatible clients.

Tokens
16K
Snippets
51
Records
73
Agent score
99%

What's inside MarkItDown

  1. Security considerations for MarkItDown-MCP

    main

    The MarkItDown-MCP server has the following security characteristics:

    • No Authentication: The server does not support authentication.
    • User Privileges: It runs with the privileges of the user executing it.
    • Local Binding: When using SSE or Streamable HTTP, it binds to localhost by default. DO NOT bind to other interfaces unless you understand the risks.
    • File Access: The convert_to_markdown tool can read any file the server's user has access to.

    Recommendation: Run the server in a sandboxed environment (like a container or VM) and ensure user permissions are strictly configured to limit access to sensitive files and network segments.

  2. Security considerations for MarkItDown

    main

    MarkItDown performs I/O with the privileges of the current process. It can access any resource that the process itself can access (similar to open() or requests.get()).

    To mitigate risks in untrusted environments:

    1. Sanitize inputs before processing.
    2. Use the narrowest possible conversion function for your specific use case (e.g., convert_stream() or convert_local()) to limit access scope.

  3. Securely use MarkItDown conversion methods

    main

    MarkItDown performs I/O with the privileges of the current process. To ensure security, especially in server-side applications, sanitize your inputs by validating file paths, URI schemes, and network destinations to prevent unauthorized access to private or metadata-service addresses.

    To follow the principle of least privilege, choose the most specific conversion method for your use case rather than using the permissive convert() method:

    • convert_local(): Use this if you only need to read local files.
    • convert_response(): Use this if you want to control URI fetching yourself (e.g., by calling requests.get() first and passing the response object).
    • convert_stream(): Use this for maximum control by opening a stream to the input you want converted.
    • convert(): A permissive method that handles local files, remote URIs, and byte streams.

  4. How the MarkItDown OCR Plugin works

    main

    The plugin integrates into MarkItDown via the markitdown.plugin entry point group. When enable_plugins=True is passed to MarkItDown():

    1. The plugin registers four OCR-enhanced converters at priority -1.0 (ensuring they run before built-in converters at priority 0.0).
    2. During conversion, the plugin extracts embedded images from the document.
    3. Each image is sent to the configured LLM with an extraction prompt.
    4. The returned text is inserted inline, wrapped in specific markers to preserve document structure.
    5. If an LLM call fails, the conversion continues without the image's text.

    Output Format: Extracted OCR text is wrapped as follows:

    *[Image OCR]
<extracted text>
[End OCR]*

  5. Install and use MarkItDown plugins

    main

    Installation

    Install your plugin package using pip. For local development, use editable mode:

    nip install -e .

    Verification

    Verify that the plugin is correctly recognized by MarkItDown using the CLI:

    markitdown --list-plugins

    CLI Usage

    To use plugins during a command-line conversion, include the --use-plugins flag:

    markitdown --use-plugins path-to-file.rtf

    Python Usage

    To enable plugins when using the MarkItDown library in Python, set enable_plugins=True in the MarkItDown constructor.

    nip install -e .
markitdown --list-plugins
markitdown --use-plugins path-to-file.rtf
    from markitdown import MarkItDown

md = MarkItDown(enable_plugins=True)
result = md.convert("path-to-file.rtf")
print(result.text_content)

  6. Develop and test MarkItDown plugins

    main
    You can extend MarkItDown by creating and sharing 3rd-party plugins. For implementation details and guidance on how to build a plugin, refer to the packages/markitdown-sample-plugin directory in the repository.

  7. Implement a custom DocumentConverter plugin

    main

    To create a MarkItDown plugin, you must implement a class that inherits from DocumentConverter. This class requires two primary methods:

    1. accepts(file_stream, stream_info, **kwargs) -> bool: Contains the logic to determine if the converter can handle the provided file stream.
    2. convert(file_stream, stream_info, **kwargs) -> DocumentConverterResult: Contains the logic to transform the file stream into a DocumentConverterResult (Markdown content).

    You can specify a priority during initialization, using PRIORITY_SPECIFIC_FILE_FORMAT to define its precedence.

    from typing import BinaryIO, Any
from markitdown import MarkItDown, DocumentConverter, DocumentConverterResult, StreamInfo, PRIORITY_SPECIFIC_FILE_FORMAT

class RtfConverter(DocumentConverter):

    def __init__(
        self, priority: float = PRIORITY_SPECIFIC_FILE_FORMAT
    ):
        super().__init__(priority=priority)

    def accepts(
        self, 
        file_stream: BinaryIO, 
        stream_info: StreamInfo, 
        **kwargs: Any
    ) -> bool:
        # Implement logic to check if the file stream is an RTF file
        raise NotImplementedError()

    def convert(
        self, 
        file_stream: BinaryIO, 
        stream_info: StreamInfo, 
        **kwargs: Any
    ) -> DocumentConverterResult:
        # Implement logic to convert the file stream to Markdown
        raise NotImplementedError()

  8. Run MarkItDown-MCP in Docker

    main

    To run the server in a containerized environment:

    1. Build the image:

      docker build -t markitdown-mcp:latest .

    2. Run for remote URIs:

      docker run -it --rm markitdown-mcp:latest

    3. Run with local file access: To access local files, mount a directory into the container using a volume. Files in the host directory will be accessible under /workdir inside the container.

      docker run -it --rm -v /home/user/data:/workdir markitdown-mcp:latest
    docker run -it --rm -v /home/user/data:/workdir markitdown-mcp:latest

  9. Configure MarkItDown-MCP for Claude Desktop

    main

    It is recommended to use the Docker image for Claude Desktop integration. Edit your claude_desktop_config.json file to include the server configuration.

    Basic configuration:

    {
  "mcpServers": {
    "markitdown": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "markitdown-mcp:latest"
      ]
    }
  }
}

    Configuration with local directory mounting: To allow Claude to access local files, mount the desired directory:

    {
  "mcpServers": {
    "markitdown": {
      "command": "docker",
      "args": [
        "run",
        "--rm",
        "-i",
        "-v",
        "/home/user/data:/workdir",
        "markitdown-mcp:latest"
      ]
    }
  }
}

  10. Register a MarkItDown plugin

    main

    A plugin must export specific metadata and a registration function to be recognized by MarkItDown:

    1. __plugin_interface_version__: An integer representing the interface version. Currently, only version 1 is supported.
    2. register_converters(markitdown: MarkItDown, **kwargs): A function called during the construction of MarkItDown instances. Inside this function, use markitdown.register_converter() to attach your converter instances.

    Additionally, you must define a package entry point in your pyproject.toml file under the [project.entry-points."markitdown.plugin"] section.

    # The version of the plugin interface that this plugin uses.
# The only supported version is 1 for now.
__plugin_interface_version__ = 1

# The main entrypoint for the plugin.
def register_converters(markitdown: MarkItDown, **kwargs):
    """
    Called during construction of MarkItDown instances to register converters provided by plugins.
    """
    markitdown.register_converter(RtfConverter())
    [project.entry-points."markitdown.plugin"]
sample_plugin = "markitdown_sample_plugin"

  12. Install the MarkItDown OCR Plugin

    main

    To use LLM-based OCR for extracting text from images in PDF, DOCX, PPTX, and XLSX files, install the markitdown-ocr package. Since the plugin relies on an OpenAI-compatible client, you should also ensure a client like openai is installed.

    pip install markitdown-ocr
pip install openai
    pip install markitdown-ocr