mistletoe

When choosing between mistletoe and mistune, consider the following trade-offs:

• mistletoe: Focuses on strict CommonMark compliance and extensibility. It uses a highly context-sensitive grammar to correctly handle complex Markdown cases (e.g., precedence of tokens like code spans vs emphasis). This contextual analysis introduces a performance penalty compared to simpler parsers.
• mistune: Focuses on raw speed. It uses a simpler lexing and parsing process that avoids heavy contextual checks, making it faster but potentially unable to handle complex edge cases or specific token precedence rules correctly.

Summary: Use mistune for maximum speed where spec-compliance is secondary. Use mistletoe when you need a spec-compliant, extensible parser that handles complex Markdown structures correctly.

For more control, you can manually create a Document (the AST) and use a renderer.

Important: The parsing phase is tightly connected to the lifecycle of the renderer. You should always call Document(...) inside a with ... as renderer block to ensure that internal token lists are correctly managed and reset.

from mistletoe import Document, HtmlRenderer

with open('foo.md', 'r') as fin:
    with HtmlRenderer() as renderer:     # or: `with HtmlRenderer(AnotherToken1, AnotherToken2) as renderer:`
        doc = Document(fin)              # parse the lines into AST
        rendered = renderer.render(doc)  # render the AST

When mistletoe parses Markdown, it produces an Abstract Syntax Tree (AST) stored in a Document instance. The AST is a hierarchy of tokens.

• Block tokens: Represent lines or blocks of lines (e.g., Document, List, Paragraph, ThematicBreak). They can contain other block tokens, span tokens, or no children.
• Span tokens (or inline tokens): Represent content within a block (e.g., RawText, Link, Emphasis). They can only have span tokens as children.

Every token has children and parent properties for traversing the hierarchy.

Install the mistletoe package using pip3. It is tested for Python 3.5 and above.

pip3 install mistletoe

To inspect the parsed structure of a Markdown file, use the AstRenderer. This outputs the AST as a JSON object, showing token types, attributes (like level for headings or target for links), and line_number for block tokens.

mistletoe text.md --renderer mistletoe.ast_renderer.AstRenderer

To install mistletoe from a local clone in editable mode (so changes to the source are immediately reflected), use the following commands:

git clone https://github.com/miyuchina/mistletoe.git
cd mistletoe
pip3 install -e .

You can benchmark mistletoe against other Markdown parsers (like markdown, mistune, and commonmark) by running the provided benchmark script. This script outputs the time taken in seconds for a specific test document and number of iterations.

To run the benchmarks, use the following command from the repository root:

$ python test/benchmark.py

Because mistletoe splits functionality into modules, function lookup overhead can impact performance compared to single-module parsers. To mitigate this and boost performance, it is recommended to run mistletoe using the PyPy interpreter instead of standard CPython.

Benchmark results indicate that mistletoe performs significantly better on PyPy. You can run the benchmark specifically comparing mistune and mistletoe using PyPy with the following command:

$ pypy3 test/benchmark.py mistune mistletoe

To produce LaTeX output, pass the LaTeXRenderer class as the renderer argument to mistletoe.markdown().

import mistletoe
from mistletoe.latex_renderer import LaTeXRenderer

with open('foo.md', 'r') as fin:
    rendered = mistletoe.markdown(fin, LaTeXRenderer)

Use MarkdownRenderer to reflow text with a specific maximum line length. Note that MarkdownRenderer should be used as a context manager.

import mistletoe
from mistletoe.markdown_renderer import MarkdownRenderer

with open('dev-guide.md', 'r') as fin:
    with MarkdownRenderer(max_line_length=20) as renderer:
        print(renderer.render(mistletoe.Document(fin)))