markdown-it-py Documentation

markdown-it-py is a high-speed Python port of the markdown-it JavaScript library. It is designed to be a highly configurable and pluggable Markdown parser. Key features include:

• CommonMark Compliance: Follows the CommonMark spec for baseline parsing.
• Configurable Syntax: Allows adding new rules or replacing existing ones.
• Pluggable Architecture: Supports syntax extensions via plugins.
• High Performance: Optimized for speed.
• Security: Easy to configure for secure parsing environments.

To improve typography, you can enable smartquotes (converts quote marks to opening/closing variants) and replacements (converts constructs like (c) to ©).

Note: Both require the typographer option to be set to True in the configuration dictionary.

md = MarkdownIt("commonmark", {"typographer": True})
md.enable(["replacements", "smartquotes"])
md.render("'single quotes' (c)")

Rules are functions that manipulate the parser state objects. They are managed by Ruler instances and can be enabled or disabled via MarkdownIt methods.

Important design principles for rules:

• Independence: Rules are designed to be independent so they can be safely added, removed, or toggled.
• Write-only Stream: During the block and inline parse stages, the token stream is treated as "write-only" in certain validation modes where rules only look ahead to find the end of a token without modifying the stream.
• Association: A single token type might be associated with multiple chains (e.g., a blockquote token is associated with blockquote, paragraph, heading, and list chains).

Instead of a traditional Abstract Syntax Tree (AST), markdown-it-py uses a low-level token stream (a simple Array/list).

Key characteristics of the token stream:

• Flat Structure: Opening and closing tags are represented as separate, paired tokens.
• Block Tokens: The top level consists of paired or single block tokens (e.g., heading_open, paragraph_close, fence).
• Inline Containers: Special tokens that contain a .children property. This property holds a nested token stream for inline content (e.g., strong_open, em_open, text).

If you require an AST, you can manually convert the token stream after parsing without a renderer.

The parser produces a flat stream of Token objects. Nesting is indicated by a nesting attribute (1 for opening, -1 for closing).

To work with a hierarchical structure instead of a flat stream, use SyntaxTreeNode to convert the tokens into a tree.

from markdown_it import MarkdownIt
from markdown_it.tree import SyntaxTreeNode

md = MarkdownIt("commonmark")
tokens = md.parse("# Header\n\nText")
node = SyntaxTreeNode(tokens)

# Traverse the tree
print(node.children)
print(node[0])

from markdown_it import MarkdownIt
from markdown_it.tree import SyntaxTreeNode

md = MarkdownIt("commonmark")
tokens = md.parse("""
# Header

Here's some text and an image ![title](image.png)

1. a **list**

> a *quote*
""")

node = SyntaxTreeNode(tokens)
print(node.pretty(indent=2, show_text=True))

The linkify component allows URI autolinks to be identified without <> brackets.

Requirement: You must have linkify-it-py installed (e.g., pip install markdown-it-py[linkify]).

To use it, set "linkify": True in the configuration and enable the linkify rule.

md = MarkdownIt("commonmark", {"linkify": True})
md.enable(["linkify"])
md.render("github.com")

When using plugins that operate on tokenized content, be aware of how they handle attributes. A specific security risk involves plugins that generate arbitrary id or name attributes based on user input.

To prevent DOM clobbering, ensure that any plugin-generated id or name attributes use consistent prefixes if they depend on user-provided data. This is particularly important when using plugins that:

• Add extended class syntax
• Autogenerate header anchors

The parsing process follows a nested chain of rules across three main stages: core, block, and inline.

1. Core Stage: Performs initial normalization and high-level parsing.
2. Block Stage: Parses block-level elements (e.g., blockquotes, lists, headings, paragraphs).
3. Inline Stage: Applied to tokens of type inline (typically within block containers) to parse inline markup like bold, italic, or links.
4. Post-processing: Additional core rules (like footnotes or linkifiers) may apply to the token stream.

The result of this process is a list of tokens which is then passed to a renderer to generate the final output (usually HTML). Each stage uses an independent state object to ensure parsing operations are isolated and can be enabled or disabled independently.

To install markdown-it-py with specific features like linkify and plugins, use the following command:

pip install markdown-it-py[linkify,plugins]

By default, MarkdownIt is initialized to comply with the CommonMark spec, which allows arbitrary HTML tags. This is convenient but unsafe for parsing user-submitted content in web applications as it may lead to Cross-Site Scripting (XSS) vulnerabilities.

To mitigate this, use the js-default preset. This follows the security strategy used by the original JavaScript markdown-it project, which is safe-by-default by disabling HTML and requiring plugins for specific markup features.

Even with default settings, markdown-it-py automatically prohibits certain dangerous link schemes:

• javascript:
• vbscript:
• file:
• data: (except for specific image types like gif, png, jpeg, and webp)

from markdown_it import MarkdownIt
MarkdownIt("js-default").render("*user-submitted* text")

You can run the continuous integration benchmarking analysis locally within the repository using tox to compare markdown-it-py performance against other Markdown parsers. This requires having tox installed and configured in your environment.

tox -e py311-bench-packages -- --benchmark-columns mean,stddev

You can install markdown-it-py using pip. It is recommended to install the [plugins] extra to ensure access to common syntax extensions.

pip install markdown-it-py[plugins]