marko Documentation

repository·master·Indexed 19 days ago

https://github.com/frostming/marko

A highly extensible pure Python markdown parser adhering to the CommonMark specification (v0.31.2). It provides a high-level API via the marko.Markdown class and a convenience function marko.convert() for HTML conversion. The library supports a variety of built-in extensions such as GFM, TOC, and footnotes, and allows users to create custom extensions by implementing the MarkoExtension class to define new block and inline elements.

Tokens
16K
Snippets
64
Records
85
Agent score
66%

What's inside marko

  1. Work with Block and Inline Elements

    master

    Marko distinguishes between two types of elements in the AST:

    • Block Elements: Managed via marko.block.BlockElement. These represent structural components like paragraphs, lists, or headers.
    • Inline Elements: Managed via marko.inline.InlineElement. These represent content within blocks, such as emphasis, strong text, or links.
  2. Traverse and manipulate the Abstract Syntax Tree (AST)

    master

    You can access the parsed document structure using the .parse() method. This returns a tree of objects that can be traversed to inspect elements like headings, paragraphs, and their properties (e.g., heading levels).

    from marko import Markdown
    
    md = Markdown()
    doc = md.parse("# Title\n\nParagraph with **bold** text.")
    
    # The document has a tree structure
    print(doc)  # Document object
    print(doc.children)  # List of block elements
    
    # Access specific elements
    heading = doc.children[0]  # Heading element
    print(heading.level)  # 1
    print(heading.children)  # ['Title']
    
    paragraph = doc.children[1]  # Paragraph element
    print(paragraph.children)  # List of inline elements
  3. Publish an extension as a standalone package

    master

    To allow users to refer to your extension via an import string (e.g., Markdown(extensions=['my_package'])) instead of importing the object directly, include a make_extension(arg) function in your package's entry file. This function should accept arguments and return a MarkoExtension object.

    def make_extension(arg):
        return GitHubWiki(arg)
  4. Create a new inline element

    master

    To add support for new syntax like GitHub wiki links ([[Page|Target]]), subclass marko.inline.InlineElement.

    Key attributes:

    • pattern: A regex string used to scan text for matches.
    • parse_children: If True, the parser will parse the content within the match groups (defaulting to group 1) into inline elements.
    • priority: Controls parsing precedence. The default is 5. Higher numbers are tried sooner. If priorities are equal, the first registered element wins.
    • override: Set to True if you want to replace an existing element's functionality.

    Use the __init__(self, match) method to map regex match groups to element attributes.

    from marko import inline
    
    class GitHubWiki(inline.InlineElement):
        pattern = r'\\\[\\[ *(.+?) *\| *(.+?) *\\\]\\]'
        parse_children = True
    
        def __init__(self, match):
            self.target = match.group(2)
  5. Use built-in Marko extensions

    master

    Marko provides several built-in extensions to add specialized Markdown parsing capabilities. You can enable these extensions by importing the specific extension module and adding it to your Marko instance.

    Available built-in extensions include:

    • marko.ext.gfm: GitHub Flavored Markdown support.
    • marko.ext.toc: Table of Contents generation.
    • marko.ext.footnote: Footnote support.
    • marko.ext.pangu: Pangu support (improves spacing for CJK characters).
    • marko.ext.codehilite: Syntax highlighting for code blocks.
  6. Add a new render function via Mixins

    master

    Marko uses mixins to control how elements are represented. To define how your custom element renders, create a mixin class with a method named render_<element_name_in_snake_case>.

    For example, an element named GitHubWiki requires a method named render_git_hub_wiki. The method receives the element instance and should return the rendered string (e.g., HTML).

    Common base renderers include:

    • marko.renderer.HTMLRenderer: The standard for HTML output.
    • marko.ast_renderer.ASTRenderer: Renders elements as JSON objects (useful for debugging).
    • marko.ast_renderer.XMLRenderer: Renders elements as XML format AST.
    • marko.ext.latex_renderer.LatexRenderer: Renders elements as LaTeX.
    class WikiRendererMixin(object):
        def render_git_hub_wiki(self, element):
            return '<a href="{}">{}</a>'.format(
                self.escape_url(element.target), self.render_children(element)
            )
  7. Pre-process and post-process Markdown content

    master

    You can intercept the parsing or rendering lifecycle by subclassing marko.Markdown:

    • Pre-processing: Override the parse(self, text) method to modify the raw text before it is parsed.
    • Post-processing: Override the render(self, parsed) method to modify the output (e.g., HTML) after the rendering process is complete.
    from marko import Markdown
    
    # Pre-processing Text
    class MyMarkdown(Markdown):
        def parse(self, text):
            # Pre-process text before parsing
            text = text.replace('TODO:', '**TODO:**')
            return super().parse(text)
    
    # Post-processing Results
    class MyPostProcessor(Markdown):
        def render(self, parsed):
            html = super().render(parsed)
            # Post-process HTML
            return html.replace('<p>', '<p class="content">')
    
    md = MyPostProcessor()
    html = md.convert("Hello world")
  8. Use extensions in Marko

    master

    Marko is highly extensible. You can add extensions in three ways:

    1. During initialization: Pass the extension function or its string identifier to the extensions list in the Markdown constructor.
    2. Using string identifiers: For built-in extensions, you can simply pass the name as a string.
    3. Registering later: Use the .use() method on an existing Markdown instance.

    Built-in extensions include: 'footnote', 'toc', 'pangu', and 'codehilite'.

    For GitHub Flavored Markdown (GFM), use marko.ext.gfm.gfm.

    from marko import Markdown
    from marko.ext.footnote import make_extension
    
    # 1. Add footnote extension via function
    markdown = Markdown(extensions=[make_extension()])
    
    # 2. Or use the string identifier for built-in extensions
    markdown = Markdown(extensions=['footnote'])
    
    # 3. Alternatively, register an extension later
    markdown.use(make_extension())
  9. Convert Markdown to HTML with a quick start

    master

    The simplest way to convert Markdown text to HTML is by using the marko.convert() convenience function. This is ideal for one-off conversions where you don't need custom configuration.

    import marko
    
    # Convert markdown to HTML
    html = marko.convert("# Hello World\nThis is **bold** text.")
    print(html)
    # Output: <h1 Hello World</h1>\n<p>This is <strong>bold</strong> text.</p>\n
  10. Load extensions in Marko

    master

    Extensions can add new elements or modify parsing/rendering. You can load them in three ways:

    1. During initialization: Pass a list of extension names to the extensions argument in Markdown().
    2. Using the .use() method: Call md.use('extension_name') on an existing instance.
    3. Passing extension objects: Pass the result of an extension factory (e.g., make_extension()) to the extensions list.
    from marko import Markdown
    
    # Method 1: Pass extensions during initialization
    md = Markdown(extensions=['footnote', 'toc'])
    
    # Method 2: Use the use() method
    md = Markdown()
    md.use('footnote')
    md.use('toc')
    
    # Method 3: Pass extension objects
    from marko.ext.footnote import make_extension
    md = Markdown(extensions=[make_extension()])