markdown-rs

repository·main·Indexed 23 days ago

https://github.com/wooorm/markdown-rs

A CommonMark-compliant markdown parser written in Rust. It features a high-performance state-machine-based parser capable of emitting HTML or a Markdown Abstract Syntax Tree (mdast) for complex manipulations. The library includes utilities for converting mdast trees back to markdown strings, managing construct stacks, and configuring HTML compilation options including security settings for dangerous HTML and protocols.

Tokens
17.2K
Snippets
39
Records
88
Agent score
81%

What's inside markdown-rs

  1. Configure markdown extensions

    main

    Extensions are not enabled by default. You can enable them by passing specific Options to to_html_with_options. Supported extensions include:

    • GFM: autolink literal, footnote, strikethrough, table, tagfilter, task list item.
    • MDX: ESM, expressions, JSX.
    • frontmatter
    • math
  2. How markdown-rs works

    main

    The library is implemented as a state machine (#![no_std] + alloc) that emits concrete tokens. The high-level process follows this flow:

    1. Parse: The input string is turned into a stream of events.
    2. Resolve: Events are processed and resolved.
    3. Compile: Events are compiled into either an HTML string or an mdast syntax tree.
  3. Install markdown-rs via cargo

    main

    To use markdown-rs in your Rust project (requires Rust edition 2018+ and version 1.56+), add it using cargo add.

    cargo add markdown
  4. Understand the Document content type

    main

    In markdown-rs, a Document represents the top-level containers that structure the markdown content. These containers hold other sections and define the document's flow.

    Common flow constructs include:

    • Block quote
    • List item
    • GFM: Footnote definition

    The document lifecycle involves handling optional components like the Byte Order Mark (BOM) and frontmatter before processing these structural containers.

  5. Configure enabled markdown constructs with `Constructs`

    main

    The Constructs struct allows you to control which markdown syntax elements are enabled during parsing. Note that some constructs, like blank lines and paragraphs, cannot be disabled.

    You can use predefined configurations:

    • Constructs::default(): Provides CommonMark constructs.
    • Constructs::gfm(): Provides GitHub Flavored Markdown (GFM) constructs.
    • Constructs::mdx(): Provides MDX constructs (turns on MDX, turns off conflicting autolink, code_indented, and html constructs).

    You can also create custom configurations by mixing and matching fields using struct update syntax.

    use markdown::Constructs;
    
    // Use the default trait to get `CommonMark` constructs:
    let commonmark = Constructs::default();
    
    // To turn on all of GFM, use the `gfm` method:
    let gfm = Constructs::gfm();
    
    // Or, mix and match:
    let custom = Constructs {
      math_flow: true,
      math_text: true,
      ..Constructs::gfm()
    };
  6. Common AST node properties

    main

    Most nodes in the markdown-rs AST share common fields for managing content and metadata:

    • children: A Vec<Node> representing the nested content model (used by parent nodes like Root, Paragraph, Table, etc.).
    • position: An Option<Position> providing the location of the node in the source text.
    • value: A String representing the literal text content (used by leaf nodes like Text, Code, Yaml, etc.).
    • stops: A Vec<Stop> used in MDX nodes to track the source slices of the value string (serialized as _markdownRsStops).
  7. Configure HTML compilation with `CompileOptions`

    main

    The CompileOptions struct defines how markdown is compiled into HTML. It includes settings for security (handling dangerous HTML and protocols) and customization (GFM footnote styling and MDX-related behaviors).

    Security Options

    • allow_any_img_src: If true, allows all values as src on images. This is considered safe as browsers do not execute code in images.
    • allow_dangerous_html: If true, allows actual HTML elements to be rendered. If false (default), HTML is treated as text.
    • allow_dangerous_protocol: If true, allows dangerous protocols (like javascript:) in links and images. If false (default), these are dropped.

    GFM Customization

    • gfm_footnote_back_label: The aria-label for footnote backreferences (default: "Back to content").
    • gfm_footnote_clobber_prefix: A prefix for footnote IDs to prevent DOM clobbering (default: "user-content-").
    • gfm_footnote_label_attributes: HTML attributes for the footnote label element (default: "class=\"sr-only\"").
    • gfm_footnote_label_tag_name: The HTML tag for the footnote label (default: "h2").
    • gfm_footnote_label: The textual label for the footnotes section (default: "Footnotes").
    • gfm_task_list_item_checkable: If true, GFM task list <input> items are not disabled, allowing users to toggle them in the browser.
    • gfm_tagfilter: A naïve XSS protection filter. Only works if allow_dangerous_html is also true. (Note: A proper HTML sanitizer is recommended instead).

    Other Options

    • default_line_ending: The line ending used when compiling to HTML if no line ending is detected in the document.
    use markdown::CompileOptions;
    
    // Use the default trait to get safe defaults:
    let safe = CompileOptions::default();
    
    // Live dangerously / trust the author:
    let danger = CompileOptions {
      allow_dangerous_html: true,
      allow_dangerous_protocol: true,
      ..CompileOptions::default()
    };
    
    // In French:
    let enFrançais = CompileOptions {
      gfm_footnote_back_label: Some("Arrière".into()),
      gfm_footnote_label: Some("Notes de bas de page".into()),
      ..CompileOptions::default()
    };
  8. Security considerations for markdown-rs

    main

    XSS Protection

    markdown-rs is safe by default. It encodes or drops embedded HTML and dangerous protocols (like javascript:) to prevent Cross-Site Scripting (XSS).

    Warning: Enabling allow_dangerous_html or allow_dangerous_protocol via options makes your application vulnerable to XSS if you are processing untrusted user input.

    Image Sources

    By default, the parser only allows http:, https:, and relative image sources. You can safely enable allow_any_img_src as modern browsers protect against scripts in images.

    Denial of Service (DoS)

    To protect against resource exhaustion attacks (e.g., extremely large files or deeply nested syntax like unclosed emphasis/links):

    1. Cap the accepted input size (e.g., 500kb).
    2. Process content in a separate thread so it can be interrupted if needed.
  9. Convert markdown to HTML

    main

    You can convert markdown strings directly to HTML using markdown::to_html. For more control, use markdown::to_html_with_options which allows you to enable extensions like GFM (GitHub Flavored Markdown). Note that to_html_with_options returns a Result<String, Message>, so you must handle potential errors (especially when using MDX).

    // Basic usage
    println!("{}", markdown::to_html("## Hi, *Saturn*! 🪐"));
    
    // Usage with GFM extensions
    fn main() -> Result<(), markdown::message::Message> {
        println!(
            "{}",
            markdown::to_html_with_options(
                "* [x] contact ~Mercury~Venus at hi@venus.com!",
                &markdown::Options::gfm()
            )?
        );
        Ok(())
    }
  10. Parse markdown into an mdast syntax tree

    main

    To perform complex manipulations on the markdown structure, you can parse it into an mdast (Markdown Abstract Syntax Tree) using markdown::to_mdast. This requires providing markdown::ParseOptions.

    fn main() -> Result<(), markdown::message::Message> {
        println!(
            "{:?}",
            markdown::to_mdast("# Hi *Earth*!", &markdown::ParseOptions::default())?
        );
        Ok(())
    }
  11. Configure `markdown-rs` via Cargo features

    main

    The crate provides several optional features to extend its functionality:

    • log: Enables logging. You can view logs by setting the environment variable RUST_LOG=debug.
    • serde: Enables serde support, allowing you to serialize and deserialize ASTs and configuration objects.

    Note: The default feature is enabled by default but provides no additional functionality (nothing is enabled by default).

  12. Customize constructs in ParseOptions

    main

    You can fine-tune the parser by modifying the constructs field within ParseOptions. This allows you to selectively enable or disable specific markdown features (e.g., turning off indented code blocks while keeping other features).

    use markdown::{to_html_with_options, Constructs, Options, ParseOptions};
    # fn main() -> Result<(), markdown::message::Message> {
    
    // Pass `constructs` to choose what to enable and disable:
    assert_eq!(to_html_with_options(
        "    indented code?",
        &Options {
            parse: ParseOptions {
              constructs: Constructs {
                code_indented: false,
                ..Constructs::default()
              },
              ..ParseOptions::default()
            },
            ..Options::default()
        }
    )?,
        "<p>indented code?</p>"
    );
    # Ok(())
    # }