DOCSEARCHbyAlgolia
Submit Docs

Marked

repository·master·Indexed 12 days ago

https://github.com/markedjs/marked

A high-speed, low-level markdown compiler designed to parse markdown into HTML. It is lightweight, supports various markdown flavors, and works across Node.js, browsers, and command-line interfaces. Version 18.0.9.

Tokens
16.7K
Snippets
51
Records
67
Agent score
97%

What's inside Marked

  1. How the Marked Pipeline works

    master

    Marked translates Markdown to HTML through a specific pipeline:

    1. Input: User provides a Markdown string.
    2. Lexer/Tokenizer: The lexer feeds segments of text into tokenizer functions. If a pattern matches, a token object is generated and organized into a nested tree structure.
    3. walkTokens: The walkTokens function traverses the token tree to perform final adjustments to token contents.
    4. Parser/Renderer: The parser traverses the token tree and feeds each token into the appropriate renderer. The renderer converts the token into an HTML string.
    5. Output: The concatenated outputs of the renderers form the final HTML result.

  2. Create Custom Extensions

    master

    Custom extensions allow you to define entirely new syntax by providing both a tokenizer and a renderer. These are added via the extensions array in marked.use() and execute before default parsing logic.

    Extension Properties

    • name: A string identifier. If it matches an existing method name, it overrides that behavior.
    • level: Either 'block' or 'inline'.
      • Block: Handled before block-level tokenizers.
      • Inline: Handled inside block-level tokens, before inline-level tokenizers.
    • start(src): A function returning the index of the next potential start of the token. Helps Marked avoid skipping text.
    • tokenizer(src, tokens): A function that returns a token object. The token should include type (matching name), raw (consumed text), and optionally tokens (child tokens).
      • Accessing Lexer: Inside tokenizer, this.lexer.blockTokens, this.lexer.inline, or this.lexer.inlineTokens can be used to parse nested content.
    • renderer(token): A function that returns the HTML string.
      • Accessing Parser: Inside renderer, this.parser.parse or this.parser.parseInline can be used to render child tokens.
    • childTokens (optional): An array of strings matching token parameter names that should be traversed by walkTokens.
    const descriptionList = {
  name: 'descriptionList',
  level: 'block',
  start(src) { return src.match(/:[^:\n]/)?.index; },
  tokenizer(src, tokens) {
    const rule = /^(?::[^:\n]+:[^:\n]*(?:\n|$))+/;
    const match = rule.exec(src);
    if (match) {
      const token = {
        type: 'descriptionList',
        raw: match[0],
        text: match[0].trim(),
        tokens: []
      };
      this.lexer.inline(token.text, token.tokens);
      return token;
    }
  },
  renderer(token) {
    return `<dl>${this.parser.parseInline(token.tokens)}\n</dl>`;
  }
};

marked.use({ extensions: [descriptionList] });

  4. Create a local Marked instance to avoid global scope mutation

    master

    By default, Marked uses a global instance. Changing options or adding extensions in one part of your application will affect all other parts using the default marked import. To ensure options and extensions are locally scoped, instantiate a new Marked class.

    Note: Avoid using marked.use(...) inside loops or functions. It should only be called immediately after importing marked or creating a new Marked instance.

    import { Marked } from 'marked';
const marked = new Marked([options, extension, ...]);

  5. Extend Marked with custom extensions

    master

    Marked can be extended by passing extensions to marked.use(). Extensions allow you to add support for new syntax, custom rendering, or specialized formatting. Common extensions include:

  6. Prevent ReDoS attacks using Workers

    master

    To protect your application from Regular Expression Denial of Service (ReDoS) attacks, run marked.parse inside a Worker thread. This allows you to set a timeout and terminate the worker if parsing takes too long, preventing a malicious markdown string from freezing your main thread.

    ### Node Worker Thread Example

```js
// markedWorker.js
import { marked } from 'marked';
import { parentPort } from 'worker_threads';

parentPort.on('message', (markdownString) => {
  parentPort.postMessage(marked.parse(markdownString));
});
    // index.js
import { Worker } from 'worker_threads';
const markedWorker = new Worker('./markedWorker.js');

const markedTimeout = setTimeout(() => {
  markedWorker.terminate();
  throw new Error('Marked took too long!');
}, timeoutLimit);

markedWorker.on('message', (html) => {
  clearTimeout(markedTimeout);
  console.log(html);
  markedWorker.terminate();
});

markedWorker.postMessage(markdownString);

  7. Use the Marked CLI

    master

    The Marked CLI allows you to parse markdown from stdin, strings, or files.

    • From stdin: Use -o <file> to output the result to a file.
    • From a string: Use -s "<string>" to parse a specific string.
    • From a file: Use -i <input-file> and -o <output-file> to convert a file.
    • Help: Use --help to see all available options.
    # Example: stdin to file
$ marked -o hello.html

# Example: string input
$ marked -s "*hello world*"

# Example: file input to file output
$ marked -i readme.md -o readme.html

# Example: print help
$ marked --help

  8. Request to be listed as a user of Marked

    master

    If you are an individual or organization using Marked, you can request to be listed in the AUTHORS.md file.

    To be listed, submit a pull request with the following optional fields:

    • Individual or Organization: The name you would like associated with the record.
    • Website: A URL to a standalone website for the project.
    • Project: A URL for the repository of the project using marked.
    • Submitted by: The name and optional honorifics for the person adding the listing.

  9. Understand the Marked Demo views

    master

    The Marked demo provides several ways to inspect the conversion process from Markdown to HTML. You can switch between these views using the dropdown menu:

    • Preview: Shows the final rendered HTML as it would appear in a web browser.
    • HTML Source: Shows the raw HTML string generated by the parser.
    • Lexer Data: Shows the internal tokenized representation (the data structure marked uses during parsing).
    • Quick Reference: Provides a guide on Markdown formatting syntax.

  10. Extend Marked using marked.use()

    master

    The recommended way to add functionality to Marked is via marked.use(). You can pass a MarkedExtension object which can contain options, hooks, renderer overrides, tokenizer overrides, and custom extensions.

    Important: Extensions should be added in the global scope of a module. If added inside a function that runs repeatedly (e.g., inside a Svelte component), they will be added multiple times, leading to recursion errors. To avoid this, use a Marked instance instead of the global marked object.

    Multiple extension objects can be passed to a single marked.use() call, which is equivalent to calling marked.use() multiple times sequentially. Options are overwritten by subsequent calls, except for renderer, tokenizer, hooks, walkTokens, and extensions, which are merged.

    import { marked } from 'marked';

// Using a Marked Extension
marked.use({
  gfm: true,
  breaks: false,
  renderer: { /* renderer overrides */ },
  tokenizer: { /* tokenizer overrides */ },
  extensions: [
    { name: 'myCustomSyntax', level: 'block', tokenizer: fn, renderer: fn }
  ]
});

  11. Use extensions in the CLI

    master

    To use extensions with the Marked CLI, create a custom script that imports marked, applies your extensions via marked.use(), and then imports the marked/bin/marked binary.

    // file: myMarked
#!/usr/bin/node

import { marked } from 'marked';
import customHeadingId from 'marked-custom-heading-id';

marked.use(customHeadingId());

import 'marked/bin/marked';
    $ ./myMarked -s "# heading {#custom-id}"
<h1 id="custom-id">heading</h1>