commonmarker

repository·main·Indexed 19 days ago

https://github.com/gjtorikian/commonmarker

A Ruby wrapper for the Rust-based comrak crate, providing a spec-complete CommonMark parser with GitHub Flavored Markdown (GFM) extensions. It enables developers to convert Markdown to HTML or programmatically manipulate the document's Abstract Syntax Tree (AST) using Commonmarker::Node. The library includes a syntax highlighter plugin with multiple themes and supports custom .tmtheme files.

Tokens
10.9K
Snippets
33
Records
35
Agent score
64%

What's inside commonmarker

  1. Modify document nodes and attributes

    main

    Once you have parsed a document, you can manipulate the AST using the following methods and attribute assignments:

    Structural Operations:

    • insert_before(node)
    • insert_after(node)
    • prepend_child(node)
    • append_child(node)
    • delete

    Attribute Modifications: You can directly modify the following attributes on supported nodes:

    • url
    • title
    • header_level
    • list_type
    • list_start
    • list_tight
    • fence_info
    • alert_type

    Source Position: Call source_position on a node to retrieve its location in the original string as a hash containing :start_line, :start_column, :end_line, and :end_column.

    doc = Commonmarker.parse("*Hello* world")
    puts doc.first_child.first_child.source_position
    # => {:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>7}
  2. Install the commonmarker gem

    main

    You can install commonmarker using Bundler or by installing the gem directly via the command line.

    To use it in a Bundler-managed application, add this to your Gemfile:

    gem 'commonmarker'

    Then run:

    bundle

    Alternatively, install it globally using:

    gem install commonmarker
    gem 'commonmarker'
  3. Develop Commonmarker locally

    main

    To set up a local development environment, clone the repository and run the bootstrap script followed by the compilation rake task. If errors occur, ensure you have followed the comrak dependency instructions.

    script/bootstrap
    bundle exec rake compile
  4. Use the Syntax Highlighter plugin

    main

    The syntax highlighter plugin is enabled by default using the "base16-ocean.dark" theme. It applies syntax highlighting to fenced code blocks that specify a language.

    You can customize the theme by passing a theme name from the pre-existing set in the plugins hash.

    Available themes:

    • "base16-ocean.dark" (Default)
    • "base16-eighties.dark"
    • "base16-mocha.dark"
    • "base16-ocean.light"
    • "InspiredGitHub"
    • "Solarized (dark)"
    • "Solarized (light)"

    To output CSS classes instead of inline style attributes, set the theme key to an empty string "".

    code = <<~CODE
      ```ruby
      def hello
        puts "hello"
      end

    CODE

    pass in a theme name from a pre-existing set

    puts Commonmarker.to_html(code, plugins: { syntax_highlighter: { theme: "InspiredGitHub" } })

    To output CSS classes instead of style attributes, set the theme key to ""

    Commonmarker.to_html(code, plugins: { syntax_highlighter: { theme: "" } })

    To disable this plugin, set the value to nil:

    Commonmarker.to_html(code, plugins: { syntax_highlighter: nil })

  5. Walk the AST to inspect or transform nodes

    main

    Use the walk method to traverse the entire document tree. This is useful for finding specific node types (like :link) and either extracting data or modifying the tree structure.

    Example: Extracting link URLs

    doc.walk do |node|
      if node.type == :link
        printf("URL = %s\n", node.url)
      end
    end

    Example: Removing links and replacing them with plain text

    doc.walk do |node|
      if node.type == :link
        node.insert_before(node.first_child)
        node.delete
      end
    end
    require 'commonmarker'
    
    doc = Commonmarker.parse("# The site\n\n [GitHub](https://www.github.com)")
    
    doc.walk do |node|
      if node.type == :link
        node.insert_before(node.first_child)
        node.delete
      end
    end
    
    doc.to_commonmark
    # => # The site\n\nGitHub\n
  6. Parse Markdown into a Document node

    main

    Use Commonmarker.parse to convert a Markdown string into a :document node. This node allows you to inspect the Abstract Syntax Tree (AST), iterate over nodes, and modify the document structure before rendering.

    Common operations on a document node include:

    • to_html: Renders the node (and its children) to an HTML string.
    • to_commonmark: Renders the node back into raw Markdown text.
    • walk: Recursively iterates over the node and all its descendants.
    • each: Iterates only over the direct children of the node.
    require 'commonmarker'
    
    doc = Commonmarker.parse("*Hello* world", options: {
        parse: { smart: true }
    })
    puts doc.to_html # => <p><em>Hello</em> world</p>\n
    doc.walk do |node|
      puts node.type # => [:document, :paragraph, :emph, :text, :text]
    end
  7. Convert Markdown strings to HTML

    main

    Use Commonmarker.to_html to quickly convert a Markdown string into an HTML string. This method accepts an optional options hash to configure parsing behavior.

    Note: This gem expects UTF-8 encoded strings. Ensure your input is correctly encoded before passing it to Commonmarker.

    require 'commonmarker'
    Commonmarker.to_html('"Hi *there*"', options: {
        parse: { smart: true }
    })
    # => <p>“Hi <em>there</em>”</p>\n
  8. Convert Markdown to HTML

    main

    Commonmarker currently supports generating output in HTML format. Use the Commonmarker.to_html method to convert a Markdown string into an HTML string.

    puts Commonmarker.to_html('*Hello* world!')
    # <p><em>Hello</em> world!</p>
  9. Configure a custom syntax highlighter theme

    main

    When converting code to HTML, you can specify a custom syntax highlighting theme by passing a path to a directory containing .tmtheme files within the plugins options hash.

    Commonmarker.to_html(code, plugins: { syntax_highlighter: { theme: "Monokai", path: "./themes" } })
  10. Configure syntax highlighting via the `syntax_highlighter` plugin

    main

    Syntax highlighting is managed through a plugin configuration hash. You can specify a theme and an optional path to a directory containing custom themes.

    • If theme is provided but no path is specified, the library looks up the theme in the default syntect theme set.
    • If a path is provided, it must be a directory. The library will attempt to load all themes from that directory using ts.add_from_folder(&path). The specified theme must then exist within that directory.
    • If no theme is provided, the adapter defaults to using CSS classes for highlighting.

    Configuration Keys:

    • syntax_highlighter: The main plugin configuration hash.
    • theme: (Inside syntax_highlighter) The name of the theme to use (e.g., a string).
    • path: (Inside syntax_highlighter) A string representing the directory path containing custom .tmTheme or similar files.
    # Example configuration structure
    plugins = {
      syntax_highlighter: {
        theme: 'base16-ocean.dark',
        path: '/path/to/custom/themes'
      }
    }
  11. Configure parsing options

    main

    When parsing Markdown, you can pass a hash of options to control the parser's behavior. Use the following keys to configure parsing:

    • smart: Boolean. Enables smart typography (e.g., curly quotes).
    • default_info_string: String. Sets the default info string for fenced code blocks.
    • relaxed_tasklist_matching: Boolean. Enables relaxed matching for task lists.
    • relaxed_autolinks: Boolean. Enables relaxed autolinks.
    • leave_footnote_definitions: Boolean. If true, leaves footnote definitions in the document.
    • sourcepos_chars: Boolean. Enables source position characters.
    # Example usage (conceptual Ruby/Hash interface)
    options = {
      smart: true,
      default_info_string: "custom_info"
    }
  12. Configure rendering options

    main

    When rendering Markdown to HTML, you can pass a hash of options to customize the output. Use the following keys:

    • hardbreaks: Boolean. Enables hard line breaks.
    • github_pre_lang: Boolean. Enables GitHub-style language classes for preformatted text.
    • full_info_string: Boolean. Enables full info strings for code blocks.
    • width: Integer. Sets the line width for wrapping.
    • unsafe: Boolean. Allows unsafe HTML.
    • escape: Boolean. Escapes HTML characters.
    • sourcepos: Boolean. Enables source position information.
    • escaped_char_spans: Boolean. Enables escaped character spans.
    • ignore_empty_links: Boolean. Ignores empty links.
    • gfm_quirks: Boolean. Enables GitHub Flavored Markdown quirks.
    • prefer_fenced: Boolean. Prefers fenced code blocks.
    • tasklist_classes: Boolean. Enables task list classes.
    • compact_html: Boolean. Produces compact HTML output.
    • alert_style: String. Sets the alert style. Use "semantic" for semantic alerts, otherwise defaults to "specific".
    • ignore_setext: Boolean. Ignores Setext-style headers.
    # Example usage (conceptual Ruby/Hash interface)
    render_options = {
      hardbreaks: true,
      unsafe: true,
      alert_style: "semantic"
    }