markdownify Documentation

repository·develop·Indexed 24 days ago

https://github.com/matthewwithanm/python-markdownify

A Python library and CLI tool designed to convert HTML markup into Markdown. It features a flexible API via the markdownify() function and MarkdownConverter class, allowing for custom tag converters, fine-grained control over styling, and configurable options for headings, bullets, and text wrapping. It supports converting both HTML strings and BeautifulSoup objects.

Tokens
2.8K
Snippets
3
Records
13
Agent score
78%

What's inside markdownify

  1. Create custom converters by subclassing MarkdownConverter

    develop

    To implement custom conversion logic, inherit from MarkdownConverter and override specific tag methods. The method naming convention is convert_{tag_name}(self, el, text, parent_tags).

    For example, to add extra newlines after images or to ignore specific tags like paragraphs, you can override convert_img or convert_p respectively.

    from markdownify import MarkdownConverter
    
    class ImageBlockConverter(MarkdownConverter):
        """
        Create a custom MarkdownConverter that adds two newlines after an image
        """
        def convert_img(self, el, text, parent_tags):
            return super().convert_img(el, text, parent_tags) + '\n\n'
    
    # Usage
    def md(html, **options):
        return ImageBlockConverter(**options).convert(html)
    
    class IgnoreParagraphsConverter(MarkdownConverter):
        """
        Create a custom MarkdownConverter that ignores paragraphs
        """
        def convert_p(self, el, text, parent_tags):
            return ''
    
    # Usage
    def md(html, **options):
        return IgnoreParagraphsConverter(**options).convert(html)
  2. Define custom tag converters

    develop

    The MarkdownConverter looks for methods following the pattern convert_<tag_name> to handle specific HTML tags. You can extend the converter by subclassing it and adding your own conversion methods.

    To handle a tag like <div>, you would implement convert_div(self, el, text, parent_tags). The parent_tags argument provides context about the nesting (e.g., if the tag is inside a <pre> block, it will contain '_noformat').

  3. Convert BeautifulSoup objects

    develop

    If you are already working with BeautifulSoup objects, you can use MarkdownConverter.convert_soup() to convert them directly.

    from markdownify import MarkdownConverter
    
    # Create shorthand method for conversion
    def md(soup, **options):
        return MarkdownConverter(**options).convert_soup(soup)
  4. Configure Markdown conversion options

    develop

    You can customize the conversion behavior by passing options to markdownify() or by instantiating MarkdownConverter directly.

    Common Configuration Options:

    • autolinks (bool): Whether to use shortcut syntax for autolinks. Defaults to True.
    • bs4_options (dict or str): Options passed to BeautifulSoup. Defaults to 'html.parser'.
    • bullets (iterable): Characters used for list bullets. Defaults to '*+-'.
    • code_language (str): Language identifier for code blocks. Defaults to ''.
    • code_language_callback (callable): A function that takes an element and returns a language string.
    • convert (list of str): A whitelist of tags to convert. If provided, only these tags are converted.
    • escape_asterisks (bool): Whether to escape * characters. Defaults to True.
    • escape_underscores (bool): Whether to escape _ characters. Defaults to True.
    • escape_misc (bool): Whether to escape miscellaneous special Markdown characters.
    • heading_style (str): Style for headings. Options: 'atx', 'atx_closed', or 'underlined' (also known as 'setext').
    • newline_style (str): Style for line breaks. Options: 'spaces' or 'backslash'.
    • strong_em_symbol (str): The symbol used for strong/emphasis. Defaults to '*'.
    • strip (list of str): A blacklist of tags to strip from the output.
    • strip_document (str): How to strip leading/trailing newlines from the document. Options: 'lstrip', 'rstrip', 'strip', or None.
    • strip_pre (str): How to strip leading/trailing newlines from <pre> blocks. Options: 'lstrip', 'rstrip', 'strip', 'strip_one', or None.
    • wrap (bool): Whether to wrap text. Defaults to False.
    • wrap_width (int): The width at which to wrap text if wrap is True.

    Note: You cannot specify both strip (blacklist) and convert (whitelist) at the same time.

  5. Reference: markdownify() configuration options

    develop

    The following options can be passed as keyword arguments to the markdownify() function to control the conversion process:

    • strip: A list of tags to strip. Cannot be used with convert.
    • convert: A list of tags to convert. Cannot be used with strip.
    • autolinks: Boolean. If True, uses "automatic link" style when an <a> tag's content matches its href. Defaults to True.
    • default_title: Boolean. If True, sets the link title to its href if no title is provided. Defaults to False.
    • heading_style: Defines heading conversion. Accepted values: ATX, ATX_CLOSED, SETEXT, and UNDERLINED (alias for SETEXT). Defaults to UNDERLINED.
    • bullets: An iterable (string, list, or tuple) of bullet styles. If one item, it's used for all levels. Otherwise, it alternates by nesting level. Defaults to '*+-'.
    • strong_em_symbol: Choose between ASTERISK (default) or UNDERSCORE for strong/emphasized text.
    • sub_symbol, sup_symbol: Characters surrounding <sub> and <sup> text. Defaults to empty string.
    • newline_style: Defines <br> conversion. SPACES (default) uses two spaces and a newline; BACKSLASH uses \n.
    • code_language: String. Assumed language for all <pre> sections. Defaults to ''.
    • code_language_callback: A function that takes a BeautifulSoup object and returns a string (the language) or None. Used to extract language from tags (e.g., classes).
    • escape_asterisks: Boolean. If False, does not escape * to \*. Defaults to True.
    • escape_underscores: Boolean. If False, does not escape _ to \_. Defaults to True.
    • escape_misc: Boolean. If True, escapes miscellaneous punctuation. Defaults to False.
    • keep_inline_images_in: A list of parent tags (e.g., ['td']) that allow images to be converted to markdown images instead of alt-text.
    • table_infer_header: Boolean. If True, uses the first body row as the header if no <thead> is present. Defaults to False.
    • wrap, wrap_width: If wrap is True, wraps text at wrap_width (default 80). wrap_width=None reflows to unlimited length.
    • strip_document: Controls leading/trailing newlines. Values: LSTRIP, RSTRIP, STRIP (default), or None.
    • strip_pre: Controls leading/trailing blank lines in <pre> tags. Values: STRIP (default), STRIP_ONE, or None.
    • bs4_options: Configuration for the underlying BeautifulSoup object. Can be a string/list for features or a dictionary for kwargs.
  6. Use the markdownify CLI

    develop

    You can use markdownify from the command line to convert HTML files or piped input. The CLI accepts the same options as the Python function.

    • Convert a file: markdownify example.html > example.md
    • Pipe input: cat example.html | markdownify > example.md
    • View help: markdownify -h
  7. Use MarkdownConverter for advanced control

    develop

    For more granular control, you can instantiate the MarkdownConverter class directly. This allows you to manage options and potentially extend the converter.

    from markdownify import MarkdownConverter
    
    converter = MarkdownConverter(heading_style='atx', bullets='-')
    html = '<ul><li>Item 1</li><li>Item 2</li></ul>'
    markdown = converter.convert(html)
  8. Convert HTML to Markdown with markdownify()

    develop

    The primary way to convert HTML strings to Markdown is by using the markdownify() function. It accepts an HTML string and an arbitrary number of keyword arguments to configure the conversion process.

    from markdownify import markdownify
    
    html = '<h1>Hello World</h1><p>This is <b>bold</b>.</p>'
    markdown = markdownify(html, heading_style='atx')
    print(markdown)
  9. Reference: markdownify CLI arguments and flags

    develop

    The following table lists the available command-line arguments for the markdownify tool:

    FlagArgumentDescription
    html(positional)The html file to convert. Defaults to STDIN if not provided.
    -s, --stripnargs='*'A list of tags to strip. Cannot be used with --convert.
    -c, --convertnargs='*'A list of tags to convert. Cannot be used with --strip.
    -a, --autolinksbooleanUse 'automatic link' style if <a> tag contents match href.
    --default-titlebooleanSet link title to its href if no title is given.
    --heading-styleATX, ATX_CLOSED, UNDERLINEDDefines how headings should be converted.
    -b, --bulletsstringBullet styles to use (e.g. *+-). Alternates based on nesting.
    --strong-em-symbolASTERISK, UNDERSCOREUse * or _ to convert strong and italics text.
    --sub-symbolstringChars that surround <sub>.
    --sup-symbolstringChars that surround <sup>.
    --newline-styleSPACES, BACKSLASHDefines <br> conversions.
    --code-languagestringAssumed language for all <pre> sections.
    --no-escape-asterisksbooleanDo not escape * to \*.
    --no-escape-underscoresbooleanDo not escape _ to \_.
    -i, --keep-inline-images-innargs='*'List of parent tags allowed to contain inline images.
    --table-infer-headerbooleanUse first body row as header if no <thead>/<th> exists.
    -w, --wrapbooleanWrap all text paragraphs.
    --wrap-widthintegerWidth for text wrapping.
    --bs4-optionsstringBeautifulSoup parser (e.g., html.parser, lxml).