DOCSEARCHbyAlgolia
Submit Docs

markdown-cjk-friendly

repository·main·Indexed 40 days ago

https://github.com/tats-u/markdown-cjk-friendly

A collection of Markdown parsing and rendering packages designed to make emphasis (**) and GFM strikethrough (~~) compatible with Chinese, Japanese, and Korean (CJK) text. It provides plugins and extensions for markdown-it (markdown-it-cjk-friendly), Marked.js (marked-cjk-friendly), and the remark/micromark ecosystem (remark-cjk-friendly, micromark-extension-cjk-friendly) to ensure correct rendering when emphasis marks are adjacent to CJK punctuation.

Tokens
21K
Snippets
71
Records
113
Agent score
51%

What's inside markdown-cjk-friendly

  1. Overview of markdown-it-cjk-friendly

    main

    markdown-it-cjk-friendly is a plugin for markdown-it designed to make Markdown emphasis (**) compatible with Chinese, Japanese, and Korean (CJK) text.

    In standard CommonMark, emphasis marks (**) are often not recognized if they are placed immediately adjacent to CJK punctuation (like 。 or parentheses) without spaces. This plugin fixes that behavior, ensuring that CJK users can use standard Markdown emphasis syntax naturally.

  2. Overview of markdown-cjk-friendly packages

    main

    The markdown-cjk-friendly project provides a suite of Markdown packages compatible with Chinese, Japanese, and Korean (CJK) characters, based on a CommonMark revision candidate. It offers ports for various Markdown engines and ecosystems.

    Available JavaScript/TypeScript Packages

    • markdown-it ecosystem:
      • markdown-it-cjk-friendly (Use this instead of the deprecated markdown-it-cj-friendly)
    • remark/micromark ecosystem:
      • remark-cjk-friendly
        • remark-cjk-friendly-gfm-strikethrough
        • micromark-extension-cjk-friendly
          • micromark-extension-cjk-friendly-util
          • micromark-extension-cjk-friendly-gfm-strikethrough
        • mdast-util-to-markdown-cjk-friendly
          • mdast-util-to-markdown-cjk-friendly-gfm-strikethrough
    • marked ecosystem:
      • marked-cjk-friendly

  3. What is remark-cjk-friendly?

    main

    A remark plugin designed to make Markdown emphasis (**) compatible with Chinese, Japanese, and Korean (CJK) text.

    In standard CommonMark, emphasis marks often fail to render if they are immediately adjacent to CJK punctuation (like 。) or parentheses, because the characters inside the ** are considered part of the punctuation rather than the emphasis content. This plugin fixes that behavior to ensure CJK users can use emphasis naturally.

  4. Available ports and extensions

    main

    The CJK-friendly specification is being ported to other languages and Markdown implementations:

    As an extension for existing parsers

    • Comrak (via commonmarker and GitLab's gitlab-glfm-markdown gem)
    • goldmark (via goldmark-cjk-friendly)
    • Markdig

    Planned

    • Further specification tweaks
    • More ports for non-JavaScript implementations

  5. What is micromark-extension-cjk-friendly and when to use it?

    main

    This is a micromark extension designed to fix a limitation in the CommonMark specification regarding emphasis (**) when used with Chinese, Japanese, and Korean (CJK) text.

    The Problem

    In standard CommonMark, emphasis marks (**) are often not recognized if the character immediately inside the marks is a CJK punctuation mark (like 。) or a parenthesis, and the character immediately outside is not a space or punctuation.

    Example of failing CommonMark:

    **このアスタリスクは強調記号として認識されず、そのまま表示されます。**この文のせいで。

    When to use this

    You should adopt this extension instead of plain CommonMark or GFM if:

    1. Handling unsupervised content: You need to display user-generated or AI-generated CJK content as-is.
    2. Translation workflows: You work with translators (using tools like Crowdin or Transifex) who may not understand CommonMark edge cases and cannot provide real-time rendering previews, and you cannot allow raw <strong> tags.
    3. Target Audience: You are building Markdown-related software specifically targeting CJK users.

  6. The CJK Markdown problem and when to adopt this specification

    main

    Standard CommonMark and GFM fail to recognize emphasis marks (like **) when they are adjacent to CJK (Chinese, Japanese, Korean) punctuation marks (e.g., 。) or parentheses, provided the character outside the mark is not a space or punctuation.

    Example of the issue: **このアスタリスクは強調記号として認識されず、そのまま表示されます。**この文のせいで。 (The ** is not recognized as emphasis).

    You should adopt this specification if:

    1. You need to display user-generated or AI-generated CJK content as-is.
    2. You work with translators using services like Crowdin or Transifex where real-time rendering previews are unavailable and <strong tags are not permitted.
    3. You are building Markdown-related software primarily targeting CJK users.

  7. Implement CJK-friendly Emphasis and Strong Emphasis

    main

    The CJK-friendly amendment modifies how delimiter runs (like _ or __) are identified as left-flanking or right-flanking to allow them to work correctly adjacent to CJK characters and punctuation.

    Left-flanking delimiter run

    A delimiter run that is:

    1. Not followed by Unicode whitespace.
    2. AND either:
      • (2a) Not followed by a non-CJK punctuation character.
      • (2b) Followed by a non-CJK punctuation character AND preceded by:
        • (2bα) Unicode whitespace.
        • (2bβ) A non-CJK punctuation sequence.
        • (2bγ) A CJK sequence.
        • (2bδ) An Ideographic Variation Selector.

    Note: The beginning and end of a line count as Unicode whitespace.

    Right-flanking delimiter run

    A delimiter run that is:

    1. Not preceded by Unicode whitespace.
    2. AND either:
      • (2a) Not preceded by a non-CJK punctuation sequence.
      • (2b) Preceded by a non-CJK punctuation sequence AND followed by:
        • (2bα) Unicode whitespace.
        • (2bβ) A non-CJK punctuation character.
        • (2bγ) A CJK character.

    Opening Emphasis/Strong Emphasis

    • A single _ can open emphasis if it is part of a left-flanking delimiter run and is either (a) not part of a right-flanking delimiter run, or (b) part of a right-flanking delimiter run preceded by a Unicode punctuation sequence.
    • A double __ can open strong emphasis if it is part of a left-flanking delimiter run and is either (a) not part of a right-flanking delimiter run, or (b) part of a right-flanking delimiter run preceded by a Unicode punctuation sequence.

  8. Compatibility with other languages in remark-cjk-friendly-gfm-strikethrough

    main
    The modifications introduced by this package only affect Chinese, Japanese, and Korean (CJK) text. It does not affect other languages. Even if your application or documents contain translations or content in other languages, they will remain unaffected. The package is guaranteed to produce the same HTML output as cmark-gfm version 0.29.0.gfm.13 for all its test cases.

  9. Compatibility and language impact of the CJK extension

    main
    The CJK-friendly specification modification only affects Chinese, Japanese, and Korean text. It does not impact other languages. Even in documents containing translations or mixed-language content, non-CJK languages will remain unaffected. The extension is designed to ensure that micromark still outputs the same HTML for all CommonMark 0.31.2 test cases.

  10. Understand CJK character and sequence definitions

    main

    To implement CJK-friendly Markdown, you must use the following definitions for character classification:

    • CJK character: A character that meets at least one of these criteria:
      • UAX #11 East Asian Width category is W, F, or H, AND it is not a default emoji presentation character (per UTS #51).
      • UAX #24 Unicode Script Property is Hangul.
    • Ideographic Variation Selector: A character in the Variation Selectors Supplement Block (U+E0100–U+E01EF).
    • Non-emoji General-use Variation Selector: A character in the Variation Selectors Block (U+FE00–U+FE0F) excluding the Emoji Presentation Selector (U+FE0F).
    • CJK sequence: A CJK character or a sequence of 2 characters where the first is a CJK character and the second is a Non-emoji General-use Variation Selector.
    • CJK punctuation character: A Unicode punctuation character that is also a CJK character.
    • Non-CJK punctuation character: A Unicode punctuation character that is NOT a CJK punctuation character.
    • Unicode punctuation sequence: A Unicode punctuation character or a sequence of 2 characters where the first is a Unicode punctuation character and the second is a Non-emoji General-use Variation Selector.
    • CJK ambiguous punctuation sequence: A Standardized Variation Sequence (from StandardizedVariants.txt) containing the phrase "fullwidth form", where the first character is a Unicode punctuation character with UAX #11 East Asian Width category A.
    • CJK punctuation sequence: A CJK punctuation character, a CJK ambiguous punctuation sequence, or a sequence of 2 characters where the first is a CJK punctuation character and the second is a Non-emoji General-use Variation Selector.
    • Non-CJK punctuation sequence: A Non-CJK punctuation character or a sequence of 2 characters where the first is a Non-CJK punctuation character and the second is a Non-emoji General-use Variation Selector.

  11. When to use markdown-it-cjk-friendly

    main

    You should use this package if you are handling CJK content that cannot be fully supervised, specifically in these scenarios:

    1. Unsupervised Content: When displaying user-generated or AI-generated content as-is.
    2. Translation Workflows: When translators (who may not be engineers) use services like Crowdin or Transifex and cannot be expected to follow strict CommonMark spacing rules, and you cannot provide real-time rendering previews or allow <strong tags.
    3. Target Audience: If you are building Markdown-related software or services primarily targeting users in China, Japan, or Korea.

  12. What problem does this extension solve?

    main

    In standard CommonMark and GFM, strikethrough (~~) and emphasis (**) marks are often not recognized when they are adjacent to CJK (Chinese, Japanese, Korean) punctuation marks (like 。 or parentheses) if the character on the other side is not a space or punctuation.

    For example, in the following strings, the marks might fail to trigger correctly:

    • **このアスタリスクは強調記号として認識されず、そのまま表示されます。**この文のせいで。
    • **该星号不会被识别，而是直接显示。**这是因为它没有被识别为强调符号。
    • **이 별표는 강조 표시로 인식되지 않고 그대로 표시됩니다(이 괄호 때문에)**이 문장 때문에.

    This extension adjusts the recognition logic to be compatible with CJK text patterns.