DOCSEARCHbyAlgolia
Submit Docs

nodejieba

repository·master·Indexed 25 days ago

https://github.com/yanyiwu/nodejieba

A high-performance Node.js implementation of the Jieba Chinese segmentation algorithm powered by a C++ core. It provides functionality for Chinese word segmentation (including default, HMM, full, search engine, and small granularity modes), part-of-speech (POS) tagging, and keyword extraction using standard and TextRank algorithms. It supports dynamic dictionary updates via insertWord() and custom dictionary configuration through load().

Tokens
1.2K
Snippets
4
Records
17
Agent score
35%

What's inside nodejieba

  2. Configure dictionary loading

    master

    By default, dictionaries are loaded automatically on the first function call. You can manually trigger loading or provide custom paths for different dictionary types using load(options).

    Options for load():

    • dict: Main dictionary (includes weights and POS tags). Use nodejieba.DEFAULT_DICT for default.
    • hmmDict: Hidden Markov Model dictionary. Use nodejieba.DEFAULT_HMM_DICT for default.
    • userDict: User-defined dictionary path. Use nodejieba.DEFAULT_USER_DICT for default.
    • idfDict: IDF information for keyword extraction. Use nodejieba.DEFAULT_IDF_DICT for default.
    • stopWordDict: Stop words list for keyword extraction. Use nodejieba.DEFAULT_STOP_WORD_DICT for default.
    // Load only a custom user dictionary
nodejieba.load({
  userDict: './path/to/your/userdict.utf8',
});

// Or load everything explicitly
nodejieba.load({
  dict: nodejieba.DEFAULT_DICT,
  hmmDict: nodejieba.DEFAULT_HMM_DICT,
  userDict: './path/to/your/userdict.utf8',
  idfDict: nodejieba.DEFAULT_IDF_DICT,
  stopWordDict: nodejieba.DEFAULT_STOP_WORD_DICT,
});

  3. Add custom words to the dictionary

    master

    Use insertWord(word) to dynamically add a new word to the dictionary during runtime so it is treated as a single unit in future segmentation calls.

    var nodejieba = require("nodejieba");
nodejieba.insertWord("男默女泪");
console.log(nodejieba.cut("男默女泪"));
// ["男默女泪"]

  4. Tag words with part-of-speech (POS) labeling

    master

    Use tag() to segment text and return each word along with its corresponding part-of-speech tag.

    var nodejieba = require("nodejieba");
var result = nodejieba.tag("红掌拨清波");
// Returns: [ { word: '红掌', tag: 'n' }, { word: '拨', tag: 'v' }, { word: '清波', tag: 'n' } ]

  5. Extract keywords from text

    master

    You can extract the most important keywords from a sentence using two different algorithms:

    1. extract(sentence, topN): Standard keyword extraction.
    2. textRankExtract(sentence, topN): Keyword extraction using the TextRank algorithm.

  6. Perform Chinese word segmentation

    master

    NodeJieba provides several segmentation modes:

    1. Default segmentation: Standard segmentation.
    2. HMM segmentation: Uses the Hidden Markov Model.
    3. Full mode: Finds all possible words in the text.
    4. Search engine mode: A mode optimized for search engines.
    5. Small granularity segmentation: Segments words into smaller units (specify granularity with a number).

  7. Troubleshoot missing native bindings

    master

    If you encounter a BINDING_NOT_FOUND error, it means the native C++ binding (nodejieba.node) was not found. This typically happens if install scripts were skipped or the build failed.

    Resolution:

    • Reinstall the package without the --ignore-scripts flag.
    • Or run npm rebuild nodejieba to trigger a rebuild of the native components.

  8. Load custom dictionaries in nodejieba

    master

    Use the load(dictJson) method to configure the dictionary paths used by the segmentation engine. If no argument is provided, it loads the default dictionaries. You can pass an object to specify custom paths for specific dictionary types.

    Supported keys in the dictJson object:

    • dict: Path to the main dictionary file.
    • hmmDict: Path to the HMM model file.
    • userDict: Path to the user dictionary file.
    • idfDict: Path to the IDF dictionary file.
    • stopWordDict: Path to the stop words dictionary file.