DOCSEARCHbyAlgolia
Submit Docs

winkNLP

repository·master·Indexed 23 days ago

https://github.com/winkjs/wink-nlp

A high-performance, developer-friendly JavaScript library for Natural Language Processing (NLP) compatible with Node.js, web browsers, and Deno. It provides capabilities for sentence boundary detection (sbd), named entity recognition (ner), part-of-speech tagging (pos), sentiment analysis, and custom entity training. Version 2.4.0 requires a compatible language model, such as wink-eng-lite-web-model.

Tokens
2.1K
Snippets
7
Records
14
Agent score
80%

What's inside wink-nlp

  1. Install winkNLP via npm

    master

    Install the core wink-nlp package using npm. Note that you must also install a language model for the library to function. The recommended model for Node.js 16 or 18 is wink-eng-lite-web-model.

    npm install wink-nlp --save

  2. Install the appropriate language model

    master

    After installing wink-nlp, you must install a language model compatible with your Node.js version:

    • For Node.js 16 or 18 (Recommended): Use wink-eng-lite-web-model.
    • For Node.js 14 or 12: Run the internal installation script.
    Node.js VersionInstallation Command
    16 or 18npm install wink-eng-lite-web-model --save
    14 or 12node -e "require('wink-nlp/models/install')"
    npm install wink-eng-lite-web-model --save

  3. Configure TypeScript for winkNLP

    master

    To use winkNLP in a TypeScript project, ensure your tsconfig.json has the following compilerOptions enabled to handle module interop correctly:

    "compilerOptions": {
    "esModuleInterop": true,
    "allowSyntheticDefaultImports": true,
    ...
}

  6. Available NLP pipeline annotations

    master

    When configuring the pipe argument in the nlp() constructor, you can choose which annotations to perform. The available annotations depend on the provided model. Supported keys include:

    • sbd: Sentence Boundary Detection
    • ner: Named Entity Recognition
    • negation: Negation detection
    • sentiment: Sentiment Analysis
    • pos: Part-of-Speech (PoS) Tagging
    • cer: Custom Entity Recognition (Patterns)

  7. Initialize wink-nlp

    master

    To use wink-nlp, you must call the main function with a language model. You can optionally specify an annotation pipeline (an array of features to process) and word embeddings for vector operations.

    If no pipeline is provided, wink-nlp will attempt to enable all available annotations supported by the model (e.g., sbd, ner, pos, etc.).

  8. Quickstart: Hello World with winkNLP

    master

    This example demonstrates how to load the package and model, instantiate winkNLP, and perform basic NLP tasks like sentence splitting, entity extraction, and token frequency analysis using the its and as helpers.

    // Load wink-nlp package.
const winkNLP = require( 'wink-nlp' );
// Load english language model.
const model = require( 'wink-eng-lite-web-model' );
// Instantiate winkNLP.
const nlp = winkNLP( model );
// Obtain "its" helper to extract item properties.
const its = nlp.its;
// Obtain "as" reducer helper to reduce a collection.
const as = nlp.as;
 
// NLP Code.
const text = 'Hello   World🌎! How are you?';
const doc = nlp.readDoc( text );
 
console.log( doc.out() );
// -> Hello   World🌎! How are you?
 
console.log( doc.sentences().out() );
// -> [ 'Hello   World🌎!', 'How are you?' ]
 
console.log( doc.entities().out( its.detail ) );
// -> [ { value: '🌎', type: 'EMOJI' } ]
 
console.log( doc.tokens().out() );
// -> [ 'Hello', 'World', '🌎', '!', 'How', 'are', 'you', '?' ]
 
console.log( doc.tokens().out( its.type, as.freqTable ) );
// -> [ [ 'word', 5 ], [ 'punctuation', 2 ], [ 'emoji', 1 ] ]

  9. Extract entities from a collection with colEntitiesOut

    master

    The colEntitiesOut function is used to extract entity data from a collection of entities within a raw document data structure (rdd). It maps over the entities and applies a reducer to return a specific JavaScript data type or structure.

    Note that word vectors do not apply to entities. The function determines the return format based on the provided itsf (mapper) and asf (reducer) functions. If the requested detail or span is needed, it defaults to as.array unless the reducer is explicitly allowed for entities.

    /**
 * @param  {obejct}   entities entities from `rdd`; could be customEntities.
 * @param  {obejct}   rdd      Raw document data structure.
 * @param  {function} itsf     Desired `its` mapper.
 * @param  {function} asf      Desired `as` reducer.
 * @return {*}                 Reduced value.
 */
var colEntitiesOut = function ( entities, rdd, itsf, asf ) { ... };

  10. Train custom entities with `learnCustomEntities()`

    master

    You can extend the NLP engine's capabilities by training it to recognize custom entities using patterns. The learnCustomEntities(examples, config) method takes an array of example objects and returns the number of learned entities.

    Example Object Structure:

    • name: A string representing the entity type.
    • patterns: An array of strings representing the patterns to match.
    • mark (optional): An array of two integers [start, end] representing token indexes.

    Configuration Options:

    • matchValue: Boolean. If true, matches the value of the token.
    • usePOS: Boolean. Whether to use Part-of-Speech information (defaults to true).
    • useEntity: Boolean. Whether to use existing entities in the matching process (defaults to true).

  12. Retrieve word vectors with `vectorOf()`

    master

    If word embeddings were provided during initialization, you can retrieve the vector for a specific word using vectorOf(word, safe).

    • word: The string to look up.
    • safe: Boolean. If true (default), returns a slice of the vector up to the l2NormIndex. If false, returns the full vector.

    If the word is not found in the embeddings, it returns the unkVector (unknown vector).