Stanza NLP Library Documentation

repository·main·Indexed 27 days ago

https://github.com/stanfordnlp/stanza

An official Python NLP library from the Stanford NLP Group providing accurate natural language processing tools for over 60 languages. It features a PyTorch-based neural pipeline and a Python wrapper for the Java Stanford CoreNLP software. Stanza supports neural pipeline training for tokenizers, POS taggers, lemmatizers, dependency parsers, and NER, and includes utilities for visualizing CoNLL-U dependency relations and Semgrex search results.

Tokens
5.2K
Snippets
16
Records
33
Agent score
93%

What's inside Stanza

  1. Train custom neural pipelines

    main

    You can train neural modules (tokenizer, MWT expander, POS/morphological tagger, lemmatizer, dependency parser, and NER) using your own data.

    Data Formats:

    • Tokenizer, MWT expander, POS/morphological tagger, lemmatizer, and dependency parser: CoNLL-U format.
    • NER model: BIOES format.

    Note: Model training is not supported via the Pipeline interface. You must clone the repository and run training from the source.

  2. Maximize pipeline speed with batching

    main
    To achieve maximum performance, avoid running a loop that processes one sentence at a time. Instead, concatenate multiple documents together, separating each document with a blank line (two line breaks \n\n). The tokenizer will treat these blank lines as sentence boundaries.
  3. Access Java Stanford CoreNLP via Stanza

    main

    Stanza provides an official wrapper to access Java Stanford CoreNLP.

    Setup Requirements:

    1. Download Stanford CoreNLP and the required language models.
    2. Place the model jars in the distribution folder.
    3. Set the CORENLP_HOME environment variable to the path of your Stanford CoreNLP installation.

    Example (nix): export CORENLP_HOME=/path/to/stanford-corenlp-4.5.3

  4. Get started with the Stanza neural pipeline

    main

    To use the neural pipeline, import stanza, download the language models (e.g., 'en' for English), and initialize a stanza.Pipeline. You can then process text strings to create a Document object.

    import stanza
    stanza.download('en')       # Optional: pre-download English models
    nlp = stanza.Pipeline('en') # Sets up a default neural pipeline in English
    doc = nlp("Barack Obama was born in Hawaii. He was elected president in 2008.")
    doc.sentences[0].print_dependencies()
  5. Run the Stanza interactive demo locally

    main

    To run the interactive demo locally, you must have stanza and flask installed.

    1. Disable Browser CORS: Ensure your browser's CORS rules are disabled (e.g., using a Chrome extension).
    2. Start the Server: From the stanza/pipeline/demo/ directory, set the FLASK_APP environment variable and run the Flask server:
      export FLASK_APP=demo_server.py
      flask run
    3. Configure Client Address: In stanza-brat.js, uncomment the serverAddress declaration and set it to your Flask server's address (typically http://localhost:5000).
    4. Launch: Open stanza-brat.html in your browser.
    export FLASK_APP=demo_server.py
    flask run
  6. Run Semgrex and Ssurgeon visualizations via Streamlit

    main

    You can use a Streamlit-based web interface to visualize Semgrex and Ssurgeon results and process files. To set up the Streamlit webpage, follow these steps:

    1. Install Streamlit: pip install streamlit.
    2. Install Stanford CoreNLP if you haven't already.
    3. Set the $CLASSPATH environment variable to point to your local Stanford CoreNLP installation.
    4. Install the required dependencies: streamlit, spacy, and ipython. (Note: You can use the visualization stanza setup option to automate this).
    5. Launch the application using the following command:
    streamlit run stanza/utils/visualization/semgrex_app.py --theme.backgroundColor "#FFFFFF"