DOCSEARCHbyAlgolia
Submit Docs

OpenHowNet

repository·master·Indexed 20 days ago

https://github.com/thunlp/openhownet

A Python API developed by THUNLP providing programmatic access to the HowNet sememe-based lexical knowledge base. It enables semantic search, sememe tree visualization, and word similarity calculations based on sememes (the minimum semantic units in linguistics). The library supports retrieving concepts (senses) for Chinese and English words, querying relationships between sememes, and integrates with BabelNet for multilingual synset information.

Tokens
6.1K
Snippets
22
Records
25
Agent score
71%

What's inside OpenHowNet

  1. Introduction to OpenHowNet

    master

    OpenHowNet is an open sememe-based lexical knowledge base developed by the Tsinghua University Natural Language Processing (THUNLP) Lab. It provides tools for querying sememe information, visualizing sememe trees, and calculating word similarity based on sememes.

    HowNet is built upon the concept of sememes (义原), which are defined as the smallest semantic units. The knowledge base uses a predefined set of over 2,000 sememes to annotate more than 200,000 concepts represented by Chinese and English words.

  2. Overview of OpenHowNet

    master
    OpenHowNet is an API developed by THUNLP that provides programmatic access to the HowNet knowledge base. It allows developers to search for information in HowNet, display sememe trees, and calculate word similarity based on sememes. It is designed to facilitate NLP tasks that require deep semantic understanding through sememes (the minimum semantic units in linguistics).

  3. Understand the HowNet Dictionary Data Format

    master

    The HowNet dictionary contains concepts (senses) represented by Chinese and English words. Each concept is annotated with metadata including POS tags, sentiment orientation, example sentences, and a sememe-based definition.

    Key fields in a concept record include:

    • NO.: Concept ID
    • W_C: Chinese word
    • G_C: POS tag of the Chinese word
    • S_C: Sentiment orientation (e.g., PlusFeeling|正面情感)
    • E_C: Example sentences for the Chinese word
    • W_E: English word
    • G_E: POS tag of the English word
    • S_E: Sentiment orientation for the English word
    • E_E: Example sentences for the English word
    • DEF: Sememe-based definition (e.g., {willing|愿意})
    • RMK: Remarks
    NO.=000000026417 	# Concept ID
W_C=不惜 	# Chinese word
G_C=verb 	# POS tag of the Chinese word
S_C=PlusFeeling|正面情感 	# Sentiment orientation
E_C=~牺牲业余时间，~付出全部精力，~出卖自己的灵魂 	# Example sentences of the Chinese word
W_E=do not hesitate to 	# English word 
G_E=verb 	# POS tag of the English word
S_E=PlusFeeling|正面情感 	# Sentiment orientation of the English word
E_E=               	# Example sentences of the English word
DEF={willing|愿意} 	# Sememe-based definition
RMK=

  4. Core Data Types in OpenHowNet

    master

    OpenHowNet is built around three primary classes:

    • HowNetDict: The main dictionary class. It encapsulates core functions like data retrieval, presentation, and similarity calculation.
    • Sense: Represents a concept in HowNet. It contains information such as Chinese/English words, Part-of-Speech (POS), and sememe-based definitions.
    • Sememe: Represents a sememe (the smallest unit of meaning). It includes Chinese/English descriptions, frequency, and relationships to other sememes.

  5. Install OpenHowNet and dependencies

    master

    To use OpenHowNet, you must have Python 3.X installed. The only external dependency required is anytree. Follow the official installation instructions to install the OpenHowNet API.

    pip install anytree
# Follow official instructions for OpenHowNet installation

  6. Initialize OpenHowNet and Download Data

    master

    To use the API, you must first initialize the HowNetDict object. If you have not downloaded the HowNet data yet, you must call OpenHowNet.download() to avoid errors.

    import OpenHowNet
# If data is not downloaded, run this first:
# OpenHowNet.download()

hownet_dict = OpenHowNet.HowNetDict()

  7. Install OpenHowNet

    master

    You can install OpenHowNet using pip (recommended) or by cloning the GitHub repository and running the setup script.

    Requirements:

    • Python >= 3.6
    • anytree >= 2.4.3
    • tqdm >= 4.31.1
    • requests >= 2.22.0
    # Method 1: via pip (recommended)
pip install OpenHowNet

# Method 2: via Github
git clone https://github.com/thunlp/OpenHowNet/
cd OpenHowNet
python setup.py install

  8. Initialize HowNetDict

    master

    To use the API, you must first initialize a HowNetDict instance.

    Note: If sememe data has not been downloaded, an error will occur. You must run OpenHowNet.download() before initializing if you haven't downloaded the data previously.

    import OpenHowNet
# Ensure data is downloaded first if necessary
# OpenHowNet.download()

hownet_dict = OpenHowNet.HowNetDict()

  9. Retrieve All Senses, Words, and Sememes

    master

    The API provides methods to export the entire HowNet dataset contents.

    all_senses = hownet_dict.get_all_senses()
zh_word_list = hownet_dict.get_zh_words()
en_word_list = hownet_dict.get_en_words()
all_sememes = hownet_dict.get_all_sememes()

  10. Query relationships between sememes

    master

    You can find the relationship between two sememes or find all sememes related to a specific sememe via a specific relation.

    # Find relationship between two sememes
# Returns the relation string (e.g., 'hyponym') or a list of triples if return_triples=True
relation = hownet_dict.get_sememe_relation('FormValue', '圆', return_triples=False)
triples = hownet_dict.get_sememe_relation('FormValue', '圆', return_triples=True)

# Find all sememes related to a sememe by a specific relation
# relation must be lowercase English
related = hownet_dict.get_related_sememes('FormValue', relation='hyponym', return_triples=True)

  11. Query sememe relations

    master

    You can explore relationships between sememes using the following methods:

    • get_sememe_relation(sememe1, sememe2, return_triples=False): Returns relations between two specific sememes. If return_triples=True, returns (head_sememe, relation, tail_sememe).
    • get_related_sememes(sememe, relation=None): Returns sememes related to a specific sememe instance. If relation is provided, it filters by that relation (e.g., 'hyponym').
    • get_related_sememes(sememe1, sememe2): (Via HowNetDict) Returns relations between two sememe strings.
    # Get triples between two sememes
triples = hownet_dict.get_sememe_relation('FormValue', '圆', return_triples=True)

# Get hyponyms of a sememe instance
sememe_instance = hownet_dict.get_sememe('FormValue')
related = sememe_instance.get_related_sememes(relation='hyponym')

  12. Get Relationships Between Sememes

    master

    You can query the relationship between two sememes using their English or Chinese names.

    • get_sememe_relation(sememe1, sememe2, return_triples=False): Returns the relation string (e.g., 'hyponym').
    • get_sememe_relation(..., return_triples=True): Returns a list of triplets (sememe1, relation, sememe2).

    To find all sememes related to a specific sememe, use get_related_sememes(sememe, relation, return_triples=True). Note that relation must be in lowercase English.

    # Get relation type
relation = hownet_dict.get_sememe_relation('FormValue', '圆', return_triples=False)

# Get relation triplets
triples = hownet_dict.get_sememe_relation('FormValue', '圆', return_triples=True)

# Find related sememes
related = hownet_dict.get_related_sememes('FormValue', relation='hyponym', return_triples=True)