wn Wordnet Interface Library

repository·main·Indexed 43 days ago

https://github.com/goodmami/wn

A Python library for exploring and querying multilingual wordnets with a high-performance SQL-based backend. It supports WN-LMF 1.4 format, interlingual queries via the Collaborative Interlingual Index (CILI), and provides tools for taxonomy exploration, similarity metrics, and lemmatization. The library includes specifiers for downloading datasets like Open English WordNet (oewn:2025+), Open Multilingual Wordnet (omw:1.4), and others.

Tokens
45.6K
Snippets
175
Records
229
Agent score
54%

What's inside wn

  1. Overview of wn features

    main

    The wn library provides several advanced linguistic and computational features:

    • Multilingual Support: First-class support for wordnets in any language.
    • Interlingual Queries: Support for queries via the Collaborative Interlingual Index.
    • Similarity Metrics: Access to six different similarity metrics.
    • Taxonomy Exploration: Functions for exploring wordnet taxonomies.
    • Linguistic Processing: Support for lemmatization (including built-in Morphy for English) and unicode normalization.
    • Format Support: Full support for the WN-LMF 1.4 format, including word pronunciations and lexicon extensions.
    • Performance: A SQL-based backend for fast startup and high-performance queries.
  2. Understand the core concepts of a Wordnet

    main

    A wordnet is an online lexicon organized by concepts. To use or build a wordnet, you must understand its three primary hierarchical units:

    1. Words: The basic building blocks of language (word forms).
    2. Senses: The unit of meaning. Because one word form can have multiple meanings (e.g., bank as a financial institution vs. bank as a river edge), senses act as the bridge between words and concepts.
    3. Synsets (Synonym Sets): The fundamental unit of a wordnet. A synset is a group of words (or word senses) that all refer to the same concept (e.g., cab, taxi, and hack all belong to one synset).

    In summary: One word form can have multiple senses, and each sense belongs to a specific synset. Multiple word forms can also map to the same synset.

  3. Differentiate between Package and Collection projects

    main

    Wordnet resources are organized into three main types of project objects:

    1. Package: Represents a directory containing a specific Wordnet package. It includes a type property and a resource_file() method.
    2. ResourceOnlyPackage: A virtual package used to model simple resource files as packages to maintain API consistency.
    3. Collection: Represents a directory that groups multiple packages together. It provides a packages() method to iterate over or access the constituent packages.
  4. Understand the Project class hierarchy

    main

    The Project class is the base abstraction for all Wordnet resources in the library. While you do not instantiate Project directly, you will interact with its subclasses. All projects share common metadata methods:

    • path: The filesystem path to the project.
    • readme(): Returns the project's README content.
    • license(): Returns the project's license information.
    • citation(): Returns the citation information for the project.
  5. How Morphy and Wordnet work together for query expansion

    main

    Morphy can be used to expand queries in wn.Wordnet.

    1. Uninitialized approach: You can pass an uninitialized Morphy instance to the lemmatizer argument when creating a wn.Wordnet object. This is useful for high-performance candidate generation where you rely on the database to filter results.
    2. Initialized approach: You can assign an initialized Morphy (one that has been built using a Wordnet instance) to the ewn.lemmatizer property. This provides pre-filtered, valid lemmas.

    In both cases, calling ewn.words('wordform') will use the assigned lemmatizer to find related lemma forms.

    import wn
    from wn import morphy
    
    # Uninitialized approach (passed at creation)
    ewn = wn.Wordnet('ewn:2020', lemmatizer=morphy.Morphy())
    print(ewn.words('lemmas'))
  6. WN-LMF Structure: Files, Packages, and Collections

    main

    Wn supports three levels of WN-LMF (WordNet Lexical Markup Format) organization:

    1. WN-LMF XML Files: Single .xml files following the WN-LMF specification.
    2. WN-LMF Packages: A directory containing exactly one .xml file and optional metadata:
      • LICENSE (.txt, .md, or .rst)
      • README (.txt, .md, or .rst)
      • citation.bib (BibTeX file)
    3. WN-LMF Collections: A directory containing multiple WN-LMF packages as subdirectories. The collection itself can also contain a README, LICENSE, and citation.bib describing the whole project.
  7. Use wn.compat for lexicon-specific functionality

    main
    The wn.compat subpackage provides compatibility modules for working with specific lexicons. While the core wn package is designed to be language-agnostic, wn.compat contains specialized functionality that is useful for certain lexicons but does not generalize across all of them. Use these modules when you need features tied to a particular lexicon's structure or conventions.
  8. Understand Wn Lexicon Terminology

    main

    To use Wn effectively, understand the hierarchy of its data structures:

    • Lexicon: An inventory of words, senses, synsets, and relations sharing a namespace.
    • Wordnet: A group of lexicons (often just one).
    • Resource: A single file containing lexicons.
    • Package: A directory containing a resource and optional metadata (e.g., LICENSE, README, citation.bib).
    • Collection: A directory containing multiple packages and optional metadata.
    • Project: A general term for a resource, package, or collection, specifically regarding its distribution and maintenance.
  9. Understand Word, Sense, and Synset objects

    main

    Primary queries in wn return one of three core objects representing different levels of lexical information:

    1. Word: Encodes word forms independent of meaning (e.g., spelling, part of speech).
    2. Sense: A pairing of a Word and a Synset, representing a specific meaning of a word.
    3. Synset: A set of words that share a single meaning (the semantic unit).

    Refer to the wn.Word, wn.Sense, and wn.Synset classes for detailed method signatures and attributes.

  10. How synset and sense relations work

    main

    Wordnets are structured through semantic relations that connect different units. These are categorized into three types:

    Synset Relations

    Relations built at the synset level. For example, a hypernym relation exists when one synset represents a more general concept than another (e.g., the synset containing car is a hypernym of the synset containing taxi).

    Sense-Sense Relations

    Relations that link one specific sense to another sense, often used for morphologically related words. For example, the adjective behavioral has a pertainym relation with the noun behavior.

    Sense-Synset Relations

    Relations that connect a specific sense to an entire synset. For example, a specific term (sense) might have a has domain topic relation with a synset representing a discipline (e.g., cursor has a domain topic relation with the computer science synset).

  11. Understand ILI object statuses

    main

    The status attribute of an ILI or ProposedILI object indicates its validity and how it was discovered in the WordNet (Wn) environment.

    StatusClassDescription
    ILIStatus.ACTIVEILIThe identifier is explicitly listed in a loaded ILI file.
    ILIStatus.PRESUPPOSEDILIA lexicon contains synsets referencing an identifier, but no corresponding ILI file has been loaded.
    ILIStatus.PROPOSEDProposedILIA synset in WN-LMF format proposes a new ILI (has a definition but no identifier).
    ILIStatus.UNKNOWNEitherThe default status, typically encountered only when manually creating objects.

    Note: You can explicitly add information about ILIs to Wn using wn.add("cili") to move them from a guessed state to a known state.

  12. Use Lexicon and Project Specifiers

    main

    Wn uses specifiers to identify specific lexicons or versions when multiple are loaded. Specifiers join a lexicon's ID and version with a colon (:).

    Specifier Formats:

    • id — The most recently added lexicon with that ID.
    • id:* — All lexicons with that ID.
    • id:version — A specific lexicon ID and version.
    • *:version — All lexicons with a specific version.

    Project Specifiers: These refer to projects in Wn's index. While often identical to lexicon specifiers (e.g., ewn:2020), a project specifier might trigger the installation of multiple lexicons (e.g., omw:1.4 installs omw-zsm:1.4, omw-cmn:1.4, etc.). Providing only an ID (e.g., ewn) selects the first version listed in the index.

    # Examples of specifier patterns
    id
    id:*
    id:version
    *:version