scispaCy

You can load arbitrary databases and ontologies into scispaCy by integrating with pyobo. This allows you to use the EntityLinker logic with external identifiers like HGNC.

Requirements:

• pip install "pyobo>=0.12.9"

Use pyobo.get_scispacy_entity_linker(database_name, ...) to create a linker instance that can be used directly as a spaCy component.

import pyobo
import spacy
from scispacy.linking import EntityLinker
from tabulate import tabulate

# Create a linker for the HGNC database
linker: EntityLinker = pyobo.get_scispacy_entity_linker("hgnc", filter_for_definitions=False)

# Use it with a standard spaCy model
nlp = spacy.load("en_core_web_sm")

text = "RAC(Rho family)-alpha serine/threonine-protein kinase is an enzyme that in humans is encoded by the AKT1 gene."

# Apply the linker to the doc
doc = linker(nlp(text))

# Process results
rows = [
    (
        span,
        span.start_char,
        span.end_char,
        f"[{identifier}](https://bioregistry.io/{identifier})",
        score,
    )
    for span in doc.ents
    for identifier, score in span._.kb_ents
]
print(tabulate(rows, headers=["text", "start", "end", "prefix", "identifier"], tablefmt="github"))

The AbbreviationDetector is a spaCy component that implements the Schwartz & Hearst (2003) algorithm to identify abbreviation definitions in biomedical text.

Once added to the pipeline, you can access identified abbreviations via the doc._.abbreviations attribute. For each abbreviation, the long form (a spacy.tokens.Span) can be accessed using the span._.long_form attribute.

Note: To ensure doc objects can be serialized (saved and loaded), initialize the pipe with make_serializable=True in the config.

import spacy
from scispacy.abbreviation import AbbreviationDetector

nlp = spacy.load("en_core_sci_sm")

# Add the abbreviation pipe to the spacy pipeline.
nlp.add_pipe("abbreviation_detector")

doc = nlp("Spinal and bulbar muscular atrophy (SBMA) is an inherited motor neuron disease...")

print("Abbreviation", "\t", "Definition")
for abrv in doc._.abbreviations:
	print(f"{abrv} \t ({abrv.start}, {abrv.end}) {abrv._.long_form}")

The EntityLinker (added via scispacy_linker) performs string overlap-based searches (char-3grams) to link named entities to concepts in a knowledge base. It sets the ._.kb_ents attribute on spaCy Spans, which contains a list of (concept_id, score) tuples.

Supported Linkers (v2.5.0):

• umls: Unified Medical Language System (~3M concepts).
• mesh: Medical Subject Headings (~30k entities).
• rxnorm: RxNorm ontology (~100k clinical drug concepts).
• go: Gene Ontology (~67k gene function concepts).
• hpo: Human Phenotype Ontology (~16k phenotypic abnormality concepts).

Configuration Options:

• resolve_abbreviations (bool, default False): If True, the linker uses the long form of abbreviations identified by an AbbreviationDetector in the pipeline. This requires the detector to be added before the linker.
• k (int, default 30): Number of nearest neighbors to look up per mention.
• threshold (float, default 0.7): Minimum score for a candidate to be added.
• no_definition_threshold (float, default 0.95): Threshold for candidates without a definition.
• filter_for_definitions (bool, default True): Only return entities that have definitions in the KB.
• max_entities_per_mention (int, default 5): Maximum entities returned per mention.

You can look up entity details using the linker's kb attribute: linker.kb.cui_to_entity[concept_id].

import spacy
import scispacy
from scispacy.linking import EntityLinker

nlp = spacy.load("en_core_sci_sm")

# Note: resolve_abbreviations requires AbbreviationDetector to be in the pipeline
nlp.add_pipe("scispacy_linker", config={"resolve_abbreviations": True, "linker_name": "umls"})

doc = nlp("Spinal and bulbar muscular atrophy (SBMA) is an inherited motor neuron disease...")

# Accessing linked entities
entity = doc.ents[1]
linker = nlp.get_pipe("scispacy_linker")
for umls_ent in entity._.kb_ents:
	print(linker.kb.cui_to_entity[umls_ent[0]])