repository·master·Indexed 12 days ago
https://github.com/flairnlp/flairA PyTorch-based NLP framework for state-of-the-art tasks including Named Entity Recognition (NER), sentiment analysis, and PoS tagging. It provides a text embedding library supporting Transformers and various word/document embeddings, featuring tools like ModelTrainer for multi-GPU training and the Sentence object for text processing.
HunFlair2 is a state-of-the-art named entity tagger and linker specifically designed for biomedical texts. It utilizes pretrained domain-specific language models to perform Named Entity Recognition (NER) and entity linking.
Supported entity types include:
The examples/ directory contains maintained examples for various NLP tasks. You can find specific implementations and usage patterns for:
examples/ner/ directory.examples/multi_gpu/ directory.A Sentence object is the primary container for text that you want to embed or tag in Flair. When you initialize a Sentence with a string, the text is automatically tokenized (segmented into words and punctuation) using the segtok library.
You can provide a custom Tokenizer during initialization if you do not want to use the default.
To create a sentence:
from flair.data import Sentence
sentence = Sentence('The grass is green.')PooledFlairEmbeddings provide a 'global' representation of each distinct word by using a pooling operation of all past occurrences. Unlike standard embeddings, these embeddings evolve over time, meaning the same word in the same sentence may have different embeddings at different points in time.
Warning: PooledFlairEmbeddings are memory-intensive because they maintain past embeddings of all words in memory. For many use cases, regular FlairEmbeddings provide similar performance with significantly lower memory requirements.
from flair.embeddings import PooledFlairEmbeddings
from flair.data import Sentence
# init embedding
flair_embedding_forward = PooledFlairEmbeddings('news-forward')
# create a sentence
sentence = Sentence('The grass is green .')
# embed words in sentence
flair_embedding_forward.embed(sentence)The WordEmbeddings constructor accepts a string identifier to determine which pre-trained model to load.
'en', 'de', 'fr') to load FastText embeddings trained over Wikipedia/news data for that language.-crawl to a language code (e.g., 'de-crawl') to use FastText embeddings trained over web crawls.'en-glove', 'en-extvec', 'en-twitter', or 'en-turian'..gensim formatted file (e.g., 'path/to/your/custom/embeddings.gensim').flair.datasets.biomedical module. You can use the specific Data Set Class names to load these corpora for model development and evaluation.Stacked embeddings allow you to combine multiple embedding types (e.g., combining static GloVe with contextual Flair embeddings) into a single representation. This is a highly recommended pattern for sequence labeling tasks.
To use them, instantiate the StackedEmbeddings class by passing a list of existing embedding objects to its constructor. The resulting object behaves like a single embedding: calling .embed(sentence) on it will concatenate the vectors from all included embeddings into a single PyTorch vector for each token.
from flair.embeddings import WordEmbeddings, FlairEmbeddings, StackedEmbeddings
from flair.data import Sentence
# 1. Initialize individual embeddings
glove_embedding = WordEmbeddings('glove')
flair_embedding_forward = FlairEmbeddings('news-forward')
flair_embedding_backward = FlairEmbeddings('news-backward')
# 2. Stack them together
stacked_embeddings = StackedEmbeddings([
glove_embedding,
flair_embedding_forward,
flair_embedding_backward,
])
# 3. Use the stacked embedding on a sentence
sentence = Sentence('The grass is green .')
stacked_embeddings.embed(sentence)
# The token.embedding is now a concatenation of all three vectors
for token in sentence:
print(token.embedding)ELMo word embeddings can be constructed by combining ELMo layers using different strategies. When initializing ELMoEmbeddings, you can specify how layers are combined:
"all": Use the concatenation of the three ELMo layers."top": Use the top ELMo layer."average": Use the average of the three ELMo layers.By default, the top 3 layers are concatenated.
Flair provides two distinct training approaches depending on whether you are modifying a pre-trained language model or training a model from scratch (or training only a prediction head on top of frozen weights).
Use fine-tuning when you are working with a pre-trained language model and want to adapt it to a specific task. In this approach, you add a prediction head with randomly initialized weights to a model that already has millions of trained parameters. Because most parameters are already optimized, you should use:
Use the ModelTrainer.fine_tune() method for this approach.
Use the classic training approach (also known as "feature-based" or "probing") if the majority of your trainable parameters are randomly initialized. This is common when:
Because most parameters need to be learned from scratch, you should use:
Use the ModelTrainer.train() method for this approach.
CharacterEmbeddings allow you to incorporate character-level word embeddings into your model.
Important Note: These embeddings are randomly initialized when the class is instantiated. They do not contain pre-trained information and are only meaningful once they have been trained on a specific downstream task.
To make these features useful, you must include them in an embedding stack and pass that stack to a training method.
from flair.embeddings import CharacterEmbeddings
# CharacterEmbeddings are randomly initialized and require training
embedding = CharacterEmbeddings()When using launch_distributed, all processes must use the exact same corpus and preprocessing. If your corpus initialization involves randomness, you must ensure consistency using one of these two methods:
flair.set_seed(seed_value) before initializing the corpus to ensure all processes generate the same data splits/order.launch_distributed and pass the object as an argument to the function. This allows the corpus to be serialized and shared across all spawned processes.import flair
from flair.distributed_utils import launch_distributed
# Method 1: Set seed
flair.set_seed(42)
def main():
# corpus initialization happens here
pass
if __name__ == '__main__':
launch_distributed(main)Flair provides several NER models optimized for different priorities:
'ner'): Good tradeoff between accuracy and speed. Uses Flair embeddings and recognizes 4 entity types.'ner-large'): Uses a large transformer. Best for accuracy, but requires more memory and time. Supports multiple languages.'ner-fast'): Optimized for speed.'ner-ontonotes-large'): Distinguishes between 18 different entity types (e.g., DATE, MONEY, WORK_OF_ART).'bioner'): Specifically for biomedical data, detecting types like Disease, Gene, and Species.'de-ner-large' (German) or 'ar-ner' (Arabic) are available for non-English text.