Happy Transformer Documentation

Happy Transformer is a library designed to simplify fine-tuning and performing inference with NLP Transformer models. It provides high-level abstractions for several common NLP tasks, supporting both basic usage (inference) and training (fine-tuning) for most workflows.

Supported tasks include:

• Text Generation: Inference and Training
• Text Classification: Inference and Training
• Question Answering: Inference and Training
• Word Prediction: Inference and Training
• Text-to-Text: Inference and Training
• Token Classification: Inference only
• Next Sentence Prediction: Inference only

Happy Transformer supports various NLP tasks for both inference and training. Note that some tasks are deprecated.

Note: Word prediction, question answering, next sentence prediction, and token classification have been deprecated.

The TTSettings class allows you to control the behavior of the text generation process. When using generate_text(), you can pass an instance of TTSettings to the args parameter to customize sampling and length constraints.

Common parameters include:

• do_sample: Boolean to enable sampling.
• top_k: Integer for top-k sampling.
• top_p: Float for nucleus sampling.
• temperature: Float to control randomness.
• min_length: Minimum length of the generated text.
• max_length: Maximum length of the generated text.
• early_stopping: Boolean to enable early stopping during generation.

from happytransformer import TTSettings

settings = TTSettings(do_sample=True, top_p=0.8, temperature=0.7)

By default, HappyTextToText uses a "greedy" algorithm which simply picks the most likely next word. To change the generation algorithm (e.g., to beam search or sampling) or to adjust constraints like length and repetition, use the TTSettings class.

Pass an instance of TTSettings to the args parameter of the HappyTextToText.generate_text() method.

from happytransformer import HappyTextToText, TTSettings

happy_tt = HappyTextToText("T5", "t5-small")

# Configure settings
settings = TTSettings(do_sample=True, temperature=0.7, max_length=20)

# Pass settings to the args parameter
output = happy_tt.generate_text(
    "translate English to French: nlp is a field of artificial intelligence ",
    args=settings
)

If you need to load a private model from Hugging Face, use the use_auth_token argument when initializing HappyTokenClassification. Pass your Hugging Face authentication token as a string to grant access to the private repository.

from happytransformer import HappyTokenClassification

happy_toc_private = HappyTokenClassification(
    "BERT", 
    "user-repo/bert-base-NER", 
    use_auth_token="123abc"
)

Depending on your hardware constraints and performance requirements, you can choose different model combinations:

• Best Performance: HappyQuestionAnswering("ALBERT", "mfeb/albert-xxlarge-v2-squad2")
• Limited Hardware (Tiny models): HappyQuestionAnswering("BERT", "mrm8488/bert-tiny-5-finetuned-squadv2")
• Standard/Default: HappyQuestionAnswering("DISTILBERT", "distilbert-base-cased-distilled-squad")
• Robust Base Models: HappyQuestionAnswering("ROBERTA", "deepset/roberta-base-squad2")

from happytransformer import HappyQuestionAnswering

# High performance
happy_qa_albert = HappyQuestionAnswering("ALBERT", "mfeb/albert-xxlarge-v2-squad2")

# Low hardware usage
happy_qa_bert = HappyQuestionAnswering("BERT", "mrm8488/bert-tiny-5-finetuned-squadv2")

# Standard usage
happy_qa_roberta = HappyQuestionAnswering("ROBERTA", "deepset/roberta-base-squad2")

To perform text-to-text generation tasks, initialize a HappyTextToText object. You must specify the model_type (e.g., "T5" or "BART") and the model_name (the Hugging Face model identifier).

Initialization Arguments:

• model_type (string): The architecture name in all caps (e.g., "T5", "BART").
• model_name (string): The specific model identifier or URL from Hugging Face (e.g., "t5-small").
• use_auth_token (string): An authentication token required to load private models.
• trust_remote_code (bool): Set to True to allow custom Python files from the model location to be executed.

from happytransformer import HappyTextToText

# Standard initialization
happy_tt = HappyTextToText("T5", "t5-small")

# Initialization with a private model using an auth token
happy_tt_private = HappyTextToText("T5", "ericfillion/t5-small", use_auth_token="123abc")

To install the current stable version of Happy Transformer and avoid breaking changes introduced in the upcoming version 4.0.0, use the following command. It is recommended to pin your version to <4.0.0 in your requirements file.

pip install "happytransformer<4.0.0"

Install the happytransformer package using pip to get started with the library.

pip install happytransformer

To load a model that was previously saved using the .save() method, initialize a new Happy Transformer object and pass the directory path where the model was saved to the model_name parameter instead of a Hugging Face model identifier.

from happytransformer import HappyGeneration

# Provide the path to the saved directory as the model_name
happy_gen = HappyGeneration(model_type="GPT-NEO", model_name="model/")

Use the HappyQuestionAnswering class to perform question answering. This model extracts a text-span from a provided body of text that answers a specific question.

When initializing, you can specify the model architecture and a specific model name from Hugging Face. For high performance, it is recommended to use HappyQuestionAnswering("ALBERT", "mfeb/albert-xxlarge-v2-squad2").

from happytransformer import HappyQuestionAnswering

# Initialize with a specific model architecture and name
happy_qa = HappyQuestionAnswering("ROBERTA", "deepset/roberta-base-squad2")