DOCSEARCHbyAlgolia
Submit Docs

SeqGPT

repository·main·Indexed 42 days ago

https://github.com/alibaba-nlp/seqgpt

A bilingual Large Language Model optimized for open-domain Natural Language Understanding (NLU) in English and Chinese. SeqGPT performs NLU tasks by treating them as combinations of classification and extraction tasks, utilizing a specific prompt template and the [GEN] special token for generation.

Tokens
847
Snippets
2
Records
3
Agent score
3%

What's inside SeqGPT

  1. How SeqGPT handles NLU tasks

    main

    SeqGPT is a bilingual (English and Chinese) Large Language Model designed for open-domain Natural Language Understanding (NLU). It is capable of handling any NLU task that can be decomposed into a combination of two atomic task types:

    1. Classification (分类): Assigning input text to specific categories or labels.
    2. Extraction (抽取): Extracting specific information or entities from the text.

    The model is trained on diverse synthesized data and high-quality NLU datasets to facilitate these transformations.

  2. Install SeqGPT

    main

    To set up the SeqGPT environment, create a new Conda environment with Python 3.8.16 and install the required dependencies using the provided requirements.txt file.

    conda create -n seqgpt python==3.8.16

conda activate seqgpt
pip install -r requirements.txt

  3. Run inference with SeqGPT

    main

    You can perform inference using the transformers library. SeqGPT supports two primary task types: classification (分类) and extraction (抽取).

    Important Note: The model relies on a specific prompt template. Changing the instruction format can harm performance. The expected format is: 输入: {sent} {task}: {labels} 输出: [GEN]

    Where [GEN] is the special token [GEN] used to trigger generation.

    from transformers import AutoTokenizer, AutoModelForCausalLM, AutoModel
import torch

model_name_or_path = 'DAMO-NLP/SeqGPT-560M'
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path)
model = AutoModelForCausalLM.from_pretrained(model_name_or_path)
tokenizer.padding_side = 'left'
tokenizer.truncation_side = 'left'

if torch.cuda.is_available():
    model = model.half().cuda()
model.eval()
GEN_TOK = '[GEN]'

# Example usage logic
sent = 'Your input text here'
task = '分类' # or '抽取'
labels = 'labelA,LabelB'

p = '输入: {}\n{}: {}\n输出: {}'.format(sent, task, labels, GEN_TOK)
input_ids = tokenizer(p, return_tensors="pt", padding=True, truncation=True, max_length=1024)
input_ids = input_ids.to(model.device)
outputs = model.generate(**input_ids, num_beams=4, do_sample=False, max_new_tokens=256)

# Extracting only the generated part
input_ids_tensor = input_ids.get('input_ids', input_ids)
outputs = outputs[0][len(input_ids_tensor[0]):]
response = tokenizer.decode(outputs, skip_special_tokens=True)
print(response)