Qwen LLM

Qwen-7B is a series of 7B-parameter large language models. The release provides two primary versions:

• Qwen-7B: The base pretrained language model, trained on over 2.2 trillion tokens with a context length of 2048.
• Qwen-7B-Chat: A version fine-tuned to align with human intent using curated task-oriented, security-oriented, and service-oriented data.

The repository includes example codes for fine-tuning, evaluation, and inference, as well as guides for long-context and tool-use inference.

Qwen provides two main types of models:

1. Qwen (Base Models): Foundational language models trained on up to 3 trillion tokens across multiple languages (primarily Chinese and English).
2. Qwen-Chat (Chat Models): Models fine-tuned using SFT (Supervised Fine-Tuning) and RLHF (Reinforcement Learning from Human Feedback) to follow human instructions. They are capable of chat, content creation, information extraction, summarization, translation, coding, and solving mathematical problems.

Available sizes include 1.8B, 7B, 14B, and 72B parameters. Quantized versions (Int4 and Int8) are also available for reduced memory usage.

The Qwen series includes Base models and Chat models across several parameter scales.

Model Specifications

Available Scales

• 1.8B: Qwen-1.8B, Qwen-1.8B-Chat
• 7B: Qwen-7B, Qwen-7B-Chat
• 14B: Qwen-14B, Qwen-14B-Chat
• 72B: Qwen-72B, Qwen-72B-Chat

Quantized versions (Int4 and Int8) are also available for various scales to reduce memory footprint.

Qwen provides an official training script finetune.py and several notebooks to facilitate fine-tuning pre-trained models for downstream applications. The supported algorithms include:

• Full-parameter fine-tuning: Updates all parameters in the model.
• LoRA fine-tuning: Updates only the parameters of adapter layers while keeping the original LLM layers frozen, reducing memory and computation costs.
• Q-LoRA fine-tuning: Uses a quantized large language model to further reduce memory costs.

Training notebooks are available for different combinations of algorithms and hardware configurations (Single GPU vs. Multiple GPUs).

The Qwen series consists of two main types of models available in various sizes (1.8B, 7B, 14B, and 72B):

1. Qwen (Base Models): Strong base language models pretrained on up to 3 trillion tokens of multilingual data. Available as Qwen-1.8B, Qwen-7B, Qwen-14B, and Qwen-72B.
2. Qwen-Chat (Chat Models): Models aligned with human preferences via SFT and RLHF, capable of chatting, content creation, summarization, coding, math solving, and tool usage. Available as Qwen-1.8B-Chat, Qwen-7B-Chat, Qwen-14B-Chat, and Qwen-72B-Chat.

Models are also released in quantized versions (Int4 and Int8) to reduce memory requirements and improve inference speed.

When using KV cache quantization, the layer_past format changes from a simple tuple of tensors to a tuple containing quantized tensors and their respective quantization parameters (scale and zero point).

Original format: layer_past=(key, value)

Quantized format: layer_past=((q_key, key_scale, key_zero_point), (q_value, value_scale, value_zero_point))

Qwen-Chat is optimized for tool use and function calling capabilities. You can develop agents, LangChain applications, or augment Qwen with a Python code interpreter.

Implementation follows the ReAct Prompting principle. Support for function calling is provided in openai_api.py based on this principle. For detailed implementation guidance, refer to the ReAct example.

Qwen-Chat is optimized for tool use and function calling, making it suitable for building Agents, LangChain applications, or Code Interpreters.

Implementation details:

• ReAct Prompting: You can implement tool calling based on the ReAct prompting principle. Detailed guidance is available in the examples/react_prompt.md file.
• Function Calling Support: The openai_api.py file provides built-in support for Function Calling, following the ReAct principle.
• Code Interpreter: Qwen demonstrates strong capabilities in generating executable Python code for tasks like mathematics, data visualization, file processing, and web crawling.

# Refer to openai_api.py for Function Calling implementation
# Refer to examples/react_prompt.md for ReAct prompting implementation

Qwen uses a tokenizer based on tiktoken, which differs from other tokenizers like SentencePiece. When performing fine-tuning, you must pay close attention to special tokens.

For detailed information regarding the tokenizer and how to use it during fine-tuning, refer to the tokenization notes.

Qwen is capable of performing general tasks such as mathematical problem solving, data visualization, file processing, and web scraping by using a Python code interpreter.

Benchmarks for evaluating these capabilities are available at the Qwen-Agent benchmark repository.