Qwen3 Large Language Model Family

Qwen3 is a family of large language models available in various sizes and modes. The latest update, Qwen3-2507, includes two primary modes:

1. Qwen3-Instruct-2507: A non-thinking mode optimized for general-purpose chat, instruction following, logical reasoning, mathematics, science, coding, and tool usage. It features enhanced long-context understanding (256K tokens, extendable up to 1 million tokens).
2. Qwen3-Thinking-2507: A thinking model designed for deep reasoning tasks (math, science, coding, and academic benchmarks) that require human-like expertise. It also supports 256K to 1 million tokens of context.

Available sizes for the 2507 series include 235B-A22B, 30B-A3B, and 4B.

Qwen3 (also known as Qwen3-2504) is a series of large language and multimodal models from the Qwen Team at Alibaba Group. The series includes both Dense and Mixture-of-Experts (MoE) architectures, with available sizes ranging from 0.6B to 235B-A22B.

Key features include:

• Dual Modes: Seamless switching between thinking mode (optimized for complex logical reasoning, math, and coding) and non-thinking mode (optimized for efficient, general-purpose chat) within a single model.
• Multilingual Support: Support for over 100 languages and dialects.
• Agent Capabilities: High performance in complex agent-based tasks and precise integration with external tools.
• Reasoning: Enhanced capabilities in mathematics, code generation, and commonsense logical reasoning.

When using Unsloth notebooks or scripts, you can adjust several key parameters to balance performance and memory usage:

• max_seq_length: The maximum context length. While Qwen3 supports up to 40960, a value of 2048 is often recommended for standard tasks.
• load_in_4bit=True: Reduces VRAM usage by approximately 4×.
• load_in_8bit=True: Enables 8-bit training.
• full_finetuning=True: Enables full fine-tuning instead of LoRA/QLoRA.

Qwen3 models follow a specific naming scheme: Qwen3[-size][-type][-date].

• size: Indicates structure and parameter counts.
  • Dense models: e.g., 4B, 32B (total saved parameters).
  • MoE (Mixture-of-Experts) models: e.g., 30B-A3B (total saved parameters and activated parameters per token).
• type: Defines the model's primary capability:
  • -Instruct: Instruction-following models using a predefined chat template for conversations.
  • -Thinking: Models using Chain-of-Thought (CoT) for complex problem solving.
  • -Base: Pre-trained models for in-context learning or fine-tuning (no predefined chat template).
  • No type: Models featuring hybrid thinking modes.
• date: Release date in YYYYMM format (e.g., 2507).

While Axolotl supports various formats, the recommended format for Qwen3 (when using chat_template) is the OpenAI Messages format.

Your dataset should be a JSON array of objects, where each object contains a messages list of role/content pairs.

To use this in your Axolotl configuration, set the type to chat_template and provide the path to your JSON file.

[
  {
    "messages": [
      {
        "role": "user",
        "content": "What is Qwen3?"
      },
      {
        "role": "assistant",
        "content": "Qwen3 is a language model..."
      }
    ]
  }
]

datasets:
  - path: path/to/your/dataset.json
    type: chat_template

Qwen uses Byte Pair Encoding (BPE) with a large vocabulary of 151,646 tokens. This ensures no unknown words (unk token) and efficient multilingual support.

Key Control Tokens:

• <|endoftext|>: End of document (eod token).
• <|im_start|>: Start of a turn (bot token).
• <|im_end|>: End of a turn (eot token).

Note on Padding and EOS:

• Qwen does not use a pad token in training; use a special token with attention masks. It is commonly set to <|endoftext|>.
• Qwen does not use a fixed bos or eos token for packed sequences. For inference frameworks that require an eos token, set it to <|im_end|>.

To separate the model's internal reasoning (thinking content) from its final answer (content), locate the </think> token (ID 151668) in the generated output IDs.

Everything before the </think> token is considered thinking_content, and everything after is the final content.

try:
    # rindex finding 151668 (</think>)
    index = len(output_ids) - output_ids[::-1].index(151668)
except ValueError:
    index = 0

thinking_content = tokenizer.decode(output_ids[:index], skip_special_tokens=True).strip("\n")
content = tokenizer.decode(output_ids[index:], skip_special_tokens=True).strip("\n")