DOCSEARCHbyAlgolia
Submit Docs

MiniMind: Lightweight Educational LLM Project

repository·master·Indexed 13 days ago

https://github.com/jingyaogong/minimind

A highly reproducible project for training ultra-small language models (64M scale) from scratch using native PyTorch. It features a Transformer Decoder-Only architecture aligned with the Qwen3 ecosystem, supporting pre-training, supervised fine-tuning (SFT), and reinforcement learning (RL/DPO). The project includes tools for CLI inference, a Streamlit-based WebUI, and compatibility with inference engines like ollama and vllm.

Tokens
17.8K
Snippets
69
Records
95
Agent score
50%

What's inside MiniMind

  1. Overview of MiniMind

    master

    MiniMind is an open-source project designed to train ultra-small language models (approximately 64M parameters) from scratch. It is optimized for low-cost, high-speed training (e.g., ~2 hours on a single NVIDIA 3090 for 1 epoch of SFT) and is intended to be a reproducible tutorial for building LLMs.

    Key features include:

    • Minimalist Architecture: Lightweight models that are significantly smaller than GPT-3, making them runnable on consumer GPUs.
    • End-to-End Pipeline: Covers data cleaning, pretraining, Supervised Fine-Tuning (SFT), LoRA, RLHF (DPO), RLAIF (PPO / GRPO / CISPO), Tool Use, Agentic RL, Adaptive Thinking, and Model Distillation.
    • Native Implementation: Core algorithms are implemented directly in native PyTorch without high-level third-party abstractions.
    • Model Variants: Includes extensions like MiniMind-V (vision), MiniMind-O (multimodal Omni), MiniMind-dLM (diffusion language model), and MiniMind-Linear (linear attention).

  2. Overview of MiniMind-LLM capabilities

    master

    MiniMind is a project designed to lower the learning barrier for Large Language Models (LLMs) by providing a complete, from-scratch implementation of the MiniMind-LLM architecture (Dense + MoE). Unlike highly abstracted frameworks, MiniMind allows developers to understand every line of code, from tokenizer training to advanced reinforcement learning.

    Key Features:

    • Architecture: Full implementation of Dense and MoE (Mixture of Experts) models, aligned with the Qwen3 / Qwen3-MoE ecosystem.
    • Training Pipelines: Supports pretraining, SFT, LoRA, RLHF-DPO, RLAIF (PPO / GRPO / CISPO), Tool Use, Agentic RL, Adaptive Thinking, and Model Distillation.
    • Compatibility: Works with transformers, trl, peft, and inference engines like llama.cpp, vllm, and ollama.
    • Advanced Capabilities: Supports reasoning (<think> tokens), tool calling (<tool_call>), and long context extrapolation via YaRN.
    • Deployment: Provides an OpenAI-compatible API server and a Streamlit-based WebUI.

  3. Compare minimind models with other small models

    master

    The repository provides a subjective comparison between different versions of minimind and other small-parameter models (around 0.2B) across several dimensions: Accuracy, Completeness, Logic, and Code Quality.

    Models compared:

    • [A] minimind-3 (0.06B)
    • [B] minimind-3-moe (0.2B-A0.06B)
    • [C] baby-llama2-chinese (0.2B)
    • [D] chatlm-mini-chinese (0.2B)

    Evaluation Summary:

    • minimind-3-moe (B): Ranked highest in overall quality due to fluent expression, clear structure, and correct code implementation, despite significant knowledge hallucinations (e.g., misidentifying the Yangtze River as Mount Everest).
    • chatlm-mini-chinese (D): Highest knowledge accuracy (correctly identifying Newton, the Yangtze River's origin, etc.), but suffers from poor code quality (incorrect logic in sorting algorithms) and repetitive generation in long outputs.
    • minimind-3 (A): Similar style to the MoE version with usable code, but lower stability and higher density of factual errors.
    • baby-llama2-chinese (C): Lowest performance; struggles with basic facts, logic, and produces unusable code.

  4. Compare RL Model Variants (SFT vs GRPO vs Agent-CISPO)

    master

    The project provides different model weight versions based on different training methodologies. You can compare their performance based on your specific use case:

    • minimind-3 (64M, SFT): Best for general Q&A, factuality, and overall robustness in daily chat. It is more stable for knowledge-based tasks but may lack advanced tool-calling capabilities.
    • minimind-3 (64M, GRPO): An intermediate reinforcement learning variant.
    • minimind-3 (64M, Agent-CISPO): Optimized for lightweight Agent tasks and ToolUse. It excels at deciding when to call a tool and utilizing the results correctly.

    Note on the 'Alignment Tax': While the agent weights show higher success rates in verifiable solving and multi-step tool calling, they may exhibit higher hallucination rates and decreased stability on open-ended factual questions compared to full_sft weights.

  5. Enable Adaptive Thinking (Explicit Reasoning)

    master

    MiniMind manages explicit thinking (Chain-of-Thought) via the chat_template rather than separate model weights. This is controlled by the open_thinking switch:

    • open_thinking=0 (Default): Injects an empty <think>\n\n</think> block; the model answers directly.
    • open_thinking=1: Pre-injects the <think> start tag, prompting the model to output its reasoning process before the final answer.

    Usage via CLI:

    python eval_llm.py --load_from ./minimind-3 --open_thinking 1

    Usage via OpenAI-API-SDK: Pass open_thinking: True inside extra_body under chat_template_kwargs.

    Limitation: Currently, the model may struggle to stably output both a thinking process and a tool call simultaneously due to a lack of joint distillation training data.

    response = client.chat.completions.create(
    model="minimind",
    messages=[{"role": "user", "content": "你是谁？"}],
    # ...
    extra_body={"chat_template_kwargs": {"open_thinking": True}} # Thinking switch
)

  6. Compare RL model versions (SFT vs GRPO vs Agent-CISPO)

    master

    The project provides comparisons between different training stages of the minimind-3 (64M) model to help users choose the right weights for their use case.

    Model Variants:

    • [A] minimind-3 (64M, SFT): The standard Supervised Fine-Tuning model. Best for general chat, factual knowledge, and overall stability.
    • [B] minimind-3 (64M, GRPO): A version trained using Group Relative Policy Optimization.
    • [C] minimind-3 (64M, Agent-CISPO): An agent-optimized version. Best for ToolUse, lightweight multi-step calling, and verifiable problem solving (e.g., math tasks).

    Selection Guide:

    • Choose agent weights if your goal is ToolUse or tasks requiring verifiable results. Note that these weights may exhibit higher knowledge hallucination and lower stability in open-ended factual Q&A.
    • Choose full_sft weights if you prioritize daily chatting, factual accuracy, and general robustness. These weights are more stable but less capable in complex tool-calling scenarios.
    Comparison Summary (Example):
full_sft: 12/20 = 60.00%
agent: 17/20 = 85.00%

  7. Configure `max_seq_len` for training

    master

    The max_seq_len parameter refers to the token length, not the character count. When choosing a value, consider the following compression ratios:

    • Chinese text: ~1.5 to 1.7 characters per token.
    • English text: ~4 to 5 characters per token.

    Example Calculation:

    • 100 Chinese characters $\approx$ 60-67 tokens.
    • 100 English characters $\approx$ 20-25 tokens.

    Adjusting max_seq_len involves a trade-off: setting it too low causes information loss via truncation, while setting it too high wastes compute due to padding.

  8. Enable RoPE length extrapolation via YaRN

    master

    MiniMind supports RoPE (Rotary Positional Embedding) length extrapolation using the YaRN algorithm, allowing the model to handle text sequences longer than its training length more stably.

    For Native Torch Models

    When using eval_llm.py for inference, add the --inference_rope_scaling flag:

    For Transformers Format Models

    Add the rope_scaling configuration to your config.json file:

    "rope_scaling": {
    "type": "yarn",
    "factor": 16.0,
    "original_max_position_embeddings": 2048,
    "beta_fast": 32.0,
    "beta_slow": 1.0,
    "attention_factor": 1.0
}
    python eval_llm.py --weight full_sft --inference_rope_scaling

  9. Summary of Policy Optimization (PO) Algorithms

    master

    All Policy Optimization algorithms in MiniMind follow a unified framework: $\mathcal{J}_{PO} = \mathbb{E} [\text{policy term} \cdot \text{advantage term} - \text{regularization term}]$. The differences lie in how these three components are instantiated.

    AlgorithmPolicy term $f(r_t)$Advantage term $g(A_t)$Regularization term $h(\text{KL}_t)$Models Required
    DPO$\log r_w - \log r_l$No explicit advantage termImplicit in $\beta$1 (2 in forward)
    PPO$\min(r, \text{clip}(r))$$R - V(s)$$\beta \cdot \mathbb{E}[\text{KL}]$2
    GRPO$\min(r, \text{clip}(r))$$\frac{R - \mu}{\sigma}$$\beta \cdot \text{KL}_t$1
    CISPO$\text{clip}(r, 0, \varepsilon_{max}) \cdot A_t \cdot \log \pi_\theta$$ rac{R - \mu}{\sigma}$$\beta \cdot \text{KL}_t$1

  10. Understand the RL data format for preference learning

    master

    The Reinforcement Learning (RL) data used for preference optimization (like DPO) in the MiniMind main branch follows a specific format in dpo.jsonl. Each entry contains a chosen response (the preferred one) and a rejected response (the weaker one). Both are represented as lists of message objects with content and role keys.

    Other RL data (for PPO/GRPO/etc.) follows the standard SFT format but typically leaves the last assistant position blank to allow for model rollout/continuation.

    {
  "chosen": [
    {"content": "Q", "role": "user"}, 
    {"content": "good answer", "role": "assistant"}
  ], 
  "rejected": [
    {"content": "Q", "role": "user"}, 
    {"content": "bad answer", "role": "assistant"}
  ]
}

  11. Understand MiniMind-3 model architecture

    master

    The minimind-3 series uses a Transformer Decoder-Only architecture aligned with the Qwen3 ecosystem, facilitating easy conversion to transformers, llama.cpp, ollama, and vllm.

    Core Architecture Features

    • Normalization: Pre-Norm + RMSNorm.
    • Activation: SwiGLU.
    • Positional Encoding: RoPE (supports YaRN extrapolation).
    • Configuration: q_heads=8, kv_heads=4, max_position_embeddings=32768, rope_theta=1e6.

    MoE Variant (minimind-3-moe)

    Extends the architecture with MoE feed-forward layers (compatible with Qwen3-MoE style, without shared experts).

    • Default: 4 experts / top-1 routing.
    • Performance Note: While MoE offers higher capacity with lower active parameters, native PyTorch implementation may be ~50% slower than dense models due to kernel scheduling overhead unless using optimized kernels (e.g., Triton, DeepSpeed-MoE).

  12. Resume training from checkpoints

    master

    All training scripts support automatic checkpoint detection and resumption. To resume training, add the --from_resume 1 flag to your command.

    python train_pretrain.py --from_resume 1
python train_full_sft.py --from_resume 1

    Details:

    • Checkpoints are saved in the ./checkpoints/ directory.
    • Naming convention: <weight_name>_<dimension>_resume.pth (e.g., full_sft_512_resume.pth).
    • Supports recovery across different GPU counts and automatically resumes WandB/SwanLab training records.
    python train_pretrain.py --from_resume 1