DOCSEARCHbyAlgolia
Submit Docs

LLM Course: Fundamentals, Science, and Engineering

repository·main·Indexed 33 days ago

https://github.com/mlabonne/llm-course

A comprehensive educational resource for mastering Large Language Models (LLMs). Includes learning paths for LLM Fundamentals (Math, Python, Neural Networks), the LLM Scientist (architecture, pre-training, fine-tuning, alignment), and the LLM Engineer (deployment, quantization, RAG, and application building). Features practical Google Colab notebooks and tutorials covering tools like Unsloth, Axolotl, MergeKit, and llama.cpp.

Tokens
3.9K
Snippets
0
Records
19
Agent score
50%

What's inside mlabonne/llm-course

  1. Overview of the LLM Course structure

    main

    The LLM course is organized into three distinct learning paths:

    1. LLM Fundamentals (Optional): Covers foundational knowledge in Mathematics (Linear Algebra, Calculus, Probability/Statistics), Python, and Neural Networks.
    2. The LLM Scientist: Focuses on advanced techniques for building high-performance LLMs.
    3. The LLM Engineer: Focuses on building LLM-based applications and deployment strategies.

    For a more comprehensive version, users can refer to the DeepWiki version of this course.

  2. Master LLM Fundamentals: Mathematics, Python, and Neural Networks

    main

    To build a foundation for LLM work, focus on these core areas:

    1. Mathematics for Machine Learning

    Focus on Linear Algebra (geometric intuition), Statistics (probability and fundamentals), and Calculus (basics).

    2. Python for Machine Learning

    • Basics: Syntax, data types, error handling, and OOP.
    • Data Science Libraries: NumPy (numerical operations), Pandas (data manipulation), Matplotlib and Seaborn (visualization).
    • Data Preprocessing: Feature scaling, normalization, handling missing data, and data splitting.
    • Machine Learning Libraries: Proficiency with Scikit-learn for algorithms like linear regression, decision trees, and dimensionality reduction (PCA, t-SNE).

    3. Neural Networks

    • Fundamentals: Layers, weights, biases, and activation functions (sigmoid, tanh, ReLU).
    • Training: Backpropagation, loss functions (MSE, Cross-Entropy), and optimizers (Gradient Descent, RMSprop, Adam).
    • Regularization: Techniques to prevent overfitting such as dropout, L1/L2 regularization, early stopping, and data augmentation.
    • Implementation: Practice building a Multilayer Perceptron (MLP) using PyTorch.

  4. Master Natural Language Processing (NLP) Fundamentals

    main

    NLP is essential for bridging human language and machine understanding. Key concepts include:

    • Text Preprocessing: Tokenization, stemming, lemmatization, and stop word removal.
    • Feature Extraction: Converting text to numerical formats using Bag-of-words (BoW), TF-IDF, and n-grams.
    • Word Embeddings: Representing words with similar meanings closely in vector space using Word2Vec, GloVe, or FastText.
    • Recurrent Neural Networks (RNNs): Understanding sequence processing via LSTMs and GRUs to learn long-term dependencies.

  5. Optimize LLM Inference

    main

    To maximize throughput and reduce costs during text generation, implement the following optimization techniques:

    • Flash Attention: Transforms attention mechanism complexity from quadratic to linear to speed up training and inference.
    • Key-Value (KV) Cache: Utilize improvements like Multi-Query Attention (MQA) and Grouped-Query Attention (GQA).
    • Speculative Decoding: Use a smaller 'draft' model to produce tokens that are then reviewed by a larger model. Popular solutions include EAGLE-3 and the vLLM Speculators library.

  6. Evaluate LLM performance

    main

    Use multiple evaluation methods to guide data generation and training.

    Evaluation Methods

    • Automated Benchmarks: Use curated datasets and metrics (e.g., MMLU). Note: prone to data contamination and struggles with abstract/creative tasks.
    • Human Evaluation: Humans grade responses (e.g., Chatbot Arena). Best for subjective tasks but less reliable for factual accuracy.
    • Model-based Evaluation: Use judge or reward models to score outputs. Highly correlated with human preference but can suffer from self-preference bias.

    Recommended Frameworks

    • LM Evaluation Harness: Popular framework for automated benchmarks.
    • Lighteval: Framework supporting both automated and model-based evaluations.

  7. Develop LLM Agents

    main

    LLM agents perform tasks autonomously by reasoning about their environment through a cycle of Thoughts (internal reasoning), Actions (executing tasks via tools), and Observations (analyzing feedback).

    Protocols and Frameworks

    • Standard Protocols: Use Model Context Protocol (MCP) to connect agents to external tools/data, or Agent2Agent (A2A) for agent interoperability.
    • Vendor Frameworks: OpenAI SDK, Google ADK, and Claude Agent SDK.
    • Orchestration Frameworks:
      • LangGraph: For designing and visualizing complex workflows.
      • LlamaIndex: For building data-augmented agents with RAG.
      • CrewAI: For role-based team workflows.
      • AutoGen: For conversation-driven multi-agent systems.

  8. Implement Advanced RAG techniques

    main

    Beyond baseline retrieval, advanced RAG (Retrieval Augmented Generation) involves complex pipelines to improve relevance and functionality:

    • Query Construction: Translating user instructions into specific query languages like SQL or Cypher to access structured data.
    • Tools: Using Agents to automatically select relevant tools (e.g., Google, Wikipedia, Python interpreter, or Jira) to augment answers.
    • Post-processing: Enhancing retrieved documents using re-ranking, RAG-fusion, or classification.
    • Programmatic Optimization: Using frameworks like DSPy to optimize prompts and weights through automated evaluations.

  9. Implement Retrieval Augmented Generation (RAG)

    main

    RAG augments LLM knowledge by retrieving contextual documents from a database. Key components include:

    • Orchestrators: Use frameworks like LangChain or LlamaIndex to connect LLMs with tools and databases. Note the Model Context Protocol (MCP) for standardized data passing.
    • Retrievers: Enhance search using query rewriters (e.g., HyDE, CoRAG) or hybrid retrieval methods (combining embeddings with keyword signals).
    • Memory: Manage conversation history using context window buffers, summarization, or vector store-backed memory.
    • Evaluation: Assess the pipeline using two stages:
      • Retrieval Quality: Context precision and recall.
      • Generation Quality: Faithfulness and answer relevancy.
      • Tools: Use Ragas or DeepEval for automated quality assessment.

  10. Access LLM Notebooks and Tutorials

    main

    The repository provides a collection of Google Colab notebooks and articles for various LLM tasks. Key categories include:

    Tools

    • LLM AutoEval: Automatically evaluate LLMs using RunPod.
    • LazyMergekit: One-click model merging using MergeKit.
    • LazyAxolotl: One-click cloud fine-tuning using Axolotl.
    • AutoQuant: One-click quantization (GGUF, GPTQ, EXL2, AWQ, HQQ).
    • ZeroSpace: Automatically create Gradio chat interfaces using free ZeroGPU.
    • AutoAbliteration: Automatically abliterate models with custom datasets.
    • AutoDedup: Automatically deduplicate datasets using the Rensa library.

    Fine-tuning

    • Unsloth: Ultra-efficient supervised fine-tuning for Llama 3.1.
    • ORPO: Single-stage, cheaper, and faster fine-tuning for Llama 3.
    • DPO: Boost performance of SFT models using Direct Preference Optimization (Mistral-7b).
    • QLoRA: Supervised fine-tuning in free-tier Google Colab using TRL.
    • Axolotl: End-to-end guide for fine-tuning CodeLlama.

    Quantization

    • GPTQ: 4-bit quantization for consumer hardware.
    • GGUF/llama.cpp: Quantization for Llama 2 models and uploading to HF Hub.
    • ExLlamaV2: Quantize and run EXL2 models.

    Other

    • MergeKit: Merge LLMs or create Mixture of Experts (MoEs).
    • Abliteration: Uncensor LLMs without retraining.
    • Knowledge Graphs: Augment ChatGPT answers with knowledge graphs.

  11. Perform Supervised Fine-Tuning (SFT)

    main

    SFT transforms base models into instruction-following assistants.

    Training Techniques

    • Full Fine-Tuning: Updates all parameters; requires high compute.
    • LoRA (Low-Rank Adaptation): A parameter-efficient method that trains a small number of adapter parameters while freezing base weights. Key parameters include rank (typically 16-128), alpha (1-2x rank), and target modules.
    • QLoRA: Combines 4-bit quantization with LoRA to significantly reduce VRAM usage.

    Implementation & Scaling

    • Frameworks: Use TRL, Unsloth, or Axolotl for implementation.
    • Distributed Training: Scale across multiple GPUs using DeepSpeed (utilizing ZeRO optimization stages) or FSDP. Both support gradient checkpointing for memory efficiency.

    Monitoring

    Track loss curves, learning rate schedules, and gradient norms. Watch for loss spikes, gradient explosions, or performance degradation.

  12. Run LLMs locally or via APIs

    main

    To use LLMs in applications, you can either consume them via APIs or run them locally:

    • LLM APIs: Use private providers (OpenAI, Google Vertex AI, Anthropic) or open-source aggregators (OpenRouter, Together AI, Hugging Face Inference API).
    • Local Execution:
      • Download models from the Hugging Face Hub.
      • Use LM Studio for a GUI-based local experience.
      • Use llama.cpp or ollama via CLI for efficient local serving.
    • Prompt Engineering: Improve outputs using techniques like zero-shot, few-shot, Chain of Thought (CoT), and ReAct.
    • Structured Outputs: Use libraries like Outlines to guide generation toward specific formats (e.g., JSON) or use native JSON schema support provided by some APIs.