DOCSEARCHbyAlgolia
Submit Docs

MLSysBook: Machine Learning Systems Engineering Curriculum

repository·dev·Indexed 12 days ago

https://github.com/harvard-edge/cs249r_book

A comprehensive curriculum for ML Systems engineering featuring a theoretical textbook, hands-on labs, TinyTorch framework building, hardware deployment, and MLSys·im infrastructure simulation. Includes the Binder CLI for managing the book lifecycle, including building HTML/PDF/EPUB outputs, content validation, bibliography hygiene, and automated formatting.

Tokens
596.2K
Snippets
1.1K
Records
2.6K
Agent score
97%

What's inside MLSysBook

  1. Overview of Math-rendering audit tools

    dev

    The math-rendering audit tools consist of three scripts designed to detect LaTeX leakage in rendered HTML and PDF outputs. These tools ensure that LaTeX syntax is correctly handled by MathJax or contained within code blocks, rather than appearing as raw text in the final rendered document.

    Note: All scripts must be run from the repo root.

    Available Scripts

    ScriptPurpose
    audit_math_rendering.pyBuilds each chapter's HTML via binder and scans the rendered output for raw LaTeX leakage outside MathJax/code zones.
    audit_math_pdf.pyBuilds per-chapter PDFs, extracts text via pdftotext, renders pages to PNG via pdftoppm for visual spot-checking, and applies the same leak detector to the extracted text.
    audit_pdf_spot_check.pyScans the PDFs produced by audit_math_pdf.py for known fix sites (regex map maintained in the script) and emits a markdown manifest pointing to the exact page numbers / PNGs to inspect.

  2. Overview of the Spaced Repetition System

    dev

    The Spaced Repetition System is a module within the SocratiQ widget (src_shadow/js/components/spaced_repetition/) designed for managing and reviewing flashcards. It provides a modal interface for deck management, supports AI-assisted card creation via DuckAI, and persists user data locally in the browser.

    Key Capabilities:

    • Deck Management: Create and curate decks, typically grouped by chapter IDs.
    • Storage: Uses IndexedDB (via SpacedRepetitionIndexDBHandler) for live builds, with localStorage used as a fallback.
    • AI Generation: Converts selected text into flashcards using a Cloudflare proxy to DuckAI.
    • Analytics: Provides statistics using Chart.js and exploratory visualizations (like network graphs or heatmaps) using D3.

  3. Overview of StaffML Interview Playbook

    dev

    StaffML is an open-source interview preparation platform designed for ML systems engineers. It provides access to over 9,000 physics-grounded systems design questions organized by competency, difficulty, and deployment track.

    Key features include:

    • Vault: Browse questions by area, topic, and difficulty.
    • Practice: Spaced repetition and daily challenges.
    • Gauntlet: Timed mock interview sessions with self-assessment.
    • Progress: Tracking coverage across competency areas and tracks.
    • Chains: Sequences of questions that deepen understanding of a topic from basic recall to high-level architecture.

    Data Sources:

    • Question corpus: vault/corpus.json (generated via vault build)
    • Taxonomy: vault/taxonomy.json
    • Application source: staffml/

  4. Overview of MLPerf EDU Review Packets

    dev

    MLPerf EDU Review Packets are structured documentation sets generated from the native registry and structured asset dossiers. They are designed to facilitate feedback and review processes for:

    • MLCommons: For official workload review.
    • Instructors: For educational oversight.
    • Artifact Reviewers: For technical validation.

    Note that while a committed evidence summary within a packet may allow a repository to pass CI (Continuous Integration), the packet itself may still act as a blocker for raw-package publication until all requirements are met.

  5. Overview of the HarvardX TinyML Professional Certificate

    dev

    The HarvardX Professional Certificate in Tiny Machine Learning (TinyML) is a program offered by Harvard University and Google TensorFlow. It focuses on bringing machine learning to performance- and power-constrained embedded systems.

    Key Learning Areas:

    • TinyML Fundamentals: Basics of ML and embedded systems (smartphones, etc.).
    • TinyML Applications: Training models for keyword spotting, visual wake words, and gesture recognition.
    • Deploying TinyML: Programming in TensorFlow Lite for microcontrollers to deploy models to hardware.

    Hardware Requirements: The program emphasizes hands-on experience using a TinyML Program Kit, which includes:

    • An Arduino board with onboard sensors.
    • An ARM Cortex-M4 microcontroller.
    • A camera module.

    Applications built with this kit include image recognition, audio processing, and gesture detection.

  6. Overview of the TinyTalks Dataset

    dev
    TinyTalks is a lightweight, educational conversational Q&A dataset designed for teaching transformer architectures. It is specifically optimized for students to train a transformer model and observe meaningful learning patterns in under 5 minutes. The dataset is part of the TinyTorch educational framework and is intended for rapid prototyping, architecture testing, and understanding transformer mechanics rather than production use.

  7. Overview of TinyTorch

    dev

    TinyTorch is an educational project designed to teach users how to build a Machine Learning (ML) framework from scratch. Unlike traditional courses that focus on using existing frameworks like PyTorch or TensorFlow, TinyTorch focuses on implementing the underlying 'AI bricks'—the engineering foundations of ML systems.

    Key Features:

    • Zero Dependencies: Everything is built from scratch using only NumPy.
    • Comprehensive Scope: Covers everything from basic tensors and autograd to CNNs, Transformers, and performance optimization (quantization, profiling).
    • Systems-Focused: Emphasizes understanding memory usage, compute, and scaling rather than just calling .fit() methods.
    • Historical Milestones: The curriculum is structured around recreating landmark ML achievements (e.g., Perceptron, Backpropagation, CNNs, Transformers).

  8. Overview of TinyTorch Datasets

    dev
    TinyTorch Datasets provides small, lightweight datasets designed for rapid iteration and offline learning in TinyTorch milestone examples. The datasets are categorized into two types: Shipped Datasets (included in the repository for instant use) and Downloaded Datasets (larger standard benchmarks downloaded on-demand).

  9. Overview of MLSysBook Tooling Areas

    dev

    The book/tools directory is organized into the following functional areas:

    AreaPurpose
    scripts/Python and shell entrypoints for generation, maintenance, release, and Quarto-adjacent workflows.
    audit/Durable audit records and audit helpers used to track book quality work.
    dependencies/Dependency and environment documentation.
    git-hooks/Repository hook documentation and related setup.

  10. Overview of Volume III: Acting Systems and the Inference-Time Frontier

    dev

    Volume III of the Machine Learning Systems curriculum shifts the focus from served requests to acting systems. The core thesis is that when a system spends inference-time compute in a loop with state and tools, it becomes an actor, and the fundamental engineering unit shifts from the individual request to the trajectory.

    The curriculum is organized into four parts:

    1. The Inference-Time Frontier: Covers the transition from trained models to acting systems, the fundamental control loop (perceive, decide, act, observe), and inference-time scaling (trading runtime compute for capability).
    2. The Substrate: Focuses on the underlying infrastructure, including memory hierarchies (state and memory), tool/environment interfaces (typed action surfaces), and multi-agent coordination (distributed systems with nondeterministic nodes).
    3. Reliability and Economics: Addresses the systems engineering challenges of error compounding over trajectories and the economic implications of multi-step inference costs and latency.
    4. The Responsible Agent: Explores safety, control, and oversight (keeping loops observable and bounded) and the security of autonomous systems (treating the autonomous loop as a unique threat model).

  11. Overview of the Machine Learning Systems Curriculum

    dev

    The cs249r_book repository is an integrated curriculum designed to teach AI Engineering: the discipline of building efficient, reliable, safe, and robust intelligent systems. The curriculum is composed of several interconnected components that move from theory to practical implementation and modeling.

    Core Components

    For Students

    • Textbook: A two-volume MIT Press textbook providing the theoretical foundation and mental models.
    • Labs: Interactive Marimo notebooks powered by MLSys·im to explore system trade-offs.
    • Tiny🔥Torch: A progressive 20-module series for building an ML framework from scratch.
    • Hardware Kits: Practical deployment exercises for Arduino, Seeed, Grove, and Raspberry Pi devices.
    • MLSys·im: A modeling engine used to calculate memory bottlenecks, network saturation, and scheduling limits at scale.
    • StaffML: A practice platform for physics-grounded ML systems interview preparation.

    For Educators

    • Instructor Hub: Provides the 'AI Engineering Blueprint', including 16-week syllabi, pedagogy guides, assessment rubrics, and a TA handbook.
    • Lecture Slides: Beamer slide decks for every chapter with multiple theme variants.
    • Newsletter: Updates on curriculum changes and community developments.

  12. Overview of Volume I: Introduction to Machine Learning Systems

    dev
    Volume I focuses on the foundations of building, optimizing, and deploying machine learning systems on a single machine with one to eight accelerators. It covers the full stack from data engineering to model serving, emphasizing physical hardware constraints such as memory hierarchies, compute throughput, and power budgets. It establishes core quantitative frameworks like the Iron Law of ML Systems and the D.A.M Taxonomy which are used throughout the volume to reason about performance bottlenecks.