Overview of MlWizardry Learning Materials
MlWizardry provides free educational materials for learning the fundamentals of Machine Learning (AI). The recommended starting point for AI development is learning the Python programming language.
ML Wizardry
website·Indexed 30 days ago
https://mlwizardry.netlify.appA comprehensive guide to machine learning and AI, covering fundamental concepts like supervised and unsupervised learning, regression, classification, and clustering (DBSCAN). The site provides practical tutorials on Python environment setup, pip, Google Colaboratory, and Mojo, as well as implementations of GPT using NumPy, Whisper for transcription, LangChain, and building ChatGPT applications with Next.js and Vercel.
What's inside ML Wizardry
Overview of DBSCAN clustering algorithm
DBSCAN (Density-Based Spatial Clustering of Applications with Noise) is a density-based clustering algorithm that identifies clusters as high-density regions and treats low-density regions as noise. Unlike k-means, it does not assume clusters are spherical and does not require the number of clusters to be specified in advance. It is particularly effective for datasets with noise and non-linear cluster shapes.
Common use cases include:
- Anomaly Detection: Identifying isolated points as noise.
- Image Segmentation: Grouping pixels with similar characteristics.
- Geospatial Analysis: Identifying physical groups like building clusters in a city.
Overview of Python for AI Development
Python is a widely used open-source language favored for AI development due to its simplicity and a rich ecosystem of machine learning libraries and frameworks, including TensorFlow, Keras, PyTorch, Scikit-learn, NumPy, Pandas, and Matplotlib.Overview of LangChain
LangChain is a library designed to leverage Large Language Models (LLMs) like GPT and GPT4ALL. It allows developers to extend LLM capabilities by integrating them with external data sources (Google Search, Notion, Google Docs), executing SQL, and performing computations that LLMs typically struggle with.Overview of the Mojo programming language
Mojo is a programming language developed by Modular, designed specifically for AI and machine learning. It is based on Python and aims to provide significant performance improvements (claimed up to 35,000x faster than Python). It supports core Python features such as async/await and error handling. Mojo files use either the.mojoor.🔥file extensions.Understand Linear Regression basics
Linear regression is a type of regression analysis that models the relationship between two variables using a straight line. It is used to predict a continuous value (y) based on an input value (x). The mathematical formula is represented asy = ax + b, whereyis the predicted value,xis the input, andaandbare coefficients determined by the model.Understand Unsupervised Learning concepts
Unsupervised Learning occurs when a model is trained using only input data (features) without any corresponding correct output data (labels). The objective is to discover the underlying structure, patterns, or natural groupings within the data.
Example: Animal Classification
- Features: Weight, body length, number of legs, etc.
- Label: None provided.
- Process: The model analyzes the features to group animals naturally. For instance, it might group animals with four legs and light weight into one cluster and animals with two legs and heavy weight into another, based solely on data patterns.
Compare Supervised vs Unsupervised Learning
The fundamental difference between Supervised and Unsupervised Learning is the presence of ground-truth labels in the training data:
Feature Supervised Learning Unsupervised Learning Data Input data + Labels Input data only Goal Predict labels for new data Find hidden patterns/structures Mechanism Learns mapping from input to output Discovers inherent data groupings Communicate and ask questions about mlwizardry content
If you have questions or uncertainties regarding the articles on the site, use the comment form located at the bottom of each article. Public discussions are encouraged to help deepen collective knowledge.Understand Classification in Supervised Machine Learning
Classification is a machine learning technique used to categorize data into specific groups. Unlike regression, which predicts continuous numerical values based on data relationships, classification assigns data to discrete categories. Common examples include determining if an email is spam or identifying whether a fruit is an apple or an orange.GPT-2 Parameter Weights Structure
Theparamsobject is a nested dictionary where leaf nodes are NumPy arrays containing the pre-trained weights. The weights are organized by their role in the architecture: weight matrices for embeddings (wpe,wte), layer normalization parameters (ln_f), and a list of transformer blocks containing attention and MLP weights.# Conceptual shape of params using hparams { "wpe": [n_ctx, n_embd], "wte": [n_vocab, n_embd], "ln_f": {"b": [n_embd], "g": [n_embd]}, "blocks": [ { "attn": { "c_attn": {"b": [3*n_embd], "w": [n_embd, 3*n_embd]}, "c_proj": {"b": [n_embd], "w": [n_embd, n_embd]}, }, "ln_1": {"b": [n_embd], "g": [n_embd]}, "ln_2": {"b": [n_embd], "g": [n_embd]}, "mlp": { "c_fc": {"b": [4*n_embd], "w": [n_embd, 4*n_embd]}, "c_proj": {"b": [n_embd], "w": [4*n_embd, n_embd]}, }, }, # ... repeat for n_layers ] }Understand Pre-training and Fine-tuning in LLMs
GPT models typically follow a two-step transfer learning strategy:
- Pre-training: The model is trained on a massive corpus of raw text (self-supervised learning) to understand general language patterns. The resulting model is often called a 'foundation model'.
- Fine-tuning: The pre-trained weights are further trained on a smaller, labeled dataset for a specific downstream task (e.g., toxicity classification).