Machine Learning Interview Preparation Guide

Two-stage object detectors perform object detection in two distinct steps:

1. Region Proposal: A Region Proposal Network (RPN) uses anchor boxes (pre-defined boxes of various sizes and aspect ratios) and CNNs to generate potential object bounding boxes and refine their coordinates.
2. Object Classification: An object classification network takes the proposed regions, extracts features using a CNN, and uses a classifier (typically a fully connected layer with softmax) to predict the object class and confidence score.

Examples: Faster R-CNN, R-FCN. Trade-offs: High accuracy and robustness, but computationally intensive and slower than one-stage detectors.

One-stage object detectors perform both region proposal and object classification in a single step (a single forward pass).

YOLO (You Only Look Once) approach: Divides the input image into a grid of cells. Each cell predicts:

• Bounding boxes
• Objectness scores (likelihood of an object being in the cell)
• Class probabilities

SSD (Single Shot Detector) and RetinaNet approach: Uses convolutional layers to extract features and generates anchor boxes at various scales and aspect ratios to predict object likelihood and refine coordinates.

Examples: YOLO, SSD, RetinaNet. Trade-offs: High speed and efficiency, but potentially lower accuracy for small or highly occluded objects.

The self-driving car architecture is typically divided into four functional layers that process sensor data into vehicle actions:

1. Perception: Converts raw sensor data (LiDAR, camera, etc.) into world understanding, including object detection (traffic lights, pedestrians, lanes) and localization (calculating vehicle position/orientation via Visual Odometry).
2. Behavior Prediction: Predicts the future trajectories of agents in the environment.
3. Planning: Performs decision-making and generates a specific trajectory based on the route, context map, and agent predictions.
4. Controller: Generates low-level control commands such as accelerate, brake, and steer.

Note on Latency: Tasks require millisecond-level latency, with some critical tasks requiring orders of 10 msec.

Modern ML interviews require breadth in Large Language Model (LLM) internals:

• Attention Mechanisms: Scaled dot-product attention (scaling by $\sqrt{d_k}$ for stable gradients), Multi-head attention (MHA), Multi-query attention (MQA), Grouped-query attention (GQA), and Multi-head latent attention (MLA).
• Positional Encodings: Absolute/learned, RoPE (rotary), ALiBi, and long-context extensions (position interpolation, YaRN).
• Efficiency & Optimization: KV cache (reduces complexity from $O(n^2)$ to $O(n)$ per token), FlashAttention (IO-aware/tiled attention), and RMSNorm.
• Architectural Components: Pre-norm vs post-norm, SwiGLU/GeGLU activations, and Mixture-of-Experts (MoE) with sparse expert routing.
• Tokenization: BPE, byte-level BPE, and SentencePiece.
• Scaling Laws: Chinchilla compute-optimal scaling.

The goal is to predict the future trajectory of objects given multiple past frames.

Input Options:

• Perception data + HDMap.
• Representations: Top-view (CNN), Vectorized (context map), or Graph (GNN).
• Temporal Modeling: Use a feature extractor (CNN) for each frame combined with an LSTM to capture temporal information.

Output Formats:

• Predict $(x, y, std)$ for future positions.
• Use LSTM networks to generate waypoints in a trajectory sequentially.

Key Challenge: Multimodality (the uncertainty of different possible future modes/paths).

To scale ML systems, you must address both general software scaling and ML-specific scaling:

General Software Scaling:

• Use distributed servers, load balancers, sharding, replication, and caching.
• Implement Train data / KB partitioning.

ML-Specific Scaling:

• Distributed ML:
  • Data parallelism: For training.
  • Model parallelism: For training and inference (e.g., Asynchronous SGD, Synchronous SGD).
• Scaling data collection: Use techniques like MT for 1000 languages or NLLB.
• Auto ML: Implement Hyperparameter (HP) tuning or Neural Architecture Search (NAS).

Model Selection

Use a Neural Network (NN) capable of handling unstructured data (text, image, video) and embedding layers for categorical features.

Training Configuration

• Loss Function: $L = \sum L_{is}$ for each task.
  • Use Cross-Entropy (CE) for binary classification tasks (e.g., P(click)).
  • Use MAE, MSE, or Huber loss for regression tasks (e.g., Dwell time).
• Dataset Handling:
  • Use features, post features, interactions, and labels.
  • For imbalanced datasets, use downsampling on negative samples.

Evaluation

• Perform hyperparameter tuning and iterative model evaluation.

Transform raw data into usable features using the following patterns:

Feature Types

• Actor-specific: Features belonging to a single entity (e.g., User profile, Item category).
• Cross-features: Interactions between actors (e.g., User-item watch history, Query-document tf-idf).
• Static vs. Dynamic: Features retrieved from a feature store vs. features computed online.

Representation and Preprocessing

• Categorical: One-hot encoding, Ordinal encoding, Count encoding.
• Numerical: Scaling and Normalization.
• Embeddings: For text, images, or graphs (can be pre-computed or learned).
• Unstructured Data Preprocessing:
  • Text: Tokenization (Normalization, subword/word level), adding special tokens.
  • Images: Resizing, normalization.
  • Video: Frame decoding, sampling, resizing, scaling.

For large-scale image retrieval (billions of images), exact search $O(N imes D)$ is too slow. Use Approximate Nearest Neighbor (ANN) search to achieve sublinear complexity (e.g., $O(D imes ext{log} N)$).

Supported ANN Methods

• Tree-based ANN: Uses structures like R-trees or Kd-trees to partition space and search specific partitions.
• Locality Sensitive Hashing (LSH): Uses hash functions to group close points into the same buckets.
• Clustering-based ANN.

Recommended Implementation

Use an existing library like Faiss (Facebook AI Similarity Search) for efficient vector search. To manage high memory usage from large embedding tables, utilize optimizations such as vector quantization or product quantization.

Review and practice responses to common behavioral questions often asked in Machine Learning and Engineering interviews, such as:

• Tell me about yourself.
• What's your proudest project?
• Why do you want to work here?
• How do you manage projects under pressure?
• How would you communicate technical challenges to non-technical stakeholders?
• Tell me about a time you had a conflict with a team member.
• Tell me about a time you made a mistake.
• How do you stay up-to-date on advances in ML?