DOCSEARCHbyAlgolia
Submit Docs

Holi's Blog

website·Indexed 29 days ago

https://xn--0qu.fun

A personal knowledge base covering topics in machine learning, specifically diffusion models (DDPM, DDIM, DDBM, DBIM, and Score Matching Models), stochastic differential equations (SDE), and reinforcement learning. It also includes technical notes on Git version control, shell scripting, Vim, and parallel programming architecture.

Tokens
20.4K
Snippets
85
Records
173
Agent score
48%

What's inside Holi's Blog

  1. Overview of AI Computational Schools

    Artificial Intelligence is generally divided into three main schools of thought:

    • Behaviorism: Based on cybernetics; builds perception-action control systems (similar to the cerebellum), often associated with Reinforcement Learning.
    • Symbolism: Uses symbolic logic to represent knowledge and solve problems.
    • Connectionism: Based on neural network models of the brain to fit intelligent behavior.

  2. Introduction to Reinforcement Learning (RL) concepts

    Reinforcement Learning is a process where an agent learns from practice by attempting actions and receiving scores (rewards) from a 'teacher' or environment.

    RL Classifications:

    • Model-Free: Learns directly from environment feedback (e.g., Q-Learning, Sarsa, Policy Gradients).
    • Model-Based: Uses a simulated model of the real world to predict future scenarios and select actions.
    • Probability-Based: Selects actions based on the probability of success (e.g., Policy Gradients).
    • Value-Based: Selects the action with the highest calculated value (e.g., Q-Learning, Sarsa).

  3. Accelerate data processing with SIMD (Single Instruction, Multiple Data)

    SIMD increases throughput by using multiple ALUs (Arithmetic Logic Units) to perform the same operation on multiple data points simultaneously. Vectorization can be implemented explicitly by the compiler or implicitly by the hardware at runtime.

    Key concepts:

    • Coherence: The requirement that the same instruction acts on multiple data points simultaneously; essential for SIMD but not for multi-core parallelism.
    • Divergent Execution: Occurs when the instruction stream lacks coherence.

  4. Understanding the relationship between hardware architecture and program efficiency

    Hardware architecture significantly impacts program writing, efficiency, and performance. When implementing architectures to support program abstractions, designers must balance trade-offs between performance, convenience, and power consumption. High processor counts do not guarantee efficiency; for example, using 10 processors to achieve only a 2x speedup indicates poor efficiency.

  5. Understand the core concept of Score Matching Models (SMM)

    Score Matching Models aim to learn the 'score' of a data distribution $q(\mathbf{x})$, defined as the gradient of the log-density: $ abla_{\mathbf{x}} \log q(\mathbf{x})$. By fitting a neural network $\mathbf{s}_\theta(\mathbf{x})$ to this score, the model avoids the need to estimate the normalization constant (partition function) of the distribution. Once trained, the model can generate samples from the distribution using Langevin dynamics, which is essentially gradient ascent with added noise to ensure diversity and convergence to high-probability regions.

  6. Inspect remote repository information

    Use git remote show <remote> to view detailed information about a specific remote repository, such as its Fetch and Push URLs, the HEAD branch, tracked remote branches, and which local branches are configured for git pull and git push.
    $ git remote show origin

  7. Implement Instruction-Level Parallelism using Super-scalar architecture

    Super-scalar architecture allows a processor to fetch and execute multiple pre-compiled instructions simultaneously for a single program. In this model, parallelism is discovered automatically by the hardware.

  8. Understand Git's core architecture and data model

    Unlike most version control systems that store information as a list of changes (deltas), Git treats data as a series of snapshots of a mini file system. Every time you commit, Git takes a snapshot of all current files. To optimize storage, if a file has not changed, Git simply stores a link to the previous identical file. This 'snapshot stream' approach allows Git to operate almost entirely locally, as the full project history is stored on the local disk, enabling fast operations and offline work.

  9. Understand Reinforcement Learning (RL) classifications

    Reinforcement Learning methods are categorized based on how they interact with the environment and update their policies:

    • Model-Free vs. Model-Based: Model-free methods (e.g., Q-Learning, Sarsa, Policy Gradients) learn directly from environmental feedback. Model-based methods use a model to simulate the environment, allowing the agent to predict future scenarios and plan actions.
    • Probability-based vs. Value-based: Probability-based models (e.g., Policy Gradients) determine the probability of taking specific actions. Value-based models (e.g., Q-Learning, Sarsa) select the action with the highest estimated value. Actor-Critic combines both: the Actor chooses actions based on probability, and the Critic evaluates the value of those actions to accelerate learning.
    • Episode Update vs. Step Update: Episode updates (e.g., Monte-Carlo learning, basic Policy Gradients) update the policy only after a game/episode ends. Step updates (e.g., Q-Learning, Sarsa) update the policy at every single step, which is generally more efficient.
    • Online vs. Offline Learning: Online learning (e.g., Sarsa) requires the agent to learn while interacting with the environment. Offline learning (e.g., Q-Learning, DQN) allows the agent to learn from recorded data or experiences of others.

  10. Map data between two domains using Denoising Diffusion Bridge Models (DDBM)

    Denoising Diffusion Bridge Models (DDBM) enable direct mapping between two data spaces (e.g., mapping low-quality images to high-quality images), whereas standard DDPMs typically map from a noise space to a data space and require conditional guidance to achieve domain-to-domain translation. DDBM is based on the theory of Stochastic Differential Equations (SDEs) and Doob’s h-transform.

  11. Understand Version Control System (VCS) types

    Version Control Systems are categorized into three main types:

    • Local VCS: Saves patch sets on the local disk (e.g., RCS) to reconstruct file versions.
    • Centralized VCS (CVCS): Uses a single central server to store all file revisions. Clients connect to this server to retrieve files or submit updates. While it allows administrators to easily manage permissions, a server failure can result in total loss of work or history.
    • Distributed VCS: (Mentioned as a category, further details not provided in the text).

  12. Improve sampling efficiency with DPM-solver

    DPM-solver increases inference efficiency by treating the forward SDE (Stochastic Differential Equation) process as a corresponding ODE (Ordinary Differential Equation) process. It uses Taylor expansion to approximate the integral of the ODE, similar to the Runge-Kutta method, allowing for significantly fewer sampling steps while maintaining quality.