DOCSEARCHbyAlgolia
Submit Docs

bobby50792 Notes and Blog

website·Indexed 30 days ago

https://bobby50792.com/

A collection of academic notes and articles covering mathematics for computer science and AI, including probability, calculus (single and multivariable), and complex analysis. The site also features documentation on deep learning, machine learning, artificial intelligence, advanced computer graphics, general physics (mechanics, thermal physics, electricity, and magnetism), and music theory.

Tokens
22.4K
Snippets
47
Records
148
Agent score
50%

What's inside bobby50792

  1. Analyze high-dimensional distributions (Gaussian, Sphere, Ball)

    In high dimensions ($d$), three fundamental distributions exhibit specific geometric properties:

    1. Gaussian Distribution $\mathcal{G} = \mathcal{N}(\boldsymbol{0}, \boldsymbol{I}/d)$: Scaled so that the expected squared norm $\mathbb{E}[|\boldsymbol{x}\_2^2] = 1$ regardless of $d$.
    2. Uniform Sphere Distribution $\mathcal{U}_{\mathbb{S}}$: Uniform distribution on the unit sphere $\mathbb{S}^{d-1} = {\boldsymbol{x} \in \mathbb{R}^d : |\boldsymbol{x}\_2 = 1}$.
    3. Uniform Ball Distribution $\mathcal{U}_B$: Uniform distribution on the unit ball $B^d = {\boldsymbol{x} \in \mathbb{R}^d : |\boldsymbol{x}\_2 \le 1}$.

    Concentration Inequalities:

    • Ball Norm: For $\boldsymbol{x} \sim \mathcal{U}_B$, $\Pr[1 - \epsilon < |\boldsymbol{x}\_2 \le 1] \ge 1 - e^{-\epsilon d}$.
    • Gaussian Norm: For $\boldsymbol{x} \sim \mathcal{G}$, $\Pr[||\boldsymbol{x}\_2^2 - 1| \ge \epsilon] \le 2e^{-d\epsilon^2/8}$.
    • Orthogonality (Inner Product): For a fixed unit vector $\boldsymbol{u}$, the inner product $\langle \boldsymbol{u}, \boldsymbol{x} \rangle$ concentrates around 0:
      • For $\boldsymbol{x} \sim \mathcal{U}_B$ ($d \ge 3$): $\Pr[|\langle \boldsymbol{u}, \boldsymbol{x} \rangle| < \epsilon] \ge 1 - 6e^{-(d-1)\epsilon^2/2}$.
      • For $\boldsymbol{x} \sim \mathcal{U}_S$: $\Pr[|\langle\boldsymbol{u}, \boldsymbol{x}\rangle| < \epsilon] \ge 1 - 7e^{-(d-1)\epsilon^2 / 8}$.

  2. Understand the mathematical formulation of Normalizing Flows

    Normalizing Flows are generative models that transform a simple distribution $\pi(z)$ into a complex data distribution $p(x)$ using an invertible function $G$.

    Key concepts include:

    • Generation: Sample $z_0 \sim \pi(z)$ and compute $x = G(z_0)$.
    • Change of Variables: For invertible networks, the probability density is given by $p(x) = \pi(z) \cdot |\det(J_{G^{-1}}(x))|$, where $z = G^{-1}(x)$ and $J_{G^{-1}}$ is the Jacobian matrix of the inverse transformation.
    • Loss Function: The log-likelihood is expressed as $\log p(x) = \log \pi(G^{-1}(x)) + \log |\det(J_{G^{-1}}(x))|$.
    • Composition: $G$ can be decomposed into a series of $K$ sub-networks: $z_K = f_{\theta_K} \circ f_{\theta_{K-1}} \circ \dots \circ f_{\theta_1}(z_0)$.
    • Chain Rule for Log-Likelihood: $\log p(z_K) = \log \pi(z_0) - \sum_{k=1}^{K} \log |\det(\frac{\partial f_{\theta_k}}{\partial z_k})|$.

  3. Understand Implicit Generative Models for image generation

    Implicit Generative Models aim to create a sampler $g(\cdot)$ that generates images approximating the data distribution $p_{data}$. Unlike explicit models, they use likelihood-free learning, where the goal is to minimize a distance metric $D(g(z;\theta), p_{data})$ that does not involve likelihood calculations. A simple generator $g(z;\theta)$ takes a latent vector $z$ sampled from a standard normal distribution $z \sim N(0, I)$ and produces an image $x = g(z;\theta)$.

  4. Collaborative Human-AI Workflow for PPT Creation

    To avoid 'AI-flavored' or soulless presentations, use a workflow where the human defines the core logic, viewpoints, and structure first, using AI as a supportive tool for summarization, logic checking, and drafting. The process is divided into seven distinct stages to ensure the human maintains creative control.
    StepHuman ResponsibilityAI's RoleDeliverable
    1. ResearchFind sources, verify credibilitySummarize, categorize, list controversiesResearch notes
    2. OutlineDefine theme, viewpoints, sequenceCheck for logic gapsHuman-written outline
    3. Detailed OutlineBreak down per-page content/examplesSupplement cases, compress expressionPer-page detailed outline
    4. Prompt PolishingDefine audience, constraints, styleConvert detailed outline into executable promptsPPT generation prompt
    5. First DraftSelect tools and template directionGenerate page draftsPPT first draft
    6. Layout & ImagesJudge accuracy and aestheticsSuggest layout/image improvementsPresentable version
    7. AnimationDecide presentation pacingOptimize animation sequence and scriptFinal presentation

  5. Implement the Gradient Descent update rule

    Gradient Descent (GD) is an iterative optimization algorithm used to minimize a differentiable loss function $L: \mathbb{R}^d \to \mathbb{R}$. Given an initial parameter vector $\theta_0 \in \mathbb{R}^d$ and a learning rate $\eta > 0$ (step size), the parameters are updated iteratively using the gradient of the loss function.
    # Iterative update rule: theta_{t+1} = theta_t - eta * grad(L(theta_t))
theta = theta - learning_rate * gradient_of_loss

  6. Create heuristics using Relaxed Problems

    Heuristics can be derived by solving a 'relaxed' version of the original search problem $P$. A relaxation $P_j$ is a problem where the costs satisfy $Cost_j(s, a) \le Cost(s, a)$.

    The relaxed heuristic is defined as $h(s) = FutureCost_j(s)$, which is the minimum cost from $s$ to an end state using the relaxed costs.

    Key properties:

    • If $h(s)$ is derived from a relaxed problem $P_j$, it is a consistent heuristic.
    • If $h_1(s)$ and $h_2(s)$ are both consistent, then $h(s) = \max{h_1(s), h_2(s)}$ is also consistent.

  7. Collaborate with AI using a human-led framework

    To avoid generic 'average' answers and unverified data, avoid outsourcing the thinking process to AI. Instead, use a workflow where the human defines the core judgment and structure, and the AI optimizes the expression and efficiency. The recommended approach is: Human defines viewpoints $ ightarrow$ AI optimizes expression $ ightarrow$ Tools generate drafts $ ightarrow$ Human makes final selections/trade-offs.
    # Non-Recommended (Outsourcing thinking):
"Help me make a high-end, content-rich PPT about prompt engineering."

# Recommended (Collaborating with a framework):
"I have researched the materials and determined the main line: Prompt engineering is about supplementing pragmatic context for AI. My outline is: Case introduction, shared context, Grice's four maxims, PPT workflow, summary. Please do not rewrite the theme; instead, help me check for logical leaps, supplement usable cases for each page, and organize it into an 8-page PPT detailed outline."

  8. Analyze Diatonic Chords and Functional Harmony

    In a major key, the sequence of triad qualities is: Major, Minor, Minor, Major, Major, Minor, Diminished.

    • Tonic (T): I (e.g., C in C Major). Stable center.
    • Pre-Dominant (PD): ii, IV (e.g., Dm, F in C Major). Leads to Dominant.
    • Dominant (D): V, vii° (e.g., G, Bdim in C Major). Leads back to Tonic.

    Basic harmonic progression: T → PD → D → T.

  9. Optimize parameters using the Gradient Descent Algorithm

    Gradient Descent is used to iteratively minimize a function $f(X)$ by moving in the opposite direction of the gradient.

    • Update Rule: $X_{k+1} = X_k - \eta_k \nabla_X f(X_k)^T$, where $\eta_k$ is the learning rate.
    • Convergence Criterion: The algorithm converges when the difference between successive iterations is smaller than a predefined threshold: $|f(X_{k+1}) - f(X_k)| < \epsilon$.

  10. Construct a bass line for harmonic progressions

    Bass lines can be constructed using root notes, inverted chord tones, passing tones, or pedal points. The smoothness of the bass line depends on its movement relative to other voices. To create a smooth transition, maintain common tones (pedal points) or use chromatic ascent (e.g., F# to G) to resolve to the target note. When writing traditional four-part harmony, ensure there are no parallel fifths or octaves.

    Example progression: G → Em → C → D → G Bass line: G → G → G → F# → G Full notation: G → Em/G → C/G → D/F# → G

  11. Collaborative Human-AI Workflow for PPT Creation

    To avoid 'AI-flavored' or soulless presentations, use a workflow where the human defines the core logic and framework first, and AI acts as an assistant for refinement and execution. The process is divided into seven distinct stages to ensure human judgment remains the primary driver.
    StepHuman ResponsibilityAI's RoleDeliverable
    1. ResearchFind sources, verify credibilitySummarize, categorize, list controversiesResearch Notes
    2. OutlineDefine theme, core views, sequenceCheck for logic gapsHuman-written Outline
    3. Detailed OutlineBreak down per-page content & examplesSupplement cases, compress phrasingPer-page Detailed Outline
    4. Prompt RefinementDefine audience, constraints, styleConvert detailed outline into executable promptsPPT Generation Prompt
    5. First DraftSelect tools and template directionGenerate page draftsPPT First Draft
    6. Layout & ImagesJudge accuracy and aestheticsSuggest layout/image improvementsPresentable Version
    7. AnimationDecide presentation pacingOptimize animation sequence and scriptFinal Presentation

  12. Workflow for melodic dictation

    To transcribe a melody by ear, use the following process:

    1. Establish the beat and determine the starting note count, rests, and grouping.
    2. Record the rhythm first, then note the direction (ascending, descending, static) and intervals (steps or leaps).
    3. Use a known tonic or reference note to locate the first note, last note, and long notes.
    4. Convert short melodic blocks into scale degrees, then translate those to note names based on the key.
    5. Review the recording to verify durations, accidentals, and cadence positions.