Learning Deep Representations of Data Distributions

repository·main·Indexed 21 days ago

https://github.com/ma-lab-berkeley/deep-representation-learning-book

Source code and LaTeX files for the book 'Learning Deep Representations of Data Distributions'. Includes Python code for figures and technical examples (replearningbook v0.1.0), build instructions for the book and its website, and detailed proofreading and notation guidelines for Chapter 8.

Tokens
7.4K
Snippets
21
Records
42
Agent score
77%

What's inside deep-representation-learning-book

  1. Standardize image dimension ordering to (C, H, W)

    main

    The book uses channels-first ordering for all image dimensions.

    • Standard: (C, H, W) (Channels, Height, Width).
    • Action: Convert any (H, W, C) orderings (e.g., \x \in \mathbb{R}^{H \times W \times 3}) to the (C, H, W) format (e.g., \mathbb{R}^{c \times h \times w}).
    % Correct (Channels-first)
    \mathbb{R}^{c \times h \times w}
    
    % Incorrect (Channels-last)
    \mathbb{R}^{h \times w \times c}
  2. Use Prompt Modules for targeted proofreading

    main

    The project provides specialized instruction modules located in modules/ that can be used to guide AI models (like Cursor or the automated script) to perform specific proofreading tasks. Using individual modules ensures higher reliability than a single large prompt.

    Math Modules

    • modules/math_macros.md: Replaces raw commands with book macros (e.g., \v, \c, \bb, \KL, \ip, \norm, \mat).
    • modules/math_delimiters.md: Converts delimiters (e.g., $ to \(\)), handles equation* to equation conversion, and manages \[...\] conversion.
    • modules/math_style.md: Manages subscript braces, bracket sizing (\bp/\bs/\bc), \colon, and display equation punctuation.

    Text Modules

    • modules/text_editing.md: Handles grammar, emphasis (\textit), dashes, tone, and titles.
    • modules/references.md: Manages \Cref, \eqref, and citation styles (\citep/\citet).
    • modules/punctuation.md: Handles punctuation in and around math.

    Chapter 8 Specific Modules

    • modules/ch8_structural.md: Tables, figure paths, floats, captions, and image dimension ordering.
    • modules/ch8_notation.md: Resolves notation conflicts (e.g., theta, encoder/decoder, dataset variables).
    • modules/ch8_cleanup.md: Typos and review artifacts.
  3. Locate label-bounded sync regions

    main

    Sync regions are defined by LaTeX labels that act as anchors between English and Chinese files. Labels use prefixes such as ch:, sec:, sub:, eqn:, fig:, thm:, and ex:.

    To find a sync region for a changed line range in an English file:

    1. Upper Bound: Search upward from the first changed line to find the nearest \label{...} line. If none exists, use the beginning of the file.
    2. Lower Bound: Search downward from the last changed line to find the nearest \label{...} line. If none exists, use the end of the file.
    3. Extraction: The sync region includes all content between these two labels (inclusive of the label lines).

    Note: If multiple changed regions share the same bounding labels, they should be merged into a single sync region.

  4. Standardize Encoder/Decoder notation in Chapter 8

    main

    Chapter 8 establishes a primary notation of f_{\theta} for the encoder and g_{\eta} for the decoder. Several sections deviate from this:

    • VAE section: Uses \mathcal{E} and \mathcal{D}.
    • RAE section: Uses f and g without subscripts.
    • Michelangelo section: Uses \mathcal{E}_{\mathrm{s}}, \mathcal{D}_{\mathrm{s}}, \mathcal{E}_{\mathrm{i}}, \mathcal{E}_{\mathrm{t}}.
    • Conditional generation section: Uses \mathcal{E} and \mathcal{D}.

    Resolution: If the VAE encoder/decoder are architecturally distinct from the transformer-based f_{\theta}/g_{\eta}, calligraphic notation is acceptable but must be acknowledged as distinct. At a minimum, convert \mathcal{E} to \cE and \mathcal{D} to \cD to maintain distinction.

  5. Handle edge cases during translation sync

    main

    When performing a translation sync, be aware of the following edge cases:

    • Missing Labels: If a changed region lacks an upper label, use the start of the file. If it lacks a lower label, use the end of the file.
    • Merged Regions: If multiple changed regions are bounded by the same set of labels, merge them into one single sync region for translation.
    • Missing Translation Files: If the corresponding _zh.tex file for an English file does not exist, the process should skip that file and issue a warning.
  6. Resolve parameter variable ` heta` conflicts in Chapter 8

    main

    In Chapter 8, the symbol heta is primarily used for neural network parameters. However, conflicts arise in specific sections:

    • Cupid section: Redefines heta as camera pose ( heta = \boldsymbol{\theta}).
    • EgoAllo section: Uses heta for both SMPL joint rotations and network parameters.

    Resolution: Use a different symbol for pose or joint rotations (e.g., \vxi) or use clear subscripts to distinguish them from network parameters.

  7. Use correct citation styles with \citep and \citet

    main

    When citing sources, use specific LaTeX citation commands instead of the bare \cite{...} command:

    1. Parenthetical citations: Use \citep{...} when the citation should appear in parentheses within the sentence (e.g., "...as shown previously \citep{smith2020}.").
    2. Textual citations: Use \citet{...} when the author's name is part of the sentence flow (e.g., "As \citet{smith2020} showed...").
    % Parenthetical
    ...as shown previously \citep{smith2020}.
    
    % Textual
    As \citet{smith2020} showed...
  8. Reference figures, tables, sections, and chapters using \Cref

    main

    When referencing structural elements like figures, tables, sections, chapters, theorems, or remarks, always use the \Cref{...} command (with a capital 'C').

    Do not use:

    • Bare \ref{...}
    • Manual labels like Figure~\ref{...} or Section~\ref{...}
    • Lowercase \cref{...}

    Note that \Cref{...} automatically includes the type name (e.g., \Cref{ch:foo} produces "Chapter ##"), so you should not manually add the word "Chapter" or "Figure" before the command.

    % Correct
    \Cref{fig:example}
    \Cref{ch:intro}
    
    % Incorrect
    \ref{fig:example}
    \cref{fig:example}
    Figure~\ref{fig:example}
  9. Use correct equation environments

    main

    To ensure consistent formatting of displayed equations, adhere to these environment rules:

    • Single-line Equations: Always use the equation environment. Do not use equation* or \[ ... \]. Convert all existing \[ ... \] display math to \begin{equation} ... \end{equation}.
    • Multi-line Equations: Always use the align environment. Do not use align*, eqnarray, or aligned.
    • Conversion Rules:
      • Convert eqnarray and aligned to align where possible.
      • If an align-like environment contains only a single line (no \\ linebreaks), convert it to an equation environment.
      • If an align environment contains linebreaks (\\), do not convert it to equation.
    • Numbering: Every equation and every line within an align environment must be numbered. Do not use \nonumber or \notag, and remove them if they are present.
    % Correct single-line equation
    \begin{equation}
        a^2 + b^2 = c^2
    \end{equation}
    
    % Correct multi-line alignment
    \begin{align}
        f(x) &= (x+1)^2 \\
        &= x^2 + 2x + 1
    \end{align}