gopatch

repository·main·Indexed 21 days ago

https://github.com/uber-go/gopatch

A tool for matching and transforming Go code using syntax-aware patches. Unlike text-based patch tools, gopatch understands Go syntax, making it suitable for safe refactoring and automated restyling. It supports metavariables for capturing Go expressions, statements, and identifiers, as well as elision using '...' to match wider criteria.

Tokens
7.6K
Snippets
27
Records
43
Agent score
76%

What's inside gopatch

  1. Understand position tracking with go/token.Pos

    main

    gopatch uses the standard Go go/token package for position tracking.

    • FileSet: A collection of files. It tracks the total length of all files combined.
    • Pos: An int64 that acts as a pointer into a FileSet. It provides a cheap way to identify a file, line number, and column number.

    Because Pos is an integer, gopatch can perform arithmetic on it to move between offsets. When transformations add or remove characters, gopatch generates PosAdjustments to ensure that subsequent parsing logic correctly maps positions back to the original source code.

  2. Understand the difference between identifiers, expressions, and statements

    main

    When writing patches with gopatch, it is important to distinguish between the three fundamental building blocks of Go code. This distinction determines which type of matcher or replacement you should use:

    • Identifiers: The names of things (e.g., variable names, function names, type names).
    • Expressions: Code segments that evaluate to a value (which can be passed to functions) or refer to types (e.g., nil, x + 1, myFunc()).
    • Statements: Instructions that perform actions (e.g., if blocks, return statements, variable declarations).

    Note: An identifier can be part of an expression, but not all identifiers are expressions. For example, in var bar Bar, bar is an identifier used in a statement, but it is not an expression in that specific context.

    ```go
    var bar Bar
    if err := foo(bar.Baz()); err != nil {
      return err
    }

    Breakdown of the example above:

    • Identifiers: err, foo, bar, Bar, Baz
    • Expressions: Bar, bar.Baz, bar.Baz(), foo(bar.Baz()), err, nil, and err != nil
    • Statements: var bar Bar, err := foo(bar.Baz()), if err := foo(bar.Baz()); err != nil { ... }, and return err
  3. Use elision with `...` to omit code portions

    main

    gopatch supports elision by using ... to represent unimportant or omitted portions of the code being matched or replaced. This allows you to focus the patch on the specific logic you want to change without specifying every line of the surrounding context.

    Elisions can be used in:

    • Expressions
    • Statements
    • Function declarations
    • Type declarations

    Important Note on Matching: Elisions in the - (find) and + (replace) sections are matched based on their position. Because this matching can sometimes be unreliable, it is recommended to restructure your patch so that elisions are on their own lines prefixed with a space ( ) to ensure stable matching.

    @@
    @@
    -foo(
    +bar(
    +  ...,
     )
  4. Understand the gopatch file and patch grammar

    main

    A gopatch file is composed of one or more patch blocks. Each patch consists of two main parts: a metavariables section and a diff.

    Metavariables Section

    Defined between two @@ markers. Metavariables are declared using Go's var syntax. They allow you to capture specific parts of the code to reuse them in the replacement section.

    Syntax: var <identifier> <type>

    Supported Types:

    • expression: Captures a Go expression.
    • identifier: Captures a Go identifier.

    Diff Section

    The diff uses standard diff prefixes to define the transformation:

    • -: Lines to be deleted (the 'Find' side).
    • +: Lines to be added (the 'Replace' side).
    • : Lines that match and should remain unchanged.

    Note that the - and + sections effectively define two separate files: the 'Find' file and the 'Replace' file. Both files may omit package clauses, imports, or function declarations, and may contain elisions (...).

    file = patch+
    
    patch = metavariables diff
    
    metavariables = '@@' metavariable* '@@'
    
    metavariable = 'var' identi metavariable_type
    
    metavariable_type = 'expression' | 'identifier'
    
    diff = '-' line | '+' line | ' ' line
  5. Declare metavariables in patches

    main

    Metavariables allow you to capture specific parts of the code being matched so they can be reused in the replacement section. They are declared at the top of a patch using the var keyword between @@ delimiters.

    Supported types include:

    • identifier: Matches any Go identifier (e.g., variable names, type names, function names).
    • expression: Matches any Go expression (e.g., function calls, variable references, attribute access, complex logic).

    When a metavariable is matched in the - section, its value is stored and can be referenced in the + section to reproduce the exact same code in the output.

    @@
    var x identifier
    @@
    -foo
    +x
  6. Understand gopatch terminology

    main

    To work with gopatch, you should understand its core structural components:

    • Program: A single .patch file containing one or more changes.
    • Change: A single transformation within a patch file, identified by an @ header.
    • Metavariables: Declarations made within the @@ (metavariables) section of a change used to capture and reuse code patterns.
    • Patch: The actual transformation logic, expressed as a unified diff (using - for removals and + for additions).
  7. How patch files are parsed

    main

    gopatch parses .patch files in several stages:

    1. Sectioning: The file is broken into independent sections: the Header (e.g., @@ or @name@), the Metavariables section, and the Patch (the diff).
    2. Metavariable Parsing: The @@ section is parsed using standard Go-style var declarations (e.g., var name type). This uses the go/scanner package.
    3. Patch Parsing: The unified diff is split into a Before version and an After version. Each version is parsed separately to ensure they are both valid syntax.
    4. PGO (Patch Go) Transformation: Since patch versions might not be valid Go (e.g., they might use ellipses ...), gopatch uses a superset called pgo. The tool scans pgo code with go/scanner and transforms it into a format that go/parser can handle by augmenting it with necessary boilerplate (like package or func declarations) and using pgo.Dots nodes to represent captured patterns.
  8. Enforce equality with metavariable repetition

    main

    If you use the same metavariable multiple times in the - (match) section of a patch, gopatch requires that every subsequent occurrence matches the exact same value captured by the first occurrence. This allows you to target specific patterns where values are repeated.

    ```diff
    @@
    var x expression
    @@
    -foo(x, x)
    +v := x
    +foo(v, v)

    Behavior:

    • foo(a, a) -> Matches
    • foo(x, y) -> Does not match
    • foo(getValue(), getValue()) -> Matches
  9. How patch files and metavariables work

    main

    A patch file contains one or more transformations formatted like unified diffs. Lines starting with - are matched and deleted; lines starting with + are added.

    Metavariables

    To avoid hard-coding exact values, use metavariables in the metavariable section (the @@ block at the top of a patch). Metavariables act as placeholders that match any Go code of a specific type (e.g., expression, statement, identifier) and can be reused in the transformation section.

    Example: Replacing a function call with a different one while preserving arguments

    @@
    var x expression
    @@
    -foo(x)
    +bar(x)

    In this example, x matches any Go expression (like 42, answer, or getAnswer()) and ensures that the exact same expression is passed to bar that was originally passed to foo.

  10. How the gopatch engine works

    main

    The gopatch engine drives the patching process by interpreting a parsed patch and operating on user-supplied Go files. It relies on two primary abstractions:

    • Matcher: Compiled from the "minus" (removal) section of a patch. It determines if a specific piece of Go code matches the pattern.
    • Replacer: Compiled from the "plus" (addition) section of a patch. It builds the AST that should replace the matched code.

    Data Sharing: Matchers and Replacers communicate via Data objects. Data is an immutable key-value store (similar to context.Context) that allows a Matcher to capture a value (like a metavariable) and a Replacer to retrieve and reuse that same value in the new code.

  11. Transform Go statements in patches

    main

    gopatch is not limited to expressions; it can transform Go statements (instructions that do not have a value, such as assignments, if statements, or variable declarations).

    For example, you can use a patch to inline an error assignment into an if statement to reduce variable scope. This requires declaring metavariables for the operation and the error identifier.

    @@
    var f expression
    var err identifier
    @@
    -err = f
    -if err != nil {
    +if err := f; err != nil {
       return err
     }
  12. Use elision with `...` to match wider criteria

    main

    To match code without specifying the exact number of arguments or elements, use elision by adding ... in the patch. This allows the patch to be more generic.

    Common use cases include:

    • Function calls: Replacing foo(...) with bar(...) matches all calls to foo regardless of argument count.
    • Return statements: Using return ..., err allows a patch to work on functions that return multiple values by eliding the other return values.
    @@
    @@
    -foo(...)
    +bar(...)
    @@
    
    @@
    var f expression
    var err identifier
    @@
    -err = f
    -if err != nil {
    +if err := f; err != nil {
       return ..., err
     }