dst

repository·master·Indexed 23 days ago

https://github.com/dave/dst

A Go package for high-fidelity manipulation of Go syntax trees. Unlike the standard go/ast package, dst uses 'Decorations' to ensure that comments and line spacing remain attached to the correct nodes during source-to-source transformations, preventing the loss or misalignment of metadata when nodes are rearranged.

Tokens
5.9K
Snippets
15
Records
39
Agent score
80%

What's inside dst

  1. What is a Decorated Syntax Tree (dst)?

    master
    The dst package provides a way to manipulate Go syntax trees with high fidelity. Unlike the standard go/ast package, which stores comments by byte offset and causes them to detach or misalign when nodes are rearranged, dst uses 'Decorations' to ensure that comments and line spacing remain attached to the correct nodes even after the tree has been modified.
  2. Control spacing and newlines with Before and After properties

    master

    The Before and After properties on a node control vertical spacing (new lines or empty lines) around that node. These spaces are rendered before any Start decorations and after any End decorations. Use dst.EmptyLine or dst.NewLine to set these. To force a newline inside a node, add a \n decoration to an attachment point.

    call.Decs.Before = dst.EmptyLine
    call.Decs.After = dst.EmptyLine
    
    for _, v := range call.Args {
    	v := v.(*dst.Ident)
    	v.Decs.Before = dst.NewLine
    	v.Decs.After = dst.NewLine
    }
  3. Map between ast.Node and dst.Node using Decorator.Mappings

    master
    The Decorator maintains mappings between ast.Node and dst.Node via Dst.Nodes and Ast.Nodes. This allows you to bridge information from tools that operate on standard ast (like go/types) back to your dst tree. For example, you can find the *dst.Ident corresponding to an *ast.Ident found in types.Info.Uses.
  4. Why use `dst` instead of `go/ast`?

    master

    The standard go/ast package is not designed for source code manipulation. When you rearrange nodes in an ast.File (for example, swapping statements in a function body), comments often end up attached to the wrong lines or lose their relative positioning because they are tracked via byte offsets rather than being properties of the nodes themselves.

    dst solves this by attaching decorations (comments, spacing, newlines) directly to the nodes, ensuring that when you move a node, its associated metadata moves with it.

  5. Configure Resolvers for automatic import management

    master

    To allow the decorator and restorer to automatically manage import blocks, you must provide implementations for two interfaces from the resolver package:

    1. DecoratorResolver: Resolves the package path of any *ast.Ident. This is necessary to handle dot-import syntax correctly.
    2. RestorerResolver: Resolves the name of a package given its path (handling vendoring and Go modules).

    Important: When setting a Resolver on a Decorator or Restorer, you must also set the Path property to the local package path.

    DecoratorResolver Implementations

    • gotypes: Provides full dot-import compatibility. Requires go/types.Info (specifically the Uses map). It is recommended to use golang.org/x/tools/go/packages.Load to generate this info. You can use decorator.Load to automate this.
    • goast: A simplified resolver that scans a single AST file. It cannot resolve identifiers from dot-imported packages and will panic if a dot-import is encountered. It uses a RestorerResolver to resolve package names.

    RestorerResolver Implementations

    • gopackages: Full compatibility with Go modules using golang.org/x/tools/go/packages. Note that this may be slow and requires the go CLI tool.
    • gobuild: Uses the legacy go/build system. It is faster than gopackages but is not Go modules aware.
    • guess: Guesses the package name based on the last part of the path.
    • simple: Resolves paths only if they exist in a provided map. Useful when performance is critical.
  6. Add comments to nodes using decoration attachment points

    master

    Comments are attached to specific points on a node. You can use the convenience methods Append, Prepend, Replace, Clear, and All on these attachment points. When adding a line comment, include the // or /**/ markers; a newline is automatically rendered for line comments. Common attachment points include Start, End, Fun, Lparen, Rparen, Lbrace, Rbrace, etc.

    call := f.Decls[0].(*dst.FuncDecl).Body.List[0].(*dst.ExprStmt).X.(*dst.CallExpr)
    
    call.Decs.Start.Append("// you can add comments at the start...")
    call.Decs.Fun.Append("/* ...in the middle... */")
    call.Decs.End.Append("// or at the end.")
  7. Manage imports automatically with NewDecoratorWithImports and NewRestorerWithImports

    master

    To automatically manage the import block, use decorator.NewDecoratorWithImports and decorator.NewRestorerWithImports.

    When using import management, you can add qualified identifiers by simply adding a *dst.Ident node and setting its Path field to the imported package path. The restorer will automatically wrap it in a *ast.SelectorExpr and update the import block as needed.

    // Adding an identifier that triggers an automatic import
    b.List = append(b.List, &dst.ExprStmt{
    	X: &dst.CallExpr{
    		Fun: &dst.Ident{Path: "fmt", Name: "Println"},
    		Args: []dst.Expr{
    			&dst.BasicLit{Kind: token.STRING, Value: strconv.Quote("Hello")},
    		},
    	},
    })
  8. Parse and print Go source code with decorator.Parse and decorator.Print

    master

    For simple tasks, use the decorator package's convenience functions. decorator.Parse converts source code into a dst.File, and decorator.Print renders a dst.File back into source code. For more control, use decorator.Decorator to convert from ast to dst and decorator.Restorer to convert back.

    code := `package main
    
    func main() {
    	println("Hello World!")
    }`
    f, err := decorator.Parse(code)
    if err != nil {
    	panic(err)
    }
    
    // ... modify f ...
    
    if err := decorator.Print(f); err != nil {
    	panic(err)
    }
  9. Core Node interfaces in dst

    master

    The dst package uses three primary interfaces to categorize nodes in the decorated syntax tree. All nodes implement the Node interface, which provides access to Decorations() (containing Before, After, Start, and End information).

    • Expr: Represents expression and type nodes.
    • Stmt: Represents statement nodes.
    • Decl: Represents declaration nodes.
  10. Statement nodes (Stmt)

    master

    The Stmt interface represents various Go statements. Common implementations include:

    • AssignStmt: Assignment (=) or short variable declaration (:=).
    • IfStmt: An if statement with Init, Cond, Body, and Else.
    • BlockStmt: A braced list of statements ({ ... }).
    • ReturnStmt: A return statement with Results.
    • ForStmt: A standard for loop with Init, Cond, Post, and Body.
    • RangeStmt: A for range loop.
    • ExprStmt: A standalone expression used as a statement.
  11. Declaration nodes (Decl)

    master

    The Decl interface represents top-level declarations.

    • GenDecl: A generic declaration for IMPORT, CONST, TYPE, or VAR. It can represent parenthesized declarations via the Lparen field.
    • FuncDecl: A function or method declaration, containing Recv (receiver), Name, Type (signature), and Body.
    • BadDecl: A placeholder for declarations with syntax errors.
  12. How Walk and Visitor work together

    master

    The Walk mechanism uses a recursive visitor pattern to navigate the tree. The key to controlling the flow is the return value of the Visit method.

    • Continue Traversal: Return the visitor itself (e.g., return v). This tells Walk to proceed to the children of the current node.
    • Stop Traversal: Return nil. This tells Walk to skip the children of the current node.

    This pattern is useful for implementing filters, search algorithms, or transformations where you might want to skip entire subtrees once a certain condition is met.