repository·main·Indexed 20 days ago
https://github.com/ruby-syntax-tree/syntax_treeA suite of tools built on the internal CRuby parser for generating, inspecting, and manipulating Ruby syntax trees. It provides a Ruby API and the `stree` CLI for building formatters, linters, and language servers. Key features include a Language Server Protocol (LSP) implementation, a visitor pattern for tree traversal, mutation capabilities via MutationVisitor, and support for custom plugins and Rake tasks.
SyntaxTree provides a Ruby API to access and manipulate the syntax tree of Ruby source code. You can read files with encoding awareness, parse source strings into trees, format code, search for specific patterns, and index declarations.
# Example of parsing and searching
program = SyntaxTree.parse("1 + 1")
SyntaxTree.search(program.source, "binary") do |node|
# do something with the matching node
endThe stree CLI can handle three types of input sources, which are queued for processing:
-e SCRIPT flag.When using -e or STDIN, the --extension flag determines how the content is parsed (defaulting to .rb).
When passing file lists to stree, it is recommended to wrap globs in quotes (e.g., '**/*.rb'). This ensures that Ruby handles the file path expansion using Ruby's Dir globbing syntax, providing consistent behavior across different shells and environments.
Example: Excluding specific files
To include all Ruby files except schema.rb in a Rails app:
stree write "**/{[!schema]*,*}.rb"stree write '**/*.rb'Since nodes are structured, you can use Ruby's built-in pattern matching to descend the tree with specific constraints. You can match against simple hashes or specific SyntaxTree class types.
program = SyntaxTree.parse("1 + 1")
# Minimal constraints
program => { statements: { body: [binary] } }
# Strict type constraints
program => SyntaxTree::Program[statements: SyntaxTree::Statements[body: [SyntaxTree::Binary => binary]]]The SyntaxTree::Visitor class implements the double dispatch visitor pattern, allowing you to operate on specific nodes without manually walking the entire tree. You define visit_* methods for the node types you are interested in.
When you define a handler for a node, you must decide how to continue the descent:
super to visit all child nodes using default behavior.visit_child_nodes manually.visit(child) for specific children.By default, SyntaxTree::Visitor walks the entire tree even if you don't define handlers for every node type.
class ArithmeticVisitor < SyntaxTree::Visitor
def visit_binary(node)
if node in { left: SyntaxTree::Int, operator: :+ | :- | :* | :/, right: SyntaxTree::Int }
puts "The result is: #{node.left.value.to_i.public_send(node.operator, node.right.value.to_i)}"
end
end
end
visitor = ArithmeticVisitor.new
visitor.visit(SyntaxTree.parse("1 + 1"))
# The result is: 2Syntax Tree provides the stree command-line interface for various tasks including AST inspection, code formatting, searching, and more.
stree ast [FILE]
Prints the Abstract Syntax Tree (AST) for the given files.stree check [FILE]
Checks if files are formatted according to Syntax Tree's rules. Returns non-zero if unformatted.stree ctags [FILE]
Generates a ctags-compatible index of the files.stree debug [FILE]
Verifies that formatting is idempotent (formatting a file twice results in the same output as formatting it once).stree doc [FILE]
Prints the doc tree used for formatting.stree expr [-e SCRIPT] [FILE]
Prints a pattern-matching Ruby expression that matches the first expression in the input.stree format [FILE]
Prints the formatted version of the input source to stdout.stree json [FILE]
Outputs the JSON representation of the source.stree match [FILE]
Prints a pattern-matching Ruby expression that matches the input.stree search PATTERN [FILE]
Searches for a specific pattern within the given files.stree write [FILE]
Reads, formats, and writes the formatted source back to the original file.stree version
Outputs the current version of Syntax Tree.stree lsp
Runs Syntax Tree in Language Server Protocol (LSP) mode.stree ast my_file.rb
stree format ./
stree search "/\w+/" lib/*.rb
stree write lib/my_file.rbSyntaxTree can be used as a standalone command-line interface or integrated into Ruby projects as a library.
To use the stree command globally, install the gem via terminal:
gem install syntax_treeTo use SyntaxTree within your own Ruby application, add it to your Gemfile and run bundle install:
gem "syntax_tree"gem "syntax_tree"To prevent Syntax Tree from formatting a specific section of source code, place the # stree-ignore comment immediately above the code block. This ensures the original formatting, including newlines and indentation, is preserved exactly as written.
# stree-ignore
numbers = [
10000,
20000,
30000
]To avoid rule conflicts between RuboCop and Syntax Tree, you can inherit Syntax Tree's configuration in your .rubocop.yml. This disables RuboCop rules that are redundant with Syntax Tree's formatting logic.
inherit_gem:
syntax_tree: config/rubocop.ymlSyntax Tree provides Rake tasks to trigger CLI commands (stree:check and stree:write) within your build workflow. Add the following to your Rakefile to enable them:
require "syntax_tree/rake_tasks"
SyntaxTree::Rake::CheckTask.new
SyntaxTree::Rake::WriteTask.newYou can customize tasks by passing arguments to .new or using a configuration block. Supported fields include:
name: Changes the default task name (e.g., SyntaxTree::Rake::WriteTask.new(:format)).source_files: Defines which files to process (defaults to lib/**/*.rb).ignore_files: A pattern used with File.fnmatch? to skip specific files.print_width: Sets the line width for formatting (defaults to 80).plugins: An array of plugin names to use.SyntaxTree::Rake::WriteTask.new do |t|
t.source_files = FileList[%w[Gemfile Rakefile lib/**/*.rb test/**/*.rb]]
t.ignore_files = "db/**/*.rb"
t.print_width = 100
t.plugins = ["plugin/single_quotes"]
endYou can extend Syntax Tree's behavior using a plugin system. To use plugins via the CLI, pass a comma-delimited list of plugin names to the --plugins option. Plugins must be defined in a file named syntax_tree/plugin_name within your Ruby load path.
Built-in plugins:
plugin/single_quotes: Forces string literals to use single quotes.plugin/trailing_comma: Adds trailing commas to multiline array literals, hash literals, and supported method calls.plugin/disable_auto_ternary: Prevents automatic conversion of if ... else blocks into ternary expressions.stree check --plugins=plugin/single_quotes,plugin/trailing_commaThe stree CLI provides several commands to inspect, search, and format Ruby code. Most commands accept file paths, content via STDIN, or inline scripts via the -e option.
ast: Prints a textual representation of the syntax tree.check: Validates that files match the expected format (useful for CI/CD).ctags: Outputs tags compatible with ctags.expr: Outputs a Ruby case-match expression for the first expression in the input.format: Outputs the formatted version of files to stdout (does not modify files).json: Outputs a JSON representation of the syntax tree.match: Outputs a Ruby case-match expression that matches the entire input.search: Searches files for nodes matching a Ruby pattern-matching expression.write: Formats files and overwrites the original source files.