node-tree-sitter

Tree-sitter requires a specific grammar for the language you intend to parse. You can install existing grammars (like tree-sitter-javascript) via npm. If a grammar is not available, you can develop one using the Tree-sitter CLI.

npm install tree-sitter-javascript

To use the Node.js bindings for the tree-sitter parsing library, install the tree-sitter package using npm.

npm install tree-sitter

To parse source code, initialize a Parser instance, set its language using a grammar module, and call .parse() with the source string. The resulting tree allows you to inspect the syntax tree via its nodes.

const Parser = require('tree-sitter');
const JavaScript = require('tree-sitter-javascript');

const parser = new Parser();
parser.setLanguage(JavaScript);

const sourceCode = 'let x = 1; console.log(x);';
const tree = parser.parse(sourceCode);

// Inspect the root node
console.log(tree.rootNode.toString());

// Access specific nodes
const callExpression = tree.rootNode.child(1).firstChild;
console.log(callExpression);

If your source text is not a single string (e.g., it is stored in a rope or an array of lines), you can provide a callback function to parser.parse() instead of a string. The callback receives (index, position) and should return the text slice starting from the requested position.

const sourceLines = [
  'let x = 1;',
  'console.log(x);'
];

const tree = parser.parse((index, position) => {
  let line = sourceLines[position.row];
  if (line) {
    return line.slice(position.column);
  }
});

When source code changes, you can update an existing syntax tree instead of performing a full re-parse. This is significantly faster. Use tree.edit() to describe the change by providing the start/end indices and positions for both the old and new source segments. Note that end indices are exclusive.

// Example: replacing 'let' (index 0-3) with 'const' (index 0-5)
const newSourceCode = 'const x = 1; console.log(x);';

tree.edit({
  startIndex: 0,
  oldEndIndex: 3,
  newEndIndex: 5,
  startPosition: {row: 0, column: 0},
  oldEndPosition: {row: 0, column: 3},
  newEndPosition: {row: 0, column: 5},
});

// Re-parse using the existing tree as a base
const newTree = parser.parse(newSourceCode, tree);

To perform incremental parsing, you must notify the Tree about changes made to the source text using the edit method. This method takes an object describing the change.

tree.edit({
  startPosition: { row: 1, column: 0 },
  oldEndPosition: { row: 1, column: 5 },
  newEndPosition: { row: 1, column: 10 },
  startIndex: 10,
  oldEndIndex: 15,
  newEndIndex: 20
});

The Parser class allows fine-grained control over the parsing process to handle edge cases or performance constraints:

• Timeouts: To prevent the parser from hanging on malicious or overly complex input, use setTimeoutMicros(micros) to set a limit in microseconds. You can retrieve the current limit using timeoutMicros().
• Included Ranges: If you only want to parse specific segments of a file, use setIncludedRanges(ranges) to provide a list of ranges. You can check the currently active ranges with includedRanges().
• Resetting: Use reset() to clear the parser state.

A TreeCursor provides a high-performance way to traverse the tree without the overhead of creating many SyntaxNode objects. It allows you to move through the tree structure step-by-step.

Key Properties:

• currentNode: Returns the SyntaxNode at the current cursor position.
• startPosition / endPosition: The coordinates of the current node.
• nodeText: The text content of the current node.

Usage Pattern: You can obtain a cursor from a node using node.walk() or from a tree using tree.walk().

The Parser class is the main entry point for generating syntax trees.

• setLanguage(language): Sets the language for the parser. This also initializes language-specific SyntaxNode subclasses.
• getLanguage(): Returns the currently set language.
• parse(input, oldTree, options): Parses the provided input.
  • input: Can be a string or a function (offset, position) => string.
  • oldTree: (Optional) An existing Tree instance used for incremental parsing.
  • options: An object containing bufferSize, includedRanges, and progressCallback.

The Query class allows you to search for patterns within a syntax tree. You can use .matches() to find patterns or .captures() to retrieve specific named captures.

Query Methods:

• matches(node, options): Returns an array of match results. Each result contains the pattern index and the captures associated with it.
• captures(node, options): Returns the specific capture for a pattern match.

Common Query Options:

• startPosition / endPosition: The range within the tree to search.
• startIndex / endIndex: The byte range to search.
• matchLimit: Maximum number of matches to return.
• timeoutMicros: Timeout for the query execution in microseconds.