go-tree-sitter

Due to known issues with runtime.SetFinalizer and CGO, you must explicitly call the Close() method on any object that allocates memory from C. Failure to do so will result in memory leaks.

The following types require manual closing:

• Parser
• Tree
• TreeCursor
• Query
• QueryCursor
• LookaheadIterator

To use the Tree-sitter Go bindings in your project, install the package using go get:

go get github.com/tree-sitter/go-tree-sitter@latest

Note that by default, this package does not include any language grammars. You must fetch specific grammars separately (e.g., go get github.com/tree-sitter/tree-sitter-javascript@latest) or load them at runtime.

Instead of importing grammars as Go dependencies, you can load them from a shared library (.so file) at runtime using purego. This is useful for loading parsers dynamically on Linux and macOS.

1. Open the shared library using purego.Dlopen.
2. Register the language function (e.g., tree_sitter_javascript) using purego.RegisterLibFunc.
3. Wrap the resulting pointer in tree_sitter.NewLanguage().

package main

import (
	tree_sitter "github.com/tree-sitter/go-tree-sitter"
	"github.com/ebitengine/purego"
	"unsafe"
)

func main() {
	path := "/path/to/your/parser.so"
	lib, err := purego.Dlopen(path, purego.RTLD_NOW|purego.RTLD_GLOBAL)
	if err != nil {
        // handle error
    }

	var javascriptLanguage func() uintptr
	purego.RegisterLibFunc(&javascriptLanguage, lib, "tree_sitter_javascript")

	language := tree_sitter.NewLanguage(unsafe.Pointer(javascriptLanguage()))
}

If you need to incorporate changes from the upstream Unicode ICU repository, follow these steps:

1. Update ICU_SHA with the new Git SHA from the upstream repository.
2. Update the LICENSE file with the current license text from the icu4c directory in the upstream repository.
3. Replace the contents of utf8.h, utf16.h, and umachine.h with the new versions from the upstream repository.

To generate completion suggestions or valid symbols in error nodes, use the LookaheadIterator and NextState methods.

1. LookaheadIterator(state uint16) *LookaheadIterator: Creates a new iterator for a specific parse state. Returns nil if the state is invalid. Newly created iterators start with an ERROR symbol.
2. NextState(state uint16, id uint16) uint16: Calculates the next parse state given the current state and a node ID. This can be combined with a LookaheadIterator to traverse valid paths.

Usage Tip: To find valid symbols in an ERROR node, use the lookahead iterator on its first leaf node state. For MISSING nodes, use the iterator on the previous non-extra leaf node.

Tree-sitter uses a bridge between Go and the C core to manage memory. When you call SetAllocator, the library stores your Go functions in atomic values and passes function pointers to the C core via ts_set_allocator.

When the C core needs to allocate or free memory, it calls exported Go functions (go_malloc, go_calloc, go_realloc, and go_free). These exported functions check if a custom allocator has been registered; if so, they execute your Go function. If not, they fall back to the standard C library functions.

A TreeCursor is a stateful object used to walk a syntax Tree efficiently with low allocations. It allows you to navigate the tree structure by moving to parents, children, and siblings.

Important Lifecycle Note: Because TreeCursor manages C memory, you must call Close() when you are finished with it to prevent memory leaks.

// Assuming 'tree' is a valid *tree_sitter.Tree
cursor := tree.Cursor() // Or via a node
defer cursor.Close()

if cursor.GotoFirstChild() {
    node := cursor.Node()
    // Do something with the node
}

While Node provides high-level methods like Child(i) for quick access, these can be expensive if used in loops because they often involve $O(\log i)$ lookups.

For high-performance tree walking, use a TreeCursor. A TreeCursor is a stateful object that can be reset to a specific Node via cursor.Reset(node) and then moved through the tree using GotoFirstChild(), GotoNextSibling(), GotoParent(), etc.

When using Node.Children(cursor), the method internally resets the cursor to the current node and iterates through its children. To minimize allocations, you should create one TreeCursor and pass it into multiple Node iteration methods rather than creating a new cursor for every node.

Captures in a query can have different quantifiers, determining how many times a node can match a pattern. The CaptureQuantifier type includes:

• CaptureQuantifierZero (0)
• CaptureQuantifierZeroOrOne (0 or 1)
• CaptureQuantifierZeroOrMore (0+)
• CaptureQuantifierOne (1)
• CaptureQuantifierOneOrMore (1+)

The Parser is a stateful object used to produce a Tree from source code.

To use a parser, create a new instance with NewParser(), assign a language using SetLanguage(l *Language), and ensure you call Close() when finished to release C memory.

If you intend to parse a completely different document using the same parser instance, call Reset() to instruct the parser to start from the beginning instead of resuming from a previous interrupted state.

parser := tree_sitter.NewParser()
defer parser.Close()

err := parser.SetLanguage(myLanguage)
if err != nil {
	// handle error
}

tree := parser.Parse(sourceBytes, nil)