tree-sitter Python Bindings

repository·master·Indexed 23 days ago

https://github.com/tree-sitter/py-tree-sitter

Python bindings for the Tree-sitter parsing library (version 0.26.0), providing high-performance incremental parsing and syntax tree manipulation. The library includes classes for managing Language grammars, Parser configurations, Node navigation, and Tree traversal via TreeCursor. It supports synchronizing trees with source edits, pattern searching using Query and QueryCursor, and predictive parsing with LookaheadIterator.

Tokens
6.1K
Snippets
9
Records
44
Agent score
81%

What's inside tree-sitter

  1. Use tree_sitter.Query for pattern matching

    master

    The tree_sitter.Query class is used to perform pattern matching on syntax trees using Tree-sitter's query syntax. It allows you to find specific nodes and capture their data based on defined patterns and predicates.

    By default, the following predicates are supported:

    • Equality: #eq?, #not-eq?, #any-eq?, #any-not-eq?
    • Regex matching: #match?, #not-match?, #any-match?, #any-not-match?
    • Set membership: #any-of?, #not-any-of?
    • Type/Identity: #is?, #is-not?
    • Set definition: #set!

    For detailed syntax rules, refer to the Tree-sitter Query Syntax documentation.

  2. Use QueryCursor to iterate over query matches

    master
    The QueryCursor class is used to execute queries against a syntax tree and iterate through the resulting matches. You can use the captures method to retrieve all captures for a match, or matches to iterate through the matches themselves. To optimize performance or limit the scope of a search, you can restrict the cursor to specific byte ranges or point ranges using set_byte_range, set_point_range, set_containing_byte_range, or set_containing_point_range.
  3. Understand Tree-sitter ABI version compatibility

    master

    The py-tree-sitter library uses ABI (Application Binary Interface) versions to ensure compatibility between the library and the language grammars generated by the Tree-sitter CLI.

    • tree_sitter.LANGUAGE_VERSION: The latest ABI version supported by the current library version.
    • tree_sitter.MIN_COMPATIBLE_LANGUAGE_VERSION: The earliest ABI version supported by the current library version.

    Compatibility Rules:

    • The library is generally backwards-compatible with languages generated using older CLI versions.
    • The library is not forwards-compatible (you cannot use a language generated by a newer CLI version than the library supports).
  4. Set up a language for parsing

    master

    To parse a specific language, install its corresponding tree-sitter language package (e.g., tree-sitter-python) and load it using the Language class. This creates a Language object that can be passed to a Parser.

    import tree_sitter_python as tspython
    from tree_sitter import Language, Parser
    
    PY_LANGUAGE = Language(tspython.language())
  5. Search for patterns using Query and QueryCursor

    master

    Use Query to define patterns in a syntax tree using a query language. To execute the query, use a QueryCursor.

    There are two ways to retrieve results:

    1. QueryCursor.captures(node): Returns a dictionary where keys are capture names and values are lists of nodes. This is useful for getting all nodes tagged with a specific name.
    2. QueryCursor.matches(node): Returns a list of matches. Each match is a tuple containing the match index and a dictionary of captures. This is preferred when captures within a query are related (e.g., a function name and its body) and you want to process them as a single unit.
    query = Query(
        PY_LANGUAGE,
        """
        (function_definition
          name: (identifier) @function.def
          body: (block) @function.block)
        """
    )
    
    query_cursor = QueryCursor(query)
    
    # Get all captures
    captures = query_cursor.captures(tree.root_node)
    # captures['function.def'] -> [node, ...]
    
    # Get grouped matches
    matches = query_cursor.matches(tree.root_node)
    # matches[0][1]['function.def'] -> [node, ...]
  6. Use the tree_sitter.Node class to navigate syntax trees

    master
    The tree_sitter.Node class represents a node in a syntax tree. It provides a comprehensive API for traversing the tree structure, accessing source text, and inspecting node properties. You can navigate the tree using parent/child relationships, find descendants within specific byte or point ranges, and access nodes by their field names or IDs.