DOCSEARCHbyAlgolia
Submit Docs

Code-to-Knowledge-Graph Toolkit

repository·main·Indexed 41 days ago

https://github.com/bevel-software/code-to-knowledge-graph

A Kotlin/JVM toolkit that parses source code into a queryable knowledge graph using VS Code's Language Server Protocol (LSP). It enables developers to analyze code structure and impact through a rich graph model, supporting incremental updates, similarity detection via MinHashing, and custom analysis via AST-QL queries defined in YAML. The toolkit can be integrated via Gradle (version 1.1.3) or used through the Bevel VS Code extension.

Tokens
7.8K
Snippets
24
Records
40
Agent score
57%

What's inside Code-to-Knowledge-Graph

  1. Overview of Code-to-Knowledge-Graph features

    main

    Code-to-Knowledge-Graph is a Kotlin/JVM toolkit that transforms source code into a queryable knowledge graph by leveraging VS Code's Language Server Protocol (LSP).

    Key Capabilities:

    • VS Code LSP Integration: Uses existing language servers for robust multi-language parsing.
    • Rich Graph Model: Provides a comprehensive representation of code entities and their relationships.
    • Incremental Updates: Supports real-time graph updates as files change.
    • Smart File Discovery: Automatically respects .gitignore patterns.
    • Similarity Detection: Uses MinHashing for code similarity analysis.
    • Extensible Architecture: Designed for building custom analysis tools on top of the generated graph.

  2. Use Captures to Reference AST Nodes

    main

    Captures allow you to reference matched AST nodes in predicates and converters using the @ prefix.

    Named and Special Captures

    • Named Captures: Defined in the captures list (e.g., funcNode). Reference as @funcNode.
    • @parentNode: The first parent node in the AST traversal that has a node converter.
    • @current: The starting AST rule of the associated pattern.
    • @parentClassNode.node: The closest parent class node (returns the Node).
    • @parentSuperClassNode.node: The closest parent class's super class (returns the Node).

    Dot Notation Access

    You can access properties of captures using the . operator:

    • Parse Tree Access:
      • @capture.label: The source code text (label) represented by the node.
      • @capture.rule: The AST rule/type name.
      • @capture.hashCode: The hash code of the node (useful for identifying anonymous items like lambdas).
    • Node Builder Access:
      • @capture.node: Accesses the graph node properties (e.g., @capture.node.name).

    Syntax Rules

    • Concatenation: Use + and 'string' literals (e.g., @class.node + '.' + @method.label).
    • Optionality: Add a ? suffix for optional captures (e.g., @returnType?).
    # Example of concatenation and dot notation
nodeName: "@parentNode.node + '.' + @funcName"

# Example of accessing a property
rule: "@capture.rule"

  3. Define AST Patterns in YAML Queries

    main

    Queries are written in YAML to match AST patterns. A pattern defines how to traverse the AST and what to capture for later use in converters.

    Pattern Structure

    • rule: The AST rule/type to match.
    • captures: A list of names to reference the matched rules.
    • children: Required child patterns that must appear in a specific order.
      • optional: (boolean) If true, the child pattern is not strictly required.
    • notChildren: A list of rule types that must not be present.
    • alternatives: A list of patterns where any one of them can match.
    • predicates: Conditions that must be met for the pattern to be considered a match.
    • converters: Actions to perform (create nodes, connections, or imports) once a pattern matches.
    patterns:
  - pattern:
      rule: string
      captures:
        - captureName
      children:
        - rule: string
          captures:
            - childCaptureName
          optional: boolean
      notChildren:
        - excludedType
    predicates:
      - function: exists/equals/...
        capture: captureName
    converters:
      - function: node/connection/import

  4. Understand the Knowledge Graph Structure

    main

    The knowledge graph generated from AST queries consists of two primary components: Nodes and Connections.

    Node Types (NodeType)

    • FUNCTION: Functions, methods, and constructors.
    • CLASS: Classes and interfaces.
    • PACKAGE: Package or namespace declarations.
    • PROPERTY: Fields, properties, and variables.
    • ALIAS: Type aliases and imports.
    • OBJECT: Singleton objects or static contexts.

    Connection Types (ConnectionType)

    • DEFINES: Parent-child relationship (e.g., a class defining a method).
    • PARENT_OF: Ownership (e.g., a receiver type owning a method).
    • INHERITS_FROM: Inheritance relationships.
    • USES: Usage relationships (e.g., a variable using a type).
    • IS_OF_TYPE: Type relationships (e.g., return or parameter types).
    • INVOKED_BY: Method invocation relationships.
    • CALLED_WITH: Parameter passing relationships.

  7. Capture Classes and Interfaces

    main

    To capture class or interface declarations, match the declaration node and capture the name, constructor (if present), and superclasses/interfaces.

    Graph Connections to Create:

    • DEFINES connection from the package.
    • INHERITS_FROM connections to superclasses.
    • Create separate nodes and connections for constructors.

    YAML Pattern Example:

    type: classDeclaration
children:
  - type: identifier           # Class name
  - type: inheritanceClause
    children:
      - type: supertype       # Base class/interface

  9. Capture Function Overloading

    main

    To capture function overloading, match function declarations with the same name and capture their unique signatures (parameter types and return types).

    Graph Strategy:

    • Create unique nodes for each overload containing the specific parameters.
    • Create a node for the parameterless version of the function (if it exists) to serve as a reference point.
    • Add a DEFINES connection from the base parameterless function node to each overloaded function node.

    YAML Pattern Example:

    type: functionDeclaration
children:
  - type: identifier          # Function name
  - type: parameterList        # Parameter types for disambiguation
  - type: returnType
    optional: true

  10. Capture Variables and Properties

    main

    To capture variable or property declarations, match the declaration node and capture the name, type (if explicit), initial value, and modifiers/annotations.

    Graph Connections to Create:

    • Create a property node with a unique name (including parent context).
    • DEFINES connection from the parent scope.
    • IS_OF_TYPE connection to the type node.

    YAML Pattern Example:

    type: propertyDeclaration
children:
  - type: identifier      # Property name
  - type: type           # Property type (optional)
    optional: true
  - type: initializer    # Initial value (optional)
    optional: true

  11. Capture Annotations and Decorators

    main

    To capture metadata annotations, match the annotation/decorator nodes and capture the annotation name, parameters/arguments, and the target element.

    Graph Connections to Create:

    • ANNOTATES connections to the target element.

    YAML Pattern Example:

    type: annotation
children:
  - type: identifier          # Annotation name
  - type: argumentList        # Annotation parameters
    optional: true

  12. Capture Functions and Methods in Knowledge Graphs

    main

    To represent functions and methods in a knowledge graph, match the function declaration node and capture the function name, parameters, return type, and receiver type (for extension methods).

    Graph Connections to Create:

    • DEFINES connection from the parent scope.
    • IS_OF_TYPE connection to the return type node.
    • PARENT_OF connection from the receiver (specifically for extension methods).

    YAML Pattern Example:

    type: functionDeclaration
children:
  - type: identifier        # Function name
  - type: parameters       # Parameter list
  - type: returnType      # Return type