Globstar Documentation

repository·master·Indexed 19 days ago

https://github.com/deepsourcecorp/globstar

An open-source, high-performance static analysis toolkit for writing custom code checkers using tree-sitter queries via YAML or Go. Distributed as a dependency-free binary, Globstar integrates into local development and CI/CD pipelines, featuring a CLI for running checks and testing custom patterns.

Tokens
14.8K
Snippets
42
Records
64
Agent score
64%

What's inside Globstar

  1. What is Globstar?

    master

    Globstar is a static analysis toolkit designed for writing and running custom checkers to detect code quality, security, or other specific issues in a codebase.

    It supports two levels of checker complexity:

    1. YAML-based checkers: Located in the .globstar folder of your repository, these use tree-sitter's S-expression syntax to match patterns.
    2. Go-based checkers: For advanced logic, you can use the Go API to write complex code with full access to the tree-sitter AST, including features like import resolution, scope analysis, and cross-file references.
  2. Overview of Globstar

    master
    Globstar is an open-source Static Analysis Security Testing (SAST) toolkit designed for speed and ease of use. It allows developers to write checkers using simple YAML files and tree-sitter S-expressions, or use the Go API for advanced analysis. It is distributed as a single binary, making it easy to integrate into CI pipelines without managing complex dependencies.
  3. Write Tree-sitter patterns for checkers

    master

    Patterns use Tree-sitter query syntax to match AST nodes.

    Predicates

    You can use predicates to match specific values within a node:

    • #eq?: Exact string match.
    • #match?: Regex pattern match.
    • #not-eq?: String doesn't match.
    • #not-match?: Regex pattern doesn't match.

    Example: Using Predicates and Captures

    To match a specific method call and use its name in the error message:

    message: "Found console.@method call"
    pattern: |
      (
        member_expression
          object: (identifier) @console (#eq? @console "console")
          property: (property_identifier) @method
      )
    # Match only when identifier is "console" and property is "log"
    pattern: |
      (
        call_expression
          (member_expression
            object: (identifier) @console (#eq? @console "console")
            property: (property_identifier) @method (#eq? @method "log"))
      ) @js_no_console_log
  4. Understand tree-sitter patterns in Globstar

    master

    Globstar uses tree-sitter's S-expression syntax to match code patterns based on the Abstract Syntax Tree (AST).

    Core Concepts

    • Tree Structure: Code is represented as a tree of nodes (e.g., call, identifier, assignment).
    • Pattern Matching: You describe the desired structure. For example, (call function: (identifier) @func) matches a function call where the function name is captured as @func.
    • Predicates: Special functions used to add logic to a match. The #eq? predicate checks for equality, such as (#eq? @func "eval") to ensure the captured identifier is exactly "eval".
    • Captures: Using the @name syntax allows you to name specific nodes in the tree for use in predicates or to mark the entire matched block.

    Tip: Use the tree-sitter playground to visualize how your code is parsed into a tree before writing your YAML pattern.

  5. Advanced Go Checker capabilities

    master

    Go checkers provide capabilities that are not possible with YAML-based checkers:

    1. State Tracking

    You can maintain state (like maps or booleans) across the entire analysis pass to track context, such as whether you are currently inside a specific function or a 'safe' wrapper.

    2. Multi-pass Analysis

    You can perform multiple traversals of the AST. For example, a first pass can identify 'tainted' variables (e.g., variables assigned from user input), and a second pass can check if those variables are used in dangerous functions.

    3. Scope and Type Information

    If the language supports it, you can access pass.FileContext.ScopeTree to perform variable lookups and check if an identifier refers to a function parameter or a local variable. For statically-typed languages, you can leverage pass.TypeInfo to inspect the types of expressions.

  6. Identify security patterns for new checkers

    master

    Globstar focuses on security-related checkers. Use the following categories and resources to find high-impact patterns:

    Common Security Patterns

    • Input Validation: Dangerous function calls (eval, exec), command injection, unsafe deserialization.
    • Authentication & Authorization: Hardcoded credentials, weak cryptography, insecure session management.
    • Data Protection: Unencrypted sensitive data, insecure random number generation, weak hashing.
    • Code Injection: SQL injection, XSS, template injection.
    • System Security: Unsafe file operations, insecure permissions, uncontrolled resource consumption.

    Prioritization Criteria

    When choosing a pattern to implement, prioritize based on:

    1. Impact: Severity of security implications and commonality of the vulnerability.
    2. False Positives: Ability to reliably detect the pattern with minimal legitimate exclusions.
    3. Scope: Relevance across multiple frameworks or large/small codebases.
  7. Implement advanced analysis: State, Context, and Scopes

    master

    For more sophisticated checkers, you can use these patterns:

    State Tracking

    Maintain state across the analysis by declaring variables within your Run function and updating them during Preorder traversals.

    Context Awareness

    Track state (e.g., inSafeContext bool) during traversal to determine if a pattern is occurring within a specific block (like a specific function name).

    Scope Analysis

    If pass.FileContext.ScopeTree is not nil, you can perform variable lookups:

    scope := pass.FileContext.ScopeTree.GetScope(node)
    if scope != nil {
        variable := scope.Lookup("someVariable")
        if variable != nil && variable.Kind == analysis.VarKindParameter {
            // Handle parameter
        }
    }
  8. How Globstar checkers work

    master

    Globstar operates by scanning the .globstar directory in your repository for checker definitions.

    • Pattern Matching: Instead of a custom DSL, Globstar uses native tree-sitter S-expressions. This allows you to map rules directly to the code's Abstract Syntax Tree (AST).
    • Execution: You run the checkers against your codebase using the command globstar check.
    • Advanced Analysis: When using the Go API, checkers can perform sophisticated tasks such as resolving imports and scopes, which are not possible with simple pattern matching.
    globstar check
  9. Choose between YAML and Go interfaces for writing checkers

    master

    Globstar provides two primary ways to define code checkers depending on your complexity requirements:

    1. YAML Interface: Best for lightweight, single-file analysis. It is easy to write and maintain and is suitable for most common use cases across 20+ supported programming languages.
    2. Go Interface: Best for sophisticated checkers that require advanced capabilities such as multi-file analysis, scope resolution, and context awareness.

    Refer to the /reference/checker-yaml and /reference/checker-go documentation for specific implementation details.

  10. Key Features of Globstar

    master

    Globstar provides several capabilities for static analysis:

    • Fast Execution: Built with Go and using native tree-sitter bindings for high-performance parsing.
    • YAML-based Checkers: Write checkers using simple YAML files and tree-sitter S-expressions instead of learning a custom DSL.
    • Simple CI Integration: Runs as a single binary with no external dependencies required in your CI environment.
    • Advanced Go API: For complex requirements, use the Go API to access the full tree-sitter AST, imports, scope resolution, and cross-file analysis.
    • Built-in Checkers: Includes pre-written checkers for common security vulnerabilities and code quality issues.
  11. Test your checkers

    master

    To verify checker behavior, create a test file with the same name as the checker but with a .test suffix and the appropriate file extension (e.g., no_console_log.yml -> no_console_log.test.js).

    Using the <expect-error> directive

    Place the // <expect-error> comment directly above a line of code that you expect the checker to flag. If the checker does not flag that line, the test fails.

    Running tests

    Execute the following command to run all checker tests in the .globstar directory:

    # Runs all tests and exits with status 1 if any fail
    globstar test
    // .globstar/no_console_log.test.js
    
    function test() {
      // This should be caught inside a function
      // <expect-error>
      console.log("inside function");
    
      try {
        something();
      } catch (err) {
        // This should NOT be caught (pattern-not-inside: catch_clause)
        console.log(err);
      }
    }
  12. Install Globstar

    master

    You can install Globstar by running the installation script via curl. By default, it downloads the binary to ./bin/globstar in your current directory.

    To install to a specific directory, set the BINDIR environment variable.

    To install Globstar globally, move the downloaded binary to a directory in your PATH (e.g., /usr/local/bin).

    # Default installation to ./bin/globstar
    curl -sSL https://get.globstar.dev | sh
    
    # Install to a custom directory
    curl -sSL https://get.globstar.dev | BINDIR=$HOME/.local/bin sh
    
    # Global installation
    mv ./bin/globstar /usr/local/bin