Rubydex

repository·main·Indexed 42 days ago

https://github.com/shopify/rubydex

A high-performance static analysis toolkit for Ruby, version 0.2.6, providing Ruby and Rust APIs for deep semantic understanding of codebases. It transforms source code into a semantic graph via indexing and resolution phases, serving as a foundation for type checkers, linters, and language servers. Features include a Cypher-based query language via CLI and Ruby, support for Ractor-safe graph sharing, and an MCP server implementation for AI assistants.

Tokens
48.4K
Snippets
182
Records
221
Agent score
55%

What's inside Rubydex

  1. Use rubydex-sys for C FFI integration

    main

    rubydex-sys provides a C FFI (Foreign Function Interface) boundary for the Rubydex Rust library. It is designed to build static and dynamic libraries and generate C bindings.

    Note: This crate is intended for low-level integration, specifically for the Rubydex Ruby native extension. Most developers should use the high-level rubydex Ruby gem or the rubydex Rust crate instead of interacting with rubydex-sys directly.

  2. Compare extend vs include in singleton classes

    main

    While extend Foo and class << self; include Foo; end are functionally equivalent (both add methods to the singleton class), they trigger different lifecycle hooks:

    • extend triggers the extended hook.
    • include inside a singleton class triggers the included hook.
    module Foo
      def self.included(base)
        puts "included hook called on #{base}"
      end
    
      def self.extended(base)
        puts "extended hook called on #{base}"
      end
    end
    
    class UsingExtend
      extend Foo  # Triggers: "extended hook called on UsingExtend"
    end
    
    class UsingSingletonInclude
      class << self
        include Foo  # Triggers: "included hook called on #<Class:UsingSingletonInclude>"
      end
    end
  3. Understand Ruby mixin deduplication

    main

    Ruby automatically removes duplicate modules from the ancestor chain to ensure a module appears only once, even if it is included multiple times through different paths (direct or indirect).

    module A; end
    module B; include A; end
    module C; include A; end
    
    class Foo
      include B
      include C
    end
    
    Foo.ancestors  # => [Foo, C, B, A, Object, ...]
    # A appears only once despite being in both B and C
  4. Understand Rubydex Skill Library (Experimental)

    main

    Rubydex includes an experimental built-in skill library designed for deterministic skill loading for AI agents.

    Instead of agents loading massive amounts of documentation, Rubydex tools return a specific skill ID (e.g., skills/send-private-method/SKILL.md) when a rule is violated. The agent can then fetch only that specific skill using the command:

    rdx skill <id>

    This pattern ensures that agents only load relevant information when a verified violation occurs, reducing noise and improving efficiency.

  5. Best practices for custom Rubydex rules

    main

    To ensure high performance and a good user experience when writing custom rules, follow these guidelines:

    1. Minimize Diagnostics: One problem in the code should equal one diagnostic. Use related_information to provide context rather than emitting multiple separate errors for the same issue.
    2. Optimize Graph Queries: Avoid looping through the entire graph (e.g., graph.declarations.select { ... }). Instead, use highly specific queries or the search API (e.g., graph["Foo"].descendants or graph.search("#method_name()")).
    3. Avoid Disk I/O: Do not read files from disk within a rule. Rules should operate exclusively on the provided Rubydex::Graph to avoid performance hits and stale data issues.
    4. Prefer Generic Rules: Instead of writing many highly specific rules, create generic, configurable rules (e.g., a RequiredParentClass rule instead of separate rules for models and controllers).
  6. Understand anonymous classes and modules via Class.new and Module.new

    main

    In Ruby, Class.new and Module.new create anonymous class or module objects. While these objects are anonymous, they can be assigned to a constant to give them a name in a namespace. Inside the block provided to these methods, self refers to the newly created class or module.

    Method definitions

    • Class methods: Defined using def self.method_name.
    • Instance methods: Defined using def method_name.

    You can interact with the object even if it is not assigned to a constant by capturing the return value in a variable.

    Foo = Class.new do
      def self.foo; end   # class method on Foo
      def bar; end        # instance method on Foo instances
      alias_method :baz, :bar
    end
    
    # Accessing without a constant assignment
    c = Class.new do
      def self.foo; end
      def bar; end
    end
    
    c.foo
    c.new.bar
  7. Share a Rubydex::Graph across Ractors safely

    main

    A resolved Rubydex::Graph can be shared across Ractors without copying. This is achieved through an RwLock on the Rust side rather than Ruby's freeze mechanism.

    Important considerations:

    • Mutation is allowed on frozen graphs: Methods like index_source, resolve, exclude_patterns, and encoding= work even if the graph has been made shareable. This allows for incremental updates (e.g., in an LSP) while worker Ractors are reading.
    • Concurrency Responsibility: Because the underlying Rust allocation is shared, the caller is responsible for managing the order of concurrent writes and reads.
    • No Cloning: You cannot use dup or clone on a graph; doing so will raise a RuntimeError to prevent memory issues or excessive memory usage.
    graph = Rubydex::Graph.new
    graph.index_workspace
    graph.resolve
    Ractor.make_shareable(graph)
    
    # Worker Ractors can now read the graph in parallel
    ractor = Ractor.new(graph) { |g| g["Foo"]&.name }
    ractor.value
  8. Core model: Definitions, Declarations, and Documents

    main

    The Rubydex graph is composed of three primary hierarchical abstractions:

    • Document: Represents a file (whether committed to disk or not). Produced during Indexing.
    • Definition: Represents a single source-level construct found at a specific location in the code (e.g., a specific class keyword usage in a file). It captures raw data without semantic meaning. Produced during Indexing.
    • Declaration: Represents the global semantic concept of an entity. It combines all Definitions that contribute to the same fully qualified name. Produced during Resolution.

    Relationship Hierarchy

    Document (1) $\rightarrow$ many Definitions $\rightarrow$ one Declaration (per fully qualified name).

    Graph Entry Points

    • Bottom-up: Use a UriId (derived from a string URI) to find all declarations defined within a specific Document.
    • Top-down: Use a DeclarationId (derived from a fully qualified name) to access data about a specific semantic entity directly.
    ┌─────────────┐
        | Declaration |  one per fully qualified name
        └─────────────┘
               ▲
               |   many definitions that contribute to the same fully qualified name -> one declaration
               ▼
        ┌─────────────┐
        | Definition  |   a single source-level construct
        └─────────────┘
               ▲
               |   one document -> the many definitions found inside it
               ▼
        ┌─────────────┐
        |  Document   | a file
        └─────────────┘
  9. Understand the Rubydex analysis phases: Indexing and Resolution

    main

    Rubydex operates in two primary phases to transform raw source code into a semantic graph:

    1. Indexing: This phase traverses ASTs (from Ruby or RBS) to extract raw information exactly as found. It is document-specific and does not attempt to derive semantic meaning or cross-file relationships. Because Ruby's file loading order is unpredictable (due to autoload or runtime require), indexing alone cannot resolve constant references that depend on other files.

    2. Resolution: This phase takes the indexed data and builds a semantic representation. It performs tasks such as:

      • Computing fully qualified names (e.g., Foo::Bar, Foo#method(), Bar#@ivar).
      • Resolving constant references to their target declarations.
      • Linearizing ancestor chains and tracking descendants.
      • Assigning semantic ownership (linking methods/constants to their classes).
      • Creating implicit singleton classes.

    Resolution uses a worklist (unit queue) loop that iterates until it reaches a fixed-point (convergence), meaning no further progress can be made in resolving constants or ancestor chains.

  10. Understand visibility behavior for aliases

    main

    Both alias and alias_method copy the source method's visibility at the time of the alias operation, regardless of the current visibility stack.

    If you alias a public method while inside a private block, the new alias will still be public because it inherits the visibility of the original method.

    class Foo
      def pub; end    # public
      private
      alias copy pub  # copy is PUBLIC (copies source, ignores stack)
    end
  11. How Ruby constant aliases work

    main

    Constants can be assigned to reference other constants, creating aliases. These can be chained or scoped within namespaces.

    • Chained Aliases: An alias can point to another alias, which eventually points to the original constant.
    • Scoped Aliases: An alias can be defined inside a module, allowing access via a qualified path.
    • Defining under Aliases: You can define a class or module under an aliased path. The resulting class/module will have its fully qualified name based on the alias target, not the alias itself.
    • Conditional Aliases: Use ||= to only assign an alias if it is not already defined.
    module Foo; end
    module Bar; end
    
    ALIAS = Foo
    
    class ALIAS::Bar; end
    ALIAS::Bar.name # => "Foo::Bar", not "ALIAS::Bar"
  12. How singleton classes work in Ruby

    main

    A singleton class is a unique class created for a specific object to hold its singleton methods and instance variables.

    Key Behaviors:

    • Lazy Materialization: Singleton classes are usually created lazily when singleton-specific syntax (like def self.*, class << self, or extend) is used, or when obj.singleton_class is called.
    • Receiver of Singleton Methods: The singleton class is the receiver for methods defined via def self.*, class << self, and modules mixed in via extend.
    • Lexical Scope vs. Singleton Receiver: Inside a class << self block, self is the singleton class. However, lexical constant lookup still follows the surrounding constant scope. Constants defined inside a class << self block do not become part of the original class's namespace.
    • Nested Singleton Classes: Using class << self inside another class << self block opens the singleton class of the current self (the singleton class of the singleton class).

    Important Distinctions:

    • Class Variables (@@var): These bypass singleton classes entirely and belong to the base class or module.
    • Instance Variables (@var): When defined in the class body, these belong to the class's singleton class.
    • Immutable Objects: Integers, floats, symbols, and frozen strings cannot have singleton classes; calling singleton_class on them raises a TypeError.
    class Foo
      class << self # reopens (or creates) Foo's singleton class
        A = 1
        def bar; A; end
      end
    
      def self.baz; end # adds the method to Foo's singleton class
    
      @ivar = 1 # assigns a class instance variable on Foo's singleton class (NOT a class variable)
    end