RefactoringMiner Documentation

repository·master·Indexed 19 days ago

https://github.com/tsantalis/refactoringminer

A Java library and API for detecting code refactorings in project history and generating high-fidelity AST-based diffs. It supports Java, Python, Kotlin, TypeScript, and JavaScript, providing capabilities for refactoring detection, cross-language diffs, and visualization via WebDiff. The tool includes a Model Context Protocol (MCP) server for AI agent integration and can be deployed via Docker as a standalone tool or a Git difftool.

Tokens
16.3K
Snippets
35
Records
54
Agent score
63%

What's inside RefactoringMiner

  1. Overview of RefactoringMiner

    master

    RefactoringMiner is a Java-based library and API designed to detect refactorings applied throughout the history of a project.

    Key capabilities include:

    • Refactoring Detection: Identifying code changes like method extractions, moves, or renames.
    • AST Diff Generation: Since version 3.0, it can generate Abstract Syntax Tree (AST) diffs at the commit, pull request, and commit range levels.
    • Cross-Language Support: Supports detecting refactorings and generating diffs for Java, Python, Kotlin, TypeScript, and JavaScript (with C++ on the roadmap).
    • Visualization: Tools to visualize diffs in a browser, including refactoring-aware tooltips, single-page views, and support for code moved between different files.
  2. AST Diff Features

    master

    The AST diff engine provides several advanced features for code review and analysis:

    • Cross-language diff: Visualizes migrations between languages (e.g., Java methods migrated to Kotlin functions).
    • Refactoring-aware tooltips: Provides context about the nature of the change during diff viewing.
    • Refactoring listing: Provides a list of detected refactorings with direct links to their corresponding diffs.
    • Single Page View: Loads all AST diffs in a single view, similar to the GitHub interface.
    • Embedded GitHub Code Review Comments: Integrates with existing review workflows.
    • Move Diff: Handles diffs for code that has been moved between different files, including overlapping refactorings within the moved code.
    • On-demand diff generation: Allows users to select any pair of files (modified, added, or deleted) to generate a specific diff.
    • Javadoc and comment reformatting: Matches Javadoc and inline comments with formatting changes. You can exclude diffs that only include comment formatting changes by using the --ignore-formatting command-line option.
  3. Use the RefactoringMiner MCP server tools

    master

    The RefactoringMiner MCP server exposes three read-only tools grouped by task. Each tool accepts a source object, allowing agents to specify the target of analysis without selecting between duplicate tools.

    Available Tools:

    • refactoringminer_analyze: Performs refactoring analysis.
    • refactoringminer_validate: Validates refactorings.
    • refactoringminer_diff: Generates an AST diff.

    The source object: Agents can provide an explicit source.type or allow the server to infer it from fields such as:

    • pullRequestId
    • url
    • commitId
    • beforeFiles
    • beforePath
  4. Generate AST Diffs

    master

    RefactoringMiner provides advanced AST diff capabilities that support multi-mappings (one-to-many, many-to-one, many-to-many) and semantic diffs in a refactoring-aware fashion.

    All AST Diff APIs return a ProjectASTDiff object. You can call .getDiffSet() on this object to obtain a Set<ASTDiff>, where each ASTDiff corresponds to a pair of Java Compilation Units. ASTDiff extends com.github.gumtreediff.actions.Diff, making it compatible with the GumTree core APIs.

    To visualize the diffs, you can use the WebDiff class.

    // Example: AST Diff with a locally cloned repository
    GitService gitService = new GitServiceImpl();
    GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
    
    Repository repo = gitService.cloneIfNotExists(
        "tmp/refactoring-toy-example",
        "https://github.com/danilofes/refactoring-toy-example.git");
    
    ProjectASTDiff projectASTDiff = miner.diffAtCommit(repo,
        "36287f7c3b09eff78395267a3ac0d7da067863fd");
    Set<ASTDiff> diffs = projectASTDiff.getDiffSet();
    
    // To visualize the diff
    new WebDiff(projectASTDiff).run();
  5. JSON output format for refactorings

    master

    When using the -json flag, RefactoringMiner produces a JSON object containing a list of commits. Each commit includes the repository details, the SHA1, the URL, and a list of detected refactorings.

    Each refactoring entry contains:

    • type: The name of the refactoring (e.g., "Pull Up Attribute").
    • description: A human-readable description of the change.
    • leftSideLocations: An array of locations where the element existed before the refactoring.
    • rightSideLocations: An array of locations where the element exists after the refactoring.

    Location objects include filePath, startLine, endLine, startColumn, endColumn, codeElementType, description, and the codeElement string.

    {
      "commits": [{
        "repository": "https://github.com/example/repo.git",
        "sha1": "36287f7c3b09eff78395267a3ac0d7da067863fd",
        "url": "https://github.com/example/repo/commit/36287f7c3b09eff78395267a3ac0d7da067863fd",
        "refactorings": [{
            "type": "Pull Up Attribute",
            "description": "Pull Up Attribute private age : int from class org.animals.Labrador to class org.animals.Dog",
            "leftSideLocations": [{
              "filePath": "src/org/animals/Labrador.java",
              "startLine": 5,
              "endLine": 5,
              "startColumn": 14,
              "endColumn": 21,
              "codeElementType": "FIELD_DECLARATION",
              "description": "original attribute declaration",
              "codeElement": "age : int"
            }],
            "rightSideLocations": [{
              "filePath": "src/org/animals/Dog.java",
              "startLine": 5,
              "endLine": 5,
              "startColumn": 14,
              "endColumn": 21,
              "codeElementType": "FIELD_DECLARATION",
              "description": "pulled up attribute declaration",
              "codeElement": "age : int"
            }]
          }]
      }]
    }
  6. Detect refactorings in a locally cloned Git repository

    master

    Use GitHistoryRefactoringMiner to analyze the history of a local Git repository. You can detect all refactorings in the history, between specific commits or tags, or at a specific commit. All detection methods require a RefactoringHandler to process the results.

    Key methods:

    • detectAll(repo, branch, handler): Detects all refactorings in the entire history.
    • detectBetweenCommits(repo, startCommit, endCommit, handler): Iterates through all non-merge commits from the start to the end commit.
    • detectBetweenTags(repo, startTag, endTag, handler): Iterates through all non-merge commits between two tags.
    • detectAtCommit(repo, commitId, handler): Analyzes a specific commit identified by its SHA key.
    GitService gitService = new GitServiceImpl();
    GitHistoryRefactoringMiner miner = new GitHistoryRefactoringMinerImpl();
    
    Repository repo = gitService.cloneIfNotExists(
        "tmp/refactoring-toy-example",
        "https://github.com/danilofes/refactoring-toy-example.git");
    
    miner.detectAll(repo, "master", new RefactoringHandler() {
      @Override
      public void handle(String commitId, List<Refactoring> refactorings) {
        System.out.println("Refactorings at " + commitId);
        for (Refactoring ref : refactorings) {
          System.out.println(ref.toString());
        }
      }
    });
  7. Run the RefactoringMiner MCP server

    master

    The Docker image supports running the RefactoringMiner stdio Model Context Protocol (MCP) server.

    Basic MCP execution

    Run the server with an OAuthToken environment variable:

    docker run --rm -i --pull always -e OAuthToken=$OAuthToken tsantalis/refactoringminer:latest mcp

    MCP with local repository (worktree mode)

    To use the refactoringminer_diff worktree mode, you must mount your repository into the container and set it as the working directory (-w):

    docker run --rm -i --pull always -v "$PWD:/workspace" -w /workspace -e OAuthToken=$OAuthToken tsantalis/refactoringminer:latest mcp

    MCP with WebDiff browser

    To use the MCP AST diff browser, also publish port 6789:

    docker run --rm -i --pull always -v "$PWD:/workspace" -w /workspace -p 6789:6789 -e OAuthToken=$OAuthToken tsantalis/refactoringminer:latest mcp

    Important Notes for MCP:

    • Paths: MCP tools do not accept host repository paths. You must mount the repository and start the process from the container's working directory (e.g., /workspace).
    • Multiple Repositories: Mount a common parent directory and use relative source.workingDirectory values.
    • Host Binding: If your Docker setup requires a different address than 127.0.0.1, override the WebDiff host settings using these environment variables:
      • REFACTORINGMINER_WEBDIFF_BIND_HOST (e.g., 0.0.0.0)
      • REFACTORINGMINER_WEBDIFF_PUBLIC_HOST (e.g., localhost)
  8. Configure Claude Code with RefactoringMiner MCP

    master

    To use RefactoringMiner with Claude Code, create a .mcp.json file in your project directory. You must use the absolute path to the built RM-fat.jar.

    Example .mcp.json configuration:

    {
      "mcpServers": {
        "refactoringminer": {
          "type": "stdio",
          "command": "java",
          "args": [
            "-jar",
            "/absolute/path/to/RefactoringMiner/build/libs/RM-fat.jar",
            "mcp"
          ]
        }
      }
    }

    Running Claude Code with the config:

    claude -p --mcp-config .mcp.json --strict-mcp-config \
      "Use RefactoringMiner to analyze my current worktree and report detected refactorings."

    Note on refactoringminer_diff: This tool starts a local WebDiff server. To view the diff, you must use an interactive client session (like claude --mcp-config .mcp.json --strict-mcp-config) so the agent can access the returned URL while the MCP process is alive.

  9. Analyze GitHub Pull Requests and Commits

    master

    You can run RefactoringMiner against a GitHub Pull Request or Commit by providing a GitHub URL and an OAuthToken.

    Using Environment Variables

    Pass your personal GitHub OAuth token (classic) via the -e OAuthToken flag:

    docker run --pull always -p 6789:6789 -e OAuthToken=ghp_Tz... tsantalis/refactoringminer diff --url https://github.com/JabRef/jabref/pull/14138

    Using a Configuration File

    To avoid leaving tokens in your bash history, you can store the token in a file inside a mounted volume.

    1. Create a directory on your host (e.g., ~/.refactoringminer on Linux/macOS or c:\users\{login}\.refactoringminer on Windows).
    2. Inside that directory, create a file named github-oauth.properties with the following content:
      OAuthToken=ghp_Tz...
    3. Run the container by mounting that directory to /diff:
    docker run -p 6789:6789 -v ~/.refactoringminer:/diff tsantalis/refactoringminer diff --url https://github.com/JabRef/jabref/pull/14138