DOCSEARCHbyAlgolia
Submit Docs

Dropbox Affected Module Detector

repository·main·Indexed 20 days ago

https://github.com/dropbox/affectedmoduledetector

A Gradle plugin that identifies which modules in a multi-module project are impacted by specific git commits. It optimizes CI/CD pipelines by running tests only on changed or dependent modules. It provides specialized tasks such as runAffectedUnitTests and runAffectedAndroidTests, supports multiple git comparison strategies (including SpecifiedBranchCommitMergeBase and SpecifiedRawCommitSha), and allows for the definition of custom tasks via the affectedModuleDetector configuration block.

Tokens
3.7K
Snippets
8
Records
16
Agent score
70%

What's inside affectedmoduledetector

  1. Choose an affected module mode

    main

    The detector supports three modes to control which projects are considered 'affected'. You can toggle these using Gradle project properties (-P):

    1. All Affected Projects (Default): The union of Changed Projects and Dependent Projects. This is recommended for merges.
    2. Changed Projects: Only projects that had files changed within them. Enabled with -Paffected_module_detector.changedProjects.
    3. Dependent Projects: Only projects that depend on projects that had changes. Enabled with -Paffected_module_detector.dependentProjects.

    To enable the detector at all, you must pass -Paffected_module_detector.enable.

  2. Use SpecifiedRawCommitSha to skip git operations

    main
    If your environment uses Git mirroring (which can lead to inaccurate common ancestor commits), you can provide a raw commit SHA directly. This tells the plugin to skip internal git operations like git rev-parse or git merge base and compare against the exact SHA provided. This requires setting compareFrom = "SpecifiedRawCommitSha" in the configuration.

  3. Compare changes using SpecifiedBranchCommit vs SpecifiedBranchCommitMergeBase

    main

    When comparing the current branch against a parent branch (e.g., origin/dev), the choice of comparison method affects the result:

    • SpecifiedBranchCommit: Uses git rev-parse. If your feature branch is not up-to-date with the parent branch, it will include changes from other developers merged into the parent branch, potentially resulting in more 'affected' files than intended.
    • SpecifiedBranchCommitMergeBase: Uses git merge base. This finds the nearest common ancestor, ensuring only the changes unique to your feature branch are considered. This is generally the preferred behavior for CI/CD.

  4. Install the Affected Module Detector Gradle Plugin

    main

    The Affected Module Detector is a Gradle plugin that determines which modules are affected by file changes in a git commit. This allows you to run tests only in modules that have changed or depend on changed modules.

    Option 1: Using the plugins block (Recommended)

    Add the plugin to your settings.gradle(.kts) to include Maven Central and the Gradle Plugin Portal, then apply it in your root build.gradle(.kts).

    Option 2: Manual buildscript dependency

    Apply the plugin to the root build.gradle using the buildscript block.

    Option 3: Developing a custom plugin

    If you are building a plugin that uses the Affected Module Detector APIs, add it to your buildSrc dependencies.

    // settings.gradle(.kts)
pluginManagement {
  repositories {
    mavenCentral()
    gradlePluginPortal()
  }
}

// root build.gradle(.kts)
plugins {
  id("com.dropbox.affectedmoduledetector") version "<latest-version>"
}
    // Alternative: Manual buildscript dependency
buildscript {
  repositories {
    mavenCentral()
  }
  dependencies {
    classpath "com.dropbox.affectedmoduledetector:affectedmoduledetector:<LATEST_VERSION>"
  }
}
//rootproject
apply plugin: "com.dropbox.affectedmoduledetector"
    // For plugin developers in buildSrc
implementation("com.dropbox.affectedmoduledetector:affectedmoduledetector:<LATEST_VERSION>")

  5. Add custom Gradle tasks for impact analysis

    main

    You can execute custom Gradle commands as part of the impact analysis by declaring them in the affectedModuleDetector configuration block. To do this, add an instance of AffectedModuleConfiguration.CustomTask to the customTasks list.

    Each CustomTask requires:

    1. A unique task name.
    2. The actual Gradle task name to execute.
    3. A description of the task.

    Note: If your task is highly complex and requires advanced Gradle APIs to function correctly, you may need to implement a custom plugin manually in a buildSrc module instead of using this configuration method.

    affectedModuleDetector {
     // ...
     customTasks = [
           new AffectedModuleConfiguration.CustomTask(
                   "runDetektByImpact", 
                   "detekt",
                   "Run static analysis tool without auto-correction by Impact analysis"
           )
     ]
     // ...
}

  6. Configure the affectedModuleDetector block

    main

    You can customize the detector's behavior in the root project using the affectedModuleDetector configuration block. Key options include defining which files trigger all modules, excluding specific modules via regex, and setting the git comparison strategy.

    Note: If using compareFrom = "SpecifiedBranchCommit" or compareFrom = "SpecifiedBranchCommitMergeBase", you must set specifiedBranch before configuring compareFrom.

    affectedModuleDetector {
    baseDir = "${project.rootDir}"
    pathsAffectingAllModules = [
            "buildSrc/"
    ]
    logFilename = "output.log"
    logFolder = "${project.rootDir}/output"
    compareFrom = "PreviousCommit" //default is PreviousCommit
    excludedModules = [
        "sample-util", ":(app|library):.+"
    ]
    ignoredFiles = [
        ".*\\.md", ".*\\.txt", ".*README"
    ]
    buildAllWhenNoProjectsChanged = true // default is true
    includeUncommitted = true
    top = "HEAD"
    customTasks = [
        new AffectedModuleConfiguration.CustomTask(
            "runDetektByImpact",
            "detekt",
            "Run static analysis tool without auto-correction by Impact analysis"
        )
    ]
}

  7. Configure Android test variants

    main

    By default, the detector looks for assembleAndroidDebugTest, connectedAndroidDebugTest, and testDebug. You can override these specific tasks for your module using the affectedTestConfiguration block.

    affectedTestConfiguration {
     assembleAndroidTestTask = "assembleAndroidReleaseTest"
     runAndroidTestTask = "connectedAndroidReleaseTest"
     jvmTestTask = "testRelease"
}

  9. Use AffectedModuleDetector APIs in custom plugins

    main

    If you are writing a custom Gradle plugin and want to integrate with the detector, use these two primary methods:

    • AffectedModuleDetector.configureTaskGuard: Applies an onlyIf guard to a task. This can be called during the configuration phase or the execution phase.
    • AffectedModuleDetector.isProjectAffected: Returns a boolean indicating if the project was affected. Note: This can only be called after the project has been configured.

  10. Configure the Affected Module Detector plugin

    main

    After applying the plugin to your root project, you can configure its behavior using the affectedModuleDetector block. This block allows you to specify the base directory, paths that affect all modules (like buildSrc/), and the log folder location.

    Note: The plugin must be applied to the root project.

    affectedModuleDetector {
    baseDir = "${project.rootDir}"
    pathsAffectingAllModules = [
        "buildSrc/"
    ]
    logFolder = "${project.rootDir}"
}

  11. Configure custom Gradle tasks via `customTasks`

    main

    You can extend the Affected Module Detector by defining custom Gradle tasks that should be executed based on impact analysis. To do this, add a list of AffectedModuleConfiguration.CustomTask instances to the customTasks property in your build.gradle file. Each custom task requires a command to run when impacted, the original Gradle command, and a description.

    Each CustomTask requires:

    • commandByImpact: The command to execute if the module is affected.
    • originalGradleCommand: The original Gradle command.
    • taskDescription: A description of the task.
    affectedModuleDetector {
     ... 
     customTasks = [ // <- list of custom gradle invokes
         new AffectedModuleConfiguration.CustomTask(
              "runSomeCustomTaskByImpact",
              "someTaskForExample",
              "Task description."
          )
     ]
     ... 
}

  12. Configure comparison strategies via `compareFrom`

    main

    The compareFrom property determines the starting point for determining which commits to compare against. It accepts one of the following values:

    • PreviousCommit (default)
    • ForkCommit
    • SpecifiedBranchCommit (requires specifiedBranch to be set)
    • SpecifiedBranchCommitMergeBase (requires specifiedBranch to be set)
    • SpecifiedRawCommitSha (requires specifiedRawCommitSha to be set)

    Note: If you use SpecifiedBranchCommit or SpecifiedBranchCommitMergeBase, you must provide a branch name using the specifiedBranch configuration key.