Incrementalist

repository·dev·Indexed 43 days ago

https://github.com/petabridge/incrementalist

A .NET tool designed to optimize CI/CD pipelines for large solutions and monorepos. It uses Git and Roslyn to identify affected projects, allowing developers to build and test only the impacted subset of the codebase. Available as a dotnet tool (Incrementalist.Cmd), it supports parallel execution, glob pattern filtering, and JSON-based configuration.

Tokens
6.5K
Snippets
19
Records
32
Agent score
53%

What's inside Incrementalist

  1. Overview of Azure Pipelines Build Files

    dev

    The project includes .yaml files designed for Windows Azure DevOps Pipelines. These pipelines automate several build and release workflows:

    • Pull request validation on Linux: Uses Mono or .NET Core.
    • Pull request validation on Windows: Uses .NET Framework or .NET Core.
    • NuGet releases: Automates NuGet releases and posts automatic release notes to a GitHub Release repository.
  2. Filter affected projects with glob patterns

    dev

    After Incrementalist identifies affected projects via Git and dependencies, you can refine the list using glob patterns:

    • --target-glob "<pattern>": Includes only projects matching the pattern(s). If multiple patterns are provided, a project is included if it matches any of them.
    • --skip-glob "<pattern>": Excludes projects matching the pattern(s) from the list. If multiple patterns are provided, a project is excluded if it matches any of them.

    Example: Build all projects in src/ but exclude any test projects:

    incrementalist run -b dev --target-glob "src/**/*.csproj" --skip-glob "**/*.Tests.csproj" -- build -c Release --nologo
    incrementalist run -b dev --target-glob "src/**/*.csproj" --skip-glob "**/*.Tests.csproj" -- build -c Release --nologo
  3. How Incrementalist works

    dev

    Incrementalist optimizes .NET build processes by determining which projects need to be rebuilt based on Git changes.

    The Analysis Process

    1. Git Analysis: Compares current changes against a target branch (e.g., dev or master) to identify modified files.
    2. Solution Analysis: Analyzes project dependencies to identify which specific projects are affected by those changes.
    3. Decision Logic:
      • Solution-wide changes: If changes are detected in files like Directory.Build.props, global.json, or .sln files, Incrementalist triggers a full build of all projects.
      • Specific changes: If changes are localized, Incrementalist identifies only the affected projects and their dependencies to generate a targeted build list.
  4. Configure Azure Pipelines for NuGet Releases

    dev

    When using the windows-release.yaml pipeline for NuGet releases, you must perform the following manual configuration steps:

    1. Update Pipeline Variables: Modify the pipeline variables within windows-release.yaml to match your specific project requirements.
    2. Set up Variable Groups: Create Azure DevOps variable groups to securely store and provide your signing information and NuGet push credentials.
  5. Create and publish NuGet packages

    dev

    To package Incrementalist for distribution, use dotnet pack. It is recommended to build in Release mode first. You can also include symbols for debugging. Once created, packages can be pushed to NuGet using dotnet nuget push.

    # 1. Build in Release mode
    dotnet build -c Release
    
    # 2. Create packages with symbols
    dotnet pack -c Release -o bin/nuget --include-symbols --no-build
    
    # 3. Push to NuGet (replace YOUR_API_KEY)
    dotnet nuget push bin/nuget/*.nupkg --api-key YOUR_API_KEY --source https://api.nuget.org/v3/index.json
  6. Install Incrementalist CLI

    dev

    Incrementalist is available as a dotnet tool named Incrementalist.Cmd. You can install it globally or locally within a project.

    Global Installation

    Install the tool globally to use the incrementalist command from anywhere:

    dotnet tool install --global Incrementalist.Cmd

    Local Installation

    To install it as a local tool for a specific repository:

    dotnet new tool-manifest
    dotnet tool install Incrementalist.Cmd
    dotnet tool install --global Incrementalist.Cmd
  7. Run unit tests for Incrementalist

    dev

    Use dotnet test to execute the unit test suite. You can run all tests, run them specifically with a Release configuration, or request detailed output using specific loggers.

    # Run all unit tests
    dotnet test
    
    # Run tests with Release configuration
    dotnet test -c Release
    
    # Run tests with detailed output
    dotnet test --logger:trx --logger:"console;verbosity=normal"
  8. How to run Incrementalist commands

    dev

    The way you invoke Incrementalist depends on how you installed it.

    Using a Global Tool

    If installed globally, use the incrementalist command directly:

    incrementalist run -b dev -- build

    Using a Local Tool

    If installed locally, use dotnet incrementalist with a -- separator before the Incrementalist arguments.

    Note: Do NOT use dotnet tool run incrementalist as it causes parsing issues. Use dotnet incrementalist instead.

    dotnet incrementalist -- run -b dev -- build
    dotnet incrementalist -- run -b dev -- build -c Release --nologo
  9. Build the Incrementalist solution

    dev

    Use the dotnet build command to compile the solution. You can build using the default Debug configuration or specify a Release configuration.

    # Build with default configuration (Debug)
    dotnet build
    
    # Build with Release configuration
    dotnet build -c Release
  10. Real-world usage examples in Akka.NET and Akka.Management

    dev

    Incrementalist is used in large-scale projects to manage conditional test execution based on environment or project type.

    Akka.NET Patterns

    • Run Only .NET Framework Tests: Uses a specific configuration file (e.g., .incrementalist/testsOnly.json) to filter for projects targeting the .NET Framework.
    • Run Only Multi-Node Tests: Uses a configuration file (e.g., .incrementalist/mutliNodeOnly.json) to isolate tests that require multi-node orchestration.

    Akka.Management Patterns

    • Skip TestContainer Projects on Windows: Uses configuration (e.g., .incrementalist/windowsDevOpsBuilds.json) to skip projects that rely on Docker/TestContainers (like Azurite) when running on Windows build agents that lack Docker support.
    • Run All Tests: Uses a default configuration (e.g., .incrementalist/incrementalist.json) on Linux agents where the full suite can run without platform restrictions.
  11. Verify changes in Incrementalist

    dev

    When contributing or making changes, follow these verification steps to ensure stability:

    1. Basic Verification

    Run a standard build and test cycle:

    dotnet build
    dotnet test

    2. Full Verification

    Perform a complete check using Release configurations and integration tests:

    dotnet build -c Release
    dotnet test -c Release --no-build
    ./scripts/integration-tests.ps1 -Configuration Release

    3. Package Verification

    For releases, create the package and inspect the contents of the .nupkg files in bin/nuget/ using a tool like NuGet Package Explorer.