kotlinx-benchmark

The kotlinx-benchmark Gradle plugin dynamically generates tasks based on your defined configuration profiles and targets.

For every combination of a configuration profile and a registered target, a specific task is created to execute that profile on that platform.

Task Naming Convention

// Example of how configuration and targets drive task generation
benchmark {
    configurations {
        named("main") { /* ... */ }
        register("smoke") { /* ... */ }
    }

    targets {
        register("jvm")
        register("js")
    }
}

// Resulting tasks include:
// - benchmark (main on jvm + js)
// - jvmBenchmark (main on jvm)
// - smokeBenchmark (smoke on jvm + js)
// - jvmSmokeBenchmark (smoke on jvm)

To use kotlinx-benchmark in a Kotlin Multiplatform (KMP) project, you must apply the plugin, configure repositories, and add the runtime dependency to your commonMain source set.

Kotlin DSL Setup

1. Apply the plugin in build.gradle.kts.
2. Configure pluginManagement in settings.gradle.kts to include gradlePluginPortal().
3. Add kotlinx-benchmark-runtime to the commonMain dependencies in build.gradle.kts.
4. Add mavenCentral() to the repositories block in build.gradle.kts.

// build.gradle.kts
plugins {
    id("org.jetbrains.kotlinx.benchmark") version "0.4.17"
}

// settings.gradle.kts
pluginManagement {
    repositories {
        gradlePluginPortal()
    }
}

// build.gradle.kts
kotlin {
    sourceSets {
        commonMain {
            dependencies {
                implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.17")
            }
        }
    }
}

repositories {
    mavenCentral()
}

Use the benchmark extension in your build.gradle.kts to define how benchmarks are executed and which platforms they run on.

• configurations: Define profiles with specific settings like iterations, warmups, iterationTime, and iterationTimeUnit. The main profile is the default.
• targets: Register the platforms (e.g., jvm, js) you want to benchmark.

// build.gradle.kts
benchmark {
    configurations {
        named("main") {
            iterations = 20
            warmups = 20
            iterationTime = 1
            iterationTimeUnit = "s"
        }
        register("smoke") {
            include("Essential") // Only run benchmarks matching this name/pattern
            iterations = 10
            warmups = 10
            iterationTime = 200
            iterationTimeUnit = "ms"
        }
    }

    targets {
        register("jvm")
        register("js")
    }
}

For Kotlin/JVM projects, you can isolate benchmarks by creating a custom source set and associating its compilation with your main or test compilation. This enables benchmarks to access internal APIs and reuse test code.

Step 1: Define the source set

Create a new source set named benchmark.

Step 2: Associate compilations

Associate the benchmark compilation with the main compilation (or test if you want to reuse test code) to allow access to internal APIs and dependency propagation.

Step 3: Register the benchmark target

Register the source set with the benchmark extension so the tool knows where to find and execute benchmarks.

Step 4: Add benchmark code

Place your benchmark code into the newly created benchmark source set.

// build.gradle.kts
sourceSets {
    create("benchmark")
}

kotlin {
    target {
        compilations.getByName("benchmark")
            .associateWith(compilations.getByName("main"))
    }
}

benchmark {
    targets {
        register("benchmark")
    }
}

To use kotlinx-benchmark in a Kotlin/JVM or Java project, follow these configuration steps in your Gradle files:

1. Apply the benchmark plugin in build.gradle.kts or build.gradle.
2. Configure plugin management in settings.gradle.kts or settings.gradle to include gradlePluginPortal().
3. Add the runtime dependency org.jetbrains.kotlinx:kotlinx-benchmark-runtime:<version> to your project dependencies.
4. Add Maven Central to your repositories block for dependency lookup.
5. Apply the allopen plugin (recommended for Kotlin) to ensure benchmark classes annotated with @State are open, allowing JMH to function correctly.
6. Register your benchmark target in the benchmark block to tell the tool where your benchmarks are located (e.g., the main source set).

// build.gradle.kts
plugins {
    id("org.jetbrains.kotlinx.benchmark") version "0.4.13"
    kotlin("plugin.allopen") version "2.2.0"
}

repositories {
    mavenCentral()
}

dependencies {
    implementation("org.jetbrains.kotlinx:kotlinx-benchmark-runtime:0.4.13")
}

allOpen {
    annotation("org.openjdk.jmh.annotations.State")
}

benchmark {
    targets {
        register("main")
    }
}