uTest Documentation

Golden testing allows you to compare an actual value against an 'expected' value that is automatically managed. This is useful for simple assertions where you want uTest to help fill in the expected value the first time you run the test and keep it up to date as your code evolves.

Supported methods:

• assertGoldenLiteral: For comparing against a literal value.
• assertGoldenFile: For comparing against a file.
• assertGoldenFolder: (Added in 0.10.0-RC1) For comparing against a folder of files.

uTest uses a vendored version of PPrint to provide pretty-printed, syntax-highlighted, and diff-enabled output for assertion errors. This includes the values of local variables found during the failure.

Formatting configuration options have been migrated from utest.TestSuite onto the utest.runner.Framework.

uTest is compatible with Scala.js and Scala-Native, but note the following:

Scala.js:

• Does not support parallelism. Use single-threaded ExecutionContexts like utest.ExecutionContext.runNow or scala.scalajs.concurrent.JSExecutionContext.runNow.
• Eventually and Continually are not supported because they rely on blocking retry-loops, which are not allowed in Scala.js.

Scala-Native:

• Support is experimental. While tested, it may have quirks (e.g., NullPointerExceptions might be fatal).

You can customize how test results are formatted by overriding methods in a Framework. If you want to customize formatting for a specific suite, override utestFormatter on that TestSuite.

uTest uses a vendored version of the Fansi library at utest.shaded.fansi to avoid dependency conflicts. You can return utest.shaded.fansi.Str objects or standard java.lang.String objects containing ANSI escape codes, which will be automatically parsed.

uTest allows arbitrary deep nesting of test blocks to organize code. However, tests must be defined statically; you cannot define tests inside if-statements or for-loops. Only the leaf nodes (the innermost blocks) are executed as tests.

If you need to run the same logic for multiple inputs, do not use a loop to define tests. Instead, factor the logic into a function and call that function from distinct test blocks.

package example

import utest._

class NestedTests extends TestSuite{
  val tests =  Tests{
    val x = 1
    test("outer1"){
      val y = x + 1

      test("inner1"){
        assert(x == 1, y == 2)
        (x, y)
      }
      test("inner2"){
        val z = y + 1
        assert(z == 3)
      }
    }
    test("outer2"){
      test("inner3"){
        assert(x > 1)
      }
    }
  }
}

Golden testing allows you to assert that a runtime value matches a literal data structure or a file's contents.

assertGoldenLiteral(actualValue, expectedLiteral) compares the two. If they differ, uTest shows a diff and provides instructions to update the expected value.

Updating Golden Literals: To automatically update the expected literal in your source code, run your tests with the environment variable UTEST_UPDATE_GOLDEN_TESTS=1. uTest uses PPrint to convert the actual value into a string and splice it into your source code. This works well for primitives, collections, and case classes.

Note: assertGoldenLiteral is only supported on Scala-JVM.

val x = List(1, 2)
assertGoldenLiteral(x, List(1, 2, 3, 4))

To enable automatic retries for all tests within a specific suite, mix in the TestSuite.Retries trait and override the utestRetryCount value with the desired number of attempts.

For retrying only specific tests or expressions, use Local Retries.

object SuiteRetryTests extends TestSuite with TestSuite.Retries{
  override val utestRetryCount = 3
  val flaky = new FlakyThing
  def tests = Tests{
    'hello{
      flaky.run
    }
  }
}

To use uTest in an SBT project, add the following dependencies to your build.sbt. Use the %% operator for Scala-JVM and %%% for Scala.js or Scala-Native.

libraryDependencies += "com.lihaoyi" %% "utest" % "0.10.0-RC1" % "test" // Scala-JVM
libraryDependencies += "com.lihaoyi" %%% "utest" % "0.10.0-RC1" % "test" // Scala.js or Scala-Native

testFrameworks += new TestFramework("utest.runner.Framework")

To perform actions that affect an entire test run (like initializing a database or cleaning up a filesystem) rather than just individual tests, subclass utest.runner.Framework and override setup() and teardown().

To use your custom framework in SBT, add it to your testFrameworks configuration:

testFrameworks += new TestFramework("test.utest.CustomFramework")

class CustomFramework extends utest.runner.Framework {
  override def setup() = {
    println("Setting up CustomFramework")
  }
  override def teardown() = {
    println("Tearing down CustomFramework")
  }
}

You can run all tests, specific suites, or individual tests using the full path. uTest supports a brace expansion syntax {} to select specific tests within a group.

SBT Commands:

• Run all: sbt myproject/test
• Run specific test: sbt 'myproject/test-only -- example.NestedTests.outer1.inner1'
• Run group: sbt 'myproject/test-only -- example.NestedTests.outer1'
• Run specific selection: sbt 'myproject/test-only -- example.NestedTests.outer1.{inner1,inner2}'

Mill Commands:

• Run all: ./mill myproject.test
• Run specific test: ./mill myProject.test example.NestedTests.outer1.inner1
• Run specific selection: ./mill myProject.test 'example.NestedTests.outer1.{inner1,inner2}'