Alcotest

Alcotest supports two test speeds to help manage execution time:

• `Quick: Standard tests intended to run on every invocation.
• `Slow: Stress tests or heavy computations intended to run occasionally (e.g., before a release).

To suppress `Slow tests and run only `Quick tests, use the -q flag.

$ ./test.exe -q # run only the quick tests
$ ./test.exe    # run quick and slow tests

To use Alcotest in an OCaml project:

1. Opam configuration: Add (alcotest :with-test) to your dune-project file or "alcotest" {with-test} to your .opam file. Use the with-test package variable to ensure it is only a dependency during testing.
2. Installation: Run opam install --deps-only --with-test . to install dependencies.
3. Dune configuration: In your dune file, declare the test dependency using (test (libraries alcotest ...) ...).
4. Execution: Run tests using dune runtest.

$ opam install --deps-only --with-test .

$ dune runtest

For asynchronous testing, use the Alcotest_lwt module. Instead of unit -> unit, your test functions will have the type unit -> unit Lwt.t.

Features:

• Async Exception Handling: If an async exception occurs, Alcotest will cancel the test case and fail it rather than crashing the process.
• Resource Cleanup: Alcotest_lwt.test_case provides a switch. You can use Lwt_switch.add_hook with this switch to ensure resources are freed when the test finishes or fails.

Wrap your suite in Lwt_main.run.

let free () = print_endline "freeing all resources"; Lwt.return ()

let test_lwt switch () =
  Lwt_switch.add_hook (Some switch) free;
  Lwt.async (fun () -> failwith "All is broken");
  Lwt_unix.sleep 10.

let () =
  Lwt_main.run @@ Alcotest_lwt.run "foo" [
    "all", [
      Alcotest_lwt.test_case "one" `Quick test_lwt
    ]
  ]

If your test functions require extra parameters (type 'a -> unit instead of unit -> unit), use Alcotest.run_with_args.

You must provide a Cmdliner.Term to define how the extra parameter is parsed from the command line. Note that only optional arguments are supported; positional arguments are not allowed.

let test_nice i = Alcotest.(check int) "Is it a nice integer?" i 42

let int =
  let doc = "What is your preferred number?" in
  Cmdliner.Arg.(required & opt (some int) None & info ["n"] ~doc ~docv:"NUM")

let () =
  Alcotest.run_with_args "foo" int [
    "all", ["nice", `Quick, test_nice]
  ]

Use the following options to modify how test.exe behaves during execution:

• --bail: Stop running tests immediately after the first failure.
• -c, --compact: Use compact output format.
• --color=WHEN: Set color output (auto, always, or never).
• -e, --show-errors: Display the details of test errors.
• --json: Output results in JSON format (ideal for scripting).
• -o DIR: Specify a directory to store test log files.
• -q, --quick-tests: Run only the tests designated as 'quick tests'.
• --tail-errors=N: In case of an error, show only the last N lines of output.
• -v, --verbose: Display full test outputs. Warning: Using this prevents output logs from being available for later inspection.

test.exe --bail --compact --color=always --show-errors --json -o ./logs -q --tail-errors=10 -v

You can control test.exe behavior without command line arguments by setting the following environment variables:

When running test.exe, the exit code indicates the result of the execution:

• 0: Success.
• 123: Indiscriminate errors reported on standard error.
• 124: Command line parsing errors.
• 125: Unexpected internal errors (bugs).

You can run specific tests or a subset of test cases using test.exe.

• NAME_REGEX: A regular expression that matches the names of the tests you want to execute.
• TESTCASES: A comma-separated list of test case numbers or ranges. Both - and .. are valid separators for ranges.

Example usage:

# Run tests matching 'user_auth' and specific test case numbers
test.exe user_auth 4,6-10,19

# Run tests matching 'api' using the '..' range separator
test.exe api 1..5

test.exe [NAME_REGEX] [TESTCASES]

You can filter tests at runtime using a regular expression matching the test names, optionally combined with a comma-separated list of test numbers or ranges (e.g., 2,4..9).

Note: You cannot filter by the human-readable test case name (e.g., Lower case); you must use the test name and its index number.

CLI Syntax: ./your_test_binary test '<regex>' ['<numbers>']

# Run tests matching 'concat'
$ ./simple.native test '.*concat*'

# Run test 'string-case' specifically at index 1 through 3
$ ./simple.native test 'string-case' '1..3'

Use the list command to see all available tests in the current suite. You can optionally specify the color mode using the --color option.

Available color modes for --color=WHEN:

• auto: Defaults to 'always' inside Dune, otherwise 'auto'.
• always
• never

test.exe list [--color=auto|always|never]

You can control the colorization of Alcotest output using the --color flag or the ALCOTEST_COLOR environment variable.

Options for --color (or ALCOTEST_COLOR):

• auto: (Default) Automatically detects if the output is a TTY. If running inside Dune, it defaults to always.
• always: Forces ANSI color output.
• never: Disables color output.

# Example using CLI flag
alcotest --color always

# Example using environment variable
ALCOTEST_COLOR=never alcotest