nf-test

nf-test is a testing framework designed for Nextflow pipelines. It uses a DSL similar to Nextflow to describe expected behavior using when and then blocks.

Key features include:

• Unit Testing: Test various levels of your pipeline, including the entire pipeline, specific processes, or custom functions.
• Snapshot Testing: Automatically capture snapshots of output channels or objects to compare against reference files, helping to detect regressions.
• Extensibility: Supports third-party libraries (e.g., JAR files), Groovy functions, and a plugin system to add custom assertion methods (e.g., FASTA file support).
• Boilerplate Generation: Provides commands to generate test code scaffolding.

Profiles allow you to customize Nextflow executions. You can define a default profile in nf-test.config or specify one via the --profile CLI argument.

Profile Combination

Use the + prefix to combine the default profile with an additional one. For example, nf-test --profile +B results in Nextflow running with -profile A,B (where A is the default from nf-test.config).

Priority Order

When multiple profiles are defined, they are evaluated in this order (highest priority first):

1. CLI Argument: Profiles provided via --profile on the command line.
2. Testcase/Testsuite: Profiles specified within a nextflow_process or test block.
3. nf-test.config: The default profile defined in the global configuration file.

The when block defines the inputs for the test execution. It describes the state of the Nextflow environment and the specific arguments passed to the process/workflow.

It contains two main components:

1. params (optional): A map used to override Nextflow's input params.
2. process: A multi-line string where you define the input array for the process. You can use Nextflow-specific functions like file(), Channel, or path() here. Everything inside this block is executed within a generated Nextflow script.

Example of setting a parameter and a process input:

when {
    params {
        outdir = "output"
    }
    process {
        """
        input[0] = file("${projectDir}/test_data/transcriptome.fa")
        """
    }
}

nf-test allows you to categorize tests using the tag directive within a testsuite or an individual testcase. This enables selective execution based on tags or names.

Defining Tags

Use the tag directive inside your test definition:

nextflow_process {
	name "suite 1"
	tag "tag1"

	test("test 1") {
		tag "tag2"
		tag "tag3"
		...
	}
}

Running Tagged Tests

• Single tag: nf-test test --tag tag2 executes tests matching that tag.
• Multiple tags: Provide a comma-separated list to execute tests that match at least one of the provided tags. Tags are not case-sensitive.
  • nf-test test --tag tag3,tag4
  • nf-test test --tag TAG3,TAG4
• By Name: Because names are automatically added to tags, you can execute a suite or test directly using its name:
  • nf-test test --tag "suite 1"

The nf-test plugin system allows developers to extend the testing framework in two ways:

1. Extending existing classes: Adding custom methods to existing objects (e.g., adding .fasta methods to a path() object).
2. Extending nf-test methods: Adding entirely new methods to the nf-test core functionality.

This separation allows for independent release cycles, easier third-party maintenance, and the ability to use private repositories for protected code.

The setup method allows you to execute dependent processes or workflows before the primary when block of a test. This is used to prepare required input data or perform essential preparatory steps.

Inside a setup block, use the run method to define the dependency. You can specify whether the dependency is a process or a workflow and provide its script path and inputs.

Warning: Changes to processes or workflows executed within the setup method can cause test runs to fail.

test("my test"){
    setup {
        // Define and execute dependent processes or workflows here
    }
}

Nextflow channels emit values in an unpredictable order. To allow for reliable assertions by index, nf-test automatically sorts channels prior to executing the then closure.

Sorting is performed using integer, string, and path comparisons. This means that even if Nextflow emits items in a random order, nf-test will present them to your assertions in a deterministic, sorted order.

• Single-item channels (e.g., val('A')) are treated as an unordered List<Object>.
• Tuple channels (e.g., [ 'A', file('B.txt') ]) are treated as an unordered List<List<Object>>.

The params block is an optional map located within the when block of a testcase. It allows you to overwrite Nextflow's input params. You can define simple key-value pairs or nested parameters using Groovy syntax that mirrors your Nextflow script structure. The params map is also accessible in the then block for assertions.

when {
    params {
        outdir = "output"
        output {
          dir = "output"
        }
    }
}

then {
    assert params.output.dir == "output"
}

You can provide entirely new, standalone methods that are not tied to an existing class.

To implement this:

1. Create a class containing static methods.
2. Register this class in your plugin manifest under the extensionMethods property.

Example: A method that can be called directly as helloFasta().

// com.askimed.nf.test.fasta.Methods
public class Methods {

  // can be used as: helloFasta()
  public static void helloFasta() {
    System.out.println("Hello FASTA");
  }
}

Topic channels are a special type of output channel used to share data between processes without explicit connections in the Nextflow script.

To use them, you must first declare them in the nextflow_workflow block using the topics keyword. You can also declare specific topics within a test block. Once declared, the content is accessible via the topics object in then or expect blocks.

nextflow_workflow {
    ... 
    topics "my_topic_channel", "your_topic_channel"
    
    test("test_name") {
        topics "test_specific_topic"

        when { ... }
        then {
            assert topics.test_specific_topic == "expected_value"
        }
    }
}