Stryker.NET Documentation

Stryker uses Node Orchestrators to determine how different C# syntax structures are mutated. The orchestration process follows a specific order: Stryker iterates through a list of orchestrators and executes the first one where CanHandle returns true.

Key behaviors to note:

• Declaration Order Matters: Orchestrators are tried in declaration order. More specific orchestrators must be declared before more general ones to ensure they are matched first.
• Debugging Mutations: If a syntax construct is not being mutated as expected, check if a DoNotMutateOrchestrator is intercepting it or if the correct orchestrator is declared in the right order.
• Orchestrator List: You can inspect the full list and the relative declaration order by looking at CSharpMutantOrchestrator.BuildOrchestratorList().

To see which orchestrators are active, refer to the CSharpMutantOrchestrator class, which serves as the entry point for mutating C# source files.

A maintainer (synonymous with a GitHub 'Collaborator') is expected to contribute regularly and help define the scope and direction of Stryker.NET.

Key Requirements:

• Embody Code of Conduct values: Maintainers must model the project's values in all public interactions.
• Security: Maintainers must have 2FA enabled on their GitHub account.
• Decision Making: Participate in defining project scope, urgency, and boundaries.

Maintainer Rights (at discretion):

• Merging pull requests.
• Modifying issues (closing, labeling, assigning).
• Interacting with CI servers (canceling builds, restarting jobs).
• Performing CRUD operations on GitHub integrations.
• Participating in the decision-making process.

Note on 'Owners': Some maintainers have administrative rights to the stryker-mutator organization and NuGet publishing access. Those with publishing access must use NuGet's 2FA.

Stryker.NET uses a technique called mutant schemata (also known as mutation switching) to achieve high performance while maintaining accuracy. Instead of modifying source code or byte code directly, the framework injects conditional logic (if statements) into the code. This allows all mutations to be compiled into a single assembly at once, enabling fast test runs and the ability to keep the mutated assembly in memory.

Key characteristics of mutant schemata:

• Speed: All mutants are compiled together, making the process faster than source code mutation.
• Accuracy: It can show the exact location of mutations to the user.
• Capabilities: Supports testing multiple mutations in a single test run and makes mutation coverage calculation easier.
• Limitations: It cannot mutate constant values, method names, or access modifiers. Additionally, all injected mutations must be syntactically correct to avoid compile errors during the single compilation phase.

if(Environment.GetEnvironmentVariable("ActiveMutation") == "1") {
  i--; // mutated code
} else {
  i++; // original code
}

xUnit has specific behaviors that affect how Stryker performs mutation testing:

Conflicting Test Cases

If multiple xUnit test cases share the same Guid (identifier), xUnit will report results for all of them under that single VsTest test case.

Ignored Tests

Tests marked as ignored in xUnit are completely ignored by the runner and are not reported to Stryker.

Theories (Parameterized Tests)

• Static Theories (InlineData): These are processed as distinct test cases unless they share identical parameters.
• Run-time Theories: These may be discovered as a single test. During discovery, xUnit attempts to map data sets to test cases. If it cannot create unique display names, it may provide multiple results for the same test case.

Execution Sequence and Risks

1. Discovery: xUnit discovers all tests at startup. If a specific list is provided, it filters them. Note that requested tests not found during discovery will not be reported.
2. Execution: xUnit runs all data sets for a theory, then reports results.

Critical Risks for Stryker:

• Coverage Spillage: Because Stryker captures coverage on testcase end, coverage from one test might be incorrectly associated with the next test if it occurs between the end and start events.
• Isolation Issues: If a mutation changes a test's name (e.g., via ToString()), the identifier changes. This prevents the test from being run in isolation via Stryker, as Stryker can no longer predict the new name.

TestSession start event
xUnit discovers test
   xUnit discovers theories
     xUnit enumerates data source (for tests)
...
TestCase start event
   xUnit runs test with first set of data
   xUnit runs test with second set of data
   ...
   xUnit runs test with last set of data

   xUnit reports first test results
TestCase end event
   xUnit reports second test result
   ...
   xUnit reports last test result
TestCase start event
   ...
TestSession end event