Pitest Documentation

When contributing to Pitest, follow these strict rules regarding dependencies and class loading to prevent conflicts with the code under test (SUT):

1. JVM Isolation: Do not load the code under test into the JVM within the pitest-entry module (e.g., via reflection).
2. pitest module restrictions: Do not introduce third-party dependencies into the pitest module, as they may conflict with the SUT's dependencies. If a dependency is absolutely unavoidable (e.g., ASM), it must be shaded.
3. Other modules: Dependencies are allowed in other modules but are discouraged; consult the maintainers before adding them. Any dependencies added to these modules must be provided on the classpath by users of the command-line tool or Ant.

Since it is impossible to reliably kill a single thread in Java, Pitest handles infinite loops or memory exhaustion by killing the entire minion (child) process.

Detection Mechanism:

• Timing-based: Pitest records the normal execution time of tests during the coverage stage. If a test takes $x$ times longer than the recorded time (plus a fudge factor), the mutant is flagged as having caused an infinite loop.
• Probe-based (Alternative): A more robust method involves inserting probes to count instruction hits. If a mutant causes a probe to be hit significantly more times than the unmutated code, the process is terminated.

Benefits of Process-level Termination:

• Robustly stops unresponsive code.
• Ensures complete isolation of JVM state (static variables, etc.) between mutations.

Pitest generates mutants by manipulating bytecode rather than source code. This approach is fast but can occasionally produce "junk" mutations (mutations that don't map to real programmer errors), especially when dealing with bytecode generated from languages like Scala.

The Two-Stage Generation Process:

1. Scanning: The main process scans the bytecode and creates a list of all possible mutations. This list is lightweight and can handle very large programs.
2. Identification: Each mutant is uniquely identified by a combination of:
   • The mutation operator used.
   • The class + method signature.
   • The index of the mutated bytecode instruction.

Note: The actual mutated bytecode is only generated by the minion process when it is ready to run the tests.

To optimize performance, Pitest uses coverage-based selection rather than running every test against every mutant.

The Workflow:

1. Coverage Phase: Before generating mutants, Pitest inserts coverage probes into the code and runs all available tests.
2. Mapping: It generates a map showing exactly which lines of code are executed by each individual test.
3. Execution: When a mutant is created at a specific instruction, Pitest only executes the subset of tests that are known to cover that instruction.

Limitation: Because coverage is determined during class loading, code in static initializers may appear to be executed only by the first test that triggers the class load.

Pitest employs two main strategies to improve performance:

1. Early Exit: Instead of running a full matrix of tests to see which tests kill which mutants, Pitest can be configured to stop analyzing a mutation as soon as a single test fails. This provides a significant performance boost (up to 50%).
2. Test Splitting: Pitest attempts to split tests into the smallest possible individually executable units. This prevents the system from having to run an entire test class if only one test within that class is required to kill a mutant.

To set up the development environment in IntelliJ:

1. Import the repository as an existing Maven project.

To maintain consistency with the project's coding standards, use the following profile files located in the root of the repository:

• code_format_profile.xml (for formatting)
• code_cleanup_profile.xml (for cleanup)

These files create profiles named henry in your IDE.

To set up the development environment in Eclipse:

1. Import the repository as an existing Maven project.
2. Groovy Note: If you do not have Groovy plugins installed, the pitest-groovy-verification module will report errors. Unless you are specifically working on Groovy-related features, it is recommended to simply close that module rather than installing the Groovy dependencies.

Pitest supports Java bytecode generated by versions as old as Java 5. However, execution requires Java 8 or higher.

Development Recommendations:

• Recommended Version: Use Java 9 for development. This ensures you can use language features and APIs available in Java 9, while remaining compatible with the requirement to support Java 8+ execution.
• Constraint: APIs and language features from Java 9 and later cannot be used if you intend to maintain compatibility with Java 8 execution environments.