Topology ToolKit (TTK)

TTK is an open-source library designed for efficient, generic, and easy topological data analysis and visualization. It is implemented in C++ and provides multiple interfaces for different workflows, including standalone programs and integration with visualization frameworks.

Available APIs and Interfaces:

• C++: Native library access.
• VTK/C++: Integration with the Visualization Toolkit.
• Python / PVPython: Scripting interfaces for data analysis and automated visualization.
• ParaView Plugin: Direct integration into the ParaView visualization environment.

For detailed setup and usage, refer to the official Installation instructions and Examples.

The packaging workflow generates user-installable binary packages for TTK by building corresponding TTK-ParaView packages.

• macOS/Windows: Uses Homebrew and Anaconda to manage build and run-time dependencies. To ensure correct targeting of brew or conda Python, separate Python installations on GitHub VMs are removed before the CMake configuration step.
• macOS Format: Relies on binary archives (.tar.gz of the installation prefix) rather than a standard package format, meaning there is no built-in uninstallation process.
• Process: The workflow first runs platform-dependent jobs to build packages, then executes separate jobs to test these packages outside the build environment.

To ensure that results remain identical regardless of the number of MPI processes used, unstructured grid data must be prepared before distribution.

Required Pipeline Pattern:

1. Open the data file in parallel with MPI.
2. Apply the TTKIdentifiers filter.
3. Apply the RedistributeDataSet filter.

Visual Pipeline: Data File $\rightarrow$ TTKIdentifiers $\rightarrow$ RedistributeDataSet

Once this sequence is completed, subsequent TTK filters will automatically trigger the necessary preconditioning.

Data File > TTKIdentifiers > RedistributeDataSet

The check_ci workflow ensures code quality through three main job types:

1. check-formatting: Uses clang-format to verify C++ code formatting. It also enforces Unix line endings, ensures files are not empty, prevents VTK includes in the base layer, and forbids using namespace directives in header files.
2. lint-code: Runs three tasks in parallel using clang-tidy:
   • Linting against rules defined in .clang-tidy.
   • Basic static analysis (this is the longest step, ~25 minutes).
   • Doxygen documentation verification against method signatures.
3. check-warnings: Uses clang-check to ensure no new compiler warnings are introduced and verifies that TTK can build across various binary configurations.

The test_build workflow is a lightweight version of the packaging workflow run on every Pull Request. It verifies that TTK builds correctly on the three main platforms and validates data integrity.

Validation Mechanism:

• Ubuntu: Runs ttk-data state files and Python scripts. It uses OSMesa for Offscreen Rendering to compare generated screenshots against a reference database in the ttk-data repository.
• macOS: Supports screenshot comparison, but rendering is significantly slower than Ubuntu.
• Windows: Offscreen Rendering is currently unsupported; the renderer tends to segfault on headless GitHub servers.
• Data Integrity: Python scripts generate output datasets in VTK formats. These are converted to SHA1 hashes and compared against a per-platform reference hash database stored in ttk-data.

To ensure code adheres to TTK formatting standards before submitting a PR, use git-clang-format to format only the files that have changed since the last commit in the origin remote:

git-clang-format origin/dev

To execute a Python script that utilizes TTK within the Docker environment, use the runTTKPythonDocker.sh script.

Important: You must use absolute paths for all arguments (Python scripts, data files, etc.).

./runTTKPythonDocker.sh [<Standard pvpython arguments: Python script, data, etc. ABSOLUTE PATHS ONLY)>]

./runTTKPythonDocker.sh /absolute/path/to/script.py /absolute/path/to/data.vtu

To run ParaView in distributed mode using a client-server architecture, use pvserver.

Constraints:

• TTK supports a number of processes equal to $2^n$ (where $n \in \mathbb{N}$).
• If using the periodic grid, you must use a number of processes equal to $8^n$ (though $2^n$ may work if data distribution is perfectly symmetric).

Workflow:

1. Start the server using mpirun with your desired number of processes and threads.
2. Launch the ParaView GUI locally.
3. In the GUI, go to File > Connect to link to the running pvserver.

# Start pvserver with 4 MPI processes and 8 threads per process using OpenMPI
OMPI_MPI_THREAD_LEVEL=1 OMP_NUM_THREADS=8 mpirun -np 4 pvserver

# In a separate terminal, start the client
paraview

Address Sanitizer (ASan) helps detect memory access errors and segfaults.

1. Enable for a module: In the module's CMakeLists.txt, add:

   target_compile_options(<module> PUBLIC -fsanitize=address)
   target_link_options(<module> PUBLIC -fsanitize=address)

2. Run-time setup: Set the LD_PRELOAD environment variable to preload the ASan libraries.
3. GDB Integration: To make GDB stop at the first ASan error, add these lines to your $HOME/.gdbinit:

   set environment ASAN_OPTIONS=abort_on_error=1:detect_leaks=0
   set environment LD_PRELOAD=/usr/lib/libasan.so

4. Execution: Launch your program via GDB: gdb -ex run --args <your command>.

Note: ParaView loads very slowly with ASan pre-loaded. It is often more efficient to debug a Python script via the Python interpreter.

For headless execution of a pipeline defined in a Python script, use pvbatch. This method does not require setting up a pvserver manually.

Constraints:

• The number of processes should be $2^n$ (or $8^n$ for periodic grids).

Example: To execute pipeline.py using 4 MPI processes and 8 threads per process with OpenMPI:

OMPI_MPI_THREAD_LEVEL=1 OMP_NUM_THREADS=8 mpirun -n 4 pvbatch pipeline.py

Use these tools to profile TTK modules:

CPU Performance with perf and hotspot:

1. Record traces: perf record --call-graph dwarf paraview --state=states/dragon.pvsm (this creates a perf.data file).
2. Visualize: Run hotspot in the directory containing perf.data to view Flamegraphs. Note: Ensure TTK is built with -ggdb -fno-omit-frame-pointer for accurate traces.

Memory Footprint with heaptrack: Run heaptrack followed by your command. It will record consumption and automatically open the visualization application:

heaptrack paraview --state=states/dragon.pvsm