NVIDIA Holoscan SDK

Holoviz is the visualization framework within the Holoscan SDK used to composite real-time streams of frames with various overlay layers, such as segmentation masks, geometry, and GUI elements.

It is built on Vulkan for maximum performance, leveraging the NVIDIA GPU driver.

There are two primary ways to use Holoviz:

1. Holoviz operator: A ready-to-use operator sufficient for many complex visualization tasks. It is used in several official Holoscan example applications.
2. Holoviz module: A C++ API designed for advanced use cases where you need to create application-specific visualization operators. The standard Holoviz operator is itself built using this module.

The Holoscan Inference module is a framework designed to facilitate the creation and execution of inference applications. To build an application, you initialize the required operators within the Holoscan SDK, define the data flow between these operators, and configure their specific parameters.

Key capabilities include:

• End-to-end CUDA-based data buffers (with optional host-based data movement).
• Multi-backend support including TensorRT, Torch (Libtorch 1.12.0), and ONNX Runtime.
• Configurable inference parameters such as single/multi-inference, multiple inputs/outputs, and parallel/sequential execution.
• Datatype support for float32, int32, and int8 for both inputs and outputs.
• GXF Message support via multiple receivers and transmitters.

Holoviz is a high-performance visualization module for real-time rendering of medical imaging and sensor data. It uses Vulkan for GPU-accelerated rendering and integrates with CUDA for efficient GPU-to-GPU data transfers, avoiding CPU round-trips.

Key capabilities include:

• Multi-layer compositing: Overlaying images, geometry, and GUI elements.
• Immediate mode API: A stateless API where you describe what to render each frame, making it easy to integrate into data pipelines.
• Flexible rendering: Support for YUV, depth maps, LUT-based coloring, and 3D camera control.
• Multiple window modes: Support for Windowed, Fullscreen, Exclusive display, and Headless modes.

NVIDIA Holoscan is an AI sensor processing platform designed for building streaming AI pipelines. It is used for low-latency sensor and network connectivity, data processing, and AI inference across embedded, edge, and cloud environments. Common use cases include medical devices, industrial inspection, and high-performance computing at the edge.

The SDK provides several key capabilities:

• C++ and Python APIs: The recommended interfaces for creating application pipelines.
• Built-in Operators: The fundamental units of work in Holoscan applications, providing functionalities like IO, machine learning inference, processing, and visualization.
• Performance Tools: Tools like Data Flow Tracking to profile applications and analyze data flow between operators in a graph.
• Extensibility: Access to HoloHub, a repository for sharing and reusing operators and sample applications.

The Core examples are designed for new users to learn the fundamental Holoscan SDK patterns:

• Hello World: Simplest application with a single operator.
• Ping series: Demonstrates workflows including custom operators, multi-port operators, distributed applications (transmitting tensors between fragments), cyclic paths, and asynchronous buffers.
• Video Replayer: Loading video from disk instead of using Tx/Rx ports.
• Flow Tracker: Latency analysis via data flow tracking.
• Custom CUDA Kernels: Ingesting 1D or multi-dimensional custom CUDA kernels.
• Flow Control: Controlling dynamic execution flow (loops and conditional paths).
• Subgraphs: Composing fragments using nested subgraphs.

To run inference within a Holoscan application, use the built-in InferenceOp (Inference operator). The behavior and specific use cases of this operator are determined by a parameter set defined in your application's configuration file.

If the built-in InferenceOp does not meet your requirements, you can implement a custom inference operator.

Key components:

• InferenceOp: The built-in operator used to execute inference tasks.
• Inference Module: The underlying framework that provides the core inference functionality and APIs used by InferenceOp.
• Parameter Set: A collection of parameters defined in the application configuration file that controls the Inference Module's behavior.

The fastdds_example application uses DDS types defined in IDL files. These types (such as CudaBuffer) are generated at CMake build time and are located in ${CMAKE_CURRENT_BINARY_DIR}/gen/ rather than the source tree.

Key IDL files for this example:

• CudaBuffer.idl: Generates CudaBuffer and CudaBufferPubSubType. This is a composite message containing a PointerDescriptor and a size.

Note that CudaBuffer.idl includes pointer_descriptor.idl from the SDK's IDL directory (public/modules/holoipc/idl/). The build system handles this inclusion and ensures that generated C++ code uses the correct SDK include paths (e.g., #include <holoscan/ipc/transport/fastdds/gen/...>) via a post-processing script.

The Clock Resource example demonstrates how to use the scheduler's clock attribute within an application. It simulates a ping application where a receiver operator uses clock methods to manage timing and measure intervals.

Key capabilities demonstrated include:

• Retrieving the current time in seconds.
• Retrieving the current timestamp in nanoseconds.
• Sleeping the compute method for a specific duration.
• Sleeping the compute method until a specific timestamp is reached.

The example architecture consists of two operators:

1. A transmitter: Sends a sequence of integers (1-3) to its out port.
2. A receiver: Prints received values and executes the clock-based timing logic.

The available skills are designed to automate Holoscan SDK workflows. Currently, they focus on installation, helping you navigate and execute supported paths such as:

• Container
• Debian package
• Python wheel
• Conda
• Building from source

The skills help you select the path that best fits your specific platform and then verify the result.

A Scheduler is a critical component in Holoscan that governs the execution of operators within a graph. It orchestrates the execution of all defined operators while tracking their individual execution states and enforcing their associated conditions.

All schedulers inherit from a base class:

• C++: holoscan::Scheduler
• Python: holoscan.core.Scheduler

Every scheduler provides access to a clock via the clock() method (C++: holoscan::Scheduler::clock, Python: holoscan.core.Scheduler.clock). This clock is used to retrieve time, timestamps, or to handle sleeping. By default, all schedulers use the holoscan::RealtimeClock (C++: holoscan::RealtimeClock, Python: holoscan.resources.RealtimeClock).

Data Flow Tracking is a profiling mechanism used to analyze fine-grained timing properties and data flow between operators in a fragment's graph. It allows you to retrieve maximum (worst-case), average, and minimum end-to-end latencies for paths, as well as the number of messages sent from root operators.

Key Concepts:

• Root Operator: An operator with no predecessor nodes.
• Leaf Operator (Sink): An operator with no successor nodes.
• Supported Paths: Currently, tracking is only supported between root and leaf operators, or within simple cycles in a graph.
• End-to-End Latency (Root to Leaf): The time elapsed from the start of a root operator to the end of a leaf operator.
• End-to-End Latency (Cyclic): The time elapsed from the start of the first operator in a cycle to when a message is again received by that same operator.