DepthAI Core

The PointCloud node can re-express output points in different coordinate frames using three primary methods:

1. Camera Board Socket

Uses extrinsic calibration between cameras. This is the preferred method for using calibrated translations between specific sockets.

Available sockets: CAM_A through CAM_J.

// Example: Transform to CAM_A coordinate system
pc->setTargetCoordinateSystem(dai::CameraBoardSocket::CAM_A);

Note: The legacy setTargetCoordinateSystem(socket, useSpecTranslation) overload is deprecated.

2. Housing Coordinate System

Uses housing calibration stored on the device. This is useful for aligning point clouds with the physical device housing or IMU.

Available targets include:

• CAM_A through CAM_J
• FRONT_CAM_A through FRONT_CAM_J
• VESA_A through VESA_J
• IMU

// Example: Transform to VESA_A coordinate system
pc->setTargetCoordinateSystem(dai::HousingCoordinateSystem::VESA_A);

3. Custom 4×4 Matrix

Apply any homogeneous transform via initialConfig or the runtime inputConfig queue. If a custom matrix is used in conjunction with a Camera Board Socket or Housing Coordinate System, the custom matrix is applied after the calibration-derived transform.

std::array<std::array<float, 4>, 4> mat = {{ 
    {{ 0.f, -1.f, 0.f, 0.f }}, 
    {{ 1.f,  0.f, 0.f, 0.f }}, 
    {{ 0.f,  0.f, 1.f, 0.f }}, 
    {{ 0.f,  0.f, 0.f, 1.f }}, 
}};
pc->initialConfig->setTransformationMatrix(mat);

// Example of applying a custom 4x4 transformation matrix
std::array<std::array<float, 4>, 4> mat = {{ 
    {{ 0.f, -1.f, 0.f, 0.f }}, 
    {{ 1.f,  0.f, 0.f, 0.f }}, 
    {{ 0.f,  0.f, 1.f, 0.f }}, 
    {{ 0.f,  0.f, 0.f, 1.f }}, 
}};
pc->initialConfig->setTransformationMatrix(mat);

A PipelineEvent is a data structure used to track various operations within a node. Events are generated via the PipelineEventDispatcher of a node.

Event Types

• CUSTOM: User-defined events for tracking specific operations (e.g., custom timings). Must be added manually.
• LOOP: Tracks main processing loop timings. Can track one loop per node. Generated via mainLoop in simple nodes or tracked events in threaded nodes.
• INPUT: Automatically tracks input queue operations and states.
• OUTPUT: Automatically tracks output queue operations and states.
• INPUT_BLOCK: Tracks a group of input operations. Only one block can be tracked per node; must be added manually.
• OUTPUT_BLOCK: Tracks a group of output operations. Only one block can be tracked per node; must be added manually.

Creating Events

• Manual: Use startEvent and endEvent methods on the dispatcher.
• Automatic (Block): Use the BlockPipelineEvent helper class (created via the blockEvent method) to automatically trigger a start event in the constructor and an end event in the destructor.

// Conceptual C++ usage for manual event tracking
// (Based on PipelineEvent class structure)
// node.getDispatcher().startEvent(type, source);
// ... code ...
// node.getDispatcher().endEvent(type, source);

The DynamicCalibration node produces several key data structures for monitoring and evaluating calibration quality.

CoverageData (coverageOutput)

Used to monitor how much of the field of view has been adequately observed.

• meanCoverage: float [0–1] (overall spatial coverage).
• dataAcquired: float [0–1] (amount of calibration-relevant data gathered).
• coveragePerCell: dict[CameraBoardSocket, list[list[float]]] (per-sensor coverage matrices).

DynamicCalibrationResult (calibrationOutput)

Returned after calibrate() or startCalibration().

• calibrationData.newCalibration: dai.CalibrationHandler containing the updated parameters.
• calibrationData.calibrationDifference: Quality deltas between current and new calibration:
  • pairwiseRotationDifference: dict[tuple[CameraBoardSocket, CameraBoardSocket], list[float]] (per-sensor-pair rotation deltas in degrees). This is the preferred field for rotation deltas.
  • sampsonErrorCurrent: float (reprojection error proxy in pixels for current calibration).
  • sampsonErrorNew: float (reprojection error proxy in pixels for the new calibration).
• info: string (human-readable status, e.g., "success").

Deprecated Fields

• CalibrationQuality (qualityOutput): Legacy command; current runtimes return empty results and log warnings. Use DynamicCalibrationResult instead.
• rotationChange: Legacy aggregate rotation delta. Use pairwiseRotationDifference instead.
• depthErrorDifference[4]: Deprecated; do not rely on this in new code.

The dai::node::DynamicCalibration node consumes raw, unrectified mono frames (typically NV12 full-res) to perform live stereo rig calibration. It provides coverage feedback and produces either a new CalibrationHandler to apply to the device or a quality report comparing current vs. achieved calibration.

Core Workflow

1. Setup: Link unrectified mono cameras to DynamicCalibration.left/right and StereoDepth.left/right.
2. Initiate: Send DynamicCalibrationControl::Commands::StartCalibration{} to begin the collection/solve process.
3. Monitor: Poll coverageOutput for spatial coverage progress and calibrationOutput for results.
4. Apply: When a result arrives, apply the new parameters using DynamicCalibrationControl::Commands::ApplyCalibration{result.calibrationData->newCalibration}.

// Example flow snippet
dai::node::DynamicCalibrationControl::Commands::StartCalibration{} 
// ... in loop ...
if (result.calibrationData) {
    controlQueue.send(DynamicCalibrationControl::Commands::ApplyCalibration{result.calibrationData->newCalibration});
}