DOCSEARCHbyAlgolia
Submit Docs

Orbbec SDK v2

repository·main·Indexed 42 days ago

https://github.com/orbbec/orbbecsdk_v2

An open-source software development kit for interfacing with Orbbec depth cameras, LiDAR, and vision sensors, including the Gemini, Astra, Femto, and Pulsar series.

Tokens
98.6K
Snippets
264
Records
385
Agent score
54%

What's inside Orbbec SDK v2

  1. Overview of spdlog features

    main

    spdlog is a fast C++ logging library with several key capabilities:

    • Performance: Very fast execution and optional asynchronous mode.
    • Formatting: Uses the fmt library for feature-rich formatting and supports custom formatting.
    • Threading: Supports both multi-threaded and single-threaded loggers.
    • Log Targets (Sinks): Supports various targets including:
      • Rotating log files
      • Daily log files
      • Console logging (with color support)
      • syslog
      • Windows event log
      • Windows debugger (OutputDebugString(..))
      • Qt widgets
      • Custom sinks (extensible)
    • Filtering: Log levels can be modified at both runtime and compile time. Supports loading log levels from argv or environment variables.
    • Backtrace Support: Can store debug messages in a ring buffer to be displayed on demand.

  2. Overview of libyuv functionality

    main

    libyuv is an open source library providing optimized YUV scaling and conversion functionality. It is designed for high-performance image processing tasks such as:

    • Scaling: Scaling YUV content using point, bilinear, or box filters to prepare for compression.
    • YUV Conversion: Converting webcam formats to YUV for compression.
    • RGB Conversion: Converting YUV to RGB formats for rendering or applying effects.
    • Rotation: Rotating images by 90, 180, or 270 degrees (useful for mobile device portrait orientation).

    Performance is optimized for several architectures:

    • x86/x64: SSSE3 and AVX2.
    • Arm: Neon.
    • Mips: MSA.

  3. Overview of Orbbec SDK C++ Examples

    main
    The Orbbec SDK provides a comprehensive set of C++ examples categorized by complexity level to help developers implement various camera functionalities. These examples cover everything from basic device enumeration to advanced multi-device synchronization and point cloud generation.

  4. Overview of Orbbec SDK Extensions

    main

    The Orbbec SDK Extensions directory contains binary files for specialized libraries developed by the Orbbec SDK team or partners. While these extensions are not open source, they are provided under the MIT license, allowing you to use them freely in your projects.

    Available extensions include:

    • filters: Private filters library for the Orbbec SDK.
    • frameprocessor: Private frame processor library for the Orbbec SDK.
    • firmwareupdater: Private firmware updater library.
    • depthengine: Private depth engine library specifically for the Femto Bolt device.

  5. Overview of LiDAR support in Orbbec SDK v2

    main

    Orbbec SDK v2.6.2 and later provides support for LiDAR devices, specifically the Pulsar ME450 and SL450. The SDK enables:

    • Device Management: Easy discovery and configuration of LiDAR hardware.
    • Data Acquisition: Real-time point cloud capture and synchronized IMU measurements.
    • Streaming & Recording: Real-time data streaming, as well as tools for recording and playing back LiDAR data.

    This support is intended for applications in robotics, mapping, and automation.

  6. Navigate the OrbbecViewer interface

    main

    The OrbbecViewer interface is divided into three primary functional areas:

    1. Equipment management area (Area 1): Displays the list of connected sensors.
    2. Control panel area (Area 2): Used for data stream switching, parameter configuration (depth, IR, color, IMU), image acquisition, and D2C control.
    3. Image preview area (Area 3): Displays sensor image previews, frame rates, and timestamps. You can click the log information at the bottom of this area to view detailed logs.

    Control Panel Tabs:

    • Single Camera Mode: Basic controls for a single sensor.
    • Advanced Mode: Provides deeper configuration, including D2C, Point Cloud, and frame synchronization settings.

  9. Calculate real-world distance from a DepthFrame

    main

    When working with a depth stream in OB_FORMAT_Y16 format, the raw pixel values represent depth data that must be scaled to get the actual distance.

    To calculate the distance in millimeters:

    1. Cast the raw frame to an ob::DepthFrame using as<ob::DepthFrame>().
    2. Retrieve the valueScale using depthFrame->getValueScale().
    3. Multiply the raw 16-bit pixel value by this scale.

    Note: If the calculated distance is 0, the camera cannot detect an object (it may be out of the detection range).

    // Get the depth Frame from depthFrameRaw.
auto depthFrame = depthFrameRaw->as<ob::DepthFrame>();

// for Y16 format depth frame, print the distance of the center pixel every 30 frames.
if(depthFrame->getIndex() % 30 == 0 && depthFrame->getFormat() == OB_FORMAT_Y16) {
    uint32_t        width  = depthFrame->getWidth();
    uint32_t        height = depthFrame->getHeight();
    float           scale  = depthFrame->getValueScale();
    const uint16_t *data   = reinterpret_cast<const uint16_t *>(depthFrame->getData());

    // pixel value multiplied by scale is the actual distance value in millimeters.
    float centerDistance = data[width * height / 2 + width / 2] * scale;

    // attention: if the distance is 0, it means that the depth camera cannot detect the object
    win.addLog("Facing an object at a distance of " + ob_smpl::toString(centerDistance, 3) + " mm. ");
}

  10. C API Error Handling and Memory Management

    main

    When using the Orbbec SDK C API, developers must follow specific patterns for resource lifecycle and error reporting:

    Error Handling

    Most functions require an ob_error *error parameter. This structure is used to provide comprehensive error reporting for every call. Always check the error state after calling SDK functions.

    Memory Management

    Unlike the C++ API, the C API requires explicit manual deletion of all allocated objects to avoid memory leaks:

    • Use ob_delete_frame(frame, &error) to release frames and framesets.
    • Use ob_delete_pipeline(pipe, &error) to release the pipeline.

    Timeout Handling

    Functions like ob_pipeline_wait_for_frameset include a timeout parameter (in milliseconds) to prevent the application from blocking indefinitely if no data is received.

  11. How Pipeline and Frameset work in the C API

    main

    The Orbbec SDK C API uses two primary abstractions to manage data streams:

    • Pipeline: A central object used to manage data stream processing. It handles multi-channel stream configuration, stream switching, frame aggregation, and frame synchronization.
    • Frameset: A container object that represents a synchronized combination of different frame types (e.g., color and depth frames) captured at a specific point in time.

  12. Choose between TurboJPEG and libjpeg APIs

    main

    libjpeg-turbo provides two distinct APIs for JPEG compression and decompression:

    • TurboJPEG API: A straightforward, easy-to-use interface for in-memory operations. It is recommended for first-time users. It supports advanced features like generating planar YUV images and performing multiple simultaneous lossless transforms. The Java interface is built on top of this API.
    • libjpeg API: The industry-standard API. It is more powerful but more complex to use. It is API/ABI-compatible and mathematically compatible with libjpeg v6b. It can optionally be configured to emulate libjpeg v7 or v8.

    There is no significant performance difference between the two APIs when performing similar operations.

    /* Refer to tjexample.c and TJExample.java for TurboJPEG usage examples. */
/* Refer to cjpeg.c and djpeg.c for libjpeg API usage examples. */