DOCSEARCHbyAlgolia
Submit Docs

StandardCyborgCocoa SDK

repository·master·Indexed 41 days ago

https://github.com/standardcyborg/standardcyborgcocoa

An SDK for real-time 3D scanning on iOS using the TrueDepth camera. It provides frameworks for 3D reconstruction, ML-based anatomical detection for feet and ears, and UI components for scanning visualization.

Tokens
14.5K
Snippets
35
Records
75
Agent score
51%

What's inside StandardCyborgCocoa

  2. Overview of JSON for Modern C++

    master

    JSON for Modern C++ is a header-only C++ library designed to make JSON feel like a first-class data type in C++, similar to how it feels in Python. It focuses on intuitive syntax using modern C++ operator overloading and trivial integration.

    Key characteristics:

    • Single Header Integration: The entire library consists of a single header file json.hpp. It has no dependencies and requires no complex build system or adjustment to compiler flags.
    • C++11 Standard: Written in vanilla C++11.
    • High Test Coverage: Heavily unit-tested with 100% coverage, including checks for memory leaks via Valgrind/Clang Sanitizers and continuous fuzz testing via Google OSS-Fuzz.
    • Data Types: By default, it uses std::string for strings, int64_t/uint64_t/double for numbers, std::map for objects, std::vector for arrays, and bool for Booleans.

  3. Overview of StandardCyborgFusion

    master

    StandardCyborgFusion is a framework that implements the PBFusion 3D reconstruction algorithm. It takes depth and color frames as input to reconstruct a 3D pointcloud.

    Key components include:

    • Real-time reconstruction: Handled by SCReconstructionManager using live device frames.
    • Offline reconstruction: Handled by SCOfflineReconstructionManager using a sequence of .bply files.
    • Depth processing: MetalDepthProcessor is a central class that takes raw depth, color frames, and camera settings to output an unprojected point cloud where each point is assigned a color, normal, and other values required by PBFusion.

  5. Overview of scsdk components

    master

    The scsdk/ directory contains the core components of the highly portable C++ library for 3D perception. The library is organized into the following modules:

    • math: Integral containers and linear algebra for 3D Perception.
    • sc3d: Core classes for 3D geometry, imaging, selection, and annotation.
    • scene_graph: A portable representation for a complete 3D scene.
    • algorithms: A suite of 3D Perception algorithms that operate on scsdk data structures.
    • io: SERDES (Serialization/Deserialization) supporting JSON, PLY, GLTF, and other file formats.
    • util: Various supporting utilities.

  6. Overview of StandardCyborgCocoa components

    master

    The repository is organized into several key components:

    • StandardCyborgFusion: The core framework for real-time 3D reconstruction via TrueDepth, including ML models for foot/ear detection and ear landmarking.
    • StandardCyborgUI: UI components for driving and visualizing scans using the Fusion framework.
    • StandardCyborgExample: A reference iOS app demonstrating the integration of Fusion and UI.
    • scsdk: A pure C++ framework containing the core reconstruction logic, data structures, and algorithms for scan analysis.
    • CppDependencies: C++ package dependencies required by StandardCyborgFusion.
    • TrueDepthFusion: A comprehensive iOS app used for framework iteration.
    • VisualTesterMac: A macOS app for iterating on StandardCyborgFusion using pre-recorded scan data.

  8. Understand the StandardCyborg PLY file format header

    master

    StandardCyborg uses a specific PLY (Polygon File Format) header structure to store point cloud data, fusion versions, and metadata. The header includes custom comment fields for versioning and metadata, and supports additional elements like gravity alongside standard vertex and face elements.

    Key Header Components:

    • comment StandardCyborgFusionVersion <version>: Specifies the fusion version used.
    • comment StandardCyborgFusionMetadata <JSON>: Contains a JSON string of metadata (e.g., {"color_space": "sRGB"}).
    • element vertex <count>: Defines the number of vertices.
    • element gravity <count>: Defines gravity vector elements.

    Compatibility Note: Some software, such as MeshLab, may impose limits on the length of comment lines. When using StandardCyborgFusionMetadata, ensure the JSON payload is not so large that it exceeds the comment length limits of your intended parser.

    ply
format ascii 1.0
comment StandardCyborgFusionVersion 1.2.0
comment StandardCyborgFusionMetadata { "color_space": "sRGB" }
element vertex 30135
property float x
property float y
property float z
property float nx
property float ny
property float nz
property uchar red
property uchar green
property uchar blue
property float surfel_radius
element face 0
property list uchar int vertex_indices
element gravity 1
property float gravity_x
property float gravity_y
property float gravity_z
end_header

  11. How the SAX interface works

    master

    For high-performance or custom parsing, you can use the SAX (Simple API for XML) interface. Instead of building a full DOM tree, the parser calls specific functions on your handler as it encounters JSON elements.

    Steps to implement:

    1. Implement a class that provides the required SAX functions (e.g., null(), boolean(bool val), start_object(size_t elements), etc.). You can inherit from nlohmann::json_sax<json>.
    2. Create an instance of your handler.
    3. Call json::sax_parse(input, &your_handler).

    Note: sax_parse returns a bool indicating success/failure and does not return a json object; you must manage the data within your handler.