VDBFusion

repository·main·Indexed 20 days ago

A utility library for flexible and efficient Truncated Signed Distance Function (TSDF) integration using the VDB data structure. It is designed to reconstruct 3D surfaces from range sensor data, such as point clouds and poses, providing both C++ and Python APIs. Key features include VDBVolume management, point cloud and VDB grid integration, triangle mesh extraction, and TSDF grid pruning.

Tokens: 4.4K
Snippets: 21
Records: 22
Agent score: 71%

What's inside vdbfusion

Implement a Dataset for VDBFusion
main
While not mandatory, providing a dataset object simplifies data loading. The API expects a collection of scans where each item provides a point cloud and its corresponding sensor origin in the global coordinate frame.
Python Implementation
Your class should implement __len__ and __getitem__. __getitem__ must return a tuple containing:
1. A PointCloud as a np.array(N, 3)
2. The sensor origin as an Eigen::Vector3d (or equivalent in the global coordinate frame).
C++ Implementation
Your class should implement size() and operator[]. The operator must return a std::tuple<Cloud, Point> where Cloud is a std::vector<Eigen::Vector3d> and Point is the sensor origin.
```
class Dataset:
    def __init__(self, *args, **kwargs):
        # Initialize your dataset here ..

    def __len__(self) -> int:
        return len(self.n_scans)

    def __getitem__(self, idx: int):
        # Returns a PointCloud(np.array(N, 3))
        # and sensor origin(Eigen::Vector3d)
        # in the global coordinate frame.
        return points, origin
```
https://github.com/prbonn/vdbfusion/blob/main/README.md

Run the TSDF Fusion pipeline

main

The core workflow involves initializing a VDBVolume with specific parameters and then integrating individual scans into it.

Parameters

voxel_size: The size of the voxels in the VDB structure.
sdf_trunc: The truncation distance for the Signed Distance Function.
space_carving: A boolean flag to enable/disable space carving.

Python Usage

import vdbfusion

vdb_volume = vdbfusion.VDBVolume(voxel_size, sdf_trunc, space_carving)
dataset = Dataset(...)

for scan, origin in dataset:
    vdb_volume.integrate(scan, origin)

C++ Usage

#include "vdbfusion/VDBVolume.h"

vdb_fusion::VDBVolume vdb_volume(voxel_size, sdf_trunc, space_carving);
const auto dataset = Dataset(...);

for (const auto& [scan, origin] : iterable(dataset)) {
  vdb_volume.Integrate(scan, origin);
}

import vdbfusion

vdb_volume = vdbfusion.VDBVolume(voxel_size,
                                 sdf_trunc,
                                 space_carving)
dataset = Dataset(...)

for scan, origin in dataset:
    vdb_volume.integrate(scan, origin)

Install vdbfusion from source for C++
main
If you intend to use the C++ API, you must build the project from source. Refer to the INSTALL.md file in the repository for detailed instructions.
https://github.com/prbonn/vdbfusion/blob/main/README.md
Install VDBFusion via pip
main
For most users, the simplest way to install VDBFusion and its Python bindings is via pip.
```
pip install vdbfusion
```
https://github.com/prbonn/vdbfusion
Build VDBFusion from source on Linux
main
The Linux build (superbuild) includes tools to pull 3rdparty dependencies automatically.
Minimal Setup (Ubuntu/Debian)
Install the necessary system tools:
```
sudo apt-get update && sudo apt-get install build-essential cmake git python3 python3-dev python3-pip
```
To install the Python bindings, run make install from the project root.
Note on C++ Bindings: The superbuild does not currently support installing C++ bindings. To install the C++ API on Linux, you must manually install all 3rdparty dependencies (OpenVDB, Eigen3, pybind11) first, then follow the Installing the C++ API instructions.
```
# Install Python bindings from root
make install
```
https://github.com/prbonn/vdbfusion

Build VDBFusion from source in a Conda environment

main

To build from source on MacOS or Linux using Conda (recommended with MambaForge), follow these steps to set up the environment, install dependencies, build OpenVDB, and finally build the VDBFusion C++ API.

1. Create and activate environment

conda create -n vdbfusion python=3.9
conda activate vdbfusion

2. Install dependencies

mamba install cmake llvm ccache ninja pkg-config blosc boost eigen tbb tbb-devel pytest numpy black pybind11 twine

3. Clone repository

git clone https://github.com/PRBonn/vdbfusion.git && cd vdbfusion

4. Build OpenVDB from source

Note: This uses a specific branch required for VDBFusion.

git clone --depth 1 https://github.com/nachovizzo/openvdb.git -b nacho/vdbfusion \
    && cd openvdb \
    && mkdir build && cd build \
    && cmake \
    -DOPENVDB_BUILD_PYTHON_MODULE=ON \
    -DUSE_NUMPY=ON \
    -DPYOPENVDB_INSTALL_DIRECTORY="/usr/local/lib/python3.9/dist-packages" \
    -DCMAKE_POSITION_INDEPENDENT_CODE=ON \
    -DUSE_ZLIB=OFF \
    ..\" \
    && make -j$(nproc) all install \
    && cd ../.. \
    && rm -rf /openvdb

5. Build C++ API

From the root of the vdbfusion repository:

mkdir -p build && cd build && cmake ..
make install

Troubleshooting TBB: If cmake cannot find TBB, set the TBB_ROOT flag to your conda environment path: -DTBB_ROOT=/path/to/conda/envs/vdbfusion

# Example of setting TBB_ROOT if cmake fails
cmake .. -DTBB_ROOT=/path/to/conda/envs/vdbfusion

Run the TSDF fusion pipeline with VDBVolume

main

To perform TSDF integration, initialize a VDBVolume object with your desired voxel size and SDF truncation parameters, then iterate through your dataset to integrate individual scans using their corresponding poses.

Parameters

voxel_size: The size of the voxels in the volume.
sdf_trunc: The truncation distance for the Signed Distance Function.
space_carving: Boolean flag to enable/disable space carving.

Usage Example

from vdbfusion import VDBVolume

# Initialize the volume
vdb_volume = VDBVolume(voxel_size=0.1, sdf_trunc=0.3, space_carving=False)

# Integrate scans from a dataset
for scan, pose in tqdm(dataset):
    vdb_volume.integrate(scan, pose)

from vdbfusion import VDBVolume

# Create a VDB Volume to integrate scans
vdb_volume = VDBVolume(voxel_size=0.1, sdf_trunc=0.3, space_carving=False)

# You need to define your own Dataset.
dataset = KITTIOdometryDataset(kitti_root_dir="./kitti-odometry/dataset/", sequence=0)

for scan, pose in tqdm(dataset):
    vdb_volume.integrate(scan, pose)

Install vdbfusion
main
Install the vdbfusion library using pip. You can verify the installation by checking the version and the file path.
```
%pip install vdbfusion
import vdbfusion
print(f"VDBFusion version {vdbfusion.__version__} properly installed at {vdbfusion.__file__}")
```
https://github.com/prbonn/vdbfusion/blob/main/examples/notebooks/kitti_odometry.ipynb
Build VDBFusion using Docker Compose
main
VDBFusion provides Docker Compose configurations to build the project components. There are two primary services available for building:
1. builder: Uses the Dockerfile located at docker/builder/Dockerfile to build the core project image.
2. pip_builder: Uses the Dockerfile located at docker/pip/Dockerfile to build a Python-specific environment (pip builder).
You can use these services to ensure a consistent build environment for the C++ and Python components.
```
version: "3.4"
services:
  builder:
    image: gitlab.ipb.uni-bonn.de:4567/ipb-team/ipb-tools/vdbfusion:latest
    build:
      context: .
      dockerfile: docker/builder/Dockerfile
  pip_builder:
    image: gitlab.ipb.uni-bonn.de:4567/ipb-team/ipb-tools/vdbfusion/pip_builder:latest
    build:
      context: .
      dockerfile: docker/pip/Dockerfile
```
https://github.com/prbonn/vdbfusion/blob/main/docker-compose.yml

Extract and visualize a triangle mesh

main

Once the fusion pipeline is complete, you can extract a triangle mesh from the VDBVolume for visualization or further processing.

Python with Open3D

To use the Python example, ensure you have installed Open3D (pip install open3d).

import open3d as o3d

# Extract triangle mesh (numpy arrays)
vert, tri = vdb_volume.extract_triangle_mesh()

# Visualize the results
mesh = o3d.geometry.TriangleMesh(
    o3d.utility.Vector3dVector(vert),
    o3d.utility.Vector3iVector(tri),
)

mesh.compute_vertex_normals()
o3d.visualization.draw_geometries([mesh])

C++ with Open3D

#include <open3d/Open3D.h>

// Extract triangle mesh (Eigen).
auto [verts, tris] = vdb_volume.ExtractTriangleMesh();

// Visualize the results
auto mesh = o3d::geometry::TriangleMesh(
    verts,
    tris,
)

mesh.ComputeVertexNormals()
o3d::visualization::DrawGeometries({&mesh})

Install the VDBFusion C++ API
main
The C++ API is currently supported for development builds where all 3rdparty dependencies are already installed locally.
Build and Install
```
mkdir -p build && cd build && cmake ..
sudo make install
```
Use in CMake projects
Once installed, you can consume VDBFusion in your own CMake projects using find_package:
```
find_package(VDBFusion REQUIRED)
add_executable(my_example my_example.cpp)
target_link_libraries(my_examples PRIVATE VDBFusion::vdbfusion)
```
https://github.com/prbonn/vdbfusion
Update TSDF values manually
main
The update_tsdf method allows for direct manipulation of the TSDF values at specific voxel indices.
- sdf: The signed distance value to set.
- ijk: A numpy.ndarray representing the voxel index.
- weighting_function (optional): A callable to determine the weight of the update.
```
# Example: Updating a single voxel
ijk = np.array([10, 20, 30])
volume.update_tsdf(sdf=0.5, ijk=ijk)
```
https://github.com/prbonn/vdbfusion/blob/main/src/vdbfusion/pybind/vdb_volume.py

VDBFusion

What's inside vdbfusion

Python Implementation

C++ Implementation

Parameters

Python Usage

C++ Usage

Minimal Setup (Ubuntu/Debian)

1. Create and activate environment

2. Install dependencies

3. Clone repository

4. Build OpenVDB from source

5. Build C++ API

Parameters

Usage Example

Python with Open3D

C++ with Open3D

Build and Install

Use in CMake projects