DOCSEARCHbyAlgolia
Submit Docs

Point Cloud Utils

repository·master·Indexed 23 days ago

https://github.com/fwilliams/point-cloud-utils

A Python library (pcu) for processing and manipulating 3D point clouds and triangle meshes. It provides tools for file I/O (PLY, STL, OBJ, etc.), mesh sampling, voxel grid and blue noise downsampling, geometric queries via nanoflann and embree, and distance metrics including Hausdorff, Chamfer, and Wasserstein. Key features include signed distance calculations, mesh smoothing, decimation, watertight manifold conversion, and Morton code utilities.

Tokens
16.7K
Snippets
39
Records
63
Agent score
81%

What's inside point-cloud-utils

  1. Overview of Point Cloud Utils functionality

    master

    Point Cloud Utils (pcu) is a Python library for processing and manipulating 3D point clouds and triangle meshes. Key capabilities include:

    • File I/O: Reading and writing common mesh formats (PLY, STL, OFF, OBJ, 3DS, VRML 2.0, X3D, COLLADA).
    • Mesh Sampling: Poisson-Disk-Sampling, Lloyd's algorithm, and Monte-Carlo sampling.
    • Downsampling: Voxel grid downsampling and blue noise distribution sampling.
    • Geometric Queries: Closest points between clouds and meshes, normal estimation, and fast k-nearest-neighbor search (via nanoflann).
    • Distance Metrics: Hausdorff distances, Chamfer distances, and Approximate Wasserstein distances (via Sinkhorn method).
    • Mesh Processing: Signed distances (via Fast Winding Numbers), mesh smoothing, connected components, decimation, and making meshes watertight.
    • Intersections: Fast ray/mesh and ray/surfel intersection (via embree).
    • Cleanup: Deduplicating point clouds and mesh vertices, and removing unreferenced vertices.

  2. How mesh-surface samples and barycentric coordinates work

    master

    Point Cloud Utils represents samples on a mesh surface using Barycentric Coordinates. Instead of storing raw 3D coordinates, a surface sample is encoded as a pair:

    1. fid: The index of the mesh face containing the sample.
    2. bc: The barycentric coordinates of the point within that face.

    This encoding is used because it allows you to interpolate any quantity stored at the mesh vertices (such as positions, normals, or colors) to the specific sample positions.

    To recover or sample vertex quantities at these positions, use: pcu.interpolate_barycentric_coords(f, fid, bc, vertex_quantity)

  3. How normal orientation works with sensor directions

    master

    Point clouds from 3D sensors often lack surface normals. While point-cloud-utils can estimate normals using PCA, the orientation (which way the normal points) may be inconsistent if sensor data is not used.

    To ensure consistent orientation, provide view_dirs (unit vectors pointing from the point to the scanner origin). The estimation functions will then flip the fitted plane normal so that it aligns with the sensor direction.

  4. How mesh-surface samples are represented in Point Cloud Utils

    master

    Point Cloud Utils represents samples on a mesh surface using Barycentric Coordinates rather than absolute 3D positions. This encoding allows you to interpolate any vertex attribute (like positions, normals, or colors) to the exact sample location.

    Each sample is returned as a pair:

    1. fid: The index of the mesh face containing the sample.
    2. bc: The barycentric coordinates of the point within that face.

    To convert these coordinates back into actual 3D positions or vertex attributes, use the pcu.interpolate_barycentric_coords function.

  5. Recover vertex attributes using Barycentric Coordinates

    master

    Point-Cloud-Utils represents mesh-surface samples using Barycentric Coordinates. Instead of storing raw XYZ coordinates for every intersection, it returns:

    1. fid: The index of the mesh face containing the sample.
    2. bc: The barycentric coordinates within that face.

    This encoding allows you to interpolate any quantity stored at the mesh vertices (such as positions, normals, or colors) to the exact sample positions on the surface.

    To recover these quantities, use: pcu.interpolate_barycentric_coords(f, fid, bc, vertex_quantity)

    Where:

    • f: Mesh faces.
    • fid: Array of face indices.
    • bc: Array of barycentric coordinates.
    • vertex_quantity: The attribute you want to sample (e.g., the vertex positions v).

  7. How 3D data is represented in Point Cloud Utils

    master

    Point Cloud Utils uses NumPy arrays as its fundamental data structure.

    • Point Clouds: Represented as a NumPy array of shape (#p, 3), where #p is the number of points.
    • Triangle Meshes: Represented by a pair of NumPy arrays v (vertices) and f (faces).
      • v has shape (#v, 3) containing vertex coordinates.
      • f has shape (#f, 3) containing integer indices into v that define each face.
    • Attributes: Per-vertex or per-face attributes (like normals, colors, or texture coordinates) are stored as separate NumPy arrays with a number of rows matching the number of vertices or faces respectively.

  9. Use shorthand functions to load and save specific mesh attributes

    master

    If you only need specific attributes (like just vertices or just vertices and faces), use the load_mesh_* and save_mesh_* shorthand functions. The file type is automatically inferred from the file extension. If an attribute is missing from the file, these functions return an empty array.

    Available shorthand patterns:

    • load_mesh_v / save_mesh_v: Points only.
    • load_mesh_vf / save_mesh_vf: Vertices and faces.
    • load_mesh_vn / save_mesh_vn: Points and per-vertex normals.
    • load_mesh_vfc / save_mesh_vfc: Vertices, faces, and vertex colors (RGBA).
    • load_mesh_vfn / save_mesh_vfn: Vertices, faces, and vertex normals.
    • load_mesh_vfnc / save_mesh_vfnc: Vertices, faces, vertex normals, and vertex colors (RGBA).

  10. Generate an SDF dataset from ShapeNet meshes

    master

    You can use point_cloud_utils to convert non-watertight ShapeNet models into a dataset of Signed Distance Functions (SDF). The process involves:

    1. Loading the mesh: Use pcu.load_mesh_vf to get vertices and faces.
    2. Creating a watertight manifold: Use pcu.make_mesh_watertight with a specified manifold_resolution. Higher resolution improves quality but increases computation time.
    3. Estimating normals: Use pcu.estimate_mesh_vertex_normals on the new watertight mesh.
    4. Sampling volume points: Generate random points in the bounding volume (ShapeNet models are typically normalized within [-0.5, 0.5]^3) and compute their signed distances using pcu.signed_distance_to_mesh.
    5. Sampling surface points: Use pcu.sample_mesh_random to get face IDs and barycentric coordinates, then use pcu.interpolate_barycentric_coords to compute the actual 3D coordinates and normals at those points.
    6. Saving results: Save the points and SDF values as a .npz file and the watertight mesh as an .obj file using pcu.save_mesh_vfn.
    import os
import numpy as np
import point_cloud_utils as pcu

# Configuration
category_path = "./02828884"
manifold_resolution = 20_000
num_vol_pts = 100_000
num_surf_pts = 100_000

for model_path in os.listdir(category_path):
    # 1. Load mesh
    v, f = pcu.load_mesh_vf(os.path.join(category_path, model_path, "model.obj"))

    # 2. Convert to watertight manifold
    vm, fm = pcu.make_mesh_watertight(v, f, manifold_resolution)
    nm = pcu.estimate_mesh_vertex_normals(vm, fm)

    # 3. Generate volume points and compute SDF
    # ShapeNet shapes are normalized within [-0.5, 0.5]^3
    p_vol = (np.random.rand(num_vol_pts, 3) - 0.5) * 1.1
    sdf, _, _  = pcu.signed_distance_to_mesh(p_vol, vm, fm)

    # 4. Sample surface points
    fid_surf, bc_surf = pcu.sample_mesh_random(vm, fm, num_surf_pts)
    p_surf = pcu.interpolate_barycentric_coords(fm, fid_surf, bc_surf, vm)
    n_surf = pcu.interpolate_barycentric_coords(fm, fid_surf, bc_surf, nm)

    # 5. Save data
    npz_path = os.path.join(category_path, model_path, "samples.npz")
    np.savez(npz_path, p_vol=p_vol, sdf_vol=sdf, p_surf=p_surf, n_surf=n_surf)

    watertight_mesh_path = os.path.join(category_path, model_path, "model_watertight.obj")
    pcu.save_mesh_vfn(watertight_mesh_path, vm, fm, nm)

  11. Consistently orient mesh faces with orient_mesh_faces()

    master

    Meshes often contain inconsistently oriented faces, which leads to flipped normals. You can use pcu.orient_mesh_faces() to ensure that all faces within each connected component of a mesh are oriented consistently.

    This function returns two arrays:

    • f_oriented: A new face array where faces within each connected component are consistently oriented.
    • f_comp: An array of shape [num_faces,] where f_comp[i] identifies the connected component index of the $i^{th}$ face.
    import numpy as np
import point_cloud_utils as pcu

v, f = pcu.load_mesh_vf("truck.ply")

# f_oriented is a new face array where faces within each connected
#   component are consistently oriented
# f_comp is a [num_faces,]-shaped array where f_comp[i] is the
#   connected component of the i^th face
f_oriented, f_comp = pcu.orient_mesh_faces(f)

  12. Filter points with oblique angles to the sensor

    master

    When estimating normals, you can filter out points whose predicted normal is at an oblique angle (close to 90 degrees) relative to the sensor direction. This helps prevent noise during surface reconstruction.

    To use this, pass a drop_angle_threshold (in radians) to estimate_normals_knn or estimate_normals_ball. The function will return n_idx, an array of indices for the points that were not dropped. You can then use these indices to filter your original point cloud.

    import point_cloud_utils as pcu
import numpy as np

pts, sensor_dirs = pcu.load_mesh_vf("point_with_sensor_dirs.ply")

# Threshold in radians (e.g., 85 degrees)
drop_angle = np.deg2rad(85.0)
num_nbrs = 32

# n_idx contains the indices of points that were NOT filtered out
n_idx, n = pcu.estimate_normals_knn(pts, num_nbrs, view_dirs=sensor_dirs, drop_angle_threshold=drop_angle)

# Filter the points using the returned indices
pts_n = pts[n_idx]