AnyGrasp SDK Documentation

Follow these steps to install the AnyGrasp SDK. Ensure you have the correct environment for your CUDA version.

1. Install PyTorch

Install PyTorch compatible with your CUDA version (11.x, 12.x, or 13.x).

2. Install MinkowskiEngine

AnyGrasp requires a modified version of MinkowskiEngine.

Note for CUDA 12.x users: You must uncomment the git checkout cuda-12-1 line during the installation process.

Note for CUDA 13.x users: You must uncomment the git checkout cuda-13 line.

Note for CUDA 12.8+ users: You may need to apply a patch to shared_ptr_base.h using sed. The exact path to this file depends on your OS (e.g., /usr/include/c++/13/bits/shared_ptr_base.h on Ubuntu 24.04).

3. Install remaining dependencies

Install requirements via pip and the pointnet2 module.

4. Install graspnetAPI

Clone and install the graspnetAPI repository.

# 1. Install MinkowskiEngine (Modified version)
mkdir dependencies && cd dependencies
conda install openblas-devel -c anaconda
export CUDA_HOME=/path/to/cuda
git clone git@github.com:chenxi-wang/MinkowskiEngine.git
cd MinkowskiEngine

# Uncomment if using CUDA 12.x:
# git checkout cuda-12-1

# Uncomment if using CUDA 13.x:
# git checkout cuda-13

# Uncomment if using CUDA 12.8+ (example path for Ubuntu 24.04):
# sed -i 's/\bauto __raw = __to_address(__r.get());/auto __raw = std::__to_address(__r.get());/' /usr/include/c++/13/bits/shared_ptr_base.h

python setup.py install --blas_include_dirs=${CONDA_PREFIX}/include --blas_library_dirs=${CONDA_PREFIX}/lib --blas=openblas
cd ../..

# 2. Install Pip requirements
pip install -r requirements.txt

# 3. Install pointnet2
cd pointnet2
python setup.py install
cd ..

# 4. Install graspnetAPI
git clone https://github.com/graspnet/graspnetAPI.git
cd graspnetAPI
pip install .

To run the AnyGrasp detection demo, you must prepare the environment by providing the correct shared object file (gsnet.so), a license folder, and model weights.

Prerequisites:

1. Python Version: Ensure you are using Python 3.6 or higher.
2. Shared Object File: You must copy the gsnet.*.so file corresponding to your Python version into the root directory and rename it to gsnet.so.
3. License: Unzip your license files into a folder named license in the root directory.
4. Model Weights: Place your model weights in the log/ directory.

Execution: You can run the demo using the provided shell script or by running demo.py directly.

# 1. Copy the correct gsnet.so for your Python version (example for Python 3.6)
cp gsnet_versions/gsnet.cpython-36m-x86_64-linux-gnu.so gsnet.so

# 2. (Manual step) Unzip license into 'license/' folder

# 3. (Manual step) Put model weights under 'log/'

# 4. Run the demo
sh demo.sh

You can guide the detector to focus on specific areas or approach directions using optional_params.

Region Steering Use a boolean mask of the same length as your point cloud to restrict grasping to a specific object or workspace.

# Example: Steering to a specific object via segmentation mask
region_mask = segmentation_labels == target_object_id

grasps = detector.get_grasp(points, {
    "region_steering": region_mask,
    "dense_grasp": True,
    "collision_detection": True,
})

Approach Steering Provide a 3D vector to prefer a specific approach direction (e.g., [0, 0, 1] for top-down). Use approach_thresh to define a cone of acceptable angles.

# Example: Accept grasps within 60 degrees of the [0, 5, 1] direction
grasps = detector.get_grasp(points, {
    "approach_steering": [0, 5, 1],
    "approach_thresh": np.pi / 3,
})

import numpy as np

# Region Steering
region_mask = segmentation_labels == 1
grasps = detector.get_grasp(points, {
    "region_steering": region_mask,
    "dense_grasp": True,
    "collision_detection": True,
})

# Approach Steering
grasps = detector.get_grasp(points, {
    "approach_steering": [0, 5, 1],
    "approach_thresh": np.pi / 3,
})

The core inference method is detector.get_grasp(points, optional_params). It takes a 3D point cloud and returns a graspnetAPI.GraspGroup containing predicted parallel-jaw grasps.

Input Requirements:

• points: A numpy.ndarray of shape (N, 3) and type numpy.float32. These should be XYZ coordinates in meters, typically in the camera coordinate frame.

Optional Parameters (optional_params dict):

• dense_grasp (bool): If True, generates denser predictions (higher coverage, higher runtime).
• collision_detection (bool): If True, filters grasps that collide with the input point cloud.
• region_steering (numpy.ndarray, shape (N,), dtype bool): A mask to select specific points/objects for grasping. Length must match points.
• approach_steering (list or numpy.ndarray, shape (3,)): A preferred 3D approach direction. The SDK normalizes this.
• approach_thresh (float): Maximum angular deviation from the approach direction in radians. Default is numpy.pi.

Output:

• Returns a graspnetAPI.GraspGroup object. Returns None if no grasps are found or if optional_params is invalid.

import numpy as np

# XYZ coordinates in meters, shape (N, 3)
points = np.asarray(point_cloud_xyz, dtype=np.float32)

grasps = detector.get_grasp(points, {
    "dense_grasp": False,
    "collision_detection": True,
    "region_steering": None,
    "approach_steering": None,
    "approach_thresh": np.pi,
})

The output of get_grasp is a GraspGroup object. You can use several built-in methods to refine the results:

• grasps.nms(): Performs Non-Maximum Suppression to remove duplicate grasps.
• grasps.sort_by_score(): Sorts grasps by descending confidence score.
• grasps.to_open3d_geometry_list(): Converts grasps to Open3D geometries for visualization.

Grasp Data Fields: Each grasp in the group contains:

• score (float): Confidence score.
• width (float): Required gripper opening (meters).
• height (float): Gripper finger height (meters).
• depth (float): Grasp insertion depth (meters).
• rotation_matrix (ndarray, 3x3): Orientation in the input frame.
• translation (ndarray, 3): Grasp center in meters.
• object_id (int): Currently always returns -1.

Important: Calculating Gripper Tip Position translation is the grasp center, not the tip. To find the tip position, use: gripper_tip_position = translation + depth * rotation_matrix[:3, 0]

if grasps is not None:
    grasps = grasps.nms()
    grasps = grasps.sort_by_score()
    best_grasp = grasps[0]

# Accessing fields
score = best_grasp.score
translation = best_grasp.translation
rotation_matrix = best_grasp.rotation_matrix
depth = best_grasp.depth

# Calculate tip
gripper_tip_position = translation + depth * rotation_matrix[:3, 0]

To use AnyGrasp, you must first initialize a detector instance using create_detector(config). The config object can be an argparse.Namespace or an easydict.EasyDict.

If license validation fails, the function returns None. You should check for this to avoid runtime errors.

Configuration Parameters:

• checkpoint_path (str): Path to the model checkpoint file.
• max_gripper_width (float): Maximum accepted gripper opening in meters. Must be in the range (0.0, 0.1].
• gripper_height (float): Gripper finger height in meters, used during collision detection.

from argparse import Namespace
from gsnet import create_detector

# Using argparse.Namespace
config = Namespace(
    checkpoint_path="/path/to/checkpoint.tar",
    max_gripper_width=0.10,
    gripper_height=0.03,
)

detector = create_detector(config)
if detector is None:
    raise RuntimeError("Failed to create detector")