PVCNN (Point-Voxel CNN)

The core implementation of PVConv is located in modules/pvconv.py. The operation follows a specific pattern: it voxelizes features and coordinates, applies voxel layers, performs trilinear devoxelization to return to point space, and finally fuses the voxel features with point-based features.

voxel_features, voxel_coords = voxelize(features, coords)
voxel_features = voxel_layers(voxel_features)
voxel_features = trilinear_devoxelize(voxel_features, voxel_coords, resolution)
fused_features = voxel_features + point_layers(features)

To train a model, run train.py with a specific configuration file and specify the target devices using --devices.

Note: Accuracy and IoU metrics reported during training are rough estimations. For accurate evaluation, you must run the script again with the --evaluate flag after training is complete.

# General training command
python train.py [config-file] --devices [gpu-ids]

# Example: Train PVCNN on S3DIS Area 5 (holding out Area 5)
python train.py configs/s3dis/pvcnn/area5/c1.py --devices 0,1

To prepare the KITTI dataset for the Frustum-PointNet backbone:

1. Download the ground truth labels from here.
2. Unzip the labels and move them to the expected directory structure.
3. Run the download script for the Frustum data.

unzip data_object_label_2.zip
mv training/label_2 data/kitti/ground_truth
./data/kitti/frustum/download.sh

To prepare the S3DIS dataset, follow these steps:

1. Download the dataset from the official source.
2. Run the preprocessing script provided in the repository, pointing to your unzipped dataset directory.

python data/s3dis/prepare_data.py -d [path to unzipped dataset dir]

To prepare the ShapeNet dataset, use the provided download script.

./data/shapenet/download.sh

Before using PVCNN, ensure your environment meets the following requirements:

• Python: >= 3.7
• PyTorch: >= 1.3
• Other dependencies: numba, numpy, scipy, six, tensorboardX (>= 1.2), tqdm, plyfile, and h5py.

Detailed dependency information can be found in the requirements.txt file.

When evaluating on the KITTI dataset using the Frustum backbone, you can specify the number of measurements to mitigate the effects of random sampling. Use the --configs.evaluate.num_tests flag.

python train.py configs/kitti/frustum/pvcnne.py --devices 0 --evaluate --configs.evaluate.best_checkpoint_path kitti.frustum.pvcnne.pth.tar --configs.evaluate.num_tests [#measurements]

To evaluate a downloaded pretrained model, use the train.py script with the --evaluate flag. You must provide a configuration file and the path to the checkpoint via --configs.evaluate.best_checkpoint_path.

# Example: Evaluate PVCNN on GPU 0,1 for S3DIS Area 5
python train.py configs/s3dis/pvcnn/area5.py --devices 0,1 --evaluate --configs.evaluate.best_checkpoint_path s3dis.pvcnn.area5.c1.pth.tar