KPConv-PyTorch

To perform scene segmentation, you must download and organize the S3DIS dataset.

1. Download Stanford3dDataset_v1.2.zip (4.8 GB) from the provided link.
2. Uncompress the file.
3. Place the uncompressed data in a directory structure where it is accessible relative to your experiment folder. By default, the project expects the data at ../../Data/S3DIS relative to XXXX/Experiments/KPConv-PyTorch.

If you wish to use a different data location, you must modify the self.path variable within the S3DISDataset class in datasets/S3DIS.py.

# Example directory structure assumed by default:
# XXXX/Data/S3DIS
# XXXX/Experiments/KPConv-PyTorch/training_S3DIS.py

When the visualization window is open, use the following keyboard shortcuts and mouse actions to interact with the point clouds and kernels:

Keyboard Shortcuts

• 'b' / 'n': Decrease or increase point size.
• 'g' / 'h': Navigate to the previous or next example in the current batch.
• 'k': Toggle between the rigid kernel (original kernel point positions) and the deformed kernel (kernel point positions after shifts are applied).
• 'z': Toggle the displayed points (input points, current layer points, or both).
• '0': Save the current example and the deformed kernel as .ply files.

Mouse Actions

• Left Click: Select a specific point to show the kernel at its location.

Window Management

• Exit Window: Closes the current view and computes the next batch of examples.

To perform object classification on ModelNet40, you must organize your data directory relative to your experiment folder.

1. Download the regularly sampled clouds from ModelNet40: https://shapenet.cs.stanford.edu/media/modelnet40_normal_resampled.zip.
2. Uncompress the data.
3. Place the uncompressed data in ../../Data/ModelNet40 (assuming your experiment folder is at XXXX/Experiments/KPConv-PyTorch).

If you need to use a different data location, modify the self.path variable in the ModelNet40Dataset class within datasets/ModelNet40.py.

Training results are saved in a results folder. Each training session creates a dated log folder containing loss values, validation metrics, and model checkpoints.

To visualize the training progress, use plot_convergence.py. You must follow the comments within the script to select the specific training log you wish to plot.

python3 plot_convergence.py

To set up KPConv-PyTorch on Windows 10, ensure you have CUDA and cuDNN installed. A tested configuration is PyTorch 1.4.0, CUDA 10.1, and cuDNN 7.5.

Follow these steps:

1. Install PyTorch following the official PyTorch installation procedure.
2. Install Python dependencies (e.g., via PyCharm venv or pip):
   • torch
   • torchvision
   • numpy
   • scikit-learn
   • PyYAML
   • matplotlib (for visualization)
   • mayavi (for visualization)
   • PyQt5 (for visualization)
3. Compile the C++ extension modules by executing the .bat files in the cpp_wrappers directory.

# Execute build scripts for C++ extensions
cpp_wrappers\cpp_neighbors\build.bat
cpp_wrappers\cpp_subsampling\build.bat

The repository provides scripts and instructions for three main types of experiments located in the doc/ directory:

• Object Classification: Instructions to train KP-CNN on the ModelNet40 dataset (doc/object_classification_guide.md).
• Scene Segmentation: Instructions to train KP-FCNN on the S3DIS dataset (doc/scene_segmentation_guide.md).
• SLAM Segmentation: Instructions to train KP-FCNN on the SemanticKitti dataset (doc/slam_segmentation_guide.md).

Start the training process by running the training_ModelNet40.py script.

This script uses a ModelNet40Config subclass (inherited from Config in utils/config.py). You can modify any parameter by editing this subclass.

Note: The first time you run the script, it will precompute structures for the dataset, which may take some time.

python3 training_ModelNet40.py

To visualize kernel deformations, you must have a dataset and a pretrained model that utilizes deformable KPConv. The visualization is performed by running a script that executes a forward pass on a batch of test examples and displays them in an interactive window.

Run the following command to start the visualization:

python3 visualize_deformations.py

Start the training process by running the training_SemanticKitti.py script.

Configuration: You can modify training parameters by editing the SemanticKittiConfig configuration subclass.

Note: The first execution may take a significant amount of time as the script precomputes dataset structures.

python3 training_SemanticKitti.py