Minkowski Engine Documentation

A Sparse Tensor in Minkowski Engine is a high-dimensional extension of a sparse matrix using the COOrdinate (COO) format. It consists of:

• Coordinates ($C$): A matrix of size $N imes D$, where $N$ is the number of non-zero elements and $D$ is the dimension of the space. Unlike traditional sparse tensors, Minkowski Engine supports negative coordinates.
• Features ($F$): A matrix of size $N imes N_F$, where $N_F$ is the number of channels.

All other elements in the tensor are implicitly zero. A $D$-dimensional sparse tensor is a rank-$D$ tensor if features are scalars, or rank-$D+1$ if features are vectors.

In Minkowski Engine, Tensor Stride is the high-dimensional counterpart to 2D strides. It represents the distance between neurons in the feature map.

• When using pooling or convolution layers with a stride $> 1$, the tensor stride of the output feature map increases by the factor of the layer's stride.
• Using transposed convolutions (deconv, upconv) reduces the stride.

You can install the stable version of MinkowskiEngine from PyPI using pip3.

Requirements:

• Ubuntu 14.04 or higher
• CUDA 10.1 or higher (for CUDA acceleration)
• PyTorch 1.3 or higher
• Python 3.6 or higher
• GCC 6 or higher
• openblas, python3-dev, torch, and numpy packages.

pip3 install -U MinkowskiEngine

To install directly on your system Python, ensure you have the build dependencies and that your PyTorch CUDA version matches your system nvcc version.

sudo apt install build-essential python3-dev libopenblas-dev
curl https://bootstrap.pypa.io/get-pip.py | python3
python3 -m pip install torch numpy ninja

git clone https://github.com/NVIDIA/MinkowskiEngine.git
cd MinkowskiEngine
python setup.py install

To specify custom parameters during system installation:

export CXX=c++; export CUDA_HOME=/usr/local/cuda-11.1; python setup.py install --blas=openblas --force_cuda

To create a MinkowskiEngine.SparseTensor, you must provide coordinates that include batch indices. This results in a tensor with $D+1$ dimensions if the original spatial coordinates were $D$-dimensional. You can use MinkowskiEngine.utils.sparse_collate to combine coordinates and features from multiple batches into a single sparse tensor.

coords0, feats0 = to_sparse_coo(data_batch_0)
coords1, feats1 = to_sparse_coo(data_batch_1)
coords, feats = ME.utils.sparse_collate(
    coordinates=[coords0, coords1], features=[feats0, feats1]
)

# Initialize the SparseTensor
A = ME.SparseTensor(coordinates=coords, features=feats)

A PointNet can be implemented as a specialization of a convolutional neural network within MinkowskiEngine. To achieve PointNet behavior using sparse convolutions, configure the network with the following constraints:

1. Kernel Size: Set all convolution layers to have a kernel size of 1.
2. Stride: Set all convolution layers to have a stride of 1.
3. Input: Use a sparse tensor where the features are normalized coordinates.

This approach allows you to treat linear layers as a specialization of convolution, enabling the processing of an arbitrary number of points while extending functionality to support:

• Arbitrary generic features (e.g., color).
• Convolutions with kernel size > 1.
• Convolutions with stride > 1.

To reconstruct a 3D sparsity pattern from a vector (e.g., a one-hot CAD index), you can build a network that sequentially upsamples voxels and then prunes them.

Key components used in this architecture:

• MinkowskiEngine.MinkowskiConvolutionTranspose: Used to upsample the voxel resolution.
• MinkowskiEngine.MinkowskiConvolution: Used for feature processing.
• MinkowskiEngine.MinkowskiPruning: Used to remove unnecessary voxels based on a classification path.

The network typically implements two paths during the forward pass: one for main features and one for sparse voxel classification to determine which voxels to keep.

# Example logic for an upsampling and pruning block
out = upsample_block(z)
out_cls = classification(out).F
out = pruning(out, out_cls > 0)

You can run the complete ModelNet40 classification demo using the following command. Note that this example caches ModelNet40 data in memory, which requires approximately 10GB of RAM.

Command:

python -m examples.modelnet40 --batch_size 128 --stat_freq 100

You can build and run Minkowski Engine using a Docker container.

Build the image:

git clone https://github.com/NVIDIA/MinkowskiEngine
cd MinkowskiEngine
docker build -t minkowski_engine docker

Run and verify installation:

docker run MinkowskiEngine python3 -c "import MinkowskiEngine; print(MinkowskiEngine.__version__)"