SpConv Documentation

In tasks like semantic segmentation, you often need to upsample features back to the original resolution.

Important Distinction: spconv.SparseInverseConv3d is not the same as spconv.SparseConvTranspose3d.

• spconv.SparseInverseConv3d: Designed to be the mathematical 'inverse' of a sparse convolution. The output contains the same indices as the input of the corresponding SparseConv3d. To use it, you must provide the same indice_key used during the downsampling step and the same kernel_size to create the weights.
• spconv.SparseConvTranspose3d: Standard upsampling (equivalent to nn.ConvTranspose3d). This is very slow and cannot recover the original point cloud structure directly. It should primarily be used in generative models.

class ExampleNet(nn.Module):
    def __init__(self, shape):
        super().__init__()
        self.net = spconv.SparseSequential(
            spconv.SparseConv3d(32, 64, 3, 2, indice_key="cp0"),
            spconv.SparseInverseConv3d(64, 32, 3, indice_key="cp0"), # Uses saved indices from cp0
        )
        self.shape = shape

    def forward(self, features, coors, batch_size):
        coors = coors.int()
        x = spconv.SparseConvTensor(features, coors, self.shape, batch_size)
        return self.net(x)

Spconv supports fusing subm residual blocks (SubMConv + BatchNorm + Add + ReLU) into a single SubMConvAddReLU node.

To enable fusion: Your residual code must avoid using spconv specific operations like .replace_feature() or accessing .features directly on the tensor, as these create standalone nodes in the torch.fx graph that prevent fusion.

Correct Pattern (Fused): Use spconv.SparseReLU and spconv.SparseIdentity to ensure the graph remains compatible with fusion.

Incorrect Pattern (Non-fused): Using out.replace_feature(...) or out.features will prevent the residual block from being fused.

class SparseBasicBlock(spconv.SparseModule):
    """residual block that supported by spconv quantization."""
    expansion = 1
    def __init__(self, in_planes, out_planes, stride=1, downsample=None):
        spconv.SparseModule.__init__(self)
        conv1 = spconv.SubMConv2d(in_planes, out_planes, 3, stride, 1, bias=False)
        conv2 = spconv.SubMConv2d(out_planes, out_planes, 3, stride, 1, bias=False)
        norm1 = nn.BatchNorm1d(out_planes, momentum=0.1)
        norm2 = nn.BatchNorm1d(out_planes, momentum=0.1)
        self.conv1_bn_relu = spconv.SparseSequential(conv=conv1, bn=norm1, relu=nn.ReLU(inplace=True))
        self.conv2_bn = spconv.SparseSequential(conv=conv2, bn=norm2)
        self.relu = spconv.SparseReLU(inplace=True)
        self.downsample = downsample
        self.iden_for_fx_match = spconv.SparseIdentity()

    def forward(self, x: spconv.SparseConvTensor):
        identity = x
        out = self.conv1_bn_relu(x)
        out = self.conv2_bn(out)
        if self.downsample is not None:
            identity = self.downsample(x)
        out = self.relu(out + identity)
        return out

QAT inserts observers and fake quantization nodes into the model during training.

Workflow:

1. Set is_qat = True.
2. Use spconvq.get_default_spconv_qconfig_mapping(is_qat) to get the appropriate mapping.
3. Use qfx.prepare_qat_fx to create the prepared_model.
4. Run your training loop using the prepared_model.
5. Once training is complete, the model can be converted for Int8 inference using the same PTQ workflow described in the PTQ guide.

import spconv.pytorch.quantization as spconvq
import torch.ao.quantization.quantize_fx as qfx

model = ...
is_qat = True
qconfig_mapping = spconvq.get_default_spconv_qconfig_mapping(is_qat)
prepare_cfg = spconvq.get_spconv_prepare_custom_config()
backend_cfg = spconvq.get_spconv_backend_config()

# prepare model with fake quantize nodes
prepared_model = qfx.prepare_qat_fx(model, qconfig_mapping, (), backend_config=backend_cfg, prepare_custom_config=prepare_cfg)

# run training
train(prepared_model)

To perform PTQ, trace the model via torch.fx, insert observers, run inference with test data to calculate scales, and then convert the model.

Workflow:

1. Define qconfig_mapping using spconvq.get_default_spconv_qconfig_mapping(is_qat=False).
2. Configure prepare_cfg and backend_cfg to handle non-traceable modules or custom attributes.
3. Use qfx.prepare_fx to create a prepared_model containing observers.
4. Run inference on a data loader to populate observer statistics.
5. Use spconvq.prepare_spconv_torch_inference(with_linear=True) (must be called before convert_fx).
6. Convert the model using qfx.convert_fx.
7. Apply spconvq.transform_qdq and spconvq.remove_conv_add_dq to clean up the graph for Int8 inference.

Debugging Tip: Set the environment variable SPCONV_INT8_DEBUG=1 to reduce compilation time by removing most candidate Int8 kernels.

Troubleshooting: If an operation does not support the CUDA backend, disable quantization for that specific layer/module in the qconfig_mapping.

import spconv.pytorch.quantization as spconvq
import torch.ao.quantization.quantize_fx as qfx

model = ...
is_qat = False
qconfig_mapping = spconvq.get_default_spconv_qconfig_mapping(is_qat)
# disable quantization for some layers here:
qconfig_mapping.set_module_name_regex("foo.*bar.*", None)
# disable quantization by type here:
qconfig_mapping.set_object_type(ModuleClass, None)

prepare_cfg = spconvq.get_spconv_prepare_custom_config()
# preserve static attrs for your module here:
prepare_cfg.preserved_attributes = [...]
# add nontraceable modules here:
prepare_cfg.non_traceable_module_classes.extend([...])

backend_cfg = spconvq.get_spconv_backend_config()
# add custom qconfig for your non-traceable operators:
backend_cfg.set_backend_pattern_config(BackendPatternConfig(some_op_or_module_class).set_observation_type(
    ObservationType.OUTPUT_USE_DIFFERENT_OBSERVER_AS_INPUT).set_dtype_configs(
        [non_weighted_op_qint8_dtype_config]))

# prepare model
prepared_model = qfx.prepare_fx(model, qconfig_mapping, (), backend_config=backend_cfg, prepare_custom_config=prepare_cfg)

# run inference to calculate scales
for x in loader:
    prepared_model(x)

# convert to int8
spconvq.prepare_spconv_torch_inference(with_linear=True)
converted_model = qfx.convert_fx(prepared_model, qconfig_mapping=qconfig_mapping, backend_config=backend_cfg)
converted_model = spconvq.transform_qdq(converted_model)
converted_model = spconvq.remove_conv_add_dq(converted_model)