repository·main·Indexed 42 days ago
https://github.com/pointrix-project/pointrixA lightweight, differentiable point-based rendering framework designed for research and application. It features modular components for 3D reconstruction, including support for Gaussian point clouds, NeRF and COLMAP dataset loaders, and renderers like MsplatRender and Gsplat. The framework includes tools for camera optimization, hyperparameter sweeps via launch_sweep.py, and integration with Dust3r for extracting point cloud and camera priors.
Pointrix is a lightweight, differentiable point-based rendering framework. It is designed with a modular architecture to be highly extensible and supports a wide variety of computer vision and graphics tasks.
Key characteristics include:
Pointrix is a lightweight differentiable point cloud rendering framework. It is designed with a modular architecture to provide high extensibility and supports a variety of tasks including:
Pointrix is a powerful and extensible framework built around point cloud rendering. Its architecture is designed to be modular, allowing researchers to customize workflows easily. The core design principles include:
Pointrix provides a modular hierarchy of classes for 3D reconstruction and rendering. The model architecture is organized into four main functional areas:
pointrix.model.base_model): Provides the foundational abstractions for all model types.pointrix.model.camera.camera_model): Defines camera parameters and intrinsic/extrinsic properties required for spatial reasoning.pointrix.model.point_cloud.points: Standard point cloud representations.pointrix.model.point_cloud.gaussian_points: Specialized representations for Gaussian Splatting.pointrix.model.renderer.base_splatting: The base class for all splatting-based rendering techniques.pointrix.model.renderer.msplat: Implementation of the M-Splat rendering method.pointrix.model.renderer.gsplat: Implementation of standard Gaussian Splatting rendering.Pointrix is a modular framework designed for point cloud rendering and Gaussian point cloud research. The architecture is divided into three main layers:
A key feature of the framework is that the rendering core is designed to be differentiable; all inputs, such as point cloud properties and camera parameters, return gradients.
A Hook is a tool class used to execute specific actions within a Trainer at predefined lifecycle points. Hooks allow you to inject custom logic (like logging or checkpointing) without modifying the core training loop code.
Common built-in examples include:
CheckPointHook: Used to save model checkpoints during the training loop.LogHook: Used to log training and validation losses.When a hook is called, it receives the Trainer instance as its first argument, providing full access to the trainer's state, model, optimizer, and configuration.
@HOOK_REGISTRY.register()
class LogHook(Hook):
def after_train_iter(self, trainer) -> None:
# Access trainer state to perform actions
loss = trainer.loss_dict['loss']
trainer.logger.write_scalar('loss', loss.item(), trainer.global_step)The Pointrix training process is organized around three core components that work together to manage parameters and optimization:
During training, the model interacts with data pipelines providing Point Cloud Prior, Observation, and Camera Prior to compute losses and updates.
The Pointrix data pipeline is composed of two primary components: Dataset and DataPipeline.
next_train() and next_val() to return batches used for rendering and loss calculation. It also provides the model with the necessary camera and point cloud initializations.Hooks are utility modules used to perform specific operations at fixed points during the training lifecycle. They allow you to inject custom logic (like logging or checkpointing) without modifying the core Trainer class.
Key concepts:
Trainer calls specific hook methods at predefined stages (e.g., before/after training iterations or validation).trainer instance as an argument, giving you full access to the model, optimizer, configuration, and data pipeline.CheckPointHook for saving model states and LogHook for recording training/validation losses.LogHook relies on a Writer object (which must implement write_scalar(), write_image(), and write_config()). For example, TensorboardWriter is a common implementation.class DefaultTrainer:
def __init__(self, cfg, exp_dir, device="cuda"):
# Hooks are built from the configuration
self.hooks = parse_hooks(self.cfg.hooks)
self.call_hook("before_run")
def train_loop(self):
self.call_hook("before_train")
for iteration in range(...):
self.call_hook("before_train_iter")
# ... training logic ...
self.call_hook("after_train_iter")
if iteration % interval == 0:
self.call_hook("before_val")
self.validation()
self.call_hook("after_val")
self.call_hook("after_train")Model training in Pointrix is organized around three core components that interact with data provided by a data pipeline (Point Cloud Priors, Observations, and Camera Priors):
During training, the BaseModel uses the Renderer to produce outputs, which are compared against observations to calculate loss, which the Optimizer then uses to update the parameters within the BaseModel.
The Pointrix framework is organized into three main functional categories:
The data flow in the Pointrix framework follows a structured sequence: