PartCrafter

To prepare a dataset for training, use the preprocess.py script. This script processes GLB files and saves the preprocessed data.

Command: python datasets/preprocess/preprocess.py --input <input_dir> --output <output_dir>

python datasets/preprocess/preprocess.py --input assets/objects --output preprocessed_data

Use scripts/inference_partcrafter.py to generate a structured 3D object from a single RGB image. The model automatically downloads the necessary PartCrafter and RMBG weights.

Key Arguments:

• --image_path: Path to the input image.
• --num_parts: Number of parts to generate.
• --tag: A label for the output directory.
• --render: Enables rendering of the result.
• --rmbg: (Recommended for custom images) Removes the background and resizes the image appropriately.

python scripts/inference_partcrafter.py \
  --image_path assets/images/np3_2f6ab901c5a84ed6bbdf85a67b22a2ee.png \
  --num_parts 3 --tag robot --render

To preprocess an entire directory of meshes at once, use the preprocess.py script. This will process all meshes in the input folder and automatically generate a configuration file named object_part_configs.json in the output directory.

python datasets/preprocess/preprocess.py --input assets/objects --output preprocessed_data

Use scripts/inference_partcrafter_scene.py to generate a full 3D scene from an image. This uses the PartCrafter-Scene model weights (automatically downloaded).

Key Arguments:

• --image_path: Path to the scene image.
• --num_parts: Number of parts/objects in the scene.
• --tag: A label for the output directory.
• --render: Enables rendering.

Note: --part_suggest and --style_transfer are also supported for scene-level generation.

python scripts/inference_partcrafter_scene.py \
  --image_path assets/images_scene/np6_0192a842-531c-419a-923e-28db4add8656_DiningRoom-31158.png \
  --num_parts 6 --tag dining_room --render

The training code requires a JSON configuration file where each entry represents an object. You can use the generated object_part_configs.json or create your own following this schema. Each object entry must include paths to the mesh, surface points, and rendered image, along with part-related metadata.

[
    {
        "mesh_path": "/path/to/object.glb",
        "surface_path": "/path/to/object.npy",
        "image_path": "/path/to/object.png",
        "num_parts": 4,
        "iou_mean": 0.5,
        "iou_max": 0.9,
        "valid": true
    }, 
    {
        "..."
    }
]

To bridge the domain gap between real-world photos and the Objaverse-style training data, use the --style_transfer flag. This converts the input photo into an Objaverse-style 3D rendering before processing.

Configuration:

• Default model: gemini-3.1-flash-image-preview.
• Use --style_provider and --style_model to override the provider or model.

# Style transfer for real-world photos
GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --num_parts 4 --style_transfer --rmbg --render

# Combining part suggestion and style transfer
GEMINI_API_KEY=your_key python scripts/inference_partcrafter.py \
  --image_path real_photo.jpg \
  --part_suggest --style_transfer --rmbg --render

Instead of manually specifying --num_parts, you can use a Vision Language Model (VLM) to analyze the image and suggest an appropriate part count. This requires a GEMINI_API_KEY environment variable.

Configuration:

• Default model: gemini-3-flash-preview.
• Use --part_provider and --part_model to override the provider or model.

When calling the pipeline, you can control the precision and performance of the 3D mesh reconstruction via several octree and decoding parameters:

The following keys are used in the dataset configuration JSON file:

• mesh_path: The path to the GLB file of the object.
• surface_path: The path to the .npy file containing the object surface points.
• image_path: The path to the rendered image of the object (after background removal).
• num_parts: The number of parts the object is composed of.
• iou_mean: The mean Intersection over Union (IoU) of the object parts.
• iou_max: The maximum IoU of the object parts.
• valid: A boolean flag. If false, the object is filtered out during training.