DOCSEARCHbyAlgolia
Submit Docs

Stable Fast 3D (SF3D)

repository·main·Indexed 23 days ago

https://github.com/stability-ai/stable-fast-3d

A model for fast, feedforward 3D mesh reconstruction from a single image. SF3D features UV-unwrapping, illumination disentanglement, and material parameter prediction. It includes a CLI for GLB export, a Gradio web interface, a ComfyUI extension, and a TextureBaker utility for rasterizing barycentric coordinates and interpolating vertex attributes into texture atlases.

Tokens
1.6K
Snippets
6
Records
10
Agent score
33%

What's inside Stable Fast 3D

  1. Configure Hardware Backends (MPS, Windows, CPU)

    main

    Mac Silicon (MPS)

    Support is experimental. To use the MPS backend:

    • Install OpenMP runtime (e.g., via mac.r-project.org/openmp/).
    • Use the latest PyTorch (2.4.0+ recommended).
    • Run with the environment variable: PYTORCH_ENABLE_MPS_FALLBACK=1.
    • Note: MPS uses more memory than CUDA. If you have < 32GB unified memory, consider the CPU version.

    Windows

    Support is experimental. Requires Visual Studio 2022 and appropriate PyTorch/CUDA versions.

    CPU

    • The CPU backend is used automatically if no GPU is detected.
    • To force CPU usage even if a GPU is present, set the environment variable: SF3D_USE_CPU=1.

  2. Install Stable Fast 3D

    main

    To install Stable Fast 3D, ensure your environment meets the following requirements:

    • Python >= 3.8
    • PyTorch installed (ensure CUDA version matches your system if using a GPU)
    • For Windows (experimental): Visual Studio 2022

    Follow these steps to install:

    1. Update setuptools: pip install -U setuptools==69.5.1
    2. Install wheel: pip install wheel
    3. Install core requirements: pip install -r requirements.txt
    4. (Optional) For the Gradio demo: pip install -r requirements-demo.txt
    pip install -U setuptools==69.5.1
pip install wheel
pip install -r requirements.txt
# For Gradio demo
pip install -r requirements-demo.txt

  3. Install the ComfyUI Extension

    main

    To use Stable Fast 3D within ComfyUI, follow these steps:

    1. Clone the repository into your custom_nodes directory:
      cd ComfyUI/custom_nodes
git clone https://github.com/Stability-AI/stable-fast-3d
    2. Install the required dependencies:
      cd stable-fast-3d
pip install -r requirements.txt
    3. Restart ComfyUI.
    $ cd ComfyUI/custom_nodes
$ git clone https://github.com/Stability-AI/stable-fast-3d
$ cd stable-fast-3d
$ pip install -r requirements.txt

  6. Run Manual Inference via CLI

    main

    Use run.py to reconstruct a 3D mesh from one or more images. The output is saved as a GLB file.

    Arguments:

    • --output-dir: Directory where the GLB file will be saved.
    • --texture-resolution: Specify the resolution in pixels for the output texture.
    • --remesh_option: Specify the remeshing operation (None, Triangle, or Quad).

    Note: Default inference requires approximately 6GB VRAM per image.

    python run.py demo_files/examples/chair1.png --output-dir output/

  7. Use the TextureBaker to bake vertex attributes to textures

    main

    The TextureBaker is a utility for rasterizing barycentric coordinates to a tensor and interpolating vertex attributes (like positions, normals, or colors) into a texture atlas based on UV coordinates. It supports both CPU and GPU execution.

    To bake attributes, follow a two-step process:

    1. Rasterize: Generate barycentric coordinates for each pixel in the texture using the mesh's UV coordinates and face indices.
    2. Interpolate: Use those coordinates to interpolate the actual vertex attributes into the texture map.

    Required mesh data:

    • uv: UV coordinates of shape (num_vertices, 2).
    • faces: Triangle indices of shape (num_faces, 3).
    • vertices: Vertex attributes (e.g., positions) of shape (num_vertices, N).
    from texture_baker import TextureBaker

mesh = ...
uv = mesh.uv # num_vertex, 2
triangle_idx = mesh.faces # num_faces, 3
vertices = mesh.vertices # num_vertex, 3

tb  = TextureBaker()
# First get the barycentric coordinates
rast = tb.rasterize(
    uv=uv, face_indices=triangle_idx, bake_resolution=1024
)
# Then interpolate vertex attributes
position_bake = tb.interpolate(attr=vertices, rast=rast, face_indices=triangle_idx)

  8. Interpolate vertex attributes with TextureBaker.interpolate

    main

    The interpolate method uses previously computed barycentric coordinates to map vertex attributes onto a texture map.

    Arguments:

    • attr: The vertex attributes to bake (e.g., positions, colors, or normals). Shape: num_vertices, N.
    • rast: The barycentric coordinates tensor obtained from rasterize.
    • face_indices: The indices of the triangles/faces (shape: num_faces, 3).

  9. Rasterize barycentric coordinates with TextureBaker.rasterize

    main

    The rasterize method computes barycentric coordinates for each pixel in a texture atlas based on the provided UV coordinates and mesh faces.

    Arguments:

    • uv: The UV coordinates of the mesh (shape: num_vertices, 2).
    • face_indices: The indices of the triangles/faces (shape: num_faces, 3).
    • bake_resolution: The resolution of the output texture (e.g., 1024).

  10. Configure Remeshing Options

    main

    When running inference, you can specify a remeshing operation via --remesh_option. This affects the topology of the generated mesh:

    • none: The mesh remains unchanged after generation. This has no CPU overhead.
    • triangle: Rearranges vertices and edges into a triangle topography. Expect CPU overhead.
    • quad: Rearranges vertices and edges into a quadrilateral topography with proper quad flow. The mesh is split into triangles for GLB export. Expect CPU overhead.

    Target Vertex Count: You can specify a target vertex count, which acts as a rough goal for the remesher (ignored if none is selected).