Unique3D

To build the Unique3D Docker image, navigate to the yourworkspace/Unique3D/docker directory and run the build command. Note that the build process uses the source from https://huggingface.co/spaces/Wuvin/Unique3D rather than the local repository. The build process may take over one hour and the resulting image will exceed 70GB in size.

docker build -t unique3d -f Dockerfile .

To perform the initial run of the Unique3D container, use the docker run command with GPU support enabled and port 7860 mapped. This will start the app.py script inside the container.

docker run -it --name unique3d -p 7860:7860 --gpus all unique3d python app.py

To set up Unique3D on Linux, use Ubuntu 22.04.4 LTS and CUDA 12.1. Follow these steps to create a Conda environment and install the necessary dependencies:

1. Create and activate a new Conda environment named unique3d with Python 3.11.
2. Install ninja and diffusers==0.27.2 via pip.
3. Install mmcv-full using the specific OpenMMLab index for CUDA 12.1 and Torch 2.3.1.
4. Install the remaining requirements from requirements.txt.

conda create -n unique3d python=3.11
conda activate unique3d

pip install ninja
pip install diffusers==0.27.2

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.3.1/index.html

pip install -r requirements.txt

To quickly install onnxruntime-gpu with TensorRT support for CUDA 12.x, run the following commands.

Note: You must replace /root/miniconda3/lib/python3.10/site-packages with your actual Python site-packages path and /root/.bashrc with your user's .bashrc path.

pip install ort-nightly-gpu --index-url=https://aiinfra.pkgs.visualstudio.com/PublicPackages/_packaging/ort-cuda-12-nightly/pypi/simple/
pip install onnxruntime-gpu==1.17.0 --index-url=https://pkgs.dev.azure.com/onnxruntime/onnxruntime/_packaging/onnxruntime-cuda-12/pypi/simple/
pip install tensorrt==8.6.0
echo -e "export LD_LIBRARY_PATH=/usr/local/cuda/targets/x86_64-linux/lib/:/root/miniconda3/lib/python3.10/site-packages/tensorrt:${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}" >> /root/.bashrc

Once the container has been created, you can start it and execute the application without running the full docker run command again. Use docker start to wake the container and docker exec to run the application script.

docker start unique3d
docker exec unique3d python app.py

To achieve better reconstruction results with Unique3D, follow these guidelines:

1. Input Image Scaling: Ensure the input image contains the longest edge of the object. Because the mesh is normalized by the longest edge of xyz during training, failing to include the longest edge may result in erroneously squashed meshes.
2. Facing Direction: Use orthographic, front-facing images in a rest pose. The model is sensitive to the facing direction due to training data distribution.
3. Occlusions: Minimize occlusions. Images with fewer occlusions lead to better results because the four-view reconstruction process cannot fully cover objects with heavy occlusions.
4. Resolution: Use the highest resolution image possible when resolution is a limiting factor.

nvdiffrast compiles a torch plugin version upon its first execution. This requires ninja and cudatoolkit to be installed, and the CUDA_HOME environment variable must be correctly set to your cudatoolkit installation directory (e.g., /usr/local/cuda).

If you cannot install cudatoolkit on your target server due to permission issues, you can use a pre-compiled version of nvdiffrast from this repository by compiling it on a similar environment first.

To achieve higher quality 3D reconstructions with Unique3D, follow these best practices:

1. Image Orientation: Use images where the object is facing forward (orthogonal view). The model is sensitive to input orientation.
2. Minimize Occlusions: Avoid images with significant occlusions. Occlusions prevent the four views from fully covering the object, which degrades reconstruction quality.
3. Resolution: Use the highest resolution images possible for input.

To run the interactive Gradio demo locally, you must first download the model weights and organize them in a ckpt directory as follows:

Unique3D
    ├──ckpt
        ├── controlnet-tile/
        ├── image2normal/
        ├── img2mvimg/
        ├── realesrgan-x4.onnx
        └── v1-inference.yaml

Weights can be downloaded from Huggingface spaces or Tsinghua Cloud Drive.

Once weights are in place, run the following command:

python app/gradio_local.py --port 7860

After installing the complex dependencies (nvdiffrast, onnxruntime-gpu, pytorch3d, torch_scatter), install the rest of the project requirements using the provided requirements file.

pip install -r requirements.txt

To set up Unique3D on Linux (optimized for Ubuntu 22.04.4 LTS and CUDA 12.1), create a Conda environment with Python 3.11 and install the required dependencies including ninja, diffusers, and mmcv-full.

conda create -n unique3d python=3.11
conda activate unique3d

pip install ninja
pip install diffusers==0.27.2

pip install mmcv-full -f https://download.openmmlab.com/mmcv/dist/cu121/torch2.3.1/index.html

pip install -r requirements.txt

You can install torch_scatter using pre-compiled packages following the official guide, or by compiling directly from the source repository.

pip install git+https://github.com/rusty1s/pytorch_scatter.git