ConsistentID

The model weights can be automatically downloaded from Hugging Face using the huggingface_hub library. Alternatively, they are available via Google Drive or Baidu Netdisk.

from huggingface_hub import hf_hub_download
ConsistentID_path = hf_hub_download(repo_id="JackAILab/ConsistentID", filename="ConsistentID-v1.bin", repo_type="model")

To train ConsistentID, organize your data in the following directory structure:

├── data
|   ├── JSON_all.json 
|   ├── resize_IMG # Images 
|   ├── all_faceID  # FaceID
|   └── parsing_mask_IMG # Parsing Mask 

The JSON_all.json file must contain an array of objects with the following keys:

• IMG: Path to the image.
• parsing_mask_IMG: Path to the parsing mask.
• vqa_llva: VQA/LLaVA related data.
• id_embed_file_resize: Path to the resized FaceID embedding file.
• vqa_llva_facial: VQA/LLaVA facial related data.

[
    {
        "IMG": "Path of image...",
        "parsing_mask_IMG": "...",
        "vqa_llva": "...",
        "id_embed_file_resize": "...",
        "vqa_llva_facial": "..."
    },
    ...
]

To start the training process, ensure you are in the root directory of the project and execute the training bash script.

bash train_bash.sh

ConsistentID requires Python >= 3.8, PyTorch >= 2.0.0, and CUDA 11.8. It is recommended to use Anaconda or Miniconda for environment management.

Follow these steps to set up the environment:

1. Create a new Conda environment with Python 3.8.10.
2. Activate the environment.
3. Upgrade pip.
4. Install the project requirements using the provided requirements.txt.

conda create --name ConsistentID python=3.8.10
conda activate ConsistentID
pip install -U pip

# Install requirements
pip install -r requirements.txt

Before running inference, it is recommended to run convert_weights.py to save weights efficiently. Ensure you are in the project root directory.

Standard Inference

Run the standard inference script:

python infer.py

Inpaint and Controlnet Inference

To use the specialized inpainting or Controlnet demos, use the following module commands:

python -m demo.inpaint_demo
python -m demo.controlnet_demo

# Convert weights first
python evaluation/convert_weights.py

# Standard inference
python infer.py

# Inpaint/Controlnet demos
python -m demo.inpaint_demo
python -m demo.controlnet_demo