FaceChain

FaceChain is a deep learning tool for generating personalized portraits. The latest FaceChain FACT version allows users to generate high-quality portraits in approximately 10 seconds using only a single photo.

Key features include:

• Infinite Style Portraits: High controllability and ID preservation.
• Fixed Template Portraits.
• Compatibility: Works with ControlNet and LoRA.
• Multiple Interfaces: Supports Gradio web UI, Python scripting for developers, and a plugin for Stable Diffusion WebUI.
• Advanced Modules: Includes SDXL support, super-resolution (up to 2048x2048), virtual try-on, and pose control.

Run evaluation on the IJBC dataset using an ONNX model. You must specify the --model-root directory containing the model, the --image-path to the IJBC dataset, and a --result-dir for the output.

CUDA_VISIBLE_DEVICES=0 python onnx_ijbc.py --model-root ms1mv3_arcface_r50 --image-path IJB_release/IJBC --result-dir ms1mv3_arcface_r50

TopoFR requires training datasets (MS1MV2 or Glint360K) and the IJB-C test dataset.

Training Datasets:

• MS1MV2: Google Drive
• Glint360K: Baidu Pan (code: o3az)

Test Dataset:

• IJB-C: Google Drive

After training, use the scripts/stable_txt2img.py script to generate images with your fine-tuned model.

python scripts/stable_txt2img.py --ddim_eta 0.0 --n_samples 8 --n_iter 2 --scale 10.0 --ddim_steps 50  --ckpt finetuned_ckpt --prompt "prompt" --seed 7 --outdir output_img_dir --unconditional_prompt "monochrome, lowres, bad anatomy, worst quality, low quality"

Run the training process using main.py. Unlike standard DreamBooth, FaceChain-SuDe requires a --class_word (the category of the subject) and a --sude_weight.

Note on --sude_weight: A higher weight helps inherit more public attributes from the category, but a weight that is too high may cause a loss of subject fidelity.

python main.py --base configs/stable-diffusion/v1-finetune_unfrozen.yaml -t --actual_resume stable-diffusion-v-1-4/sd-v1-4-full-ema.ckpt -n name --gpus 0, --data_root subject_dir --reg_data_root reg_data_dir --class_word <category> --sude_weight 0.8 --logdir output_checkpoint_dir

To install PyTorch v1.9.0 with CUDA 10.2 support on Linux or Windows, use the following pip command pointing to the PyTorch stable wheel repository.

pip install torch==1.9.0+cu102 torchvision==0.10.0+cu102 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html

To test models:

1. Modify the path to the IJB-C dataset in eval_ijbc.py.
2. Execute the evaluation script with the appropriate --model-prefix, --result-dir, and --network arguments.

python eval_ijbc.py --model-prefix work_dirs/glint360k_vit_s/model.pt --result-dir work_dirs/glint360k_vit_s --network vit_s_dp005_mask_0 > ijbc_glint360k_vit_s.log 2>&1 &

To use WebFace42M with Nvidia DALI, you must generate shuffled .rec and .idx files. Standard InsightFace style .rec files are not compatible with DALI and will cause performance degradation.

Follow these steps to prepare your dataset:

1. Download and Unzip: Obtain WebFace42M from https://www.face-benchmark.org/download.html. After unarchiving, WebFace42M should contain 10 directories (0 through 9).
2. Generate List File: Create a train.lst file from your dataset root.
3. Generate Record Files: Use the list file to create train.rec and train.idx files, which are required for training.

Use the official ModelScope Docker image to avoid complex environment setup.

Steps:

1. Pull Image: Use the appropriate registry based on your location (China Mainland vs. International).
2. Run Container: Start the container with GPU support and port mapping for Gradio (7860).
3. Install Dependencies: Install the required Python packages inside the container.
4. Clone and Run: Clone the FaceChain source code and run app.py.

Note: If you have multiple GPUs, use CUDA_VISIBLE_DEVICES=0 python3 app.py to specify a single card.

Run evaluation on the IJBC dataset using PyTorch by executing the eval_ijbc.py script. This method allows for multi-GPU evaluation and batch processing.

CUDA_VISIBLE_DEVICES=0,1 python eval_ijbc.py \
--model-prefix ms1mv3_arcface_r50/backbone.pth \
--image-path IJB_release/IJBC \
--result-dir ms1mv3_arcface_r50 \
--batch-size 128 \
--job ms1mv3_arcface_r50 \
--target IJBC \
--network iresnet50

To use FaceChain-SuDe, you must first set up the ldm environment following the instructions from the original Stable Diffusion repository. Additionally, you need to download pre-trained Stable Diffusion models. SuDe is evaluated to work well with:

• sd-v1-4-full-ema.ckpt (SD-v1.4)
• v1-5-pruned.ckpt (SD-v1.5)

To install PyTorch v1.9.0 with CUDA 11.1 support on Linux or Windows, use the following pip command pointing to the PyTorch stable wheel repository.

pip install torch==1.9.0+cu111 torchvision==0.10.0+cu111 torchaudio==0.9.0 -f https://download.pytorch.org/whl/torch_stable.html