HiDream-E1 Documentation

The pipeline includes a refine_strength parameter to balance the editing and refinement stages:

• Editing Stage: Occurs during the first (1 - refine_strength) portion of denoising steps.
• Refinement Stage: The remaining refine_strength portion uses HiDream-I1-Full for img2img refinement to enhance the result.
• Disable Refinement: Set refine_strength to 0.0 to skip the refinement stage.

When using HiDream-E1-Full, you must provide instructions in the following specific format:

Editing Instruction: {instruction}. Target Image Description: {description}

Example: Editing Instruction: Convert the image into a Ghibli style. Target Image Description: A person in a light pink t-shirt with short dark hair, depicted in a Ghibli style against a plain background.

If you are using HiDream-E1-Full, you can use the instruction_refinement.py script to automatically generate the required target image descriptions. This script requires a VLM API key (either via a local vllm instance or OpenAI's API).

python ./instruction_refinement.py --src_image ./test.jpeg --src_instruction "convert the image into a Ghibli style"

HiDream-E1-Full operates at a fixed resolution of 768x768. Unlike version 1.1, it requires instructions to be formatted with both an editing instruction and a target image description. Use the inference.py script to generate images.

python ./inference.py

HiDream-E1.1 supports dynamic resolution (up to 1M pixels) and accepts direct instructions (e.g., convert the image into a Ghibli style) without needing manual prompt refinement. Use the inference_e1_1.py script to generate images.

python ./inference_e1_1.py

To set up the environment for HiDream-E1, you must install the project requirements along with Flash Attention and the latest version of Diffusers. It is recommended to use CUDA version 12.4 for manual installations.

Prerequisites:

• Flash Attention
• Latest Diffusers
• HuggingFace account with access to meta-llama/Llama-3.1-8B-Instruct (required for automatic model downloading).

pip install -r requirements.txt
pip install -U flash-attn --no-build-isolation
pip install -U git+https://github.com/huggingface/diffusers.git

You can run interactive Gradio demos for either model version:

For HiDream-E1.1:

python gradio_demo_1_1.py

For HiDream-E1-Full:

python gradio_demo.py

# For E1.1
python gradio_demo_1_1.py 

# For E1-Full
python gradio_demo.py 

The following models are available in the repository:

| Name | Script | Inference Steps | Resolution | HuggingFace repo |
| --- | --- | --- | --- | --- |
| HiDream-E1-Full | inference.py | 28 | 768x768 | 🤗 [HiDream-E1-Full](https://huggingface.co/HiDream-ai/HiDream-E1-Full) |
| HiDream-E1.1 | inference_e1_1.py | 28 | Dynamic(1M pixels) | 🤗 [HiDream-E1.1](https://huggingface.co/HiDream-ai/HiDream-E1-1) |