Step1X-Edit Documentation

Step1X-Edit is a state-of-the-art image editing model designed to compete with closed-source models like GPT-4o and Gemini 2 Flash. It utilizes a Multimodal LLM to process a reference image and user editing instructions, extracting latent embeddings that are integrated with a diffusion image decoder to produce the target image.

Key features include:

• High-quality image editing based on genuine user instructions.
• Support for text-to-image (T2I) generation (added in v1.1).
• Reasoning capabilities in newer versions (v1.2/ReasonEdit-S) that combine instruction reasoning with reflective correction.
• Support for LoRA fine-tuning on a single 24GB GPU.
• Compatibility with ComfyUI via community plugins.

The evaluation produces scores across several dimensions. Metrics are reported as higher-is-better.

GPT-4o Metrics (G-series)

• G_SC: Semantic Consistency
• G_PQ: Perceptual Quality
• G_O: Overall Performance

Qwen2.5-VL Metrics (Q-series)

• Q_SC: Semantic Consistency
• Q_PQ: Perceptual Quality
• Q_O: Overall Performance

Note on Reproducibility: GPT-4o scores may exhibit volatility (typical variation of ~0.1). For full reproduction of the values reported in the Step1X-Edit papers, it is recommended to use the Qwen model.

You can reduce GPU memory usage by using the scripts/run_examples.sh script with specific flags:

• --quantized: Quantifies weights to fp8.
• --offload: Offloads some modules to CPU.

For example, to use both: bash scripts/run_examples.sh --quantized --offload.

To use version 1.1, install the step1xedit branch of diffusers.

git clone -b step1xedit https://github.com/Peyton-Chen/diffusers.git
cd diffusers
pip install -e .

Before running the evaluation scripts, your generated images must follow a specific directory structure. The structure is organized by method name, then by full set, then by edit task, and finally split into Chinese (cn) and English (en) instruction subdirectories.

Required Directory Structure:

results/
├── {method_name}/
│   └── fullset/
│       └── {edit_task}/
│           ├── cn/  # Chinese instructions
│           │   ├── key1.png
│           │   └── ...
│           └── en/  # English instructions
│               ├── key1.png
│               └── ...

After running the scoring script, use calculate_statistics.py to obtain aggregate metrics across different dimensions, including semantics, quality, and overall performance. This script provides scores broken down by edit category.

Arguments:

• --model_name: The name of your model.
• --save_path: The directory containing the scores generated by the scoring script.
• --backbone: The backbone used for scoring (gpt4o).
• --language: The language scope for analysis (e.g., all).

python calculate_statistics.py --model_name your_model --save_path score_dir --backbone gpt4o --language all

To use version 1.2, ensure transformers==4.55.0 is installed. Then, clone and install the specific diffusers branch and optionally install RegionE for faster inference.

git clone -b step1xedit_v1p2 https://github.com/Peyton-Chen/diffusers.git
cd diffusers
pip install -e .

pip install RegionE # optional, for faster inference

Step Image Edit 2 is a lightweight model optimized for ultra-fast, real-time interactive image generation and editing, capable of completing tasks within 2 seconds.

To use this model via the StepFun Open Platform:

• Try it here: https://platform.stepfun.com/docs/zh/guides/models/step-image-edit-2
• API Documentation: https://platform.stepfun.com/docs/zh/step-plan/integrations/image-api

To use the preview version, install the dev/MergeV1-2 branch of diffusers.

git clone -b dev/MergeV1-2 https://github.com/Peyton-Chen/diffusers.git
cd diffusers
pip install -e .

You can evaluate your model's performance using either GPT-4o or Qwen2.5VL-72B-Instruct-AWQ.

GPT-4o Evaluation

1. Set up your API keys in a file named secret_t2.env.
2. Run run_gedit_score.py with the --backbone gpt4o flag.

Qwen Evaluation

Run run_gedit_score.py with the --backbone qwen25vl flag.

Arguments:

• --model_name: The name of your model.
• --save_dir: Directory where scores will be saved.
• --backbone: The evaluation model to use (gpt4o or qwen25vl).
• --edited_images_dir: Path to your organized edited images.

Install the base requirements using pip install -r requirements.txt.

To install flash-attn, use the provided script to find a suitable pre-built wheel for your system:

python scripts/get_flash_attn.py

Follow the flash-attn instructions to install the generated wheel.

Fine-tune the model using the ./scripts/finetuning.sh script.

Dataset Format: Your custom dataset should be organized via a .toml config (e.g., ./library/data_configs/step1x_edit.toml). The metadata_file must be a JSON file where keys are target image paths and values are dictionaries containing:

• ref_image_path: Path to the source image.
• caption: The editing instruction.

Example JSON structure:

{
  "/path/to/target.jpg": {
    "ref_image_path": "/path/to/source.jpg",
    "caption": "add a hat"
  }
}

bash ./scripts/finetuning.sh