repository·main·Indexed 24 days ago
https://github.com/qwenlm/qwen-image-layeredA model designed to decompose a single image into multiple independent RGBA layers for high-fidelity editing. It features the QwenImageLayeredPipeline for image decomposition and supports exporting layers to .pptx, .zip, and .psd formats. The repository includes Gradio interfaces for decomposition, RGBA image editing via Qwen-Image-Edit, and layer combination, with additional support for vLLM-Omni.
To use Qwen-Image-Layered, ensure you have transformers>=4.51.3 (to support Qwen2.5-VL) and install the latest version of diffusers along with python-pptx and psd-tools for layer export capabilities.
pip install git+https://github.com/huggingface/diffusers
pip install python-pptx
pip install psd-toolsThe repository provides several Gradio-based web interfaces for different stages of the layered image workflow:
Decomposition & Export: Start a web interface to decompose images and export layers into .pptx, .zip, or .psd files.
python src/app.pyRGBA Image Editing: Launch an interface to edit specific decomposed layers using transparency (powered by Qwen-Image-Edit).
python src/tool/edit_rgba_image.pyLayer Combination: Use this script to combine edited layers back into a single image. Important: Upload layers in order from the bottom layer to the top layer.
python src/tool/combine_layers.pyThe infer function provides paths to several export formats for the generated layers:
.png files (named layer_1.png, layer_2.png, etc.).Layer 1, Layer 2).The QwenImageLayeredPipeline from diffusers allows you to decompose an input image into multiple RGBA layers.
Key Input Parameters:
image: The input image (must be converted to RGBA).layers: The number of layers to decompose the image into.resolution: Recommended resolution is 640.true_cfg_scale: Guidance scale.negative_prompt: Text prompt for what to avoid.cfg_normalize: Boolean to enable/disable CFG normalization.use_en_prompt: Boolean to automatically caption the image in English if no caption is provided.Note: The text prompt should describe the overall content of the image (including occluded elements) rather than controlling individual layers explicitly. The model is fine-tuned for image-to-multi-RGBA decomposition; text-to-multi-RGBA generation performance is limited.
from diffusers import QwenImageLayeredPipeline
import torch
from PIL import Image
pipeline = QwenImageLayeredPipeline.from_pretrained("Qwen/Qwen-Image-Layered")
pipeline = pipeline.to("cuda", torch.bfloat16)
pipeline.set_progress_bar_config(disable=None)
image = Image.open("asserts/test_images/1.png").convert("RGBA")
inputs = {
"image": image,
"generator": torch.Generator(device='cuda').manual_seed(777),
"true_cfg_scale": 4.0,
"negative_prompt": " ",
"num_inference_steps": 50,
"num_images_per_prompt": 1,
"layers": 4,
"resolution": 640, # Using different bucket (640, 1024) to determine the resolution. For this version, 640 is recommended
"cfg_normalize": True, # Whether enable cfg normalization.
"use_en_prompt": True, # Automatic caption language if user does not provide caption,
}
with torch.inference_mode():
output = pipeline(**inputs)
output_image = output.images[0]
for i, image in enumerate(output_image):
image.save(f"{i}.png")infer function performs the core decomposition task. It accepts an input image (as a file path, PIL Image, or numpy array) and several configuration parameters to control the decomposition process. It returns the decomposed images along with paths to exported files (PPTX, ZIP, and PSD).The infer function performs image editing using the QwenImageEditPlusPipeline. It takes an input image (ideally with an alpha channel), a text prompt, and several generation parameters. The function automatically handles background blending (using a green background for the diffusion process) and post-generation background removal to return an RGBA image with a transparent background.
Parameters:
image: The input PIL image.prompt: Text instruction describing the edit.seed: Integer for reproducibility (default: 42).randomize_seed: Boolean to toggle random seed generation.true_guidance_scale: Guidance scale for the diffusion process (default: 1.0).num_inference_steps: Number of denoising steps (default: 50).progress: A gr.Progress object for tracking progress in Gradio environments.def infer(
image,
prompt,
seed=42,
randomize_seed=False,
true_guidance_scale=1.0,
num_inference_steps=50,
progress=gr.Progress(track_tqdm=True),
):
# ... implementation ...
return edited_image, seedThe QwenImageLayeredPipeline from the diffusers library is used to decompose an input image into multiple layers. You can load the pipeline using from_pretrained and move it to a GPU (e.g., using cuda and torch.bfloat16 for efficiency).
from diffusers import QwenImageLayeredPipeline
import torch
pipeline = QwenImageLayeredPipeline.from_pretrained("Qwen/Qwen-Image-Layered")
pipeline = pipeline.to("cuda", torch.bfloat16)The blend_with_green_bg helper function is used to prepare an RGBA image for the diffusion pipeline by compositing it onto a solid green background (30, 215, 96). This is a common technique in layered image editing to provide a stable background for the model to work against before the alpha channel is re-applied.
def blend_with_green_bg(input_img):
bg = Image.new("RGB", input_img.size, (30, 215, 96)).convert("RGBA")
input_rgba = input_img.convert("RGBA")
blended = Image.alpha_composite(bg, input_rgba).convert("RGB")
return blendedWhen calling infer(), you can use the following parameters:
input_image: The source image (string path, PIL.Image.Image, or numpy.ndarray).seed (int, default 777): The random seed for generation.randomize_seed (bool, default False): If True, a random seed is generated.prompt (str, optional): A description of the overall content of the image. Note: It is not intended to control individual layers explicitly.neg_prompt (str, default " "): Negative prompt to guide the model away from certain features.true_guidance_scale (float, default 4.0): Guidance scale for the model.num_inference_steps (int, default 50): Number of denoising steps.layer (int, default 4): The number of layers to decompose the image into.cfg_norm (bool, default True): Whether to enable CFG (Classifier-Free Guidance) normalization.use_en_prompt (bool, default True): If True, uses English for automatic captioning if no prompt is provided. If False, uses Chinese (ZH).