Use Flux Gym for LoRA Training
mainThe basic training workflow involves:
- Entering LoRA information.
- Uploading images and captioning them (ensure you include the trigger word in captions).
- Clicking "start".
repository·main·Indexed 25 days ago
https://github.com/cocktailpeanut/fluxgymA simplified web UI for training FLUX LoRAs, designed for low VRAM environments (12GB to 20GB). It combines a Gradio UI with the Kohya Scripts backend, featuring AI captioning via Florence-2, custom base model support, and direct publishing to Hugging Face. Supports installation via Pinokio, manual setup, or Docker Compose.
The basic training workflow involves:
Instead of manual captioning in the UI, you can upload pre-made caption files. Follow this naming convention:
.txt file..txt file must match an image file name exactly.img0.png must have a corresponding img0.txt.To install manually, you must clone both the fluxgym repository and the sd-scripts repository (specifically the sd3 branch), set up a Python virtual environment, and install the required dependencies for both.
# 1. Clone repositories
git clone https://github.com/cocktailpeanut/fluxgym
cd fluxgym
git clone -b sd3 https://github.com/kohya-ss/sd-scripts
# 2. Create and activate virtual environment
# Windows:
python -m venv env
env\Scripts\activate
# Linux:
python -m venv env
source env/bin/activate
# 3. Install dependencies
cd sd-scripts
pip install -r requirements.txt
cd ..
pip install -r requirements.txt
# 4. Install PyTorch Nightly
# Standard:
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
# For NVIDIA RTX 50-series (e.g., 5090):
pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128
pip install -U bitsandbytes
# 5. Start the application
python app.pyYou can automatically install and launch Flux Gym locally using the Pinokio 1-click launcher.
https://pinokio.computer/item?uri=https://github.com/cocktailpeanut/fluxgymTo publish your models:
HF_TOKEN.By default, sample images are not generated. To enable them, configure the following in the UI:
models.yaml file in the repository. This allows you to use models other than flux-dev or flux-schnell.Flux Gym allows you to upload images and generate captions for training:
.image or .txt files.Add AI captions with Florence-2 button to automatically generate captions for your uploaded images using the Florence-2 model. This process uses the concept_sentence to assist in captioning.Caption boxes to ensure the trigger word is correctly included.The Flux Gym WebUI provides several configuration options for training FLUX LoRAs. Key settings include:
LoRA Info:
lora_name: A unique name for your LoRA.concept_sentence: The trigger word or sentence (e.g., p3rs0n or in the style of CNSTLL).base_model: The foundation model (can be extended by editing models.yaml).vram: Target VRAM capacity (20G, 16G, or 12G).num_repeats: Number of repeat trains per image.max_train_epochs: Maximum training epochs.sample_prompts: Text prompts for generating sample images (separated by new lines).sample_every_n_steps: Frequency of sample image generation.resolution: The target resize resolution for dataset images.Advanced options:
--seed: Random seed for reproducibility.--max_data_loader_n_workers: Number of data loader workers.--learning_rate: The training learning rate (default: 8e-4).--save_every_n_epochs: Frequency of saving checkpoints.--guidance_scale: Guidance scale value.--timestep_sampling: Timestep sampling method (e.g., shift).--network_dim: LoRA Rank (range: 4 to 128, step 4).The 'Publish' tab in the WebUI allows you to upload your trained LoRAs directly to HuggingFace:
Huggingface Token and click Login.Account: Your HuggingFace username (automatically populated after login).Repository Name: The desired name for the new repository.Repository Visibility: Set to 'public' or 'private'.Upload to HuggingFace to initiate the transfer.You can deploy Fluxgym using Docker Compose. The configuration maps port 7860 to the host and requires an NVIDIA GPU.
Important CUDA Configuration:
If you are running CUDA 12.4 drivers, you must change the dockerfile setting in the build section from Dockerfile to Dockerfile.cuda12.4. Otherwise, leave it as Dockerfile.
Environment Variables:
PUID: User ID for the container (defaults to 1000).PGID: Group ID for the container (defaults to 1000).Volumes:
/etc/localtime and /etc/timezone are mounted as read-only to sync container time with the host../ is mounted to /app/fluxgym inside the container.services:
fluxgym:
build:
context: .
# change the dockerfile to Dockerfile.cuda12.4 if you are running CUDA 12.4 drivers otherwise leave as is
dockerfile: Dockerfile
image: fluxgym
container_name: fluxgym
ports:
- 7860:7860
environment:
- PUID=${PUID:-1000}
- PGID=${PGID:-1000}
volumes:
- /etc/localtime:/etc/localtime:ro
- /etc/timezone:/etc/timezone:ro
- ./:/app/fluxgym
stop_signal: SIGKILL
tty: true
deploy:
resources:
reservations:
devices:
- driver: nvidia
count: all
capabilities: [gpu]
restart: unless-stopped