Flux Gym

repository·main·Indexed 25 days ago

https://github.com/cocktailpeanut/fluxgym

A simplified web UI for training FLUX LoRAs, designed for low VRAM environments (12GB to 20GB). It combines a Gradio UI with the Kohya Scripts backend, featuring AI captioning via Florence-2, custom base model support, and direct publishing to Hugging Face. Supports installation via Pinokio, manual setup, or Docker Compose.

Tokens
2.9K
Snippets
5
Records
19
Agent score
85%

What's inside Flux Gym

  1. Upload Caption Files for Training

    main

    Instead of manual captioning in the UI, you can upload pre-made caption files. Follow this naming convention:

    • Every caption must be a .txt file.
    • Each .txt file must match an image file name exactly.
    • Example: img0.png must have a corresponding img0.txt.
  2. Install Flux Gym Manually

    main

    To install manually, you must clone both the fluxgym repository and the sd-scripts repository (specifically the sd3 branch), set up a Python virtual environment, and install the required dependencies for both.

    # 1. Clone repositories
    git clone https://github.com/cocktailpeanut/fluxgym
    cd fluxgym
    git clone -b sd3 https://github.com/kohya-ss/sd-scripts
    
    # 2. Create and activate virtual environment
    # Windows:
    python -m venv env
    env\Scripts\activate
    # Linux:
    python -m venv env
    source env/bin/activate
    
    # 3. Install dependencies
    cd sd-scripts
    pip install -r requirements.txt
    cd ..
    pip install -r requirements.txt
    
    # 4. Install PyTorch Nightly
    # Standard:
    pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
    
    # For NVIDIA RTX 50-series (e.g., 5090):
    pip install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128
    pip install -U bitsandbytes
    
    # 5. Start the application
    python app.py
  3. Configure Automatic Sample Image Generation

    main

    By default, sample images are not generated. To enable them, configure the following in the UI:

    • Sample Image Prompts: Enter prompts used to generate images during training. Use a new line for multiple separate prompts.
    • Sample Image Every N Steps: Set the interval for generation. For example, if 'Expected training steps' is 960 and this is set to 100, images will generate at steps 100, 200, ..., 900 for each prompt.
  4. Manage datasets and AI captioning

    main

    Flux Gym allows you to upload images and generate captions for training:

    1. Upload Images: Use the image upload component to provide .image or .txt files.
    2. AI Captioning: You can use the Add AI captions with Florence-2 button to automatically generate captions for your uploaded images using the Florence-2 model. This process uses the concept_sentence to assist in captioning.
    3. Manual Editing: After uploading or captioning, you can manually edit the text in the Caption boxes to ensure the trigger word is correctly included.
  5. Configure LoRA training settings in Flux Gym

    main

    The Flux Gym WebUI provides several configuration options for training FLUX LoRAs. Key settings include:

    • LoRA Info:

      • lora_name: A unique name for your LoRA.
      • concept_sentence: The trigger word or sentence (e.g., p3rs0n or in the style of CNSTLL).
      • base_model: The foundation model (can be extended by editing models.yaml).
      • vram: Target VRAM capacity (20G, 16G, or 12G).
      • num_repeats: Number of repeat trains per image.
      • max_train_epochs: Maximum training epochs.
      • sample_prompts: Text prompts for generating sample images (separated by new lines).
      • sample_every_n_steps: Frequency of sample image generation.
      • resolution: The target resize resolution for dataset images.
    • Advanced options:

      • --seed: Random seed for reproducibility.
      • --max_data_loader_n_workers: Number of data loader workers.
      • --learning_rate: The training learning rate (default: 8e-4).
      • --save_every_n_epochs: Frequency of saving checkpoints.
      • --guidance_scale: Guidance scale value.
      • --timestep_sampling: Timestep sampling method (e.g., shift).
      • --network_dim: LoRA Rank (range: 4 to 128, step 4).
  6. Publish LoRAs to HuggingFace

    main

    The 'Publish' tab in the WebUI allows you to upload your trained LoRAs directly to HuggingFace:

    1. Authentication: Enter your Huggingface Token and click Login.
    2. Repository Setup:
      • Account: Your HuggingFace username (automatically populated after login).
      • Repository Name: The desired name for the new repository.
      • Repository Visibility: Set to 'public' or 'private'.
    3. Upload: Select the LoRA from the list and click Upload to HuggingFace to initiate the transfer.
  7. Deploy Fluxgym via Docker Compose

    main

    You can deploy Fluxgym using Docker Compose. The configuration maps port 7860 to the host and requires an NVIDIA GPU.

    Important CUDA Configuration: If you are running CUDA 12.4 drivers, you must change the dockerfile setting in the build section from Dockerfile to Dockerfile.cuda12.4. Otherwise, leave it as Dockerfile.

    Environment Variables:

    • PUID: User ID for the container (defaults to 1000).
    • PGID: Group ID for the container (defaults to 1000).

    Volumes:

    • /etc/localtime and /etc/timezone are mounted as read-only to sync container time with the host.
    • The current directory ./ is mounted to /app/fluxgym inside the container.
    services:
    
    fluxgym:
        build:
          context: .
          # change the dockerfile to Dockerfile.cuda12.4 if you are running CUDA 12.4 drivers otherwise leave as is
          dockerfile: Dockerfile
        image: fluxgym
        container_name: fluxgym
        ports:
          - 7860:7860
        environment:
          - PUID=${PUID:-1000}
          - PGID=${PGID:-1000}
        volumes:
          - /etc/localtime:/etc/localtime:ro
          - /etc/timezone:/etc/timezone:ro
          - ./:/app/fluxgym
        stop_signal: SIGKILL
        tty: true
        deploy:
          resources:
            reservations:
              devices:
              - driver: nvidia
                count: all
                capabilities: [gpu]
        restart: unless-stopped