DOCSEARCHbyAlgolia
Submit Docs

LoRA Block Weight Plot Helper

repository·main·Indexed 40 days ago

https://github.com/zuntan03/lorablockweightplothelper

A tool for fine-tuning specific layer weights of a LoRA using X/Y/Z plots to identify layer contributions and reduce artifacts. It integrates with Stable Diffusion web UI extensions including LoRA Block Weight, SuperMerger, and sd-scripts, providing weight string parsing (ParseWeight17, ParseWeight26) and export formats for the Effective Block Analyzer and layer-wise learning rate configurations.

Tokens
2.4K
Snippets
6
Records
11
Agent score
50%

What's inside LoRA Block Weight Plot Helper

  1. Use Weighted Tags for training

    main

    Newer training features allow for weighted tag training using prefixes. This allows you to adjust the influence of specific tags during the training process.

    • _a: Affects all tags.
    • _k: Affects the specific keyword tag (e.g., gobuhime).
    • _s: Affects specific tags (e.g., greenskin).

    Weighting Syntax: Following the prefix, a two-digit number represents the multiplier:

    • 05 $\rightarrow$ 0.5 weight
    • 15 $\rightarrow$ 1.5 weight
    • 30 $\rightarrow$ 3.0 weight

    Note: class_tokens are not affected by this weighting.

  2. Understanding the difference between LoRA Block Weight and Layer-wise Learning Rates

    main

    It is critical to understand that applying LoRA Block Weight values directly to Layer-wise Learning Rates (C3Lier/sd-scripts) will not produce identical results.

    Key Observations

    • Layer-wise Learning Rate (LLR) vs. Block Weight: While Block Weights are used during inference to scale existing LoRA weights, LLR is used during the training process to scale the learning rate of specific layers.
    • Zeroing out layers: In LLR, setting a layer's learning rate to 0 might cause the model to compensate by increasing learning in other layers, making control more difficult compared to simply disabling a block in LoRA Block Weight.
    • Sensitivity: LLR often responds more predictably to changes in 'learning amount' than adjusting the number of training steps. Reducing the learning rate is often more effective at preserving the training intent than halving the step count.
    • Balance: Using LLR values that do not create extreme differences between layers (e.g., 876, 654, 432) tends to produce more stable images, whereas extreme values (e.g., 777, 555, 333) can significantly reduce the effectiveness of layer-specific control.

  3. Investigate layer influence using Effective Block Analyzer

    main

    The Effective Block Analyzer feature in the LoRA Block Weight extension allows you to identify which layers of a LoRA are responsible for specific visual changes.

    Usage Steps:

    1. Prepare Prompt: Ensure your LoRA call in the prompt ends with :XYZ>. Example: <lora:your_lora_name:1:XYZ>
    2. Configure LoRA Block Weight Settings:
      • Select Effective Block Analyzer from the LoRA Block Weight dropdown.
      • Check change X-Y.
    3. Configure Scripts:
      • Set the Script dropdown (bottom left of WebUI) to None (do not enable the standard X/Y/Z plot while using the analyzer).
    4. Generate: Click Generate to produce a grid.

    Interpreting Results:

    • Top Row: The original image.
    • Middle Row: Images where layer weights were reduced to 0.5.
    • Bottom Row: The difference between the top and middle rows. Colored areas in the difference image indicate where increasing the weight from 0.5 to 1.0 has the most impact.

    Note: If you are using hires.fix, ensure your LoRA Block Weight extension is updated to a version that supports it.

    1girl, gobuhime, longhair, hairornament, choker <lora:gobuhime-Defmix2:1:XYZ>

  4. Use LoRA Block Weight Plot Helper to define layer weights

    main

    Once you have identified important and unimportant layers using the Effective Block Analyzer, use the LoRA Block Weight Plot Helper (Google Spreadsheet) to generate a weight string.

    Configuration in the Spreadsheet:

    • AddSubValue: Set to 0.5 (default).
    • NAME: Enter your desired tag name (e.g., Gobuhime).
    • Important Layers: Set weight to 1 and check Fix.
    • Unimportant Layers: Set weight to 0 and check Fix.
    • Uncertain Layers: Set weight to 0.5 and do not check Fix.

    Applying to WebUI:

    1. Copy the generated string (e.g., Gobuhime:1,0,0.5,1...).
    2. In WebUI, set Effective Block Analyzer to Disable.
    3. Paste the string into the Weights setting field.
    4. Click Save Presets and Reload Tags to activate the tag.
    5. Change your prompt's LoRA suffix from :XYZ> to :ALL>. Example: <lora:gobuhime-Defmix2:1:ALL>
    Gobuhime:1,0,0.5,1,0.5,0.5,0.5,0,1,1,0,0,0,1,0.5,0.5,1,1,1,1,0.5,0.5,0.5,0.5,0,0.5

  5. Merge optimized LoRA weights into a model using SuperMerger

    main

    Once you have determined the optimal layer weights, you can merge them directly into the LoRA file using the SuperMerger extension.

    Steps:

    1. In LoRA Block Weight, set the Weights setting to your finalized weight string and click Save Presets and Reload Tags.
    2. Open the SuperMerger tab in WebUI and select the LoRA sub-tab.
    3. Click update list and select the LoRA you wish to merge.
    4. In the LoRAname1:ratio1:Blocks1,... field, append your weight identifier (e.g., if the field shows filename:1.0, change it to filename:1.0:Gobuhime).
    5. Click Merge LoRAs. The new LoRA file will be saved (the path is shown in the Message log).
    filename:1.0:Gobuhime

  6. Fine-tune LoRA layers using X/Y/Z plots

    main

    You can use the tool to find the optimal weights for each LoRA layer by generating X/Y/Z plots that add or subtract weight from non-fixed layers. This helps reduce negative effects like color bleeding or color shifting.

    Workflow:

    1. Prepare Prompt: Change your LoRA tag from ALL to Sub (e.g., <lora:name:1:ALL> $\rightarrow$ <lora:name:1:Sub>).
    2. Configure Weights: Copy SubWeights into the LoRA Block Weight Weights setting.
      • Tip: Backup your current weights before pasting to allow easy restoration via Ctrl+A/Ctrl+V.
    3. Update Tags: Click Save Presets and Reload Tags to refresh the tag list.
    4. Set Plot Parameters: Copy SubPromptSR into the Prompt S/R field of the X/Y/Z plot.
    5. Generate and Analyze: Run the generation. The tool will subtract the AddSubValue from the current weight (default 0.5) of non-fixed layers.
    6. Fix Layers: Observe the grid. If a layer's weight can be reduced to 0 without losing quality, set that layer to 0 and click Fix. Repeat this process until all desired layers are fixed.
    7. Finalize: Once all layers are fixed, the resulting weight string is displayed in the WEIGHT field on the right.
    1girl, gobuhime, longhair, hairornament, choker <lora:gobuhime-Defmix2:1:Sub>

  8. Parse weight strings for different LoRA types

    main

    The tool can convert weight strings into a readable format using the weight parsing feature. Use the appropriate parser based on the LoRA type:

    • ParseWeight26: Use this for C3Lier, Locon, and LoHa weights.
    • ParseWeight17: Use this for standard LoRA weights.

    If a standard LoRA does not support specific layers, you can specify a FillValue to handle them.

  9. Default LoRA Block Weight presets

    main

    The following preset weight strings are available by default in the tool for quick testing:

    NONE:0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
ALL:1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
INS:1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0
IND:1,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0
INALL:1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0
MIDD:1,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0,0
OUTD:1,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0
OUTS:1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1
OUTALL:1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1
ALL0.5:0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5
    NONE:0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0
ALL:1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1,1
INS:1,1,1,1,0,0,0,0,0,0,0,0,0,0,0,0,0
IND:1,0,0,0,1,1,1,0,0,0,0,0,0,0,0,0,0
INALL:1,1,1,1,1,1,1,0,0,0,0,0,0,0,0,0,0
MIDD:1,0,0,0,1,1,1,1,1,1,1,1,0,0,0,0,0
OUTD:1,0,0,0,0,0,0,0,1,1,1,1,0,0,0,0,0
OUTS:1,0,0,0,0,0,0,0,0,0,0,0,1,1,1,1,1
OUTALL:1,0,0,0,0,0,0,0,1,1,1,1,1,1,1,1,1
ALL0.5:0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5,0.5

  11. Configure Layer-wise Learning Rates in sd-scripts

    main

    To apply layer-wise learning rates during training (using sd-scripts or similar tools), you must provide specific weight strings for the down, mid, and up blocks.

    Example configuration for a specific layer setup:

    • down_lr_weight: A comma-separated list of weights for the down blocks.
    • mid_lr_weight: A single weight for the middle block.
    • up_lr_weight: A comma-separated list of weights for the up blocks.

    Note: The weight labels 1.0 and 1 (or 0.0 and 0) are treated interchangeably in the command line arguments.

    "down_lr_weight=0.6,0.8,1,0.8,0.8,0.6,0.6,0.6,1,0.6,0.6,0.6" "mid_lr_weight=1" "up_lr_weight=0.8,0.6,1,1,1,1,0.8,0.8,0.8,0.6,0.6,0.8"