DOCSEARCHbyAlgolia
Submit Docs

StableSwarmUI Documentation

repository·master·Indexed 26 days ago

https://github.com/stability-ai/stableswarmui

Documentation for StableSwarmUI, an interface for image and video generation. It features support for multiple GPUs, ComfyUI and Auto WebUI backends, and specialized tools like the Grid Generator, Image Batch Editor, and Dynamic Thresholding. The guide covers installation on Google Colab, API integration, model support, and the development of custom extensions.

Tokens
21.4K
Snippets
33
Records
163
Agent score
89%

What's inside StableSwarmUI

  1. Use Dynamic Thresholding in StableSwarmUI

    master

    Dynamic Thresholding is a built-in StableSwarmUI extension that allows you to generate images using very large CFG Scale values without causing image burning.

    To access the Dynamic Thresholding parameters, you must enable the advanced settings in the UI by checking the Display Advanced Options checkbox. Once enabled, you can configure parameters for Dynamic Thresholding and related features like CFG Scale Scheduling.

  2. Explore StableSwarmUI Documentation

    master

    StableSwarmUI provides several documentation guides for different user needs:

  3. Configure multi-GPU and multi-machine setups

    master

    Swarm allows you to connect multiple GPUs, multiple machines, or even remote machines over a network to form a cluster.

    • Generations are automatically queued across available backends.
    • You can split a single custom ComfyUI workflow across multiple backend GPUs (subject to certain limitations).

    For specific details on multi-GPU implementation, refer to the Using More GPUs documentation.

  5. Explore StableSwarmUI Features

    master

    StableSwarmUI includes several advanced features for image and video generation. Detailed documentation for specific features is organized into individual guides:

    • Presets: Information on using presets and accessing official high-quality SDXL preset lists.
    • Prompt Syntax: Details on the various syntax features available directly within the prompt box.
    • Autocompletions: Documentation for the prompt autocompletion engine.
    • IP Adapter and ReVision: Instructions for image-prompting using IP-Adapter and ReVision.
    • ControlNet: Guidance on utilizing ControlNets for generation control.
    • Video: Instructions for video generation workflows.

  6. Interact with the StableSwarmUI Network API

    master

    StableSwarmUI provides a full-capability network API for external programs to generate images and manage the Swarm instance (e.g., modifying backends).

    Most API calls are POST requests sent to (your server)/API/(route) with JSON inputs and JSON outputs.

    Key Concepts:

    • Session IDs: Almost all API routes (except GetNewSession) require a session_id in the JSON body. You must call /API/GetNewSession first to obtain one.
    • WebSocket Routes: Routes with a WS suffix (e.g., GenerateText2ImageWS) use WebSockets to provide streaming updates like progress bars or preview images.
    • Error Handling: Routes may return an error (display text for users) or an error_id. If error_id is invalid_session_id, you must call /API/GetNewSession to refresh your session and retry the request.

  7. Use the StabilityAPI Extension for remote Stability API calls

    master

    The StabilityAPI extension adds a backend to StableSwarmUI that allows you to make remote calls to the Stability API instead of running models locally.

    Requirements:

    Configuration Note: Currently, to use this extension, you must select a local model and then select the "Engine" as a parameter within the UI.

  8. Configure Stable Diffusion 3 (SD3) parameters

    master

    Stable Diffusion 3 Medium is supported natively. On the first run, Swarm will automatically download the required text encoders.

    Key Parameters:

    • SD3 TextEncs (under Sampling group): Select CLIP, T5, or both. Using CLIP only provides better performance on resource-constrained systems with near-identical results.
    • Sigma Shift (under Advanced Sampling): Defaults to 3. You can experiment with values around ~1.5, but extreme values are not recommended.
    • Refiner Do Tiling: Highly recommended when performing upscaling with SD3, as the model does not respond well to regular upscaling without tiling.

  9. Generate an image via API

    master

    To generate an image, follow these steps:

    1. Obtain a Session ID: Call the /API/GetNewSession endpoint via POST.
    2. Generate Image: Call the /API/GenerateText2Image endpoint via POST, providing the session_id, prompt, model, and other parameters in the JSON body.
    3. Retrieve Image: The API returns a JSON object containing a list of image paths. Use the /View/* route to download or view the generated image.

    Example Workflow (Bash):

    # 1. Get session ID
curl -H "Content-Type: application/json" -d "{}" -X POST http://localhost:7801/API/GetNewSession

# 2. Generate image (using session_id from step 1)
curl -H "Content-Type: application/json" -d '{"session_id":"YOUR_SESSION_ID","images":1,"prompt":"a cat","model":"OfficialStableDiffusion/sd_xl_base_1.0","width":1024,"height":1024}' -X POST http://localhost:7801/API/GenerateText2Image

# 3. Download the image (using the path returned in step 2)
wget "http://localhost:7801/View/local/raw/path/to/image.png"
    # First, get a usable session ID:
curl -H "Content-Type: application/json" -d "{}" -X POST http://localhost:7801/API/GetNewSession

# Now, generate the image
curl -H "Content-Type: application/json" -d '{"session_id":"9D3534E30DA38499DE782BC38211976A58555AA6","images":1,"prompt":"a cat","model":"OfficialStableDiffusion/sd_xl_base_1.0","width":1024,"height":1024}' -X POST http://localhost:7801/API/GenerateText2Image

# Now download the image to look at it
wget "http://localhost:7801/View/local/raw/2024-05-19/a cat-OfficialStableDiffusionsd_xl_base_10s-1872258705.png"

  12. Use From-To prompt swapping

    master

    Swap between two phrases at a specific timestep during the generation process.

    • Syntax: <fromto[timestep]:before, after>
    • Timestep formats:
      • Integer (e.g., 10 for step 10)
      • Float (e.g., 0.5 for halfway through)
    • Requirements: Must have exactly two entries separated by a comma.
    • Example: <fromto[0.5]:cat, dog> swaps from cat to dog halfway through.
    <fromto[0.5]:cat, dog>