DOCSEARCHbyAlgolia
Submit Docs

ComfyScript

repository·main·Indexed 20 days ago

https://github.com/chaoses-ib/comfyscript

A Python frontend and library for ComfyUI (version 0.6.1) that allows users to interact with ComfyUI nodes as Python functions. It enables writing workflows as readable code to facilitate automation, LLM-driven generation, and complex logic like loops. Features include a transpiler to convert JSON workflows to Python, support for CivitAI model loading, asynchronous task queue management, and integration with ipywidgets and Solara for interactive UIs.

Tokens
19.8K
Snippets
72
Records
90
Agent score
71%

What's inside comfy-script

  1. Understand Standalone vs Client runtime

    main

    ComfyScript supports two runtime modes depending on how the ComfyUI server is managed:

    1. Standalone runtime: The runtime starts and manages the ComfyUI server itself. Supports both Virtual and Real modes.
    2. Client runtime: The runtime connects to an existing, running ComfyUI server. Supports Virtual mode only.
    FeatureStandaloneClient
    Virtual mode✔️✔️
    Real mode✔️❌

    Standalone runtime also provides additional metadata in type stubs, such as the node's module name, to help identify the source of a node.

  2. Understand Node Compatibility in ComfyScript

    main
    ComfyScript supports all ComfyUI built-in nodes and almost all custom nodes. Built-in and custom nodes are treated identically: they are loaded on-the-fly and have type stubs (nodes.pyi) generated automatically. The only major exception is UI-only JS nodes, which cannot be executed by the server or accessed by the ComfyScript transpiler/runtime.

  3. Enable Autocomplete and Type Safety with Type Stubs and Enumerations

    main

    ComfyScript generates type stubs at comfy_script/runtime/nodes.pyi after calling load(). This enables autocompletion and type checking in editors like VS Code.

    Additionally, Python enumerations are generated for arguments that have a fixed set of values. Instead of using raw strings, you can use these enumerations for better reliability:

    • Checkpoints: Use Checkpoints.filename or CheckpointLoaderSimple.ckpt_name.filename.
    • Embeddings: Use Embeddings.name (which resolves to 'embedding:name').
    # Using enumerations instead of strings
model, clip, vae = CheckpointLoaderSimple(Checkpoints.AOM3A1B_orangemixs)
# Or for embeddings
neg = CLIPTextEncode(Embeddings.easynegative, clip)

  4. Implement ComfyUI UI features in ComfyScript

    main

    ComfyScript does not use the ComfyUI Web UI's internal formats (like S&R, mute, or groups) because they are UI-only elements. Instead, use standard Python patterns to achieve the same results:

    • Search & Replace (S&R): Use Python variables and f-strings.
    • Mute/Bypass: Use Python comments (#) to disable code or use if statements to conditionally execute nodes.
    • Groups: Organize related code into logical blocks or encapsulate them within Python functions.
    • String Handling: Use triple-quoted strings (''') for multi-line prompts or raw strings (r'') for file paths containing backslashes.
    # Search & Replace via variables
pos = 'beautiful scenery nature glass bottle landscape'
neg = 'text, watermark'

# Mute/Bypass via if statements
sample = False
if sample:
    latent = KSampler(model, seed, steps, 8, 'euler', 'normal', CLIPTextEncode(pos, clip), CLIPTextEncode(neg, clip), latent, 1)

# Groups via functions
def generate_image(pos, neg, seed, steps):
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    # ... logic

  5. Automatic script saving when installed as custom nodes

    main

    If ComfyScript is installed as a ComfyUI custom node, it hooks into nodes like SaveImage. When these nodes are executed, the corresponding ComfyScript code is automatically saved into the image's metadata and printed to the terminal.

    Configuration for these features can be managed via settings.example.toml.

  6. Use ComfyScript as a Python library for ComfyUI nodes

    main

    ComfyScript allows you to use ComfyUI nodes as standard Python functions. This enables advanced logic like loops, conditional branching, and integration with other Python libraries that are difficult to implement in a graph-based GUI.

    Core Workflow Pattern:

    1. Import comfy_script.runtime and comfy_script.runtime.nodes.
    2. Use load(url_or_path) to connect to a ComfyUI server or local path.
    3. Wrap node calls within a with Workflow(wait=True): context manager to execute the graph.
    from comfy_script.runtime import *
load('http://127.0.0.1:8188/')
from comfy_script.runtime.nodes import *

with Workflow(wait=True):
    # Node calls look like standard Python functions
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    # ... rest of workflow

  7. Use Virtual mode for workflow generation

    main

    In Virtual mode, calling a node does not execute it immediately. Instead, the script builds a representation of the workflow. The entire workflow is only executed when sent to the ComfyUI server.

    Key characteristics:

    • Workflow Generation: You can export the workflow to ComfyUI's API JSON format using wf.api_format_json().
    • Limitation: You cannot access node outputs (like images or tensors) directly in Python before the full workflow runs.
    • Async: Virtual mode is internally asynchronous but exposes synchronous APIs for ease of use. To use async APIs, prefix methods with an underscore and use await (e.g., await runtime._load()).
    with Workflow(queue=False) as wf:
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    conditioning = CLIPTextEncode('beautiful scenery...', clip)
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 123, 20, 8, 'euler', 'normal', conditioning, conditioning, latent, 1)
    SaveImage(VAEDecode(latent, vae), '0')

# Generate the API JSON
json = wf.api_format_json()
with open('prompt.json', 'w') as f:
    f.write(json)

  8. Use Real mode for direct execution and research

    main

    In Real mode, calling a node executes it immediately in the current Python process. This is achieved by importing from comfy_script.runtime.real.

    Use cases:

    • ML research and direct tensor manipulation.
    • Reusing custom nodes in other projects.
    • Developing and debugging custom nodes.
    • Integrating ComfyUI into other environments like sd-webui.

    Key characteristics:

    • Direct Access: You get actual ComfyUI objects (like ModelPatcher, CLIP, VAE) and tensors directly in your script.
    • No Caching: It does not use ComfyUI's built-in cache system; users must manage variable lifetimes to optimize speed.
    • No API execution: Scripts in Real mode cannot be executed via a ComfyUI server's API.
    • Output types: Output nodes (like SaveImage) return dictionary metadata rather than ComfyScript result classes.
    from comfy_script.runtime.real import *
load()
from comfy_script.runtime.real.nodes import *

with Workflow():
    model, clip, vae = CheckpointLoaderSimple('v1-5-pruned-emaonly.ckpt')
    # model is a <comfy.model_patcher.ModelPatcher> object
    
    latent = EmptyLatentImage(512, 512, 1)
    latent = KSampler(model, 123, 20, 8, 'euler', 'normal', conditioning, conditioning2, latent, 1)
    
    image = VAEDecode(latent, vae)
    # image is a torch.Tensor
    
    print(SaveImage(image, 'ComfyUI'))
    # Returns metadata: {'ui': {'images': [...]}}

  9. Work around UI-only JS nodes during transpilation

    main

    Some nodes exist only in the ComfyUI web interface (written in JS) and are removed or converted before execution. Because ComfyScript's transpiler cannot access these, certain JSON workflows in the web UI format may fail to transpile.

    To resolve transpilation failures caused by UI-only nodes, use one of these two methods:

    1. Transpile from an image: If the image metadata includes the API format workflow, the transpiler will automatically fallback to the API format if the web UI format fails.
    2. Export in API format: Instead of using the standard web UI JSON, export the workflow using the API format.
      • Enable Dev Mode in ComfyUI settings.
      • Click Save (API format) in the ComfyUI interface.
      • Attempt to transpile this exported file.

  10. Install ComfyScript with Solara UI support

    main

    To use Solara widgets (like MetadataViewer) in Jupyter Notebooks or web pages, you must install ComfyScript with the solara extra.

    If you are developing within a ComfyUI directory (editable install):

    pip install -e ".[default,solara]"

    If you are installing the standalone ComfyScript package:

    pip install "comfy-script[default,solara]"
    pip install -e ".[default,solara]"
# OR
pip install "comfy-script[default,solara]"

  11. Enable Naked Mode for direct ComfyUI compatibility

    main

    Naked mode provides a 'real real' execution mode that bypasses ComfyScript's internal modifications designed for developer experience. In naked mode, ComfyScript will not execute any code after load() except for code wrapped in a Workflow() block (which can be conceptually replaced with torch.inference_mode()). This mode is fully compatible with scripts generated by the ComfyUI-to-Python-Extension.

    import random
from comfy_script.runtime.real import *
load(naked=True)
from comfy_script.runtime.real.nodes import *

with Workflow():
    checkpointloadersimple = CheckpointLoaderSimple()
    # ... rest of your workflow

  12. Install ComfyScript for external ComfyUI servers

    main

    If you want to use ComfyScript to develop apps or libraries that connect to an external ComfyUI server (e.g., a cloud server), install the package via pip. It is recommended to use the [default] extra to ensure common dependencies are included.

    To test your installation, you can run a script that uses load() to connect to your server address.

    python -m pip install -U "comfy-script[default]"

# Test script
from comfy_script.runtime import *
load('http://127.0.0.1:8188/')
from comfy_script.runtime.nodes import *

with Workflow(wait=True):
    image = EmptyImage()
    images = util.get_images(image, save=True)