DOCSEARCHbyAlgolia
Submit Docs

ComfyUI API

repository·main·Indexed 19 days ago

https://github.com/saladtechnologies/comfyui-api

A stateless, extendable wrapper for ComfyUI designed for high-scale, production-ready deployments. It transforms ComfyUI into a scalable API supporting asynchronous workflows, automated model management via manifest files, and flexible output delivery to S3-compatible storage, Azure Blob, Huggingface, or HTTP endpoints. Features include dynamic model loading, LRU caching, and support for image-to-image workflows.

Tokens
22.5K
Snippets
62
Records
94
Agent score
62%

What's inside comfyui-api

  1. Overview of ComfyUI API features

    main

    ComfyUI API is a stateless wrapper for ComfyUI that enables horizontal scaling and flexible output handling.

    Key capabilities include:

    • Full ComfyUI Support: Executes any API-formatted prompt, including custom nodes and workflows.
    • Statelessness: Designed for scaling; handles input downloads (like images) before execution.
    • Flexible Output: Supports synchronous base64 responses or asynchronous delivery via webhooks to S3-compatible storage, Azure Blob, Huggingface, or HTTP endpoints.
    • Model Management: Supports dynamic model loading from URLs (with local caching), on-demand downloads via API, and automatic installation via a manifest file.
    • Observability: Provides /health and /ready probes, and can forward ComfyUI websocket events to a webhook.
    • Image Processing: Supports returning images in PNG, JPEG, or WebP formats using sharp parameters.

  2. How storage providers work in ComfyUI-API

    main

    Storage providers are modular components responsible for downloading models and input media, and uploading completed outputs. The server uses a matching mechanism to select the correct provider based on the URL provided in a request body.

    Key Concepts

    • Automatic Selection: The server iterates through registered providers and calls their testUrl(url) method. The first provider that returns true for a given URL is selected to handle the request.
    • Optional Capabilities: Providers are not required to implement all methods. Some may only support downloading, while others may only support uploading.
    • Catch-all Pattern: The HTTPStorageProvider should always be the last provider in the registration list to act as a fallback for URLs that do not match specialized providers.
    export interface StorageProvider {
  /**
   * The key in a request body that indicates this storage provider should be used for upload.
   * Must be unique across all storage providers, and must be included if `uploadFile` is implemented.
   */
  requestBodyUploadKey?: string;

  /**
   * The zod schema for the request body field that indicates this storage provider should be
   * used for upload. Must be included if `requestBodyUploadKey` is defined.
   */
  requestBodyUploadSchema?: z.ZodObject<any, any>;

  /**
   * Takes the inputs from the request body and generates a URL for uploading.
   * @param inputs
   */
  createUrl(inputs: any): string;

  /**
   * Test if the given URL can be handled by this storage provider.
   * @param url URL to test
   */
  testUrl(url: string): boolean;

  /**
   * Upload a file to the given URL.
   * @param url URL to upload to
   * @param fileOrPath File path or buffer to upload
   * @param contentType MIME type of the file
   * 
   * @returns An Upload object that can be used to start and abort the upload.
   */
  uploadFile?(
    url: string,
    fileOrPath: string | Buffer,
    contentType: string
  ): Upload;

  /**
   * Download a file from the given URL to the specified output directory.
   * @param url URL to download from
   * @param outputDir Directory to save the downloaded file
   * @param filenameOverride Optional filename to use instead of auto-generated one
   * 
   * @resolves The path to the downloaded file
   */
  downloadFile?(
    url: string,
    outputDir: string,
    filenameOverride?: string
  ): Promise<string>;
}

  3. Use Workflow Webhooks with .webhook_v2

    main

    To receive asynchronous notifications for workflow completion or failure, include the .webhook_v2 field in your request to /prompt or other workflow endpoints.

    Unlike the legacy .webhook field (which sends individual requests for every output), .webhook_v2 sends a single webhook request once the entire workflow has completed, containing all outputs.

    Webhooks are sent as Standard Webhooks and are signed. You can validate them using the WEBHOOK_SECRET environment variable and a library like svix.

    // Example prompt.complete webhook schema
{
  "type": "prompt.complete",
  "timestamp": "2025-01-01T00:00:00Z",
  "id": "request-id",
  "images": ["base64-encoded-image-1", "base64-encoded-image-2"],
  "filenames": ["output-filename-1.png", "output-filename-2.png"],
  "prompt": {},
  "stats":{}
}

  4. Understand synchronous vs asynchronous response formats

    main

    The server responds differently based on whether you requested an asynchronous workflow (using a webhook or S3/Azure upload).

    Synchronous Requests (No webhook or s3.async is false):

    • Returns 200 OK immediately after the prompt completes.
    • The response body contains the outputs (images as base64 strings, filenames, and execution stats).

    Asynchronous Requests (Webhook or S3/Azure upload provided):

    • Returns 202 Accepted immediately.
    • The actual outputs are sent to your webhook_v2 URL or uploaded to your storage provider in the background.

  5. How ComfyUI API handles workflows

    main

    The server acts as a proxy for the ComfyUI /prompt API. When you submit a prompt:

    1. The server intercepts the request.
    2. It downloads any required input media (e.g., images) specified in the workflow.
    3. It overrides the filename_prefix field to ensure output files have unique names.
    4. It queues the prompt in ComfyUI.
    5. Once complete, it delivers the output based on your request parameters (direct response, webhook, or cloud storage upload).

  6. Core design principles for ComfyUI-API contributors

    main

    When contributing to the project, adhere to these architectural guidelines:

    • Asynchronous Operations: Use async programming to prevent blocking the event loop and ensure server responsiveness.
    • Modularity: Keep components modular and loosely coupled to support broad use cases.
    • Don't Duplicate Existing ComfyUI functionality: Leverage existing ComfyUI API endpoints instead of re-implementing them. The API server can access local ComfyUI via config.comfyURL.
    • Error Handling: Implement robust error handling with clear messages. Errors should not crash the server unless recovery is impossible.
    • Testing: Include tests for significant features or bug fixes to maintain codebase integrity.

  7. How the server handles file downloading and caching

    main

    The server uses a content-addressable cache to avoid redundant downloads.

    Caching Mechanism:

    • Files are identified by hashing their URL.
    • The default cache directory is $HOME/.cache/comfyui-api.
    • Files are stored using the first 32 characters of the URL hash plus the file extension, and are symbolically linked to the requested local_path.
    • If a download for a specific URL is already in progress, subsequent requests for that same URL will wait for the first one to finish.

    Storage Backend Detection:

    • S3: URLs starting with s3:// (requires AWS credentials via env vars).
    • Huggingface: URLs starting with https://huggingface.co/ (uses hf cli; requires HF_TOKEN).
    • Azure Blob: URLs matching https://<account>.blob.core.windows.net/ (requires Azure credentials via env vars).
    • HTTP(S): All other URLs (uses fetch to stream to disk). Supports Basic Auth via https://user:pass@host and custom headers via HTTP_AUTH_HEADER_NAME/HTTP_AUTH_HEADER_VALUE env vars.

  9. Configure Warmup Prompts

    main

    You can specify a warmup prompt to be loaded and parsed at startup. This allows you to define a default workflow (and its associated checkpoints) without rebuilding your Docker image.

    • WARMUP_PROMPT_FILE: Path to a local warmup prompt file. This takes precedence if both variables are set.
    • WARMUP_PROMPT_URL: URL to download a remote warmup prompt from.

    The checkpoint used in the warmup prompt can be used as the default for workflow models.

  10. How the stateless ComfyUI API works

    main
    The ComfyUI API server is stateless, meaning it does not store state between requests. This architecture enables horizontal scaling behind a load balancer. To ensure readiness and model loading, the server uses a configurable warmup workflow. Documentation is available via self-hosted Swagger docs and an OpenAPI spec at the /docs endpoint.

  11. Create custom workflow endpoints

    main

    Custom workflows allow you to abstract the complex ComfyUI node-based prompt format into simple, specific API endpoints for your use cases.

    • Implementation: Workflows can be written in either JavaScript or TypeScript.
    • Runtime Loading: Workflows are loaded at runtime. This means you can add or update workflows without rebuilding your Docker images or pre-compiled binary releases.
    • Complexity: Workflows can range from simple parameter mappings to highly complex logic.

    For detailed instructions, refer to the guide on generating new workflow endpoints.

  12. Upload outputs to S3 or Azure Blob Storage

    main

    You can configure the API to automatically upload generated images to cloud storage instead of returning them as base64 strings.

    S3 Configuration: Provide an s3 object in your request with bucket, prefix, and optionally async (defaults to true if not specified in some contexts, but explicitly set to false for synchronous responses).

    Azure Blob Configuration: Provide an azure_blob_upload object with container and blob_prefix.

    When using these, the images array in the response (or webhook payload) will contain the full URLs to the uploaded files.

    // S3 Example
{
  "prompt": { ... },
  "s3": {
    "bucket": "my-bucket",
    "prefix": "outputs/",
    "async": false
  }
}

// Azure Blob Example
{
  "prompt": { ... },
  "azure_blob_upload": {
    "container": "my-container",
    "blob_prefix": "outputs/",
    "async": false
  }
}