DOCSEARCHbyAlgolia
Submit Docs

ComfyTV

repository·main·Indexed 42 days ago

https://github.com/jtydhr88/comfytv

A canvas-based media workbench and LibTV-style pipeline for ComfyUI that extends it into a production environment for image, video, audio, and 3D content. It features project-centric workflows with independent stage execution, a built-in Bot agent, an MCP server with 45 tools for AI agents, and comprehensive toolsets for 2D layer editing, storyboarding, and audio/music synthesis.

Tokens
346.8K
Snippets
358
Records
1.7K
Agent score
62%

What's inside ComfyTV

  1. Overview of ComfyTV Bot

    main

    The ComfyTV Bot is a chat interface embedded in the sidebar that acts as an agent to drive your ComfyUI canvas. Instead of manual node placement, you can use natural language to build workflows, run renders, manage assets, and iterate on results.

    Key Capabilities:

    • Workflow Construction: "Add a Z-Image Turbo node with a prompt for a neon cat at night, 16:9, and run it."
    • Video/Audio Tasks: "Use that image as a reference for a 5-second image-to-video, wait for it to finish, and check the first frame."
    • Canvas Inspection: "Look at my canvas and tell me why the video node failed."
    • Timeline Control: "Open the director's timeline and re-shoot segment 3 with slower camera movement."

    Security Design: The Bot does not store API keys. It drives agent CLIs already installed on your machine or connects to local LLM providers running on your own hardware.

  2. Overview of ComfyTV

    main

    ComfyTV is a canvas-based media workbench that transforms ComfyUI into a comprehensive pipeline for media production. It operates on a node-based architecture where every operation is a stage, and results flow downstream automatically.

    Users can chain stages into complete pipelines following the pattern: generate → pick → edit → composite → export. The system supports approximately 190 stages across several domains:

    • Image: Inpainting, outpainting, upscaling, and browser-side editing (crop, rotate, mirror, grid split).
    • Video: ~100 nodes including color grading, keying, roto/tracking, compositing (39 blend modes), and FX chains.
    • Audio: Dynamics, EQ, denoise, stem splitting, and audio-reactive automation.
    • Music: Composition via MusicXML, piano-roll editors, and SoundFont synthesis.
    • Panorama & 3D: 360° panorama tools and a Scene3D workbench with keyframed cameras and PBR materials.
    • 2D Layers: Layered editor with PSD import/export and storyboard workbench.

  3. Overview of ComfyTV current capabilities and roadmap

    main

    ComfyTV is currently in a pre-release state. The project provides a comprehensive suite of tools for image, video, audio, 3D, and panorama generation and editing.

    Key completed features include:

    • Image Generation & Editing: Local SD1.5 (T2I/I2I), Ideogram 4, Flux Canny Edit, Flux Fill Inpaint/Outpaint, LaMa Erase, and BiRefNet Cutout.
    • Video Generation: Local LTX 2.3 (T2V, I2V, FLF2V, IA2V) and MiniMax H3 (T2V, FLF2V, R2V with sound).
    • Audio & Music: MiniMax Music 3 and ACE-Step v1 Music.
    • 3D & Panorama: Hunyuan3D 2.1, MoGe-2, TripoSplat, and Qwen-based Image-to-Panorama/Text-to-Panorama.
    • Production Tools: Director's Console (timeline control), MCP Service (Agent endpoints), and Bridge nodes for third-party ComfyUI plugin integration.

    Core non-AI suites available:

    • Video Suite: ~100 nodes for editing, color grading, keying, rotoscoping, and compositing.
    • Audio Suite: 30+ nodes for dynamics, EQ, noise reduction, and stem splitting.
    • Music Suite: Symbolic music tools including MusicXML, MIDI editing, and SoundFont synthesis.
    • 2D Layer Editor: Raster/Vector/Text layers with PSD import/export and Storyboard support.

  4. Overview of ComfyTV capabilities

    main

    ComfyTV provides approximately 190 stages across several media domains:

    • Image: Generation (T2I, I2I, inpaint, outpaint, etc.), SAM-based part splitting, and browser-side tools (crop, rotate, etc.).
    • 2D Layer Editor & Storyboard: A layered editor (raster, text, vector, adjustment layers) with PSD import/export. The storyboard includes onion skinning and timeline playback.
    • Video: ~100 nodes covering editing (clip, split, concat), color (LUT, curves), keying (chroma key), roto/tracking, compositing (39 blend modes), and FX (glow, lens distortion, etc.).
    • Audio: 30+ nodes for dynamics, EQ, reverb (convolution and algorithmic), stem splitting, and audio-reactive automation.
    • Music: Symbolic music tools including MusicXML, piano-roll MIDI editors, and SoundFont (SF2/SF3) synthesis.
    • 3D & Panorama: 360° viewers, text/image-to-panorama, and a Scene3D DCC-style stage with PBR material binding and geometry workshops.

  5. Overview of ComfyTV Feature Status

    main

    ComfyTV is currently in a pre-release state. The project's capabilities are divided into completed features (✅ Done), non-AI suites that work instantly on local footage (✅ Suites), and upcoming features (⏳ Pending).

    Key Capabilities

    • Image Generation & Editing: Supports T2I, I2I, Inpainting (Flux/SDXL), Outpainting, Erasing (LaMa), Relighting (Flux2 Klein), and Background Removal (BiRefNet).
    • Video Generation: Includes Local LTX 2.3 (T2V, I2V, FLF2V, IA2V) and MiniMax H3 (Video with sound).
    • Audio & Music: Text-to-music (MiniMax Music 3, ACE-Step), audio stem splitting, and symbolic music editing (MIDI/MusicXML).
    • 3D & Panorama: Image-to-3D (Hunyuan3D, MoGe-2, TripoSplat) and Image/Text-to-Panorama (Qwen-Image-Edit).
    • Production Tools: A Director console for shot-by-shot production, an MCP server for agentic workflows, and a master timeline for clip management.

  6. Overview of ComfyTV content capabilities

    main

    ComfyTV provides specialized nodes across several media domains:

    • Image: Generation (T2I, I2I, Inpaint, Outpaint, etc.), browser-based tools (crop, rotate, split), SAM segmentation, and line art extraction.
    • 2D Layer Editor & Storyboard: Full layer editor (raster, text, vector, parametric fills) with mask support, non-destructive transforms, and PSD import/export. Storyboard tools include onion skinning and timeline playback.
    • Video (~100 nodes): Editing (cut, split, speed, proxy generation), Color Grading (wheels, curves, LUTs), Keying (PIK, Despill, matte monitoring), Roto/Tracking (Bezier masks, point tracking), Compositing (39 blend modes, 57 transitions), and VFX (glow, particles, lens distortion, etc.).
    • Audio (30+ nodes): Dynamic processing (compression, limiter, de-esser), EQ, noise reduction, convolution reverb, stem separation (vocals/drums/bass/etc.), and audio-to-video level automation.
    • Music (Symbolic): MusicXML support, piano roll MIDI editor, SoundFont (SF2/SF3) synthesis, and chord accompaniment.
    • Panorama: 360° viewer and workflows for text/image-to-panorama.
    • 3D: DCC-style workspace (multi-camera, keyframes), 3D model generation, geometry workshop (mesh ops, booleans), and PBR material nodes.
    • Orchestration: Pickers (A/B comparison), Sequence assembly, and a Director's Station (clip-based timeline with transitions and content-addressed caching).

  7. How to use the Video Editor skill

    main

    The video-editor skill is used for conversation-driven video editing on the ComfyTV canvas. It is designed for editorial tasks such as rough cuts, silence removal, pacing adjustments, montages, trims, concatenation, speed changes, color grading, and burning subtitles on existing footage.

    When to use it:

    • When the user explicitly invokes $video-editor.
    • When the user asks to edit footage, cut out dead air/filler, tighten pacing, assemble clips, grade a video, or burn subtitles.

    Key distinction: Do not use this skill to generate new footage. Generation is the responsibility of the Director stage. The Video Editor's job is purely editorial: deciding what to keep, where to cut, and how to pace existing material.

  8. Use the Storyboard Editor for shot blocking and animatics

    main

    The Storyboard Editor is a workbench for building an ordered set of shot boards. Each board is a drawable artboard that reuses the full Layer Editor engine (brushes, shapes, text, masks, transforms) and includes story metadata like dialogue, action, and timing.

    It is primarily used to:

    • Block out sequences shot-by-shot before generation.
    • Convert scripts (Fountain) or reference images into timed boards.
    • Preview pacing via a playable animatic.
    • Export assets as video, GIF, PDF, ZIP (PNGs), or layered PSD.

    Note on Integration: The editor outputs ComfyTV snapshots, not native tensors. To feed these into native ComfyUI nodes, you must insert a Bridge node.

    Workflow:
1. Add Storyboard Editor node.
2. Populate boards (import script, images, or start blank).
3. Draw/annotate using Layer Editor tools and fill metadata in the Board tab.
4. Refine using guides and onion-skinning.
5. Play animatic to review.
6. Export animatic to video or download files.
7. Wire outputs (images/video) downstream.

  9. What is ComfyTV and its core concepts

    main

    ComfyTV is a canvas-based media workstation built on top of ComfyUI. It transforms the node-based workflow into a structured pipeline for media production: Generate → Select → Edit → Compose → Export.

    Key Concepts

    • Per-node Run: Each stage (node) runs independently and does not enter the global ComfyUI queue. Downstream stages consume the most recent snapshot of the upstream output. Re-running a node does not force the entire chain to re-run.
    • Project-based Workflow: Stages belong to a project. Every output includes full history, allowing automatic recovery after page refreshes or restarts.
    • Local Models & Workflows: Uses your local models. You can import any ComfyUI workflow as JSON and bind inputs via the sidebar GUI.
    • Ecosystem Integration: Supports subgraphs and third-party plugins. Use Bridge nodes to connect any plugin into the ComfyTV pipeline. Remote ComfyUI machines can be registered as additional Runners in the Servers tab.
    • Built-in Asset Management: Includes project-level Asset Libraries (images/video/audio/3D), Resource Libraries (LUTs, fonts, SoundFonts), and reusable Prompt Snippets. Use the @ symbol in any prompt to reference these assets.
    • Eagle Integration: Connects to the Eagle app to browse/search libraries, drag assets onto the canvas, and automatically archive renders with full generation provenance (workflow/prompt/parameters).
    • Rich Node Editors: Many stages feature embedded editors like layer editors, storyboards, piano rolls, 3D viewports, and oscilloscopes.

  10. Verify Character Identity and NPC Diversity

    main

    To maintain visual consistency, compare characters against a single canonical face source. For secondary views, use them only for angle and rear construction.

    Character Identity Checklist:

    • Face: age impression, shape, jaw, nose, ears, brow, eye shape/color.
    • Hair: color, hairline, part, bangs, silhouette, strand grouping, rear shape.
    • Body: height, head-to-body ratio, shoulders, limb proportions, hand scale.
    • Costume: layers, collar, closures, seams, hardware, hem, gloves, footwear.
    • Details: accessories, scars, makeup, dirt, wetness, damage, and carried props.

    NPC Diversity Matrix: When generating crowds, avoid repeating templates by varying:

    • Age band, face shape, and facial hair.
    • Hairstyle and color.
    • Body height, width, posture, and mobility.
    • Clothing silhouette, dominant color, layers, and occupation cues.
    • Accessories (eyeglasses, bags, devices).
    • Action and attention target.

    Note: Do not duplicate the protagonist's face, hair, costume, or color signature among NPCs.

  11. Convert COMFYTV_IMAGE to native ComfyUI IMAGE

    main

    ComfyTV uses specific data types (COMFYTV_PANORAMA, COMFYTV_IMAGE) which are references to stored results (URLs). These are not the same as native ComfyUI IMAGE or VIDEO data (which reside in memory).

    To use a screenshot from a Panorama · Current View node with native ComfyUI nodes like IPAdapter or ControlNet, you must use a Bridge node to convert the type.

    • To use in native nodes: Use the ← ComfyTV Image bridge node.
    • To bring native data into ComfyTV: Use the → ComfyTV Image bridge node.

  12. Handle media references and 1-based indexing

    main

    ComfyTV uses 1-based positions for media mentions within a stage's prompt.

    Media Lists

    Every stage maintains an ordered media list per type, consisting of wired inputs and asset_refs in the order reported by get_stage under media.

    Prompt Tokens

    Use tokens like @image_1 or @video_1 to address the media list by position.

    • @image_1 refers to the first image in the list.
    • Automatic Renumbering: If you remove or reorder media, ComfyTV automatically renumbers the tokens in the stage's prompt.
    • Safety: Out-of-range tokens expand to nothing, and the tools will issue a warning.