WebVideoCreator (WVC)

repository·master·Indexed 42 days ago

https://github.com/vinlic/webvideocreator

A Node.js framework for rendering web animations into high-quality videos using Puppeteer and FFmpeg. It supports deterministic frame-by-frame capture of CSS, SVG, Lottie, and JavaScript-driven animations from libraries like GSAP, Three.js, and D3.js. Version 0.0.34 provides capabilities for single-scene and multi-scene rendering with transitions, audio integration, transparent background exports in WebM format, and distributed rendering primitives via ChunkVideo and MultiVideo.

Tokens
25.4K
Snippets
69
Records
96
Agent score
49%

What's inside WebVideoCreator

  1. What is WebVideoCreator?

    master

    WebVideoCreator (WVC) is a Node.js framework that renders web animations into videos using Puppeteer, Chrome, and FFmpeg. It provides a deterministic rendering environment, ensuring perfect frame-by-frame capture of HTML5 animations (CSS3, SVG, Lottie, GIF, APNG, WEBP) and timeline-based animations driven by requestAnimationFrame (RAF).

    Key capabilities include:

    • Deterministic Rendering: Uses Chrome's headless experimental API to control timing and prevent frame skipping.
    • High Performance: Supports GPU acceleration and parallel rendering.
    • Advanced Compositing: Supports single-scene and multi-scene videos with transition effects, chunked rendering for long videos, and transparent WebM exports.
    • Automation: Designed for backend integration to automate video production from web templates.
  2. Access the CaptureContext via window.captureCtx

    master
    The CaptureContext is the central interface for controlling and monitoring the video capture process. WebVideoCreator (WVC) automatically exposes this instance to the global window.captureCtx object, allowing your page's scripts to access capture parameters, trigger actions, or monitor progress.
  3. Understand WebVideoCreator limitations

    master

    Before deploying, be aware of these constraints:

    • Secure Contexts: Due to browser restrictions, WVC can only access localhost / 127.0.0.1 or domains using HTTPS with valid certificates. You can bypass URL restrictions by using the content option to set page content directly.
    • MacOS Stability: The headless experimental API on Mac may crash; use compatibleRenderingMode if necessary, though it is less efficient.
    • Module System: WebVideoCreator is a pure ESM package. You cannot use require() to import it in CommonJS environments without specific workarounds.
    • Page Navigation: Page jump requests during rendering are intercepted to prevent the loss of capture context.
  4. Avoid Page Navigation During Rendering

    master
    Requests that trigger page navigation (redirects/jumps) will be intercepted during the rendering process. This is because navigating to a new page causes the capture context to be lost, which would break the video generation.
  5. Distributed Rendering with MultiVideo and ChunkVideo

    master

    For high-performance requirements, WebVideoCreator supports a distributed rendering strategy using MultiVideo and ChunkVideo.

    Concept:

    1. Divide: Split a long animation into segments (e.g., 0-10s, 10-20s, etc.).
    2. Distribute: Send the parameters for each segment to different devices running WVC.
    3. Render: On each device, create a ChunkVideo instance to render the segment into a .ts video chunk in parallel.
    4. Merge: Return the chunks to a core node, which uses MultiVideo to merge the segments, apply transitions, and synthesize audio tracks for the final output.

    Note: The automated distribution and return workflow is not yet built into WVC, but the primitives (MultiVideo and ChunkVideo) are available for you to implement your own orchestration layer.

  6. Understanding Deterministic vs. Non-deterministic Rendering Environments

    master

    When creating videos via web technologies, standard browser rendering is non-deterministic. Browsers execute animations in 'real-time', meaning high system load can cause dropped frames, and tasks like video decoding are offloaded to other threads, causing timing desynchronization (e.g., <video> elements). This makes standard browser rendering unreliable for video production where every frame must be consistent.

    web-video-creator provides a deterministic rendering environment. In this mode, the code executes as usual, but the flow of time is controlled by the renderer. The following time-sensitive APIs are synchronized to the current rendering progress rather than the system clock:

    • requestAnimationFrame (the currentTime value returned)
    • setTimeout / setInterval callbacks
    • Date
    • performance.now()

    To handle media that is typically outside developer control (like dynamic images and embedded videos), the project uses the experimental WebCodecs API to take control of the frame timing.

  7. Supported Animation Libraries

    master

    WVC is compatible with most web animation and graphics libraries. Verified compatible libraries include:

    • Anime.js
    • GSAP
    • D3.js
    • Three.js
    • Echart
    • Lottie-Web
    • PixiJS
    • Animate.css
    • Mo.js
    • Tween.js

    Important Note for RAF users: If you manually use requestAnimationFrame to drive animations, you must use the timestamp parameter provided by the callback to set the animation's progress. Failing to do so may cause frame rate asynchrony.

  8. Insert video into rendered HTML

    master

    Include a <video> element in your HTML to embed video content.

    Requirements:

    • Supported formats: mp4 and webm.
    • If the src does not end in .mp4, ensure you use the capture attribute or a <canvas video-capture> element to help WVC identify it.
    • For front-end frameworks that hide/show elements, use <canvas src="test.mp4" video-capture></canvas> to ensure WVC notices changes.

    Attributes:

    • src: Path to video.
    • volume: Volume level (e.g., 0.7).
    • startTime / endTime: Playback window in ms.
    • seekStart / seekEnd: Loop segment in ms.
    • fadeInDuration / fadeOutDuration: Video fades in ms.
    • loop / muted: Standard video attributes.
    <!-- Standard video usage -->
    <video src="test.mp4" volume="0.7" startTime="3000" endTime="10000" loop></video
    
    <!-- For framework-based animations or non-standard elements -->
    <canvas src="test.mp4" video-capture></canvas>
  9. Insert audio into videos

    master

    You can add audio to your video using three different methods:

    1. HTML <audio> element: Add an <audio> tag directly to your HTML. WVC automatically detects it and merges the track. You can use attributes like loop, volume, startTime, endTime, seekStart, seekEnd, fadeInDuration, and fadeOutDuration.
    2. Capture Context API: Use captureCtx.addAudio() or captureCtx.addAudios() within your page code.
    3. WVC High-Level API: Use video.addAudio() or video.addAudios() on a SingleVideo, MultiVideo, or ChunkVideo instance.

    Supported HTML attributes for <audio>:

    • volume: Set volume level (e.g., 0.5).
    • startTime / endTime: Control playback timing.
    • seekStart / seekEnd: Define a specific segment to play.
    • fadeInDuration / fadeOutDuration: Control audio transitions.
    • loop: Enable looping.
    <!-- HTML Method -->
    <audio src="bgm.mp3" volume="0.5" startTime="3000" endTime="10000" loop></audio>
    // WVC API Method
    video.addAudio({
        path: "bgm.mp3",
        startTime: 500,
        loop: true,
        volume: 80
    });
  10. Control rendering start and timing

    master

    Delayed Rendering Start

    By default, WVC starts rendering immediately after page navigation. To perform tasks before rendering, set autostartRender: false. You must then manually call captureCtx.start() within your page code to prevent the process from blocking indefinitely.

    Start at a specific time

    To begin capturing from a specific timestamp instead of 0, use the startTime option (in milliseconds).

    Pre-render Page Operations

    Use pagePrepareFn to execute Puppeteer operations (like clicking buttons) before the capture begins.

    // 1. Disable autostart
    const video = wvc.createSingleVideo({
        url: "http://localhost:8080/test.html",
        autostartRender: false
    });
    
    // 2. In your HTML/Page code, trigger start manually
    // <script>captureCtx.start();</script>
    
    // 3. Or use pagePrepareFn for Puppeteer actions
    const videoWithPrep = wvc.createSingleVideo({
        url: "http://localhost:8080/test.html",
        pagePrepareFn: async page => {
            const _page = page.target;
            await _page.tap("#play-button");
        }
    });
  11. Render multi-scene videos using ChunkVideo and MultiVideo

    master

    You can create complex videos by composing multiple individual scenes (chunks). Each chunk is rendered from a URL or raw HTML content. You can set transitions (like fades) between these chunks and then combine them into a single multi-scene video.

    1. Create Chunks: Use createChunkVideo to define individual scenes.
    2. Wait for Rendering: Use chunk.startAndWait() to ensure all chunks are rendered before proceeding.
    3. Set Transitions: Use chunk.setTransition() to define effects between chunks.
    4. Compose Multi-Video: Use createMultiVideo with an array of chunks to merge them into a final file.
    5. Start Synthesis: Call .start() on the multi-video instance and listen for the completed event.
    import WebVideoCreator, { VIDEO_ENCODER, TRANSITION, logger } from "web-video-creator";
    
    const wvc = new WebVideoCreator();
    
    wvc.config({
        mp4Encoder: VIDEO_ENCODER.NVIDIA.H264
    });
    
    // 1. Create chunks
    const chunk1 = wvc.createChunkVideo({
        url: "http://localhost:8080/scene-1.html",
        width: 1280,
        height: 720,
        fps: 30,
        duration: 10000,
        showProgress: true
    });
    
    const chunk2 = wvc.createChunkVideo({
        url: "http://localhost:8080/scene-2.html",
        width: 1280,
        height: 720,
        fps: 30,
        duration: 10000,
        showProgress: true
    });
    
    // 2. Wait for chunks to render
    await Promise.all([chunk1.startAndWait(), chunk2.startAndWait()]);
    
    // 3. Set transitions
    chunk1.setTransition({ id: TRANSITION.FADE, duration: 500 });
    
    // 4. Create multi-video
    const video = wvc.createMultiVideo({
        width: 1280,
        height: 720,
        fps: 30,
        chunks: [chunk1, chunk2],
        outputPath: "./test.mp4",
        showProgress: true
    });
    
    // 5. Listen and start
    video.once("completed", result => {
        logger.success(`Render Completed!!!\nvideo duration: ${Math.floor(result.duration / 1000)}s\ntakes: ${Math.floor(result.takes / 1000)}s\nRTF: ${result.rtf}`)
    });
    
    video.start();
  12. Optimize Video Rendering Performance

    master

    Rendering performance depends on animation and media complexity. Use the following strategies to reduce rendering time:

    Animation Strategies

    • Segmented Rendering: For long animations, split them into multiple segments. You can pass a seek parameter to your page URL to start playback at a specific timestamp, then render and compose them as a multi-scene video.

    Hardware and Configuration

    • Parallelism: To maximize resource usage, set the number of parallel video chunk renders to match your CPU thread count (ensure sufficient RAM).
    • GPU Acceleration: Use GPU-accelerated rendering and composition. If your GPU is not being utilized, check your configuration.
    • Hardware Encoders: By default, WVC uses software encoders (libx264 for mp4, libvpx for webm). For better performance, configure the library to use hardware encoders supported by your integrated or discrete graphics card (e.g., NVENC).
    • Storage: Use an SSD to improve disk cache write performance during parallel rendering.
    • Network: Minimize network latency by hosting static files on the same server or accessing them via a local network.

    Output Settings

    • Resolution and Framerate: Reducing the output video resolution and frame rate is the most effective way to decrease rendering time.