DOCSEARCHbyAlgolia
Submit Docs

Waveform Playlist

repository·main·Indexed 23 days ago

https://github.com/naomiaro/waveform-playlist

A multi-track web audio editor and player built with React, Tone.js, and the Web Audio API. It features canvas-based waveform visualization, drag-and-drop clip editing, professional audio effects, and support for time-synchronized text annotations. The library provides multiple playback paths, including a full multitrack engine and a lightweight MediaElement-based player for single-track use cases.

Tokens
213.4K
Snippets
414
Records
890
Agent score
82%

What's inside waveform-playlist

  1. Overview of @waveform-playlist/ui-components features

    main

    The @waveform-playlist/ui-components package provides React components for building custom playlist interfaces. Key features include:

    • Canvas-based rendering: Channel (waveform), SpectrogramChannel, and PianoRollChannel use canvas for efficient rendering.
    • Track Controls: Includes mute, solo, volume, and track menus.
    • Performance: Uses virtual scrolling with chunked canvases and viewport tracking for long timelines.
    • Metering: SegmentedVUMeter for standalone level visualization.
    • Error Handling: PlaylistErrorBoundary to catch render errors independently of the theme provider.

  2. Understand the Waveform-playlist architecture and technology stack

    main

    Waveform-playlist is a multitrack Web Audio editor and player featuring canvas-based waveform visualizations. It is organized as a monorepo using pnpm workspaces and is split into two primary technology stacks depending on the package:

    1. React Stack (@waveform-playlist/* packages): Uses React, Tone.js, and styled-components. This is the primary stack for high-level UI and React integration.
    2. Web Components Stack (@dawcore/* packages): Uses Lit Web Components and native Web Audio. These packages are framework-agnostic, making them suitable for use in any environment (Vanilla JS, Vue, etc.) as a Digital Audio Workstation (DAW) UI.

    Developers should choose between these stacks based on whether they need a React-integrated experience or a framework-agnostic, low-level DAW component set.

  3. Understand the UI component architecture

    main

    The UI layer is built with React and styled-components, organized into several functional areas:

    Core Layout Components

    • Playlist: The main container component.
    • Track: Represents an individual waveform track.
    • Clip: An audio clip, which can include a draggable ClipHeader and ClipBoundary (trim handles).

    Rendering Channels

    • Channel / SmartChannel: Handles waveform rendering with device pixel ratio support.
    • PianoRollChannel: Renders MIDI piano roll canvases using a chunked approach.
    • SpectrogramChannel: Renders spectrogram canvases using a chunked approach.

    UI Overlays and Indicators

    • Playhead: Indicates the current playback position.
    • Selection: An overlay for selected regions.
    • LoopRegion: An overlay for loop regions.
    • FadeOverlay: Visualizes fade in/out effects.
    • SegmentedVUMeter: An LED-style VU meter supporting multi-channel, configurable dB ranges, and peak hold.

    Controls and Inputs

    • TimeScale / SmartScale: The timeline ruler. SmartScale can switch between beats/bars or time scales.
    • TimeInput / SelectionTimeInputs: Inputs for entering specific time values.
    • TrackControls: A suite of controls including Mute, Solo, Volume, and Pan.

  4. Choose your integration path for Waveform Playlist

    main

    Waveform Playlist offers two distinct integration paths depending on your project requirements:

    1. React Integration: Best for React 18+ or 19+ projects. It uses a hooks-and-components pattern, providing access to real-time parameter control for effects via React state.
    2. Web Components Integration: Best for vanilla HTML or projects using other frameworks. It uses framework-agnostic custom elements (built with Lit) and does not require React.

  5. Configure multi-channel metering

    main

    The channelCount option determines how many channels are metered.

    • Mono Input: Default is 1. If you use channelCount: 2 with a mono microphone, the single channel level is mirrored to both L/R channels.
    • Stereo Output: Default is 2.
    • Multi-channel Interfaces: Set channelCount to the number of channels available (e.g., 4).

    When channelCount is 1, the scalar level and peakLevel return values are identical to levels[0] and peakLevels[0]. When channelCount is greater than 1, the scalar values represent the maximum across all channels.

    // 4-channel input metering
const { levels, peakLevels } = useMicrophoneLevel(stream, { channelCount: 4 });

// SegmentedVUMeter auto-labels channels as 1, 2, 3, 4
<SegmentedVUMeter
  levels={levels}
  peakLevels={peakLevels}
  channelLabels={['Front L', 'Front R', 'Rear L', 'Rear R']}
/>

  6. How to handle track swaps and lifecycle events

    main

    When building a single-track player (like a podcast player), you can use setSource() to swap tracks in place. This method preserves any existing Web Audio routing or effects and ensures that on() event listeners registered on the MediaElementPlayout instance are retained across the swap.

    Example of observing lifecycle events:

    playout.on('loadedmetadata', () => console.log('duration:', playout.duration));
playout.on('play', () => updateTransportUI('playing'));
playout.on('pause', () => updateTransportUI('paused'));
playout.on('error', (err) => surfaceError(err));

// To swap tracks without losing listeners:
playout.setSource({ source: '/audio/episode-2.mp3', name: 'Episode 2' });

  7. How virtual scrolling and chunked rendering work

    main

    To maintain performance with large playlists, the UI uses a virtual scrolling system and chunked rendering for heavy components like Channel, SpectrogramChannel, and PianoRollChannel.

    Viewport Management

    • ScrollViewportProvider: Wraps the scrollable container and tracks scroll position using useSyncExternalStore with a RAF-throttled listener and ResizeObserver.
    • useScrollViewport(): Returns the full viewport state.
    • useScrollViewportSelector(): Provides fine-grained subscriptions to viewport changes.
    • useVisibleChunkIndices(totalWidth, chunkWidth, originX?): Returns a memoized array of visible chunk indices. The originX parameter is critical for converting local chunk coordinates to global viewport space.

    Correcting Clip Culling

    Because components like Channel use absolute positioning (left: chunkIndex * 1000px), clips that do not start at position 0 require coordinate correction.

    • ClipViewportOriginProvider: Wraps each clip's channels to supply the clip's pixel left offset. This ensures that chunk culling is calculated correctly relative to the global viewport.

  8. Understand the Waveform Data Service architecture

    main

    The service is designed to offload audio decoding and peak computation from the browser to the edge using Cloudflare Workers and WASM.

    Workflow:

    1. API Worker: Accepts uploads/URLs and dispatches processing.
    2. Processing Worker: Uses a WebAssembly module (FFmpeg or lightweight decoder) to decode audio and compute peaks (supporting formats like FLAC/OGG).
    3. R2 Storage: Stores computed peak data as JSON/binary, content-addressed by the hash of the audio file and resolution.
    4. CDN: Serves peak data from the edge for indefinite caching.

  9. Use Annotations with dual-view synchronization

    main

    Annotations in Waveform Playlist can be managed through two synchronized views: a timeline track view and a text list view.

    1. Timeline View (<daw-annotation-track>): Contains <daw-annotation> elements. Users can drag these boxes to edit timing.

    • Attributes: editable, link-endpoints, continuous-play, keyboard-controls, box-label, name.

    2. Text List View (<daw-annotation-list>): A scrollable panel that links to a track via the for attribute. Edits made here (text or time) automatically update the timeline view.

    • Attribute: for (matches the id of the <daw-annotation-track>).
    • Attribute: time-display ('time' default | 'bars'). Setting this to 'bars' shows bar.beat ranges instead of clock time.

    Tick-based vs Seconds-based:

    • An annotation is tick-based if both start-tick and end-tick are set. This is the preferred method for musical accuracy.
    • If only start and end (seconds) are set, the annotation is seconds-based. The editor maintains a derived cache of ticks for these.
    • Warning: Setting only one of the two tick attributes results in a half-configured state that defaults to seconds-based and logs a warning.
    <!-- Annotations: single source of truth, dual view -->
<daw-editor id="my-editor">
  <daw-annotation-track id="lyrics" editable link-endpoints>
    <daw-annotation start="0.0" end="2.5">First line of lyrics</daw-annotation>
    <daw-annotation start="2.5" end="5.1">Second line of lyrics</daw-annotation>
    <daw-annotation start="5.1" end="8.0">Third line of lyrics</daw-annotation>
  </daw-annotation-track>
  <daw-track src="/audio/vocals.mp3" name="Vocals"></daw-track>
</daw-editor>

<!-- Text list view linked to the same data via for/id -->
<daw-annotation-list for="lyrics"></daw-annotation-list>

  10. Understand the Waveform Playlist architecture

    main

    Waveform Playlist is built on a shared headless engine (@waveform-playlist/engine) that handles the core logic.

    • React path: Wraps the engine using React Providers and Hooks.
    • Web Components path: Wraps the engine using Lit-based custom elements (via @dawcore/components).
    • Audio Backends: Both paths utilize a pluggable PlayoutAdapter to interface with different audio backends.

  11. Integrate the @waveform-playlist/engine timeline engine

    main

    The @waveform-playlist/engine is a framework-agnostic, stateful timeline engine. It is built using a two-layer architecture: pure operation functions (for logic like dragging, trimming, and zooming) and a stateful PlaylistEngine class that manages state and emits events.

    Key Concepts

    • PlaylistEngine: The central class that manages selection, loop, zoom, master volume, and tracks. It uses an event emitter to notify subscribers of statechange, play, pause, or stop events.
    • PlayoutAdapter: A pluggable audio backend interface. You can implement this to connect the engine to different audio APIs (like Web Audio API). It supports addTrack() for incremental track additions.
    • EngineState: A snapshot of the current engine state, including tracksVersion and mixerVersion.

    Clip Mutations

    Mutations like moveClip(), trimClip(), and splitClip() update the internal tracks, sync the provided adapter via adapter.setTracks(), and emit a statechange event.

    • Performance Optimization: Both moveClip() and trimClip() accept an optional skipAdapter parameter. Use this during high-frequency operations (like dragging) to avoid unnecessary adapter synchronization overhead.

  12. Combine Clip and Annotation dragging

    main

    To allow both clips and annotations to be draggable within a single DragDropProvider, you must manually route drag events in the onDragStart, onDragMove, and onDragEnd handlers.

    Inspect event.operation?.source?.data to determine the source. If data.annotationId is present, route the event to useAnnotationDragHandlers. Otherwise, route it to useClipDragHandlers.

    const onDragStart = (event) => {
  const data = event.operation?.source?.data;
  if (data?.boundary) {
    // Annotation boundary or clip boundary
    if (data.annotationId) {
      annotationDragStart(event);
    } else {
      clipDragStart(event);
    }
  } else {
    clipDragStart(event);
  }
};