YTPTube

YTPTube is a web-based GUI for yt-dlp that facilitates downloading videos, playlists, channels, and live streams.

Key Capabilities:

• Automation: Schedule downloads for channels/playlists and create custom download feeds from non-supported sites.
• Notifications: Integration with Apprise to send notifications based on events.
• Advanced yt-dlp Support: Powerful presets system, support for curl-cffi (Docker only), and bundled pot provider plugin (Docker only).
• Bypassing Protections: Support for FlareSolverr or Trawl to bypass Cloudflare.
• UI Modes: Dual view mode (Technical vs. Simple) and multi-language support (English, العربية, Français, 中文, 日本語).
• Media Playback: Built-in video player with sidecar external subtitle support (requires ffmpeg in PATH for non-Docker setups).

YTPTube can automatically scrape pages and enqueue new links using a generic task handler. This works by turning JSON definitions into site-specific scrapers.

Workflow

1. Create a Definition: In the WebUI, go to tasks > Definitions and create a JSON definition.
2. Create a Task: Create a task referencing the target URL. Ensure the task uses a preset with --download-archive enabled to avoid duplicates.
3. Execution: The handler scans definitions, matches the task URL against match_url (supports Glob and Regex), fetches the page, and extracts items.

Definition Schema

Each definition is a JSON object. Key fields include:

• name: Identifier for logs.
• match_url: List of Glob or Regex strings to match task URLs.
• engine: The fetch engine (httpx default, or selenium for remote Chrome sessions).
• request: HTTP settings (method, url, headers, params, data, json_data, timeout).
• response: The response format (html default, or json).
• parse: Extraction logic.
  • items: Container for per-item extraction. Requires a selector (CSS or XPath) and fields.
  • fields: Mapping of keys to extraction rules. Each field requires a type (css, xpath, or jsonpath), an expression, and an attribute (e.g., text, html, href).

Parsing Rules

• Link Requirement: Every definition must provide a link field (either at the top level or inside parse.items.fields).
• Attributes:
  • text / inner_text: Applies normalize-space().
  • html / outer_html: Returns raw HTML fragment.
  • For link fields, if attribute is omitted, it defaults to href.
• JSON Responses: If response.type is json, set type and field type to jsonpath and use JMESPath expressions.

Fetch Engines

• httpx: Default. Supports custom headers, params, JSON payloads, and proxies.
• selenium: Requires a remote Chrome session. Provide the hub URL in engine.options.url. Supports arguments, wait_for (CSS/XPath), wait_timeout, and page_load_timeout.

{
  "name": "example",
  "match_url": ["https://example.com/articles/*"],
  "engine": { "type": "httpx" },
  "request": { "method": "GET", "url": "https://example.com/articles/latest" },
  "response": { "type": "html" },
  "parse": {
    "items": {
      "selector": ".columns .card",
      "fields": {
        "link": { "type": "css", "expression": ".card-header a", "attribute": "href" },
        "title": { "type": "css", "expression": ".card-header a", "attribute": "text" }
      }
    }
  }
}

The WebSocket API provides real-time bidirectional communication for download queue management and status updates.

Connection Details:

• Endpoint: /ws (or {base_path}/ws if a base path is configured).
• Protocol: ws:// or wss://.
• Heartbeat: 10-second interval.
• Auto-reconnect: Clients are responsible for implementing reconnection logic.

Authentication: If YTP_AUTH_USERNAME and YTP_AUTH_PASSWORD are set, authenticate via:

1. Query Parameter (Recommended): ws://localhost:8080/ws?apikey=<base64_urlsafe_credentials>
2. HTTP Basic Auth: Include Authorization: Basic base64("<username>:<password>") during the handshake.

const ws = new WebSocket('ws://localhost:8080/ws');
ws.onopen = () => console.log('Connected');
ws.onmessage = (event) => {
  const message = JSON.parse(event.data);
  console.log('Event:', message.event, 'Data:', message.data);
};