VideoDL Documentation

repository·master·Indexed 25 days ago

https://github.com/charlespikachu/videodl

A lightweight Python-based video downloader, distributed as `videofetch` on PyPI. It supports a wide range of Chinese and overseas platforms, including Bilibili, YouTube, and Reddit, as well as various generic video parsers. Designed for academic research, development workflows, and personal use, it provides capabilities for URL parsing and video downloading via specialized clients.

Tokens
10.7K
Snippets
10
Records
30
Agent score
79%

What's inside VideoDL

  1. Introduction to VideoDL

    master

    VideoDL (also referred to as videofetch in PyPI) is a fast, lightweight, and fully Python-based video downloader. It is designed for simplicity, efficiency, and flexibility, making it suitable for:

    • Academic research: Collecting and organizing video data for dataset construction.
    • Development workflows: Integrating into multimedia processing pipelines.
    • Personal projects: Saving online videos for offline access.

    The project features an easy-to-understand Python codebase and a lightweight design, allowing for both direct use and extension into custom applications.

  2. Explore recommended projects by CharlesPikachu

    master
    The videodl repository documentation provides a list of related lightweight downloader and utility projects developed by CharlesPikachu. These include tools for music, video, images, academic papers, proxy collection, and GPT interfaces.
  3. Understand the BaseVideoClient abstraction

    master

    In videodl, BaseVideoClient is the abstract base class for all site-specific video clients. While most users will interact with the high-level VideoClient or a specific concrete subclass (like YouTubeVideoClient or BilibiliVideoClient), understanding BaseVideoClient is useful for knowing how the underlying client logic is structured.

    Concrete subclasses implement the actual parsing logic for specific websites, while BaseVideoClient provides the common interface and shared utilities like request retries and session management.

  4. Inspect and modify VideoInfo objects

    master

    The parsefromurl() method returns a list of VideoInfo objects. These objects behave like both a dataclass and a dictionary, allowing you to access properties via dot notation or key access.

    Common Fields

    • source: The client that produced the result.
    • title: The video title.
    • download_url: The resolved media URL.
    • save_path: The output file path.
    • ext: The file extension.
    • err_msg: Any parsing error message.
    • download_with_ffmpeg: Boolean to indicate if ffmpeg should be used.
    • download_with_aria2c: Boolean to indicate if aria2c should be used.
    • enable_nm3u8dlre: Boolean to indicate if N_m3u8DL-RE should be used.

    Modifying Download Behavior

    You can modify the VideoInfo objects before calling .download() to force specific download methods.

    from videodl import videodl
    
    video_client = videodl.VideoClient()
    video_infos = video_client.parsefromurl("URL")
    
    for info in video_infos:
        # Access via dot notation or dict key
        print(info.title)
        print(info['title'])
        
        # Modify download settings
        info["download_with_aria2c"] = True
        info["enable_nm3u8dlre"] = True
        info["download_with_ffmpeg"] = True
    
    video_client.download(video_infos)
  5. How videodl selects video parsers

    master

    When attempting to parse a video URL, videodl follows a specific fallback logic to ensure maximum coverage:

    1. Primary Parsers: It first attempts to use the parsers explicitly listed in the project's supported list.
    2. Generic Parsers: If the primary parsers fail, videodl then invokes the generic parsers (such as AnyFetcherVideoClient, APICXVideoClient, etc.) one by one until a successful parse is achieved.
  6. Install videodl

    master

    You can install videodl (via the videofetch package) using one of the following three methods:

    1. Via pip: The simplest method using the PyPI package.
    2. Via GitHub (Direct Pip): Install directly from the master branch of the GitHub repository.
    3. Via GitHub (Manual Clone): Clone the repository and run the setup script locally.
    # from pip
    pip install videofetch
    
    # from github repo method-1
    pip install git+https://github.com/CharlesPikachu/videodl.git@master
    
    # from github repo method-2
    git clone https://github.com/CharlesPikachu/videodl.git
    cd videodl
    python setup.py install
  7. Use videodl from the Command Line

    master

    You can use videodl directly from your terminal to download videos.

    Basic Usage

    • Download a video: Provide a URL with the -i flag. If a matching client is found, it will parse and download automatically.
    • Interactive Mode: Run videodl without the -i flag to enter a terminal UI where you can enter URLs, press q to quit, or r to restart the UI.

    Advanced CLI Options

    • Restrict to specific clients: Use -a to specify one or more clients (comma-separated) to speed up parsing.
    • Use generic parsers: Use -g to only apply common/generic video clients.
    • Configure clients: Use -c to pass client-specific configurations (like work_dir) as a JSON string.
    • Override requests: Use -r to pass custom headers or proxies as a JSON string for specific clients.
  8. Use videodl from Python

    master

    Integrate videodl into your Python applications using the VideoClient class. The client can choose suitable parsers, parse URLs into VideoInfo objects, and download the resulting media.

    Basic Workflow

    1. Initialize: Create a videodl.VideoClient() instance.
    2. Parse: Call .parsefromurl(url) to get a list of VideoInfo objects.
    3. Download: Pass the list of VideoInfo objects to .download(video_infos).

    Configuration via Constructor

    • allowed_video_sources: A list of strings specifying which clients to use.
    • apply_common_video_clients_only: Boolean to restrict parsing to generic clients.
    • init_video_clients_cfg: A dictionary for client-specific settings like work_dir or default_parse_cookies.
    • requests_overrides: A dictionary to provide headers or proxies for specific clients.
    • clients_threadings: A dictionary to set the number of threads per client.
  9. Install required CLI dependencies for videodl

    master

    Many video downloaders in videodl require external CLI tools for decryption, stream parsing, and accelerated downloading. For full functionality, it is highly recommended to install the following tools and ensure they are available in your system PATH.

    FFmpeg

    Required for all HLS (HTTP Live Streaming) streams. Verify installation:

    ffmpeg -version

    N_m3u8DL-RE

    Highly recommended for performance and compatibility. Specialized for m3u8 streams, it handles encryption, anti-leech headers, and parallel segment downloading. Without it, many clients (e.g., CCTVVideoClient, FoxNewsVideoClient, TencentVideoClient, RedditVideoClient, IQiyiVideoClient, etc.) may fail to parse or download. Verify installation:

    N_m3u8DL-RE --version

    Bento4

    Required for certain encrypted media workflows. Provides utilities like mp4decrypt used by N_m3u8DL-RE for clients like TBNUKVideoClient and PlayerPLVideoClient. Verify installation:

    mp4decrypt --version

    Node.js

    Required for specific JavaScript-based parsing. Only necessary if using YouTubeVideoClient, CCTVVideoClient, or TencentVideoClient. Verify installation:

    node -v
    npm -v

    aria2c

    Optional for download acceleration. Used to accelerate downloads (e.g., MP4 files) and enable resuming interrupted downloads. Verify installation:

    aria2c --version
  10. Configure a BaseVideoClient instance

    master

    When instantiating a client (or a subclass of BaseVideoClient), you can pass several arguments to control request behavior, security, and output:

    • auto_set_proxies (bool): Automatically fetch and apply proxies. Useful for sites that block direct requests.
    • random_update_ua (bool): Randomly refresh the User-Agent for new sessions to reduce fingerprinting.
    • enable_parse_curl_cffi, enable_search_curl_cffi, enable_download_curl_cffi (bool): Use curl_cffi instead of requests for specific operations. Use this for sites with strict request checks.
    • max_retries (int): Maximum number of HTTP request retries.
    • maintain_session (bool): If True, keeps cookies and session state between requests.
    • work_dir (str): The root directory where outputs will be saved (default: "videodl_outputs").
    • freeproxy_settings (dict): Settings for the proxy client when auto_set_proxies=True.
    • default_search_cookies, default_download_cookies, default_parse_cookies (dict): Default cookies for search, download, or parsing operations (useful for authenticated sessions).
  11. Verify installation of CLI dependencies

    master

    After installing the dependencies, verify they are correctly added to your system PATH by running the following commands in your terminal:

    ToolVerification CommandSuccess Indicator
    FFmpegffmpeg -versionDetailed version information
    N_m3u8DL-REN_m3u8DL-RE --versionVersion string (e.g., 0.5.1+...)
    Bento4mp4decrypt --versionVersion information
    Node.jsnode -v or npm -vNode/npm version (e.g., v22.11.0)
    aria2caria2c --version or aria2c -vVersion information