TVerRec

repository·master·Indexed 42 days ago

https://github.com/dongaba/tverrec

A batch downloader for the TVer streaming service that automates the saving of ad-free video files based on genres, performers, or titles. It supports parallel downloading, metadata embedding, and scheduling. Compatible with Windows, Mac, and Linux via PowerShell Core, and can be deployed using Docker or Google Colab.

Tokens
3.8K
Snippets
4
Records
15
Agent score
54%

What's inside TVerRec

  1. Overview of TVerRec

    master

    TVerRec is a downloader and download support tool designed to batch download and save programs from the TVer video streaming service. It allows users to specify genres, performers, or program names to automate the downloading process.

    Key features include:

    • Batch Downloading: Download multiple programs based on specific criteria.
    • Ad-free Content: Downloads do not include commercials, allowing you to keep clean versions of your favorite shows.
    • Automation: Once started, it can automatically download new programs as they are released.

    Important Compatibility Note:

    • TVerRec does not support Windows PowerShell. You must use PowerShell Core.
    • It runs on Windows, Mac, and Linux where PowerShell Core is installed.
    • On Windows, if PowerShell Core is missing, TVerRec will attempt to install it automatically.
  2. Key features of TVerRec

    master

    TVerRec provides the following capabilities for downloading TVer content:

    • Bulk Downloading: Download programs in bulk using keywords such as genre, talent, or program name.
    • My Page Integration: Bulk download programs and special features registered to your TVer My Page.
    • Full Recording: Supports full recording of TVer content.
    • Metadata Embedding: Automatically embeds program thumbnails and subtitle information (if available) into the downloaded files.
    • High-Speed Downloads: Supports parallel downloading for high throughput.
    • Individual Downloads: Ability to specify and download individual programs.
    • Integrity Verification: Verifies that downloaded programs are not corrupted.
    • Automatic Organization: Automatically organizes downloaded files into final destinations.
    • Self-Updating Components: Automatically downloads the latest versions of youtube-dl, ffmpeg, and other required components.
    • Notifications: Uses toast notifications to report status and notifies you when a stable version update is available.
    • Scheduling: Allows specifying download execution availability based on day of the week and time of day.
    • Proxy Support: Supports specifying a Proxy server to bypass Geo IP checks (Proxy is used only for Geo IP bypass).
  3. Main features of TVerRec

    master

    TVerRec is a bulk download and save tool for TVer programs. Key capabilities include:

    • Bulk Downloading: Download by keywords (genre, talent, program name) or from TVer's 'My Page' (registered programs, specials, etc.).
    • High Performance: Supports ultra-high-speed parallel downloading.
    • Media Enrichment: Automatically embeds program thumbnails and available subtitles into downloaded files.
    • Automation: Automatically organizes files into media libraries, verifies file integrity, and handles component updates (ffmpeg, youtube-dl).
    • Flexible Scheduling: Specify downloading availability by day of the week and time.
    • Platform Support: Provides a Windows GUI version and CUI (Command Line) versions for Windows, Linux, and Mac.
    • Geo-IP Bypass: Supports proxy server specification for Geo-IP checks.

    For detailed information on specific features, refer to the TVerRec Wiki.

  4. Understand the TVerRec directory structure

    master

    TVerRec uses a structured directory system to separate configuration, databases, logs, and execution scripts for different platforms (Windows vs. Unix). Understanding this layout is essential for managing settings and locating logs or binaries.

    Key Directories

    • conf/: Contains configuration files like ignore.conf (exclusion rules) and keyword.conf (download targets).
    • db/: Stores history.csv (download history) and list.csv (download lists).
    • log/: Contains error logs for ffmpeg and youtube-dl.
    • src/: Contains the core PowerShell (.ps1) source scripts for Windows.
    • unix/: Contains shell scripts (.sh) for Linux/Mac users.
    • win/: Contains command files (.cmd) for Windows users.
    • resources/: Contains assets, language files (message.json), and Docker/Colab samples.
  5. Run TVerRec using Docker

    master

    TVerRec can be run as a containerized application using Docker. The image is based on Ubuntu Linux and comes pre-configured with all necessary tools (PowerShell Core, youtube-dl, ffmpeg, etc.).

    To use the Docker version:

    1. Pull the image from Docker Hub.
    2. Prepare and modify your configuration file.
    3. Set up the necessary disk mount bindings to ensure downloaded content is persisted outside the container.
  6. Prerequisites for running TVerRec

    master

    TVerRec supports Windows, Mac, and Linux. Depending on your OS, different software requirements apply:

    Required Software

    • PowerShell Core: Automatically installed on Windows.
    • youtube-dl: Automatically downloaded.
    • ffmpeg: Automatically downloaded.
    • Python: Required for Linux/Mac only. Not required on Windows.
    • PyCryptodome: Recommended for Linux/Mac environments to speed up download processes.

    Docker Option

    Alternatively, you can run TVerRec as a container. The image is based on Ubuntu Linux and comes with all necessary tools pre-configured. To use this method, you must:

    1. Prepare and modify the configuration files.
    2. Set up disk mounting/binding for your files.
  7. Configure download targets and exclusions

    master

    TVerRec manages what to download and what to ignore via configuration files located in the conf/ directory. If these files do not exist, they are automatically created.

    • conf/keyword.conf: Define the keywords used to identify programs for bulk downloading.
    • conf/ignore.conf: Define settings to exclude specific programs from being downloaded.
    • conf/user_setting.ps1: A file you can create to define your own custom user settings.

    Sample files for these configurations can be found in resources/sample/ (e.g., keyword.sample.conf, ignore.sample.conf).

  8. Set up TVerRec in Google Colab

    master

    To run TVerRec in a Google Colab environment, you must perform a multi-step setup: install system dependencies (including PowerShell), clone the repository, configure paths for the Colab filesystem, and mount Google Drive for persistent storage.

    Follow these steps in order:

    1. Install System Dependencies: Update apt and install wget, git, and powershell via the Microsoft repository.
    2. Clone Repository: Clone https://github.com/dongaba/TVerRec and ensure shell scripts are executable.
    3. Configure Colab Paths: Use sed to fix relative paths in shell scripts and append Colab-specific directory settings to conf/user_setting.ps1.
    4. Mount Drive: Mount Google Drive to allow saving downloads to your Drive storage.
    5. Execute: Run the startup script.
    # 1. Install dependencies and PowerShell
    !sudo apt-get update
    !sudo apt-get install -y wget apt-transport-https software-properties-common git
    !wget -q "https://packages.microsoft.com/config/ubuntu/$(lsb_release -rs)/packages-microsoft-prod.deb"
    !sudo dpkg -i packages-microsoft-prod.deb
    !rm -rf packages-microsoft-prod.deb*
    !sudo apt-get update
    !sudo apt-get install -y powershell
    
    # 2. Clone and prepare repository
    !if [ ! -d /content/TVerRec ] ; then git clone https://github.com/dongaba/TVerRec ; else cd /content/TVerRec && git pull && cd - ; fi
    !chmod a+x /content/TVerRec/unix/*.sh
    
    # 3. Configure for Colab environment
    !sed -i -e "s|\.\./src|/content/TVerRec/src|g" /content/TVerRec/unix/*.sh
    !echo '$script:downloadBaseDir = "/content/drive/MyDrive/Work"' >> /content/TVerRec/conf/user_setting.ps1
    !echo '$script:downloadWorkDir = "/tmp"' >> /content/TVerRec/conf/user_setting.ps1
    !echo '$script:saveBaseDir = "/content/drive/MyDrive/Save"' >> /content/TVerRec/conf/user_setting.ps1
    
    # 4. Mount Google Drive
    from google.colab import drive
    drive.mount('/content/drive')
    
    # 5. Start TVerRec
    !/content/TVerRec/unix/start_tverrec.sh
  9. Deploy TVerRec using Docker Compose

    master

    You can run TVerRec in a containerized environment using Docker Compose. The configuration uses the dongaba/tverrec:latest image and sets up several persistent volumes for work, temporary files, and saved content.

    Key container settings:

    • Restart Policy: unless-stopped ensures the container restarts automatically unless manually stopped.
    • Interactive Mode: tty: true and stdin_open: true are enabled to allow for interactive terminal usage if needed.
    • Telemetry: The environment variable POWERSHELL_TELEMETRY_OPTOUT: 1 is used to opt out of PowerShell telemetry.
    version: "3.8"
    
    services:
      tverrec:
        container_name: TVerRec-loop
        image: dongaba/tverrec:latest
        hostname: "tverrec"
        restart: unless-stopped
        tty: true
        stdin_open: true
        environment:
          POWERSHELL_TELEMETRY_OPTOUT: 1
        tmpfs:
          - /mnt/Temp
        volumes:
          - temp:/mnt/Temp
          - work:/mnt/Work
          - save:/mnt/Save
          - container-data:/app/TVerRec/container-data
    
    volumes:
      temp:
      work:
      save:
      container-data:
  10. Configure TVerRec directory settings for Colab

    master

    When using TVerRec in Colab, you must override the default directory settings in conf/user_setting.ps1 to ensure files are saved to your mounted Google Drive rather than the ephemeral Colab local storage.

    Append the following variables to /content/TVerRec/conf/user_setting.ps1:

    • $script:downloadBaseDir: The base directory for downloads (e.g., "/content/drive/MyDrive/Work").
    • $script:downloadWorkDir: The temporary working directory (e.g., "/tmp").
    • $script:saveBaseDir: The base directory where files are permanently saved (e.g., "/content/drive/MyDrive/Save").
    !echo '$script:downloadBaseDir = "/content/drive/MyDrive/Work"' >> /content/TVerRec/conf/user_setting.ps1
    !echo '$script:downloadWorkDir = "/tmp"' >> /content/TVerRec/conf/user_setting.ps1
    !echo '$script:saveBaseDir = "/content/drive/MyDrive/Save"' >> /content/TVerRec/conf/user_setting.ps1
  11. Locate logs for troubleshooting

    master

    Logs are stored in the log/ directory. Note that some logs are temporary and may be automatically deleted after a certain period of time.

    • ffmpeg_err_*.log: Errors related to ffmpeg processing.
    • ytdl_err_*.log: Execution logs for youtube-dl.
    • ytdl_out_*.log: Error logs for youtube-dl.