Zilean

repository·main·Indexed 42 days ago

https://github.com/ipromknight/zilean

A service for searching DebridMediaManager-sourced content that exposes it as a Torznab indexer. Zilean supports scraping content from Zurg and other Zilean instances, provides IMDB metadata integration, and includes a dedicated API for managing blacklists, DMM imports, and torrent streaming.

Tokens
5.6K
Snippets
24
Records
32
Agent score
54%

What's inside Zilean

  1. What is Zilean?

    main

    Zilean is a service designed to search for content sourced from DebridMediaManager that has been shared by users.

    Key features include:

    • Torznab Integration: Can be configured as a Torznab indexer for use in various content applications.
    • DMM Import: Automatically reruns on missing pages based on a configured time interval.
    • Scraping Capabilities: Can scrape content from a running Zurg instance or from other running Zilean instances.
  2. How database migrations are handled in Zilean

    main

    In Zilean, database migrations and index management are automated. You do not need to run migrations manually.

    When the application starts, Zilean automatically:

    1. Runs Migrations: It checks the database for the latest migration and executes any pending migrations that have not yet been applied.
    2. Manages Indexes: It checks for missing database indexes and creates them if they do not exist to ensure the database is optimized for the application.

    While some migrations may take longer to execute than others, the process is entirely handled by the system during the startup lifecycle.

  3. How Torznab Indexers work

    main

    A Torznab indexer acts as a standardized bridge between media management applications (like Sonarr, Radarr, or Prowlarr) and torrent/Usenet trackers. It uses a RESTful API based on the RSS feed format to provide a uniform interface for searching and retrieving metadata.

    Workflow

    1. Setup: The user configures the indexer URL and API key in their media application.
    2. Request: The application sends a GET request to the Torznab API with parameters like search terms (q), categories (cat), and limits.
    3. Response: The indexer returns an XML feed containing metadata such as Title, Link (magnet/torrent URL), Description, Category, and Size.
    GET /api?t=search&q=example_movie&cat=5000
  4. Enable and Configure Ingestion

    main

    To enable ingestion, you must set Ingestion.EnableScraping to true and populate instance lists.

    1. Define Instances

    Populate one or more of the following lists with objects containing Url and EndpointType:

    • Ingestion.ZurgInstances
    • Ingestion.ZileanInstances
    • Ingestion.GenericInstances

    EndpointType Values:

    • 0: Zilean
    • 1: Zurg
    • 2: Generic

    Requirements for Zilean Instances: If scraping from Zilean instances, you must also:

    1. Set an ApiKey in the Zilean section.
    2. Set Zilean.Torrents.EnableEndpoint to true.
    3. Set Zilean.Torrents.EnableScrapeEndpoint to true.

    2. Kubernetes Service Discovery (Optional)

    If running in Kubernetes, configure these keys:

    • Ingestion.Kubernetes.EnableServiceDiscovery: Set to true to find Zurg instances automatically.
    • Ingestion.Kubernetes.KubernetesSelectors: List of selectors for discovery.
    • Ingestion.Kubernetes.KubeConfigFile: Path to kube config (default: /$HOME/.kube/config).
    • Ingestion.Kubernetes.AuthenticationType: 0 = None, 1 = Kubernetes RBAC.
    {
      "Ingestion": {
        "EnableScraping": true,
        "ZurgInstances": [
          {
            "Url": "http://zurg:9000",
            "EndpointType": 1
          }
        ],
        "ScrapeSchedule": "0 0 * * *"
      }
    }
  5. Configure Zilean as a Generic Torznab Indexer in Prowlarr

    main

    To use Zilean as an indexer within Prowlarr, follow these steps:

    1. Open Prowlarr and navigate to Indexers -> Add.
    2. Search for generic and select the type private.
    3. Select Generic Torznab.
    4. Set the Name to Zilean.
    5. Set the Url to http://zilean:8181/torznab.
    6. Set the API box to /api.
    7. Click Sync to Apps to propagate the settings.
    Name: Zilean
    Url: http://zilean:8181/torznab
    API: /api
  6. Access the Zilean API documentation

    main

    By default, the interactive API documentation (Scalar v2) is available at: http://localhost:8181/scalar/v2.

    Note that some endpoints may be disabled via configuration. If an endpoint is disabled, it will not be mapped and will not exist while Zilean is running. Refer to the Configuration guide to enable or disable specific endpoints.

  7. Configure Zilean via JSON or Environment Variables

    main

    Zilean configuration is defined in a JSON file. If running in a container, mount your configuration file as a volume at /app/data/settings.json.

    Load Order

    Settings are loaded in the following order (later values override earlier ones):

    1. /app/data/settings.json
    2. Environment variables (highest precedence)

    Environment Variable Format

    To override a configuration key using environment variables, use the following pattern: Zilean__{Section}__{Key}

    Examples:

    • To set ApiKey in the Zilean section: Zilean__ApiKey
    • To set EnableScraping in the Dmm section: Zilean__Dmm__EnableScraping
    # Example of setting an environment variable for DMM scraping
    export Zilean__Dmm__EnableScraping=true
  8. Install Zilean using Docker Compose

    main

    The recommended way to install Zilean is via docker-compose. This setup starts two services:

    1. Zilean
    2. Postgres (version 17)

    Prerequisites

    Ensure you have one of the following installed:

    • Docker
    • Docker Desktop
    • Podman

    Setup Steps

    1. Generate a Postgres Password Generate a secure password and save it to a .env file using the following command:
    echo "POSTGRES_PASSWORD=$(openssl rand -base64 42 | tr -dc A-Za-z0-9 | cut -c -32 | tr -d '\n')" > .env
    1. Configure Compose File Use the provided compose-file.yaml to start the services.

    Note on Storage: By default, configuration and persistent storage are stored in Docker volumes. If you are not running on Windows, it is recommended to change the storage configuration to use a local ./data directory located next to your compose file.

  9. Run the Zilean Scraper manually

    main

    The Zilean scraper is the service responsible for content ingestion, including IMDB metadata. While it typically runs as a scheduled task, you can run it manually via Docker.

    CRITICAL: Before running the scraper manually, you must stop your API instance. Running the scraper and the API scheduler simultaneously will cause data corruption.

    docker compose stop zilean
    
    docker run -it --rm \
        -v ./settings.json:/app/data/settings.json \
        --name zileanscraper \
        ipromknight/zilean:latest \
        /app/scraper <args>
  10. Authenticate with the Zilean API

    main

    Some API endpoints require authentication. To perform authenticated requests, you must include your API key in the request headers.

    1. Locate your API Key: The ApiKey is generated on the first run of Zilean and is stored in your %settings-file%.
    2. Add Header: Include the key using the following header configuration:
      • Header Key: X-Api-Key
      • Header Value: <YOUR_API_KEY>
    HTTP/1.1
    X-Api-Key: YOUR_API_KEY_HERE
    ...