tuliprox

repository·develop·Indexed 19 days ago

https://github.com/euzu/tuliprox

A high-performance IPTV proxy and playlist processor written in Rust. It unifies media sources including M3U/M3U8 playlists, Xtream providers, Stalker/Ministra portals, Plex, Emby, and Jellyfin into reshaped outputs for IPTV players and media clients. Features include Role-Based Access Control (RBAC) via Argon2 password hashing and support for deployment via Docker with Traefik, Gluetun, CrowdSec, and IPTV-org-epg.

Tokens
105.7K
Snippets
247
Records
426
Agent score
67%

What's inside tuliprox

  1. Overview of Tuliprox Capabilities

    develop

    Tuliprox is a high-performance IPTV proxy and playlist processor written in Rust. It is designed for several use cases:

    IPTV Enthusiasts

    • Merge multiple providers into one unified playlist.
    • Filter, rename, and sort channels.
    • Automatic EPG assignment with fuzzy matching.
    • Integration with Kodi, Jellyfin, Emby (via STRM), and Plex (via HDHomeRun).

    Self-Hosted & Homelab Users

    • Single Docker container deployment with no external database required (uses a custom B+Tree engine).
    • Low resource footprint suitable for Raspberry Pi or small VPS.
    • VPN/SOCKS5 routing for upstream traffic with public IP verification via Web UI.

    Multi-User Operations

    • Multi-tenant edge gateway to publish multiple virtual endpoints.
    • User management with connection limits and priority levels.
    • RBAC (Role-Based Access Control) with 14 granular permissions.
    • Panel API for automated account management.

    Developers & Power Users

    • Mapper DSL: For arbitrary playlist transformations.
    • Management REST API: For configuration, processing, and operational automation.
    • CLI Mode: For scripting and CI/CD integration.
  2. Supported Input and Output Formats

    develop

    Tuliprox acts as a bridge between various IPTV input sources and multiple output formats, making it compatible with both IPTV players and media-server workflows.

    Supported Inputs

    • M3U / M3U8 playlists
    • Xtream inputs
    • Local library content

    Supported Outputs

    • M3U
    • Xtream-style outputs
    • HDHomeRun
    • STRM Files
  3. Operational and Management Features

    develop

    Tuliprox includes several tools for maintaining and monitoring the service:

    • Automation: Scheduled playlist refreshes and hot config reload support.
    • Resilience: Provider failover and DNS-aware connection rotation.
    • Recording: Integrated download and recording manager with provider-aware fairness, retries, and RBAC.
    • Monitoring: Notifications, monitoring hooks, and a Web UI for monitoring and web-based configuration.
  4. Transform Playlists with the Mapper DSL and Filter Engine

    develop

    Use the built-in processing pipeline to merge, filter, and transform multiple input sources into a single target.

    Filter Engine: Use complex boolean expressions for selection, e.g., (Group ~ "^DE.*") AND NOT (Name ~ ".*XXX.*").

    Mapper DSL: A custom language for transformations including:

    • Regex: Renaming with capture groups and backreferences.
    • Logic: Variables, if/else blocks, and for_each loops.
    • Functions: replace, pad, format, first, capitalize, lowercase, uppercase, template.
    • Counters: Support for padded counters (e.g., 001, 002).

    Sorting: Use regex sequences for groups/channels or order: none to preserve source order. Supports multi-level sorting via named capture groups (c1, c2, etc.).

  5. Manage HLS and Catchup session affinity

    develop

    To prevent provider bans for 'Account Hopping' caused by frequent connects/disconnects, Tuliprox uses Virtual Reservations to maintain account affinity.

    • hls_session_ttl_secs: (Default: 15) Keeps the virtual provider slot open between HLS segment (.ts) requests.
    • catchup_session_ttl_secs: (Default: 45) Keeps the reservation alive during seeking and reconnects for Archive/Catchup TV.

    Note: Channel switches from the same client immediately take over the reservation, bypassing these TTLs. Regular TS/VOD/local playback is socket-bound and does not use these session-holding principles.

  6. How the MetadataUpdateManager works

    develop

    The MetadataUpdateManager is an asynchronous background engine (activated via resolve_background: true on an input) that processes metadata without blocking the main playlist update.

    Key Behaviors:

    • Isolation: A dedicated Tokio Task (Worker) is started per Provider-Input to prevent slow providers from affecting others.
    • Task-Merging: If a stream needs both TMDB info and an FFprobe, they are merged into a single Task.
    • Preemption: FFprobe tasks run at the lowest priority (user_priority: 127). If a user starts streaming, the FFprobe process is immediately aborted/preempted to free the connection slot for the user.
    • Persistence: Retry, exhaustion, and cooldown states are saved in metadata_retry_state.db. This ensures Tuliprox remembers the status of broken streams across server restarts.
    • Cascading Updates: Metadata updates are saved to the Input DB and immediately cascaded into all Target DBs without requiring a full playlist rebuild.
  7. Handle ICS Timezones and Formats

    develop

    Tuliprox converts all imported calendar events into an internal EPG time format. The ics.timezone must be a valid IANA timezone (e.g., UTC, Europe/Budapest, America/New_York).

    Supported ICS Time Forms:

    • DTSTART:20260306T123000Z: Treated as UTC.
    • DTSTART;TZID=Europe/Berlin:20260306T1230: Interpreted in the specified TZID and converted internally.
    • DTSTART:20260306T123000: Treated as a floating timestamp and interpreted using the configured ics.timezone.
    • DTSTART;VALUE=DATE:20260306: All-day events (these are ignored).

    Note on End Times: An ICS event must provide a valid start and end time. DTEND is preferred. If DTEND is missing, Tuliprox may use DURATION. Events without a usable end time are skipped.

  8. Understand HLS Cache Session Identifiers

    develop

    Tuliprox uses several distinct identifiers to manage HLS playback, ranging from shared content identity to individual user playback sessions. Understanding these is critical for debugging URL structures and session behavior.

    Key Identifiers

    • HlsSessionKey (Shared Content): A stable tuple of input_id, the literal HLS kind, and stream_ref (the original input_stream_id). This identifies the content itself, regardless of which target or mirror is used.
    • proxy_session_id (Shared Public URL Identity): An opaque token derived from the HlsSessionKey and a configured secret. It is used to construct canonical URLs for shared content.
    • HlsPlaybackFamilyKey (User/Client Family): A tuple of the Tuliprox username and a client fingerprint key, used to group playback attempts by user and device.
    • hls_access_lease_id (Per Playback URL Identity): A random lookup key for a specific server-side HlsAccessLease. This is not a shared-content identity; it is unique to a single playback attempt.
    • HLS cache user session token: An internal token associated with an access lease for admission control.

    Canonical URL Structure

    Public canonical paths follow this pattern:

    /hls/shared/live/<proxy_session_id>/<hls_access_lease_id>/manifest.m3u8
    /hls/shared/live/<proxy_session_id>/<hls_access_lease_id>/<segment_file>
    /hls/shared/live/<proxy_session_id>/<hls_access_lease_id>/map/<map_file>
    /hls/shared/live/<proxy_session_id>/<hls_access_lease_id>/r/<resource_file>
  9. Understand the Connection Handling Runtime Flow

    develop

    The Tuliprox runtime manages the lifecycle of a playback request through a series of stages: resolving identity, classifying the request, resolving admission (user and provider), activating the session, and managing cleanup.

    High-level lifecycle:

    1. Endpoint Resolution: Resolves user, target, item, and input to build a session fingerprint.
    2. Request Classification: Determines if the request is Prepare, Activate, FollowUp, or Terminate.
    3. Admission Resolution: Evaluates if the user is allowed to play (user admission) and if the provider has capacity (provider admission).
    4. Activation: Creates a session placeholder and acquires the playback transition gate.
    5. Streaming: Opens the provider stream or reuses a shared stream.
    6. Cleanup: Handles disconnection via removal, preservation, or expiration.
  10. The 5 Pillars of Tuliprox Configuration

    develop

    Tuliprox follows a Separation of Concerns design pattern. Instead of a single monolithic file, configuration is split into five distinct files that must be placed in your config/ directory to fully utilize the system.

    FileResponsibilityArchitecture Level
    config.ymlThe Core System. Defines physical execution: ports, reverse proxy buffers, paths, TMDB API keys, metadata worker limits, Web UI settings, and global logging.Infrastructure & Engine
    source.ymlThe Data Flow. Defines inputs (Provider URLs, Xtream credentials, Panel API limits) and outputs (Targets, Filter assignments, Formats like STRM or M3U).Data Sources & Targets
    api-proxy.ymlThe Gateway. Defines virtual server endpoints for clients (VLC, TiviMate) and handles Access Management (user-to-target mapping, proxy modes, and user priority).Network & Auth
    mapping.ymlThe Transformation. Uses an embedded DSL to dynamically rename streams, reassign groups, or map IDs/counters using regex filters.Data Enrichment
    template.ymlThe DRY Principle. Contains globally reusable Regular Expressions (Regex) and logic macros invoked in source.yml and mapping.yml via !MACRO_NAME! syntax.Structuring
  11. Understand Tuliprox Home Directory and Path Resolution

    develop

    Tuliprox uses a central Home Directory to resolve all relative paths defined in your configuration files (e.g., storage_dir: ./data). To ensure predictable behavior, you should explicitly define or understand how this directory is determined.

    Path Resolution Order

    Tuliprox determines the Home Directory using the following priority:

    1. CLI Argument: --home or -H (Highest Priority)
    2. Environment Variable: TULIPROX_HOME
    3. Fallback: The physical directory where the tuliprox binary is located.

    Default Directory Structure

    When initialized, Tuliprox creates a standard directory tree under the Home Directory:

    tuliprox_home/
     ├─ config/         # Contains config.yml, source.yml, mapping.yml, user.txt
     ├─ data/           # Primary storage_dir for B+Tree databases (*.db)
     ├─ data/backup/    # Backups initiated by the Web UI
     ├─ data/user/      # User-specific configurations (like favorites)
     ├─ downloads/     # Downloaded VODs
     └─ web/            # Frontend assets for the Web UI
     └─ cache/          # Cached resources
    # Example: Setting the home directory via CLI
    tuliprox --home /opt/tuliprox
    
    # Example: Setting the home directory via Environment Variable
    export TULIPROX_HOME=/opt/tuliprox
    tuliprox
  12. Understand the mapping processing pipeline

    develop

    Tuliprox processes input playlists through a specific lifecycle. Understanding this helps you decide which stage to use for your mapping rules.

    1. Input playlist
    2. Identity Freeze: Removes duplicates.
    3. Processing Pipe (Order configurable via processing_order):
      • F: Filter
      • R: Rename
      • M: Mapper (Blocks with stage: processing)
    4. Metadata Resolution: Resolves series, VOD, and stream probes.
    5. EPG Enrichment: Adds EPG data.
    6. Late Mapping: Mapper blocks with stage: after_epg (can access epg_channel_id and logo).
    7. Consolidation: Collects inputs, applies Favourites/Trakt categories.
    8. Finalization: Merges groups $\rightarrow$ Sorts $\rightarrow$ Assigns channel numbers $\rightarrow$ Mapping Counters $\rightarrow$ Watches $\rightarrow$ Persistence.

    Note: counter blocks always run at the very end, after merging and sorting, regardless of the stage defined in the mapper blocks.