Sootio Stremio Addon

repository·main·Indexed 40 days ago

https://github.com/sooti/sootio-stremio-addon

An intelligent, multi-source streaming engine for Stremio (v1.9.1) that prioritizes high-quality cached torrents, Usenet downloads, and direct HTTP streams from Debrid services. It includes a Stream Provider module for fetching movie and series streams and a Media File Server for managing personal media libraries and automated Usenet downloads via SABnzbd.

Tokens
24.8K
Snippets
94
Records
131
Agent score
100%

What's inside Sootio

  1. How the Stream Provider module is structured

    main

    The Stream Provider is a modular system designed for separation of concerns. Its architecture is divided into several functional layers:

    • Configuration (config/): Manages timeouts and stream name prefixes (via STREAM_NAME_MAP).
    • Utilities (utils/): Handles URL validation, torrent sorting (by resolution/size), and stream filtering by size.
    • Caching (caching/): A dual-layer system using SQLite and in-memory storage. It includes cache-manager.js for retrieval/storage, background-refresh.js for async updates, and deduplication.js to prevent concurrent duplicate requests.
    • Debrid Providers (debrid/): Contains implementation logic for services like RealDebrid, AllDebrid, Premiumize, DebridLink, TorBox, OffCloud, Debrider.app, and Personal Cloud.
    • Formatters (formatters/): Converts raw data into Stremio-compatible stream objects. stream-formatter.js is the standard, while debrider-formatter.js handles NZB routing and personal cloud files.
    • Resolvers (resolvers/): The url-resolver.js module handles the logic of turning identifiers into streaming URLs for all supported debrid services.
    • Alternative Services (alternative-services/): Integrations for Easynews, Usenet/Newznab, Home Media Servers, and HTTP streaming scrapers (e.g., 4KHDHub, HDHub4u).
  2. Sootio Project Architecture Overview

    main

    The Sootio project is organized into several key functional areas:

    • Core Server: server.js (Express setup) and cluster.js (multi-worker management).
    • Addon Logic: addon.js defines the Stremio addon interface.
    • Providers (lib/):
      • stream-provider.js: Orchestrates stream discovery.
      • {provider}.js: Individual integrations for various Debrid services.
      • catalog-provider.js: Manages personal download catalogs.
    • Common Utilities (lib/common/):
      • scrapers.js: Contains 14 different torrent scrapers.
      • cache-store.js: Handles backend selection (SQLite or Postgres).
      • debrid-cache-processor.js: Processes debrid cache data.
    • Integrations (lib/):
      • http-streams.js (4KHDHub), uhdmovies.js, usenet.js, newznab.js, sabnzbd.js, and home-media.js.
    • Usenet File Server (media-file-server/):
      • Python-based servers (usenet_file_server.py or fastapi_file_server.py) for progressive streaming.
  3. How Usenet progressive streaming works

    main

    The Usenet streaming feature enables near-instant playback by leveraging SABnzbd's download and extraction process.

    The Workflow

    1. Search: The add-on queries your Newznab indexer.
    2. Selection & Submission: Selecting a result submits the NZB to SABnzbd.
    3. Progressive Download: Once the download reaches 3%, the add-on begins looking for the video file.
    4. Instant Streaming: If Direct Unpack is enabled in SABnzbd, the video file is extracted during the download. The add-on streams directly from the _UNPACK_ folder or the incomplete folder.
    5. Background Processing: Download and extraction continue while you watch.

    Key Features

    • Range Request Support: You can seek/skip forward in videos. If the requested part isn't downloaded yet, the player will show a "please wait" message. Because SABnzbd downloads sequentially, earlier parts are always available.
    • Smart File Detection: The add-on automatically identifies the correct video file in multi-file NZBs (e.g., matching episodes in TV shows) and selects the largest video file available.
    • Download Management:
      • Downloads already in your SABnzbd queue are marked with a 💾 "Personal" label.
      • Incomplete downloads are automatically cleaned up after 10 minutes of inactivity.
      • The system monitors disk space: it warns at <10GB and blocks downloads at <2GB.
  4. How universal archive support works via fuse-archive

    main

    The server uses fuse-archive (Google's universal archive mounting tool) to provide transparent access to compressed files without manual extraction. This saves disk space and time.

    Key Details:

    • Mount Point: Files are served from /mnt/archivefs.
    • Supported Formats: RAR (including multi-part), 7z, ZIP, TAR (.tar.gz, .tar.bz2, .tar.xz), ISO, CPIO, and more via libarchive.
    • Docker Requirements: You must run the container with the --privileged flag OR use --device /dev/fuse --cap-add SYS_ADMIN to grant FUSE access.
  5. How Python File Server authentication works

    main

    The Python file server validates access using three possible methods. The Node.js server automatically includes the password from your configuration in these formats:

    1. HTTP Header: X-API-Key: <password>
    2. Query Parameter: ?key=<password> (useful for direct browser or app access)
    3. Authorization Header: Authorization: Bearer <password>

    Security Note: Because the password is sent in the addon URL, you should use HTTPS in production environments. You can provide different passwords to different users by generating unique addon URLs.

  6. Set up Sootio for Development

    main

    To contribute to or develop Sootio, clone the repository, install dependencies using pnpm, and use the provided scripts to run the development server or tests.

    # Clone and install
    git clone https://github.com/your-username/sootio-stremio-addon.git
    cd sootio-stremio-addon
    pnpm install
    
    # Run in development mode
    npm run dev
    
    # Run tests (if available)
    npm test
  7. Recover corrupted SQLite databases

    main

    If you encounter errors like database disk image is malformed, use the recover-sqlite.sh script.

    Recovery process:

    1. Creates a backup of the corrupted database.
    2. Attempts to checkpoint the WAL (Write-Ahead Log) file.
    3. Runs integrity checks.
    4. Attempts to rebuild from recoverable data.
    5. If recovery fails, it creates a fresh database.

    Note: The script automatically restarts the container after recovery. If the database was too corrupted to recover, the cache will be empty and will repopulate as you use the addon.

    Environment Variables:

    • SOOTIO_CONTAINER: Specify a custom Docker container name (defaults to sootio).
    # Recover cache.db (most common)
    ./scripts/recover-sqlite.sh cache.db
    
    # Recover hash-cache.db
    ./scripts/recover-sqlite.sh hash-cache.db
    
    # Use custom container name
    SOOTIO_CONTAINER=my-sootio ./scripts/recover-sqlite.sh cache.db
  8. Use the Python File Server for Usenet Streaming

    main

    The easiest way to serve Usenet files is using the included Python script. It has zero dependencies (uses Python's built-in HTTP server), supports range requests (enabling seeking), and has CORS enabled for remote access.

    Basic Usage: Run the script on your SABnzbd machine pointing to your video directory:

    python3 usenet_file_server.py /home/sooti/Videos

    Command-line options:

    • --port <port>: Specify a custom port.
    • --bind <address>: Bind to a specific interface. Use 127.0.0.1 for localhost only (more secure) or 0.0.0.0 to allow remote connections.
    # Custom port
    python3 usenet_file_server.py /home/sooti/Videos --port 9000
    
    # Localhost only (more secure)
    python3 usenet_file_server.py /home/sooti/Videos --bind 127.0.0.1
    
    # Allow remote connections
    python3 usenet_file_server.py /home/sooti/Videos --bind 0.0.0.0
  9. Integrate with Stremio

    main

    The Media File Server can be used in two primary ways within Stremio via the Sootio addon.

    Home Media Server (HM+)

    Use this for your personal media library.

    1. Start the file server on port 3003.
    2. In Stremio addon configuration, add a service:
      • Provider: Home Media Server
      • Server URL: http://your-server-ip:3003
      • API Key: Your configured API key.
    3. Files will appear in Stremio with a ☁️ cloud icon.

    Usenet (UN+)

    Use this with SABnzbd for automated downloads.

    1. Configure SABnzbd to download to your target directory.
    2. Start the file server pointing to that directory.
    3. In the Stremio addon, configure Usenet with:
      • Newznab URL & API Key
      • SABnzbd URL & API Key
      • File Server URL: http://your-server-ip:3003
    4. New downloads will be queued in SABnzbd and streamed as soon as they are complete.
  10. Set up Usenet Progressive Streaming

    main

    You can enable progressive streaming for Usenet content by setting up a Python-based file server. This allows streams to start once a certain download threshold is met.

    1. Install the Python file server: Navigate to the media-file-server directory and install dependencies (if using the FastAPI version) or run the zero-dependency version.
    2. Configure environment variables: Set USENET_FILE_SERVER_URL and USENET_FILE_SERVER_API_KEY in your .env file.
    3. Configure Stremio UI: In the Stremio interface, add your Newznab indexer URLs/API keys and your SABnzbd URL/API key. Streams will begin playing once the download reaches 3% completion.
    # 1. Install Python file server
    cd media-file-server
    pip install -r requirements.txt  # if using FastAPI version
    python usenet_file_server.py    # zero-dependency version
    # 2. Configure in .env
    USENET_FILE_SERVER_URL=http://localhost:8765
    USENET_FILE_SERVER_API_KEY=your_secret_key
  11. Scale Sootio using Clustering

    main

    To handle high loads, Sootio supports multi-worker clustering. You can run the application in different modes depending on your environment:

    • Single process: Use this for debugging purposes to avoid the complexity of multiple workers.
    • Multi-worker (Default): Use npm start to launch with default worker settings.
    • Custom worker count: Set the MAX_WORKERS environment variable to specify the number of workers.
    # Single process (debugging)
    npm run standalone
    
    # Multi-worker with default settings
    npm start
    
    # Custom worker count
    MAX_WORKERS=16 npm start
  12. Customize the HTTP Health Check script

    main

    You can modify scripts/test-http-streams-status.mjs to change the testing parameters:

    1. Change test content: Update the TEST_MOVIE object with a different imdbId, title, and type.
    2. Add/remove providers: Modify the PROVIDERS array to include or exclude specific provider functions.
    3. Change timeout: Adjust the timeout variable (e.g., 60000 for 60 seconds).
    // Example customization in scripts/test-http-streams-status.mjs
    
    const TEST_MOVIE = {
      imdbId: 'tt1375666',  // Inception instead
      title: 'Inception',
      type: 'movie'
    };
    
    const PROVIDERS = [
      { name: 'CineDoze', fn: getCineDozeStreams },
    ];
    
    const timeout = 60000; // 60 seconds