DOCSEARCHbyAlgolia
Submit Docs

AltMount

repository·main·Indexed 43 days ago

https://github.com/javi11/altmount

A WebDAV server backed by NZB/Usenet that provides seamless access to Usenet content through standard WebDAV protocols. It can be deployed via Docker or installed using Go, featuring support for FUSE device access, auto-updates via the Docker socket, and Windows long path support.

Tokens
72.2K
Snippets
200
Records
290
Agent score
56%

What's inside AltMount

  1. Overview of PAR2 Repair Cancellation

    main

    The PAR2 repair cancellation feature allows users to stop queued or in-flight PAR2 repairs from the Health page. The system ensures that all transient scratch and temporary artifacts are provably deleted before the API responds to the cancellation request.

    Key Architectural Components:

    • par2repair.Service: Maintains an in-process registry of active jobs. Each job is tracked with a context.CancelFunc and a done channel.
    • Cancellation Flow: Calling Cancel triggers the job's context cancellation. The system waits for the worker goroutine to unwind, ensuring that RunJob's arena defer statements execute (which delete mmap scratch files) before the service sweeps any remaining orphaned scratch/tmp files.
    • Bookkeeping: The worker goroutine is responsible for terminal bookkeeping (e.g., failing parked imports, deleting rows) to ensure a single owner for state transitions.

  2. Overview of nntppool Priority Dispatch Implementation

    main

    The nntppool Priority Dispatch implementation aims to ensure priority lanes are strictly prioritized, prevent streaming BODY commands from occupying connections that are already draining a BODY, and eliminate lock contention in StatMany's dispatch without reducing overall throughput.

    Architectural Changes

    1. Strict Lane Probing: tryNextRequest is modified to probe lanes in a strict order instead of using an unbiased select statement.
    2. Body Steering: A new unbuffered group channel, hotIdleBodyCh, is introduced. A connection offers this channel only when it has no BODY in flight (it provides a nil channel otherwise). This allows priority bodies to be steered to free connections rather than reserving capacity.
    3. Atomic Cursor Dispatch: In StatMany's worker dispatch, an unbuffered channel is replaced with an atomic cursor to reduce contention.

    Constraints and Requirements

    • No Throughput Regression: Import MB/s and aggregate STAT/s must remain stable. Mechanisms must have zero cost when no priority traffic is present.
    • No Static Reservation: Capacity or connections must not be statically reserved.
    • API Stability: The public API of nntppool remains unchanged. StatMany retains its signature and its contract that "results arrive out of order."

  3. Avoid body pipelining for imports in AltMount

    main

    While it may seem intuitive to increase import throughput by pipelining multiple bodies per connection, benchmarks show this is often harmful on bandwidth-constrained links.

    Increasing the import budget (e.g., doubling it) can lead to:

    • Increased Latency: Stream p99 latency can more than double.
    • Reduced Playback Bandwidth: The extra in-flight bodies compete for the same saturated link, reducing the bandwidth available for active streams.

    Instead of increasing import concurrency, use import.stream_headroom_connections to manage how much pool slack is allocated to playback versus imports.

  4. Design for Alias-aware matching in Stremio stream gate

    main

    To resolve title mismatches for anime and other localized content, the system uses an alias-aware matching strategy:

    1. Alias Fetching: resolveSeriesMetadataFromIMDb fetches aliases from https://api.tvmaze.com/shows/{id}/akas once per IMDb ID (cached). It collects names where country == "en" (falling back to any country if none found).
    2. Title Propagation: A slice of strings []string containing all valid titles/aliases is passed through the Stremio search and scoring path.
    3. Matching Logic: releaseMatchesContent succeeds if MatchesSeries passes for ANY alias in the list.
    4. Strictness: The strict 1:1 comparison is maintained per-alias to prevent false positives (e.g., correctly rejecting "ARK: The Animated Series" when searching for "The Ark").
    5. Resilience: Aliases are cached by imdbID, and the system is designed to tolerate failures in fetching aliases (treating aliases as nil).

  5. Understanding the Anime title-matching gap in Stremio stream gate

    main

    A known issue exists where classic anime requests in Stremio may return "No streams available" even when indexers have valid releases. This occurs because the resolveSeriesMetadataFromIMDb function uses the primary name from TVMaze (often Romaji, e.g., "Meitantei Conan"), while release names use different localizations (e.g., "Detective Conan").

    Matching fails because:

    1. MatchesSeries performs a strict 1:1 comparison of CleanSeriesTitle.
    2. releaseMatchesContent (the fallback) requires $\ge 0.8$ token coverage. In the case of "Detective Conan" vs "Meitantei Conan", the release might only contain "conan", resulting in $0.5$ coverage, which causes the release to be dropped.

  6. How demand-shaping and speculative budgets work together

    main

    AltMount uses a tiered approach to streaming to balance throughput and resource usage:

    1. Demand Fetches: These are high-priority fetches for data immediately needed by the reader. They bypass the SpeculativeBudget to ensure that even if the pool is heavily utilizing read-ahead slots, the active stream's immediate needs are met without delay.
    2. Speculative Fetches: These are read-ahead fetches intended to buffer data for future use. They are governed by the pool.SpeculativeBudget. If the budget is exhausted, these fetches are deferred.
    3. The Budget: The SpeculativeBudget acts as a global throttle for all speculative fetches across the entire pool.Manager. This prevents a single stream or a group of streams from consuming all available provider connections for read-ahead, which would otherwise starve other streams or the primary demand path.

  7. How keyword filters work in Stremio integration

    main

    AltMount uses token-boundary matching for keyword filters (languages, qualities, and excludes) rather than plain substring matching. This prevents accidental matches inside larger words.

    • Token Matching: A keyword like TS matches Movie.2024.TS.1080p but will not match DTS-HD.MA or Knights.
    • Multi-word phrases: Phrases like Korean Dub will match across delimiters such as Korean.Dub, Korean_Dub, or Korean-Dub.
    • Regular Expressions: You can use explicit regex. Use bare patterns like \b(cam|ts)\b or slash-delimited patterns like /(cam|ts)/i.

    Note for Upgraders: If you are upgrading from an older version that used substring matching, you must update your exclude lists. For example, if you previously used ip to block WEBRip, you should now use \bip\b|webrip or the full token webrip.

  9. Understand Faithful vs. Synthesized stores

    main

    During migration, AltMount attempts to create two types of .nzbz stores for a group of files:

    1. Faithful Store

    If the source_nzb_path is still present on disk, AltMount builds a 'faithful' store by parsing the original NZB. This store contains the real subjects, posters, dates, groups, and segment numbers. It is preferred because it preserves original metadata and is highly compatible with external clients.

    2. Synthesized Store

    If the original NZB is missing, AltMount creates a 'synthesized' store by walking the segments currently present in the legacy .meta files.

    • Subject: Set to the virtual filename.
    • Groups: Uses the value defined in metadata.migration.default_group (default: alt.binaries.misc).
    • Efficiency: It uses segment_runs to collapse contiguous segment indices, maximizing space savings.

  10. How the Segment Cache memory tier works

    main

    The Segment Cache uses a tiered storage architecture to serve recently decoded articles from memory, reducing disk I/O and download overhead.

    • segcache.MemoryCache: A sharded, byte-bounded cache using a CLOCK (second-chance) eviction algorithm. It is keyed by message-ID. It is enabled by default with a 256 MB capacity if MemoryMB is configured.
    • segcache.TieredStore: Implements the usenet.SegmentStore interface. It coordinates between the memory tier and the disk tier:
      • Get: Attempts to retrieve data from memory first. If a hit occurs in the disk tier, the data is promoted to memory.
      • Put: Stores data in memory immediately and queues the disk write to a bounded asynchronous writer (queue size 64). If the queue overflows, writes are dropped and tracked via Dropped().
    • segcache.Source.Store(): Returns the long-lived TieredStore instance. The memory tier is active if MemoryMB > 0, and the disk tier is active if a disk store is provided.
    // Conceptual flow of TieredStore.Get
// 1. Check MemoryCache (fast hit)
// 2. If miss, check Disk (slow hit)
// 3. If disk hit, promote to MemoryCache

  11. Use the API key for compatibility routes

    main

    The per-user API key (found in System → Settings) is not a substitute for JWT authentication on most endpoints. It is only accepted by a specific subset of compatibility routes via the ?apikey= query parameter.

    Supported endpoints for API key usage:

    • /api/sabnzbd/*: SABnzbd-compatible API (also supports ma_username + ma_password).
    • POST /api/arrs/webhook: Sonarr/Radarr webhooks (requires ?apikey=).
    • POST /api/import/file: Manual NZB file import (requires ?apikey=).

    For all other categories (Queue, Health, Config, etc.), you must use the JWT authentication flow.

  12. Streaming benchmark scenario metrics

    main

    When running streaming benchmarks, the following scenarios and metrics are used to evaluate performance:

    ScenarioMetric NameDescription
    B1-cold-openttfb_p50, ttfb_p99, articlesTime to first byte (percentiles) and total articles processed.
    B2-sequentialthroughput, waste_ratioThroughput in MB/s and the ratio of fetched bytes to read bytes.
    B3-seek-stormread_p50, read_p99, articles_per_readLatency of read operations and density of articles per read.
    B8-pause-resumebytes_during_pauseAmount of data (MB) processed during a pause/resume cycle.

    Scenario names are typically formatted as {scenario_name}/{profile_name} (e.g., B1-cold-open/premium-750k).