podfetch

repository·main·Indexed 19 days ago

https://github.com/samtv12345/podfetch

A self-hosted podcast management web application for downloading, online listening, and advanced transcript-based searching. Built with a Rust backend and React frontend, it features GPodder integration, Podcasting 2.0 support, and transcript generation via OpenAI-compatible Whisper APIs. The project includes a mobile app built with Expo and supports deployment via Docker and Terraform.

Tokens
97.5K
Snippets
258
Records
372
Agent score
67%

What's inside podfetch

  1. Overview of Podcast Transcript Support Implementation

    main

    The Podcast Transcript support feature enables PodFetch to parse <podcast:transcript> tags, archive transcript files, display transcripts in the player, enable full-text search, and generate missing transcripts using an OpenAI-compatible Whisper API.

    Architecture Model

    The implementation follows a hybrid storage approach: original files are archived, while normalized segments are stored in the database. It utilizes native database full-text search (SQLite FTS5 or Postgres tsvector) and a database-backed job queue with a Tokio worker for Whisper processing.

    All components follow the established pattern:

    1. Trait definition: in podfetch-domain
    2. Diesel Repository: in podfetch-persistence (with adapters in adapters.rs)
    3. Service layer: in podfetch-web/src/services/
    4. Controller layer
    5. React UI
  2. Overview of Podfetch

    main

    Podfetch is a self-hosted podcast manager designed for downloading and listening to podcasts online.

    Key features include:

    • GPodder Integration: Allows users to sync and continue using their existing podcast applications.
    • Podcasting 2.0 Support: Provides full-text search across episode transcripts.
    • Transcript Generation: Can generate missing transcripts using any OpenAI-compatible Whisper API.
    • Tech Stack: Built with Rust (backend) and React (frontend).
  3. Overview of the Podfetch UI frontend

    main

    The podfetch-ui package provides the user interface for the Podfetch ecosystem. It is designed to allow users to interact with podcast content through a web-based or client-side interface.

    Key features include:

    • Search and download podcasts: Users can discover new content and initiate downloads.
    • Listen to podcasts: Integrated playback capabilities for consuming podcast audio.
  4. Navigate the Podfetch UI views

    main

    The Podfetch interface is organized into several primary views:

    • Home Screen: The main landing area to access episodes and continue listening.
    • Timeline View: A view for managing or viewing episode timelines.
    • Info View: Detailed information pages for podcasts or episodes.
    • Settings: Configuration for user preferences.
    • Administration: Administrative controls and management interfaces.
  5. Understand Transcript Preference Logic

    main

    PodFetch manages multiple transcript variants (e.g., JSON, VTT, SRT, HTML) per episode. It uses a preference system to decide which one to display and index. The is_preferred flag is recalculated whenever transcripts change.

    Preference Rules:

    1. Status Check: Only variants with status='parsed' are eligible. Non-parseable feed transcripts do not block the display or the Whisper fallback.
    2. Source Priority: feed (from RSS) always takes precedence over generated (via Whisper).
    3. Format Priority: Within the same source, the order is: JSON > VTT > SRT > HTML/Text.
    4. Language Match: If formats are equal, the variant matching the podcast's language is chosen.
    5. Fallback: If a new parseable feed transcript is discovered during a refresh, the preference switches from generated to the feed version, and segments are rebuilt accordingly.
  6. Technical Stack for Podcast Transcripts

    main

    The implementation of podcast transcript features relies on the following technology stack:

    • Backend: Rust (Axum, Diesel sync, Tokio)
    • HTTP Client: reqwest (requires the multipart feature enabled)
    • Frontend: React 19, TypeScript, TanStack Query, Zustand
    • Real-time: socketioxide
    • Parsing: Custom-written VTT/SRT parsers (approximately 50 lines each) instead of external crates to ensure reliability and maintainability for these specific formats.
  7. Understand the Chromecast permission model

    main

    Chromecast devices are categorized by their visibility and ownership:

    • chromecast_personal: Visible only to the user who owns them. Devices reported by an agent default to this type, owned by the user whose API key was used to authenticate the agent.
    • chromecast_shared: Household devices visible to every user on the PodFetch instance. These are created when an admin promotes a personal device to shared status.

    Session Ownership: When casting, you can select from your personal devices or any shared devices. Active sessions are owned by the user who initiated them. Only the owner can perform actions like pause, stop, or seek. If a different user starts a session on the same device, the previous session is implicitly stopped.

  8. E2E Test Stability and Error Handling Best Practices

    main

    To ensure reliable E2E tests, the following patterns are used:

    • Web-First Assertions: Avoid fixed sleep calls. Use Playwright's polling assertions like expect(...).toBeVisible() to handle asynchronous UI updates.
    • Isolation: Use a fresh SQLite database in a temporary directory for every run to prevent cross-run state contamination.
    • Network Binding: All mock servers are configured to bind strictly to 127.0.0.1.
    • Authentication: The instance is configured to run without auth. If the UI requires a login, the global setup performs a single minimal login step and stores the resulting storage state for subsequent tests.
  9. How PodFetch handles transcripts

    main

    PodFetch provides end-to-end support for Podcasting 2.0 transcripts through several integrated features:

    • Feed transcripts: Automatically records <podcast:transcript> tags during feed refreshes. It downloads and archives the best matching format (preferring VTT > SRT > JSON > HTML) alongside the audio file.
    • Player integration: The audio player includes a Transcript tab that supports auto-scroll and allows jumping to specific segments by clicking them.
    • Full-text search: Parsed transcripts are indexed (using SQLite FTS5 or PostgreSQL tsvector). The episode search page includes a Transcripts mode to find spoken words and jump to the matching audio position.
    • Generated transcripts: For episodes lacking feed transcripts, PodFetch can generate them using any OpenAI-compatible Whisper API.
    • RSS re-export: Archived transcripts are included as <podcast:transcript> tags in the RSS feeds generated by PodFetch.
  10. Architecture of Playwright E2E Tests for Transcript Features

    main

    The E2E testing suite uses Playwright to drive a full-stack environment consisting of the real PodFetch binary and two mock servers. The lifecycle of these processes is managed by Playwright's webServer configuration.

    Component Stack

    1. Fixture Feed Server (ui/e2e/servers/fixture-feed.mjs): A Node.js server running on 127.0.0.1:9123 that serves a test feed.xml containing a <podcast:transcript> VTT tag, a minimal MP3, and a transcript.vtt file.
    2. Mock Whisper Server (ui/e2e/servers/mock-whisper.mjs): An OpenAI-compatible Node.js server on 127.0.0.1:9998. It handles POST /v1/audio/transcriptions requests, returning fixed verbose_json segments after a ~1s delay to allow testing of 'pending' UI states.
    3. PodFetch Binary: The built application running on 127.0.0.1:8000. It requires the following environment configuration:
      • DATABASE_URL: Points to a temporary SQLite file.
      • TRANSCRIPTION_API_BASE_URL: Set to http://127.0.0.1:9998.
      • Working Directory: Must contain a static/ directory populated with the built UI distribution.

    Data Seeding

    Tests do not rely on pre-existing database state. Instead, a global setup uses the Playwright request context to seed data via the API:

    • Call POST /api/v1/podcasts/feed to add the fixture podcast.
    • Wait for the episode download and the parsing of the feed transcript before proceeding to UI-driven tests.