SoulSync Documentation

repository·main·Indexed 22 days ago

https://github.com/nezreka/soulsync

An intelligent automation platform for music and video libraries that automates discovery, downloading, metadata enrichment, and organization. It supports hybrid download sources including Soulseek, Deezer, Tidal, and YouTube, and integrates with media servers like Plex, Jellyfin, and Navidrome. Features include a visual Automation Engine for custom workflows, AcoustID audio verification, and comprehensive metadata workers for both audio and video content.

Tokens
48.5K
Snippets
82
Records
196
Agent score
84%

What's inside SoulSync

  1. Overview of SoulSync Platform

    main
    SoulSync is an intelligent music and video automation platform designed to bridge streaming services with self-hosted libraries. It automates the entire lifecycle of media management: from discovering new releases and generating playlists (similar to Spotify's Release Radar or Discovery Weekly) to downloading tracks from multiple sources (Soulseek, Deezer, Tidal, etc.), verifying audio quality via AcoustID, enriching metadata using 14 different workers, and organizing files into clean folder structures. It supports integration with media servers like Plex, Jellyfin, and Navidrome, or can be used as a standalone library manager.
  2. Understand Spotify and iTunes fallback logic

    main

    The SpotifyClient provides fallback capabilities to the iTunesClient to ensure high availability of metadata. The behavior depends on the method type:

    Search Methods

    search_tracks, search_albums, and search_artists always try Spotify first. If any exception occurs, they fall through to iTunes.

    ID-based Methods

    get_track_details, get_album, get_album_tracks, get_artist, and get_artist_albums try Spotify first. They only fall through to iTunes if the provided ID is numeric (id_str.isdigit()). If the ID is alphanumeric (Spotify format) but Spotify fails, the method returns None or [] and does not attempt an iTunes lookup.

    User-specific Methods

    Methods like get_user_playlists and get_saved_tracks are Spotify-only and have no fallback to iTunes.

  3. Add delays to automation actions

    main
    Action blocks include an optional Delay field (measured in minutes). The action will wait for the specified duration after the trigger fires before executing. This is useful for ensuring preceding processes (like file moves or database writes) have completed.
  4. Understand UnifiedTrack, UnifiedArtist, and UnifiedAlbum data structures

    main

    To ensure consistency across different metadata providers (Spotify vs. iTunes), SoulSync uses unified dataclasses. These objects normalize fields like id, name, artists, and album regardless of the source.

    Key Fields in UnifiedTrack:

    • id: A unique identifier (prefixed with itunes: if from iTunes).
    • source: The MetadataSource used (e.g., SPOTIFY_OAUTH, ITUNES).
    • spotify_id / itunes_id: Original source IDs for cross-referencing.
    • popularity: Available for Spotify sources; defaults to 0 for iTunes.
    • isrc: International Standard Recording Code (Spotify only).

    Key Fields in UnifiedArtist:

    • followers: Available for Spotify sources.

    Key Fields in UnifiedAlbum:

    • total_tracks: Number of tracks in the album.
    • explicit: Boolean indicating explicit content.
  5. Authenticate with the SoulSync API

    main

    All /api/v1/ endpoints (except the bootstrap endpoint) require authentication. Keys are prefixed with sk_.

    MethodDetails
    HeaderAuthorization: Bearer sk_...
    Query?api_key=sk_...

    Authentication Error Codes

    StatusCodeMeaning
    401AUTH_REQUIREDNo API key provided
    403INVALID_KEYAPI key is wrong or revoked

    Rate Limiting

    Requests are limited to 60 per minute per IP address. Exceeding this limit returns 429 RATE_LIMITED.

  6. Understand the Import route data and invalidation model

    main

    The Import route uses TanStack Query to manage data fetching and synchronization. Understanding the invalidation rules is critical for ensuring the UI reflects the current state of the staging folder.

    Query Options

    • Critical Loader Data: importStagingFilesQueryOptions()
    • Prefetch Data: importStagingGroupsQueryOptions(), importStagingSuggestionsQueryOptions()
    • Nested Route Data: autoImportStatusQueryOptions(), autoImportSettingsQueryOptions(), autoImportResultsQueryOptions(autoFilter)
    • Lazy Search Data: importAlbumSearchQueryOptions(query), importTrackSearchQueryOptions(query)

    Invalidation Rules

    To keep the UI in sync with backend changes, the following invalidation patterns are used:

    • Processing files (Album or Singles): Invalidates staging files, staging groups, staging suggestions, auto-import results, and any route-local queue completion summaries.
    • Auto-import actions: Invalidates auto-import status and results.
    • Auto-import settings writes: Invalidates settings and status.
    • Manual Refresh: Invalidates staging files, groups, and suggestions.
  7. Understand the SoulSync Metadata Fallback Strategy

    main

    SoulSync uses a tiered priority system for metadata and search operations. This allows the platform to provide immediate functionality to users without requiring them to set up Spotify Developer credentials or complete OAuth flows.

    Priority Order of Access Methods:

    1. Spotify OAuth: Full feature set including playlists, library, search, and metadata. Requires user credentials and OAuth flow.
    2. Spotify Client Credentials: Provides search and metadata functionality but requires application client_id and client_secret.
    3. Anonymous Spotify Access: Provides search and metadata functionality with no credentials required.
    4. iTunes Search API: Provides search and metadata functionality with no credentials required, using a different data source.
  8. Use Multi-Profile Support with X-Profile-Id

    main

    SoulSync supports multiple user profiles. To perform actions (like adding to a watchlist) or retrieve discovery data for a specific profile, include the X-Profile-Id header in your request. If omitted, the request typically defaults to the primary profile.

    Example: Adding an artist to the watchlist for profile ID 2 using Python:

    requests.post(
        f"{API_URL}/watchlist",
        headers={**headers, "X-Profile-Id": "2"},
        json={"artist_id": "4tZwfgrHOc3mvqYlEYSvnL", "artist_name": "Daft Punk"}
    )
  9. How the DownloadEngine and Plugins interact

    main

    SoulSync uses a centralized DownloadEngine to manage the complexities of downloading media from multiple sources. This architecture separates high-level orchestration from source-specific logic.

    The DownloadEngine Responsibilities

    • Concurrency & Threading: Spawns and manages BackgroundDownloadWorker instances.
    • State Management: Tracks active_downloads and manages global state_lock.
    • Rate Limiting: Uses a rate_limiter pool to ensure plugins respect source-specific limits (e.g., engine.rate_limit.acquire(source)).
    • Reliability: Manages fallback chains (engine.fallback_chain) and result deduplication.

    The Plugin Responsibilities

    Plugins are lightweight adapters that only handle:

    • Authentication: Managing OAuth tokens, session cookies, or API keys.
    • Protocols: Handling specific transport layers (e.g., HTTP REST, HLS demux, or yt-dlp subprocesses).
    • Atomic Operations: Executing the actual search_raw or download_raw calls.

    Implementation Nuances

    • Event-Driven Sources (e.g., Soulseek/slskd): For sources that are not thread-based, download_raw returns immediately, and the engine subscribes to source events for state updates.
    • Subprocess Sources (e.g., YouTube/yt-dlp): The plugin wraps the subprocess call; the engine treats the subprocess execution as the managed thread.
  10. Use the Automation Engine to build workflows

    main

    The Automation Engine is a visual drag-and-drop builder for creating custom workflows using Triggers, Actions, and Signal Chains.

    Triggers

    Common triggers include:

    • Schedule / Daily/Weekly Time
    • Track Downloaded / Batch Complete
    • Playlist Changed / Discovery Complete
    • Watchlist Match / Wishlist Item Added
    • Library Scan Complete

    Actions

    Common actions include:

    • Process Wishlist / Scan Watchlist
    • Sync Playlist / Discover Playlist
    • Scan Library / Database Update
    • Quality Scan / Full Cleanup
    • Discord/Telegram/Pushbullet notifications

    Signal Chains

    Automations can communicate via signals. One automation can fire a signal (e.g., signal:foo), which another automation listens for. SoulSync includes cycle detection, chain depth limits, and cooldowns to prevent runaway loops.

  11. Understand the Mirrored Playlist Sync Pipeline

    main

    To prevent raw metadata (like YouTube video titles) from polluting your library, use a three-step automation chain for mirrored playlists.

    The Pipeline Workflow

    1. Refresh: Re-fetch the playlist from the source (e.g., every 6 hours). This emits a Playlist Changed event if changes are detected.
    2. Discover: Triggered by Playlist Changed. Matches raw tracks to official Spotify/iTunes metadata via the matching engine and caches the result. This emits a Discovery Complete event.
    3. Sync: Triggered by Discovery Complete. Pushes only the verified, discovered tracks to your media server.

    Note for Spotify users: Spotify-sourced playlists skip the Discover step because their metadata is already official. You can chain Playlist Changed directly to Sync Playlist.

    ### Step 1: Refresh

    WHEN: Schedule (every 6 hours) DO: Refresh Mirrored Playlist (all)

    
    ### Step 2: Discover

    WHEN: Playlist Changed DO: Discover Playlist (all)

    
    ### Step 3: Sync

    WHEN: Discovery Complete DO: Sync Playlist (select playlist)

  12. Handle API response type inconsistencies

    main

    SoulSync returns different data structures depending on the method called. Consumers must be prepared to handle both Dataclass instances and raw Dictionaries.

    Return Type Mapping

    MethodReturns
    search_tracks/albums/artistsDataclass instances (Track, Album, Artist)
    get_track_detailsDict (enhanced, same shape both sources)
    get_albumDict (Spotify raw / iTunes normalized)
    get_album_tracksDict with items list
    get_artistDict (Spotify raw / iTunes normalized)
    get_artist_albumsDataclass instances (List[Album])

    Access Patterns

    • For Dataclasses: Use attribute access: track.name, track.artists.
    • For Dicts: Use key access: track_details['name'], track_details['album']['name'].