DOCSEARCHbyAlgolia
Submit Docs

Sushitrain Documentation

repository·main·Indexed 24 days ago

https://github.com/pixelspark/sushitrain

A file synchronization application for iOS and macOS that implements the Syncthing protocol. It features a hybrid architecture with a Go-based core (SushitrainCore) and a Swift/SwiftUI frontend. Key capabilities include on-demand file access, selective synchronization via .stignore, remote thumbnail generation, and specialized iOS photo synchronization through photo back-up and virtual photo folders.

Tokens
9.4K
Snippets
7
Records
71
Agent score
84%

What's inside Sushitrain

  1. Choose between photo back-up and photo folder mechanisms

    main

    Synctrain offers two distinct ways to synchronize iOS photos via Syncthing. Choose based on your storage and synchronization needs:

    1. Photo back-up: Automatically and periodically exports photos, videos, and live photos from your library into a specific location within a synchronized folder.

      • Best for: Backing up all media types (including videos/live photos) to other devices.
      • Trade-off: Exported files take up physical disk space on the device until they are synchronized and potentially deselected via selective synchronization.

    2. Photo folder: Exposes specific photo albums as if they were a virtual (send-only) folder. Photos are read directly from the system photo library on demand and are never actually exported to the device's disk.

      • Best for: Exposing albums without consuming local disk space.
      • Trade-off: Does not support videos or live photos; deletions in the iOS library are immediately propagated as deletions in the synchronized folder.

  2. How the Go and Swift layers interact

    main

    Sushitrain uses a hybrid architecture: a Go-based core (SushitrainCore) embedded into a Swift/SwiftUI frontend via gomobile bindings.

    Key Concepts

    • Gomobile Bridge: Since gomobile does not support complex Go types like slices (arrays), the SushitrainCore framework abstracts these into simpler types, such as a 'list of strings', which the Swift side can iterate over (e.g., for lists of devices or folders).
    • The Client Struct: The primary interface on the Go side is the Client struct. The Swift side creates one instance of this struct and assigns a Swift class as a delegate to receive events.
    • State Management: Events received from the Go delegate update the state on the Swift side, which triggers SwiftUI UI updates.
    • Threading: State changes must occur on the main thread. While calling Go from Swift typically happens on the main thread (in response to UI actions), you must explicitly defer work to the main thread when receiving callbacks from the Go side to avoid crashes or undefined behavior.
    • In-Process Node: The app runs a full Syncthing node in-process, supporting standard features like discovery and ignore files, though not all features are exposed to the UI for iOS/macOS usability.

  3. Configure the thumbnail disk cache

    main

    Sushitrain offers two ways to manage the thumbnail disk cache:

    • System Default: By default, thumbnails are stored in a system-defined cache folder. Note that the system may purge this cache if disk space becomes low.
    • Synced Folder: Users can choose a synced folder as the thumbnail cache. This allows thumbnails to be shared across multiple devices. For example, you can generate thumbnails on a device with fast access to large media libraries (like a desktop) and then browse those thumbnails on other devices without re-generating them.

  4. Use selective synchronisation to save storage

    main

    Sushitrain implements selective synchronisation using Syncthing's .stignore mechanism. This allows you to see the global index of files without downloading them all to your mobile device.

    How it works

    When a folder is in 'selective' mode, an .stignore file is created with a * pattern, causing all files to be ignored for automatic synchronisation. To sync specific files or directories, 'exception patterns' are added to the top of the .stignore file.

    Pattern Rules for the UI

    For the Sushitrain UI to correctly manage selective synchronisation, the .stignore file must follow these rules:

    1. The last pattern in the file must be *.
    2. It may start with zero or more patterns starting with (?d) that do not contain / or ** (used for global ignores like .DS_Store).
    3. All other patterns must start with !/ and must not contain * (these indicate the paths you have 'selected').
    4. Empty lines are permitted.

    Selecting Files and Directories

    • To select a file: Add !/path/to/file.txt before the * pattern.
    • To select a directory (and all its contents): Add !/path/to/folder before the * pattern. This makes all current and future files in that folder 'implicitly selected'.

    Handling Extraneous and Renamed Files

    • Extraneous files: Files that exist locally but are not 'selected' in the UI. The app will prompt you to either 'select' them or delete them.
    • Local Renames: If you rename a selected file locally, Syncthing treats it as a removal and a new creation. The new file will be ignored by the * pattern until you manually select it.
    • Remote Renames: If a file is renamed on another node, the selected file will disappear from your device and the new filename will not appear until you select it.
    !/some/file/I/want.txt
*

  5. Access files via on-demand downloads

    main

    Sushitrain allows you to access files from the 'global index' without full synchronisation. When you select a file to view, the app downloads it from a peer that has the necessary blocks available.

    Media Streaming

    To enable streaming playback of media files, Sushitrain runs an HTTP server on localhost on a random port. When a video file is selected for streaming, the app uses a special URL format that supports HTTP range requests. This ensures that only the necessary blocks are fetched from the remote Syncthing peer, rather than downloading the entire file first.

    Streaming URL Format: /file?folder=X&path=Y&signature=Z

  6. Understand Folder Server HTTPS and security

    main

    The folder server operates over HTTPS at https://localhost to ensure that secure-context-only web APIs are available.

    Because the server uses a self-signed certificate that is regenerated every time a FolderServer is instantiated, the internal app web view is specifically configured to automatically accept the fingerprint of these certificates.

  7. How thumbnails are generated in Sushitrain

    main

    Sushitrain uses three distinct methods to provide thumbnails depending on the file's availability:

    1. Local Files: The app uses the system's QuickLook-framework to generate thumbnails. This works for media files and other types like PDFs. If a true thumbnail cannot be produced, QuickLook generates a representative icon image.
    2. On-demand Remote Files: For files not present locally, the app fetches only the necessary parts of the file to generate a thumbnail. For video, it uses AVAssetImageGenerator to avoid downloading the entire file.
    3. Disk Cache: Generated thumbnails can be stored in a disk cache. By default, only thumbnails for remote files are cached, but users can manually trigger thumbnail generation for local files using the 'generate thumbnails' button in the app.

  8. Understand Folder Server Authentication

    main

    To prevent unauthorized local applications from accessing the served website, the folder server implements cookie-based authentication.

    • Mechanism: The server requires a specific cookie to be present in requests.
    • Cookie Lifecycle: Both the cookie name and its value are determined by the FolderServer (Go implementation) and are generated as a random string each time a FolderServer is instantiated.
    • Client Handling: The web view integrated within the app is pre-configured to automatically include this required cookie with its requests.

  9. Format Swift code with swift-format

    main

    To maintain code style, use swift-format. You can lint the project or apply formatting changes directly using the following commands:

    brew install swift-format
swift format lint -r .
swift format -r -i .

  10. Load custom configurations and device identities

    main

    By default, Sushitrain creates a device identity and configuration in the Library/Application Support directory. Synced folders are stored in the Documents directory, making them visible in the iOS 'Files' app.

    To override default settings (useful for testing), you can place specific files in the Documents folder:

    • Custom Configuration: Place a config.xml file in the Documents folder. A warning will appear on startup indicating the custom config is loaded.
    • Custom Device Identity: Place both cert.pem and key.pem in the Documents folder.

  11. Translate strings using localize.mjs

    main

    The project provides a localize.mjs script that uses ChatGPT to fill in missing translations.

    1. Set your OPENAI_KEY environment variable.
    2. Run the script using Node.js.
    3. After the script completes, open Localizable.xcstrings in Xcode and save it once to ensure the JSON formatting matches Xcode's expectations and avoids whitespace pollution in git diffs.

    Note: You must edit the script to define which locales are supported.

    OPENAI_KEY=sk-xxxx node ./localize.mjs

  12. Build the Sushitrain iOS/macOS app in Xcode

    main

    Once the Go framework is built, follow these steps in Xcode to build the application:

    1. Entitlements: Remove the com.apple.developer.device-information.user-assigned-device-name entitlement from the iOS project configuration. Without this, the app will use localhost as the peer name. You can manually change the device name in the app settings later.
    2. Signing: Set up code signing by updating the 'team ID' in project settings to a valid developer ID. Xcode will then automatically provision certificates. You may need to change the bundle ID to a unique value.
    3. Build: Press Cmd-B to build the project.

    Note: Xcode invokes the Go Makefile automatically during the build process, but building the framework manually first is recommended for easier debugging of compiler output.