Hydrus Network Documentation

Hydrus Network is a file-management application designed for large media collections. Unlike traditional folder-based systems, it uses a tag-based browsing system similar to a booru. It supports various image, video, audio, and image project filetypes. Key features include:

• Tag-based browsing: Organize and find files using tags instead of directories.
• Metadata parsing: Download files and automatically parse tags/metadata from websites using user-made downloaders.
• Subscription system: Automatically repeat gallery searches every few days to keep collections updated.
• Privacy-focused: The program does not 'phone home' and allows for anonymous tag sharing via a public server.
• Media Playback: Supports audio and video playback via an mpv embed or a native Qt player.

Hydrus distinguishes between three types of file relationships:

• Potential duplicates: Pairs of files identified by the background search algorithm that might be duplicates. These are candidates for review.
• Duplicates: Pairs or groups of files that you have manually confirmed are different quality or resolution copies of the same source. Every duplicate group has one designated 'king' (the best quality file).
• Alternates: Pairs or groups of files that are related (e.g., costume changes, work-in-progress versions) but are not duplicates of the same source.

To protect users from historical data leaks and account cross-referencing, Hydrus repositories implement an anonymisation routine.

• Mechanism: The server assigns ownership of every file, mapping, or tag sibling/parent to a special null account. This removes all traces of your account's involvement with that content.
• Timing: Anonymisation occurs after a set delay. On the PTR, this is typically 90 days after upload, though local administrators can adjust this period. You can check the current period under review services.
• Administrative impact: Once content is anonymised, it is associated with the null account. Janitors will see the null account for old content. The null account is specially labelled, cannot be used by anyone, and cannot be banned or altered.

Hydrus uses several specialized downloader types depending on the source and desired behavior. A fresh installation contains no downloaders; you must add them manually.

• Gallery: Used for booru-style sites. You provide tags/queries, and it walks through gallery pages to queue files.
• Subscriptions: Automatic, repeating gallery jobs used to keep up-to-date with specific searches. Best practice: Use a Gallery downloader to get existing content first, then set up a Subscription for new content.
• Watcher: Used for monitoring imageboard threads. It checks URLs repeatedly for new posts. The API can send URLs to this.
• URL download: For one-off jobs using specific URLs (pasted or drag-and-dropped). It does not perform multi-page searching.
• Simple downloader: For advanced, one-off tasks with simple parsing rules (e.g., 'get all linked images from this page').

The Hydrus parsing system is composed of three hierarchical components used to transform HTML or JSON data into Hydrus metadata:

1. Formulae: The base layer. They take parsable data, perform a search/extraction, and return zero to $n$ strings.
2. Content Parsers: The middle layer. They take parsable data, apply a Formula to extract strings, and assign a specific Hydrus metadata 'type' (and optional modifiers) to those strings.
3. Page Parsers: The top layer. They take parsable data, execute multiple Content Parsers, and aggregate the resulting metadata into a structured format.

You can explore and manage these components via the Hydrus UI at: network -> downloader components -> manage parsers.

When syncing with a repository, the Hydrus client maintains anonymity through the following mechanisms:

• Anonymous Metadata: Metadata is downloaded in generic chunks. The server does not know which specific tags or files you are interested in.
• Local Indexing: The client downloads all metadata and performs all queries locally on your own hardware. The client never requests information about a specific file or tag from the server.
• File Downloads: In file repositories, thumbnails are downloaded automatically, but actual files are downloaded separately. The server does not log which specific files you download.
• Anonymity: Even if a server owner identifies your IP address, they only know that you are syncing; they cannot see the contents of your local files.

In the Hydrus network, a repository is a service that stores information (such as files or tag mappings) submitted by users.

Key architectural behavior:

• Clients do not send queries to repositories.
• Instead, clients perform queries against a local cache of the repository's data stored on the same computer.
• Users periodically synchronize with the repository to update this local cache.

Hydrus downloaders are modular components used to fetch files and metadata from the web. The system is designed to be flexible, allowing users to create and share custom downloaders using HTML or JSON parsing.

Note for developers: Creating custom downloaders requires proficiency in HTML or JSON. If you only want to add existing downloaders to your client, refer to the adding_new_downloaders.md guide instead.

Hydrus consists of two primary components:

• The Client: A file database that manages files within its own internal folder structure. It is designed for high-scale management of tens or hundreds of thousands of files using booru-style tagging.
• The Server: An advanced component that can be run to store tags or files for sharing between multiple users.

Note: The server component is considered advanced and is not recommended for exploration until you are familiar with the client's core functionality. By default, the client operates with complete privacy and does not 'phone home' or share data unless you take conscious action to connect to a public tag repository.