Kemono Downloader

Kemono Downloader is a PyQt5-based desktop application designed for downloading and organizing content from various sites. Key capabilities include:

• Downloading Engine: Supports multi-threading, multi-part downloading (chunked), and session management (pause/resume).
• Filtering: Filter by content type (images, video, audio, archives), keyword skipping, minimum file size, and specific character/series names.
• Organization: Automatic subfolder creation, file renaming (by title, date, ID, etc.), and filename cleaning.
• Specialized Modes:
  • Manga Mode: Chronological sorting.
  • Favorites Mode: Direct download from account favorites.
  • Export/Extraction: Extract external links (Mega/Google Drive) or save post text/comments as PDF, DOCX, or TXT.
• Advanced Tools: Cookie support for logged-in sessions, duplicate detection, and optional .webp image compression.

To resume a previous download session, use the restore_data parameter in start_session. The manager uses this data to avoid re-downloading posts that were already processed.

restore_data should be a dictionary containing:

• processed_post_ids (list of str): A list of IDs that have already been successfully handled.
• all_posts_data (list of dict): The full list of post data objects from the previous session. This allows the manager to calculate remaining work and update the overall_progress immediately.

When restore_data is provided, the manager merges the processed_post_ids from the session and the creator's profile to ensure no duplicates are processed.

# Example restore data structure
restore_data = {
    'processed_post_ids': ['post_id_1', 'post_id_2'],
    'all_posts_data': [
        {'id': 'post_id_1', 'title': 'Post 1', ...},
        {'id': 'post_id_2', 'title': 'Post 2', ...},
        {'id': 'post_id_3', 'title': 'Post 3', ...}
    ]
}

manager.start_session(config, restore_data=restore_data)

The DownloadManager can use proxies for network requests. When providing a config dictionary to start_session, you can specify proxy settings in two ways:

1. Pre-built proxies: If the config already contains a 'proxies' key with a dictionary mapping http and https to a correctly schemed proxy string (e.g., http://user:pass@host:port), the manager will use it directly.
2. Manual configuration: If 'proxies' is missing, you can provide individual fields. The manager will construct the proxy string using the following keys:
   • proxy_enabled (bool): Must be True to use proxies.
   • proxy_host (str): The proxy server address.
   • proxy_port (int/str): The proxy port.
   • proxy_type (str): One of HTTP, SOCKS5, or SOCKS4. Defaults to HTTP.
   • proxy_username (str, optional): Username for authentication.
   • proxy_password (str, optional): Password for authentication.

Note: For SOCKS5, the manager uses the socks5h scheme to ensure DNS resolution happens through the proxy.

# Example manual proxy configuration
config = {
    'proxy_enabled': True,
    'proxy_host': '127.0.0.1',
    'proxy_port': '8080',
    'proxy_type': 'SOCKS5',
    'proxy_username': 'myuser',
    'proxy_password': 'mypassword',
    'api_url': 'https://example.com/api'
}

The utility provides two advanced matching functions to identify which character/folder a file or post belongs to by comparing against a list of name objects (containing name and aliases).

match_folders_from_title

Matches folder names within a post title. It first cleans the title by removing common metadata patterns (like [OC], NSFW, HD, etc.) before searching for aliases. It returns a list of matched primary names and the count of unique non-overlapping matches found.

match_folders_from_filename_enhanced

Matches folder names within a filename. It uses a tiered matching strategy (Strict -> Relaxed -> CJK substring). If multiple candidates are found in the filename, it uses the post_title as a tie-breaker by checking which candidates also appear in the title.

# names_to_match is a list of dicts: [{'name': 'Primary', 'aliases': ['alias1', 'alias2']}]
# unwanted_keywords is a list of strings to ignore

# Example usage for filename matching
candidates, count = match_folders_from_filename_enhanced(
    filename="file_with_alias.jpg",
    names_to_match=names_to_match,
    unwanted_keywords=unwanted_keywords,
    post_title="The actual post title"
)

The fetch_erome_data function identifies and extracts media files from an Erome album URL. It handles Cloudflare bypass attempts and returns a structured folder name and a list of media file dictionaries.

Parameters:

• url (str): The Erome album URL (e.g., https://www.erome.com/a/albumID).
• logger (function): A callback function used for logging progress and errors.
• proxies (dict, optional): A dictionary of proxies to use for the session.

Returns:

• A tuple containing (album_folder_name, list_of_file_dicts).
• Returns (None, []) if the URL is invalid or extraction fails.

File Dictionary Structure: Each dictionary in the returned list contains:

• url: The direct media file URL.
• filename: A sanitized filename following the pattern {album_id}_{sanitized_title}_{index}.{extension}.
• headers: A dictionary containing necessary request headers (e.g., {'Referer': page_url}).

from src.core.erome_client import fetch_erome_data

def my_logger(msg):
    print(f"[LOG] {msg}")

url = "https://www.erome.com/a/example_id"
folder_name, files = fetch_erome_data(url, my_logger)

if folder_name:
    print(f"Folder: {folder_name}")
    for file in files:
        print(f"File: {file['filename']} -> {file['url']}")

The format_custom_date(date_str, format_string) function converts various date string formats into a standardized format defined by the user.

Supported Input Formats:

• YYYY-MM-DD, DD-MM-YYYY, YYYY/MM/DD, DD/MM/YYYY, MM-DD-YYYY, MM/DD/YYYY, YYYY.MM.DD, DD.MM.YYYY.
• ISO format strings.
• If the input contains NoDate, it returns "NoDate".

Custom Format Tokens:

• YYYY $\rightarrow$ 4-digit year
• MM $\rightarrow$ 2-digit month
• DD $\rightarrow$ 2-digit day

If the date cannot be parsed, the function returns the original date_str (cleaned of time components).

from src.utils.file_utils import format_custom_date

# Standard conversion
print(format_custom_date("2023-05-20", "DD-MM-YYYY")) # Output: "20-05-2023"

# Using custom tokens
print(format_custom_date("2023-05-20", "YYYY_MM_DD")) # Output: "2023_05_20"

# Handling unparseable dates
print(format_custom_date("UnknownDate", "YYYY"))      # Output: "UnknownDate"