DOCSEARCHbyAlgolia
Submit Docs

waifuc Documentation

repository·main·Indexed 18 days ago

https://github.com/deepghs/waifuc

An efficient train data collector for anime-style character datasets. waifuc automates crawling from booru-style sites and Pixiv, applying image processing pipelines—including filtering, cropping, tagging, and resizing—to export data in formats suitable for machine learning training like LoRA. It features a pipeline pattern consisting of Sources, Actions, and Exporters, and includes CCIP (Contrastive Character Image Pretraining) for character-specific dataset filtering and feature extraction.

Tokens
16.3K
Snippets
60
Records
74
Agent score
63%

What's inside waifuc

  1. Overview of waifuc capabilities

    main

    waifuc is a Python-based image toolbox designed for high-freedom DIY workflows. Its functionality is divided into three main categories:

    1. Data Sourcing: Quickly crawl image data from integrated high-quality image websites or load data from local directories.
    2. Image Processing & Filtering: A collection of independent, practical operations to process and filter image data. You can configure these into a custom pipeline.
    3. Data Export: Export processed data in various formats, including images only, metadata, or tagged data.

    Hardware Requirements: Unlike many image toolboxes, waifuc does not strictly depend on high-end GPU performance. It supports running on CPU and can function on low-spec hardware (e.g., 2-core, 6GB GPU environments), although using a GPU will increase efficiency.

  2. Overview of waifuc

    main
    waifuc is a module designed to parse and manage configuration file structures and their versions. It provides a framework for crawling images and videos, processing them (such as background preprocessing), and saving them to directories in specified formats. It includes features for pre-filtering images based on ratings (e.g., 'safe', 'r15', 'r18').

  3. Hardware requirements for Waifuc

    main

    Waifuc can operate normally in environments without a GPU. It is highly adaptable to low-configuration hardware and can run in free cloud environments like GitHub Actions or Hugging Face Spaces.

    Minimum tested configuration:

    • 2 CPU cores
    • 6GB RAM
    • No GPU required

    Waifuc is designed to run effectively on almost any modern desktop or laptop computer.

  4. What is CCIP (Contrastive Character Image Pretraining)

    main

    CCIP is a contrastive learning model designed for character feature extraction. Unlike CLIP, which aligns images and text, CCIP aligns images of anime characters. It is used to determine if two images contain the same character by calculating a visual dissimilarity value.

    The model consists of two components:

    1. Feature Extractor: Extracts feature vectors from character images.
    2. Feature Comparator: Performs similarity calculations on those vectors.

    Key Use Cases:

    • Character Similarity Determination: Comparing two images to see if they represent the same character.
    • Dataset Filtering: Cleaning noisy web data (e.g., from Danbooru) to extract high-quality, character-specific datasets.
    • Image Sorting: Using feature vectors with clustering algorithms (like OPTICS) to automatically categorize anime screenshots.
    • Quality Assessment (RecScore): Evaluating character LoRAs by measuring the similarity between generated images and the training dataset images.

  5. How CCIPAction works for dataset filtering

    main

    CCIPAction is a flagship feature in waifuc used to filter out irrelevant characters from noisy web datasets. It operates as a state machine to identify the 'key features' of a target character without requiring manual labeling of every image.

    State Machine Workflow

    1. Initialization Phase (INIT):

      • Used when no trusted images are provided.
      • The action extracts features from input images and stores them.
      • Once a user-defined quantity of images is reached, it transitions to the Stepping Phase.
      • Alternative: Initialization with Trusted Source Phase (INIT_WITH_SOURCE) is used if you provide known 'trusted' images of the character. It records these features and moves directly to the Inference Phase.

    2. Stepping Phase (APPROACH):

      • The action performs clustering on the collected feature vectors.
      • If one cluster dominates (e.g., reaches a user-set threshold like 70%), those features are identified as the 'key character features'.
      • The action then transitions to the Inference Phase.

    3. Inference Phase (INFER):

      • The action uses the identified key features to filter the remaining data source.
      • As it finds more matching images, it updates the key feature set, increasing filtering accuracy over time.

    Important Considerations

    • Data Distribution: CCIP works best when the target character is the majority in the data source and other characters are randomly distributed.
    • Limitations: It is a 0-shot model; it can tell you if two images are the same, but it cannot tell you the name of the character. It is sensitive to hairstyles but less so to skin or hair color.

  6. Combine multiple data sources using concatenation and union

    main

    waifuc allows you to merge multiple data sources using two primary operators:

    • Concatenation (+): Executes sources sequentially. For example, source_a + source_b crawls source_a first, then source_b.
    • Union (|): Merges sources by randomly picking images from the available sources until the total limit is reached. This is useful when you want a diverse dataset from multiple sites without knowing the exact count per site.

    You can also perform complex nested operations and apply transformations (via .attach()) to specific parts of the combined source.

    from waifuc.source.danbooru import DanbooruSource
from waifuc.source.zerochan import ZerochanSource
from waifuc.exporter import SaveExporter

s_db = DanbooruSource(keyword='amiya', limit=30)
s_zc = ZerochanSource(keyword='amiya', limit=30)

# Concatenation: 30 from Danbooru, then 30 from Zerochan
s_concat = s_db + s_zc
s_concat.attach(SaveExporter(path='/data/combined')).run()

# Union: Randomly pick 60 images total from both sources
s_union = s_db | s_zc
s_union.attach(SaveExporter(path='/data/union')).run()

# Complex nested operation
# 50 from Zerochan + (50 randomly picked from Danbooru and Pixiv)
s_complex = s_zc[:50] + (s_db | s_pixiv)[:50]

  7. How waifuc pipelines work: Source, Action, and Exporter

    main

    waifuc uses a modular pipeline architecture to automate the process of crawling, processing, and exporting character datasets. The pipeline consists of three main components:

    1. Source: Loads image data into the pipeline. Examples include DanbooruSource for crawling Danbooru or LocalSource for loading local directories.
    2. Action: Processes or filters images within the pipeline. Actions can be added, removed, or reordered to customize the workflow. Common actions include ModeConvertAction (format conversion), FaceCountAction (filtering by number of people), and TaggingAction (auto-tagging).
    3. Exporter: Saves the processed images to a directory in a specific format. For example, TextualInversionExporter saves images along with their corresponding .txt annotation files.

    Users can build custom workflows by calling the .attach() method on a Source object to chain multiple Action modules together.

  8. Understand and manage image metadata JSON files

    main

    When using SaveExporter, waifuc generates a .json file for every image. These files contain critical metadata such as:

    • Site-specific information (e.g., Danbooru tags, dimensions, ID, upload time)
    • Source URL information
    • Filename information
    • Tag information used for generating training datasets (like LoRA)

    If you do not want to save these JSON files, set the no_meta parameter to True in your exporter configuration.

    # Example: Saving images without metadata JSON files
# Set no_meta=True to skip JSON generation
exporter = SaveExporter(path='output_dir', no_meta=True)

  9. How to process images using Actions

    main

    In waifuc, you process image data (whether crawled or local) by using Action objects. You apply an Action to a Source by calling its .attach() method.

    Crucial Behavior: Every time .attach() is called, it returns a new Source instance. It does not modify the original Source. To chain multiple actions, you must capture the returned object from each step.

    Correct Pattern (Chaining):

    source = LocalSource('/data/raw').attach(Action1()).attach(Action2())

    Incorrect Pattern (Logical Error):

    source = LocalSource('/data/raw')
source.attach(Action1()) # This returns a new source, but it is discarded
source.attach(Action2()) # Action1 is never applied to the source used here
    # Correct way to chain actions
source = LocalSource('/data/raw').attach(Action1()).attach(Action2())

  10. How the waifuc Pipeline Works: Source, Action, and Exporter

    main

    The waifuc library operates on a modular pipeline architecture composed of three primary components:

    1. Data Source (Source): Responsible for loading data into the pipeline.
      • Examples: DanbooruSource (crawls Danbooru), LocalSource (loads local files).
    2. Data Processing (Action): Transforms or filters the data as it flows through the pipeline.
      • Examples: ModeConvertAction, NoMonochromeAction, ClassFilterAction, FilterSimilarAction, FaceCountAction, PersonSplitAction, CCIPAction, AlignMinSizeAction, TaggingAction, FirstNSelectAction, RandomFilenameAction.
    3. Data Export (Exporter): Saves the final processed data to a specific format and location.
      • Example: TextualInversionExporter (saves images and .txt labels).

    This structure allows you to easily swap sources (e.g., from Danbooru to Pixiv), add/remove processing steps, or change the output format.

  11. How waifuc works: Sources, Actions, and Exporters

    main

    waifuc follows a pipeline pattern to collect and process image datasets:

    1. Source: The starting point that provides data (e.g., DanbooruSource, PixivSearchSource, LocalSource, or GcharAutoSource).
    2. Action: A series of processing steps attached to the source using .attach(). Actions can filter images (e.g., FaceCountAction), transform them (e.g., ModeConvertAction, PaddingAlignAction), or tag them (e.g., TaggingAction).
    3. Exporter: The final step that saves the processed data to a destination (e.g., SaveExporter for local files or TextualInversionExporter for LoRA training formats) using .export().

    You can chain multiple actions together and even slice the source (e.g., [:10]) to limit the number of images processed.

  12. Use Metadata to control filenames and tags

    main

    Metadata in an ImageItem is used to drive downstream processing. Two key fields are:

    • filename: Controls the name used when saving the image. It supports relative paths (e.g., folder/image.png), allowing you to automatically generate nested directory structures during export.
    • tags: A mapping (dictionary) where keys are tag text and values are confidence scores. This is specifically used by the TextualInversionExporter to generate .txt files required for LoRA training.

    By providing these in your BaseDataSource or WebDataSource, you can automate complex dataset organization.