IMDL-BenCo: Image Manipulation Detection & Localization Benchmark

The model_zoo directory contains the implementations of all models supported by IMDLBenCo. Each model is organized into its own sub-folder. To use a specific model, navigate to its corresponding directory to find implementation details and weight download instructions.

For example, the APSC-Net model (CVPR 2024) is implemented as a pure-PyTorch inference adapter that is compatible with official checkpoints.

- [APSC-Net](./apsc_net/README.md) (CVPR 2024): pure-PyTorch inference
  adapter with official-checkpoint compatibility.

The imdlbenco version of the APSC-Net adapter includes several modifications to the original upstream implementation to ensure compatibility with the imdlbenco ecosystem:

• Runtime Decoupling: MMCV/MMSEG runtime components have been replaced with plain PyTorch modules.
• Interface Standardization: The forward interface has been adapted to match the IMDLBenCo output contract.
• Checkpoint Management: Added checkpoint-prefix normalization and checkpoint conversion utilities.
• Workflow Integration: Added IMDLBenCo-specific evaluation and experimental fine-tuning launchers.

Trufor's training process consists of three stages: noiseprint++, localization, and detection. When configuring the training, use the phase parameter to select between localization and detection training:

• Phase 2 (Localization Training): Focuses on the localization stage. The det_resume_ckpt parameter must be left empty.
• Phase 3 (Detection Training): Focuses on the detection stage. This phase requires the det_resume_ckpt parameter to be set to the path of the weights obtained from the localization training stage to allow the model to continue training the detection head.

When performing inference or evaluation with the APSC-Net adapter, follow these protocols:

Input Requirements:

• Resize: Images should be resized to 512 x 512.
• Precision: Use FP32 for audited numerical precision.
• Batch Size: Typically 1 for standard evaluation.

Thresholding:

• The generic IMDLBenCo runner uses a fixed mask threshold of 0.5.
• The official APSC-Net implementation can derive a dynamic threshold (range 0.3 to 0.7) based on the classification score.
• Warning: Because the IMDLBenCo runner uses a fixed threshold, results may differ from the official paper's reported metrics if the paper uses the dynamic thresholding method. Results should be labeled accordingly.

To use SparseViT within the IMDL-BenCo framework, you can download recommended checkpoints from Hugging Face. The recommended checkpoint for training is uniformer_base_in1k.pth.

Download link: https://huggingface.co/Sense-X/uniformer_image/tree/main

# Download uniformer_base_in1k.pth from:
# https://huggingface.co/Sense-X/uniformer_image/tree/main

# You can then load this file directly to train SparseViT.

To use the Span models within the IMDL-BenCo model zoo, you must manually download the pretrained weights file IMTFEv4.pt. These weights are hosted by the ManTraNet-pytorch repository.

Required File: IMTFEv4.pt Source: ManTraNet-pytorch GitHub

IMDL-BenCo is a Python library available on PyPI. You can install it using pip to access the comprehensive benchmark and modular codebase for Image Manipulation Detection & Localization (IMDL).

pip install imdlbenco

Mesorch requires two distinct sets of pretrained weights to function.

1. ConvNeXt weights: These are handled automatically. Set the conv_pretrain parameter to True to trigger an automatic download.
2. SegFormer weights: These must be downloaded manually. You need to obtain the MiT-B3 pretrained model from the official SegFormer repository: https://github.com/NVlabs/SegFormer.

# Example configuration logic
# Set conv_pretrain to True to enable automatic ConvNeXt weight downloading
conv_pretrain = True

# Note: SegFormer MiT-B3 weights must be downloaded manually from 
# https://github.com/NVlabs/SegFormer

To use MantraNet within the IMDL-BenCo framework, you must manually provide the pretrained weights.

1. Download the MantraNetv4.pt weights from the ManTraNet-pytorch repository.
2. Place the MantraNetv4.pt file directly into the IMDLBenCo/model_zoo/mantra_net/ directory.
3. Run the modify_weights.py script located in that directory to convert the weights into the format required by IMDL-BenCo.

# After placing MantraNetv4.pt in the directory:
python modify_weights.py

Use benco init base or benco init to scaffold a new project in your current working directory. These commands generate the necessary scripts for:

• Training
• Testing
• Robustness testing
• A template model file (MyModel.py) demonstrating the recommended development pattern.

Note that the generated Python and shell scripts are intended to be modified to suit your specific research or development needs.