OVCamo: Open-Vocabulary Camouflaged Object Segmentation

Install the required Python packages using pip. Note that specific versions for torch and torchvision are specified within the requirements.txt file.

pip install -r requirements.txt

To evaluate the official pretrained model:

1. Download the model weights from this link.
2. Run the evaluation command:

python .\main.py --config .\configs\ovcoser.py --model-name OVCoser --evaluate --load-from model.pth

To reproduce the results reported in the paper:

1. Download the results archive here.
2. Unzip the archive into a directory (e.g., <path>/ovcoser-ovcamo-te).
3. Run the evaluation script pointing to that directory:

python .\evaluate.py --pre <path>/ovcoser-ovcamo-te

Use main.py to manage training and inference tasks. You must provide a configuration file and a model name.

To train the model:

python .\main.py --config .\configs\ovcoser.py --model-name OVCoser

To run inference (evaluation mode): Provide the --evaluate flag and the path to a local .pth checkpoint using --load-from.

python .\main.py --config .\configs\ovcoser.py --model-name OVCoser --evaluate --load-from <path_to_local_pth_file>

# Training
python .\main.py --config .\configs\ovcoser.py --model-name OVCoser

# Inference
python .\main.py --config .\configs\ovcoser.py --model-name OVCoser --evaluate --load-from <path_to_local_pth_file>

To evaluate a trained model on a test set, use the test(model, cfg, metric_names=...) function. It uses ImageTestDataset to load test data and recorder.OVCOSMetricer to compute performance.

Supported metrics (default): sm, wfm, mae, fm, em, iou.

If cfg.test.save_results is set to True, the function will save predicted masks as images in the directory specified by cfg.path.save, using the naming convention [predicted_class]mask_name.ext.

# Example conceptual usage
@torch.no_grad()
def test(model, cfg, metric_names=("sm", "wfm", "mae", "fm", "em", "iou")):
    # ... implementation details ...
    outputs = model(data=batch_images, gt_classes=gt_classes, class_names=dataset_classes)
    # ... calculates metrics and saves results ...

To train the model, use the train(model, cfg) function. This function initializes the ImageTrainDataset, sets up a DataLoader, constructs an optimizer and scheduler via the pipeline module, and executes the training loop. It handles gradient accumulation using cfg.train.grad_acc_step and logs progress including loss, learning rate, and GPU memory usage. At the end of training, the final model weights are saved to cfg.path.final_state_net with a -final suffix.

# Example conceptual usage within the main entrypoint
model = model_class()
model.to(cfg.device)

if not cfg.evaluate:
    train(model=model, cfg=cfg)

The main.py script can be executed from the command line to either train a model or evaluate an existing one.

Arguments:

• --config: (Required) Path to the configuration file.
• --root-info: Path to the YAML file containing dataset directory information (default: env/splitted_ovcamo.yaml).
• --model-name: The name of the model to instantiate from the methods module.
• --load-from: Path to a checkpoint to load weights from.
• --evaluate: Flag to switch from training mode to evaluation mode.
• --info: Additional info string.

Example:

python main.py --config configs/my_config.yaml --model-name MyModel --evaluate --load-from checkpoints/model.pth

The ImageTestDataset class is a PyTorch data.Dataset designed for testing. It requires a dataset_info (an mmengine.Config object) and input_hw (a list containing target height and width).

It returns a dictionary containing:

• data: {"image": tensor}
• info: {"text": class_name, "mask_path": mask_path, "group_name": "image"}

class ImageTestDataset(data.Dataset):
    def __init__(self, dataset_info: Config, input_hw: list):
        # ...

    def __getitem__(self, index):
        # Returns dict(data={"image": image}, info=dict(text=class_name, mask_path=mask_path, group_name="image"))

The ImageTrainDataset class is a PyTorch data.Dataset designed for training. It requires a dataset_info (an mmengine.Config object) and input_hw (a dictionary or list containing target height and width).

It loads images, masks, and depth maps, applies Albumentations transformations (HorizontalFlip, Rotate, RandomBrightnessContrast, HueSaturationValue), and returns a dictionary containing:

• data: {"image": tensor, "mask": tensor, "depth": tensor}
• info: {"text": class_name}

class ImageTrainDataset(data.Dataset):
    def __init__(self, dataset_info: Config, input_hw: dict):
        # ...

    def __getitem__(self, index):
        # Returns dict(data={"image": image, "mask": mask, "depth": depth}, info={"text": class_name})