Detectron2
repository·main·Indexed 12 days ago
https://github.com/facebookresearch/detectron2A modular PyTorch-based library for state-of-the-art detection and segmentation algorithms developed by Facebook AI Research. It provides tools for training and deploying models like Faster R-CNN, Mask R-CNN, and Keypoint R-CNN, with support for datasets including COCO, LVIS, Cityscapes, and Pascal VOC. The library includes a Model Zoo, lazy instantiation for configurations, and export capabilities for TorchScript and ONNX.
What's inside Detectron2
- MViTv2 (Improved Multiscale Vision Transformers) provides detection configurations and models integrated into Detectron2. This implementation focuses on object detection and segmentation tasks. For image classification tasks using MViTv2, users should refer to the standalone MViTv2 repository.
Overview of Detectron2 example scripts
mainThe
tools/directory contains several scripts for common tasks in the Detectron2 workflow:train_net.py: An example training script for training built-in Detectron2 models. Refer toGETTING_STARTED.mdfor detailed usage.plain_train_net.py: A training script that implements a manual training loop instead of using theTrainerabstraction. This is intended for users who want more control ('hackers').benchmark.py: Benchmarks training, inference, or data loading speeds.analyze_model.py: Analyzes model FLOPs, parameters, and activations.visualize_json_results.py: Visualizes JSON results fromCOCOEvalutororLVISEvaluator.visualize_data.py: Visualizes ground truth annotations or augmented training data.
Overview of ViTDet in Detectron2
mainViTDet provides implementations and configurations within the Detectron2 framework for exploring plain Vision Transformer (ViT) backbones for object detection. It includes support for ViTDet, MViTv2, and Swin backbones, following the settings described in the ViTDet paper.Overview of TridentNet in Detectron2
mainTridentNet (Scale-Aware Trident Networks) is an object detection architecture designed to generate scale-specific feature maps with uniform representational power. It uses a parallel multi-branch architecture where branches share transformation parameters but utilize different receptive fields.
This repository implements TridentNet-Fast, a fast approximation version that provides significant performance improvements without increasing parameters or computational cost.
Access the Detectron2 Model Zoo
mainDetectron2 provides a large set of baseline results and pre-trained models available for download in the Model Zoo. You can use these models for inference or as starting points for fine-tuning on your own datasets.Use detectron2.utils.visualizer for image visualization
mainThedetectron2.utils.visualizermodule provides tools for visualizing model predictions (such as bounding boxes, segmentation masks, and keypoints) on images. It is the primary interface for creating visual representations of detection results.Use detectron2.utils.events for event tracking
mainThedetectron2.utils.eventsmodule is used for managing and tracking events during training or inference, often used for logging metrics or triggering specific behaviors at certain intervals.Use detectron2.utils.analysis for model evaluation and analysis
mainThedetectron2.utils.analysismodule contains tools for analyzing model performance, such as computing metrics or visualizing error patterns.Prepare Detectron2 models for deployment
mainThe
detectron2/exportdirectory provides tools to prepare trained Detectron2 models for deployment in production environments. Currently, the export functionality supports the following formats:- TorchScript: For deployment in environments that support PyTorch JIT.
- ONNX: For deployment across various hardware accelerators and runtimes (e.g., TensorRT, ONNX Runtime).
- Caffe2: (Deprecated)
For detailed usage instructions and tutorials, refer to the official deployment documentation.
Use detectron2.utils.colormap for color mapping
mainThedetectron2.utils.colormapmodule provides utilities for generating and managing colormaps, typically used to assign distinct colors to different object classes in segmentation masks.Use detectron2.utils.memory for memory management
mainThedetectron2.utils.memorymodule provides utilities to monitor or manage memory usage, which is critical when working with large models and high-resolution images on limited GPU memory.Explore DensePose Model Zoo and Baselines
mainThe DensePose Model Zoo provides various pre-trained models categorized by their architecture, training schedule, and feature sets. Users can choose models based on their specific needs for accuracy (AP), inference speed, or additional features like confidence estimation.
Model Categories:
- Legacy Models: Baselines trained using schedules from Güler et al, 2018.
- Improved Baselines (Original Fully Convolutional Head): Models using improved training schedules and Panoptic FPN heads.
- Improved Baselines (DeepLabV3 Head): Models incorporating the DeepLabV3 head for enhanced performance.
- Baselines with Confidence Estimation: Models that estimate confidence in regressed UV coordinates (using
WC1orWC2types). - Baselines with Mask Confidence Estimation: Models that estimate confidence for UV coordinates AND coarse/fine segmentation (using
WC1MorWC2Mtypes). - Bootstrapping Baselines: Master and student models trained via a bootstrapping pipeline, specifically evaluated on the DensePose Chimps dataset.