DOCSEARCHbyAlgolia
Submit Docs

Computer Vision Recipes

repository·staging·Indexed 27 days ago

https://github.com/microsoft/computervision-recipes

A collection of Jupyter notebooks and utility functions for building, optimizing, and deploying computer vision systems using PyTorch and Azure. It includes implementations for action recognition (I3D, R(2+1)D), crowd counting (MCNN, OpenPose), document image cleanup (Confidence-based Sauvola Binarization), and image similarity workflows using RESNET50.

Tokens
13.8K
Snippets
30
Records
92
Agent score
95%

What's inside Computer Vision Recipes

  2. Overview of Image Classification Recipes

    staging
    This directory provides examples and best practices for building image classification systems, covering both single-object and multi-object classification. The recipes leverage PyTorch and the fast.ai library to enable high-accuracy training on custom datasets. The project includes notebooks for model design, evaluation, and operationalization, as well as integration with Azure for large-scale training and deployment.

  3. Overview of Object Detection recipes

    staging

    The scenarios/detection directory provides examples and best practices for building object detection systems. It is designed to allow users to bring their own datasets and train high-accuracy models using pre-set default parameters. The implementation is based on torchvision's Faster R-CNN.

    Hardware Recommendations:

    • It is highly recommended to run these samples on a machine with a GPU.
    • Supported OS: Linux or Windows (note: Windows may be ~20% slower).
    • Training on a CPU is technically possible but becomes prohibitively slow even with small datasets.

  4. Overview of Action Recognition implementations

    staging

    The contrib/action_recognition directory provides resources for building video-based action recognition systems, which classify actions from sequences of frames. The repository implements two state-of-the-art approaches:

    1. I3D: Supports RGB, Optical flow, and Two-Stream implementations.
    2. R(2+1)D: Recommended for its competitive accuracy, fast inference speed, and fewer package dependencies.

    Example notebooks are available for tasks such as scoring webcam footage or fine-tuning on the HMDB-51 dataset.

  5. Overview of R(2+1)D for Action Recognition

    staging

    The implementation in this repository utilizes the R(2+1)D model architecture. This model is designed for high accuracy and high inference speed by using video frames directly as input, avoiding the computationally expensive pre-computation of optical flow fields required by many other state-of-the-art methods.

    Key characteristics:

    • Accuracy: Derived from an extra pre-training step using large-scale datasets (e.g., IG-Kinetics).
    • Speed: Faster inference compared to 'Two-stream' or 'Flow' based models because it operates directly on video frames.

  7. Use Confidence-based Sauvola Binarization for document cleanup

    staging
    Confidence-based Sauvola binarization is an improved technique for segmenting foreground from background pixels in scanned document images. It improves upon the standard Sauvola binarization method by introducing a confidence score for background pixels, which helps preserve more foreground information in the resulting binarized images.

  8. Explore Computer Vision Scenarios

    staging

    The computervision-recipes repository provides implementations and examples for several core computer vision tasks. You can explore specific implementations for the following scenarios:

    • Classification: Predict the category of a given image (supervised learning).
    • Similarity: Compute similarity scores to identify images in a dataset most similar to a query image.
    • Detection: Detect the bounding box (rectangular coordinates) of objects within an image.
    • Keypoints: Detect specific points on an object (e.g., human pose estimation using body joints).
    • Segmentation: Assign a category to every individual pixel in an image.
    • Action Recognition: Classify actions from a sequence of video frames (e.g., 'reading' or 'drinking').
    • Tracking: Detect and track multiple objects across a video sequence over time.

  9. Keypoint Detection Overview

    staging
    This scenario provides examples and best practice guidelines for building keypoint detection systems, where keypoints are defined as points-of-interest on objects (e.g., bottle lids, body joints for human pose estimation). The implementation uses an extension of Mask R-CNN that simultaneously detects objects and their keypoints, based on Torchvision's Mask R-CNN architecture.

  10. Setup Multiple Virtual Machines with JupyterHub via VMSS

    staging
    This tool automates the deployment of an Azure Virtual Machine Scale Set (VMSS) consisting of multiple Data Science Virtual Machines (DSVMs). Each VM is pre-configured with the computervision-recipes repository, a Conda environment, and multiple JupyterHub user accounts. This is ideal for hands-on labs or sessions requiring multiple isolated VM environments that can be easily managed and shut down.

  11. Use FairMOT tracking implementations

    staging

    The fairmot folder contains code ported from the original FairMOT repository (https://github.com/ifzhang/FairMOT/src/lib). This code is intended to be used as-is, with minimal modifications made to ensure compatibility within the computervision-recipes environment.

    Note that the following modifications have been applied to the original FairMOT source:

    • Import statements have been updated (e.g., import utils changed to from . import utils).
    • Input resolution values are no longer hard-coded in datasets/dataset/jde.py.
    • The logging level is set to WARNING.