DOCSEARCHbyAlgolia
Submit Docs

EgoBlur

repository·main·Indexed 42 days ago

https://github.com/facebookresearch/egoblur

A face and license plate anonymization system designed to process VRS (Video Recording Studio) formatted sensor data, images, and videos. It provides Gen1 and Gen2 models to detect and blur sensitive information in time-stamped sensor streams, featuring Python CLI tools and C++ VRS utilities for Ubuntu 22.04.

Tokens
5.2K
Snippets
13
Records
17
Agent score
55%

What's inside EgoBlur

  1. How to choose between Gen1 and Gen2 models

    main

    EgoBlur provides different model generations based on the device used to capture the data. It is critical to use the generation that matches your data:

    • Aria Gen1 data: Use Gen1 models and the egoblur-gen1 CLI.
    • Aria Gen2 data: Use Gen2 models and the egoblur-gen2 CLI.

    Models must be downloaded separately from the EgoBlur website.

  3. Install NVIDIA Drivers and CUDA Toolkit

    main

    If drivers or CUDA are missing:

    1. Identify GPU: Use lspci | grep nvidia -i.
    2. Install Drivers: Use sudo ubuntu-drivers devices to find the recommended driver, then sudo apt install nvidia-driver-535 (or the version recommended).
    3. Install CUDA 12.1: Download the runfile from NVIDIA and execute it.
    4. Configure Environment: Add CUDA to your ~/.bashrc:
      export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
    5. Verify: Run nvidia-smi and nvcc --version.
    # Example: Adding CUDA to PATH
export PATH=/usr/local/cuda/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-12.1/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
source ~/.bashrc

  4. Install EgoBlur via PyPI

    main

    To install the EgoBlur Python command line tools, it is recommended to use a virtual environment.

    1. Create and activate a virtual environment:
    mkdir -p ~/venvs
python3 -m venv ~/venvs/egoblur
source ~/venvs/egoblur/bin/activate
pip install --upgrade pip
    1. Install the package:
    pip install egoblur
    pip install egoblur

  5. Install EgoBlur VRS Utilities on Ubuntu 22.04

    main

    This guide covers the installation for Ubuntu 22.04 using libtorch 2.1 and CUDA toolkit 12.1.

    Prerequisites:

    • NVIDIA drivers, CUDA, OpenCV, make, and gcc must be installed.
    • ninja-build is required for projectaria_tools.

    Steps:

    1. Download and unpack CMake (v3.28.0-rc4).
    2. Download and unpack libtorch (v2.1.0 with CUDA 12.1).
    3. Install VRS dependencies via apt (including libfmt-dev, libturbojpeg-dev, liblz4-dev, and libboost-*).
    4. Install ninja-build.
    5. Build Torchvision: Clone pytorch/vision (branch v0.16.0), build with CMake using your libtorch path and TORCH_CUDA_ARCH_LIST.
    6. Build EgoBlur: Clone the EgoBlur repository and build the ego_blur_vrs_mutation tool using CMake, pointing to your libtorch and TorchVision directories.
    # Example: Building ego_blur_vrs_mutation
cd ~/repos/EgoBlur/tools/vrs_mutation
rm -rf build && mkdir build && cd build
~/cmake/bin/cmake .. -DTorch_DIR=/home/$USER/libtorch/share/cmake/Torch -DTorchVision_DIR=~/repos/vision/cmake
make -j ego_blur_vrs_mutation

  6. Use the EgoBlur Gen1 CLI for image and video blurring

    main

    The demo_ego_blur_gen1.py script is a CLI tool used to detect and blur faces and license plates in images (PNG, JPEG) or videos (MP4) using Gen1 models. It supports both face and license plate detection, allowing you to specify model paths, confidence thresholds, and scaling factors for the blur area.

    To use the tool, you must provide at least one model path (face or license plate) and at least one input file (image or video).

    # Example: Blur faces in an image
python gen1/script/demo_ego_blur_gen1.py \
    --face_model_path /path/to/face_model.pt \
    --input_image_path input.jpg \
    --output_image_path output.jpg

# Example: Blur license plates in a video
python gen1/script/demo_ego_blur_gen1.py \
    --lp_model_path /path/to/lp_model.pt \
    --input_video_path input.mp4 \
    --output_video_path output.mp4 \
    --output_video_fps 30

  7. Use the EgoBlur Gen2 CLI for image and video blurring

    main

    The demo_ego_blur_gen2.py script is a CLI tool used to perform automated blurring of faces and license plates in images (PNG, JPEG) or videos (MP4) using EgoBlur Gen2 models.

    To use the tool, you must provide paths to the model files and either an input image or an input video. You can optionally specify camera names to use optimized default thresholds or adjust detection sensitivity via score thresholds and NMS IOU thresholds.

    # Example: Blur an image
python demo_ego_blur_gen2.py \
    --face_model_path /path/to/face_model.pth \
    --lp_model_path /path/to/lp_model.pth \
    --input_image_path input.jpg \
    --output_image_path output.jpg

# Example: Blur a video with specific camera thresholds
python demo_ego_blur_gen2.py \
    --camera_name camera-rgb \
    --face_model_path /path/to/face_model.pth \
    --lp_model_path /path/to/lp_model.pth \
    --input_video_path input.mp4 \
    --output_video_path output.mp4

  8. Use the VRS blur Gen2 CLI tool

    main

    The vrs_blur_gen2.py script provides an end-to-end tool for blurring faces and license plates in VRS camera streams using Gen2 EgoBlur models. It reads camera streams from an input VRS file, performs detection and blurring, and writes the results (along with non-camera streams like IMU or VIO copied verbatim) into a new output VRS file.

    Key features:

    • Supports batched inference for improved GPU throughput.
    • Automatically uses NVIDIA GPU encoding (hevc_nvenc) if available, otherwise falls back to CPU (libx265).
    • Handles multiple camera streams (RGB and SLAM) by grouping them for efficient batching.
    • Can be used in debug_encoding_only mode to test the VRS write path without running model inference.
    python gen2/script/vrs_blur_gen2.py \
    --input_vrs_path input.vrs \
    --output_vrs_path output.vrs \
    --face_model_path path/to/face_model.pt \
    --lp_model_path path/to/lp_model.pt

  9. Blur faces and license plates in Aria Gen2 VRS files

    main

    The egoblur-vrs-blur tool (available in version 2.0.1+) allows for end-to-end blurring directly within Aria Gen2 VRS files. It processes camera streams and writes the blurred results back into a single output VRS file, while copying non-camera streams (IMU, VIO, etc.) verbatim.

    Note: Running this on macOS is not recommended as models run on CPU only, resulting in slow inference. For best performance, use a Linux machine with a CUDA-capable GPU.

    egoblur-vrs-blur \
  --input_vrs_path /path/to/input.vrs \
  --output_vrs_path /path/to/output.vrs \
  --face_model_path ${EGOBLUR_MODELS}/ego_blur_face_gen2.jit \
  --lp_model_path ${EGOBLUR_MODELS}/ego_blur_lp_gen2.jit \
  --batch_size 5
    egoblur-vrs-blur \
  --input_vrs_path /path/to/input.vrs \
  --output_vrs_path /path/to/output.vrs \
  --face_model_path ${EGOBLUR_MODELS}/ego_blur_face_gen2.jit \
  --lp_model_path ${EGOBLUR_MODELS}/ego_blur_lp_gen2.jit \
  --batch_size 5

  10. Supported environments for EgoBlur

    main

    Python CLI Tools (Gen1 and Gen2)

    • Python version: 3.10-3.12
    • Platforms: Fedora 43, Ubuntu 20.04, Ubuntu 22.04, Ubuntu 24.04, MacOS-14, MacOS-15
    • CUDA toolkit (optional): 12.8

    Gen1 C++ VRS Utilities

    • Platform: Ubuntu 22.04
    • Dependencies: libtorch 2.1, CUDA toolkit 12.1

  11. Reference: CLI arguments for egoblur-gen1 and egoblur-gen2

    main

    The following arguments are used by the Python CLI tools. Note that Gen2 includes --camera_name and handles video FPS differently than Gen1.

    ArgumentDescription
    --camera_name (Gen2 only)Specify camera type for automatic threshold selection. Valid options: slam-front-left, slam-front-right, slam-side-left, slam-side-right, camera-rgb.
    --lp_model_pathAbsolute path to the EgoBlur license plate model file. (Required if not providing --face_model_path).
    --face_model_pathAbsolute path to the EgoBlur face model file. (Required if not providing --lp_model_path).
    --face_model_score_thresholdFace model score threshold (0.0 to 1.0) to filter low confidence detections.
    --lp_model_score_thresholdLicense plate model score threshold (0.0 to 1.0) to filter low confidence detections.
    --nms_iou_thresholdNMS IoU threshold (0.0 to 1.0) for overlapping boxes. Defaults to 0.3.
    --scale_factor_detectionsScale factor for blurring area (e.g., 1.1 scales UP by 10%, 0.9 scales DOWN by 10%).
    --input_image_pathAbsolute path to the input image.
    --output_image_pathAbsolute path for the blurred output image. (Required if --input_image_path is used).
    --input_video_pathAbsolute path to the input video.
    --output_video_pathAbsolute path for the blurred output video. (Required if --input_video_path is used).
    --output_video_fps (Gen1 only)FPS for the output video (positive integer). Defaults to 30.

  12. Use the ego_blur_vrs_mutation CLI

    main

    The ego_blur_vrs_mutation tool performs face and license plate anonymization on VRS files.

    Mandatory Arguments:

    • -i, --in: Absolute path to the input VRS file.
    • -o, --out: Absolute path for the output blurred VRS file.

    Optional Arguments:

    • -f, --faceModelPath: Path to the EgoBlur face model file. (Required if you want to blur faces).
    • -l, --licensePlateModelPath: Path to the EgoBlur license plate model file. (Required if you want to blur license plates).
    • --face-model-confidence-threshold: Score threshold for face detection (0.0 to 1.0, default: 0.1).
    • --license-plate-model-confidence-threshold: Score threshold for license plate detection (0.0 to 1.0, default: 0.1).
    • --scale-factor-detections: Scale the blurred region (e.g., 1.15 scales UP by 15%, 0.9 scales DOWN by 10%). Default: 1.15.
    • --nms-threshold: NMS IoU threshold for overlapping boxes (0.0 to 1.0, default: 0.3).
    • --use-gpu: Flag to enable GPU acceleration (highly recommended).
    # Sample command using all arguments
./ego_blur_vrs_mutation \
    --in your_vrs_file \
    --out your_output_vrs_file \
    -f ~/models/ego_blur_face.jit \
    --face-model-confidence-threshold 0.75 \
    -l ~/models/ego_blur_lp.jit \
    --license-plate-model-confidence-threshold 0.99 \
    --scale-factor-detections 1.15 \
    --nms-threshold 0.3 \
    --use-gpu