DOCSEARCHbyAlgolia
Submit Docs

FaceForensics++

repository·master·Indexed 25 days ago

https://github.com/ondyari/faceforensics

A forensics dataset and research tool for detecting manipulated facial images. It includes tools for classification, segmentation, and video manipulation detection using scripts like detect_from_video.py. The repository provides access to various manipulation methods including Deepfakes, FaceSwap, FaceShifter, and NeuralTextures, along with scripts for downloading datasets and generating models.

Tokens
6.2K
Snippets
16
Records
52
Agent score
80%

What's inside faceforensics

  1. Overview of FaceShifter Manipulation in FaceForensics++

    master
    FaceShifter manipulation involves videos that have been manipulated by the original authors of the FaceShifter project. These manipulated videos are included in the FaceForensics++ dataset. For detailed information regarding the creation process of these manipulations, users should refer to the official FaceShifter project page or the associated research paper.

  3. Overview of FaceForensics++ dataset contents

    master

    FaceForensics++ is a forensics dataset designed for detecting manipulated facial images. It consists of 1000 original video sequences manipulated using four automated methods:

    • Deepfakes
    • Face2Face
    • FaceSwap
    • NeuralTextures
    • FaceShifter (added in later updates; high-fidelity identity-preserving face swapping)

    Key Features:

    • Data Source: 977 YouTube videos containing trackable, mostly frontal faces without occlusions.
    • Usage: Includes binary masks, making it suitable for image/video classification and segmentation tasks.
    • Augmentation: Provides 1000 Deepfakes models to generate and augment new data.
    • Additional Data: Includes the Google & JigSaw Deep Fake Detection Dataset (over 3000 manipulated videos).

  4. Understand NeuralTextures Manipulation

    master

    NeuralTextures videos in this dataset are manipulated using a face model from Face2Face to track and render UV masks. These masks are processed through an encoder-decoder architecture optimized via Neural Textures.

    Note that because the network was trained using a generative adversarial loss, the reenactment of face motions might not always be perfectly precise, though the process is optimized for high visual quality.

  7. Install dependencies for Faceswap

    master

    Before running the setup script, ensure the following dependencies are installed on your system:

    • Python: Version >= 3.2.
    • Nvidia GPU Users: Install CUDA and CUDNN. If you are not building TensorFlow yourself, ensure you install no higher than CUDA 9.0 and CUDNN 7.0.x.
    • dlib (Face Recognition): Requires specific system applications to compile:
      • Windows: Visual Studio 2015, CMake v3.8.2
      • Linux: build-essential, cmake
      • macOS: xquartz

  8. Run face manipulation detection from video files

    master

    Use the detect_from_video.py script within the classification folder to run detection on a single video or a folder containing .mp4 or .avi files.

    Note: The provided models are trained on the FaceForensics++ dataset and may not be fine-tuned for general compressed videos. You can download the trained models from this link.

  9. Install requirements for dataset processing

    master

    To use the extraction and compression scripts, ensure the following environment is set up:

    • OS: Ubuntu 16.04 or 18.04 (tested)
    • Python: Python 3
    • Python Packages:
      • tqdm (pip install tqdm)
      • opencv-python (pip install opencv-python)
    • System Tools: ffmpeg built with h264 support.

  10. Set up the DeepFakes environment

    master

    To set up a fresh environment on an Ubuntu machine (e.g., a new AWS instance), you can run the provided setup.sh script. This script installs the specific CUDA, cuDNN, and Python versions used in the FaceForensics++ experiments.

    Requirements for setup.sh:

    • OS: Ubuntu 16.04 (ideally without a prior Anaconda installation).
    • Hardware: An NVIDIA GPU with at least 8GB of memory (tested on GeForce 1080ti and K80).

    Environment Versions:

    • Python 3.5/3.6
    • Cuda 9.0.176-1
    • Cudnn 7.0.5.15-1
    • nccl 2_2.1.4-1
    • tensorflow_gpu-1.10.1

    Post-setup step: After running the setup, you must download the face alignment cache and merge it with the faceswap-master folder.

    bash setup.sh

  12. Run Faceswap using Docker (GPU accelerated)

    master

    For a pre-configured environment, you can use Docker. To use an Nvidia GPU from within the container, you must use nvidia-docker.

    1. Build the image

    docker build -t deepfakes-gpu -f Dockerfile.gpu .

    2. Run the container

    Option A: Without GUI (Command Line only) Use this if you only need the command line tools.

    docker run -p 8888:8888 \
    --hostname faceswap-gpu --name faceswap-gpu \
    -v /opt/faceswap:/srv \
    faceswap-gpu

    Option B: With GUI (X11 forwarding) To enable the GUI, you must allow local access to the X11 server and map the display/audio/video groups.

    xhost +local:

nvidia-docker run -p 8888:8888 \
    --hostname faceswap-gpu --name faceswap-gpu \
    -v /opt/faceswap:/srv \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e DISPLAY=unix$DISPLAY \
    -e AUDIO_GID=`getent group audio | cut -d: -f3` \
    -e VIDEO_GID=`getent group video | cut -d: -f3` \
    -e GID=`id -g` \
    -e UID=`id -u` \
    faceswap-gpu

    3. Interact with the project

    Open a new terminal and execute the GUI tool inside the running container:

    docker exec faceswap-gpu python /srv/tools.py gui
    docker build -t deepfakes-gpu -f Dockerfile.gpu .

# Example for GUI run
nvidia-docker run -p 8888:8888 \
    --hostname faceswap-gpu --name faceswap-gpu \
    -v /opt/faceswap:/srv \
    -v /tmp/.X11-unix:/tmp/.X11-unix \
    -e DISPLAY=unix$DISPLAY \
    -e AUDIO_GID=`getent group audio | cut -d: -f3` \
    -e VIDEO_GID=`getent group video | cut -d: -f3` \
    -e GID=`id -g` \
    -e UID=`id -u` \
    faceswap-gpu