Face2Face

repository·main·Indexed 41 days ago

https://github.com/socaity/face2face

A generative AI tool for swapping faces in images and videos. Version 1.5.4 provides a high-level API for one-shot face swapping, face recognition-based swapping using embeddings, and face restoration using models like gfpgan_1.4 and the gpen family. It can be used as a local Python library, via the socaity SDK, or as a job-based asynchronous web service.

Tokens
3.9K
Snippets
13
Records
18
Agent score
100%

What's inside Face2Face

  1. Face swapping with face embeddings

    main

    Face embeddings allow you to name a face and reuse it later without providing the source image again.

    1. Create and save an embedding: Use add_face with save=True. If the source image has multiple people, providing a list of names will create embeddings for each person from left to right.
    2. Perform the swap: Use the swap method, passing the saved name to the faces argument. This works for both images and videos.
    # create a face embedding and save it to disk. 
    embedding = f2f.add_face("my_new_face", "path/to/my_portrait_image.jpg", save=True)
    
    # Swap all faces in the image or video with the face(s) in the face embedding
    swapped = f2f.swap(media="path/to/my_img_or_video.jpg", faces="my_new_face")
  2. How the Face2Face Web Service works

    main

    The web service is built using APIPod and operates on a job-based asynchronous pattern:

    1. Submit a Request: When you send a request to an endpoint (like swap, add_face, or enhance_face), the service creates a job.
    2. Receive Job ID: The service immediately returns a JSON payload containing a job_id.
    3. Poll for Status: You must poll the status endpoint (/status/{job_id}) to check the progress and retrieve the final result once the job is complete.
  3. Use plain web requests to interact with Face2Face

    main

    You can interact with the service using any HTTP client (e.g., requests, httpx, curl). The service provides endpoints for swap, add_face, and enhance_face functionality.

    1. Submit a request

    First, load your image as bytes, then send a POST request to the desired endpoint. For example, to use add-face:

    import requests
    
    # load images from disc
    with open("myimage.jpg", "rb") as image:
        my_image = image.read()
    
    submit = requests.post(
        "http://localhost:8020/add-face",
        files={"image": ("face.jpg", my_image, "image/jpeg")},
        data={"face_name": "biden"},
    )
    my_job = submit.json()

    2. Poll for results and parse the response

    Use the job_id from the initial response to poll the /status/{job_id} endpoint. Once the job is finished, the response will contain the result (a media payload) and metrics.

    import requests
    
    # check status of job
    status = requests.get(f"http://localhost:8020/status/{my_job['job_id']}").json()
    
    # Access execution metrics
    metrics = status.get("metrics", {})
    print("execution_time_s:", metrics.get("execution_time_s"))
    
    # Retrieve the result image
    result = status.get("result", {})
    image_url = result.get("content")
    
    if image_url:
        image_bytes = requests.get(image_url, timeout=60).content
        with open("swapped_img.png", "wb") as f:
            f.write(image_bytes)
    import requests
    
    # 1. Submit request
    with open("myimage.jpg", "rb") as image:
        my_image = image.read()
    
    submit = requests.post(
        "http://localhost:8020/add-face",
        files={"image": ("face.jpg", my_image, "image/jpeg")},
        data={"face_name": "biden"},
    )
    my_job = submit.json()
    
    # 2. Poll status
    status = requests.get(f"http://localhost:8020/status/{my_job['job_id']}").json()
    
    # 3. Parse result
    result = status.get("result", {})
    image_url = result.get("content")
    if image_url:
        image_bytes = requests.get(image_url, timeout=60).content
        with open("swapped_img.png", "wb") as f:
            f.write(image_bytes)
  4. Install Face2Face via pip

    main

    You can install Face2Face locally using pip. Choose between the lightweight version for script-based inference or the full package if you intend to run it as a web service.

    Requirements:

    • Python 3.7 or higher
    • Minimum 5GB RAM (running out of memory may cause a 'Killed' message)
    • Recommended: GPU with at least 8GB VRAM for fast inference (CPU is supported but slower).
    • For VideoFile support in the web service, you must also install ffmpeg.

    Models are downloaded automatically upon first use.

    # face2face without service (only for inference from script)
    pip install socaity-face2face 
    
    # full package with service
    pip install socaity-face2face[full]
  5. Use the Socaity SDK (No GPU required)

    main

    If you want to avoid local installation and GPU requirements, you can use the socaity SDK which runs on socaity.ai. This allows you to perform face swaps via a hosted service.

    from socaity import Face2Face
    swapped_img = Face2Face().swap_img_to_img("path/to/src.jpg", "path/to/target.jpg").get_result()
  6. Start the Face2Face Web Service

    main

    To start the Face2Face web service, run the server module from your command line. Note that the first time you start the server, it will automatically download the required models, which may take some time.

    python -m face2face.server
  7. Configure Face2Face Web Service via Environment Variables

    main

    You can customize the service behavior using the following environment variables:

    ENV VariableDescription
    MODELS_DIRPath to the folder where models (e.g., inswapper, GPEN) are stored.
    EMBEDDINGS_DIRPath to the folder where face embeddings are stored for reuse.
    ALLOW_EMBEDDING_SAVE_ON_SERVERIf True, embeddings can be saved on the server. Set to False in multi-user scenarios or to avoid memory overflow.
    APIPOD_BACKENDSet to runpod to use the Runpod backend (requires checking the DOCKERFILE for Runpod deployment).
  8. Face swapping with defined swap pairs (Recognition)

    main

    If you have multiple embeddings saved, you can use swap_pairs to perform specific swaps within a single media file. By passing a dictionary to the faces argument, the system recognizes the faces in the media and swaps them according to your mapping.

    Example: {"person_in_video": "target_embedding_name"}

    # This function will swap the faces of trump with hagrid and biden with ron.
    # assumption the faces [trump, hagrid, biden, ron] are already added with f2f.add_face
    swapped = f2f.swap_pairs(
      media="path/to/my_img_or_video.mp4", 
      faces={
          "trump": "hagrid",
          "biden": "ron"
      }
    )
  9. Initialize the Face2Face class for local inference

    main

    To use Face2Face directly in your scripts, import the Face2Face class. You can specify a device_id to target a specific GPU, which is useful for running multiple processes across different GPUs.

    from face2face import Face2Face
    f2f = Face2Face(device_id=0) 
  10. Swap faces from one image to another

    main

    Use the swap_img_to_img method to perform a one-shot face swap between a source image (the face you want to use) and a target image (the image to be modified).

    swapped_img = f2f.swap_img_to_img("path/to/src.jpg", "path/to/target.jpg")
  11. Enhance face quality with face restoration models

    main

    Because the underlying swap model (inswapper) operates at low resolution, you can use face enhancement models to upscale and improve portrait quality.

    Available Models:

    • gfpgan_1.4
    • gpen family (e.g., gpen_bfr_512). The GPEN model can upscale up to 2048, providing higher quality at the cost of higher runtime.

    Usage:

    • During swapping: Pass enhance_face_model to the swap method. The model is automatically downloaded.
    • Standalone enhancement: Use enhance_faces for all faces in an image, or enhance_single_face for a specific detected face.
    # enhance during swap
    swapped_img = f2f.swap(media="path/to/my_img_or_video.mp4", enhance_face_model='gpen_bfr_512')
    
    # enhance all faces in an image
    enhanced_img = f2f.enhance_faces(image="path/to/my_img.jpg", enhance_face_model='gpen_bfr_512')
    
    # enhance a specific face
    target_img = cv2.imread("path/to/my_img.jpg")
    detected_face = f2f.detect_faces(target_img)[0]
    enhanced_img = f2f.enhance_single_face(target_img, detected_face, enhance_face_model='gpen_bfr_512')
  12. Iterative face swapping with generators

    main

    For processing large batches of images or video frames efficiently, use the generator-based methods. These allow you to yield images (e.g., from cv2.imread) and receive swapped images iteratively.

    • swap_to_face_generator: Swaps a specific face embedding onto a stream of target images.
    • swap_pairs_generator: Performs recognition-based pair swapping on a stream of target images.
    import cv2
    
    def my_image_generator():
      for i in range(100):
        yield cv2.imread(f"image_{i}.jpg")
    
    # for swapping to always the same face
    for swapped_img in f2f.swap_to_face_generator(faces="my_embedding", target_img_generator=my_image_generator()):
      cv2.imshow("swapped", swapped_img)
      cv2.waitKey(1)
    
    # including face recognition
    for swapped_img in f2f.swap_pairs_generator(target_img_generator=my_image_generator(), swap_pairs={"trump": "hagrid"}):
      cv2.imshow("swapped", swapped_img)
      cv2.waitKey(1)