curl-impersonate

repository·main·Indexed 25 days ago

https://github.com/lexiforest/curl-impersonate

A specialized build of curl designed to impersonate the TLS and HTTP/2 handshakes of major web browsers, including Chrome, Edge, Safari, and Firefox. It allows users to bypass TLS and HTTP/2 fingerprinting used by web services to identify non-browser HTTP clients. Key features include support for Encrypted Client Hello (ECH), ZSTD compression, HTTP/3 and QUIC fingerprint support, and various browser profiles provided via wrapper scripts.

Tokens
28.6K
Snippets
23
Records
137
Agent score
78%

What's inside curl-impersonate

  1. Overview of curl-impersonate

    main

    curl-impersonate is a specialized build of curl designed to impersonate the TLS and HTTP/2 handshakes of major web browsers, including Chrome, Edge, Safari, and Firefox. This allows users to bypass TLS and HTTP/2 fingerprinting used by web services to identify and restrict non-browser HTTP clients.

    Key features of this fork include:

    • Support for Encrypted Client Hello (ECH) and ZSTD compression.
    • Support for X25519Kyber768/X25519MLKEM curves.
    • Ability to change extension orders and toggle TLS grease.
    • Single binary support for Chrome, Safari, and Firefox.
    • HTTP/3 and QUIC fingerprint support.
    • Python bindings via curl_cffi and Node.js TypeScript bindings via impers.
    • Support for various architectures including Windows, Arm, and RISC-V.
  2. Modify HTTP headers for impersonation

    main
    The wrapper scripts set a default group of HTTP headers (like User-Agent and Accept-Encoding) to match a browser's first visit. To avoid duplicate headers or incorrect ordering, the safest way to change, reorder, or add headers is to modify the wrapper scripts directly.
  3. Use curl-impersonate via Docker images

    main

    Docker images based on Alpine Linux and Debian are available on Docker Hub. These images come with curl-impersonate pre-built, including the binary and all wrapper scripts.

    docker pull lexiforest/curl-impersonate:2.0.0
    docker run --rm lexiforest/curl-impersonate:2.0.0 curl_chrome145 https://www.example.com
  4. Build curl-impersonate on OpenBSD

    main

    Building on OpenBSD requires specific configuration flags: -DPREFER_GCC=ON and -DENABLE_AVX512=ON. You must also apply the OpenBSD patchfile before configuring.

    doas pkg_add -U gcc pkgconf cmake ninja curl autoconf automake libtool gmake gperf go
    
    # Apply OpenBSD patch
    patch -p1 < ./patches/OpenBSD.patch
    
    # Configure with required flags
    cmake -B build -S . -DPREFER_GCC=ON -DENABLE_AVX512=ON
    cmake --build build --parallel 4
    
    # Install
    doas make install
  5. Use libcurl-impersonate in PHP on Linux

    main

    To use libcurl-impersonate in PHP on Linux, you must first patch the library to change its SONAME, then load it at runtime using LD_PRELOAD and the CURL_IMPERSONATE environment variable.

    To verify the setup, run the command below. A successful installation will show [ssl_version] => BoringSSL in the output.

  6. Understand TLS and HTTP/2 fingerprinting

    main

    Web services often use TLS fingerprinting and HTTP/2 fingerprinting to identify the client accessing them.

    • TLS Fingerprinting: Occurs during the TLS handshake. The Client Hello message produced by most HTTP libraries differs significantly from a real browser.
    • HTTP/2 Fingerprinting: Occurs during the HTTP/2 handshake when settings are exchanged. Standard libraries use settings that differ from real browsers.

    curl-impersonate mitigates this by patching curl to use BoringSSL (used by Chrome and Safari) and modifying TLS extensions, SSL options, and HTTP/2 connection settings so the network signature is identical to a real browser.

  7. Build curl-impersonate on FreeBSD

    main

    FreeBSD is built natively. The build process currently disables libidn2. To build for local development from macOS, use the provided helper script to run a FreeBSD VM via QEMU.

    # Native FreeBSD build
    pkg install -y pkgconf cmake ninja curl autoconf automake libtool gmake gperf go
    
    cmake_args="-G Ninja -DCMAKE_INSTALL_PREFIX=$PWD/freebsd-install"
    cmake_args="$cmake_args -DUSE_LIBIDN2=OFF"
    
    gmake configure BUILD_DIR=build-freebsd CMAKE_CONFIGURE_ARGS="$cmake_args"
    gmake build BUILD_DIR=build-freebsd CMAKE_CONFIGURE_ARGS="$cmake_args"
    gmake install-strip BUILD_DIR=build-freebsd CMAKE_CONFIGURE_ARGS="$cmake_args"
    
    # macOS helper for FreeBSD VM development
    scripts/freebsd-vm-macos.sh setup
    scripts/freebsd-vm-macos.sh start
    scripts/freebsd-vm-macos.sh build
    scripts/freebsd-vm-macos.sh fetch-artifacts
    scripts/freebsd-vm-macos.sh stop
  8. Install curl-impersonate

    main

    You can install curl-impersonate using one of the following methods:

    1. Prebuilt Binaries: Download the ready-to-use binaries from the GitHub releases page.
    2. Docker Images: Use the official images from Docker Hub:
      • lexiforest/curl-impersonate (Debian-based)
      • lexiforest/curl-impersonate:alpine (Alpine-based)
    3. Build from Source: Follow the instructions in INSTALL.md or docs/install.rst to compile the project yourself.