curl_cffi

curl_cffi is a Python binding for curl-impersonate via cffi. It is designed to bypass bot protection by impersonating browser TLS/JA3 and HTTP/2 fingerprints.

Key Features:

• Fingerprint Impersonation: Supports JA3/TLS and HTTP/2 fingerprints, including recent browsers and custom fingerprints.
• Protocol Support: Supports HTTP/2, HTTP/3 (including fingerprints and UDP proxy as of v0.15.0), and WebSockets.
• API Compatibility: Mimics the requests API, making it easy to transition from requests or httpx.
• Performance: Faster than requests/httpx, comparable to aiohttp/pycurl.
• Async Support: Supports asyncio with proxy rotation on each request.
• Ease of Use: Pre-compiled binaries are provided, so no local compilation is required for most users.

Fingerprinting is a technique used by WAFs (Web Application Firewalls) to identify if a request originates from a real browser or an automated script based on the specific combinations of extensions and cipher suites used during the handshake.

• TLS Fingerprinting: Uses the combination of extensions and cipher suites. The most common method is JA3.
• HTTP/2 Fingerprinting: Uses specific settings in the HTTP/2 connection. The most common method is the Akamai http2 fingerprint.
• HTTP/3 Fingerprinting: Implemented over QUIC. While less publicly exploited than TLS/HTTP2, it is increasingly used by strict WAF vendors. Users often report less detection when using HTTP/3.

curl_cffi provides impersonation for TLS and HTTP/2, and supports the HTTP/3 protocol.

Use curl-cffi when you require browser impersonation (TLS fingerprinting), HTTP/2 or HTTP/3 support, or a CLI designed for reliability in scripts and AI agent workflows.

Key differences:

• Browser Impersonation & HTTP/2/3: Supported by curl-cffi and curl-impersonate, but not by HTTPie or standard curl.
• Syntax: curl-cffi uses a readable key=value or key:=json syntax, whereas curl uses flags like -H and -d.
• Method Declaration: curl-cffi requires explicit subcommands (e.g., post, get), unlike HTTPie which may implicitly guess the method based on data presence.
• Batch Execution: curl-cffi includes a native run command for executing multiple requests from files, a feature not natively available in curl or HTTPie.

The TLS PSK extension is managed automatically by curl_cffi (version 0.11.0 and later, requiring libcurl 8.13.0 or higher).

• Automatic Behavior: The client automatically handles the extension and offers it on subsequent requests to the same site, mimicking a real browser.
• Manual Control: There is currently no option to manually enable or disable the PSK extension.
• Pretending to be a first-time visitor: If you need to prevent the PSK extension from being sent (to appear as a new visitor), you must either use an older version of curl_cffi or create a completely new session for each request.

Warning: Do not attempt to manually inject a PSK extension with a random value, as this is a high-signal indicator of non-browser traffic and will likely lead to being blocked.

Use the following commands to manage fingerprints for impersonate.pro:

• Download fingerprints: Use curl-cffi update to download the latest fingerprints to your local cache. A successful update will print the total number of fingerprints stored.
• List fingerprints: Use curl-cffi list to display both native and cached fingerprints in a table format.
• List fingerprints as JSON: Use curl-cffi list --json to output the fingerprint data in JSON format.

You can configure proxies using the proxy parameter for a single proxy, or the proxies dictionary for different protocols (e.g., http vs https). Additionally, curl_cffi respects the following environment variables:

• http_proxy
• https_proxy
• ws_proxy
• wss_proxy

Warning: Do not add the https:// prefix to an https proxy URL.

import curl_cffi

# Single proxy
curl_cffi.get(url, proxy="http://user:pass@example.com:3128")

# Multiple proxies (requests-style)
proxies = {
    "http": "http://localhost:3128",
    "https": "http://localhost:3128"
}
curl_cffi.get(url, proxies=proxies)

To install the latest unstable version directly from the GitHub repository, clone the repo, run the preprocess command, and install the local directory.

git clone https://github.com/lexiforest/curl_cffi/
cd curl_cffi
make preprocess
pip install .

To set cookies on a request via the CLI, use the + separator followed by the key=value pair.

# Set a single cookie
curl-cffi get https://httpbin.org/cookies +session=abc123

# Set multiple cookies
curl-cffi get https://httpbin.org/cookies +session=abc123 +theme=dark

Instead of using standard curl flags like -H (headers) and -d (data), curl-cffi uses a more readable syntax with request items:

• key=value for standard fields.
• key:=json for JSON data.
• Header:Value for custom headers.

Note that the HTTP method must always be specified as a subcommand (e.g., post, get, put, etc.).

When using PyInstaller, you may need to include hidden imports and collect all data for curl_cffi. Use the following command structure:

pyinstaller -F example.py --hidden-import=_cffi_backend --collect-all curl_cffi

Alternatively, you can manually specify paths and add the library data:

pyinstaller --noconfirm --onefile --console \
    --paths "C:/Users/Administrator/AppData/Local/Programs/Python/Python39" \
    --add-data "C:/Users/Administrator/AppData/Local/Programs/Python/Python39/Lib/site-packages/curl_cffi.libs/libcurl-cbb416caa1dd01638554eab3f38d682d.dll;." \
    --collect-data "curl_cffi" \
    "C:/Users/Administrator/Desktop/test_script.py"

To achieve maximum consistency and avoid performance penalties from process migration, you can pin the server and client to specific CPU cores.

# Linux: Use taskset
# Terminal 1
taskset -c 0 python ws_bench_1_server.py
# Terminal 2
taskset -c 1 python ws_bench_1_client.py

# Windows: Use start /affinity (hexadecimal mask)
# PowerShell/CMD 1
start /affinity 1 python ws_bench_1_server.py
# PowerShell/CMD 2
start /affinity 2 python ws_bench_1_server.py

To install a local editable version of curl_cffi on macOS, ensure you have the necessary dependencies installed via Homebrew and that the required directory permissions are set.

Note: The IMPERSONATE_BUILD_DIR must contain libcurl-impersonate.dylib and any required versioned symlinks. The directory is recorded as a runtime search path in the extension module.

# This is for using the libcurl-impersonate built by GitHub actions

sudo mkdir /Users/runner
sudo chmod 777 /Users/runner

# Dependencies

brew install libidn2 zstd

# Then install

pip install -e .[test]
pip install -e .[dev]