httpmorph

You can use the browser parameter in httpmorph.Session to mimic specific browser profiles. This includes User-Agent, TLS cipher suites, HTTP/2 settings, and JA3/JA4 fingerprint matching.

Supported browser values:

• 'chrome': Defaults to Chrome 143 profile.
• 'chrome[version]': Specific Chrome version (127-143 supported, e.g., 'chrome127').
• 'random': Selects a random browser profile.

Use the os parameter to simulate different operating systems, which affects the User-Agent string:

• 'macos' (default)
• 'windows'
• 'linux'

import httpmorph

# Mimic Chrome on Windows
session = httpmorph.Session(browser='chrome', os='windows')

# Use a specific Chrome version
session = httpmorph.Session(browser='chrome143')

# Random browser selection
session = httpmorph.Session(browser='random')

Connections are automatically pooled and reused when using httpmorph.Client() or httpmorph.Session(). For multiple requests to the same host, always use a Session to benefit from connection reuse and improved performance.

# Using Session for connection reuse
with httpmorph.Session() as session:
    for i in range(100):
        response = session.get(f'https://example.com/page{i}')

The simplest way to install httpmorph is using pip. Pre-built wheels are available for the following platforms:

• Windows (x86_64)
• macOS (Intel and Apple Silicon)
• Linux (x86_64, ARM64)

pip install httpmorph

To build the HTML documentation from the source files, you must first install the necessary Python dependencies and then use the make command within the docs directory. The resulting HTML files will be located in build/html/.

# Install dependencies
pip install -r requirements.txt

# Build HTML documentation
cd docs
make html

You can perform simple HTTP requests using top-level functions in httpmorph. Supported methods include get, post, put, delete, head, patch, and options.

import httpmorph

# Simple GET request
response = httpmorph.get('https://httpbin.org/get')
print(response.status_code)
print(response.text)

# POST with JSON data
response = httpmorph.post(
    'https://httpbin.org/post',
    json={'name': 'value'}
)
print(response.json())

# POST with form data
response = httpmorph.post(
    'https://httpbin.org/post',
    data={'field': 'value'}
)

Files can be uploaded using the files parameter in post requests. You can upload single files, multiple files, or specify custom filenames and content types.

import httpmorph

# Single file
files = {'file': open('report.pdf', 'rb')}
response = httpmorph.post('https://httpbin.org/post', files=files)

# Multiple files
files = {
    'file1': open('report.pdf', 'rb'),
    'file2': open('data.csv', 'rb')
}
response = httpmorph.post('https://httpbin.org/post', files=files)

# With filename and content type
files = {'file': ('report.pdf', open('report.pdf', 'rb'), 'application/pdf')}
response = httpmorph.post('https://httpbin.org/post', files=files)

If you need to build from source, follow these steps to clone the repository, set up vendor dependencies (like BoringSSL), build the Python extensions, and install in development mode. Note that the initial build may take 5-10 minutes due to BoringSSL compilation.

git clone https://github.com/arman-bd/httpmorph.git
cd httpmorph

# Build vendor dependencies (BoringSSL, nghttp2)
./scripts/setup_vendors.sh

# Build Python extensions
python setup.py build_ext --inplace

# Install in development mode
pip install -e ".[dev]"

The response object contains metadata about the request and performance metrics. You can inspect the URL, status code, and headers, as well as timing breakdowns in microseconds (_us).

To convert microsecond timings to milliseconds, divide the values by 1000.

response = httpmorph.get('https://httpbin.org/get')

# Request details
print('URL:', response.url)
print('Status:', response.status_code)
print('Headers:', response.headers)

# Timing breakdown (converted to ms)
print('Connect:', response.connect_time_us / 1000, 'ms')
print('TLS:', response.tls_time_us / 1000, 'ms')
print('Total:', response.total_time_us / 1000, 'ms')

Use response.raise_for_status() to trigger exceptions for unsuccessful HTTP status codes. Common exceptions include httpmorph.Timeout, httpmorph.ConnectionError, and httpmorph.HTTPError.

When implementing retries, it is recommended to use exponential backoff to handle transient network issues.

import time
import httpmorph

def fetch_with_retry(url, max_retries=3):
    for attempt in range(max_retries):
        try:
            response = httpmorph.get(url, timeout=10)
            response.raise_for_status()
            return response
        except (httpmorph.Timeout, httpmorph.ConnectionError):
            if attempt < max_retries - 1:
                time.sleep(2 ** attempt)  # Exponential backoff
            else:
                raise

You can manually set and retrieve cookies using an httpmorph.Session instance. Assign a dictionary of cookie names and values to session.cookies to include them in subsequent requests. After a request, session.cookies will contain the cookies returned by the server.

Note: httpmorph provides basic cookie support and does not handle advanced cookie attributes such as Domain, Path, or Expires.

session = httpmorph.Session()

# Set cookies manually
session.cookies = {'session_id': 'abc123'}

# Make request with cookies
response = session.get('https://example.com')

# Get cookies from response
print(session.cookies)

Depending on your operating system, you must install specific build tools before compiling from source.

# macOS
brew install cmake ninja libnghttp2

# Ubuntu/Debian
sudo apt-get install cmake ninja-build libssl-dev pkg-config \ 
                     autoconf automake libtool libnghttp2-dev

# Fedora/RHEL
sudo dnf install cmake ninja-build openssl-devel pkg-config \ 
                    autoconf automake libtool libnghttp2-devel

# Windows
choco install cmake golang nasm visualstudio2022buildtools -y

Install the httpmorph library using pip. Note that the installation process builds BoringSSL from source and requires libnghttp2 for HTTP/2 support.

pip install httpmorph