HTTPX2 Documentation

repository·main·Indexed 20 days ago

https://github.com/pydantic/httpx2

A next-generation, fully featured HTTP client for Python maintained by Pydantic. It provides sync and async APIs supporting HTTP/1.1 and HTTP/2, with a broadly requests-compatible interface. Features include connection pooling, type safety, and support for Basic, Digest, and NetRC authentication. The project also includes httpcore2, a minimal low-level HTTP client designed as a reusable networking foundation for higher-level libraries.

Tokens
66.4K
Snippets
229
Records
284
Agent score
72%

What's inside httpx2

  1. What is HTTPCore?

    main

    HTTPCore is a minimal, low-level HTTP client designed to do one thing: send HTTP requests. It provides a foundation for higher-level libraries rather than providing high-level abstractions.

    Key Features:

    • Sending HTTP requests.
    • Thread-safe and task-safe connection pooling.
    • HTTP(S) proxy and SOCKS proxy support.
    • Support for HTTP/1.1 and HTTP/2.
    • Both synchronous and asynchronous interfaces.
    • Async backend support for asyncio and trio.
  2. HTTPX2 Features Overview

    main

    HTTPX2 is a next-generation HTTP client for Python with the following key capabilities:

    • Protocol Support: Both HTTP/1.1 and HTTP/2.
    • API Styles: Standard synchronous interface and full async support.
    • Compatibility: Broadly requests-compatible API.
    • Application Integration: Ability to make requests directly to WSGI or ASGI applications.
    • Type Safety: Fully type-annotated.
    • Standard Features: Includes connection pooling, keep-alive, cookie persistence, SSL verification, authentication (Basic/Digest), multipart file uploads, and proxy support.
  3. What is httpcore and when to use it?

    main

    HTTP Core is a minimal, low-level HTTP client designed to do one thing: send HTTP requests.

    Key Features:

    • Thread-safe / task-safe connection pooling.
    • HTTP(S) proxy & SOCKS proxy support.
    • Supports HTTP/1.1 and HTTP/2.
    • Provides both sync and async interfaces (supporting asyncio and trio).

    What it does NOT do: It does not provide high-level abstractions. It does not handle:

    • Redirects
    • Multipart uploads
    • Authentication header building
    • Transparent HTTP caching
    • URL parsing
    • Session cookie handling
    • Content/charset decoding or JSON handling

    When to use it:

    • You are building a low-level tool like a proxy service.
    • You are building a higher-level library (like httpx) and need a reusable networking foundation.

    When NOT to use it:

    • For most application development, you should use a higher-level client library like httpx instead.
  4. Map requests to proxies using mounts

    main

    HTTPX2 uses the mounts argument for proxying and transport routing instead of the proxies dictionary used in requests. When using httpx2.Client(mounts={...}), use full URL schemes (e.g., http:// or https://).

    Note that httpx2.Client.request(...) does not accept a mounts parameter; configuration must be done at the client level.

    # Example of mapping schemes to transports
    client = httpx2.Client(mounts={"http://": ..., "https://": ...})
  5. Use event hooks to implement client-wide logic

    main

    HTTPX provides event hooks that are triggered automatically during the request/response lifecycle. These are ideal for implementing cross-cutting concerns like logging, monitoring, tracing, or automatic error handling.

    There are two types of hooks:

    1. request: Called after a request is fully prepared but before it is sent to the network. It receives the request instance.
    2. response: Called after the response is fetched from the network but before it is returned to the caller. It receives the response instance.

    Important Constraints:

    • Hooks must be provided as a list of callables.
    • You can register multiple hooks for each event type.
    • Async Support: If using httpx2.AsyncClient, the registered hooks MUST be async functions.
    def log_request(request):
        print(f"Request event hook: {request.method} {request.url} - Waiting for response")
    
    def log_response(response):
        request = response.request
        print(f"Response event hook: {request.method} {request.url} - Status {response.status_code}")
    
    client = httpx2.Client(event_hooks={'request': [log_request], 'response': [log_response]})
  6. Use `httpx2.Client` for efficient requests

    main

    For any production use case beyond simple prototyping, use a Client instance instead of the top-level API. A Client implements HTTP connection pooling, allowing it to reuse underlying TCP connections for multiple requests to the same host. This reduces latency (no handshaking), lowers CPU usage, and decreases network congestion.

    httpx2.Client() is the equivalent of requests.Session() in the requests library.

    import httpx2
    
    # Recommended: Use as a context manager to ensure connection cleanup
    with httpx2.Client() as client:
        r = client.get('https://example.com')
    
    # Alternative: Explicitly close the connection pool
    client = httpx2.Client()
    try:
        r = client.get('https://example.com')
    finally:
        client.close()
  7. Fine-tune timeout configurations with httpx2.Timeout

    main

    For granular control, use the httpx2.Timeout object. This allows you to specify different durations for the four distinct stages of an HTTP request lifecycle:

    • connect: Maximum time to wait until a socket connection to the host is established. Raises ConnectTimeout.
    • read: Maximum duration to wait for a chunk of data to be received (e.g., response body). Raises ReadTimeout.
    • write: Maximum duration to wait for a chunk of data to be sent (e.g., request body). Raises WriteTimeout.
    • pool: Maximum duration to wait for acquiring a connection from the connection pool. Raises PoolTimeout.

    You can pass a single float to httpx2.Timeout(timeout) to set all four to that value, or use keyword arguments to override specific ones.

    # A client with a 60s timeout for connecting, and a 10s timeout elsewhere.
    timeout = httpx2.Timeout(10.0, connect=60.0)
    client = httpx2.Client(timeout=timeout)
    
    response = client.get('http://example.com/')
  8. How extensions work in httpcore

    main

    The httpcore request/response API uses a simple core model (status code, headers, and stream). To handle non-trivial or protocol-specific features without bloating the core API, httpcore uses 'extensions'.

    Extensions are implemented as a dict of optional additional information. They can be passed in the extensions argument of a request or accessed via the .extensions attribute on a response object.

    # Request with extensions
    r = httpcore.request(
        "GET",
        "https://www.example.com",
        extensions={"timeout": {"connect": 5.0}}
    )
    
    # Accessing response extensions
    print(r.extensions["http_version"])
  9. Disable environment variable configuration in HTTPX2

    main

    By default, HTTPX2 uses environment variables for configuration (such as proxies and SSL certificates). To ignore these environment variables and use only the explicit configuration provided in your code, set trust_env=False.

    You can disable environment variable lookup in two ways:

    1. When initializing a client: httpx2.Client(trust_env=False).
    2. When using the top-level API: httpx2.get("<url>", trust_env=False).
    import httpx2
    
    # Using the top-level API with trust_env=False to ignore environment variables
    httpx2.get('http://example.com', trust_env=False)
    
    # Using a Client with trust_env=False
    with httpx2.Client(trust_env=False) as client:
        client.get('http://example.com')
  10. Understanding Forwarding vs Tunnelling proxy mechanisms

    main

    When using a proxy, the request is handled in one of two ways:

    • Forwarding: The proxy server makes the actual request to the destination server on your behalf and returns the response to you.
    • Tunnelling (HTTP Tunnel): The proxy establishes a TCP connection to the destination server. The client then uses this connection to send its own request and receive the response. This is the mechanism used to access HTTPS websites through an HTTP proxy, as it allows the client to perform a TLS handshake directly with the destination server over the established TCP tunnel.
  11. How network backends work in httpcore

    main

    A network backend is the API layer where httpcore interacts with the network. It handles low-level operations like opening TCP connections, socket stream operations (reading, writing, closing), and SSL/TLS upgrades.

    By default, httpcore automatically selects an appropriate backend, but you can explicitly provide one when initializing a ConnectionPool or AsyncConnectionPool using the network_backend argument. This allows you to switch between synchronous, asynchronous, mock, or custom implementations depending on your runtime context.

    import httpcore
    
    # Explicitly selecting a synchronous backend
    network_backend = httpcore.SyncBackend()
    with httpcore.ConnectionPool(network_backend=network_backend) as http:
        response = http.request('GET', 'https://www.example.com')
  12. Use Async support in httpcore2

    main

    For asynchronous programming, use the Async prefixed versions of the core components:

    • httpcore.AsyncConnectionPool: An asynchronous connection pool.
    • httpcore.AsyncHTTPConnection: The base async connection class.
    • httpcore.AsyncHTTP11Connection: Async support for HTTP/1.1.
    • httpcore.AsyncHTTP2Connection: Async support for HTTP/2.