DOCSEARCHbyAlgolia
Submit Docs

pytest-httpserver

repository·master·Indexed 42 days ago

https://github.com/csernazs/pytest-httpserver

A fake HTTP server for pytest designed for testing HTTP clients in Python. It allows developers to mock server behavior by pre-defining expected requests and responses on localhost. It provides a simple API via the `httpserver` fixture, a flexible API using `respond_with_handler` for custom logic, and a `BlockingHTTPServer` for synchronous testing. The library is compatible with pytest-xdist and uses werkzeug for its Request and Response objects.

Tokens
17.3K
Snippets
57
Records
82
Agent score
53%

What's inside pytest-httpserver

  1. Overview of pytest-httpserver

    master

    pytest-httpserver is a Python package designed to start a real HTTP server for testing purposes.

    Key characteristics:

    • Programmatic Configuration: Configure request handlers directly in your code without external configuration files.
    • Threaded Execution: The server runs in a separate thread, allowing your test code to run concurrently.
    • Client Agnostic: Since it uses a real TCP port, you can use any HTTP client library (e.g., requests, httpx, urllib).
    • Flexible Integration: While optimized for pytest via fixtures, it can be used with other frameworks via its Context API.

  2. Core API components of pytest_httpserver

    master

    The pytest_httpserver package provides several core classes for managing mock HTTP servers in pytest.

    • HTTPServer: The primary server class.
    • RequestHandler: Used to define how the server responds to incoming requests.
    • RequestMatcher: Used to define criteria for matching incoming requests.
    • RequestMatcherKwargs: Configuration for request matching.
    • BakedHTTPServer: A specialized server implementation.
    • BlockingHTTPServer: A server implementation that blocks execution.
    • BlockingRequestHandler: A request handler designed for blocking server modes.
    • WaitingSettings: Configuration for how the test waits for server operations.
    • HeaderValueMatcher: A specialized matcher for validating HTTP header values.
    • URIPattern: A utility for matching URI patterns.

    Errors that can be encountered include HTTPServerError and NoHandlerError.

  3. Understand the server lifecycle and crosstalk prevention

    master
    The HTTP server is started when the first test uses the httpserver fixture and remains running for the duration of the entire test session to avoid the overhead of restarting. To prevent state leakage (crosstalk) between tests, the server's state is automatically cleaned up between test runs.

  4. Configure expected responses

    master

    After defining a request expectation, you must define how the server responds. You can fine-tune the response using:

    • HTTP status: The status code to return.
    • headers: Response headers.
    • data: The response body.

    Supported response types include:

    • Arbitrary data (string or bytearray).
    • JSON (a Python dict converted to JSON in-place).
    • A werkzeug.Response object.
    • A custom function.
    # Example: responding with JSON
httpserver.expect_request("/foobar").respond_with_json({"foo": "bar"})

  5. Understand server lifecycle and fixture scope

    master

    Server Lifecycle

    The HTTP server starts when the first test using the httpserver fixture is run. It continues to run until the end of the pytest session. To prevent cross-talk between tests, the library performs a cleanup between tests (clearing registered handlers, emptying logs, etc.) rather than restarting the server. This avoids the ~1 second delay required by the underlying werkzeug server to shut down.

    Fixture Scope

    Because the server runs once per session, it is recommended to treat the httpserver fixture as session scoped. However, the library handles the necessary per-test cleanup automatically.

    Parallel Testing

    The library is compatible with pytest-xdist. When running tests in parallel, distinct test sessions will be created, each with its own independent HTTP server instance.

  6. Configure expected requests

    master

    When defining an expectation using expect_request(uri, ...), you can fine-tune the match criteria. The URI is mandatory. Other available parameters include:

    • method: The HTTP method (e.g., GET, POST).
    • headers: Expected HTTP headers.
    • query_string: Expected query parameters.
    • data: The raw HTTP body of the request.
    • json: The expected JSON body of the request.
    # Example of a POST request with an expected JSON body
httpserver.expect_request(
    "/foobar", 
    method="POST", 
    json={"id": 12, "name": "foo"}
).respond_with_json({"foo": "bar"})

  7. Match Authorization headers

    master
    By default, pytest-httpserver includes a built-in parser for the Authorization header. This ensures that even if the parameters in a Digest authentication header (like username, realm, nonce, etc.) arrive in a different order than expected, the matcher will still succeed.

  8. Configure request matching behavior (permanent, oneshot, ordered)

    master

    You can control how many times an expected request is served:

    • permanent: The request will always be served as long as it matches. You can make multiple identical requests.
    • oneshot: The request is served only once. Subsequent identical requests will not match this expectation.
    • ordered: Similar to oneshot, but the requests must be matched in the exact order they were defined in your test setup.

  9. Configure IPv4 vs IPv6 binding

    master

    The server's address family is determined by the host string provided. Since it uses Werkzeug, the binding behavior follows these rules:

    • IPv4: Provide a host string without a colon (e.g., localhost or 127.0.0.1).
    • IPv6: Provide a host string containing a colon (e.g., ::1).

    Note on Dual-Stack systems: If you use localhost, clients might attempt to connect via IPv6 (::1) before falling back to IPv4 (127.0.0.1). If you are testing TLS errors or specific connection failures, this fallback can cause confusing error messages (like Connection reset instead of a TLS error). To prevent this, fix the host to 127.0.0.1 so the client uses the direct IPv4 address.

  11. Understand the pytest-httpserver API design

    master

    The library follows a tiered API approach to balance simplicity with flexibility:

    1. Simple API: Uses a flat structure with a handful of functions that accept built-in Python types (bytes, strings, ints, etc.). This allows you to define an expected request and its response in a single line.
    2. Flexible API: For complex logic, use the respond_with_handler method. This accepts a callable (e.g., a function) that receives a Request object and returns a Response object, allowing you to implement custom logic.
    3. Higher-level API: Built on top of low-level protocol elements (status, headers, etc.) to provide specialized matchers like HeaderValueMatcher and authorization support.

    Transparency Note: The library is transparent. The Request and Response objects used are the actual objects defined by werkzeug. There is no wrapping, so you can use werkzeug's API directly.

  12. Use a custom request handler for dynamic responses

    master

    If you need a response that is not static (e.g., it depends on the request content), use respond_with_handler(). You pass a function that accepts a request object and returns a Response object.

    Handling errors in handlers: Because the server runs in a separate thread, assert statements inside a handler will not fail the test immediately. Instead, they will cause an HTTP 500 error. To catch these, use:

    • httpserver.check_assertions(): Checks for failed assertions.
    • httpserver.check_handler_errors(): Checks for unhandled exceptions.
    • httpserver.check(): Checks for both assertions and errors.
    from werkzeug.wrappers import Response

def my_handler(request):
    # Logic to generate dynamic response
    return Response("Dynamic Content", status=200)

httpserver.expect_request("/dynamic").respond_with_handler(my_handler)

# In your test, verify no errors occurred in the handler thread
httpserver.check()