aiohttp

repository·master·Indexed 12 days ago

https://github.com/aio-libs/aiohttp

An asynchronous HTTP client/server framework for Python built on the asyncio ecosystem. It provides tools for building high-performance web servers, performing asynchronous HTTP requests, and managing connection pooling via TCPConnector, UnixConnector, and NamedPipeConnector. Features include a robust client exception hierarchy (e.g., ClientResponseError, ClientConnectionError), cookie management with CookieJar, multipart data handling via FormData, and request tracing with TraceConfig.

Tokens
124.1K
Snippets
371
Records
542
Agent score
95%

What's inside aiohttp

  1. Overview of aiohttp components and APIs

    master

    aiohttp is an asyncio-based HTTP client/server framework for Python. It is divided into three main functional areas:

    1. Server (aiohttp.web)

    Provides an HTTP/1.1 server including routing, middleware, WebSocket support, static-file serving, and a Gunicorn worker. Key APIs:

    • aiohttp.web.Application
    • web.RouteTableDef
    • web.run_app
    • web.AppRunner
    • web.WebSocketResponse
    • web.FileResponse

    2. Client (aiohttp.ClientSession)

    Provides an HTTP/1.1 client including connection pooling, TLS, proxy support, redirects, cookie handling, and WebSockets. Key APIs:

    • aiohttp.ClientSession
    • aiohttp.TCPConnector
    • aiohttp.ClientResponse
    • aiohttp.WSMessage
    • aiohttp.BasicAuth

    3. Shared Wire-Protocol Code

    Core logic used by both client and server for protocol handling. Key APIs:

    • aiohttp.MultipartReader / MultipartWriter
    • aiohttp.CookieJar
    • aiohttp.TraceConfig
    • aiohttp.resolver.AsyncResolver
  2. Overview of aiohttp.web exceptions

    master

    In aiohttp.web, exceptions are used to signal specific HTTP status codes to the client. Every exception is a subclass of HTTPException and corresponds to a single HTTP status code. While status codes in the 200-300 range are technically successful, they can still be raised within handlers to trigger specific responses.

    Common usage pattern:

    async def handler(request):
        raise aiohttp.web.HTTPFound('/redirect')
    async def handler(request):
        raise aiohttp.web.HTTPFound('/redirect')
  3. Access all cookies including duplicates

    master

    The response.cookies attribute returns a SimpleCookie object. Because SimpleCookie uses the cookie name as a key, cookies with the same name but different domains or paths will overwrite each other in this attribute.

    To access all cookies, including duplicates with the same name, use the headers attribute to retrieve all Set-Cookie headers:

    # To get all cookies including duplicates:
    all_cookies = resp.headers.getall('Set-Cookie')
  4. Understand the Resource and Route relationship in aiohttp

    master

    In aiohttp, routing is decoupled into two distinct concepts: Resources and Routes.

    1. Resource: Represents a URI/location (a path). A resource has a path (which can be constant or dynamic, e.g., /path/{to}) and an optional unique name. A resource acts as a container for one or more routes.
    2. Route: Represents a specific HTTP method applied to a resource. A route binds an HTTP method (like GET or POST) and a web-handler to a resource.

    This decoupling allows you to define a single location (Resource) and then attach multiple handlers for different HTTP methods to that same location.

    When you access a named entity via app.router['name'], you receive an AbstractResource instance rather than a specific route.

    # 1. Define a resource with a path and a unique name
    resource = router.add_resource('/path/{to}', name='name')
    
    # 2. Add specific routes (methods + handlers) to that resource
    route = resource.add_route('GET', handler)
  5. Share resources like database connections via Application

    master

    The aiohttp.web.Application object supports the dict interface. You can store shared resources (like database connections) using web.AppKey to ensure type safety and avoid key collisions. Handlers can then access these resources via request.app[app_key].

    db_key = web.AppKey("db_key", DB)
    
    async def go(request):
        db = request.app[db_key]
        cursor = await db.cursor()
        await cursor.execute('SELECT 42')
        return web.Response(status=200, text='ok')
    
    async def init_app():
        app = Application()
        db = await create_connection(user='user', password='123')
        app[db_key] = db
        app.router.add_get('/', go)
        return app
  6. Implement Token Refresh middleware

    master

    Middlewares are ideal for managing authentication lifecycles. You can implement several strategies:

    1. Reactive Refresh: Check for a 401 Unauthorized response, refresh the token, and retry the request.
    2. Expiry-based Refresh: Check the token's expiry time before making the request and refresh if it is near expiration.
    3. Preemptive Refresh: Refresh the token in a background task to avoid latency during the request flow (often better implemented outside of middleware).

    These middlewares can also be adapted to handle proxy authentication by modifying ClientRequest.proxy_headers.

    # Example: Reactive 401 Refresh
    class TokenRefresh401Middleware:
        async def __call__(self, session, method, url, **kwargs):
            response = await session.request(method, url, **kwargs)
            if response.status == 401:
                # Logic to refresh token
                new_token = await refresh_token()
                kwargs['headers']['Authorization'] = f'Bearer {new_token}'
                return await session.request(method, url, **kwargs)
            return response
  7. Choose the right HTTP response class

    master

    aiohttp.web provides three main response classes depending on your use case:

    1. Response: The most common choice. Use this for standard HTTP responses where you have the entire body ready. It automatically calculates the Content-Length header.
    2. StreamResponse: The base class for streaming data. Use this when you need to send data in chunks (e.g., large files or real-time streams).
    3. FileResponse: Specialized for sending files from the filesystem. It supports Content-Range and If-Range headers for efficient file transfers.

    Note: Response and FileResponse are derived from StreamResponse.

    # Common case: returning a standard Response
    async def handler(request):
        return Response(text="All right!")
  8. Manage middleware scope: Session vs Request level

    master

    It is critical to understand how middlewares are applied. Request-level middlewares replace session-level middlewares; they do not merge with them. This is different from headers, which are merged.

    • If you define middlewares in ClientSession(middlewares=[...]), they apply to all requests.
    • If you pass middlewares=[...] to a specific request method (like get() or post()), the session-level middlewares are ignored for that request.
    • To use both, you must explicitly pass both lists in the request call.
    # Session middleware is used
    session = ClientSession(middlewares=[middleware_session])
    await session.get("http://example.com")
    
    # Session middleware is NOT used, only request middleware
    await session.get("http://example.com", middlewares=[middleware_request])
    
    # To use both, explicitly pass both
    await session.get(
        "http://example.com",
        middlewares=[middleware_session, middleware_request]
    )
  9. Understand Application Freezing

    master

    An aiohttp.web.Application instance can act as either a main application (using app.make_handler()) or as a sub-application, but not both simultaneously.

    Once an application is either connected via add_subapp() or started as a top-level web server, it becomes frozen.

    While frozen, you cannot:

    • Register new routes.
    • Register new signals.
    • Register new middlewares.
    • Change the application state (e.g., app['name'] = 'value') — changing state in a frozen application is deprecated.
  10. Mitigate DoS via nested multipart recursion

    master

    The MultipartReader.next() API allows for nested multipart bodies and does not impose a default recursion depth limit, which can lead to a RecursionError (DoS).

    Note: Request.post() is safe as it automatically rejects nested multiparts with a ValueError. However, if you are using the bare MultipartReader API directly, you must implement your own depth limiting to prevent stack exhaustion.

  11. What are Connectors in aiohttp?

    master

    Connectors are the transport layer for the aiohttp client API. They manage the underlying connections used to send requests.

    There are two standard types of connectors:

    1. TCPConnector: The most common transport, used for regular HTTP and HTTPS requests via TCP sockets. If you are unsure which connector to use, use TCPConnector.
    2. UnixConnector: Used for connecting via UNIX sockets, primarily useful for testing or high-speed inter-process communication on the same host.

    All connectors support keep-alive connections by default, which allows reusing connections for multiple requests to improve performance. This behavior is controlled by the force_close parameter.

    # Example of using a UnixConnector
    from aiohttp import ClientSession, UnixConnector
    
    async def main():
        conn = UnixConnector(path='/path/to/socket')
        session = ClientSession(connector=conn)
        async with session.get('http://python.org') as resp:
            ... 
        await session.close()
  12. Handle non-UTF-8 header values safely

    master

    aiohttp's parser is designed to be bytes-preserving. This means header values are not automatically decoded into strings and may contain non-UTF-8 sequences.

    Action for Users: When working with headers, you must manually re-validate any header value before reflecting it into responses, logs, or sub-requests to prevent injection or encoding issues.