Async::HTTP

repository·main·Indexed 18 days ago

https://github.com/socketry/async-http

A high-performance, asynchronous HTTP client and server implementation for Ruby built on the Async ecosystem. It supports HTTP/1.0, HTTP/1.1, and HTTP/2 with TLS and streaming for both requests and responses. Key components include Async::HTTP::Client for persistent connections, Async::HTTP::Internet for high-level request patterns, and Async::HTTP::Server for hosting services.

Tokens
10.4K
Snippets
46
Records
55
Agent score
60%

What's inside async-http

  1. Overview of Async::HTTP

    main

    Async::HTTP is an asynchronous client and server implementation supporting HTTP/1.0, HTTP/1.1, and HTTP/2, including TLS support. It is designed for high-performance asynchronous I/O and supports streaming for both requests and responses.

    The library is built on the following ecosystem components:

    For production Rack-compatible server deployments, it is recommended to use falcon, which is built on top of async-http.

  2. Core concepts of Async::HTTP

    main

    Understanding the primary abstractions in Async::HTTP:

    • Async::HTTP::Client: The main class used for making HTTP requests.
    • Async::HTTP::Internet: A high-level interface providing a simple way to make requests to any server on the internet.
    • Async::HTTP::Server: The main class for handling and responding to incoming HTTP requests.
    • Async::HTTP::Endpoint: A utility to parse HTTP URLs to create clients or servers.
    • protocol-http: Provides the underlying abstract HTTP protocol interfaces.
  3. Manage response persistence and closing

    main

    Because responses are streamed, you must ensure they are closed. If you manage the response manually (without a block), use an ensure block to call close.

    By default, Async::HTTP::Internet maintains connection persistence by creating an Async::HTTP::Client for each remote host and keeping connections open to reduce latency. Connections are closed automatically when the event loop exits.

    require 'async/http/internet/instance'
    
    Sync do
    	response = Async::HTTP::Internet.get("https://httpbin.org/get")
    	puts response.read
    ensure
    	response&.close
    end
  4. Mock HTTP responses with Async::HTTP::Mock::Endpoint

    main

    To avoid making real network requests during testing, you can use Async::HTTP::Mock::Endpoint. This feature runs a real server in a separate background task and intercepts requests made by a client.

    To implement this:

    1. Create an instance of Async::HTTP::Mock::Endpoint.
    2. Start a background server task using mock_endpoint.run where you define the logic to return a Protocol::HTTP::Response.
    3. Wrap your target Async::HTTP::Endpoint using mock_endpoint.wrap(endpoint).
    4. Initialize your Async::HTTP::Client with the wrapped endpoint.
    require 'async/http'
    require 'async/http/mock'
    
    mock_endpoint = Async::HTTP::Mock::Endpoint.new
    
    Sync do
      # Start a background server:
      server_task = Async(transient: true) do
        mock_endpoint.run do |request|
          # Respond to the request:
          ::Protocol::HTTP::Response[200, {}, ["Hello, World"]]
        end
      end
      
      endpoint = Async::HTTP::Endpoint.parse("https://www.google.com")
      mocked_endpoint = mock_endpoint.wrap(endpoint)
      client = Async::HTTP::Client.new(mocked_endpoint)
      
      response = client.get("/")
      puts response.read
      # => "Hello, World"
    end
  5. Transparently mock Async::HTTP::Client using Sus

    main

    You can achieve 'transparent mocking' by using your test framework (like Sus) to intercept Async::HTTP::Client.new. This allows your application code to instantiate a standard client while the test framework automatically injects a mocked endpoint.

    When using Sus::Fixtures::Async::ReactorContext, you can wrap the :new method of Async::HTTP::Client to return a client initialized with a mock_endpoint.wrap(endpoint) instead of the original endpoint.

    require 'async/http'
    require 'async/http/mock'
    require 'sus/fixtures/async/reactor_context'
    
    include Sus::Fixtures::Async::ReactorContext
    
    let(:mock_endpoint) {Async::HTTP::Mock::Endpoint.new}
    
    def before
      super
      
      # Mock the HTTP client:
      mock(Async::HTTP::Client) do |mock|
        mock.wrap(:new) do |original, endpoint|
          original.call(mock_endpoint.wrap(endpoint))
        end
      end
      
      # Run the mock server:
      Async(transient: true) do
        mock_endpoint.run do |request|
          ::Protocol::HTTP::Response[200, {}, ["Hello, World"]]
        end
      end
    end
    
    it "should perform a web request" do
      client = Async::HTTP::Client.new(Async::HTTP::Endpoint.parse("https://www.google.com"))
      response = client.get("/")
      # The response is mocked:
      expect(response.read).to be == "Hello, World"
    end
  6. Set request timeouts

    main

    Wrap your request logic in task.with_timeout(seconds) to prevent requests from hanging indefinitely. This will raise an Async::TimeoutError if the timeout is reached.

    require 'async/http/internet/instance'
    
    Sync do |task|
    	# Request will timeout after 2 seconds
    	task.with_timeout(2) do
    		response = Async::HTTP::Internet.get "https://httpbin.org/delay/10"
    	entensure
    		response&.close
    	end
    rescue Async::TimeoutError
    	puts "The request timed out"
    end
  7. Make HTTP requests with Async::HTTP::Internet

    main

    Use Async::HTTP::Internet for simple request patterns. It supports several HTTP methods including :patch, :options, :connect, :post, :get, :delete, :head, :trace, and :put.

    Automatic Resource Management

    When you pass a block to a request method, the response is automatically closed when the block completes:

    require 'async/http/internet/instance'
    
    Sync do
      Async::HTTP::Internet.get("https://httpbin.org/get") do |response|
        puts response.read
      end
    end

    Manual Resource Management

    If you do not use a block, you must ensure the response is closed manually to avoid leaking resources, as responses are streamed. Use an ensure block to guarantee closure:

    require 'async/http/internet/instance'
    
    Sync do
      response = Async::HTTP::Internet.get("https://httpbin.org/get")
      puts response.read
    ensure
      response&.close
    end

    Connection Persistence

    Async::HTTP::Internet automatically manages connection persistence. It creates an Async::HTTP::Client for each remote host and keeps connections open to reduce latency for subsequent requests to the same host. Connections are closed automatically when the event loop exits.

  8. How Async::HTTP::Client handles retries

    main

    The Async::HTTP::Client#call method implements a retry strategy based on the type of error encountered:

    1. Protocol::HTTP::RefusedError: This indicates the request was not processed by the server. The client will retry the request if attempt < @retries and request.rewind! is successful, even for non-idempotent requests.
    2. Network/Protocol Errors: For errors like Protocol::HTTP::RemoteError, SocketError, IOError, EOFError, Errno::ECONNRESET, or Errno::EPIPE, the client will retry only if attempt < @retries and the request is marked as retryable via request.retry! (idempotency check).

    If the retry limit is reached or the request cannot be rewound/retried, the error is raised to the caller.