Req HTTP Client for Elixir

repository·main·Indexed 23 days ago

https://github.com/wojtekmach/req

A batteries-included, extensible HTTP client for Elixir featuring a step-based architecture. Req provides high-level functions like Req.get!/2 and Req.post!/2, supports request/response body streaming, and includes built-in support for JSON, form, and multipart encoding. It offers advanced features such as AWS Signature Version 4 signing, response body checksum verification, and a modular plugin system for extending functionality via custom steps.

Tokens
4.4K
Snippets
15
Records
30
Agent score
78%

What's inside Req

  1. How Req steps work

    main

    Req's functionality is modularized into steps. A request is processed by running a request struct through a series of these steps. Steps are regular functions that can be reused, rearranged, or custom-written. You can append custom steps to a request using Req.Request.append_request_steps/2.

    req =
      Req.new(base_url: "https://api.github.com")
      |> Req.Request.append_request_steps(
        debug_url: fn request ->
          IO.inspect(URI.to_string(request.url))
          request
        end
      )
    
    Req.get!(req, url: "/repos/wojtekmach/req").body["description"]
  2. Extend Req with plugins

    main

    Custom steps can be packaged into plugins. Plugins are typically used by calling an attach() function on a Req struct to chain multiple steps together.

    req =
      (Req.new(http_errors: :raise)
      |> ReqEasyHTML.attach()
      |> ReqS3.attach()
      |> ReqHex.attach()
      |> ReqGitHubOAuth.attach())
    
    Req.get!(req, url: "https://elixir-lang.org").body[".entry-summary h5"]
  3. Perform basic HTTP requests with Req.get!/2 and Req.post!/2

    main

    Req provides high-level functions for common HTTP methods. Req.get!/2 and Req.post!/2 automatically handle response body decoding and follow redirects.

    Example of a GET request accessing a decoded JSON body:

    Req.get!("https://api.github.com/repos/wojtekmach/req").body["description"]

    Example of a POST request with JSON data:

    Req.post!("https://httpbin.org/post", json: %{x: 1, y: 2}).body["json"]
    Req.get!("https://api.github.com/repos/wojtekmach/req").body["description"]
    #=> "Req is a batteries-included HTTP client for Elixir."
  4. Decode response bodies using decoders

    main

    Req can automatically decode response bodies based on the Content-Type header. You can control this behavior using the decoders option.

    Built-in decoders include: :json, :json_api, :zip, :tar, :tgz, :gz, :zst, and :csv.

    To use a custom decoder, pass a tuple in the format {format, codec} where codec is a function or a module that implements a decode/1 function.

    Note: Setting raw: true in your request options disables response body decoding and decompression.

    # Decode JSON (default behavior)
    response = Req.get!("https://httpbin.org/json")
    response.body["slideshow"]["title"]
    
    # Decode a ZIP archive (opt-in)
    response = Req.get!("https://example.com/archive.zip", decoders: [:zip])
    response.body["file.txt"]
  5. Stream request and response bodies

    main

    Req supports streaming for both sending and receiving data.

    Request Body Streaming: Set body: enumerable to stream data to the server.

    Response Body Streaming: Set into: fun | collectable | :self to stream the response body into a function, a collectable, or to the process itself.

  6. Reuse request configurations with Req.new/1

    main

    If you are making multiple similar requests, use Req.new/1 to create a request struct with common options (like base_url) and reuse it across multiple calls.

    req = Req.new(base_url: "https://api.github.com")
    
    Req.get!(req, url: "/repos/sneako/finch").body["description"]
    Req.get!(req, url: "/repos/elixir-mint/mint").body["description"]
  7. Configure request redirect behavior

    main

    Req automatically follows redirects by default. You can customize this using the following options:

    • :redirect: Set to false to disable automatic redirects. Defaults to true.
    • :redirect_trusted: If true, authorization credentials will be sent to any host during a redirect. If false (default), credentials are only sent to redirects with the same host, scheme, and port.
    • :redirect_log_level: The log level for redirect messages. Defaults to :debug. Can be set to false to disable logging.
    • :max_redirects: The maximum number of redirects allowed. Defaults to 10. Reaching this limit raises a Req.TooManyRedirectsError.

    Method Handling during Redirects:

    • Status codes 301, 302, and 303 will change the request method to GET.
    • Status codes 307 and 308 will preserve the original request method.
  8. Configure request retries

    main

    Req can automatically retry requests when they fail due to transient errors. Use the :retry option to configure this:

    • :safe_transient (default): Retries GET or HEAD requests on specific transient errors (HTTP 408, 429, 500, 502, 503, 504, or specific transport/HTTP2 errors).
    • :transient: Retries all HTTP methods on the same transient errors as :safe_transient.
    • fun: A 2-arity function fn request, response_or_exception -> ... end that returns:
      • true: Retry with default delay.
      • {:delay, milliseconds}: Retry with a specific delay.
      • false: Do not retry.

    Retry Options:

    • :max_retries: Maximum number of retry attempts. Defaults to 3 (total of 4 attempts).
    • :retry_delay:
      • If not set, Req uses the Retry-After header if available, otherwise uses exponential backoff with jitter.
      • Can be a function fn retry_count -> delay_ms end where retry_count starts at 0.
    • :retry_log_level: Log level for retry messages. Defaults to :warning. Can be set to false to disable.

    Example:

    # Retry on 500 errors
    Req.get!("https://httpbin.org/status/500,200")
  9. Handle HTTP error responses

    main

    By default, Req returns the response even if the status code indicates an error (4xx or 5xx). You can change this using the :http_errors option:

    • :return (default): Returns the response object as is.
    • :raise: Raises a RuntimeError containing the status code and the response body.

    Example:

    # Returns the response with status 404
    Req.get!("https://httpbin.org/status/404").status
    
    # Raises an error
    Req.get!("https://httpbin.org/status/404", http_errors: :raise)
    Req.get!("https://httpbin.org/status/404", http_errors: :raise)
  10. Handle Req.ChecksumMismatchError

    main

    When using Req.Steps.checksum/1, a Req.ChecksumMismatchError is raised if the calculated checksum of the response body does not match the expected value. This exception contains two keyword list keys to help with debugging:

    • :expected: The checksum value that was anticipated.
    • :actual: The checksum value that was actually calculated from the response.

    You can catch this exception to handle cases where data integrity is compromised during transfer.

  11. Use HTTP Digest Authentication

    main

    Req supports HTTP Digest authentication. When you set the :auth option to {:digest, "user:pass"}, Req will automatically handle the 401 challenge-response flow.

    Example:

    resp = Req.get!("https://httpbin.org/digest-auth/auth/user/pass", auth: {:digest, "user:pass"})
    resp.status
    # 200
    resp = Req.get!("https://httpbin.org/digest-auth/auth/user/pass", auth: {:digest, "user:pass"})
    resp.status