go-retryablehttp

The retryablehttp package is a thin wrapper over the standard Go net/http client library. It provides a familiar HTTP client interface that automatically handles retries and exponential backoff.

Retry Logic

retryablehttp automatically triggers a retry under the following conditions:

• An error is returned by the client (e.g., connection errors).
• A 500-range response code is received (except for 501 Not Implemented).

If these conditions are met, the client waits for a period (using exponential backoff) before attempting the request again. Otherwise, the response is returned to the caller for interpretation.

Request Bodies

Unlike the standard net/http client, retryablehttp supports 'rewinding' request bodies for methods that require them (like POST or PUT). This allows the client to re-send the full request body if an initial attempt fails.

Because retryablehttp performs retries, it must be able to rewind the request body for subsequent attempts. When creating a request via NewRequest or SetBody, you can provide several types of input:

• ReaderFunc: A function returning an io.Reader and an error. This is the most efficient way to provide multiple readers.
• []byte: A raw byte slice.
• *bytes.Buffer: A buffer where the underlying slice is used.
• *bytes.Reader: A reader that can be re-used.
• io.ReadSeeker: An interface that can be seeked back to the start.
• io.Reader: If a standard io.Reader is provided, the client will read the entire body into memory once to allow for re-use during retries.

Note: Avoid using io.ReadSeeker if you observe data races between the net/http library and the Seek functionality.

Using retryablehttp is designed to be nearly identical to using the standard net/http library. You can use package-level functions like Get to perform requests. If the request fails due to connection errors or 500-range status codes, the call will block and retry automatically with exponential backoff before returning the final result.

resp, err := retryablehttp.Get("/foo")
if err != nil {
    panic(err)
}

To use retryablehttp in existing codebases that expect a standard *http.Client, you can use the StandardClient() method. This allows you to configure advanced retry behavior in a *retryablehttp.Client and then pass the resulting standard client into any function or library that requires a *http.Client.

retryClient := retryablehttp.NewClient()
retryClient.RetryMax = 10

standardClient := retryClient.StandardClient() // *http.Client

You can customize how the client behaves during failures by setting fields on the Client struct:

• RetryMax: Maximum number of retries.
• RetryWaitMin / RetryWaitMax: Bounds for the backoff duration.
• CheckRetry: A CheckRetry function to define which responses or errors trigger a retry. Use DefaultRetryPolicy or ErrorPropagatedRetryPolicy.
• Backoff: A Backoff function to define the wait time between attempts. Options include DefaultBackoff, LinearJitterBackoff, and RateLimitLinearJitterBackoff.
• ErrorHandler: A custom ErrorHandler to handle cases where retries are exhausted.
• PrepareRetry: A PrepareRetry function to modify the request (e.g., re-signing) before each retry attempt.

You can attach a ResponseHandlerFunc to a *retryablehttp.Request using SetResponseHandler. This function is called when a response is received and the CheckRetry policy indicates no retry is needed. If the handler returns an error, the CheckRetry policy is invoked to see if the error from the handler should trigger a retry of the entire request.

Warning: The response body is not automatically closed by the handler. You must close it either in the handler or by the caller to avoid memory leaks.

The Client provides several methods to execute requests. You can use the full Do method with a *retryablehttp.Request for maximum control, or convenience methods for common verbs.

Using Do with Request

For complex requests, use NewRequest or NewRequestWithContext to create a *retryablehttp.Request, then pass it to client.Do(req).

Convenience Methods

• Get(url string)
• Head(url string)
• Post(url, bodyType string, body interface{})
• PostForm(url string, data url.Values)

The RoundTripper type implements the standard library's http.RoundTripper interface. This allows you to wrap a retryablehttp.Client and use it within a standard *http.Client. When you call client.Do(req) on a standard client using this RoundTripper, the request will automatically benefit from the retry logic configured in the underlying retryablehttp.Client.

Note that the behavior of the RoundTripper is highly dependent on the configuration of the Client it uses. If no Client is explicitly provided to the RoundTripper, it will initialize a default retryablehttp.Client using NewClient() upon the first request.

Use NewClient() to create a new Client instance with default settings. The default configuration includes:

• RetryWaitMin: 1 second
• RetryWaitMax: 30 seconds
• RetryMax: 4 retries
• CheckRetry: DefaultRetryPolicy (retries on connection errors and 500-range status codes)
• Backoff: DefaultBackoff (exponential backoff, respects Retry-After header)

The Client is a thin wrapper over the standard net/http client and is designed to be easy to drop into existing programs.

The Client.Logger field accepts either a Logger or a LeveledLogger interface.

• Logger: Requires Printf(string, ...interface{}).
• LeveledLogger: Requires Error, Info, Debug, and Warn methods. If you provide a LeveledLogger, the client will use these levels for its internal logging.

The RoundTripper struct is an implementation of http.RoundTripper that uses a retryablehttp.Client to execute requests. It is designed to bridge the gap between the standard net/http package and the retryablehttp library.

Fields:

• Client (*Client): The specific retryablehttp.Client to use for requests. If this is nil, the RoundTripper will automatically initialize a default client using NewClient() during the first call to RoundTrip.