ureq

repository·main·Indexed 24 days ago

https://github.com/algesten/ureq

A minimal, blocking HTTP client for Rust. ureq provides a simple API for performing HTTP requests with support for connection pooling via Agents, TLS, JSON handling through serde, and proxy support (HTTP and SOCKS). Version 3.3.0 features a Sans-IO architecture and integration with the standard http crate.

Tokens
16.3K
Snippets
40
Records
73
Agent score
77%

What's inside ureq

  1. Understand the `unversioned` module and API stability

    main

    In ureq 3.x, the library aims for strict semver adherence. However, certain experimental or low-level APIs are located in the unversioned module.

    The unversioned module does not follow semver.

    This module includes:

    • Transport: A trait for writing bespoke transports.
    • Resolver: A trait for writing bespoke DNS resolvers.
    • unversioned::multipart: Multipart support.

    Use these features with caution as their APIs are not yet solidified and may change in minor or patch releases.

  2. Handle character encoding with the charset feature

    main

    By enabling the charset feature, ureq supports receiving character sets other than utf-8.

    When calling Body::read_to_string(), the library inspects the Content-Type header (e.g., Content-Type: text/plain; charset=iso-8859-1). If a charset is specified, it attempts to decode the body using that encoding. If the charset is missing or uninterpretable, it falls back to utf-8.

    Note: ureq does not currently support encoding specific character sets when sending request bodies.

  3. Configure lossy UTF-8 reading for text bodies

    main

    When reading text bodies (where Content-Type starts with text/), ureq can handle invalid UTF-8 characters by replacing them with a question mark ? (not the standard UTF-8 replacement character).

    • For Body::read_to_string(), this lossy behavior is enabled by default.
    • For Body::as_reader(), this behavior is disabled by default but can be enabled.

    To precisely configure this behavior, use Body::with_config().

  4. Configure Root Certificates in ureq 3.x

    main

    In ureq 3.x, you can configure root certificates using the RootCerts enum within TlsConfig. This configuration can be applied at either the Agent level or the Request level.

    Available RootCerts variants:

    • PlatformVerifier: For rustls, this delegates to the system. For native-tls, this uses the root certificates that native-tls is already picking up (the recommended setting for most users).
    • WebPki: Uses the root certificates bundled with ureq.
    • Specific: Allows you to provide and set your own specific root certificates.
  5. Migrate from ureq 2.x to 3.x

    main

    Migrating from ureq 2.x to 3.x involves several breaking changes due to a complete ground-up rewrite. Key changes include:

    • Sans-IO Architecture: The HTTP protocol is now implemented in the ureq-proto crate using a Sans-IO style. This allows users to implement their own Transport or Resolver for alternative TLS or non-socket communication.
    • HTTP Crate Integration: ureq 3.x no longer uses custom Request and Response structs. It now uses the standard [http] crate, providing a unified API compatible with the broader Rust ecosystem.
    • Removed Automatic Retries: Unlike 2.x, ureq 3.x does not automatically retry idempotent methods (like GET or HEAD). Retries must be implemented manually by the user.
    • Reduced Re-exports: To maintain stability, ureq 3.x re-exports fewer crates. Specifically:
      • TLS configuration and Cookie APIs are now built into ureq rather than re-exported from external crates.
      • The json! macro has been dropped.
    • Body Charset: ureq 3.x cannot change the charset of a request body, though it can still do so for response bodies.
  6. Perform simple HTTP requests with ureq

    main

    For basic HTTP tasks, you can use the crate-level convenience functions like ureq::get or ureq::post. These functions allow you to chain methods for headers and then execute the request with .call().

    Example of a simple GET request:

    let body: String = ureq::get("http://example.com")
        .header("Example-Header", "header value")
        .call()?
        .body_mut()
        .read_to_string()?;
  7. Use an Agent for connection pooling and configuration

    main

    For more complex scenarios, use an Agent. An Agent manages a connection pool for reusing connections and can hold a cookie store (if the cookies feature is enabled). Agents can be cheaply cloned using Arc internally, and all clones share the same state.

    Use an Agent when you need to:

    • Reuse connections across multiple requests.
    • Set global configurations like timeouts.
    • Configure specific TLS settings.

    Example of configuring and using an Agent:

    use ureq::Agent;
    use std::time::Duration;
    
    let mut config = Agent::config_builder()
        .timeout_global(Some(Duration::from_secs(5)))
        .build();
    
    let agent: Agent = config.into();
    
    let body: String = agent.get("http://example.com/page")
        .call()?
        .body_mut()
        .read_to_string()?;
    
    // Reuses the connection from previous request.
    let response: String = agent.put("http://example.com/upload")
        .header("Authorization", "example-token")
        .send("some body data")?
        .body_mut()
        .read_to_string()?;
  8. Handle HTTP errors and status codes

    main

    ureq returns errors via Result<T, ureq::Error>. This includes I/O and protocol errors. By default, any HTTP status code in the 4xx or 5xx range is treated as an Error::StatusCode(code).

    You can handle these errors using pattern matching:

    use ureq::Error;
    
    match ureq::get("http://mypage.example.com/").call() {
        Ok(response) => { /* it worked */},
        Err(Error::StatusCode(code)) => {
            /* the server returned an unexpected status
               code (such as 400, 500 etc) */
        }
        Err(_) => { /* some kind of io/transport/etc error */ }
    }

    To prevent 4xx/5xx status codes from being treated as errors, use the .http_status_as_error() method on the request builder.

  9. Configure HTTP and SOCKS proxies

    main

    Proxies are configured on an Agent. All requests sent through that agent will use the configured proxy.

    • HTTP (CONNECT) proxies: Always available.
    • SOCKS4/SOCKS5 proxies: Must be enabled using the socks-proxy feature.

    Environment Variables

    When creating a default Agent, ureq automatically reads proxy configuration from the following environment variables (checked in order, including lowercase variants):

    1. ALL_PROXY
    2. HTTPS_PROXY
    3. HTTP_PROXY

    NO_PROXY can be used to specify hosts that bypass the proxy. It supports:

    • Exact hosts
    • Wildcard suffixes (*.example.com)
    • Dot suffixes (.example.com)
    • Match-all (*)
  10. Configure ureq features via Cargo

    main

    ureq uses a minimal dependency tree by disabling several features by default. You can enable them in your Cargo.toml.

    Example configuration:

    ureq = { version = "3", features = ["socks-proxy", "charset"] }

    Available Features:

    • rustls: Enables the rustls TLS implementation (default for crate-level calls).
    • native-tls: Enables the native TLS backend (must be configured on an Agent).
    • platform-verifier: Enables verifying server certificates using the host OS method.
    • socks-proxy: Enables proxy configuration using socks4://, socks4a://, socks5://, and socks:// prefixes.
    • cookies: Enables cookie support.
    • gzip: Enables gzip-compressed response requests and decompression.
    • brotli: Enables brotli-compressed response requests and decompression.
    • charset: Enables interpreting the charset part of the Content-Type header.
    • json: Enables JSON sending and receiving via serde_json.
    • multipart: Enables multipart/form-data sending.
    • rustls-webpki-roots: Enables webpki-roots for root certificates when using rustls.
    • native-tls-webpki-roots: Enables webpki-root-certs for root certificates when using native-tls.

    Unstable Features:

    • rustls-no-provider
    • native-tls-no-default
    • vendored (compiles and statically links non-Rust vendors like OpenSSL)
  11. Difference between shared and owned body readers

    main

    When accessing the body, you can choose between a shared reader (borrowing the body) or an owned reader (consuming the body).

    • Shared Reader (as_reader() / with_config()): Borrows the Body. Use this when you want to keep the Response object alive and perform multiple operations.
    • Owned Reader (into_reader() / into_with_config()): Consumes the Body. This returns a reader with a 'static lifetime, allowing you to move the reader to another thread or disconnect it from the original response object.
  12. How Form and Part work together

    main

    A Form is a collection of Part objects. When a Form is sent as a request body, it generates a unique boundary string. Each Part within the form is then encapsulated between these boundaries, including its own headers (like Content-Disposition and Content-Type).

    1. Form::new() initializes a new multipart container with a random boundary.
    2. Form::text() or Form::file() are high-level helpers that create and append Parts.
    3. Form::part() allows injecting a manually configured Part.
    4. RequestBuilder::send(form) triggers the serialization of the entire multipart structure into the request body.