Pingora

The Pingora Server does not have a native concept of a 'Proxy'. Instead, it operates on Services. A proxy is implemented by creating a struct that implements specific traits and is then wrapped in a Service.

For an HTTP proxy, the hierarchy of responsibility is:

1. HttpProxy (struct): Handles the high-level proxying workflow. It is customized using the ProxyHttp trait.
2. HttpServerApp (trait): Handles protocol-specific details like selecting between H1 and H2 streams and managing H2 handshakes.
3. ServerApp (trait): Handles dispatching application instances as individual tasks per Session.
4. Service<A> (struct): Handles dispatching application instances as individual tasks per Listener.

┌─────────────┐        ┌──────────────────────────────────────┐
│  HttpProxy  │        │Handles high level Proxying workflow, │
│  (struct)   │─ ─ ─ ─ │   customizable via ProxyHttp trait   │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│HttpServerApp│        │ Handles selection of H1 vs H2 stream │
│   (trait)   │─ ─ ─ ─ │   handling, incl H2 handshake       │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│  ServerApp  │        │ Handles dispatching of App instances │
│   (trait)   │─ ─ ─ ─ │   as individual tasks, per Session   │
└──────┬──────┘        └──────────────────────────────────────┘
       │
┌──────▼──────┐        ┌──────────────────────────────────────┐
│ Service<A>  │        │ Handles dispatching of App instances │
│  (struct)   │─ ─ ─ ─ │   as individual tasks, per Listener   │
└─────────────┘        └──────────────────────────────────────┘

The Pingora proxy lifecycle follows a specific sequence of phases for handling a request from inception to completion. When caching is not involved, the flow is divided into request filtering, upstream connection, upstream request handling, and response filtering.

1. Request Phase

• new request: The lifecycle begins when a new request is received.
• early_request_filter: Initial filtering stage.
• request_filter: Main request filtering stage. If a response is sent here, the flow moves directly to logging.
• upstream_peer: The phase where the proxy determines the upstream destination.

2. Upstream Connection & Request Phase

• Connect: The IO operation to connect to the upstream.
  • If connection fails and is retryable, it returns to upstream_peer.
  • If connection fails and is not retryable, it moves to fail_to_proxy, sends an error response, and goes to logging.
• connected_to_upstream: Successful connection established.
• upstream_request_filter: Filtering applied to the request destined for the upstream.
• request_body_filter: Processing the request body.
• SendReq: The IO operation to send the request to the upstream.
• RecvResp: The IO operation to read the response from the upstream.

3. Response Phase

• adjust_upstream_modules: A feature point for adjusting upstream modules after receiving a response.
• upstream_response_filter: Filtering applied to the upstream response.
• response_filter: Main response filtering stage.
• upstream_response_body_filter: Processing the upstream response body.
• response_body_filter: Final response body processing.
• logging: Final stage where request details are logged before the request is marked as done.

Error Handling

• IOFailure: Occurs during SendReq or RecvResp. Leads to error_while_proxy.
• error_while_proxy: If the error is retryable, the process returns to upstream_peer. If not, it moves to fail_to_proxy.
• Response Filter Errors: Any error during a response filter leads to error_while_proxy.

 graph TD;
    start("new request")-->early_request_filter;
    early_request_filter-->request_filter;
    request_filter-->upstream_peer;

    upstream_peer-->Connect{{IO: connect to upstream}};

    Connect--connection success-->connected_to_upstream;
    Connect--connection failure-->fail_to_connect;

    connected_to_upstream-->upstream_request_filter;
    upstream_request_filter --> request_body_filter;
    request_body_filter --> SendReq{{IO: send request to upstream}};
    SendReq-->RecvResp{{IO: read response from upstream}};
    RecvResp-.feature: adjust_upstream_modules.->adjust_upstream_modules;
    adjust_upstream_modules-->upstream_response_filter-->response_filter-->upstream_response_body_filter-->response_body_filter-->logging-->endreq("request done");

    fail_to_connect --can retry-->upstream_peer;
    fail_to_connect --can't retry-->fail_to_proxy--send error response-->logging;

    RecvResp--failure-->IOFailure;
    SendReq--failure-->IOFailure;
    error_while_proxy--can retry-->upstream_peer;
    error_while_proxy--can't retry-->fail_to_proxy;

    request_filter --send response-->logging


    Error>any response filter error]-->error_while_proxy
    IOFailure>IO error]-->error_while_proxy

While CTX is used for state scoped to a single request, you can share data across all requests (such as global counters, caches, or shared resources) using standard Rust concurrency primitives.

Common patterns include:

• Proxy Struct Members: Storing state within the MyProxy struct itself (e.g., using Mutex<T> or AtomicUsize).
• Global Statics: Using static variables with synchronization (e.g., static COUNTER: Mutex<usize>).
• Shared Ownership: Using Arc to share data across threads/requests.

Because these resources are accessed concurrently by multiple requests, you must use thread-safe types like Mutex, RwLock, or Atomic types to prevent data races.

// Example: Using a global static and a struct member for cross-request state
static REQ_COUNTER: Mutex<usize> = Mutex::new(0);

pub struct MyProxy {
    beta_counter: Mutex<usize>,
}

#[async_trait]
impl ProxyHttp for MyProxy {
    type CTX = MyCtx;
    // ...
    async fn upstream_peer(
        &self,
        _session: &mut Session,
        ctx: &mut Self::CTX,
    ) -> Result<Box<HttpPeer>> {
        // Increment global counter
        let mut req_counter = REQ_COUNTER.lock().unwrap();
        *req_counter += 1;

        // Increment per-proxy member counter
        let mut beta_count = self.beta_counter.lock().unwrap();
        *beta_count += 1;

        // ...
    }
}

Pingora's default behavior is to strip downstream hop-by-hop request fields and fields nominated by the Connection header before forwarding to an HTTP upstream. This ensures RFC compliance and prevents routing metadata leakage.

Header Policy Modes

1. Preserve all behavior (RFC-non-compliant)

To bypass all stripping and use the previous passthrough behavior, use HttpUpstreamRequestPolicy::preserve(). Warning: You are responsible for all valid hop-by-hop handling in your upstream_request_filter().

2. Retain only Connection-nominated fields

To keep the default behavior but stop stripping fields that are explicitly nominated in the Connection header, set strip_connection_nominated = false.

3. Preserve HTTP/1 upgrade handshakes

To preserve complete HTTP/1 upgrade handshakes (including all metadata) while maintaining ordinary request normalization, set the h1_upgrade policy to H1UpgradePolicy::Preserve. Warning: This is RFC-non-compliant.

Security Constraints

When strip_connection_nominated is enabled, Pingora rejects requests if Connection nominates sensitive routing headers like Host, X-Forwarded-For, X-Forwarded-Host, X-Forwarded-Proto, or pseudo-headers like :authority. It also rejects requests with ten or more connection nominations.

// Preserve all passthrough behavior (RFC-non-compliant)
use pingora_core::upstreams::peer::HttpUpstreamRequestPolicy;
peer.options.http_upstream_request_policy = HttpUpstreamRequestPolicy::preserve();

// Retain only fields nominated in Connection
peer.options
    .http_upstream_request_policy
    .strip_connection_nominated = false;

// Preserve complete HTTP/1 upgrade handshakes (RFC-non-compliant)
use pingora_core::upstreams::peer::H1UpgradePolicy;
peer.options.http_upstream_request_policy.h1_upgrade = H1UpgradePolicy::Preserve;