Trino Gateway
repository·main·Indexed 42 days ago
https://github.com/trinodb/trino-gatewayA specialized proxy and load balancer for Trino environments that provides a centralized entry point to route queries across multiple Trino clusters based on configurable rules. It includes a web application built with React and TypeScript, and supports deployment via Docker and Kubernetes.
What's inside Trino Gateway
- Trino Gateway is a load balancer, proxy server, and configurable routing gateway designed for managing multiple Trino clusters. It provides a centralized entry point to route queries to various Trino clusters based on configurable rules.
Understand proxyTo vs externalURL for Trino clusters
mainEach Trino cluster configured in the Gateway has two distinct URL types:
proxyToURL: Used internally by the Trino Gateway to route requests based on routing rules and query identifiers. This is typically a service-to-service address (e.g., a Kubernetes service).externalURL: A publicly reachable or UI-accessible address. This is used by users to access the Trino Web UI.
Example (Kubernetes):
proxyTo:trino-backend-service.trino-namespace.svc.cluster.local:8083externalURL:trino.domain.com
Key features and use cases of Trino Gateway
mainTrino Gateway is used to solve several common Trino management challenges:
- Workload Distribution: Provides a single connection URL for client tools while distributing queries across multiple Trino clusters.
- Automatic Routing: Routes queries to specific Trino clusters based on workload requirements, specific query characteristics, or data sources.
- Zero-Downtime Upgrades: Enables blue/green or canary deployment models for Trino clusters, allowing upgrades without interrupting users.
- Capacity Management: Allows for transparent changes to Trino cluster capacity without user interruption.
Use an external service for routing rules
mainWhen
rulesTypeis set toEXTERNAL, Trino Gateway sends a POST request to the configuredurlPath. The body contains a map of all request headers (excluding those inexcludeHeaders). IfrequestAnalyzerConfig.analyzeRequestistrue, the body also includesTrinoRequestUserandTrinoQueryPropertiesdata.Included HTTP Information:
remoteUser,method,requestURI,queryString,session,remoteAddr,remoteHost,parameterMap.
External Service Response Requirements:
- Status code must be
200 OK. - Response must be a JSON object.
- Must return exactly one
routingGroup. - If the
errorsfield is not null, the query is routed to the default group.
Dynamic Header Modification: The service can return an
externalHeadersmap to inject or modify headers (like session properties or client tags) before the request reaches the Trino cluster.{ "routingGroup": "test-group", "errors": [ "Error1", "Error2", "Error3" ], "externalHeaders": { "x-trino-client-tags": "['etl']", "x-trino-session": "query_max_memory=50GB,optimize_metadata_queries=false" } }Understand Trino Gateway routing logic
mainTrino Gateway determines how to route incoming requests using two main approaches:
- Related Requests (Sticky Routing): If a request is identified as related to a previous one, it is sent to the same Trino cluster that handled the earlier request. This ensures stateful operations or ongoing processes are not interrupted.
- New Requests: For requests with no existing relationship, the Gateway:
- Consults Routing rules to find the appropriate 'Routing Group'.
- Selects a cluster from that group using either an adaptive or round-robin strategy.
- Only selects from clusters that are currently healthy.
Note: Only healthy clusters are eligible for routing.
Understand the built-in routing mechanisms
mainTrino Gateway provides two entry-level routers that make decisions based on cluster load reported in
ClusterStats.- StochasticRoutingManager: A decentralized approach that dispatches incoming queries randomly without advanced optimization.
- QueryCountBasedRouterProvider: A more advanced mechanism using near real-time, user-level cluster load statistics. It tracks running query counts and query queue lengths to direct queries to the least loaded cluster for a specific user, optimizing for successful execution.
Configure Certificate Identity and User Mapping
mainWhen using client certificates, you must define how the certificate identity is extracted via
requestAnalyzerConfig.clientCertificateIdentityField.SUBJECT_DN: Matches Trino's native certificate authentication by mapping the whole subject DN. Use this if you want to copy existing Trino user-mapping rules (http-server.authentication.certificate.user-mapping.*) directly to the gateway.- Other RDNs (e.g.,
CN,OU): The gateway extracts the specific RDN value from the subject DN. Note: Rules written for Trino (which expect a full DN) will not work in this mode. You must rewrite your mapping rules to match the extracted value instead of the full DN.
Constraint: If your mapping file contains rules with
"allow": false, you must useclientCertificateIdentityField: SUBJECT_DN.Configure Trino cluster proxy settings
mainTrino Gateway acts as a transparent proxy. To ensure correct routing and authentication (especially when TLS is terminated at the Gateway), you must configure the proxied Trino clusters:
- Enable Process Forwarded Headers: In the Trino cluster configuration, set
http-server.process-forwarded=true. This ensures subsequent requests use the Gateway's URL instead of the cluster's local URL. - Disable Gateway Header Forwarding (Optional): If you want to prevent Trino Gateway from sending
X-Forwarded-*andForwardedheaders, setrouting.forwardedHeadersEnabled: falsein the Gateway configuration.
# In Trino cluster configuration http-server.process-forwarded=true# In Trino Gateway configuration routing: forwardedHeadersEnabled: false- Enable Process Forwarded Headers: In the Trino cluster configuration, set
How Trino cluster health checks work
mainTrino Gateway performs periodic health checks to maintain an in-memory
TrinoStatusfor each backend. If a backend fails a check, it is marked asUNHEALTHYand routing stops.Note:
TrinoStatus(programmatic health) is distinct from the active/inactive state (manual toggle).Control rule execution with priorities and state
mainRules are evaluated in order. If multiple rules match, the last one to execute its
result.put("routingGroup", ...)action wins. To manage complex logic, use these two mechanisms:- Rule Priority: Assign an integer
priorityto a rule. Lower integers fire earlier. Use higher priorities for more specific rules that should override general rules. - Passing State: Use the
stateobject to store information in an early rule (e.g.,state.put("is_airflow", true)) and check it in a later rule (e.g.,condition: 'state.get("is_airflow") == true').
Example: Using Priority and State
--- name: "initialize state" priority: 0 condition: "true" actions: - | state.put("triggeredRules", new HashSet()) --- name: "airflow" priority: 1 condition: | request.getHeader("X-Trino-Source") == "airflow" actions: - | result.put("routingGroup", "etl") - | state.get("triggeredRules").add("airflow") --- name: "airflow special" priority: 2 condition: | state.get("triggeredRules").contains("airflow") && request.getHeader("X-Trino-Client-Tags") contains "label=special" actions: - 'result.put("routingGroup", "etl-special")'- Rule Priority: Assign an integer
Understand the request forwarding flow and header behavior
mainWhen Trino Gateway forwards a request to a Trino cluster, it follows these steps:
- Routing: Determines the target Trino cluster for the query.
- Header Preparation:
- By default, most headers are forwarded unmodified, and the Gateway appends
X-Forwarded-*proxy headers. - Security/Direct Routing: If you set
routing.forwardedHeadersEnabledtofalsein theProxyRequestHandlerconfiguration, the Gateway will case-insensitively strip all client-provided legacy (X-Forwarded-*) and modern (Forwarded) headers. It will also skip adding its own proxy headers. This configuration is useful when you want Trino backend responses (likenextUrivalues) to point directly to the backend rather than back through the Gateway or load balancer.
- By default, most headers are forwarded unmodified, and the Gateway appends
- Special Handling: Certain URIs require specific logic, such as retrieving a
queryIdfrom a Trino response or setting session cookies for OIDC compatibility in the web UI. - Execution: The actual execution of the request to Trino and the subsequent response to the client are managed by
airlift.jaxrs.AsyncResponseHandler.
Understand changes to service ports and endpoints
mainIn versions 10+, multiple service ports have been merged into a single HTTP port (default 8080).
Port Consolidation:
- Old behavior: Separate ports for request service (9080), application service (9081), and Admin service (9082).
- New behavior: A single port (default 8080) serves both Trino Gateway (Web UI and RESTful API) and requests forwarded to Trino. The destination is determined by the HTTP URI.
Admin Service & JMX:
- The Dropwizard admin service at port 9082 has been removed.
- JMX can now be accessed at
/v1/jmx.
WebUI Path Change:
- The root path for the WebUI has changed from
/to/trino-gateway. - By default,
/redirects to/trino-gateway. - To route
/directly to Trino instead of the WebUI, add/toextraWhitelistPathsusing regex format.