Trino Gateway

repository·main·Indexed 42 days ago

https://github.com/trinodb/trino-gateway

A specialized proxy and load balancer for Trino environments that provides a centralized entry point to route queries across multiple Trino clusters based on configurable rules. It includes a web application built with React and TypeScript, and supports deployment via Docker and Kubernetes.

Tokens
21K
Snippets
74
Records
112
Agent score
53%

What's inside Trino Gateway

  1. Overview of Trino Gateway

    main
    Trino Gateway is a load balancer, proxy server, and configurable routing gateway designed for managing multiple Trino clusters. It provides a centralized entry point to route queries to various Trino clusters based on configurable rules.
  2. Understand proxyTo vs externalURL for Trino clusters

    main

    Each Trino cluster configured in the Gateway has two distinct URL types:

    1. proxyTo URL: Used internally by the Trino Gateway to route requests based on routing rules and query identifiers. This is typically a service-to-service address (e.g., a Kubernetes service).
    2. externalURL: A publicly reachable or UI-accessible address. This is used by users to access the Trino Web UI.

    Example (Kubernetes):

    • proxyTo: trino-backend-service.trino-namespace.svc.cluster.local:8083
    • externalURL: trino.domain.com
  3. Key features and use cases of Trino Gateway

    main

    Trino Gateway is used to solve several common Trino management challenges:

    • Workload Distribution: Provides a single connection URL for client tools while distributing queries across multiple Trino clusters.
    • Automatic Routing: Routes queries to specific Trino clusters based on workload requirements, specific query characteristics, or data sources.
    • Zero-Downtime Upgrades: Enables blue/green or canary deployment models for Trino clusters, allowing upgrades without interrupting users.
    • Capacity Management: Allows for transparent changes to Trino cluster capacity without user interruption.
  4. Use an external service for routing rules

    main

    When rulesType is set to EXTERNAL, Trino Gateway sends a POST request to the configured urlPath. The body contains a map of all request headers (excluding those in excludeHeaders). If requestAnalyzerConfig.analyzeRequest is true, the body also includes TrinoRequestUser and TrinoQueryProperties data.

    Included HTTP Information:

    • remoteUser, method, requestURI, queryString, session, remoteAddr, remoteHost, parameterMap.

    External Service Response Requirements:

    • Status code must be 200 OK.
    • Response must be a JSON object.
    • Must return exactly one routingGroup.
    • If the errors field is not null, the query is routed to the default group.

    Dynamic Header Modification: The service can return an externalHeaders map to inject or modify headers (like session properties or client tags) before the request reaches the Trino cluster.

    {
        "routingGroup": "test-group",
        "errors": [
            "Error1",
            "Error2",
            "Error3"
        ],
        "externalHeaders": {
            "x-trino-client-tags": "['etl']",
            "x-trino-session": "query_max_memory=50GB,optimize_metadata_queries=false"
        }
    }
  5. Understand Trino Gateway routing logic

    main

    Trino Gateway determines how to route incoming requests using two main approaches:

    1. Related Requests (Sticky Routing): If a request is identified as related to a previous one, it is sent to the same Trino cluster that handled the earlier request. This ensures stateful operations or ongoing processes are not interrupted.
    2. New Requests: For requests with no existing relationship, the Gateway:
      • Consults Routing rules to find the appropriate 'Routing Group'.
      • Selects a cluster from that group using either an adaptive or round-robin strategy.
      • Only selects from clusters that are currently healthy.

    Note: Only healthy clusters are eligible for routing.

  6. Understand the built-in routing mechanisms

    main

    Trino Gateway provides two entry-level routers that make decisions based on cluster load reported in ClusterStats.

    1. StochasticRoutingManager: A decentralized approach that dispatches incoming queries randomly without advanced optimization.
    2. QueryCountBasedRouterProvider: A more advanced mechanism using near real-time, user-level cluster load statistics. It tracks running query counts and query queue lengths to direct queries to the least loaded cluster for a specific user, optimizing for successful execution.
  7. Configure Certificate Identity and User Mapping

    main

    When using client certificates, you must define how the certificate identity is extracted via requestAnalyzerConfig.clientCertificateIdentityField.

    • SUBJECT_DN: Matches Trino's native certificate authentication by mapping the whole subject DN. Use this if you want to copy existing Trino user-mapping rules (http-server.authentication.certificate.user-mapping.*) directly to the gateway.
    • Other RDNs (e.g., CN, OU): The gateway extracts the specific RDN value from the subject DN. Note: Rules written for Trino (which expect a full DN) will not work in this mode. You must rewrite your mapping rules to match the extracted value instead of the full DN.

    Constraint: If your mapping file contains rules with "allow": false, you must use clientCertificateIdentityField: SUBJECT_DN.

  8. Configure Trino cluster proxy settings

    main

    Trino Gateway acts as a transparent proxy. To ensure correct routing and authentication (especially when TLS is terminated at the Gateway), you must configure the proxied Trino clusters:

    1. Enable Process Forwarded Headers: In the Trino cluster configuration, set http-server.process-forwarded=true. This ensures subsequent requests use the Gateway's URL instead of the cluster's local URL.
    2. Disable Gateway Header Forwarding (Optional): If you want to prevent Trino Gateway from sending X-Forwarded-* and Forwarded headers, set routing.forwardedHeadersEnabled: false in the Gateway configuration.
    # In Trino cluster configuration
    http-server.process-forwarded=true
    # In Trino Gateway configuration
    routing:
      forwardedHeadersEnabled: false
  9. How Trino cluster health checks work

    main

    Trino Gateway performs periodic health checks to maintain an in-memory TrinoStatus for each backend. If a backend fails a check, it is marked as UNHEALTHY and routing stops.

    Note: TrinoStatus (programmatic health) is distinct from the active/inactive state (manual toggle).

  10. Control rule execution with priorities and state

    main

    Rules are evaluated in order. If multiple rules match, the last one to execute its result.put("routingGroup", ...) action wins. To manage complex logic, use these two mechanisms:

    1. Rule Priority: Assign an integer priority to a rule. Lower integers fire earlier. Use higher priorities for more specific rules that should override general rules.
    2. Passing State: Use the state object to store information in an early rule (e.g., state.put("is_airflow", true)) and check it in a later rule (e.g., condition: 'state.get("is_airflow") == true').

    Example: Using Priority and State

    ---
    name: "initialize state"
    priority: 0
    condition: "true"
    actions:
      - | 
        state.put("triggeredRules", new HashSet())
    ---
    name: "airflow"
    priority: 1
    condition: |
      request.getHeader("X-Trino-Source") == "airflow"
    actions:
      - | 
        result.put("routingGroup", "etl")
      - | 
        state.get("triggeredRules").add("airflow")
    ---
    name: "airflow special"
    priority: 2
    condition: |
      state.get("triggeredRules").contains("airflow") && request.getHeader("X-Trino-Client-Tags") contains "label=special"
    actions:
      - 'result.put("routingGroup", "etl-special")'
  11. Understand the request forwarding flow and header behavior

    main

    When Trino Gateway forwards a request to a Trino cluster, it follows these steps:

    1. Routing: Determines the target Trino cluster for the query.
    2. Header Preparation:
      • By default, most headers are forwarded unmodified, and the Gateway appends X-Forwarded-* proxy headers.
      • Security/Direct Routing: If you set routing.forwardedHeadersEnabled to false in the ProxyRequestHandler configuration, the Gateway will case-insensitively strip all client-provided legacy (X-Forwarded-*) and modern (Forwarded) headers. It will also skip adding its own proxy headers. This configuration is useful when you want Trino backend responses (like nextUri values) to point directly to the backend rather than back through the Gateway or load balancer.
    3. Special Handling: Certain URIs require specific logic, such as retrieving a queryId from a Trino response or setting session cookies for OIDC compatibility in the web UI.
    4. Execution: The actual execution of the request to Trino and the subsequent response to the client are managed by airlift.jaxrs.AsyncResponseHandler.
  12. Understand changes to service ports and endpoints

    main

    In versions 10+, multiple service ports have been merged into a single HTTP port (default 8080).

    Port Consolidation:

    • Old behavior: Separate ports for request service (9080), application service (9081), and Admin service (9082).
    • New behavior: A single port (default 8080) serves both Trino Gateway (Web UI and RESTful API) and requests forwarded to Trino. The destination is determined by the HTTP URI.

    Admin Service & JMX:

    • The Dropwizard admin service at port 9082 has been removed.
    • JMX can now be accessed at /v1/jmx.

    WebUI Path Change:

    • The root path for the WebUI has changed from / to /trino-gateway.
    • By default, / redirects to /trino-gateway.
    • To route / directly to Trino instead of the WebUI, add / to extraWhitelistPaths using regex format.