Trino Python Client

repository·master·Indexed 19 days ago

https://github.com/trinodb/trino-python-client

A Python client for interacting with the Trino distributed SQL engine. It provides a low-level DBAPI 2.0 implementation, a SQLAlchemy adapter compatible with versions 1.3.x, 1.4.x, and 2.0.x, and a high-level TrinoQuery interface. The client supports various authentication methods including Basic, JWT, OAuth2, Certificate, Kerberos, and GSSAPI, as well as features like user impersonation, session property management via ClientSession, and the Trino spooling protocol.

Tokens
7.2K
Snippets
23
Records
30
Agent score
15%

What's inside trino-python-client

  1. Use Transactions with Isolation Levels

    master

    By default, the client operates in autocommit mode. To use transactions, set the isolation_level parameter to a value other than IsolationLevel.AUTOCOMMIT (e.g., IsolationLevel.REPEATABLE_READ).

    When using a with context manager:

    1. The transaction starts when the first SQL statement is executed.
    2. trino.dbapi.Connection.commit() is called automatically if the block exits successfully.
    3. trino.dbapi.Connection.rollback() is called if an exception occurs.
    from trino.dbapi import connect
    from trino.transaction import IsolationLevel
    
    with connect(
            isolation_level=IsolationLevel.REPEATABLE_READ,
            ...
    ) as conn:
        cur = conn.cursor()
        cur.execute('INSERT INTO sometable VALUES (1, 2, 3)')
        cur.fetchall()
        cur.execute('INSERT INTO sometable VALUES (4, 5, 6)')
        cur.fetchall()
  2. Implement User Impersonation and Extra Credentials

    master

    User Impersonation

    If the user submitting the query is different from the authenticated user, set user to the target username. The principal_id is automatically extracted from the auth object (e.g., username in BasicAuth, sub in JWT, or service-name in Kerberos). Ensure the principal_id has permission to impersonate the user in Trino.

    Extra Credentials

    You can send extra credentials as a list of tuples to trino.dbapi.connect using the extra_credential parameter.

    import trino
    conn = trino.dbapi.connect(
        host='localhost',
        port=443,
        user='the-user',
        extra_credential=[('a.username', 'bar'), ('a.password', 'foo')],
    )
    
    cur = conn.cursor()
    cur.execute('SELECT * FROM system.runtime.nodes')
    rows = cur.fetchall()
  3. Use the Trino DBAPI 2.0 interface

    master

    The trino.dbapi.connect function provides a standard Python DBAPI 2.0 implementation. If the host provided is a valid URL, the port and http_schema will be automatically determined (e.g., https://my-trino-server:9999 sets http_schema to https and port to 9999).

    To control how many rows are fetched at once, adjust trino.dbapi.Cursor.arraysize. By default, Cursor.fetchmany() fetches one row.

    from trino.dbapi import connect
    
    conn = connect(
        host="<host>",
        port=<port>,
        user="<username>",
        catalog="<catalog>",
        schema="<schema>",
    )
    cur = conn.cursor()
    cur.execute("SELECT * FROM system.runtime.nodes")
    rows = cur.fetchall()
  4. Use the Trino SQLAlchemy adapter

    master

    The trino.sqlalchemy adapter is compatible with SQLAlchemy 1.3.x, 1.4.x, and 2.0.x.

    Prerequisite: Trino server version must be >= 351.

    Connection String Format: trino://<username>:<password>@<host>:<port>/<catalog>/<schema> Note: password and schema are optional.

    You can pass additional connection attributes via the connect_args parameter in create_engine or directly in the connection string.

    from sqlalchemy import create_engine
    from sqlalchemy.schema import Table, MetaData
    from sqlalchemy.sql.expression import select, text
    
    # Basic connection
    engine = create_engine('trino://user@localhost:8080/system')
    connection = engine.connect()
    
    # Execute raw text
    rows = connection.execute(text("SELECT * FROM runtime.nodes")).fetchall()
    
    # Using SQLAlchemy schema
    nodes = Table(
        'nodes',
        MetaData(schema='runtime'),
        autoload=True,
        autoload_with=engine
    )
    rows = connection.execute(select(nodes)).fetchall()
  5. Understand the Trino Spooling Protocol segments

    master

    The Trino Spooling protocol breaks query results into segments. The client handles two types of segments via the SegmentType enum:

    • inline: Data is base64 encoded and included directly in the response. Use InlineSegment to access this.
    • spooled: Data is stored remotely (e.g., S3). The client must fetch the data via a URI and optionally acknowledge processing. Use SpooledSegment to handle these.

    Segments provide access to metadata (containing uncompressedSize and segmentSize) and the data itself.

  6. Configure SQLAlchemy connection arguments

    master

    When using SQLAlchemy, you can pass Trino-specific session properties, client tags, or roles using connect_args or the connection string URL.

    Using connect_args: Pass a dictionary to create_engine containing keys like session_properties, client_tags, and roles.

    from sqlalchemy import create_engine
    from trino.sqlalchemy import URL
    
    # Using URL object and connect_args
    engine = create_engine(
        URL(
            host="localhost",
            port=8080,
            catalog="system"
        ),
        connect_args={
          "session_properties": {'query_max_run_time': '1d'},
          "client_tags": ["tag1", "tag2"],
          "roles": {"catalog1": "role1"},
        }
    )
    
    # Using connection string with query parameters
    engine = create_engine(
        'trino://user@localhost:8080/system?'
        'session_properties={"query_max_run_time": "1d"}'
        '&client_tags=["tag1", "tag2"]'
        '&roles={"catalog1": "role1"}'
    )
    
    # Using URL factory method
    engine = create_engine(URL(
      host="localhost",
      port=8080,
      client_tags=["tag1", "tag2"]
    ))
  7. Authenticate with Kerberos or GSSAPI

    master

    For Kerberos authentication, install the extra: pip install trino[kerberos]. Use trino.auth.KerberosAuthentication.

    For GSSAPI authentication, install the extra: pip install trino[gssapi]. Use trino.auth.GSSAPIAuthentication. GSSAPI uses requests-gssapi under the hood.

    Both support DBAPI and SQLAlchemy via the auth parameter in connect() or connect_args.

    # Kerberos DBAPI Example
    from trino.dbapi import connect
    from trino.auth import KerberosAuthentication
    
    conn = connect(
        user="<username>",
        auth=KerberosAuthentication(...),
        http_scheme="https",
        ...
    )
    
    # GSSAPI SQLAlchemy Example
    from sqlalchemy import create_engine
    from trino.auth import GSSAPIAuthentication
    
    engine = create_engine(
        "trino://<username>@<host>:<port>/<catalog>",
        connect_args={
            "auth": GSSAPIAuthentication(...),
            "http_scheme": "https",
        }
    )
  8. Authenticate with OAuth2

    master

    Use trino.auth.OAuth2Authentication for OAuth2-configured clusters.

    Redirect Handlers: You can provide a redirect_auth_url_handler.

    • trino.auth.WebBrowserRedirectHandler (default): Launches a web browser.
    • trino.auth.ConsoleRedirectHandler: Outputs the URL to stdout.
    • trino.auth.CompositeRedirectHandler: Combines multiple handlers.

    Token Caching: Tokens are cached per instance/username. If keyring is installed (pip install 'trino[external-authentication-token-cache]'), tokens are stored in a secure backend (e.g., MacOS keychain).

    Warning: If user is not specified, the cache is shared per host.

    # DBAPI
    from trino.dbapi import connect
    from trino.auth import OAuth2Authentication
    
    conn = connect(
        user="<username>",
        auth=OAuth2Authentication(),
        http_scheme="https",
        ...
    )
    
    # SQLAlchemy
    from sqlalchemy import create_engine
    
    engine = create_engine(
        "trino://<username>@<host>:<port>/<catalog>",
        connect_args={
            "auth": OAuth2Authentication(),
            "http_scheme": "https",
        }
    )
  9. Authenticate with Basic Authentication

    master

    Use trino.auth.BasicAuthentication for clusters configured with Password file, LDAP, or Salesforce authentication.

    DBAPI: Pass the auth object to connect(). SQLAlchemy: Include credentials in the connection string or pass the auth object via connect_args.

    # DBAPI
    from trino.dbapi import connect
    from trino.auth import BasicAuthentication
    
    conn = connect(
        user="<username>",
        auth=BasicAuthentication("<username>", "<password>"),
        http_scheme="https",
        ...
    )
    
    # SQLAlchemy
    from sqlalchemy import create_engine
    
    engine = create_engine("trino://<username>:<password>@<host>:<port>/<catalog>")
    # OR via connect_args
    from trino.auth import BasicAuthentication
    engine = create_engine(
        "trino://<username>@<host>:<port>/<catalog>",
        connect_args={
            "auth": BasicAuthentication("<username>", "<password>"),
            "http_scheme": "https",
        }
    )
  10. Configure Authorization Roles

    master

    You can specify authorization roles for different catalogs using the roles parameter in trino.dbapi.connect().

    • To set specific roles for specific catalogs, pass a dictionary where keys are catalog names and values are the roles: {"catalog1": "roleA", "catalog2": "roleB"}.
    • To set a single role for the system catalog, pass a string: roles="role1" (equivalent to {"system": "role1"}).
    import trino
    
    # Specific roles per catalog
    conn = trino.dbapi.connect(
        host='localhost',
        port=443,
        user='the-user',
        roles={"catalog1": "roleA", "catalog2": "roleB"},
    )
    
    # Single role for the system catalog
    conn = trino.dbapi.connect(
        host='localhost',
        port=443,
        user='the-user',
        roles="role1"
    )
  11. Authenticate with Certificate Authentication

    master

    Use trino.auth.CertificateAuthentication for certificate-based authentication. This requires paths to a valid client certificate and private key.

    DBAPI: Pass the auth object to connect(). SQLAlchemy: Pass cert and key as query parameters in the connection string, or via connect_args.

    # DBAPI
    from trino.dbapi import connect
    from trino.auth import CertificateAuthentication
    
    conn = connect(
        user="<username>",
        auth=CertificateAuthentication("/path/to/cert.pem", "/path/to/key.pem"),
        http_scheme="https",
        ...
    )
    
    # SQLAlchemy
    from sqlalchemy import create_engine
    
    engine = create_engine("trino://<username>@<host>:<port>/<catalog>/<schema>?cert=<cert>&key=<key>")
    # OR via connect_args
    from trino.auth import CertificateAuthentication
    engine = create_engine(
        "trino://<username>@<host>:<port>/<catalog>",
        connect_args={
            "auth": CertificateAuthentication("/path/to/cert.pem", "/path/to/key.pem"),
            "http_scheme": "https",
        }
    )