DOCSEARCHbyAlgolia
Submit Docs

Cloud SQL Auth Proxy Documentation

repository·main·Indexed 23 days ago

https://github.com/googlecloudplatform/cloud-sql-proxy

A utility providing secure, encrypted, and IAM-authorized connections to Google Cloud SQL instances without manual certificate management. Documentation covers v2 configuration, including Kubernetes sidecar deployment, HTTP health checks (/startup, /liveness, /readiness), disaster recovery automation using Secret Manager, and integration with PgBouncer for connection pooling.

Tokens
20.6K
Snippets
47
Records
83
Agent score
76%

What's inside Cloud SQL Auth Proxy

  1. Overview of Cloud SQL Auth Proxy

    main

    The Cloud SQL Auth Proxy is a utility designed to provide secure connections to Cloud SQL instances. It handles IAM authorization (allowing you to control access via IAM permissions) and provides TLS 1.3 encryption without requiring manual certificate management.

    Key Features:

    • Automatic IAM Authentication (for Postgres and MySQL only).
    • Support for Metrics (Cloud Monitoring, Cloud Trace, and Prometheus).
    • HTTP Healthchecks.
    • Service account impersonation.
    • Configuration via environment variables and POSIX-compliant flags.

    Important Limitations & Compatibility:

    • Go Compatibility Warning: Go versions 1.25.2 and 1.24.8 are NOT compatible because they break SAN verification due to how they handle trailing dots in DNS names.
    • MySQL 8.4 Support: The proxy does not currently support Unix domain socket connections to MySQL 8.4 instances due to the caching_sha2_password authentication plugin.
    • Network Connectivity: The proxy does not configure the network. You must ensure the environment where the proxy runs has network access to the Cloud SQL instance (via VPC for Private IP or via Public IP configuration).

  2. Configure Cloud SQL Proxy credentials and IAM roles

    main

    The Proxy uses Application Default Credentials to authorize connections.

    Required IAM Roles

    Any IAM principal connecting to a Cloud SQL database must have one of the following roles:

    • Cloud SQL Client (preferred)
    • Cloud SQL Editor
    • Cloud SQL Admin

    Alternatively, you can manually assign these permissions:

    • cloudsql.instances.connect
    • cloudsql.instances.get

    Authentication Flags

    • --token: Use an OAuth2 token.
    • --credentials-file: Use a service account key file.
    • --impersonate-service-account: Act as a different service account. The impersonating principal needs roles/iam.serviceAccountTokenCreator on the target account. The target account must have Service Usage Consumer and Cloud SQL Client permissions.

  3. Enable and configure HTTP health checks

    main

    When the --health-check flag is enabled (or via the CSQL_PROXY_HEALTH_CHECK environment variable), the proxy starts an HTTP server on localhost (or the configured address) providing three endpoints for Kubernetes probes:

    • /startup: Returns 200 when the proxy has finished starting up; otherwise returns 503.
    • /liveness: Always returns 200. If this endpoint is not responding, the proxy is in a bad state and should be restarted.
    • /readiness: Returns 200 when the proxy has started, has available connections (if --max-connections is set), and can connect to all registered instances. Otherwise, returns 503.

    To configure the server, use the following flags or environment variables:

    • --http-address / CSQL_PROXY_HTTP_ADDRESS: The address to listen on (e.g., 0.0.0.0 to allow Kubernetes management components to reach the endpoints).
    • --http-port / CSQL_PROXY_HTTP_PORT: The port for the HTTP server (default is not specified, but 9801 is used in examples).

  4. Use DNS for instance identification and automatic failover

    main

    The Proxy can use DNS names instead of INSTANCE_CONNECTION_NAME to identify instances. This is useful for Advanced Disaster Recovery or custom domain management.

    Setup via TXT Records

    Create a private TXT record where the name is your desired domain and the value is the INSTANCE_CONNECTION_NAME:

    • Record Type: TXT
    • Name: prod-db.mycompany.example.com
    • Value: my-project:region:my-instance

    Usage

    ./cloud-sql-proxy prod-db.mycompany.example.com

    Automatic Failover

    When configured with a domain name, the Proxy periodically polls the DNS record (every 30 seconds by default). If the TXT record value changes, the Proxy closes existing connections to the old instance and directs new connection attempts to the new instance defined in the record.

  5. How the Cloud SQL Auth Proxy works

    main

    The Cloud SQL Auth Proxy is a utility that authorizes and encrypts connections to Cloud SQL.

    Core Mechanism: For every provided instance connection name, the Proxy creates:

    1. A socket that mimics a database running locally.
    2. An encrypted connection using TLS 1.3 back to your Cloud SQL instance.

    Security & Authentication:

    • IAM Authorization: It provides IAM authorization, allowing you to control access via IAM permissions.
    • Encryption: It uses TLS 1.3 encryption without requiring manual certificate management. It uses ephemeral certificates that are refreshed hourly. Existing client connections are not affected by the refresh cycle.

    Network Requirements: The Proxy does not configure the network. You must ensure the Proxy can reach your Cloud SQL instance (e.g., via a VPC with access to a Private IP instance or by configuring Public IP).

  6. Use Cloud SQL Connectors instead of the Proxy

    main

    If your application is written in Go, Java, Python, or Node.js, you can use the language-specific Cloud SQL connectors. These connectors provide the same functionality as the Proxy but run in-process within your application, eliminating the need to manage a separate proxy binary.

  7. Enable and use Health Check endpoints

    main

    By enabling the --health-check flag, the Proxy starts an HTTP server on localhost (configurable via --http-address and --http-port) with three endpoints used for monitoring lifecycle and readiness:

    • /startup: Returns 200 when the Proxy has finished starting up; otherwise 503.
    • /readiness: Returns 200 when the Proxy has started, can connect to all registered instances, and has available connections (if --max-connections is set); otherwise 503.
    • /liveness: Always returns 200. If this endpoint does not respond, the Proxy is in a bad state and should be restarted.

    Example: Enabling health checks

    ./cloud-sql-proxy --health-check my-project:us-central1:my-db-server

  9. Set up Cloud SQL Auth Proxy via environment variables in Kubernetes

    main

    Instead of using CLI flags, you can manage the proxy configuration using environment variables in your Kubernetes deployment. This is the recommended approach when using the cloud-sql-proxy-operator.

    Key environment variables:

    • CSQL_PROXY_HEALTH_CHECK: Set to "true" to enable HTTP health checks.
    • CSQL_PROXY_HTTP_PORT: The port for health checks (e.g., "9801").
    • CSQL_PROXY_HTTP_ADDRESS: Set to 0.0.0.0 so Kubernetes can reach the endpoints.
    • CSQL_PROXY_PORT: The port the proxy opens for database connections.
    • CSQL_PROXY_EXIT_ZERO_ON_SIGTERM: Set to "true" to ensure graceful exits when Kubernetes sends a SIGTERM.
    env:
- name: CSQL_PROXY_PORT
  value: <DB_PORT>
- name: CSQL_PROXY_HEALTH_CHECK
  value: "true"
- name: CSQL_PROXY_HTTP_PORT
  value: "9801"
- name: CSQL_PROXY_HTTP_ADDRESS
  value: 0.0.0.0
- name: CSQL_PROXY_EXIT_ZERO_ON_SIGTERM
  value: "true"

  10. Start the Cloud SQL Auth Proxy

    main

    To start the Proxy, you need your Cloud SQL instance connection name. You can find this in the Cloud SQL instance overview page or by using gcloud:

    gcloud sql instances describe INSTANCE --format='value(connectionName)'

    Run the Proxy by passing the connection name as an argument. By default, the Proxy determines the database engine and starts a listener on localhost using the default port (MySQL: 3306, Postgres: 5432, SQL Server: 1433).

    ./cloud-sql-proxy my-project:us-central1:my-db-server

  11. Persist cloud-sql-proxy completions for all zsh sessions

    main

    To make autocompletion available automatically every time you open a new zsh shell, you must write the completion script to a directory in your $fpath. The command depends on your operating system:

    Linux:

    cloud-sql-proxy completion zsh > "${fpath[1]}/_cloud-sql-proxy"

    macOS (via Homebrew):

    cloud-sql-proxy completion zsh > $(brew --prefix)/share/zsh/site-functions/_cloud-sql-proxy

    Note: You must start a new shell session for these changes to take effect.

    # Linux
cloud-sql-proxy completion zsh > "${fpath[1]}/_cloud-sql-proxy"

# macOS
cloud-sql-proxy completion zsh > $(brew --prefix)/share/zsh/site-functions/_cloud-sql-proxy

  12. Deploy PgBouncer with Cloud SQL Auth Proxy as a sidecar

    main

    This pattern deploys PgBouncer and the Cloud SQL Auth Proxy in the same Kubernetes pod. The proxy acts as a sidecar, listening on a port (e.g., 5431) and forwarding traffic to the Cloud SQL instance, while PgBouncer manages connection pooling.

    Key Configuration Steps:

    1. Mount Secrets: Mount the CA and server certificates into the PgBouncer container.
    2. Configure PgBouncer: Use environment variables to set database credentials and TLS settings. Set DB_PORT to a port other than 5432 (e.g., 5431) to avoid conflict with the service port.
    3. Add Proxy Sidecar: Add the cloud-sql-proxy as an initContainer (or container) in the deployment, passing the --port flag to match the DB_PORT used by PgBouncer.
    # PgBouncer Volume Mounts
volumeMounts:
- name: pgbouncer
  image: <PG-BOUNCER-CONTAINER>
  ports:
  - containerPort: 5432
  volumeMounts:
  - name: cacert
    mountPath: "/etc/ca"
    readOnly: true
  - name: servercert
    mountPath: "/etc/server"
    readOnly: true

# PgBouncer Environment Variables
env:
- name: DB_HOST
  value: "127.0.0.1"
- name: DB_PORT
  value: "5431"
- name: CLIENT_TLS_SSLMODE
  value: "require"
- name: CLIENT_TLS_CA_FILE
  value: "/etc/ca/cert.pem"
- name: CLIENT_TLS_KEY_FILE
  value: "/etc/server/key.pem"
- name: CLIENT_TLS_CERT_FILE
  value: "/etc/server/cert.pem"

# Cloud SQL Proxy Sidecar
initContainers:
- name: cloud-sql-proxy
  restartPolicy: Always
  image: gcr.io/cloud-sql-connectors/cloud-sql-proxy:2.24.1
  args:
    - "--port=<DB_PORT>"
    - "<INSTANCE_CONNECTION_NAME>"
  securityContext:
    runAsNonRoot: true