DOCSEARCHbyAlgolia
Submit Docs

terraform-google-modules/terraform-google-kubernetes-engine

repository·main·Indexed 23 days ago

https://github.com/terraform-google-modules/terraform-google-kubernetes-engine

A Terraform module for deploying and managing Google Kubernetes Engine (GKE) clusters on GCP. It supports complex configurations including node pools, network policies, and IP masquerading. The module includes a Safer Cluster option for hardened setups following the GKE hardening guide and CIS framework, as well as integrations for GKE Workload Identity, Anthos Config Management (ACM) Config Sync, Policy Controller, and Config Connector.

Tokens
58K
Snippets
128
Records
329
Agent score
79%

What's inside terraform-google-kubernetes-engine

  1. Overview of the Terraform Google Kubernetes Engine Module

    main

    This module provides an opinionated way to create and configure Google Cloud Platform (GCP) Kubernetes Engine (GKE) clusters. It automates the creation of clusters, node pools, and several key configurations including:

    • GKE Cluster Creation: Provisioning the cluster with specified addons.
    • Node Pool Management: Creating and attaching one or more node pools with custom configurations.
    • DNS Configuration: Replacing the default kube-dns configmap if stub_domains are provided.
    • Network Policy: Activating network policy when network_policy is set to true.
    • IP Masquerade: Adding the ip-masq-agent configmap with non_masquerade_cidrs if configure_ip_masq is set to true.

    Sub-modules are available for specialized use cases, such as private clusters, beta private clusters, and beta public clusters, allowing access to GKE beta features.

  2. Overview of the GKE Standard Cluster submodule

    main

    The gke-standard-cluster submodule is designed to create a basic private cluster on Google Kubernetes Engine. This submodule has beta features enabled by default.

    Note: If you require a complete configuration that includes GKE Node Pools, DNS configuration, IP masq, Service account creation, and Firewall configuration, use the beta-private-cluster module instead.

  3. Use the Safer Cluster module for hardened GKE setups

    main
    The safer-cluster-update-variant module provides an opinionated GKE configuration designed to reduce exposure and follow security best practices, including the GKE hardening guide and CIS framework. It automates the setup of hardened cluster parameters, project configurations, and basic Kubernetes objects to minimize the blast radius of security incidents.

  4. Deploy a basic GKE Autopilot cluster

    main

    Use the gke-autopilot-cluster submodule to create a basic private cluster with Autopilot enabled. Note that this submodule enables beta features by default.

    If you require a complete configuration that includes the GKE cluster, IP masq, Service account creation, and Firewall configuration, use the beta-autopilot-private-cluster module instead.

  5. Configure GKE authentication with the Auth Module

    main

    The GKE Auth Module configures authentication to a GKE cluster using an OpenID Connect token retrieved from GCP. It can provide a kubeconfig file or specific outputs designed for use with the kubernetes and helm providers.

    The module retrieves a token for the account configured with the google provider used by the Terraform runner. If you are using a private cluster, set use_private_endpoint to true to return the GKE private endpoint IP address.

    module "gke_auth" {
  source               = "terraform-google-modules/kubernetes-engine/google//modules/auth"

  project_id           = "my-project-id"
  cluster_name         = "my-cluster-name"
  location             = module.gke.location
  use_private_endpoint = true
}

  7. Implement GKE Workload Identity for application service accounts

    main

    To limit the permissions of GKE nodes, use Workload Identity. This allows you to associate a specific GCP Service Account with a Kubernetes Service Account.

    Instead of giving nodes broad permissions, you create a dedicated GCP Service Account for each application and authorize the Kubernetes Service Account to impersonate it via an IAM binding.

    Pattern for Workload Identity Binding:

    1. Create a GCP Service Account for the application (e.g., myproduct-sa@project.iam.gserviceaccount.com).
    2. Create a binding that allows the Kubernetes Service Account [K8s-namespace]/[K8s-service-account-name] to use the role roles/iam.workloadIdentityUser on that GCP Service Account.
    # Example IAM Policy Binding for Workload Identity
email: myproduct
displayName: Google Service Account for containers running in the myproduct k8s namespace
policy:
  bindings:
  - members:
    - serviceAccount:product-prod.svc.id.goog[myproduct/default]
    role: roles/iam.workloadIdentityUser

  8. Handle ASM configuration changes in v20.0+

    main

    In versions v20.0 and later, the ASM module no longer provides variables for option configuration such as custom_overlay or options. These options were previously tied to the installer script used in older versions.

    To use these configurations with the new module, you must manage them separately outside the module by moving the configuration to the mesh configuration for the managed revision. You can find corresponding configuration patterns in the Anthos Service Mesh packages repository.

  9. ASM version and installation requirements in v14.0

    main

    The ASM submodule in version 14.0 defaults to Anthos Service Mesh (ASM) v1.8. The module utilizes a new ASM installation script with the following constraints:

    • Minimum Version: Does not support installation or upgrades for ASM versions older than 1.7.3.
    • Upgrade Path: Supports upgrades only from versions 1.7.3+ or a 1.8 patch release.
    • Istio Migration: Supports migrations from open source Istio 1.7 or 1.8 to ASM.

  10. Configure node pools for GKE Standard Cluster

    main

    The node_pool input allows you to define a list of node pools associated with the cluster.

    Warning: Node pools defined directly inside the cluster resource cannot be modified after creation without destroying and recreating the entire cluster. For production environments where you need to manage node pool lifecycles independently, it is highly recommended to use the google_container_node_pool resource instead of this property.

  11. Architectural patterns for secure GKE deployments

    main

    The Safer Cluster module follows several architectural patterns to reduce blast radius and improve security:

    Application Isolation

    • Different Applications: Place applications with different data sensitivity levels in separate clusters to limit blast radius.
    • Communicating Applications: Place communicating applications (e.g., frontend and backend) in the same cluster but in different namespaces for administrative isolation and fast networking.

    Project Separation

    • Cluster Projects: Use dedicated projects for different environments (e.g., separate projects for dev, staging, and prod).
    • Shared Registry Projects: Use a central project for GCR/Artifact Registry to simplify image sharing across environments.
    • Data Projects: For shared clusters, store application data in separate projects to allow application teams to manage their own Cloud IAM policies independently of the cluster infrastructure team.

    Identity Separation

    • Workload Identity: Use Workload Identity to separate node-level permissions from application-level permissions. This ensures that a compromised application does not automatically gain the permissions of the underlying GKE node.