DOCSEARCHbyAlgolia
Submit Docs

Google Kubernetes Engine Samples

repository·main·Indexed 23 days ago

https://github.com/googlecloudplatform/kubernetes-engine-samples

A collection of sample applications and implementation patterns for Google Kubernetes Engine (GKE). Includes demonstrations for agentic AI workloads using the Agentic Development Kit (ADK) with Vertex AI or vLLM, and a secure Anthropic Agent Sandbox utilizing gVisor isolation, the GKE Agent Sandbox add-on, and the Anthropic CLI.

Tokens
78.1K
Snippets
202
Records
383
Agent score
80%

What's inside kubernetes-engine-samples

  1. Overview of the Distributed Tracing example

    main
    This example demonstrates how to build, instrument, and deploy a containerized Go web server to a Kubernetes cluster using OpenTelemetry for distributed tracing. The application architecture consists of a frontend and a backend service, where the frontend makes requests to the backend, which in turn makes outbound requests to an external destination (e.g., google.com).

  3. Overview of the Workload Recommender Module

    main

    The Workload Recommender Module provides tools to recommend Horizontal Pod Autoscaler (HPA) or static Vertical Pod Autoscaler (VPA) configurations for Kubernetes workloads in Google Kubernetes Engine (GKE). It aims to balance cost efficiency and reliability by evaluating workloads using historical metric data.

    Key Capabilities:

    • Fetches and aggregates CPU and memory metrics from Cloud Monitoring.
    • Calculates workload startup time (including pod initialization and cluster autoscaler delays).
    • Simulates resource scaling using Dynamic Minimum Replicas (DMR) and Dynamic CPU Requests (DCR) algorithms.
    • Generates specific resource recommendations for both HPA and VPA.

    Note: This solution is currently tested only for Kubernetes Deployments.

  4. Overview of the Prometheus dummy exporter

    main

    The prometheus-dummy-exporter is a simple container designed to expose a single Prometheus metric with a constant value. It is intended for testing and demonstrating how metrics are collected and used in GKE.

    Key features:

    • Configurable Metrics: You can specify the metric name, the metric value, and the port via command-line flags.
    • Cloud Monitoring Integration: The metrics are exported to Google Cloud Monitoring using Google Cloud Managed Service for Prometheus via a PodMonitoring resource.

  5. Overview of whereami

    main

    whereami is a Python-based Kubernetes application designed to describe the location and attributes of the Pod serving a request. It returns metadata such as the cluster name, region, pod name, namespace, and service account. This is primarily used for demonstrations to visualize how traffic flows through a cluster.

    It supports two modes:

    • Flask (HTTP): The default mode.
    • gRPC: An alternative mode for gRPC-based communication.

    Observability

    • Tracing: In Flask mode, it is instrumented with OpenTelemetry and exports traces to Cloud Trace when running on GCP. You can configure the sampling likelihood via the TRACE_SAMPLING_RATIO environment variable in the ConfigMap.
    • Metrics: Prometheus metrics are exposed at x.x.x.x/metrics. In gRPC mode, this endpoint is exposed on port 8000 via HTTP.

    Note: If using Workload Identity, ensure the associated Google Service Account (GSA) has permissions to write to Cloud Trace (e.g., roles/cloudtrace.agent).

  6. Overview of the Stackdriver dummy exporter

    main

    The sd-dummy-exporter is a simple container designed to export a metric with a constant value to Stackdriver (Google Cloud Observability) in a continuous loop. This is useful for testing observability pipelines or custom metrics autoscaling.

    Key configuration capabilities:

    • Metric Name: Set via flags.
    • Metric Value: Set via flags.
    • Pod Name: Set via flags.
    • Namespace: Set via flags.

  9. Prepare data and batch-predict on TPU with Ray Data

    main

    This sample demonstrates two primary workflows using Ray Data for TPU-based machine learning tasks:

    1. Data Preparation (prepare_preference_data.py): A CPU-distributed job that streams a raw preference dataset (e.g., from HuggingFace), reshapes it into the specific columns required by Tunix's DPOTrainer (prompts, chosen_responses, rejected_responses), and writes the cleaned JSONL output to Google Cloud Storage (GCS).

    2. Batch Prediction (batch_predict_tpu.py): An offline inference job that streams prompts through a model on a TPU slice. It uses iter_jax_batches to yield batches as device-sharded JAX arrays directly to TPU hosts, allowing the CPU-based tokenization pipeline to overlap with the TPU forward pass. Predictions are written back to GCS.

    This sample is designed to run against the cluster configured in the ../cluster/ directory.

  12. Integrate Model Armor guardrails with GKE

    main

    This sample demonstrates how to secure a Large Language Model (LLM) running on Google Kubernetes Engine (GKE) by using Model Armor to inspect traffic and defend against harmful inputs and outputs.

    Key components of the implementation include:

    • Infrastructure: Provisioning a GKE cluster equipped with NVIDIA L4 GPUs and Hyperdisk ML storage.
    • Model Serving: Using vLLM to serve the Gemma 1.1 7b-it model.
    • Networking: Deploying a GKE Gateway for regional external load balancing.
    • Security: Implementing a security checkpoint via GKE Service Extensions to apply Model Armor policies that filter both prompts (inputs) and responses (outputs).