minikube
repository·master·Indexed 12 days ago
https://github.com/kubernetes/minikubeA tool that runs a local Kubernetes cluster on macOS, Linux, and Windows, primarily intended for local Kubernetes application development. It includes support for various addons such as gVisor, KubeVirt, and registry-aliases, as well as performance testing tools like mkcmp and CPU usage benchmarks.
What's inside minikube
- minikube implements a local Kubernetes cluster on macOS, Linux, and Windows. It is designed to be a tool for local Kubernetes application development, supporting standard Kubernetes features such as LoadBalancers, NodePorts, Persistent Volumes, and various container runtimes.
Overview of minikube capabilities
masterminikube is a tool for quickly setting up a local Kubernetes cluster on macOS, Linux, and Windows. It is designed primarily for application developers and new Kubernetes users.
Key Capabilities:
- Kubernetes Versions: Supports the latest release plus up to 6 previous minor versions.
- Deployment Modes: Can be deployed as a Virtual Machine (VM), a container, or on bare-metal.
- Container Runtimes: Supports multiple Container Runtime Interfaces (CRI) including
CRI-O,containerd, anddocker. - AI/GPU Support: Supports GPU acceleration for AI development via
nvidia,amd, orappledrivers. - Advanced Networking & Storage: Supports
LoadBalanceraccess, filesystem mounts,FeatureGates, andnetwork policy. - Extensibility: Uses
Addonsto easily install common Kubernetes applications. - Developer Workflow: Provides a direct API endpoint for fast image loading and building.
Supported container runtimes in minikube
masterminikube supports several container runtimes to satisfy Kubernetes requirements. When starting a cluster, you can choose from the following supported runtimes:
- containerd
- cri-o
- docker
Each runtime has specific configuration requirements and behaviors. Refer to the specific documentation for the runtime you intend to use to ensure proper setup.
Access Kubernetes features in minikube
masterminikube provides specific commands to enable and access standard Kubernetes networking and management features:
- LoadBalancer access: Use
minikube tunnelto provide access to services of typeLoadBalancer. - NodePort access: Use
minikube serviceto access services of typeNodePort. - Dashboard: Use
minikube dashboardto launch the web-based Kubernetes user interface. - Multi-cluster support: Create and manage multiple independent clusters using the
-p(profile) flag:minikube start -p <name>. - Container Runtimes: Specify a runtime during startup using the
--container-runtimeflag:minikube start --container-runtime <runtime>.
# Example: Start a specific cluster profile minikube start -p my-cluster # Example: Access a LoadBalancer service minikube tunnel # Example: Open the dashboard minikube dashboard- LoadBalancer access: Use
Difference between Initial and Iterative benchmark modes
masterBenchmarks are categorized into two modes to simulate different real-world container loading scenarios:
Initial
Simulates the very first time an image is loaded. To replicate a clean environment, all existing images and caches are cleared from both
minikubeandDockerbetween every run.Iterative
Simulates a development workflow (similar to how Skaffold operates) where only the application code changes. In this mode, only the final layer (the Go binary) is updated, while the existing image and Docker cache are preserved between runs to measure the impact of incremental changes.
Use custom registry domains for container images
masterOnce the
registry-aliasesaddon is configured, you can use the following domains as container registry URLs in your deployment manifests (e.g.,skaffold.yamlor Kubernetes YAML files). These domains will resolve to the internal minikube registry:example.comexample.orgtest.comtest.org
To test this locally with Docker, ensure your shell is using the minikube docker daemon context:
eval $(minikube -p demo docker-env)eval $(minikube -p demo docker-env)Data loss warnings for the none driver
masterThe
nonedriver performs destructive operations on certain system paths:Overwritten on start:
/etc/kubernetes(configuration files)
Erased on
minikube delete:/data/minikube/etc/kubernetes/manifests/var/lib/minikube
Because Kubernetes has full access to your filesystem and docker images, unexpected data loss may occur.
Benchmark script components and workflow
masterThe CPU usage benchmark consists of three main components:
benchmark_local_k8s.sh: Executes the actual benchmark. It compares minikube drivers (e.g.,hyperkit,virtualbox,dockerwithauto-pauseaddon,kvm2) againstkind,k3d, andDocker for Mac Kubernetes(on macOS).update_summary.sh: Aggregates results into a summary CSV file for each driver and product.chart.go: Generates a bar chart graph as a PNG file from the results.
Benchmark Workflow
The
benchmark_local_k8s.shscript follows these steps to compareauto-pausevs.non auto-pausestates:- Start the local Kubernetes tool.
- Deploy a sample Nginx application.
- Wait 1 minute for stability.
- Measure idle CPU usage using
cstat(Initial state). - For minikube only: Enable the
auto-pauseaddon. - For minikube only: Wait 1 minute for the control plane to transition to
Pausedstatus. - For minikube only: Verify the control plane is indeed paused.
- For minikube only: Wait 3 minutes.
- For minikube only: Measure idle CPU usage using
cstat(Auto-paused state).
Understand how minikube reports test flake rates in PRs
masterTo help distinguish between new regressions and existing flaky tests, minikube automatically comments on Pull Requests (PRs) when integration tests fail.
When integration tests complete, a script named
report_flakes.shprocesses thegopoghsummaries and the publicgs://minikube-flake-rate/flake_rates.csvdataset. The resulting PR comment includes:- A list of all failed tests.
- The recorded flake rate for each failed test.
- Direct links to the flake charts for that specific test and environment.
This allows developers and reviewers to quickly determine if a failure is a known unreliable test (a flake) or a potential new bug introduced by the PR.
Understand the core goals and principles of minikube
masterminikube is designed to simplify running Kubernetes locally for day-to-day development workflows and learning. Its primary focus is providing a user-friendly, cross-platform, and reliable experience with minimal resource overhead and third-party dependencies.
Key features that support these goals include:
- Single command setup and teardown UX.
- Support for local storage, networking, auto-scaling, and load balancing.
- A unified UX across different operating systems.
Note on Non-Goals: minikube is not intended to simplify Kubernetes production deployment experiences, nor does it aim to support every possible Kubernetes deployment configuration (such as all variations of storage or networking types).
Reference minikube binaries in mkcmp
masterWhen running
mkcmp, you can specify the binaries to compare using two methods:- Direct Path: Provide the absolute or relative path to a specific minikube binary file.
- PR Reference: Use the syntax
pr://<PR number>to automatically use the minikube binary built during that specific Pull Request.
Limitations of minikube HA clusters
masterWhile a minikube HA cluster is designed to continue operating in a degraded mode if a single control-plane node is lost, certain components may not be fully redundant. Specifically, some components might be attached only to the primary control-plane node (for example, thestorage-provisioner).