Akash Provider

repository·main·Indexed 41 days ago

https://github.com/akash-network/provider

A daemon that bridges the Akash blockchain with physical compute resources, currently Kubernetes. It automates bidding on marketplace orders and deploying workloads to clusters. The provider supports NGINX Ingress and the Kubernetes Gateway API for ingress management.

Tokens
33.2K
Snippets
146
Records
179
Agent score
50%

What's inside Akash Provider

  1. Overview of the Akash Provider Daemon

    main

    The Akash Provider is a critical component of the Akash Network that offers compute resources to the decentralized marketplace. It functions by listening to events emitted from the Akash blockchain and executing actions on a connected Kubernetes cluster. Its primary responsibilities include:

    • Resource Bidding: Managing the lifecycle of bids based on provider configuration.
    • Deployment Management: Handling user deployments based on provided manifests.
    • Workload Orchestration: Managing compute capacity on connected clusters.

    The daemon provisions compute capacity specifically for the bids that the configured provider key wins.

  2. Understand the Kubernetes cluster configuration variants

    main

    The provider uses three main methods to set up a Kubernetes cluster, which are organized into configuration variants within the _run directory:

    • kube: Uses kind to set up a local cluster. The node and provider run as host services, while operators run as Kubernetes deployments. This is the standard for e2e testing.
    • single: Uses kind to set up a local cluster where both the node, provider, and all operators run as deployments within the cluster.
    • ssh: Expects a cluster to be already configured and running. This is used for testing advanced features like GPU or IP leases.
    • minikube: Currently not in use.

    All commands are implemented as make targets. If you need to reset a cluster, navigate to the specific runbook directory and run:

    make kube-cluster-delete
    make clean
  3. How the Akash Provider Daemon is structured

    main

    The provider daemon is composed of several specialized packages:

    • bidengine: Queries on-chain orders and places bids based on configured selling prices. It supports dynamic configuration updates without requiring a daemon restart.
    • cluster: Manages the compute resources. Currently, it supports kubernetes as the backend for deploying orders based on manifests.
    • cmd: The entry point of the application, providing a cobra-based CLI utility to run the daemon.
    • event: Defines the pubsub events (such as won leases and received manifests) that trigger daemon actions.
    • gateway: Implements the REST server exposed by the provider.
    • manifest: Handles the logic related to deployment manifests.
  4. How the Inventory operator works

    main

    The Inventory operator automates the process of labeling and annotating nodes with specific features (like GPUs or persistent storage), removing the need for manual configuration. It consists of two components:

    1. Inventory Service: Scrapes cluster resource information and watches all online node scrapers via service discovery.
    2. Node Scraper: Deployed as a DaemonSet, it scrapes hardware-specific information and streams resource changes and hardware snapshots to the Inventory Service via a gRPC endpoint.

    By default, the operator labels and annotates all healthy nodes to make them available to the provider.

  5. Compare NGINX Ingress vs Gateway API in Akash

    main

    The Akash provider supports two ingress modes. Gateway API is the recommended standard as NGINX Ingress reaches EOL in March 2026.

    FeatureNGINX Ingress (default)Gateway API
    Resource TypeIngressHTTPRoute
    Class/ReferenceIngressClass: akash-ingress-classGateway: akash-gateway
    Annotationsnginx.ingress.kubernetes.io/*nginx.org/*

    HTTP Options Translation:

    The provider automatically maps SDL options to the correct annotations:

    SDL OptionNGINX Ingress AnnotationGateway API Annotation
    Read Timeoutnginx.ingress.kubernetes.io/proxy-read-timeoutnginx.org/proxy-read-timeout
    Send Timeoutnginx.ingress.kubernetes.io/proxy-send-timeoutnginx.org/proxy-send-timeout
    Max Body Sizenginx.ingress.kubernetes.io/proxy-body-sizenginx.org/client-max-body-size
    Next Upstream Triesnginx.ingress.kubernetes.io/proxy-next-upstream-triesnginx.org/proxy-next-upstream-tries
  6. Configure the Certificate Issuer for REST and gRPC Gateways

    main

    The built-in certificate issuer enables automatic SSL/TLS certificate generation and renewal for Akash provider gateways using Let's Encrypt. It utilizes the DNS-01 challenge validation method, which requires access to your DNS provider to prove domain ownership.

    To use the certificate issuer, you must provide a --cert-issuer-email for ACME account registration. By default, the issuer uses the Let's Encrypt staging environment, which is recommended for testing to avoid production rate limits.

    # Example enabling the issuer with staging (default)
    ./provider-services ... \
      --cert-issuer-enabled=true \
      --cert-issuer-email="admin@yourdomain.com"
  7. Network Policy Enforcement for Tenant Namespaces

    main

    The Provider uses Kubernetes NetworkPolicy (ideally via a CNI like Calico) to isolate tenant namespaces.

    Ingress Rules

    • Deny All: By default, all ingress into the Tenant namespace is denied.
    • Intra-Namespace: Allows traffic between containers within the same Tenant namespace.
    • Ingress Controller: Allows traffic from the Ingress Controller (e.g., ingress-nginx).

    Egress Rules

    • Deny All: By default, all egress from the Tenant namespace is denied.
    • Intra-Namespace: Allows egress to other pods within the same Tenant namespace.
    • Internet Access: Allows egress to 0.0.0.0/0 EXCEPT the internal cluster subnet 10.0.0.0/8.
    • DNS: Allows UDP Port:53 egress to coredns in the kube-system namespace.
  8. Understand the Event Bus implementation stages

    main

    The Akash Provider Event Bus is transitioning from a single-process implementation to a distributed message queue system. The evolution is divided into two stages:

    Stage 1: Substitution

    In this stage, the underlying implementation is replaced with a selected message queue (e.g., Redis), but the public API remains unchanged.

    • Behavior: Subscribers are automatically subscribed to all emitted events. There is no routing or filtering; subscribers must perform their own filtering.
    • Requirement: All existing tests must pass.

    Stage 2: API Enhancement

    This stage introduces advanced messaging features:

    • Topic Management: Support for subscribe to topic and unsubscribe to topic.
    • Separation of Concerns: The Event Bus is renamed to Publisher, and a distinct Subscriber entity is introduced.
    • Filtering & Routing: Support for wildcard and logical subscriptions.
  9. Understand Akash Provider's Hard Multi-Tenancy Model

    main

    The Akash Provider implements 'Hard Multi-Tenancy' to protect the provider and other tenants from malicious workloads. This is achieved through multiple layers of security across different threat domains, primarily focusing on Container/Kernel isolation and Networking isolation.

    Threat Domains

    1. Container/CGroups/Kernel/Service Accounts: Prevents tenant containers from gaining host-level root privileges, escalating privileges via syscalls, or accessing the Kubernetes API via service account tokens.
    2. Networking: Restricts tenant containers to communicating only within their own namespace or to the open web, preventing unauthorized access to other tenant namespaces, system utilities, or the Kubernetes API.

    Providers can manage specific risks (like GPU workloads requiring root) through selectable attributes, but by default, the provider blocks root workloads using a PodSecurityPolicy.

  10. Set up the Single Dev Environment

    main

    The 'Single' dev environment builds a single-node blockchain network and an Akash Provider Services Daemon (PSD) within a kind (Kubernetes in Docker) environment. This is used for testing bidding, running workloads, and interacting with the network locally.

    Prerequisites

    • kubectl installed
    • Port 80 must be free (unless using a different configuration)

    Local DNS Configuration

    To ensure DNS routing works for the Akash-Node and Akash-Provider, add the following to your /etc/hosts file:

    127.0.0.1   akash.localhost
    127.0.0.1   akash-provider.localhost

    Alternatively, you can use 127.0.0.1 *.localhost if it does not conflict with other local rules.

    # No specific single command, but requires kubectl and /etc/hosts updates.
  11. Initialize Kubernetes cluster (kind)

    main

    You have two primary options for setting up the Kubernetes cluster via kind.

    Option 1: Random local port (Recommended for stability) Maps a random local port to port 80 of your workload. This is less error-prone and allows multiple instances to run concurrently, but makes browser access more difficult.

    make kind-cluster-setup

    Option 2: Localhost port 80 Maps localhost port 80 directly to the workload. This allows you to browse your app via http://hello.localhost, but will fail if any other service is already listening on port 80.

    KIND_CONFIG=kind-config-80.yaml make kind-cluster-create
    make kind-cluster-setup
    # OR
    KIND_CONFIG=kind-config-80.yaml make kind-cluster-create
  12. Configure the Dev Environment: "SSH" configuration

    main

    The SSH dev environment builds a single-node blockchain network, an Akash Provider Services Daemon (PSD) for bidding and running workloads, and a Kubernetes cluster (using kind) for the PSD to run workloads on.

    Default Configuration Variables

    Most make commands are configurable via environment variables. If not provided, the following defaults are used:

    NameDefaultDescription
    KEY_NAMEmainstandard key name
    PROVIDER_KEY_NAMEprovidername of key to use for provider
    DSEQ1deployment sequence
    GSEQ1group sequence
    OSEQ1order sequence
    PRICE10uaktprice to bid