DOCSEARCHbyAlgolia
Submit Docs

Azure Service Operator (ASO)

repository·main·Indexed 21 days ago

https://github.com/azure/azure-service-operator

A Kubernetes operator that enables the provisioning and management of Azure resources using Kubernetes Custom Resource Definitions (CRDs) and standard tooling like kubectl. ASO v2 is the actively maintained version, providing stable lifecycle management, status reporting based on Azure OpenAPI specs, and integration with Open Policy Agent (Gatekeeper) for resource constraints.

Tokens
430.9K
Snippets
505
Records
1.1K
Agent score
69%

What's inside Azure Service Operator

  1. Overview of Azure Service Operator (ASO) v2

    main

    Azure Service Operator (ASO) v2 allows you to deploy and maintain Azure resources directly from within your Kubernetes cluster. Instead of managing Azure resources separately from your Kubernetes applications, ASO enables you to manage them together using standard Kubernetes tooling.

    Key capabilities include:

    • K8s Native: Uses Custom Resource Definitions (CRDs) and Golang API structures to manage Azure resources.
    • Azure Native: CRDs model the Azure resource lifecycle and utilize Kubernetes garbage collection via ownership references.
    • Cloud Scale: CRDs are generated from Azure Resource Manager (ARM) schemas to ensure rapid support for new Azure features.
    • Async Reconciliation: Resource creation processes do not block the operator.

  2. What is Azure Service Operator v2?

    main
    Azure Service Operator (ASO) v2 allows you to manage Azure resources directly from within your Kubernetes cluster using Kubernetes tooling. It provides Custom Resource Definitions (CRDs) and Golang API structures that model the Azure resource lifecycle using Kubernetes garbage collection via ownership references. This enables you to deploy and maintain Azure resources (like Redis Cache or PostgreSQL) alongside your Kubernetes applications, ensuring they are configured and managed together.

  3. What is Azure Service Operator (ASO)?

    main

    Azure Service Operator (ASO) allows you to provision and manage Azure resources directly from within Kubernetes using Kubernetes tooling and primitives like kubectl apply.

    It works by using:

    • Custom Resource Definitions (CRDs): Specific definitions for each Azure service you wish to provision.
    • A Kubernetes Controller: A management loop that synchronizes the desired state defined in your Kubernetes Custom Resources with the actual state of the resource in Azure. The controller handles creating, updating, or deleting Azure resources to match your configuration.

  5. Locate Azure Service Operator v2 source and configuration

    main

    The core of Azure Service Operator v2 is located in the v2/ directory. Use the following directory structure to find specific components:

    • API Definitions: v2/api/ contains Kubernetes API packages organized by Azure service group.
    • Helm Charts: v2/charts/ contains the ASO v2 Helm chart source and packaged releases.
    • Binaries: v2/cmd/ contains the entrypoints for the controller and the asoctl CLI.
    • Deployment Manifests: v2/config/ contains Kubernetes deployment manifests and Kustomize overlays.
    • Resource Samples: v2/samples/ contains example Kubernetes resource manifests for use as user examples.
    • API Specifications: v2/specs/ contains the Azure REST API specifications used as input for the code generator.
    • Generator Configuration: v2/azure-arm/ contains per-service generator configuration, while v2/azure-arm.yaml and v2/azure-stack.yaml are the top-level configurations.

  6. CEL expression inputs: self and secret

    main

    When writing CEL expressions in ASO, you have access to specific input variables depending on the output type:

    Output TypeAvailable VariablesDescription
    ConfigMapselfThe resource itself.
    Secretself, secretThe resource itself and a set of associated secrets.

    The self variable

    self represents the resource at the version you last applied. You can access fields in self.spec, self.metadata, or self.status. Note: If a field was removed in a newer API version, you can still reference it via self as long as you are applying the resource using the API version that contained that field.

    The secret variable

    secret provides access to a set of secrets associated with the resource. The available keys in secret vary by resource type. For example, an eventhub Namespace supports:

    • primaryConnectionString
    • primaryKey
    • secondaryConnectionString
    • secondaryKey

  7. Implementation details for change detection and operatorStatus

    main

    The change detection mechanism relies on specific structural additions to the generated code and resource status types:

    • DiffWith() method: Every spec type includes a DiffWith() method (added via the code generator) that accumulates a list of differences found during comparison.
    • genruntime/diff package: Provides helper methods for common comparison tasks during the differencing process.
    • operatorStatus: A field introduced on each top-level status type. It contains a knownDifferences field used to store the differences identified during the reconciliation process to prevent loop-triggering updates.

  8. Configure IP address and subnet mask for restrictions

    main

    When defining an IP restriction rule, you can specify the target IP address using two different methods:

    1. Pure IPv4 Address: Provide a standard IPv4 address in the ipAddress property and you must also specify the subnetMask property.
    2. CIDR Notation: Provide the address in CIDR notation (e.g., ipv4/mask) in the ipAddress property. If you use CIDR notation, the subnetMask property must not be specified.

    Other relevant properties include action (to Allow or Deny) and priority to control the order of rule evaluation.

  9. How Azure-generated secrets lifecycle works

    main

    Azure-generated secrets are managed by the operator using the following lifecycle model:

    1. Creation: After the Azure resource reaches a Ready state, the operator calls the appropriate Azure API (like ListKeys) to retrieve the values.
    2. Storage: The operator creates or updates a Kubernetes secret (or KeyVault secret) based on the operatorSpec.secrets configuration.
    3. Ownership: The Kubernetes secret is marked with an owner reference pointing to the Azure resource. Deleting the resource triggers automatic deletion of the secret.
    4. Updates: If the keys are rotated in Azure, the operator will reconcile and update the Kubernetes secret with the new values.

  11. Retrieve Azure-generated secrets into Kubernetes

    main

    Azure resources often produce secrets (like connection strings or primary keys) that you need to use in your cluster. ASO can automatically capture these and write them to Kubernetes Secret objects using the .spec.operatorSpec.secrets or .spec.operatorSpec.secretExpressions fields.

    Using .spec.operatorSpec.secrets

    Use this field for a direct mapping where you specify the destination Secret name and the key where the value should be stored.

    Using .spec.operatorSpec.secretExpressions

    Use this field if you need to format the secret or map values from the resource's status (e.g., mapping self.status.documentEndpoint to a secret key).

    # Example using secretExpressions
operatorSpec:
  secretExpressions:
    - name: mysecret
      key: primarymasterkey
      value: secret.primaryMasterKey
    - name: myendpoint
      key: endpoint
      value: self.status.documentEndpoint

# Example using secrets
operatorSpec:
  secrets:
    primaryMasterKey:
      name: mysecret
      key: primarymasterkey

  12. Understand Azure resource relationships in ASO

    main

    Azure Service Operator (ASO) models two primary types of relationships between Azure resources to manage their lifecycles and connectivity within Kubernetes:

    1. Related/Linked Resources ("has-a" relationship)

    These are resources that refer to one another without a formal ownership bond. This relationship is typically one-way.

    • Example: A Virtual Machine Scale Set (VMSS) refers to a Subnet. The VMSS needs the Subnet's information, but the Subnet does not need to know about the VMSS.
    • ASO Implementation: ASO uses decomposed properties (like resourceGroup, virtualNetworkName, and subnetName) which are combined to form a fully qualified Azure Resource ID.

    2. Owner and Dependent Resources

    These resources have a formal ownership bond where one resource (the owner) controls the lifecycle of another (the dependent).

    • Example: A ResourceGroup owns all resources inside it, or a RouteTable owns multiple Routes.
    • Lifecycle Behavior: When an owner is deleted, the dependent resource should also be deleted (cascading deletion).
    • ASO Implementation: Dependent resources include properties that map to their owner's identity (e.g., a MySQLFirewallRule includes resourceGroup and server to identify its owners).