Provider Upjet-AWS
repository·main·Indexed 42 days ago
https://github.com/crossplane-contrib/provider-upjet-awsA Crossplane provider that leverages Upjet code generation to provide managed resources for Amazon AWS. It enables the management of AWS services using Kubernetes-style declarative configuration and supports multiple authentication mechanisms including Secret, IRSA, WebIdentity, and PodIdentity via the aws.upbound.io/v1beta1/ProviderConfig resource.
What's inside Provider Upjet-AWS
- Provider Upjet-AWS is a Crossplane provider built using Upjet code generation tools. It exposes XRM-conformant managed resources for Amazon AWS, allowing you to manage AWS infrastructure using Crossplane's declarative API.
Understand the AWS ProviderConfig E2E testing architecture
mainThe
providerconfig-aws-e2e-testpackage provides a base environment to test variousProviderConfigscenarios forprovider-upjet-aws.It uses a nested control plane pattern: a local Crossplane control plane manages a remote EKS cluster, which in turn runs its own Crossplane control plane. This remote cluster is bootstrapped with
provider-kubernetesandprovider-helmfrom the local control plane.Key components provisioned in the test environment include:
- An AWS EKS cluster.
- IAM resources for testing IRSA (IAM Roles for Service Accounts) and WebIdentity authentication.
- Crossplane installed in the EKS cluster.
DeploymentRuntimeConfigs to enable IRSA for XP providers.provider-family-awsand example providers (provider-aws-ec2,provider-aws-rds).- Various AWS
ProviderConfigmanifests and example Managed Resources (MRs).
Authentication mechanisms supported by provider-aws
mainThe
provider-awsauthenticates to the AWS API using theaws.upbound.io/v1beta1/ProviderConfigresource. You select the authentication mechanism by setting thespec.credentials.sourcefield.Supported values for
spec.credentials.source:Secret: Uses long-term IAM user credentials stored in a Kubernetes Secret.IRSA: Uses IAM Roles for Service Accounts (requires EKS and IRSA configuration).WebIdentity: Uses an assumed Web identity token.PodIdentity: Uses EKS Pod Identity (requires EKS Pod Identity configuration).
If no mechanism is specified, the default is
Secret.Note: For Upbound managed control-planes (MCP), only the
Secretmechanism is supported until EKS clusters can be utilized.Authenticate using EKS Pod Identity
mainEKS Pod Identity is an alternative to IRSA that does not require an OIDC provider. It relies on the EKS Pod Identity Agent and the
pods.eks.amazonaws.comservice principal.Setup Steps:
- Install the EKS Pod Identity Agent on your cluster.
- Create an EKS Pod Identity association between your
ServiceAccountandnamespaceand the target IAM role. - Ensure the
ServiceAccountandnamespaceused byprovider-awsmatch the association.
Troubleshooting:
- Verify the
namespaceandServiceAccountmatch the association. - Confirm the Pod Identity Agent is functional.
- Review the IAM role's trust policy for Pod Identity.
# 1. Configure ServiceAccount via DeploymentRuntimeConfig apiVersion: pkg.crossplane.io/v1beta1 kind: DeploymentRuntimeConfig metadata: name: podidentity-drc spec: serviceAccountTemplate: metadata: name: provider-aws deploymentTemplate: {} --- apiVersion: pkg.crossplane.io/v1 kind: Provider metadata: name: upbound-provider-aws-ec2 spec: package: xpkg.upbound.io/upbound/provider-aws-ec2:v1.4.0 runtimeConfigRef: name: podidentity-drc --- # 2. Configure ProviderConfig apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: podidentity-example spec: credentials: source: PodIdentityAuthenticate using AWS access keys
mainTo authenticate the AWS provider using access keys, you must store your credentials in a Kubernetes secret and reference that secret in a
ProviderConfigobject.1. Prepare the credentials file
Create a text file (e.g.,
aws-credentials.txt) containing your AWS credentials in INI format:[default] aws_access_key_id = <aws_access_key> aws_secret_access_key = <aws_secret_key>2. Create a Kubernetes secret
Generate a secret from this file using
kubectl:kubectl create secret generic <secret-name> --from-file=key-file=<aws_credentials_file.txt>3. Create a ProviderConfig
Apply a
ProviderConfigthat points to the secret. Thespec.secretRefmust include thenamespace, thenameof the secret, and thekey(which corresponds to theDatafield name used when creating the secret, e.g.,key-file).apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: namespace: default name: aws-secret key: key-fileInstall the Up command-line
mainDownload and install the Upbound
upCLI to manage Upbound Universal Crossplane (UXP) and providers. Official providers requireupversion v0.13.0 or later.curl -sL "https://cli.upbound.io" | sh sudo mv up /usr/local/bin/Verify the installation:
up --versionUpdate IAM role trust policy for IRSA
mainWhen using IRSA, the IAM role created (e.g., via
eksctl) must have itsTrust relationshipupdated to allow the AWS provider to assume the role.- Identify the OIDC issuer: Use
aws eks describe-cluster --name <cluster-name> --query "cluster.identity.oidc.issuer" --output textand strip thehttps://prefix. - Construct the Condition: Use
StringLikeinstead ofStringEquals. The condition key should follow the pattern:"<oidc-issuer>:sub": "system:serviceaccount:<namespace>:provider-aws-*". - Apply the policy: Use
aws iam update-assume-role-policywith a JSON file containing the new policy.
{ "Version": "2012-10-17", "Statement": [ { "Effect": "Allow", "Principal": { "Federated": "arn:aws:iam::000000000000:oidc-provider/oidc.eks.us-east-2.amazonaws.com/id/266A01FA1DBF8083FA1C23EB7D4736E4" }, "Action": "sts:AssumeRoleWithWebIdentity", "Condition": { "StringLike": { "oidc.eks.us-east-2.amazonaws.com/id/266A01FA1DBF8083FA1C23EB7D4736E4:sub": "system:serviceaccount:upbound-system:provider-aws-*" } } } ] }- Identify the OIDC issuer: Use
Authenticate using Web Identity
mainWeb Identity authentication allows you to specify a Web Identity token in the
ProviderConfig. This is preferred over using environment variables (AWS_WEB_IDENTITY_TOKEN_FILEandAWS_ROLE_ARN), which are now deprecated.You can configure the token source in
spec.credentials.webIdentity.tokenConfigusing either aSecretor aFilesystempath.Note: If using
Filesystem, the token must be mounted into the provider pod (e.g., viaDeploymentRuntimeConfig).# Example: Using a Kubernetes Secret for the token apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: webidentity-example spec: credentials: source: WebIdentity webIdentity: roleARN: arn:aws:iam::123456789012:role/providerexamplerole tokenConfig: source: Secret secretRef: key: token name: example-web-identity-token-secret namespace: upbound-system --- # Example: Using a Filesystem path for the token apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: webidentity-example-fs spec: credentials: source: WebIdentity webIdentity: roleARN: arn:aws:iam::123456789012:role/providerexamplerole tokenConfig: source: Filesystem fs: path: /path/to/token/fileBuild and publish the provider locally
mainTo build the provider locally and prepare it for a local Kubernetes cluster, use the
make build.allcommand. You must specify theDOCKERHUB_ORG, theSUBPACKAGESyou are working on (e.g.,configandec2), and setBRANCH_NAMEtomainto facilitate image publishing to your Docker Hub account.Example for building the
configandec2sub-packages:DOCKERHUB_ORG=<your-docker-name> BUILD_ARGS="--load" XPKG_REG_ORGS_NO_PROMOTE="" XPKG_REG_ORGS="index.docker.io/$DOCKERHUB_ORG" make build.all publish BRANCH_NAME=main SUBPACKAGES="config ec2"Authenticate with long-term IAM user credentials
mainThis method uses a Kubernetes Secret containing long-term IAM user credentials. It is considered less secure than other methods.
The Secret must contain a
credentialskey with a base64-encoded file formatted as an AWS shared config file with a[default]profile:[default] aws_access_key_id=<AWS_ACCESS_KEY_ID> aws_secret_access_key=<AWS_SECRET_ACCESS_KEY>Then, configure the
ProviderConfigto reference this secret.apiVersion: v1 kind: Secret metadata: name: example-aws-creds namespace: upbound-system type: Opaque data: credentials: <base64-encoded credentials file> --- apiVersion: aws.upbound.io/v1beta1 kind: ProviderConfig metadata: name: default spec: credentials: source: Secret secretRef: name: example-aws-creds namespace: upbound-system key: credentialsAuthenticate using IAM Roles for Service Accounts (IRSA)
mainFor clusters running in Amazon EKS, you can use IAM Roles for Service Accounts (IRSA) to authenticate the AWS provider. This method avoids managing long-lived access keys.
Prerequisites
- Enable IAM OIDC provider: The EKS cluster must have an IAM OIDC provider enabled.
- Create an IAM Policy: Define the permissions the provider needs.
- Create an IAM Role: An IAM role must be created and associated with a Kubernetes service account.
- Update Role Trust Policy: The role's trust relationship must be updated to allow the provider to assume the role using
StringLikeconditions. - Create a DeploymentRuntimeConfig: This associates the IAM role ARN with the provider's deployment.
- Configure the Provider: The
Providermust reference theDeploymentRuntimeConfig. - Configure ProviderConfig: The
ProviderConfigmust be set to useIRSAcredentials.
Configure AWS credentials in Kubernetes
mainTo manage AWS resources, you must provide credentials via a Kubernetes secret in the
upbound-systemnamespace.- Generate a credentials file (e.g.,
aws-credentials.txt) with the following format:
[default] aws_access_key_id = <aws_access_key> aws_secret_access_key = <aws_secret_key>- Create the Kubernetes secret:
kubectl create secret \ generic aws-secret \ -n upbound-system \ --from-file=creds=./aws-credentials.txt- Generate a credentials file (e.g.,