Elemental Documentation

To develop for the Elemental stack, you must provision a test cluster and build several local images. This process requires cloning the elemental, elemental-toolkit, and elemental-operator repositories.

1. Provision the test cluster

Use the elemental-operator repository to provision a local cluster:

LOCAL_BUILD=true make setup-full-cluster

Once successful, Rancher is accessible at https://172.18.0.2.sslip.io with the password rancherpassword (uses self-signed certificates).

2. Build local development images

Build the following images in order to reference them in your development ISO:

• elemental-register: In the elemental-operator repo:

  make build-docker-register

  Produces docker.io/local/elemental-register:dev.

• elemental-toolkit: In the elemental-toolkit repo:

  make VERSION=dev build

  Produces docker.io/local/elemental-toolkit:dev.

• Elemental Dev ISO: In the elemental repo:

  make kind-load-dev-iso

  Generates and loads docker.io/local/elemental-iso:dev into the test cluster.

3. Download the Dev ISO

Apply a test manifest and download the generated ISO for use in hypervisors:

kubectl apply -f tests/manifests/elemental-dev-example.yaml
kubectl wait --for=condition=ready pod -n fleet-default fire-img

# Download the ISO
wget --no-check-certificate `kubectl get seedimage -n fleet-default fire-img -o jsonpath="{.status.downloadURL}"` -O elemental-dev.x86_64.iso

Note: Machines provisioned with this ISO must connect to the test Rancher at 172.18.0.2:443 and the test registry at 172.18.0.2:30000.

To test upgrades or downgrades, build specific versions of the components and push a new OS image to the test registry.

1. Build component images for the target version

Checkout your feature branch and build the images:

• elemental-register (elemental-operator repo):

  git checkout my-next-feature-branch
  make build-docker-register

• elemental-toolkit (elemental-toolkit repo):

  git checkout my-next-feature-branch
  make VERSION=dev GIT_COMMIT=test-upgrade build

2. Build and push the OS image

In the elemental repo, build the OS image and push it to the test registry (172.18.0.2:30000).

Prerequisite: You must add the registry to your /etc/docker/daemon.json to allow insecure connections:

{ "insecure-registries":["172.18.0.2:30000"] }

Then restart docker: sudo systemctl restart docker.

Build and Push:

ELEMENTAL_OS_IMAGE="172.18.0.2:30000/elemental-os:dev-next" make build-dev-os
docker push 172.18.0.2:30000/elemental-os:dev-next

3. Trigger and monitor the upgrade

Apply the upgrade manifest and monitor the cattle-system namespace for the upgrader pod:

kubectl apply -f tests/manifests/elemental-dev-upgrade-example.yaml

# Monitor for the upgrader pod
kubectl -n cattle-system get pods -w

# Follow upgrade logs
kubectl -n cattle-system logs <apply-os-upgrader-pod-name> -f

4. Verify the upgrade

On the upgraded machine, run:

elemental version --long

The output should include the GIT_COMMIT value used during the build.

In the development environment, Rancher uses a private CA managed by cert-manager. To test certificate rotation, follow these steps:

1. Backup current CA: Save the current cacerts from https://172.18.0.2.sslip.io/cacerts to verify the change later.
2. Delete the CA secret:

   kubectl -n cattle-system delete secret tls-ca

   cert-manager will recreate it immediately.
3. Patch the secret: Copy the tls.crt from the new secret into cacerts.pem so Rancher mounts it correctly:

   export TEST_CA="$(kubectl -n cattle-system get secret tls-ca -o jsonpath="{.data['tls\.crt']}")"
   kubectl -n cattle-system patch secret tls-ca -p "{\"data\":{\"cacerts.pem\":\"$TEST_CA\"}}"

4. Rotate Ingress certificate: Delete the ingress secret to trigger a new issuance:

   kubectl -n cattle-system delete secret tls-rancher-ingress

5. Restart Rancher:

   kubectl rollout restart deploy/rancher -n cattle-system

6. Force Agent Redeployment: On the downstream cluster, force the Rancher agents to redeploy to pick up the new CA:

   kubectl annotate clusters.management.cattle.io <CLUSTER_ID> io.cattle.agent.force.deploy=true

7. Update Fleet: In the Rancher UI, select 'Force Update' for the clusters in the Continuous Delivery view.