OSBuild Composer

repository·main·Indexed 41 days ago

https://github.com/osbuild/osbuild-composer

A set of HTTP services that orchestrate the composition of operating system images by translating high-level requests into osbuild pipelines. It acts as a middleman between user interfaces and the osbuild execution engine, supporting the lorax-composer API and providing a CLI tool called composer-cli. It supports the creation of various image types, including Red Hat Enterprise Linux 8, and can be configured via TOML files and environment variables.

Tokens
37.3K
Snippets
81
Records
167
Agent score
74%

What's inside OSBuild Composer

  1. Overview of OSBuild Composer Image Types

    main
    The osbuild-composer tool generates various image types categorized by their target operating system. Each image type is designed for specific purposes and use cases, and some may have unique characteristics or configurations that distinguish them from a standard operating system installation.
  2. Overview of OSBuild Composer

    main

    OSBuild Composer is a set of HTTP services designed for composing operating system images. It acts as an orchestrator and middleman between the osbuild pipeline execution engine and various user interfaces.

    Composer translates high-level image composition requests into pipeline descriptions for osbuild. Once a build is complete, the resulting pipeline output is either returned to the user or uploaded to a specified target.

    It supports multiple APIs, including the lorax-composer API, making it a drop-in replacement for lorax-composer.

  3. What is OSBuild Composer?

    main

    OSBuild Composer is a set of HTTP services designed for composing operating system images. It leverages the osbuild pipeline execution engine and defines a specific class of images for building.

    It is designed to be compatible with the lorax-composer API, allowing it to serve as a drop-in replacement for that service.

  4. Implementation details of osbuild-composer GCE RHEL Guest images

    main

    The osbuild-composer GCE RHEL Guest image is specifically designed for the Google Compute Engine environment.

    Core Characteristics:

    • Import Method: Built as a RAW image in a tarball archive and imported via the manual import process.
    • Default Size: 20 GB.
    • Partitioning: Uses the default x86_64 partition table.
    • Google Guest Tools: Includes google-compute-engine, google-osconfig-agent, and gce-disk-expand. These services are enabled by default for initial instance setup (user creation, SSH keys, disk resizing).
    • Note on cloud-init: cloud-init is not installed on this image.
    • Networking: dhcp-client is not installed because NetworkManager uses its internal DHCP implementation.
    • Subscription Model: Built as a Bring Your Own Subscription (BYOS) image. It does not include the Google RHUI client; users must register via subscription-manager.
  5. How to interact with OSBuild Composer

    main

    You can control an OSBuild Composer instance through several interfaces:

    1. Direct APIs: Access the service directly via its provided HTTP APIs (including the lorax-composer compatible API).
    2. Command Line: Use the composer-cli tool.
    3. Web UI: Use higher-level interfaces like the Cockpit Module.
  6. Note on missing Oracle Guest Agent in OCI images

    main

    Images generated using the Oracle OCI image type do not include the Oracle Guest Agent.

    The Oracle Guest Agent is a collection of agents used by the Oracle Cloud web console to collect metrics, report status, and manage disk usage or OS updates. It is excluded from this image type because the source code is unavailable for inclusion in the build process.

  7. When to use panic in osbuild-composer

    main

    In osbuild-composer, panic should be reserved exclusively for programmer errors—situations where the error can only occur if other code within the project is incorrect. Using panic for these cases allows developers to catch logic flaws during unit testing.

    When panicking due to a programmer error, provide only the error itself in the panic call to ensure a clean stacktrace. Do not add additional descriptive messages.

    bytes, err := json.Marshal(knownStruct)
    if err != nil {
            panic(err)
    }
  8. Implementation details for GCE RHEL Guest Images

    main

    The osbuild-composer implementation for GCE RHEL Guest images includes several automated configuration steps during the %post phase of the image build:

    • GPG Keys: Imports Google's RPM and Yum package keys.
    • Networking: Sets the firewall default zone to trusted and reduces the dhclient retry interval to 10 seconds.
    • DNF Configuration: Disables IPv6 for DNF (ip_resolve=4) and configures dnf-automatic to apply security updates automatically.
    • GCE Guest Tools: Installs google-compute-engine, google-osconfig-agent, gce-disk-expand, and google-cloud-sdk.
    • SSH Hardening: Disables password authentication and root login via SSH, and sets ServerAliveInterval/ClientAliveInterval to 420 to prevent disconnections.
    • Cleanup: Removes installer-specific files (like ens4 network configs and boto.cfg) and cleans the DNF cache to minimize image size.
  9. Install SAP HANA packages for GCE RHEL images

    main

    For RHEL images intended for SAP HANA workloads on Google Compute Engine, you must install the google-rhui-client-rhel8-sap client and a specific set of SAP-related packages. This ensures the environment is correctly tuned and has the necessary resource agents for GCE.

    Required packages include:

    • compat-sap-c++-9
    • fence-agents-gce
    • resource-agents-sap-hana
    • tuned-profiles-sap-hana
    • (and others listed in the implementation snippet)
    %post --erroronfail
    set -x
    exec &> /dev/ttyS0
    dnf -y install google-rhui-client-rhel8-sap
    %end
    
    %post --erroronfail
    set -x
    exec &> /dev/ttyS0
    
    # Configure SAP HANA packages.
    SAP_PKGS="
    compat-sap-c++-9
    fence-agents-gce
    libatomic
    libtool-ltdl
    lvm2
    numactl
    numactl-libs
    nfs-utils
    pacemaker
    pcs
    resource-agents
    resource-agents-gcp
    resource-agents-sap
    resource-agents-sap-hana
    rhel-system-roles-sap
    tuned-profiles-sap
    tuned-profiles-sap-hana
    "
    
    dnf install -y ${SAP_PKGS}
    %end
  10. Configure osbuild-composer worker instance via cloud-init

    main

    The built AMI requires a configuration file located at /tmp/cloud_init_vars to function. This file must contain specific key-value pairs for authentication, cloud provider credentials, and logging. The simplest way to provide this file is using the cloud-init write_files directive.

    #cloud-config
    
    write_files:
      - path: /tmp/cloud_init_vars
        content: |
          COMPOSER_HOST=api.stage.openshift.com
          COMPOSER_PORT=443 
          OFFLINE_TOKEN_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:offline-token-abcdef
          SUBSCRIPTION_MANAGER_COMMAND_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:subscription-manager-command-abcdef
          GCP_SERVICE_ACCOUNT_IMAGE_BUILDER_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:gcp_service_account_image_builder-abcdef
          AZURE_ACCOUNT_IMAGE_BUILDER_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:azure_account_image_builder-abcdef
          AWS_ACCOUNT_IMAGE_BUILDER_ARN=arn:aws:secretsmanager:us-east-1:123456789012:secret:aws_account_image_builder-abcdef
          SYSTEM_HOSTNAME_PREFIX=staging-worker-aoc
          SECRETS_MANAGER_ENDPOINT_URL=https://secretsmanager.us-east-1.amazonaws.com/
          CLOUDWATCH_LOGS_ENDPOINT_URL=https://logs.us-east-1.amazonaws.com/
          CLOUDWATCH_LOG_GROUP=staging_workers_aoc
          TOKEN_URL="https://sso.redhat.com/auth/realms/redhat-external/protocol/openid-connect/token"
  11. Configure SSH and User Access for GCE RHEL

    main

    To secure the image for GCE deployment, apply these SSH and user settings:

    • Root User: Lock the root account and set an invalid password.
    • SSH Password Auth: Disable password authentication in /etc/ssh/sshd_config.
    • SSH Root Login: Disable root login via SSH.
    • SSH Timeouts: Set ServerAliveInterval and ClientAliveInterval to 420 to prevent disconnections.

    Example SSH hardening commands:

    sed -i -e '/^PasswordAuthentication /s/ yes$/ no/' /etc/ssh/sshd_config
    sed -i -e $'/^	ServerAliveInterval/d' /etc/ssh/ssh_config
    sed -i -e $'/^Host \*$/a \	ServerAliveInterval 420' /etc/ssh/ssh_config
    sed -i '/ClientAliveInterval/s/^.*/ClientAliveInterval 420/' /etc/ssh/sshd_config
    sed -i '/PermitRootLogin yes/s/^.*/PermitRootLogin no/' /etc/ssh/sshd_config
    # Disable password authentication by default.
    sed -i -e '/^PasswordAuthentication /s/ yes$/ no/' /etc/ssh/sshd_config
    # Set ServerAliveInterval and ClientAliveInterval to prevent SSH disconnections.
    sed -i -e $'/^	ServerAliveInterval/d' /etc/ssh/ssh_config
    sed -i -e $'/^Host \*$/a \	ServerAliveInterval 420' /etc/ssh/ssh_config
    sed -i '/ClientAliveInterval/s/^.*/ClientAliveInterval 420/' /etc/ssh/sshd_config
    # Disable root login via SSH by default.
    sed -i '/PermitRootLogin yes/s/^.*/PermitRootLogin no/' /etc/ssh/sshd_config
  12. Deploy osbuild-composer to QEMU

    main

    The tools/deploy-qemu script deploys osbuild-composer into an ephemeral virtual machine using QEMU.

    Key Characteristics:

    • Ephemeral: The qcow2 image provided as <config> is not modified. All changes made to the virtual machine are lost once the QEMU process is stopped.
    • Networking: Two ports are forwarded from the VM to the host via QEMU user networking:
      • Port 22 (SSH) is forwarded to host port 2222.
      • Port 443 (HTTPS) is forwarded to host port 4430.

    Usage:

    ./tools/deploy-qemu <path-to-qcow2-image> <userdata>
    ./tools/deploy-qemu <path-to-qcow2-image> <userdata>