BOSH Bootloader (BBL)

repository·main·Indexed 41 days ago

https://github.com/cloudfoundry/bosh-bootloader

A command-line utility designed to automate the provisioning of BOSH on various Infrastructure-as-a-Service (IaaS) providers, including AWS, GCP, Azure, OpenStack, vSphere, and CloudStack.

Tokens
41.3K
Snippets
127
Records
169
Agent score
51%

What's inside BBL

  1. Overview of BOSH Bootloader (bbl)

    main

    BOSH Bootloader, also known as bbl (pronounced "bubble"), is a command-line utility used to stand up CloudFoundry or Concourse installations on various Infrastructure-as-a-Service (IaaS) platforms.

    Supported IaaS providers include:

    • AWS
    • GCP
    • Microsoft Azure
    • Openstack
    • vSphere
  2. Use Plan Patches to customize BBL environments

    main

    Plan patches allow you to customize the IaaS environment and the BOSH director created by bbl up.

    A patch is a directory containing files organized in the same hierarchy as the bbl-state directory. When you run bbl up, the bootloader attempts to merge your terraform overrides and apply provided jumpbox, director, and cloud-config ops files, provided they are placed in the correct directories.

    Important Notes:

    • bbl down will not delete files that were not originally created by BBL.
    • Warning: Plan patches are not tested as thoroughly as the rest of the bosh-bootloader.
    • Many patches require specific preparation steps or are designed for specific downstream BOSH deployments. Always read the README.md within the specific patch directory before applying it.
  3. Understand the bbl state directory

    main

    The bbl state directory contains all files used to create your BOSH director. You must retain the entire directory to update, delete, or run BOSH commands against the environment.

    Warning: The state directory contains sensitive IaaS credentials. Store it securely (e.g., in an encrypted bucket) and avoid committing it to version control. Specifically, the vars/ directory should be kept outside of version control.

    FilenameContents
    bbl-state.jsonEnvironment name and bbl version metadata
    terraform/Terraform templates used to pave the IaaS
    vars/Environment-specific variables
    jumpbox-deployment/The latest tested jumpbox-deployment
    create-jumpbox.shBOSH CLI command used to create the jumpbox
    bosh-deployment/The latest tested bosh-deployment
    create-director.shBOSH CLI command used to create the director (via bbl up)
    cloud-config/Cloud-config YAML used to map IaaS resources to BOSH
    delete-director.shBOSH CLI command used to delete the director
    delete-jumpbox.shBOSH CLI command used to delete the jumpbox
  4. How the `bbl up` workflow works

    main

    The bbl up command executes a multi-step orchestration process to provision infrastructure and configure BOSH. The workflow follows these stages:

    1. Terraform Generation: Generates a Terraform template (terraform/bbl-template.tf) based on your IaaS, region, and load balancer type.
    2. Terraform Application: Runs Terraform to apply the template using variables from vars/bbl.tfvars.
    3. Output Mapping (BOSH vars): Transforms Terraform outputs (like subnet CIDRs and IP addresses) into BOSH create-env input files: vars/jumpbox-vars-file.yml and vars/director-vars-file.yml.
    4. BOSH Deployment: Executes bosh create-env via the BOSH CLI to create the jumpbox VM (using jumpbox-deployment) and the director VM (using bosh-deployment). The exact shell commands are saved to create-jumpbox.sh and create-director.sh.
    5. Cloud-Config Generation: Generates a base cloud-config template based on the IaaS, region, and load balancer configuration.
    6. Output Mapping (Cloud-Config): Maps Terraform outputs (network names, security groups, CIDR ranges, etc.) to cloud-config variables.
    7. Cloud-Config Update: Updates the director's cloud configuration by running bosh update-cloud-config.
  5. What are Plan Patches?

    main

    A plan patch is a method of applying multiple modifications to a bbl plan using override files. Instead of modifying the core bbl logic, you provide override files in specific locations that bbl reads and merges into its plan during bbl up.

    Supported override locations:

    • terraform/ directory
    • cloud-config/ directory
    • {create,delete}-{jumpbox,director}.sh scripts

    Plan patches are considered experimental and should be used with caution. They are useful for complex deployments like BOSH-lite on GCP, CF Isolation Segments on public clouds, or managing K8s clusters via cfcr.

  6. Deploy CFCR with bbl using the cfcr-aws patch

    main

    To deploy Cloud Foundry Container Runtime (CFCR) on AWS using the Bosh Bootloader (bbl), follow these steps to apply the cfcr-aws patch and deploy the necessary manifests.

    1. Initialize the environment with the patch

    Create a new environment and copy the patch files into it before running bbl up:

    mkdir banana-env && cd banana-env
    bbl plan --name banana-env
    cp -r bosh-bootloader/plan-patches/cfcr-aws/. .
    
    bbl up
    eval "$(bbl print-env)"

    2. Prepare the kubo-deployment repository

    Clone the kubo-deployment repository and export its path as KD to simplify subsequent commands:

    git clone git@github.com:cloudfoundry-incubator/kubo-deployment.git
    export KD=$(pwd)/kubo-deployment

    3. Upload the required stemcell

    Use the version specified in the CFCR manifest to upload the correct AWS Xen HVM Ubuntu stemcell:

    bosh upload-stemcell https://bosh.io/d/stemcells/bosh-aws-xen-hvm-ubuntu-xenial-go_agent?v=$(bosh int ${KD}/manifests/cfcr.yml --path=/stemcells/0/version)

    4. Apply custom cloud configuration

    Update the cloud configuration using the AWS-specific VM extensions manifest:

    bosh update-config --name cfcr \\n${KD}/manifests/cloud-config/iaas/aws/use-vm-extensions.yml \\n--var deployment_name=cfcr \\n--type cloud \\n--vars-file <(bbl outputs)

    5. Deploy the CFCR manifest

    Deploy the CFCR deployment using the required ops files and variables:

    bosh deploy -d cfcr ${KD}/manifests/cfcr.yml \\n-o ${KD}/manifests/ops-files/iaas/aws/cloud-provider.yml \\n-o ${KD}/manifests/ops-files/use-vm-extensions.yml \\n-o ${KD}/manifests/ops-files/add-hostname-to-master-certificate.yml \\n-o ${KD}/manifests/ops-files/iaas/aws/lb.yml \\n--var deployment_name=cfcr \\n-l <(bbl outputs)
  7. SSH to the BOSH Director

    main
    Use the bbl ssh --director command to access the BOSH Director. This command performs a double SSH invocation: first, it opens a tunnel forwarding a random port to the jumpbox, and second, it initiates an interactive SSH session through that port to the director.
    bbl ssh --director
  8. Use the n-cf-gcp plan patch for multiple CF environments

    main

    The n-cf-gcp plan patch extends the default bbl behavior, which typically supports only one Cloud Foundry (CF) deployment, to support multiple CF environments.

    When using this patch:

    • It creates separate load balancers and DNS records for each environment.
    • Domain names for each environment are derived from the --lb-domain flag (e.g., cf0.banana-env.com, cf1.banana-env.com).
    • Important: To deploy CF correctly, you must use the cf-lb-ops.yml file to ensure load balancers are set up properly.
    bosh -d cf0 deploy $CF_D/cf-deployment.yml \
      -o $CF_D/operations/rename-network-and-deployment.yml \
      -v deployment_name=cf0 \
      -v network_name=default \
      -o cf-lb-ops.yml \
      -v env=0 \
      -v system_domain=cf0.banana-env.com
  9. Deploy a BOSH director on AWS using bbl up

    main

    Use the bbl up command to pave infrastructure, create a jumpbox, and deploy a BOSH director on AWS. This process typically takes 5-8 minutes.

    Upon completion, bbl will have generated:

    • A BOSH director
    • A jumpbox
    • BOSH director credentials
    • An SSH keypair for the director and deployed instances
    • The deployment manifest
    • A basic cloud config

    Important: The bbl state directory contains all files used to create the director. You should check this directory into version control to ensure you can later destroy or update the environment.

    bbl up \
    	--aws-access-key-id <INSERT ACCESS KEY ID> \
    	--aws-secret-access-key <INSERT SECRET ACCESS KEY> \
    	--aws-region us-west-1 \
    	--iaas aws
  10. Back up a BBL BOSH Director using BBR

    main

    To back up the BOSH Director in a BBL environment, use the bbr director command. You need the Director's IP, the SSH username (which is jumpbox for BBL), and the path to the Director's private key.

    Required Information:

    • DIRECTOR-IP: The internal or public IP of the BOSH Director. Use bbl director-address to find it (remove the port number).
    • BBR-USERNAME: For BBL, this is always jumpbox.
    • BOSH-DIRECTOR-PRIVATE-KEY-PATH: The path to the BOSH Director's private key. Use bbl director-ssh-key to find the context/path.

    Note: If using the BOSH_ALL_PROXY method, ensure your environment is correctly configured before running the command.

    bbr director \
        --host DIRECTOR-IP \
        --username jumpbox \
        --private-key-path BOSH-DIRECTOR-PRIVATE-KEY-PATH \
    backup
  11. Clean up orphaned IaaS resources with bbl cleanup-leftovers

    main

    Use bbl cleanup-leftovers when bbl destroy is insufficient. This is useful if:

    • bbl destroy failed and lost state information.
    • You no longer have access to the bbl-state-dir.
    • You want to delete resources created outside of bbl.
    • Acceptance tests failed and orphaned IaaS resources.

    Unlike bbl destroy, this command does not require a bbl-state-dir. It functions similarly to a forced destruction but relies on filtering resources by name or tag.

    Warning: You MUST use the --filter flag. If omitted, bbl will attempt to delete ALL resources in your IaaS account.

    # Example: Cleaning up AWS resources with a specific filter
    export BBL_IAAS=aws
    export BBL_AWS_SECRET_ACCESS_KEY=foo
    export BBL_AWS_ACCESS_KEY_ID=bar
    
    bbl cleanup-leftovers --filter malawi