BOSH CLI

repository·main·Indexed 41 days ago

https://github.com/cloudfoundry/bosh-cli

Command-line interface for interacting with Bosh Directors to manage deployments, releases, stemcells, and UAA services. Includes tools for environment management, task control, resource cleanup, and programmatic access to the Director API via Go packages.

Tokens
33.6K
Snippets
114
Records
157
Agent score
52%

What's inside BOSH CLI

  1. Understand the inputs and outputs of the `bosh delete` command

    main

    The bosh delete command is used to tear down existing BOSH deployments.

    Inputs required for deletion:

    • Combo manifest: Specifically the installation manifest.
    • CPI release: The release associated with the CPI.

    Actions performed by deletion:

    • Produces: A local installation of the CPI.
    • Deletes: The previously deployed remote VM, its associated disk(s), and the stemcell.
  2. Understand the inputs and outputs of the `bosh deploy` command

    main

    The bosh deploy command is used to install a Cloud Platform Interface (CPI) locally and then deploy BOSH and its associated jobs onto remote cloud infrastructure.

    Inputs required for deployment:

    • Combo manifest: Contains both installation and deployment manifests.
    • Stemcell: The root file system used for the deployment.
    • CPI release: The release containing the CPI.
    • BOSH release: The release containing BOSH itself.

    Outputs produced by deployment:

    • A local installation of the CPI.
    • A remote deployment of BOSH (and its multiple jobs) running on a single VM or container within the cloud infrastructure targeted by the CPI.
  3. Understand the BOSH CLI deployment flow

    main

    The bosh-init deploy command follows a specific sequence of steps to manage the lifecycle of a Virtual Machine (VM) and its associated components. The CLI automates the transition from a deployment manifest to a running VM by interacting with a Cloud Provider Interface (CPI) and an agent running on the VM via an SSH tunnel and Mbus.

    Deployment Lifecycle Steps:

    1. Validation: The CLI checks for changes in the manifest, release, or stemcell. It parses the manifest into a deployment manifest (defining networks, resource_pools, disk_pools, and jobs) and a CPI configuration (derived from the cloud_provider section).
    2. CPI Installation: The CPI release is compiled locally. The CPI job (specified by cloud_provider.template.job) uses properties from cloud_provider -> properties to render templates.
    3. Stemcell Upload: The CLI calls the CPI create_stemcell method.
    4. VM Cleanup: If a VM already exists, the CLI connects to the agent, stops services, unmounts disks, and deletes the VM.
    5. VM Creation: The CLI calls the CPI create_vm method and persists the VM CID in a deployment state file.
    6. SSH Tunneling: A reverse SSH tunnel is established to allow the VM agent to communicate back to the machine running the CLI.
    7. Agent Readiness: The CLI pings the agent via the provided Mbus URL until it responds.
    8. Disk Management: If requested via persistent_disk_pool or persistent_disk in the manifest, the CLI calls create_disk and attach_disk via the CPI, then issues a mount_disk request to the agent.
    9. Job Application:
      • Stop: A stop message is sent to prepare the agent.
      • Apply: The CLI renders job templates, uploads them to the agent's blobstore (<mbus URL>/blobs), and sends an apply message containing package lists and template specs.
      • Start: Once applied, a start message is sent to begin running the jobs.
  4. Programmatic access to the Director API via Client Libraries

    main

    The bosh-cli repository provides the director and uaa Go packages, which are intended for developers who need to interact with the Bosh Director API programmatically within their own Go projects.

    To implement programmatic access, you can use these packages to perform operations typically handled by the CLI. For a practical implementation guide, refer to the docs/example.go file in this repository.

    // See docs/example.go for a live short usage example
  5. Set up a development workstation for bosh-cli

    main

    To develop on bosh-cli, you need gcc (or an equivalent) and a package manager (like brew or apt-get). You must also install go, git, and hg (Mercurial), as Go requires these for dependency management.

    1. Install Dependencies

    On macOS using Homebrew:

    brew update
    brew install go
    brew install git
    brew install hg

    2. Clone the Repository

    Use go get to download the source code into your $GOPATH:

    go get -d github.com/cloudfoundry/bosh-cli
    cd $GOPATH/src/github.com/cloudfoundry/bosh-cli
    brew update
    brew install go
    brew install git
    brew install hg
    
    go get -d github.com/cloudfoundry/bosh-cli
    cd $GOPATH/src/github.com/cloudfoundry/bosh-cli
  6. Build the bosh CLI binary

    main

    Once you have cloned the repository and are working within the $GOPATH/src/github.com/cloudfoundry/bosh-cli directory, you can build the CLI using the provided build script. The resulting bosh binary will be placed in the out/ directory.

    bin/build
  7. Configure persistent disks in a deployment manifest

    main

    To request a disk during deployment, you can use one of two methods in your deployment manifest:

    1. Using Disk Pools (Recommended): Use this method if you need to specify specific cloud_properties for the disk. Add the persistent_disk_pool property to a job, referencing a pool defined in the top-level disk_pools section.
    2. Specifying Size Directly: Add the persistent_disk property to a job to specify the size of the persistent disk.

    When a disk is requested, the CLI performs the following:

    • Calls the CPI create_disk method.
    • Persists the disk CID in the deployment state file.
    • Calls the CPI attach_disk method.
    • Issues a mount_disk request to the agent on the VM.
    # Example manifest structure for disks
    
    disk_pools:
      - name: my_disk_pool
        cloud_properties:
          # ... specific cloud properties
    
    jobs:
      - name: my_job
        persistent_disk_pool: my_disk_pool
        # OR
        # persistent_disk: 20G
  8. Run BOSH CLI tests

    main
    The BOSH CLI uses a multi-tiered testing strategy. Each package contains its own unit tests, and the integration package contains integration tests. You can execute the entire test suite using the provided bin/test script.
    bin/test
  9. Execute BOSH CLI acceptance tests via Fly

    main

    Acceptance tests are designed to exercise core CLI commands such as deployment, deploy, and delete. They are not intended for verifying CPI compatibility or BOSH releases.

    To run these tests using fly, you should base your one-off build on the pipeline's test-acceptance job to ensure the correct image, CPI release, and stemcell are used. You only need to override the source under test.

    Note: The -p flag is required because the task initializes its own Docker daemon by sourcing start-docker.

    fly -t bosh execute -p \
      -c ci/tasks/test-acceptance.yml \
      --inputs-from bosh-cli/test-acceptance \
      --image docker-cpi-image \
      -i bosh-cli=<path-to-source-dir>
  10. How Bosh CLI autocompletion works

    main

    The Bosh CLI implements autocompletion by dynamically discovering commands, flags, and arguments from the internal BoshOpts structure using reflection.

    When the CLI is invoked with the completion or __complete commands, it builds a command tree where:

    1. Commands are derived from struct fields tagged with command and description.
    2. Flags are derived from fields tagged with long, short, default, and description. If a field is tagged with env, the environment variable name is included in the flag's usage description.
    3. Dynamic Completion is provided for specific flags and arguments via a CompleteFunctionsMap. This allows the CLI to suggest valid values (like environment names or deployment names) based on the current context.
    4. Context Awareness: The completion engine tracks context such as environment, config (path), and deployment to resolve flag values used in completion logic.
  11. How CLI autocompletion caching works

    main

    The bosh-cli uses a local cache to store autocompletion results to improve performance. The cache is managed by the CompleteCache component and follows these principles:

    • Storage Location: The cache is stored in a directory named completion-cache located in the same parent directory as your Bosh configuration file.
    • Cache Keying: Cache entries are uniquely identified based on a combination of the environment name, the deployment name, and an optional group name. This ensures that autocompletion suggestions for one environment or deployment do not leak into another.
    • Expiration: Cached values have a hardcoded expiration period of 15 seconds. If the cached data is older than this period, it is considered invalid and is automatically erased from the store.
    • Data Format: Values are stored as JSON objects containing a timestamp and a list of string values.
  12. How Bosh CLI handles UAA authentication

    main

    The Bosh CLI uses a session-based approach to handle UAA authentication. When the Director's authentication type is set to uaa, the session determines how to obtain tokens based on the credential type:

    1. UAA Client: If the credentials are identified as a UAA client (creds.IsUAAClient()), the session uses boshuaa.NewClientTokenSession to manage token acquisition.
    2. UAA User: If the credentials are a standard user (creds.IsUAA()), the session uses boshuaa.NewAccessTokenSession with a refreshable access token (using AccessToken and RefreshToken) to manage the session.

    This ensures that the boshdir.Director always has a valid TokenFunc to call when making requests to the Director API.