melange

repository·main·Indexed 20 days ago

https://github.com/chainguard-dev/melange

A tool for building secure, multi-architecture apk packages using declarative, pipeline-oriented YAML configurations. Melange is often used as a precursor to building OCI container images with apko and supports various build runners including Docker, Bubblewrap, and QEMU.

Tokens
42.6K
Snippets
134
Records
165
Agent score
69%

What's inside melange

  1. How Melange contains builds using bubblewrap

    main

    To ensure build isolation and prevent pipeline commands from modifying the host system, Melange executes all runs commands inside a virtual container created by bubblewrap (bwrap).

    The guest directory serves as the root filesystem for this container. This allows commands to assume standard filesystem locations (like /usr/bin/gcc) while actually interacting with the files laid out in the guest directory (e.g., ${GUEST_DIR}/usr/bin/gcc).

  2. Understand the Melange directory model

    main

    Melange utilizes three distinct directory types to manage the build lifecycle and ensure source integrity:

    1. Source directory: The location of your original build sources (defaults to the current directory).
    2. Workspace directory: A directory where sources are copied to allow for compilation and manipulation without altering the original source files. The workspace is bind-mounted into the guest at /home/build.
    3. Guest directory: A temporary directory (often in /tmp) where the actual build process occurs. It contains the laid-out apk package dependencies and the environment where commands are executed.

    Note on Persistence: Because the workspace is bind-mounted into the guest, any changes made by the pipeline are persisted in the workspace directory even after the guest is cleaned up. If you set the workspace directory to be the same as your source directory, all build changes will persist in your source folder.

  3. Understand Post-build Linting in Melange

    main
    After a package is built, Melange automatically runs a series of lint checks to catch common misconfigurations and mistakes. These linters ensure that packages follow standard filesystem conventions and security best practices. By default, all available linters are enabled.
  4. Building for alternate architectures

    main

    Melange supports cross-architecture builds using the following mechanisms:

    • Native Builds: If building for the host architecture (e.g., amd64 on amd64), all runs commands execute natively.
    • Cross-Architecture Builds: If building for a different architecture (e.g., arm64 while on amd64), commands are executed via binfmt_misc user-mode emulation.

    Requirement: To enable cross-architecture builds, binfmt_misc must be installed on the host system.

  5. Configure package repositories and build dependencies in `contents`

    main

    The contents node within environment defines where to find packages, how to validate them, and which packages to install as build-time dependencies.

    Local Building

    When building locally, you must include Wolfi package repository information. This is not required when submitting to the Wolfi OS repository.

    repositories

    List of repositories to fetch packages from. Warning: Do not mix Alpine apk repositories with Wolfi apk repositories.

    keyring

    List of keys used to validate the authenticity of a repository.

    packages

    List of packages to install in the build environment for running the pipeline (build-time dependencies).

    You can specify versions using the following syntax:

    • go>1.21: Installs anything newer than 1.21 (excluding 1.21).
    • foo=~4.5.6: Installs any version starting with "4.5.6" (e.g., 4.5.6-r7).
    • python3: Installs the latest stable version.
    environment:
      contents:
        repositories:
          - https://packages.wolfi.dev/os
        keyring:
          - https://packages.wolfi.dev/os/wolfi-signing.rsa.pub
        packages:
          - busybox
          - ca-certificates-bundle
          - go>1.21
          - foo=~4.5.6
          - python3
  6. Configure package update strategies in melange

    main

    Melange allows you to describe how a package can be updated by specifying an update: configuration block in your melange file. This does not automatically keep packages up to date, but provides the metadata necessary for automation tools to identify and fetch new versions.

    There are four primary update sources:

    1. release-monitor: Queries https://release-monitoring.org/ using a specific identifier.
    2. github: Queries the GitHub GraphQL API using an org/repo identifier.
    3. git: Uses local git checkouts (useful for unsupported providers).
    4. oci: Queries OCI container registry image tags using the go-containerregistry library.
    package:
      name: example-pkg
      version: 1.0.0
    
    update:
      enabled: true
      # Choose one of the following:
      release-monitor:
        identifier: 123
      github:
        identifier: org/repo
      git: {}
      oci:
        identifier: cgr.dev/image/name
  7. Use Go workspace mode with `go/bump`

    main

    The go/bump pipeline supports Go workspace mode (using go.work files). When the work parameter is set to true, the pipeline will use go work vendor instead of go mod vendor for dependency management.

      - uses: go/bump
        with:
          deps: github.com/sirupsen/logrus@v1.9.3
          work: true
  8. Define test blocks in YAML

    main

    You can define test: blocks at the top level (to test the 'main' package) or inside a subpackages definition. Each block can specify an environment (to define required packages) and a pipeline (to define the actual test commands).

    If you are decoupling your tests into a separate file from your build file, you must include a mandatory (but empty) pipeline: key at the top level to satisfy the configuration parser.

    package:
      name: php-8.2-msgpack
      version: 2.2.0
      epoch: 0
      description: "Tests for PHP extension msgpack"
      copyright:
        - license: BSD-3-Clause
    
    # This is mandatory to avoid config parsing errors
    pipeline:
    
    test:
      environment:
        contents:
          packages:
            - wolfi-base
            - apk-tools
      pipeline:
        - runs: |
            # Test commands go here
  9. Use melange packages with apko

    main

    To use an .apk package built with Melange in an image built with apko, you have two primary options:

    1. Package Repository: Upload the built .apk to a standard package repository.
    2. Local Repository: Use a "local" repository. This is useful for running both Melange and apko builds in the same directory or GitHub repository without requiring external storage. This pattern allows for seamless integration in CI/CD workflows.
  10. How Melange testing works

    main

    Melange allows you to test packages using a test command. Tests are defined using a test: keyword block, which follows a similar structure to the build pipeline.

    There are two distinct environments involved in a test run:

    1. Test environment (workspace): A single shared workspace mounted as the CWD (Current Working Directory) for each test run. You can use the --source-dir flag to make external files (like test fixtures) available in this workspace.
    2. Execution environment (guest): A fresh container built via apko for each test. This container contains the Package Under Test (PUT) by default, plus any additional packages you specify in test.environment.contents.packages. This ensures tests run in a clean, isolated environment containing only the necessary runtime dependencies.
  11. How the Melange build process works

    main

    The Melange build process follows these high-level steps:

    1. Dependency Resolution: Evaluates pipeline steps for needs sections and merges them into environment.contents.
    2. Environment Setup: Uses apko to create a tar stream of environment.contents packages and lays them out in the guest directory.
    3. Workspace Preparation: Creates the workspace directory, copies files from the source directory to it, and bind-mounts the workspace into the guest at /home/build.
    4. Execution: Runs each step in the pipeline (either uses or runs) inside the workspace.
    5. Output: Emits the final .apk package (and any subpackages) and generates/signs an APKINDEX if requested.
    6. Cleanup: Removes the guest and workspace directories.