DOCSEARCHbyAlgolia
Submit Docs

pi-gen

repository·master·Indexed 25 days ago

https://github.com/rpi-distro/pi-gen

A tool used to create Raspberry Pi OS images and custom images based on Raspberry Pi OS/Raspbian. It utilizes a stage-based build system to construct the root filesystem and deploy final images, supporting both 32-bit (master branch) and 64-bit (arm64 branch) architectures. The tool provides options for building on Debian-based systems or via Docker for isolation and cross-platform support.

Tokens
1.7K
Snippets
5
Records
11
Agent score
35%

What's inside pi-gen

  1. Configure the build via the config file

    master

    When you run build.sh, it automatically sources a file named config in the current working directory. This file is a bash shell fragment used to set environment variables that control the build process. You can also override these values by passing a specific config file to the build script using the -c flag.

    ./build.sh -c myconfig

  2. Clone the pi-gen repository

    master

    Clone the repository to your build machine.

    Important Requirements:

    • The base path must NOT contain spaces, otherwise debootstrap will fail.
    • For 32-bit images, use the master branch.
    • For 64-bit images, use the arm64 branch.
    • Avoid using --depth 1 (shallow clone) on development machines.
    git clone https://github.com/RPI-Distro/pi-gen.git

  3. Configure build stages and skip stages

    master

    The build process is divided into stages (0 through 5). You can skip specific stages to speed up development or build a specific version (like Lite or Desktop).

    Skipping Stages

    • To skip a stage: Place an empty file named SKIP in the directory of the stage you want to exclude (e.g., ./stage3/SKIP).
    • To skip image exports: Place an empty file named SKIP_IMAGES in the directories containing EXPORT_* files (currently stage2, stage4, and stage5).

    Example: Building a Lite System

    To build up to Stage 2 (Lite), skip stages 3, 4, and 5, and skip image exports in stages 4 and 5:

    echo "IMG_NAME='raspios'" > config
touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
sudo ./build.sh  # or ./build-docker.sh
    # Example for building a lite system
echo "IMG_NAME='raspios'" > config
touch ./stage3/SKIP ./stage4/SKIP ./stage5/SKIP
touch ./stage4/SKIP_IMAGES ./stage5/SKIP_IMAGES
sudo ./build.sh  # or ./build-docker.sh

  4. Configure Cloud-Init for Raspberry Pi OS

    master

    Cloud-Init support is available for Raspberry Pi OS via specific configuration files. To ensure the imager correctly creates filesystem entries and to provide instance configuration, you must include the following files in the configuration directory:

    • files/network-config: Required for network configuration. Without this, the imager may fail to create the correct filesystem entry.
    • files/user-data: Required for user data and example configurations.
    • files/meta-data: Used for Cloud-init instance configuration.

    For detailed information on available Raspberry Pi specific modules, refer to the official Cloud-Init documentation.

  5. Build using Docker

    master

    You can use Docker to perform builds inside a container, which provides isolation and allows building on non-Debian based systems (e.g., Fedora).

    Prerequisites: Ensure binfmt-support is installed on the host machine to allow ARM emulation via qemu-user-static.

    Basic Build Workflow:

    1. Edit your config file.
    2. Run ./build-docker.sh.
    3. Finished images will be located in the deploy/ folder.

    Advanced Docker Options:

    • Continue after errors: Use CONTINUE=1 ./build-docker.sh to edit scripts and resume.
    • Preserve container: Use PRESERVE_CONTAINER=1 ./build-docker.sh to prevent the build container from being removed after a successful build (useful for incremental changes).
    • Inspect container on failure: Enter a shell within the failed container using: sudo docker run -it --privileged --volumes-from=pigen_work pi-gen /bin/bash
    • Custom Docker arguments: Pass additional arguments to the docker run command via the PIGEN_DOCKER_OPTS environment variable. For example, to add a host entry: export PIGEN_DOCKER_OPTS="--add-host foo:192.168.0.23"
    vi config         # Edit your config file. See above.
./build-docker.sh

  6. Understand the pi-gen build process and stage anatomy

    master

    The build process iterates through stage directories in alphanumeric order.

    Stage Execution Flow:

    1. Skip Check: If a stage directory contains a file named SKIP, it is bypassed.
    2. Prerun: prerun.sh is executed (typically to copy the build directory).
    3. Subdirectory Execution: Inside each stage, pi-gen iterates through subdirectories and runs install scripts.

    Install Script Naming: Scripts must be named with a two-digit padded number (e.g., 01-name.sh).

    Supported Files in Stage Subdirectories:

    • 00-run.sh: A shell script executed during the build. Must be executable.
    • 00-run-chroot.sh: A shell script executed inside the image's chroot. Must be executable.
    • 00-debconf: File containing configuration passed to debconf-set-selections.
    • 00-packages: A list of packages to install (space-separated).
    • 00-packages-nr: Packages installed using --no-install-recommends.
    • 00-patches: A directory of patch files applied via quilt. If an EDIT file exists, the build pauses for manual editing.

    Image Generation: If a stage contains EXPORT_NOOBS or EXPORT_IMAGE, that stage is added to the list of images to generate.

  8. Build 64-bit images

    master

    To generate a 64-bit image, you must use the arm64 branch of the repository.

    Workflow:

    1. Clone the specific branch: git clone --branch arm64 https://github.com/RPI-Distro/pi-gen.git
    2. Follow the standard build instructions.

    Note for Raspberry Pi users: If you are building a 64-bit image from a Raspberry Pi currently running a 32-bit OS, you must add arm_64bit=1 to your config.txt and reboot to ensure the kernel is running in 64-bit mode. This requires a 64-bit capable processor (Pi Zero 2, Pi 3, or Pi 4).

    git clone --branch arm64 https://github.com/RPI-Distro/pi-gen.git

  9. Troubleshoot binfmt_misc and ARM emulation errors

    master

    If you encounter errors like Exec format error or Couldn't load the binfmt_misc module when building on an x86_64 host, it means the kernel cannot execute ARM binaries.

    Resolution Steps:

    1. Load the module: Run sudo modprobe binfmt_misc.
    2. Verify files: Ensure the following are available on your host:
      • /lib/modules/$(uname -r)/kernel/fs/binfmt_misc.ko
      • /usr/bin/qemu-arm-static
    3. WSL Users: If using Windows Subsystem for Linux, you may need to run: sudo update-binfmts --enable.

  11. Reference: pi-gen configuration environment variables

    master
    The following environment variables can be set in your config file to customize the image build. Note that build.sh parses the config file first, and command-line arguments passed via -c are parsed after, allowing for overrides.