DOCSEARCHbyAlgolia
Submit Docs

Apptainer (formerly Singularity) Documentation

repository·master·Indexed 25 days ago

https://github.com/apptainer/singularity

An open-source container platform designed for shared systems and High Performance Computing (HPC) environments. It features a single-file SIF format, integration with GPUs and parallel filesystems, and support for SCIF images. Documentation covers building images, managing CLI plugins, end-to-end testing, and the MAKEIT build system.

Tokens
12.6K
Snippets
23
Records
106
Agent score
83%

What's inside Apptainer

  1. Overview of MAKEIT build system

    master
    MAKEIT is a small, platform-independent build system designed to generate native, non-recursive Makefiles tailored to the host system. It transforms Makefile fragments and module configuration files into Makefiles that are compatible with various versions of Make, including GNU, BSD, and SVR4. It is designed to be lightweight enough to be included directly within individual projects.

  3. Build a Singularity image with SCIF support

    master

    Use the singularity build command to create a Singularity image from a definition file. When using the Scientific Filesystem (SCIF), you can build containers that include multiple entrypoints, modular environments, libraries, and executables.

    sudo singularity build cowsay.simg Singularity.cowsay

  4. Compile and install Singularity from source

    master

    To build and install Singularity from the cloned repository, use mconfig, make, and make install.

    Basic build sequence:

    1. Run ./mconfig to configure.
    2. Enter the ./builddir directory.
    3. Run make.
    4. Run sudo make install.

    Customizing the build: You can use ./mconfig options to modify the build. For example, to specify a different build folder and installation prefix: ./mconfig -b ./buildtree -p /usr/local

    Check ./mconfig -h for all available options.

    ./mconfig
cd ./builddir
make
sudo make install

singularity --version

  5. Run Singularity help end-to-end tests

    master

    To run the help content end-to-end tests, use the make command from the builddir directory with the SINGULARITY_E2E environment variable set.

    To verify specific tests or run with verbosity, you may need to modify the Makefile to include the -v (verbose) and -r (run specific test) flags for the Go test command (e.g., go test -v -r helpContentTests).

    SINGULARITY_E2E=1 make -C builddir e2e-test

  6. Add a new topic to End-to-End (E2E) testing

    master

    To add a new functional area (topic) to the Singularity E2E test suite, follow these steps:

    1. Register the topic in suite.go: Add a new t.Run entry in suite.go using your topic name as a runtime-hook.
    2. Create a topic directory: Create a dedicated directory for your tests (e.g., e2e/yourtopic).
    3. Initialize test files: Create a source file for the tests (e.g., yourtopic.go) and an optional file for helpers (e.g., yourtopic_utils.go).
    4. Define the package: Ensure the package declaration in your test file matches the name used in suite.go.
    5. Configure testing environment: Define a struct to hold testing settings (using github.com/kelseyhightower/envconfig) and a package-level variable to store them.
    6. Implement the entry point: Create a RunE2ETests(t *testing.T) function that processes environment variables via envconfig.Process("E2E", &testenv) and then triggers individual tests using t.Run.
    7. Write tests: Implement standard Go tests (e.g., func TestYourTopic(t *testing.T)) and call them from your entry point.

  7. Report software bugs and code issues via GitHub

    master

    For issues involving the Singularity source code or when you need to share debug output, use the GitHub issue boards:

    • Singularity Software Issues: Use the Singularity Issues board for most software-related bugs.
    • User Documentation Feedback: Use the User Documentation issues board for questions, feedback, or suggestions regarding user guides.
    • Admin Documentation Feedback: Use the Admin Documentation issues board for questions, feedback, or suggestions regarding administrative guides.

  8. Naming convention for contributing new definition files

    master

    When contributing new definition files to the examples/contrib/ directory, use a hyphen (-) delimited format for the filename. The format should follow this structure:

    1. Base distribution name and version (e.g., centos7 or ubuntu_trusty)
    2. Target nomenclature describing the container (e.g., tensorflow)
    3. Relevant version strings for the application or workflow
    4. The file must end in .def

    Example filename: examples/contrib/debian84-tensorflow-0.10.def

    examples/contrib/debian84-tensorflow-0.10.def

  9. Install golangci-lint for Singularity development

    master

    If you are contributing to the Singularity source code, install golangci-lint to run local checks. Every pull request must pass these linting checks.

    After installation, ensure $(go env GOPATH)/bin is added to your PATH.

    curl -sSfL https://raw.githubusercontent.com/golangci/golangci-lint/master/install.sh | sh -s -- -b $(go env GOPATH)/bin v1.43.0

echo 'export PATH=$PATH:$(go env GOPATH)/bin' >> ~/.bashrc
source ~/.bashrc

  10. Install Go for Singularity

    master

    Singularity is written in Go and may require a version newer than your distribution's repositories. It is recommended to install the latest version from official binaries.

    1. Download the Go tar.gz archive to /tmp.
    2. Extract it to /usr/local (remove any existing /usr/local/go first if updating).
    3. Add /usr/local/go/bin to your PATH.

    Note: Ensure you adjust GOVERSION, OS, and ARCH as needed for your environment.

    export GOVERSION=1.17.3 OS=linux ARCH=amd64

wget -O /tmp/go${GOVERSION}.${OS}-${ARCH}.tar.gz \
  https://dl.google.com/go/go${GOVERSION}.${OS}-${ARCH}.tar.gz
sudo tar -C /usr/local -xzf /tmp/go${GOVERSION}.${OS}-${ARCH}.tar.gz

echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc
source ~/.bashrc

  11. Build and install Singularity as an RPM

    master

    On RHEL, CentOS, or Fedora, you can build Singularity as an RPM package for easier distribution or management via yum/dnf.

    Method 1: Build from a release source tarball

    1. Install rpm-build, wget, and golang.
    2. Download the specific version's tarball.
    3. Run rpmbuild -tb on the tarball.
    4. Install the resulting RPM using rpm -ivh.

    Method 2: Build from the latest master

    1. Clone the repo.
    2. Run ./mconfig --only-rpm (this allows using older system Go versions by applying a patch).
    3. Run make -C builddir rpm.

    Customizing the RPM prefix: Use the RPMPREFIX variable during the make step: make -C builddir rpm RPMPREFIX=/opt/singularity

    # Method 1: From tarball
export VERSION=3.8.4
wget https://github.com/hpcng/singularity/releases/download/v${VERSION}/singularity-${VERSION}.tar.gz
rpmbuild -tb singularity-${VERSION}.tar.gz
sudo rpm -ivh ~/rpmbuild/RPMS/x86_64/singularity-${VERSION}-1.el7.x86_64.rpm

# Method 2: From master
./mconfig --only-rpm
make -C builddir rpm
sudo rpm -ivh ~/rpmbuild/RPMS/x86_64/singularity-3.8.4*.x86_64.rpm

  12. Build a Singularity CLI plugin

    master

    To build a CLI plugin, you must use source code that matches the version of the Singularity binary you are targeting. You can identify the exact commit used to build your current Singularity binary by running singularity version. The suffix (e.g., .g7998470e7) indicates the Git commit hash.

    1. Clone the Singularity repository.
    2. Checkout the commit hash identified from your Singularity version.
    3. Use the singularity plugin compile command pointing to your plugin directory.

    Note: Currently, the plugin source code must reside inside the Singularity source code tree during the compilation process.