In Summer 2024, the CS department has made available Apptainer software (formally named Singularity) for containers in the environment. This is available as a software module and can be used throughout the environment (including on SLURM nodes). Containers should not be run on portal nodes.

There are also three general purpose servers, apptainer[01-03], that may be used for building and running containers. These servers are accessible via SSH.

Apptainer is available as a module. To load and use it simply execute the following

~$ module load apptainer

By default for Apptainer containers, mountpoints such as home, project, and bigtemp directories are available along with certain system directories. This means that changes to files you make when using an image will persist outside of the image as well.

System directories examples (local to the server):

• /tmp
• /proc

Mounted Directories (remote storage):

• /u
• /l
• /s
• /p
• /bigtemp

To disable such directories from becoming available within a container, add the flag -c Apptainer commands such as exec or run.

Apptainer image files are named with the suffix Singularity Image Format (.sif). To create your own image, an Apptainer Definition file must be written first. A full guide on def files can be found on the official Apptainer website.

Once a def file has been created, the following can be done (replace <imagename> with the name of the resulting SIF image, and <defname> with the name of your def file)

~$ apptainer build <imagename>.sif <defname>.def

Apptainer Definition files are similar to a Dockerfile, and comparisions can be found here.

Note, when building images, be sure to check the file size. By default, Apptainer will create the SIF file in the current directory where the command was executed. For large images, it's recommended to use project directories for persistent storage or bigtemp for temporary image storage.

Apptainer provides built-in compatibility for Docker container images. For example, container images from Docker Hub can often be ran and built directly using Apptainer without modification. In such instances, Apptainer will automatically convert the container image to an Apptainer SIF image when using the build command

~$ apptainer build mylolcow.sif docker://ghcr.io/apptainer/lolcow

Further, Docker Hub images can also be run directly

~$ apptainer run docker://sylabsio/lolcow:latest

Dockerfiles can be rewritten to a syntactically similar Apptainer Definition file. For comparisions between the two, see the Apptainer page for Apptainer Definition file vs. Dockerfile.

More information on Apptainer's support for Docker and OCI containers can be found here.

To open an interactive shell within a container

~$ apptainer shell <imagename>.sif
Apptainer>

Note, after opening a shell, commands such as ls are available. As mentioned previously, home directories are available by default within an image

Apptainer> ls -al

To run commands within a given container, for example when a container is made for running a python program

~$ apptainer exec <imagename>.sif python3 mypythonprogram.py 

When an image is created from a def file, a runscript can also be defined to turn the image into more of an executable program. Read more about runscripts here.

~$ apptainer run <imagename>.sif
or
~$ ./<imagename>.sif

When using a GPU image, the host system's GPU drivers can be loaded into an image by including the flag --nv when running an Apptainer command

For example,

~$ apptainer exec --nv <imagename>.sif python3 mygpuprogam.py 

Be sure to reference the CS wiki page about slurm for additional information here.

After initializing an interactive job in slurm

~$ module purge
~$ module load apptainer
~$ apptainer shell /bigtemp/example/myimage.sif
Apptainer>

Below is a simple example of a possible sbatch script for running an image on a GPU system

#!/bin/bash

#SBATCH --cpus-per-task=1
#SBATCH --gpus=1
#SBATCH -t 02:00:00
#SBATCH -p gpu

module purge
module load apptainer
apptainer exec --nv myimage.sif python3 mygpuprogam.py