Biocluster Singularity: Difference between revisions

Revision as of 15:40, 20 January 2023

Containers contain applications in a way that keep them isolated from the host system that they run on. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
Singularity is a program that can launch containers securely in a HPC environment. Other similar programs are Docker, LXD, and Kubernetes.
This guide allows you to run pre-built singularity and docker images. It DOES NOT allow you to create the container. Creating a container requires root access. If you need to create a container, you will need to do this on another machine and then you can copy it over to the biocluster to run.

module load singularity/3.8.1

This can not be done on the Biocluster. It has to be done on your local machine. This is a security risk to allow users to create or edit containers on a HPC resource

You need to use the biologin nodes (biologin-0 and biologin-1) to download containers
This will not work from the compute nodes as they do not have access to the internet
Download a pre-existing container from Singularity Hub - https://singularity-hub.org/. singularity-r is an example R container

singularity pull --name r-base_latest.sif shub://nickjer/singularity-r

Download a pre-existing container from Docker - https://hub.docker.com/. r-base is an example R container

singularity pull docker://r-base

Do not use the biologin nodes (biologin-0 and biologin-1) to run containers. The login nodes are only allowed to be used to download containers
From a compute node or a SLURM job script, run the following. r-base_latest.sif is name of the container. R is the program to run in the container. This can be bash to get a command line.

singularity exec r-base_latest.sif R

Your home folder is automatically mounted
You should have access to any group folders in /home/groups, /home/labs, and /private_stores
The local /scratch space is accessible.

If the container does run properly or can't find the command it is trying to run, its possible that the program exists in the /home folder within the singularity container. Since we automatically mount the biocluster's /home folder in the container, it then can't find the command. To get around this, you can remount the biocluster's /home folder in a new directory

singularity exec --no-mount hostfs --bind /home:/newhome container.sif

@@ Line 3: / Line 3: @@
 * Containers contain applications in a way that keep them isolated from the host system that they run on.  Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
 * Singularity is a program that can launch containers securely in a HPC environment.  Other similar programs are Docker, LXD, and Kubernetes.
-* This guide allows you to run pre-built singularity and docker images.  It does not allow you to create the container.  Creating a container requires root access.  If you need to create a container, you will need to do this on another machine and then you can copy it over to the biocluster to run.
+* This guide allows you to run pre-built singularity and docker images.  It '''DOES NOT''' allow you to create the container.  Creating a container requires root access.  If you need to create a container, you will need to do this on another machine and then you can copy it over to the biocluster to run.
 == Module ==