Difference between revisions of "Biocluster Singularity"
Jump to navigation
Jump to search
(Created page with "__TOC__ == Module == *Load Singularity Module to run any singularity commands <pre>module load singularity/3.4.1</pre> == Creating Containers == * This can not be done on the...") |
(→Download Containers) |
||
| (18 intermediate revisions by the same user not shown) | |||
| Line 1: | Line 1: | ||
__TOC__ | __TOC__ | ||
| + | == About == | ||
| + | * Containers contain applications in a way that keep them isolated from the host system that they run on. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package. | ||
| + | * Singularity is a program that can launch containers securely in a HPC environment. Other similar programs are Docker, LXD, and Kubernetes. | ||
| + | |||
== Module == | == Module == | ||
*Load Singularity Module to run any singularity commands | *Load Singularity Module to run any singularity commands | ||
| − | <pre>module load singularity/3. | + | <pre>module load singularity/3.8.1</pre> |
== Creating Containers == | == Creating Containers == | ||
| − | * | + | * You '''CAN NOT''' create containers on the Biocluster. It has to be done on your local machine. Creating a container requires root access which we do not allow. |
== Download Containers == | == Download Containers == | ||
| − | * You need to use the biologin nodes (biologin- | + | * You need to use the biologin nodes (biologin-2 and biologin-3) to download containers |
* This will not work from the compute nodes as they do not have access to the internet | * This will not work from the compute nodes as they do not have access to the internet | ||
* Download a pre-existing container from Singularity Hub - https://singularity-hub.org/. singularity-r is an example '''R''' container | * Download a pre-existing container from Singularity Hub - https://singularity-hub.org/. singularity-r is an example '''R''' container | ||
<pre> | <pre> | ||
| − | singularity pull --name | + | singularity pull --name r-base_latest.sif shub://nickjer/singularity-r |
</pre> | </pre> | ||
* Download a pre-existing container from Docker - https://hub.docker.com/. r-base is an example '''R''' container | * Download a pre-existing container from Docker - https://hub.docker.com/. r-base is an example '''R''' container | ||
| Line 25: | Line 29: | ||
singularity exec r-base_latest.sif R | singularity exec r-base_latest.sif R | ||
</pre> | </pre> | ||
| + | |||
| + | == Storage == | ||
| + | * Your home folder is automatically mounted | ||
| + | * You should have access to any group folders in /home/groups, /home/labs, and /private_stores | ||
| + | * The local /scratch space is accessible. | ||
| + | |||
| + | == Issues == | ||
| + | * If the container does run properly or can't find the command it is trying to run, its possible that the program exists in the /home folder within the singularity container. Since we automatically mount the biocluster's /home folder in the container, it then can't find the command. To get around this, you can remount the biocluster's /home folder in a new directory | ||
| + | <pre> | ||
| + | singularity exec --no-mount hostfs --bind /home:/newhome container.sif | ||
| + | </pre> | ||
| + | |||
| + | == References == | ||
| + | * [https://singularity.lbl.gov/ https://singularity.lbl.gov/] | ||
Latest revision as of 09:27, 1 June 2023
Contents
About[edit]
- Containers contain applications in a way that keep them isolated from the host system that they run on. Containers allow a developer to package up an application with all of the parts it needs, such as libraries and other dependencies, and ship it all out as one package.
- Singularity is a program that can launch containers securely in a HPC environment. Other similar programs are Docker, LXD, and Kubernetes.
Module[edit]
- Load Singularity Module to run any singularity commands
module load singularity/3.8.1
Creating Containers[edit]
- You CAN NOT create containers on the Biocluster. It has to be done on your local machine. Creating a container requires root access which we do not allow.
Download Containers[edit]
- You need to use the biologin nodes (biologin-2 and biologin-3) to download containers
- This will not work from the compute nodes as they do not have access to the internet
- Download a pre-existing container from Singularity Hub - https://singularity-hub.org/. singularity-r is an example R container
singularity pull --name r-base_latest.sif shub://nickjer/singularity-r
- Download a pre-existing container from Docker - https://hub.docker.com/. r-base is an example R container
singularity pull docker://r-base
Running Containers[edit]
- Do not use the biologin nodes (biologin-0 and biologin-1) to run containers. The login nodes are only allowed to be used to download containers
- From a compute node or a SLURM job script, run the following. r-base_latest.sif is name of the container. R is the program to run in the container. This can be bash to get a command line.
singularity exec r-base_latest.sif R
Storage[edit]
- Your home folder is automatically mounted
- You should have access to any group folders in /home/groups, /home/labs, and /private_stores
- The local /scratch space is accessible.
Issues[edit]
- If the container does run properly or can't find the command it is trying to run, its possible that the program exists in the /home folder within the singularity container. Since we automatically mount the biocluster's /home folder in the container, it then can't find the command. To get around this, you can remount the biocluster's /home folder in a new directory
singularity exec --no-mount hostfs --bind /home:/newhome container.sif
