|
Size: 4484
Comment:
|
Size: 5319
Comment:
|
| Deletions are marked like this. | Additions are marked like this. |
| Line 1: | Line 1: |
| ## page was renamed from Services/Singularity | |
| Line 3: | Line 4: |
| = Singularity = This article explains the purpose and use of [[https://sylabs.io/singularity/|Singularity]]. |
= Apptainer = This article explains the purpose and use of [[https://apptainer.org/|Apptainer]], a fork of [[https://sylabs.io/singularity/|Singularity]]. [[https://apptainer.org/docs/user/latest/singularity_compatibility.html|Compatibility is explained here]] in detail. |
| Line 7: | Line 8: |
| Singularity is a container system for isolating software and computing environments from the host system. It allows to package software in '''highly portable''' and '''reproducible environments''' encapsulating all dependencies, including the operating system.<<BR>> It has similarities to Docker but it's security model is specifically engineered to be used in HPC environments and easy integration in managed client environments.<<BR>> |
Apptainer is a container system for isolating software and computing environments from the host system. It allows to package software in '''highly portable''' and '''reproducible environments''' encapsulating all dependencies, including the operating system.<<BR>> It has similarities to Docker but it's security model is specifically engineered to be used in HPC environments and for easy integration in managed client environments.<<BR>> |
| Line 10: | Line 11: |
| A typical use case is to install a specific software with all its dependencies in order to run the application in a batch job on our [[Services/SLURM|Slurm cluster]]. Providing such dependencies on our managed Linux systems would be time consuming in preparation and maintenance. On the other hand such installations can be quickly done with elevated (sudo or root) privileges in a container.<<BR>> | A typical use case is to install a specific software with the latest tool versions and all its dependencies from third-party repositories in order to run the application in a batch job on our [[Services/SLURM|Slurm cluster]]. Providing such dependencies on our managed Linux systems would be time consuming in preparation and maintenance. On the other hand such installations can be quickly done with elevated (sudo or root) privileges in a container.<<BR>> |
| Line 12: | Line 13: |
| Docker is not secure as it provides a means to gain root access on the host it's running on, therefore we provide Singularity as a secure alternative. | Docker is not secure as it provides a means to gain root access on the host it's running on, therefore we provide Apptainer as a secure alternative. |
| Line 16: | Line 17: |
| [[https://www.sylabs.io/docs/|The official documentation]] for users is a great starting point to get up and running with Singularity. | [[https://apptainer.org/docs/user/latest/|The official documentation]] for users is a great starting point to get up and running with Apptainer.<<BR>> The analogue of a ''Docker file'', a so-called ''Apptainer recipe'' or ''Apptainer Definition File'' is documented in detail section [[https://apptainer.org/docs/user/latest/definition_files.html|Definition Files]]. |
| Line 21: | Line 23: |
| * '''Singularity Image File (SIF)''': An immutable container runtime image containing the container's directory structure * '''!SingularityBuilder''': A [[Services/SingularityBuilder|virtual machine to allow Singularity operations with elevated privileges]] |
* '''Apptainer (Singularity) Image File (SIF)''': An immutable container runtime image containing the container's directory structure * '''!SingularityBuilder''': A [[Services/SingularityBuilder|virtual machine to allow Apptainer operations with elevated privileges]] |
| Line 32: | Line 34: |
| The simplest use case is to convert an existing Docker image and run a command inside it. A well made Docker image can be converted without elevated privileges. | The simplest use case is to convert an existing Docker image, run commands inside it and customize it for personal requirements. A well made Docker image can be converted without elevated privileges. |
| Line 35: | Line 37: |
| Directly from Docker Hub: | |
| Line 36: | Line 39: |
| singularity build lolcow.sif docker://godlovedc/lolcow | apptainer build lolcow.sif docker://godlovedc/lolcow |
| Line 38: | Line 41: |
| Converting a Docker image stored locally as tar file: {{{ apptainer build docker-image.sif docker-archive:///path/to/docker-image.tar }}} |
|
| Line 41: | Line 49: |
| singularity exec lolcow.sif cowsay "Mooh!" | apptainer exec lolcow.sif cowsay "Mooh!" |
| Line 43: | Line 51: |
| If the command to be executed from the SIF is not found in the SIF's `PATH` variable, it's path needs to be specified explicitely: | If the command to be executed from the SIF is not found in the SIF's `PATH` variable, it's path needs to be specified explicitly: |
| Line 45: | Line 53: |
| singularity exec lolcow.sif /usr/games/cowsay "Mooh!" | apptainer exec lolcow.sif /usr/games/cowsay "Mooh!" |
| Line 49: | Line 57: |
| === Build your own container === A more complex use case is to install software on a specific Linux distribution with all its dependencies. This is easier with elevated privileges, as available step-by-step guides may be followed. To run Singularity operations as root we provide [[Services/SingularityBuilder|SingularityBuilder]] . |
=== Build your own SIF === A more complex use case is to install software on a specific Linux distribution with all its dependencies. This is easier with elevated privileges, as available step-by-step guides may be followed. To run Apptainer operations as user `root` we provide [[Services/SingularityBuilder|SingularityBuilder]] . |
| Line 52: | Line 60: |
==== Build a writeable sandbox to build your own SIF ==== From the Singularity Library: |
==== Create a writeable sandbox ==== From the Apptainer Library, where the example sandbox directory `ubuntu_16.04` is created in the current directory: |
| Line 56: | Line 63: |
| singularity build --sandbox ubuntu_16.04 library://library/default/ubuntu:16.04 | apptainer build --sandbox ubuntu_16.04 library://library/default/ubuntu:16.04 |
| Line 60: | Line 67: |
| singularity build --sandbox ubuntu docker://ubuntu:16.04 | apptainer build --sandbox ubuntu docker://ubuntu:16.04 |
| Line 62: | Line 69: |
| Sandbox directory paths may be relative or absolute paths. | |
| Line 64: | Line 72: |
| To install the software according to an installation guide or as you see fit, start a shell in your sandbox: | To install the software according to an installation guide or as you see fit, start a shell in your sandbox. The `--writable` option is necessary for file system modifications: |
| Line 66: | Line 74: |
| singularity shell ubuntu_16.04 | apptainer shell --writable ubuntu_16.04 |
| Line 72: | Line 80: |
| singularity build my_image.sif my_sandbox | apptainer build my_image.sif my_sandbox |
| Line 78: | Line 86: |
| singularity build --sandbox my_sandbox my_image.sif | apptainer build --sandbox my_sandbox my_image.sif |
| Line 82: | Line 90: |
| ==== Using a GPU ==== | ==== Using GPU resources ==== |
| Line 85: | Line 93: |
| singularity exec --nv lolcow.sif cowsay "Mooh!" | apptainer exec --nv lolcow.sif cowsay "Mooh!" |
| Line 89: | Line 97: |
| A user's home directory is available per default from a SIF. To be able to access additional storage paths they need to be defined by using the option `--bind`, either by using the same path as on the host: | A user's home directory is available per default from a SIF. To be able to access additional storage paths they need to be defined by using the option `--bind`, either by automatically creating the same path `/scratch/$USER` inside the SIF as it exists on the host: |
| Line 91: | Line 99: |
| singularity shell --bind /scratch/$USER lolcow.sif | apptainer shell --bind /scratch/$USER lolcow.sif |
| Line 93: | Line 101: |
| or specifying a different path in the SIF: | or mapping a specific path on the host `/scratch/$USER` to a different path `/mnt/$USER` inside the SIF: |
| Line 95: | Line 103: |
| singularity shell --bind /scratch/$USER:/mnt/$USER lolcow.sif | apptainer shell --bind /scratch/$USER:/mnt/$USER lolcow.sif |
| Line 97: | Line 105: |
== To be documented ? == * Singularity recipes * Convert local Docker image stored as tar file {{{ singularity build --sandbox my_sandbox docker-archive:///path/to/docker-image.tar }}} |
Apptainer
This article explains the purpose and use of Apptainer, a fork of Singularity. Compatibility is explained here in detail.
Description
Apptainer is a container system for isolating software and computing environments from the host system. It allows to package software in highly portable and reproducible environments encapsulating all dependencies, including the operating system.
It has similarities to Docker but it's security model is specifically engineered to be used in HPC environments and for easy integration in managed client environments.
A typical use case is to install a specific software with the latest tool versions and all its dependencies from third-party repositories in order to run the application in a batch job on our Slurm cluster. Providing such dependencies on our managed Linux systems would be time consuming in preparation and maintenance. On the other hand such installations can be quickly done with elevated (sudo or root) privileges in a container.
Docker is not secure as it provides a means to gain root access on the host it's running on, therefore we provide Apptainer as a secure alternative.
Official documentation
The official documentation for users is a great starting point to get up and running with Apptainer.
The analogue of a Docker file, a so-called Apptainer recipe or Apptainer Definition File is documented in detail section Definition Files.
Concepts
Sandbox: A writeable directory containing the container's directory structure
Apptainer (Singularity) Image File (SIF): An immutable container runtime image containing the container's directory structure
SingularityBuilder: A virtual machine to allow Apptainer operations with elevated privileges
Image sources
Pre-built SIFs and Docker images are available in the Sylabs Cloud Library or on the Docker hub.
Common use cases
Using a Docker image
The simplest use case is to convert an existing Docker image, run commands inside it and customize it for personal requirements. A well made Docker image can be converted without elevated privileges.
Convert a Docker image to a SIF
Directly from Docker Hub:
apptainer build lolcow.sif docker://godlovedc/lolcow
Converting a Docker image stored locally as tar file:
apptainer build docker-image.sif docker-archive:///path/to/docker-image.tar
Run a command in a SIF
apptainer exec lolcow.sif cowsay "Mooh!"
If the command to be executed from the SIF is not found in the SIF's PATH variable, it's path needs to be specified explicitly:
apptainer exec lolcow.sif /usr/games/cowsay "Mooh!"
Build your own SIF
A more complex use case is to install software on a specific Linux distribution with all its dependencies. This is easier with elevated privileges, as available step-by-step guides may be followed. To run Apptainer operations as user root we provide SingularityBuilder .
Create a writeable sandbox
From the Apptainer Library, where the example sandbox directory ubuntu_16.04 is created in the current directory:
apptainer build --sandbox ubuntu_16.04 library://library/default/ubuntu:16.04
Or from the Docker Hub:
apptainer build --sandbox ubuntu docker://ubuntu:16.04
Sandbox directory paths may be relative or absolute paths.
Start a shell in a sandbox
To install the software according to an installation guide or as you see fit, start a shell in your sandbox. The --writable option is necessary for file system modifications:
apptainer shell --writable ubuntu_16.04
Convert a sandbox to a SIF
After a sandbox is set up and tested it can be converted to a SIF:
apptainer build my_image.sif my_sandbox
Convert a SIF to a sandbox
It's also possible to convert a SIF back to a sandbox.:
apptainer build --sandbox my_sandbox my_image.sif
Running commands from SIFs
Using GPU resources
For software packaged into a SIF which requires access to the GPU resources of its host, the option --nv can be added:
apptainer exec --nv lolcow.sif cowsay "Mooh!"
Accessing storage
A user's home directory is available per default from a SIF. To be able to access additional storage paths they need to be defined by using the option --bind, either by automatically creating the same path /scratch/$USER inside the SIF as it exists on the host:
apptainer shell --bind /scratch/$USER lolcow.sif
or mapping a specific path on the host /scratch/$USER to a different path /mnt/$USER inside the SIF:
apptainer shell --bind /scratch/$USER:/mnt/$USER lolcow.sif