Singularity: What's the point of singularity?

[stefanoborini]

Sorry but it's not clear from a 30 seconds overview of the documentation or the website.
What's the point of singularity compared to using docker? What problem does it solve? Why do I need it?

Thanks

[sk8forether]

Singularity originally started at the Lawrence Berkeley National Laboratory as a containerization solution with different requirements and goals than Docker. It has been primarily used for scientific computing and HPC, and solves problems such as security, running as a user instead of root (since cluster admins typically do not want to allow root access), etc. A company called Sylabs has since been started and continues to maintain/update Singularity, but I believe the focus of the tool has not changed. If you are not doing anything related to scientific computing or HPC, or don't have similar security concerns, you may not be very interested in Singularity. If you are interested and have been using Docker, check this out.

Anyone else feel free to add, that's just my 2 cents. It would be nice if there were a brief explanation of this in the more recent docs.

[sk8forether]

I just found _this_, which is linked from the main page of this repo. It seems they are also interested in supporting Enterprise Performance Computing.

[GodloveD]

Great question @stefanoborini ...

A brief history of Singularity

Singularity was born out of a need for simple, secure containers in High Performance Computing (HPC). Since then it has become the container platform of choice for HPC users, security-minded admins, and AI geeks.

The problem

Around 2015 many HPC admins were getting the same request from users:

_Dear Admin,_

_Please install Docker because I need it to run xyz and/or it is super cool._

_-sincerely_
_Every User_

This prompted us to research Docker. Most of us quickly realized that installing Docker and adding users to the appropriate group was the same as giving them root access. HPC environments tend to be massively shared resources. A single system may have thousands of users. Critically, these users do not have root and they are not getting it any time soon.

The solution

@gmkurtzer at Lawrence Berkley National Lab was also getting the same requests from users for Docker, but he stopped and asked users what they really needed. Turned out they needed massive portability, reproducibility, and the ability to easily leverage specialized HPC resources (like GPUs, Infiniband, batch schedulers, and MPI) with containers. Meanwhile admins needed a solution that was architected with security in mind.

So Greg went to work. He created Singularity v1.0 and polled the audience. Users thought it was pretty cruddy but it had potential. Some of them got involved in the development, and 2.0 was born along with the Singularity community.

Singularity is a different than other solutions

Unlike most container solutions, Singularity was developed largely by people who were not heavy Docker users. So it's a very different take on containers. In particular, the security context is totally different. Containers are run without root privs by default.

The container file format is also different from most other platforms. Singularity containers are packaged in the Singularity Image File (SIF) format. This enables things like cryptographically signing and verifying your containers. And most recently, building and running encrypted containers without any intermediate decrypting step.

This is just the tip of the iceberg. If you really want to see what is the point of Singularity, you should play with it. The most recent docs are here, and you can find some convenient vagrant boxes for experimenting here.

Dancing cats

One final differentiation to note is the Singularity inventor's/founders's profound joy and love of dancing cat gifs. I believe this is @gmkurtzer's very favorite. (It's art.) Enjoy.