Nomad: [driver plugin] OCI / containerd support

[unixfox]

https://containerd.io

An industry-standard container runtime with an emphasis on simplicity, robustness and portability

Currently it's possible to interact with containerd on nomad only trough the Docker daemon.
It would be great to add the support for interacting directly with containerd without the Docker daemon just like Kubernetes is already doing: https://kubernetes.io/blog/2018/05/24/kubernetes-containerd-integration-goes-ga/
This would mean less memory usage because we wouldn't have to run the Docker daemon to launch the containers just like it's already possible with rkt.

[endocrimes]

Hey @unixfox,

We've definitely chatted about the possibility of an OCI driver for Nomad, because it would open up support for a wide range of container runtimes in one go.

However, this is something that's likely to be best served by an external driver plugin from 0.9 onwards, rather than as an internal driver.

I'm going to leave this open, tagged as a plugin idea for discoverability, because I'd definitely like to see this happen.

[stale[bot]]

Hey there

Since this issue hasn't had any activity in a while - we're going to automatically close it in 30 days. If you're still seeing this issue with the latest version of Nomad, please respond here and we'll keep this open and take another look at this.

Thanks!

[unixfox]

is it possible to remove the stale bot on this feature request/issue?

[shishir-a412ed]

I have a working version of this driver nomad-driver-containerd under https://github.com/Roblox.
Right now it's private. I plan to open source it within 3 weeks. We (Roblox) are also trying to move in a similar direction, where we want to integrate directly with containerd (like kubelet).

Docker is a monolithic service with everything from containers, images, networking, volume management, secrets packed into one. Containerd was launched as a solution to this, to be able to handle any container-related management using this lightweight daemon (gRPC server).

This driver will allow us to directly integrate and launch containers using containerd (bypassing docker), thereby reducing the CPU and memory footprint.

[bithavoc]

@shishir-a412ed awesome, then maybe there's a chance https://github.com/google/gvisor-containerd-shim can be used in Nomad too.

[shishir-a412ed]

@bithavoc Yes I think that would be possible, I am exposing containerd_runtime as a configurable parameter to nomad driver configuration agent.hcl

Right now, it's using runc as a container runtime, but it could be configured to use runsc

[bithavoc]

wow @shishir-a412ed, freaking awesome, can't wait 🚀

[bithavoc]

@shishir-a412ed let me know if you need testers for the driver.

[shishir-a412ed]

@bithavoc The driver is ready, and I am just going through the process to open source it. The codebase needs to be scanned by our security teams, to make sure we are not releasing any Roblox related code or IP + a few approvals before the project can be open-sourced.

I am hoping it should be released within the next 2 weeks.
Once it's open I would love for you (or anyone else) to try it out!

[shishir-a412ed]

@bithavoc https://github.com/Roblox/nomad-driver-containerd

[bithavoc]

Awesome @shishir-a412ed nicely done!