Nomad: [feature] add Amazon's Firecracker as a task driver

[rushmorem]

https://firecracker-microvm.github.io:

Firecracker is an open source virtualization technology that is purpose-built for creating and managing secure, multi-tenant containers and functions-based services

I've just discovered this gem from Amazon. I think having first class support for it in Nomad would be awesome!

[yuriy-yarosh]

I'm working on it.

[dadgar]

@yuriy-yarosh Just to be clear, Nomad 0.9.0 is bring drivers as plugins, so you should target the new style of drivers if you are working on this.

Unfortunately we don't have documentation yet on writing third party drivers but these links should be useful:

• Definition of driver interface
• Ported drivers: These should be very useful references
• Example main for plugins

Essentially to build a custom driver, meet the driver interface and then create a main like the one I linked. You will then build that into a binary that you can place in a folder. Then run Nomad with the following flag: -plugin-dir=<path>. Nomad will then detect the plugin!

Hope this helps! Better docs will be coming as we get close to 0.9 releasing! Have fun!

[yuriy-yarosh]

Thanks, Alex, for being supportive.

I've dealt with hashicorp's stuff before, and as far as I know, I'll have to target stuff from the master branch right away. Surely, there are some undocumented things worth tinkering here and there, but overall, it's not that bad. I really doubt I'll caught myself in trouble with any existing hashicorp solutions, QA is simply good enough.

Hopefully, I'll be funded properly for this one...
Wanna port some Amazon's solutions like API Gateway, Lambda, StepFunctions, SQS, SES, SNS, SWF so it would be possible to deploy a serverless stack locally, using HashiCorp solutions, for the development purposes, and then move to AWS when needed. Probably, it would be some rust, but I'll have to finish this one first.

Bringing AWS to the customer and the developers doesn't work that well right now, from the Amazon's perspective. I hope I'll be able to fix that.

Anyway, I've been struggling with k8s stuff lately and it's proven to be quite unreliable, so Nomad is the only option for me right now. I'm focusing some well HW-tuned deployments with decent HW utilization, not the best one, for sure - DPDK/SPDK and all that jazz.

[yuriy-yarosh]

Finished firecracker client for golang, it took more time to get a linux box with KVM :dango: than actually implementing it.

@dadgar what do you think about adding direct containerd support for nomad ?
Docker driver by design became a bit obsolete...

[c4milo]

@yuriy-yarosh, that's a great idea. Many runtimes may be abstracted out by Containerd and Nomad's code and scope would simplify as a result. i.e: https://github.com/firecracker-microvm/firecracker-containerd

[bithavoc]

what's the official report for the driver @yuriy-yarosh, I found https://github.com/bitsnap/nomad-firecracker but no code.

[yuriy-yarosh]

@bithavoc I've finished it, but have been quite displeased with the firecracker design and some lack of features, so after a bit of tinkering I'm not considering firecracker a viable solution.
I had started my own firecracker-ish KVM runner, but have no free time and resources to finish it right now - priorities had shifted a bit.

Surely, I could publish this driver, but it's not something I could recommend people to use.

Firecracker should and will be __REFACTORED__ and partially __REWRITTEN__, and no one really knows when that day will come. So, there's just too much pointless hype going on around it.

[pires]

@yuri-yarosh can you please share what you came up with?

And maybe your thoughts on what parts of the design displease you, what features you're lacking, and so on. A _blog_ post would be sweet 🙏

Last but not least, where did you find that Firecracker will be redesigned and rewritten?

[yuriy-yarosh]

@yuri-yarosh can you please share what you came up with?

Will do, but not not right now.

A _blog_ post would be sweet 🙏

No one will pay me for that one, and I'm not a person who'll blindly contribute to a flawed product.
So, why bother ?

My current engagements are much more meaningful and at least can shake up the market a bit.

maybe your thoughts on what parts of the design displease you

• Absence of proper kernel build toolchain
• Inconsistent API's
• Security issues
• Overall Immaturity which contradicts with existing PR material

... worth hating just for the amount of hypocrisy and ignorance involved in that one

where did you find that Firecracker will be redesigned and rewritten?

Well, it's a no brainer if you'll search through and review the existing codebase.

I'd say that it's worth waiting few months 'till things settle down a bit, because right now Firecracker is too WIP.

[yuriy-yarosh]

Yup, it looks like Firecracker became something decent, at last.
So, it's time to release the driver...

[cneira]

I just created a task driver for firecracker firecracker-task-driver, I'll submit it to the community drivers next week. Let me know if more features are needed, my use case is pretty simple at this point.

[tgross]

We've got that driver in our community driver docs. Going to close out this issue. Thanks!