Gitpod: Vagrant support

[BSFishy]

I have been using Gitpod for a little bit now, because it lets me write code on any machine I use while I'm at school. This is great, because I always have an environment where I can write code without worrying about clutter, dependencies, or whatever else. While this is all fine and good, I also want my projects to be easily accessible to anyone, even those not using Gitpod. To help with this process, in terms of dependencies and whatnot, I came across Vagrant. It has, in fact, been exactly what I've been wanting for a while now, because I've wanted a Docker-esque system that is targeted for development. Vagrant, in theory, could eliminate the friction of getting the dependencies needed for my project, no matter the platform or setup. I am interested in integrating my projects with it, but am somewhat skeptical about how it would work using Gitpod.

I use Docker files in all of my projects to install dependencies, so I know how all of that works. My curiosity comes in with using a VM inside of Gitpod. Is that even supported and/or recommended? I don't exactly know the specs of the machines running the containers, so I don't know how running an entire VM could affect the performance/storage.

Furthermore, I would like to raise Vagrant as an alternative instead of Docker inside of Gitpod. I'll admit that I don't entirely know how the backend works, but I think it would be great to be able to just define a single Vagrant file and have independent users as well as Gitpod users use that file, rather than having Gitpod users using a Docker file and independent users trying to figure out dependencies on their own, or using a separate Vagrant file that is potentially out of sync with the Docker file.

[Kreyren]

Disclaimer: i am not a gitpod staff

Can you give me a summary of that? Maybe all you need is to add RUN apt install vagrant in dockerfile ?

[BSFishy]

Vagrant is a tool to easily manage virtual machines, aimed towards software development. It is a flexible system built to work using other software such as VMWare or VirtualBox. The idea is that you can clone a repository and run vagrant up to setup and start a virtual machine that is configured using a Vagrantfile in the repository's root. Then you can run vagrant ssh to use that virtual machine's terminal.

While I don't doubt the possibility of installing and using it in Gitpod, I am more concerned with using unnecessary resources associated with setting up and starting an entirely new virtual machine within a container. It would essentially be a completely new OS installation, running alongside the already existing OS used by Gitpod. That is where my concern stems from, and is why I think it could be useful to incorporate Vagrant as an alternative to Dockerfiles.

[Kreyren]

FWIW: this configuration https://github.com/Kreyren/gitpod-tests1/tree/9b8020e48b5ebdbe7ba4ef4be9368e9be76ce8cd results in

gitpod /workspace/gitpod-tests1 $ VBoxManage --version
WARNING: The character device /dev/vboxdrv does not exist.
         Please install the virtualbox-dkms package and the appropriate
         headers, most likely linux-headers-4.14.138+.

         You will not be able to start VMs until this problem is fixed.
6.0.6_Ubuntur129722

[Kreyren]

Apparently it is possible to pass --device argument in docker run that should give us the required access.

Checking weather it is sane to use

[JesterOrNot]

Hello @BSFishy and welcome to Gitpod! 🎉
I've heard of vagrant, however, I have never used it myself does it require KVM?

Also, it's up to the whole team but I would not like to see Vagrant replace Docker
But running VMs inside Gitpod would be awesome

[Kreyren]

I've heard of vagrant, however, I have never used it myself does it require KVM?

vagrant is just a command line tool, the providers it uses are where the work is actually done. essentially vagrant captures the config you define and calls out to tools like vboxmanage or docker to do any work - From #vagrant on freenode

Seems that the virtual box is an issue here where we need /dev/vboxdrv as mensioned in https://github.com/gitpod-io/gitpod/issues/1251#issuecomment-589754601 to fix this issue

EDIT: sanity check required to make sure that sharing /dev/vboxdrv is sane

[JesterOrNot]

cc @geropl @csweichel @meysholdt

[Kreyren]

FWIW i'm in #vbox waiting for response

[Kreyren]

may be relevant: https://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2008-3431

[Kreyren]

Relevant: https://www.scmagazine.com/home/security-news/oracles-virtualbox-vulnerability-leaked-by-disgruntled-researcher/

“The E1000 has a vulnerability allowing an attacker with root/administrator privileges in a guest to escape to a host ring3.” Zelenyuk wrote in a technical write-up posted to his GitHub account in technical write-up. ”Then the attacker can use existing techniques to escalate privileges to ring 0 via /dev/vboxdrv.

https://www.cybersecurity-help.cz/vdb/SB2018110701?affChecked=1

[BSFishy]

Also, it's up to the whole team but I would not like to see Vagrant replace Docker

I just wanted to clarify: I didn't mean to imply that Vagrant should outright replace Docker. What I was trying to say is that there could be an option in the config file to specify that the repository uses a Vagrantfile, which would be used to setup the container, rather than a Dockerfile.

[JesterOrNot]

I see you mean for users to have the option to write their config in a Vagrant or Dockerfile?

[JesterOrNot]

@BSFishy would running Vagrant in a workspace require root or Docker see #52

[Kreyren]

afaiu this expects the ability to run vagrant up which reads Vagrantfile to set up VMs for development, i.e testing the software on freebsd kernel.

[BSFishy]

BSFishy would running Vagrant in a workspace require root or Docker see #52

I wouldn't assume so. I haven't actually worked with Vagrant yet, as I haven't had access to a development environment outside of Gitpod, but I think it only manipulates the underlying providers, i.e. VMWare or VirtualBox. If the provider needs root access to create or interact with virtual machines, it probably would too.

[JesterOrNot]

I know but would the VMs that vagrant is spinning up require root

[BSFishy]

In order to install packages through something like apt, I think so. Although since it is in a separate virtual environment, I would expect it to be fine...

[JesterOrNot]

This depends on #39 being fixed as we don't allow root inside our workspaces for security reasons @csweichel or @geropl can explain it better than I can

[Kreyren]

Based on #vagrant it should work on rootless system, investigating atm

[BSFishy]

Just read through a tiny bit of VirtualBox documentation. From what I saw, actually creating and running virtual machines doesn't actually require root permissions. That would mean that I could technically install Vagrant and VirtualBox in a Gitpod container right now (ignoring the issue with installing VirtualBox). I could then create a Vagrantfile and spin up a virtual machine using Vagrant, where I would have root privilege.

Update:
Apparently, when you use VirtualBox, you can use it without root permission as long as you don't create the VM's as root.

https://forums.virtualbox.org/viewtopic.php?f=7&t=85541

[BSFishy]

Breaking: Vagrant actually has native support for using Docker a its provider. Potentially this could be used instead of using virtual machines? Although being able to make virtual machines in Gitpod would be very cool.

Additionally, Vagrant also allows you to make your own provider. Potentially that could be used to better integrate it into Gitpod if the team does decide to go that route, or even just to be able to use it in the workspaces.

[Kreyren]

Breaking: Vagrant actually has native support for using Docker a its provider. Potentially this could be used instead of using virtual machines?

Vagrant allows configuration of docker images this does not mean that it can run be used as invidual sandboxing method.

I'm currently having minor success with vagrant on Docker using KVM/QEMU which does not have the mensioned security issue.

[Kreyren]

cc @meysholdt @svenefftinge this configuration https://github.com/Kreyren/gitpod-tests1/tree/809456e4bf16825911b1e5f4e1695d9c210f8466 should be able to provide working vagrant using QEMU/KVM on gitpod assuming kvm module providing /dev/kvm for udev to create /var/run/libvirt/libvirt-sock.

Can you provide /dev/kvm for this purpose?

::: SECURITY ADVISORY :::

1. Provider for virtualbox should be disabled to avoid https://www.cybersecurity-help.cz/vdb/SB2018110701?affChecked=1
2. Currently it is allowed to assign end-user in libvirt group which with combination of allowing access to /dev/kvm could allow end-user to gain root access.

See https://gist.githubusercontent.com/Kreyren/374f12a1f380bd23900719f785702165/raw/057d4e6d1572f8049341298599e549780caeb7d6/gistfile1.txt which is content of /usr/share/doc/libvirt-daemon/README.Debian.gz

[JesterOrNot]

We cannot use KVM @Kreyren

[JesterOrNot]

I'll start some testing in https://github.com/JesterOrNot/Gitpod-Vagrant

[Kreyren]

We cannot use KVM @Kreyren

Elaborate

[JesterOrNot]

From our Slack channel

[Kreyren]

@JesterOrNot can you elaborate further? The hardware supports it based on available info

Forcing me to use slack

Grrr

EDIT: Gitpod does not mension any slack channel based on available info

[JesterOrNot]

You don't have to use slack that is an internal TypeFox chat the issue that has more info has become private

[Kreyren]

Make it public then so that we can evaluate the options :p

EDIT: Meaning public tracking for the issue

[JesterOrNot]

I can't I don't even have access to it anymore 😅

[Kreyren]

@JesterOrNot Get the access then? Allowing KVM shoudn't be an issue in terms of security and maintainance we have the resources to configure it.

[Kreyren]

btw. we also have the resources to run rootless docker which is based on available info a vulnerability waiting to hapend where i wasn't able to find any legitimate usecase for it to run as root.

[JesterOrNot]

https://blog.trendmicro.com/trendlabs-security-intelligence/why-running-a-privileged-container-in-docker-is-a-bad-idea/

[Kreyren]

@JesterOrNot From quickread i dont see anything that would conflict with proposed since i was told that running root on a rootless dockerd is sane assuming sane configuration.

[JesterOrNot]

[Kreyren]

@JesterOrNot

That's why i mensioned sane configuration (linux's permission system is very good compared to XNU if that's where the concern comes from), trying to fetch security checklist for you atm.

[Kreyren]

Referencing https://github.com/vagrant-libvirt/vagrant-libvirt/issues/1078#issuecomment-590098120 for more info on running vagrant on QEMU.

[Kreyren]

Referencing https://github.com/docker/compose/issues/7244#issuecomment-589941607 for rootless docker.

EDIT: note https://docs.docker.com/engine/security/rootless/#known-limitations for limitations (we can in theory workaround these if needed)

[BSFishy]

Afaik, this issue relies more on the support of virtual machines in general, rather than the support of Vagrant. Should another issue be created to document VM support in general?

Although it would be nice for native support of Vagrant like that of Docker.

[BSFishy]

I've been working quite a bit, trying to get this to work these past two days. As @Kreyren said, the kvm kernel module needs to be loaded. To my knowledge, I think that is the last thing that needs to happen to allow all of this to work (although I very much could be wrong).

I have brought my repository here to a point where it could be working, but I am getting an error:

Error while creating domain: Error saving the server: Call to virDomainDefineXML failed: invalid argument: could not find capabilities for domaintype=kvm

This indicates that the kernel module is not loaded. Again, I could be wrong that this is the last thing that needs to happen, but it would be great if we could have the option to enable this module.

[BSFishy]

@JesterOrNot So would it be possible to have the KVM kernel module enabled? Or is there some other outlying problem that would prevent that? I would like to be able to use Vagrant as soon as possible, so having the kernel module loaded would be the next step I believe.