Kitematic: Add integration for one or many external Docker hosts (different docker-machine providers)

[develar]

I cannot use VirtualBox because I don't have enough memory for it — Parallels Desktop is much better and I have configured vagrant box. https://github.com/Parallels/vagrant-parallels/issues/115

So — I just want to open Kinematic and Kinematic should not install VirtualBox/boot2docker — use existing installation (DOCKER_HOST env).

[ryansch]

I'd be interested in this to enable our use of vmware fusion.

[proudlygeek]

:+1: I really dig this one!

[develar]

@ryansch Exactly. Regardless of performance (Vmware is closer, http://www.tekrevue.com/parallels-10-fusion-7-virtualbox-benchmark/), vagrant allows to customize VM more precisely (for example, by default, VirtualBox consumes more than 1GB RAM — vagrant parallels box just restricted to 512 MB and it is enough).

[hadronzoo]

:+1:

[caseywebdev]

:+1:

[tgoeke]

I would vote for supporting VMware Fusion with docker machine.

[mainiak]

:+1:

[dougborg]

+1 for the ability to specify a driver for docker-machine!

[dougborg]

Looks like you can work around this for now by using docker-machine to create and configure a box with whatever driver you like. Just call it dev and it looks like Kitematic will just use that one.

After installing kitematic and starting it up for the first time:

# Kitematic creates a virtualbox vm via docker-machine - this is hard-coded for now.
$ docker-machine ls
NAME           ACTIVE   DRIVER         STATE     URL                          SWARM
dev            *        virtualbox     Running   tcp://192.168.99.100:2376

# close kitematic

# remove the default vm
$ docker-machine rm dev
$ docker-machine ls
NAME           ACTIVE   DRIVER         STATE     URL                          SWARM

# create a new `dev` machine with whatever driver you like:
$ docker-machine create dev -d vmwarefusion
INFO[0000] Creating SSH key...
INFO[0000] Creating VM...
INFO[0000] Starting dev...
INFO[0000] Waiting for VM to come online...
INFO[0042] "dev" has been created and is now the active machine.
INFO[0042] To point your Docker client at it, run this in your shell: $(docker-machine env dev)
$ docker-machine ls
NAME   ACTIVE   DRIVER         STATE     URL                          SWARM
dev    *        vmwarefusion   Running   tcp://192.168.133.132:2376

# start Kitematic - it will say it is "Starting Docker VM",
# but it is really just using the one you created:

$ docker-machine ls
NAME   ACTIVE   DRIVER         STATE     URL                          SWARM
dev    *        vmwarefusion   Running   tcp://192.168.133.132:2376 

You can now start containers, etc using the Kitematic UI and they will use the dev vm you created manually.

So far the only thing I have found that doesn't work is using the Kismatic UI to browse/edit files you have mounted as volumes in the container. They must be setting that up using a mount from the virtualbox vm created with vbox commands, or something along those lines. That is one limitation for the work-around and something else that will have to be implemented for each driver or in a more generic way.

[tgoeke]

Using the manual method is fine - except I don't want to install VirtualBox at all, it messes with networks etc, so it would be nice to be able to skip the VB install.

[dougborg]

Yep. Good point. I tried uninstalling VirtualBox, but Kitematic installed it again the next time it started up.

[sammcj]

Just went to try it and saw it started downloading Virtualbox without even prompting - had to quickly kill it off.

I already have boot2docker and docker machine setup with VMware Fusion which is a _lot_ faster and more reliable than Virtualbox - it would be nice if Kitematic recognised this.

[jmorganca]

Hi all. Thanks for creating this issue!

The good news is that we use docker-machine under the hood, which is quickly adding support for different providers such as VMware fusion etc.

I'll mark this as a feature for adding different providers, but also understanding that it's inconvenient to automatically install VirtualBox if that's not what's expected.

[dougborg]

Just to follow up on my earlier comment, it looks like the host volume support is provided by docker machine host drivers and host volume mounting is not yet supported by the vmwarefusion driver: https://github.com/docker/machine/issues/641.

[rickard-von-essen]

@dougborg Is host volume mounting required to get kitematic to work? I wrote the Parallels Desktop driver and is eager to add support for it in kitematic but it currently lacks host volume mounting because of legal issues with the drivers.

Is there any work done on abstracting out the "driver" layer?

[develar]

@rickard-von-essen I don't get your note about parallels. Host volumes works perfectly – specified in my vagrant file (config.vm.synced_folder "/Users/develar", "/Users/develar").

[rickard-von-essen]

@develar But not in docker-machine + boot2docker.iso since it requires that boot2docker.iso to include Parallels Tools which is currently not supported by the EULA see _1.7. Parallels Tool_ on http://www.parallels.com/about/legal/eula/ .

[dougborg]

@rickard-von-essen: I think host volume support is important in the context of this user story / feature: adding support for other docker-machine providers. While I am sure there are some situations where mounting volumes from the OSX host is not needed, I find it is necessary in most of the work I do with Docker. Kitematic already has a nice interface built up around mounting and viewing contents of host volumes and I think it would be a shame for those features to be unavailable if you use another provider. If you provide support for other providers without the host volume mounting feature, I suspect it would confuse users.

[jamroks]

   + 1 on this one !!!  On Mac Os x nothing compare to vmware Fusion when using virtualization : 

• Speed
• Relability
• Rock Solid Feature
• etc

It used to be only Virtual Box around me, But nowadays i see a lot of people now swithing to Fusion for all the reason mention above . It's a growing user base of Fusion on Mac OS happening . And Yes please Host Volume support is a must for any productive workflow between Host-VM-Docker

[nuarch]

Please add this!

[jchannon]

I just came across KiteMatic and thought this looks cool!

But then I saw it used VirtualBox

Please add the ability to use VMFusion!

Thank you

[jmorganca]

@jchannon Hah! Loved this comment. Thanks for the feedback :smiley:

[philjones88]

:+1: Fusion user here

[develar]

See details here — https://github.com/docker/machine/pull/939#issuecomment-94168085 "Parallels are working on Parallel Tools for boot2docker."

[jchannon]

@jeffdm how goes the progress for this feature?

[jmorganca]

@jchannon Right now we're working to nail the (default) virtualbox experience but this is strongly in demand.

That said, this is strongly in demand. Both so KM users can deploy an app to a cloud server or use alternative Desktop virtualization system.

[MattRogish]

:+1: Also want VMWare Fusion support. :D Thanks!!

[andrerom]

Fusion supported wanted :+1: _(will try the manual way in between, but I also have limited space on my macbook, so this would be great to be able to uninstall virtualbox)_

[taiidani]

Definitely a :+1: here too. I support the developer machines at my workplace and we use a mix of Vagrant+VMWare and Vagrant+Virtualbox. It would be great if Virtualbox wasn't a requirement.

[stunthamster]

Another thumbs up from me on this feature - would be awesome to be able to use VMware Fusion on my Macbook.

[iby]

:+1: for Parallels or anything else but VirtualBox!

[ilboud]

[nnam]

[justinmayer]

Yet another bloke looking forward to VMware Fusion support. (^_^)

[danielmt]

VMWare Fusion support, please. Think of the children.

[fgimian]

+1 for me too, I'm all about VMware Fusion :smile: Perhaps at a minimum, please don't install Virtualbox automatically and provide some instructions on creating the required VM in the terminal.

Should go something like this:

# Download and install docker-machine
curl -L -o /usr/local/bin/docker-machine https://github.com/docker/machine/releases/download/v0.3.0/docker-machine_darwin-amd64

# Make docker-machine executable
chmod +x /usr/local/bin/docker-machine

# Create the dev docker machine for use with Kitematic
docker-machine create --driver=vmwarefusion dev

# Add docker-machine initialisation to bash profile
echo 'eval "$(docker-machine env dev)"' >> ~/.bash_profile 

I can confirm this procedure works as expected. Right now the biggest issue is that Kitematic forces us to install Virtualbox (ewwwww ....).

Cheers
Fotis

P.S.: Kitematic is awesome, thank you so much for developing and also making it open source!!

[HardieBoeve]

+1 for not installing Virtualbox automatically, but provide an option to select the desired vm provider (vmware, parallels, etc).

[dekobon]

+1 for not installing Virtualbox automatically from me as well.

Supporting the settings:

DOCKER_CERT_PATH
DOCKER_CLIENT_TIMEOUT
DOCKER_HOST
DOCKER_TLS_VERIFY

Will let Kitematic be more than a toy. If these are supported, you could deploy to production with it on a platform like Joyent's Triton.

[timfallmk]

Working on this with upcoming PR. https://github.com/timfallmk/kitematic. Will add support for all docker-machine supported backends eventually. This is the continuation of a project from DockerCon Hackathon 2015.

[fgimian]

Thank you so much, that would be awesome!! :smile:

[DieterReuter]

@timfallmk just checked your readme and the small screencast. This looks really awesome, would love to see this merged soon. What about to use docker-machine ls to discover all locally defined Docker engines and displays this list to the user for a preselection?
Ping @jeffdm.

[tonypiazza]

+1 for supporting alternate providers such as VMware Fusion (my first choice)
+1 for not requiring installation of VirtualBox

[rendhalver]

I also prefer VMware Fusion over Virtualbox.
I would also be happy to he test code to get this working.

[timfallmk]

Just a status update to this issue. Progress is being tracked over here and we're moving along quite well!

[legal90]

@timfallmk Thanks! Just for your information: Parallels driver for Docker Machine is on the way: https://github.com/docker/machine/pull/939
It would be great to get supported in Kitematic as a yet another local provider.

[timfallmk]

@legal90 That's awesome! As we discussed for the project (and you can see in the timeline), we're probably going to stick to a limited set of drivers for the first "MVP" release in a few months, but the idea is to add support for everything machine supports eventually.

[dekobon]

@timfallmk What about providers that don't need any special settings? For example a remote Docker server setup with the standard docker environment variables like:

DOCKER_CERT_PATH
DOCKER_HOST
DOCKER_TLS_VERIFY

[timfallmk]

@dekobon Are you suggesting using the machine "generic" driver for that kind of input? If so, how would you like to see the UI flow for inputting that kind of endpoint?

[dekobon]

@timfallmk I envision a generic driver that can be selected where you can
enter in values for all of the docker variables or optionally load them
from a config file. If this was possible it would be easy to test kitematic
against many platforms. In fact, this seems like it should be on the top of
the list for support because it is vendor agnostic and in fact supporting
the docker way of doing things.
On Jul 29, 2015 2:50 PM, "Tim" [email protected] wrote:

@dekobon https://github.com/dekobon Are you suggesting using the driver
"generic" driver for that kind of input? If so, how would you like to see
the UI flow for inputting that kind of endpoint?

—
Reply to this email directly or view it on GitHub
https://github.com/kitematic/kitematic/issues/231#issuecomment-126107101
.

[timfallmk]

@dekobon Actually the "generic" driver is just meant to cover platforms that machine doesn't _specifically_ support. Since we want the process to be as easy as possible, we wouldn't necessarily encourage that right away. However, seeing that it would be relatively trivial to add a settings page for manual "generic" provider input, this could possibly be added to the list.

First and foremost we'd like to get the specific drivers working so people can "one click" use them.

[dekobon]

@timfallmk If you had a configuration file with a registered extension, that could act as a "one-click" mechanism for supporting a given provider. Just a thought.

[mohamnag]

+1 here. I would not even restrict the idea to virtual machines, it could be docker on a EC2 host machine.

[timfallmk]

Thanks for the suggestions, I think the ideas are worth considering. Cloud providers will definitely be an option (Digital Ocean will probably be the only one supported in the first release).

[robneu]

:+1: on this. I'm really intrigued by this project but VirtualBox makes it kind of a non-starter for me. 😕

[nfarrar]

Any status on adding support for VMWare Fusion? Kitematic is great and I'd love to start using it full time, but have a hard requirement for Fusion.

[ap0phi5]

VMWare Fusion here too please!

[jascodes]

+1

[cajurabi]

+1 VMWare Fusion

[felixphew]

Possibly, you could make Kitematic check if there is already a docker machine called dev, and if so then don't install and set up Virtualbox? It's a bit of a workaround but I think many people would take advantage of it, and it means no changes to the "default" user experience.

Obviously "full" choice of what docker-machine backend to use (remote servers as well as different hypervisors) would be good in the future though.

[jdickey]

Is there anyone from the dev team working on this? Or has The Word come down that it's not glitzy enough?

[felixphew]

@jdickey I think it's in the works, but I wouldn't hold my breath.

[timfallmk]

@jdickey @felixphew . I know it's annoying waiting, but it _is_ actually happening. This project is heavily dependent on https://github.com/docker/machine. Currently the timeline was set back significantly, as the maintainers of that project have chosen to move all extra drivers back to the proposal stage (https://github.com/docker/machine/issues/1626). We're continuing to work on the Kitematic changes that don't require these changes to come through, just at a somewhat diminished pace.

[frandevel]

Yet another VMWare Fusion user here. Would be awesome being able to use your preferred virtualization engine

[inanimatt]

:+1: Another plea for a virtualbox-free Kitematic experience :)

[mikeroySoft]

I'm the product owner of VMware Fusion and Workstation, and if there's anything I can do to help please let me know =)
mroy at vmware

[FrenchBen]

thanks @mikeroySoft for reaching out - I think a lot of it has to do with getting machine selection integrated into kitematic- There's a wiki-page with some guidelines: https://github.com/kitematic/kitematic/wiki/Machine-Management

I created a small PR for a kitematic cfg/rc - which perhaps maybe an easier route to get vmware fusion up and going? https://github.com/kitematic/kitematic/pull/992

[diogoviannaaraujo]

+1 for vmware integration!

[FrenchBen]

A big thank you to @mikeroySoft for allowing me to work on this fully.

config file setup should be something like .kitematicfg in your user's home directory:

{
    "driver": "vmwarefusion",
    "name": "dockerfuse",
    "memory": "1024",
    "vbinstall": false
}

Description:

• driver: the docker-machine compatible driver (should work with all driver supported by docker-machine)
• name: name of the vm to use
• memory: the ram that can be allocated to the vm
• vbinstall: true/false on installing VirtualBox. Not having VirtualBox installed, setting this option to false and using virtualbox as the VM driver is obviously going to create some issues.

If some of you feel like doing a bit of QA on confirming that their vmware fusion VM works with this Kitematic patch, I'd appreciate it:
https://github.com/FrenchBen/kitematic/tree/cfg-vm

One option which has been discussed is adding support for the 'none' driver option in docker-machine, allowing the user to specify a Docker-friendly url. Thoughts?

DISCLAIMER: the above github code is provided "as-is" and considered to be for "advanced" users. I'm not responsible for any loss of data, destruction of your VM/laptop/house/relationship/pet/etc.; do praise me however if it worked as expected.

[felixphew]

@FrenchBen This seems to be working OK for me so far - thanks! As for the idea about the "none" driver - good idea, but might it be easier to _always_ specify it as a docker URL (more consistent)?

[dekobon]

@felixphew This is what I've been advocating for a while. If I want to give an experimental or somehow different setup to a newbie to Docker, the easiest path would be to allow them to enter in the DOCKER_HOST and other DOCKER_* variables.

[FrenchBen]

@felixphew @dekobon Always providing the docker URL creates the assumption that the VM has already been created with proper settings. The docker URL is unknown until the VM is created, which I thought was the original request (use 'xxxx' driver instead of virtualbox)

Furthermore, if just a URL is provided you lose a lot of the nice docker-machine features, such as the ability to ssh in or have it regen the certs, since the docker-machine keys probably weren't added? (not 100% on this as I don't have machines with just URL setup)

[dekobon]

@FrenchBen Making the assumption that the Docker URL was properly created seems very sensible if you are using an abstraction like Joyent's Triton. Essentially they abstract an entire data center as a single docker host thereby allowing you to connect to the Docker URL. Project Bonneville from VMWare will in all likelihood be also taking a similar approach. I've also heard from the Microsoft Docker folks that they plan on using the single host abstraction as well. Also, you can imagine a scenario where in an organization someone has provisioned a very beefy box and they give access to users to provision against it using kitematic.

As for Docker Machine, we've been living in the reality for the last couple of months where all of the pull requests were rejected and that team is in the middle of refactoring to support a driver based model. However, we are still waiting to see what that model will look like. While this all gets sorted out with Docker Machine, it would be nice to have user friendly tools like Kitematic be able to support an "advanced option".

Regarding the ability to ssh in or regenerate the certs - platforms like Triton handle that for you or those patterns just don't apply. For example, you don't need to ssh in because there is no VM host running.

[FrenchBen]

@dekobon Thanks for the feedback- I'll add the docker URL support via the 'none' driver, and 'url' entry in the above JSON, to add the entry to docker-machine and get Kitematic working with it, unless someone here is against this?

[Silvanoc]

+1 for virtualbox-free and remote host support

[astewart-twist]

+1

[iby]

@FrenchBen just have successfully got rid of virtual box and moved to parallels with the latest docker-machine release, but found out that kitematic doesn't play along nicely. Looks like you are the one who's working on this? When do you think kitematic will support all that?

[FrenchBen]

@ianbytchek If you're familiar with compiling Kitematic from source, you can check out the latest PR: https://github.com/kitematic/kitematic/pull/1161 which allows machine selection

[iby]

@FrenchBen npm fails to install contextify, seems there's unresolved issue with that. Judging by the way this is going this soon will be released anyway, will wait.

[xyc]

+1

[FrenchBen]

@ianbytchek What version of Node/NPM are you using? v4.2.1 is what I'm using

[iby]

@FrenchBen 5.0.0 & 3.3.10. This must be related to brianmcd/contextify/issues/188 or one of similar issues.

[FrenchBen]

@ianbytchek very possible. May have to wait for this to be merged.

[tiborvass]

Ping @FrenhBen a LOT of people are asking for this feature at dockercon eu ! I think it shouldn't be too difficult to integrate it with machine. :+1:

[FrenchBen]

@tiborvass Needs a little extra QA and we'll be ready for it :dancer:

[badlydrawnrob]

:+1: For an alternative vmware fusion integration. Currently using Vagrant with Deploybot integration ... so if I can figure out their Atomic deployment with Docker integration, that might lead to a hassle-free process. Anything to save me manually setting up servers!

[felixphew]

@FrenchBen sorry to keep hassling you, but when is this patch likely to make it into a release? If you need any extra testing done, I'm happy to help out.

[FrenchBen]

@felixphew no problem - Monitoring it myself. I'll update this thread when it does!

[felixphew]

Just thought I'd leave this here: https://github.com/docker/kitematic/wiki/Machine-Management

[luckydonald]

I can't use my created machine. Also, the delete vm button does nothing.

/Applications/Docker/Kitematic (Beta).app/Contents/Resources/resources/docker-machine -D create -d virtualbox --virtualbox-memory 2048 default returned non zero exit code. Stderr: Error creating machine: Machine default already exists You will want to check the provider to make sure the machine and associated resources were properly removed

$ docker-machine ls
NAME      ACTIVE   DRIVER      STATE     URL                      SWARM   DOCKER   ERRORS
default   *        parallels   Running   tcp://10.211.55.8:2376           v1.8.3

Mac OS 10.9.5, Parallels 10.3.0 (29227)

[iby]

Yep, same thing with Parallels. Thought that was supposed to be working in the latest release?

[FrenchBen]

Unfortunately it isn't - There was some early work done to support those, but the 'default' VM still is expected to be a Virtualbox driver and why you're seeing the error.

[iby]

@FrenchBen:

[jdickey]

Agreed with @ianbytchek. My boss hears me raving about Docker, but absent Kitematic working with our existing VMware installation (that we're otherwise committed to), there are still too many shiny knobs and sharp edges for me to be comfortable having him set anything up by himself. Ditto for any future new hires. :disappointed:

[luckydonald]

Is there a particular reason for that check of the driver?

[FrenchBen]

@jdickey Sorry to hear this being a pain point. It's not easy to please all 'setup', but feel free to comment in the wiki on the machine work.
@luckydonald We rely on vbox for a proper VM setup and have it integrated for retries, reset, etc. Allowing more drivers becomes a pretty big task during the initial setup.

[luckydonald]

Heh. Even VMware doesn't really works, I got problems on my windows machine all the time, something like 'could not get ip'. But that probably is another issue.

[jameswilson]

Also interested in this. I tried installing Toolkit, run Kitematic for first time, it creates the VirtualBox default VM. Then tried removing the default with docker-machine rm default, and replacing it with one using the Parallels driver with docker-machine create --driver=parallels default. When I go to restart Kitematic, it completely replaces my Parallels VM with a new VirtualBox one!

[unlucio]

:+1:
I use as well simply my general linux vm on vmware (so no vbox nor docker-machine) and I'd love to be able to have kitematic attching to that :)

[joshuajeeson]

+1 to be able to remotely control a docker engine installed on VM's, with Kinematic installed locally on my desktop machine.

[bt]

Is there any update or roadmap for this issue?

Maybe some of us can contribute to the efforts as well?

[FrenchBen]

@bt there's a bit of info in the wiki under machine management. A lot of the code needs to be updated, but it'll give you an idea of general direction

[gsingla294]

I don't want to use Virtual box. Instead, I have my docker daemon installed on one of the VMs in Data center.
can I connect that daemon using Kitematic using IP of the VM? so that I can manage my docker deamon using Kitematic UI.

[mrkeuz]

:+1:

[kastork]

+1

For me it would be enough to simply have the ability to tell Kitematic which docker-machine of mine to use. The local stuff on a VM is nice, but I use docker-machine to create multiple environments on multiple clouds as well as locally. I don't need or even, necessarily, want Kitematic to manage the construction of machines locally or remotely (docker-machine command line is perfectly adequate for that), I just want to use it to manage containers in whatever machine I need to work with since it has a nice interface for environments, ports and so forth.

[kane-c]

I figured out a basic workaround to use this without VirtualBox if anyone is interested.
https://github.com/kane-c/kitemagic

I'm using it now with Parallels and Dinghy.

[FrenchBen]

@kane-c I appreciate the quick hack, but it would be much nicer to have a proper solution implemented. Do you think you could create a PR that does something similar?

[luckydonald]

For reverence, @kane-c's script is currently 59eae3

[yuba]

+1

[chrisandchris]

+1 for adding other machine vm providers

[stormbeta]

This makes it pretty difficult to recommend Kitematic right now. Putting support for creating machines with other drivers on the back-burner makes sense, but the inability to even select an pre-existing machine config, especially when Kitematic already uses docker-machine, is frustrating.

Almost every other tool related to docker I've seen allows you to define the docker config yourself, or pick from a list of visible docker-machines.

[jdickey]

@stormbeta The only logical conclusion seems to be that Kitematic is being deprecated with plausible deniability; after all, it still works if you saddle yourself with VirtualBox, but it won't work even with Docker's bundled hypervisor (xhyve). As you note, it's now simply unethical to recommend anyone even try Kitematic, and that stance is likely to spread to Docker tools as a whole without expedited vendor action.

Translation: Docker, if you're trying to encourage people to sign up for paid support/etc contracts, you're proceeding along a highly implausible path.

[hellonearthis]

I would like to see and will try and see if the config and other settings can be exposed in Kitematic as it is a very useful feature. It is open source so you can do it yourself and if you don't like kitematic then just use the Docker.exe

[luckydonald]

As the problem is not that it doesn't work with other machines.

Like @kane-c's comment shows, it is possible with non virtual box VM's.

[ssbarnea]

Does anyone knows another client for macos that can be used to manage multime docker hosts?

[luckydonald]

Portainer might be suited.
I also heard about the rancher frontend being able to connect to multiple machines