Machine: managing different versions of docker difficult with docker-machine

[nnordrum]

For presumably relatively obvious reasons, I may not have all my versions of docker running on the same version. This causes issues in that the docker client isn't smart enough to fall back on the api version if it's newer than the daemon it's trying to connect to.

It would be nice if docker-machine env could provide a hint that I could use to adjust my path.

e.g.

export DOCKER_TLS_VERIFY="1"
export DOCKER_HOST="tcp://192.168.99.100:2376"
export DOCKER_CERT_PATH="/Users/xxx/.docker/machine/machines/dev"
export DOCKER_MACHINE_NAME="dev"
export DOCKER_VERSION="1.8.3"
export DOCKER_API_VERSION="v1.20"
# Run this command to configure your shell: 
# eval "$(docker-machine env dev)"

Obviously the best would be if the client was smarter and could fall back, or specify an api version, but I feel like this has a better chance of making it in "soon" than that would.

[nathanleclaire]

For presumably relatively obvious reasons, I may not have all my versions of docker running on the same version

It's not really obvious to me as many users just use the latest -- can you elaborate?

At any rate, I'm really hesitant to add this feature to env, but maybe we can figure out another solution.

[nnordrum]

yes, _many_ users just use latest, but I can't necessarily update everything immediately. I also may not have the ability to upgrade to latest on say a shared build server, but I'm running latest locally.

I can relatively easily roll my own solution, but I figured others would have the issue as well.

[nathanleclaire]

Yeah, I agree it can be pretty annoying to manage multiple different daemon versions. In general my approach to solving this issue is one of:

1. Use an older client for all work (so don't upgrade Docker client to greater version than your oldest server)
2. Rename and use older clients like so: docker-1.8.0 ps. Yeah, I know it's extra hassle, but it works.
3. docker-machine ssh into the machine where you want to work and the docker there should always match the daemon version there.

[outcoldman]

@nathanleclaire do you think that it can be useful if docker-machine will do all of these steps automatically? Download the latest docker and make it work in the same way as virtualenv works for python?

[nathanleclaire]

@outcoldman If you think about it Docker Machine already kind of does this. You could pretty easily create a few different VMs with different versions and ssh into them to use different versions of Docker. However, the latest version of Docker is almost ubiquitously the best, so I'm not really sure why you'd feel the need to do so.

[outcoldman]

@nathanleclaire simple example - you have a dev, staging and production. You don't want to restart/reboot production on every docker upgrade, just when it is only necessary, because of security issues or maybe because you need new docker features. But at the same time for the staging and production you are always using the latest docker client/daemon.

[nathanleclaire]

@outcoldman I don't understand. Either you stick to an older client binary (since older clients will talk to newer daemons just fine), or just ssh in to run docker commands on the respective servers (therefore you will always have the correct client version). Or, you can just have client binaries locally named docker1.9, docker1.8, etc. Either way, using docker-machine to manage Docker client versions is not something we want to support.

[outcoldman]

Either way, using docker-machine to manage Docker client versions is not something we want to support.

this is a fair argument, my question was basically if you think that it can be possible that docker-machine could save right docker client into ~/.docker/machine/machinename/bin and kind of manage it in the same way as virtualenv

[nathanleclaire]

Ah yeah, that's just not really in the scope of the project, sorry.

[darkermatter]

@outcoldman I bashed up against this the other day so I knocked together some scripts to ease the use of multiple versions. The gist is here https://gist.github.com/darkermatter/c57c112c0b24197ea99d

I know it's been a few months but I hope it helps somebody.

BTW - the scripts are for ZSH and will need tweaking for Bash, etc.

[outcoldman]

@darkermatter :+1: thanks! I still believe it should be managed by docker-machine, if they want it to make production ready.

At current point I see that docker-machine team (including @nathanleclaire ) targeting their project only for developers. Meaning that I do not see how this product can be production ready as with every update of docker-machine:

It can break machine setup.

So many times I needed manually change config of my machines to just keep them alive. You can find a lot of issues related to that.

It will require you to upgrade docker on host machine.

This just does not work for production.

If you are building 12-factor apps - you don't care about docker-machine. If you are not there yet - you may manually upgrade your machines.

Let's just take a look on one example

1. You have Floating IP + Docker Droplet, which you created with docker-machine vX
2. New docker and docker-machine came out with vNext. You decided that you want to upgrade. And you want to do that with minimum downtime.
3. Create new droplet with docker-machine
4. Prepare setup (maybe docker-compose or just scripts). Verify that this droplet handles everything as you would expect.
5. Switch floating IP from old machine to new one.

Sounds good right? What would you do if something bad will happen between 1 and 4 and you want really quick fix/check something on old machine. You cannot do that - because it has old docker, so you need to downgrade to previous version of docker-machine (and obviously you need to know exactly which one).

Another example

You have dev and production environments. You upgraded docker-machine to start working integrating/testing new docker features on dev environment. But at the same moment something bad happens with production, how can you connect to it? Yes - you need to downgrade - do you know to which version of docker-machine? Probably not.

Concluding

Docker-machine is a great product. But I don't think that it should be used for production. Only dev environments. And I believe @nathanleclaire should document it somewhere.

[darkermatter]

Aaaaand compose just choked up a hairball so I have to start fixing the scripts. I'll update the gist when it's working.

[darkermatter]

@outcoldman To be fair, the last time I looked at the website, it stated that it shouldn't be used for production. Mind you, that was a year ago 😜

[outcoldman]

@darkermatter for some reason @nathanleclaire removed it https://github.com/docker/machine/commit/b2d8bcdd75558e5112b97c239c2fe29a60cf386c

[outcoldman]

@darkermatter @nathanleclaire actually created new issue for that #2817

[darkermatter]

Okay, a simplified Gist is up. Hope it helps.
https://gist.github.com/darkermatter/c57c112c0b24197ea99d

[narqo]

According to docker 1.10 rcX DOCKER_API_VERSION is supported in docker 1.10. It'd be great if docker-machine could use it so one dm client could properly manage hosts with different docker versions.

[twang2218]

I agree with @narqo , I was going to create a feature request to add DOCKER_API_VERSION to docker-machine env command, which will be a great help for managing multiple docker host with different versions.

And it's also very helpful for testing Docker for Mac/Windows, which ships 1.12-rc. The difference of the API broke my workflow, I have to manually switch the DOCKER_API_VERSION back and forth, besides docker-machine env, to manage those docker hosts.

So, please add DOCKER_API_VERSION to docker-machine env command.

[nathanleclaire]

@twang2218 To me that seems a reasonable request -- I'd be happy to evaluate a PR which:

1. Checks docker binary locally for version if exists. Skips printing DOCKER_API_VERSION if not.
2. Sets DOCKER_API_VERSION in env and config to the local client binary version.

[iBobik]

Any progress with PR?
@twang2218 @nathanleclaire

[s-koba]

I have made a PR #3839 to add DOCKER_API_VERSION to docker-machine env. There are some differences from the above proposal:

1. It uses docker-machine's internal docker client (libmachine/mcndockerclient) to retrieve the remote docker engine's API version. It does not check local docker binary.
2. docker-machine config remains unchanged because docker command does not seem to have an option to specify the API version.

I would appreciate your comments.