Compose: docker-compose create should create network in default v2 setup

[clkao]

with v2 services are default to use project-scoped network, however the _default network isn't created during docker-compose create, at least for the first time the project is run with.

This simple docker-compose.yml fails with docker-compose create:

version: '2.0'
services:
  foo:
    image: nginx

[NeoMcFly]

+1

Was my blocking issue by doing create. With docker-compose up now it'is ok. Create should works as well.

[clkao]

I just noticed that down actually removes the network, so I can see that network creation being part of up is more symmetric.

If having the network creation in create is not desired default behaviour, perhaps we can add a additional option to create to do so?

[dnephin]

create is only for creating containers. The network is created by up, this is by design.

[NeoMcFly]

So create fails by design ?

[NeoMcFly]

I am fine with the idea that create doesnt create networks.

In the same time, create should not stop with error about missing ot the default network. The command docker-compose create need a fix. I see two ways :

1) create command create networks. (New design which is questionnable i guess)

2) or create doesnt matter about missing default network. It justs validate the whole networks consistency without creating it.

[dnephin]

Please include the output you get when running docker-compose create. It may be that we need to create the network first.

[clkao]

With the test yml file above as a.yml:

% docker-compose --version
docker-compose version 1.6.0, build d99cad6
% docker-compose -p foo -f a.yml create
Creating foo_foo_1
ERROR: network foo_default not found

[rpeleg1970]

+1
the workaround now is to create the default network upfront (using docker network create foo_default) but it feels awkward mixing docker and docker-compose calls, especially since you explicitly need to reference generated resource names.

[KalleDK]

I got the same problem - I'm guessing the same can be said about the volumes

[squeaky-pl]

This was also strange to me. Kind of against principle of least astonishment.

[aanand]

docker-compose create is supposed to be a close analogue to docker create, so that's an argument against overloading it with responsibilities.

However, given there's no way to create the networks/volumes beforehand, I'm not sure we can avoid doing that in create. (We use the same rationale with docker-compose run to pre-create networks, volumes _and_ containers.)

[KalleDK]

I wouldn't mind if it was seperate from docker-compose create.. as in docker-compose volume create .. A command to create the volumes and/or the network used by a service.. So that I would be able to do

docker-compose ... volumene ..
docker-compose ... network ...
docker-compose create

And the reverse order for removal

[dnephin]

I'd actually prefer we remove docker-compose create entirely, and just add a flag to up to not start the containers. That's basically what the code is doing now, but we have all this extra stuff to support a new command that is almost identical the up.

[tvollmer]

I landed here for this issue, as well as differences between "docker-compose -p foo rm -v" and "docker-compose -p foo down -v"; Only the later removes Named-Volumes.

My use case

My use case is that I want to spin up a stack ( postgres, tomcat, http ) using docker-compose, however there appears to be no clean way to bring the stack up from scratch without hacking at it. My tomcat container not only relies on my postgres container, but tomcat won't start up if the db connection pool points to a database that doesn't exist. Likewise, my httpd container doesn't do much good to reverse-proxy to a tomcat container via ajp where the war doesn't exist. Additionally, I absolutely love the concept of named-volumes for clarity, and I may use data-containers to bundle sets of named-volumes.

So for now, my workflow was really clunky ( as a fresh set of eyes; using docker-compose with named volumes )

let's fire this thing up from scratch

docker-compose -p foo create # bang! this blows up because the of the network error
docker-compose -p foo up # hack1 : wait, then CTRL+C, because create didn't make a network
docker run ... # hack2 : 1 or more times to initially populate the named volumes
docker-compose -p foo up -d

Goal: see spot run

docker-compose -p foo ps # <-- this part works great, where have you been all my life?

Goal: I'm done playing now, so let's clean up

docker-compose -p foo down # removes the network, but not the named volumes, hmmm

Goal: [optionally make backups, then ] try to remove the volumes

docker-compose -p foo rm -v # doesn't actually remove the named volumes
docker-compose -p foo down -v # doesn't remove the named volumes either (bc the network error)
docker-compose -p foo up # hack3 : wait, CTRL+C, only ran this to be able to remove the volumes cleanly
docker-compose -p foo down -v # Oh! this time it removed the named volumes ... wth?

Conclusion ( my 2 cents [ as a newb ] on what it should do ):

Instead of being in the weeds with specific definitions of commands, I would like to see docker-compose do a better job of handling a full life cycle (setUp,do stuff,tearDown) of let's say, the top 10 or so stacks. Maybe someone already has a crafty way to solve what I'm after, and I just haven't found what I'm looking for. For example, even if I put my war inside my image, how would folks address the db-init problem? What about creating directories within named volumes BEFORE the containers need them?
1) 'create' should create the network, named volumes, and containers
2) 'down' should never remove the network or 'named volumes' or bind-mounts; may remove live container volumes and/or may leave containers in 'Exited' state
3) 'up' should create any missing pieces ... network, named volumes, containers ... to get the stack up.
4) 'rm -v' should be able to remove the dangling named volumes defined within the scope of the yml
5) Generally speaking create/remove should act in concert and up/down should act in concert. Additionally, I would expect 'up' to also do 'create' things [where they don't exist], and I would expect 'rm' to also do 'down' things [where they exist].

[dnephin]

docker-compose -p foo create # bang! this blows up because the of the network error

This is wrong. You want docker-compose -p foo up

create is a special case, it's not for bringing up the stack.

I think the problem is that we added create as a command. We should have added a --no-start flag to up instead of a create command.

docker-compose down should be fixed in 1.7.x to continue the removal of all resources, even if removing one of the earlier steps fail.

[sanmai-NL]

What's the current status of work on this issue?

[alexykot]

There is a design inconsistency here in my opinion.

There are commands:

docker-compose build
docker-compose create
docker-compose start

which are supposedly combined inside docker-compose up that essentially does all that in one go.

This is mirrored from Docker itself, where following works pretty well:

docker build
docker create
docker start

but it does fail for Docker Compose exactly because of this networking issue. I recognise the notion of trying to keep docker-compose create simple, but if that results in the larger process being effectively broken - the concept of keeping it simple doesn't make much sense.
The larger process has to work, this way or another.

[aanand]

If the job of create is to do everything that up does _except_ start the containers, I think @dnephin is right that it'd be better to deprecate create and have a --no-start flag for up instead.

Otherwise, if we really want to be fully consistent, we'll have to add to create every flag that currently exists on up, which feels pretty bloated to me.

[funkyfuture]

whatever the change will be, it'd be great to see it backported to the 1.8 series or even earlier.

atm it's not possible to create project volumes from a config without starting containers. i do consider this a bug as it doesn't contemplate docker-engine's functionality. the create command is unusable anyway.

as the consensus seems to be up --no-start, is there anything stopping this issue from being solved? create could be an alias of it. as a complement to docker-engine's create it seems intuitive.

i can implement and supply patches. shall i?

---

ps: up --no-start seems paradox on a semantic level.

[brunocascio]

@funkyfuture @aanand

Which is the current status of the feature up --no-start?

Thanks!

[soulne4ny]

Adding my two pennies, it should be possible to create every part of the setup without starting everything. It should be possible to inspect network, volume, etc without creating all the setup, and for sure without starting everything.

up is to monolithic to be flexible or at least easy to use while debugging.

up --no-start would be useful, but sounds quite crazy. up --only-create would be more precise.

What is there so special about up that it could not be done in separate steps? It looks to be a holy cow.

If the job of create is to do everything that up does except start the containers, I think @dnephin is right that it'd be better to deprecate create and have a --no-start flag for up instead.

Otherwise, if we really want to be fully consistent, we'll have to add to create every flag that currently exists on up, which feels pretty bloated to me.

@aanand, create already has _everything that it has in common_ with up. And up is already quite _ugly_. --no-start won't make it neither more nice nor clear.

Just to compare (docker-compose version 1.9.0, build 2585387)

$ docker-compose help create
...
Usage: up [options] [SERVICE...]

Options:
    -d                         Detached mode: Run containers in the background,
                               print new container names.
                               Incompatible with --abort-on-container-exit.
    --no-color                 Produce monochrome output.
    --no-deps                  Don't start linked services.
    --force-recreate           Recreate containers even if their configuration
                               and image haven't changed.
                               Incompatible with --no-recreate.
    --no-recreate              If containers already exist, don't recreate them.
                               Incompatible with --force-recreate.
    --no-build                 Don't build an image, even if it's missing.
    --build                    Build images before starting containers.
    --abort-on-container-exit  Stops all containers if any container was stopped.
                               Incompatible with -d.
    -t, --timeout TIMEOUT      Use this timeout in seconds for container shutdown
                               when attached or when containers are already
                               running. (default: 10)
    --remove-orphans           Remove containers for services not
                               defined in the Compose file

$ docker-compose help create
Creates containers for a service.

Usage: create [options] [SERVICE...]

Options:
    --force-recreate       Recreate containers even if their configuration and
                           image haven't changed. Incompatible with --no-recreate.
    --no-recreate          If containers already exist, don't recreate them.
                           Incompatible with --force-recreate.
    --no-build             Don't build an image, even if it's missing.
    --build                Build images before creating containers.

And up has already ugly flags --abort-on-container-exit and --remove-orphans.

• --remove-orphans should be a separate command. I don't see any reason why to limit its use only to starting everything.
• --abort-on-container-exit does not abort really, all the started stuff is left running, so it is just partial start, not an abort.

[nochtap]

@funkyfuture @aanand @dnephin
which is the status of the docker-compose up --no-start option?

[funkyfuture]

@brunocascio @nochtap i wouldn't move a finger before there's an agreement on the ux design. right now i wouldn't have neither time nor motivation anyway.

btw, i assume that network issue also applies to rm. this command already allows to specify whether volumes are handled or not via options. imo, that should also be possible with create.

[eexit]

Hello, what should we do in the case when up doesn't create the network?
I ran into that case today I've been struggled for a hour because my containers couldn't talk to each other.
It is very painful to manually create the network and assign all containers to the network :(

[indywidualny]

I was furious when I've encountered this problem. create is obviously broken and it's not a feature for sure :)

[washtubs]

What is so questionable about docker-compose create creating networks? There's a command called docker network create, I think it would make sense there.

[royalwood]

Currently, I'm running docker-compose up but, showing errors like as below.

Creating network "project_default" with the default driver
ERROR: could not find an available, non-overlapping IPv4 address pool among the defaults to assign to the network

how can I fix this problem? anyone could help me?

[rprots]

Clean-up unused networks:
```

Remove unneeded networks. If a network is used

then specific "docker network rm" command will fail and that is expected.

sudo docker network ls | awk '{print $1}' | xargs --no-run-if-empty sudo docker network rm```