Compose: Docker Compose does not support defining port ranges in long form.

Created on 26 Jan 2018 · 18Comments · Source: docker/compose

Issue

docker-compose file format does not support a method of declaring port ranges in the long form syntax defined here: https://docs.docker.com/compose/compose-file/#long-syntax-1

Steps to Reproduce

docker-compose.yml:

version: '3.4'
services:
  helloWorld:
    image: hello-world
    ports:
      - target: 9900-9999
        published: 9900-9999
        protocol: tcp
        mode: host

command executed:

docker-compose -f ./docker-compose.yml config
docker-comopse -f ./docker-compose.yml up -d

Expected Output:

PS D:\issue> docker-compose -f ./docker-compose.yml config
services:
  helloWorld:
    image: hello-world
    ports:
      - target: 9900-9999
        published: 9900-9999
        protocol: tcp
        mode: host
version: '3.4'
PS D:\issue> docker-compose -f ./docker-compose.yml up -d
<ContainerId>

Actual Output:

PS D:\issue> docker-compose -f ./docker-compose.yml config
Traceback (most recent call last):
  File "docker-compose", line 6, in <module>
  File "compose\cli\main.py", line 71, in main
  File "compose\cli\main.py", line 118, in perform_command
  File "compose\cli\main.py", line 304, in config
  File "compose\cli\command.py", line 47, in get_config_from_options
  File "compose\config\config.py", line 375, in load
  File "compose\config\config.py", line 506, in process_config_file
  File "compose\config\config.py", line 497, in interpolate_config_section
  File "compose\config\interpolation.py", line 44, in interpolate_environment_variables
  File "compose\config\interpolation.py", line 44, in <genexpr>
  File "compose\config\interpolation.py", line 39, in process_item
  File "compose\config\interpolation.py", line 39, in <genexpr>
  File "compose\config\interpolation.py", line 54, in interpolate_value
  File "compose\config\interpolation.py", line 77, in recursive_interpolate
  File "compose\config\interpolation.py", line 74, in recursive_interpolate
  File "compose\config\interpolation.py", line 74, in <genexpr>
  File "compose\config\interpolation.py", line 70, in recursive_interpolate
  File "compose\config\interpolation.py", line 184, in convert
  File "compose\config\interpolation.py", line 141, in to_int
ValueError: invalid literal for int() with base 0: '9900-9999'
Failed to execute script docker-compose
PS D:\issue> docker-compose -f ./docker-compose.yml up -d
Traceback (most recent call last):
  File "docker-compose", line 6, in <module>
  File "compose\cli\main.py", line 71, in main
  File "compose\cli\main.py", line 121, in perform_command
  File "compose\cli\command.py", line 37, in project_from_options
  File "compose\cli\command.py", line 91, in get_project
  File "compose\config\config.py", line 375, in load
  File "compose\config\config.py", line 506, in process_config_file
  File "compose\config\config.py", line 497, in interpolate_config_section
  File "compose\config\interpolation.py", line 44, in interpolate_environment_variables
  File "compose\config\interpolation.py", line 44, in <genexpr>
  File "compose\config\interpolation.py", line 39, in process_item
  File "compose\config\interpolation.py", line 39, in <genexpr>
  File "compose\config\interpolation.py", line 54, in interpolate_value
  File "compose\config\interpolation.py", line 77, in recursive_interpolate
  File "compose\config\interpolation.py", line 74, in recursive_interpolate
  File "compose\config\interpolation.py", line 74, in <genexpr>
  File "compose\config\interpolation.py", line 70, in recursive_interpolate
  File "compose\config\interpolation.py", line 184, in convert
  File "compose\config\interpolation.py", line 141, in to_int
ValueError: invalid literal for int() with base 0: '9900-9999'
Failed to execute script docker-compose

Docker Version

PS D:\issue> docker version
Client:
 Version:       17.12.0-ce
 API version:   1.35
 Go version:    go1.9.2
 Git commit:    c97c6d6
 Built: Wed Dec 27 20:05:22 2017
 OS/Arch:       windows/amd64

Server:
 Engine:
  Version:      17.12.0-ce
  API version:  1.35 (minimum version 1.24)
  Go version:   go1.9.2
  Git commit:   c97c6d6
  Built:        Wed Dec 27 20:15:52 2017
  OS/Arch:      windows/amd64
  Experimental: true

Docker-Compose Version

D:\issue> docker-compose version
docker-compose version 1.18.0, build 8dd22a96
docker-py version: 2.6.1
CPython version: 2.7.14
OpenSSL version: OpenSSL 1.0.2k  26 Jan 2017

Observations

docker-compose only accepts a single port in long form syntax.
docker currently appears to only support a single port in long form syntax. See https://github.com/moby/moby/issues/32551

PS D:\issue> docker run -p mode=host,target=9900-9999,published=9900-9999,protocol=tcp -d hello-world
C:\Program Files\Docker\Docker\Resources\bin\docker.exe: Invalid containerPort: mode=host,target=9900-9999,published=9900-9999,protocol=tcp.
See 'C:\Program Files\Docker\Docker\Resources\bin\docker.exe run --help'.

Impacts

Customers with needs to define a large range of port bindings must declare each port individually if they need to use the long form syntax. This is burdensome and I would prefer to see parity between the long and short form syntax.

Workaround

Customers who do not need to use the long form syntax can use the short form syntax to declare a range of ports.
Customers who must use the long form syntax, to control which protocol is used or to bind the ports on the ingress network when deploying to swarm, need to declare each port individually in their compose files.

areconfig formav3 kinquestion

Source

estenrye

👍16

Most helpful comment

I also ran into this issue today and wold love to see it resolved!

SchoolGuy on 16 Dec 2019

👍6

All 18 comments

The error message should be improved in the next release.

As for actually supporting port ranges, the v3 format would need to be updated first - these changes happen at the docker/cli level, then get ported to docker-compose. Please open an issue there.

shin- on 26 Jan 2018

@shin- got it, will do

estenrye on 26 Jan 2018

It also does not support not stating which port published should be used. It would be useful to be able to say published: *.

MetalArend on 22 Feb 2018

It also does not support not stating which port published should be used.

Actually, you can do that by not stating which port to publish, e.g.

version: '3.5'
services:
  foo:
    image: busybox
    ports:
      - target: 9900
        protocol: udp

shin- on 22 Feb 2018

I found out what I did wrong: using mode: host without a published port does not show the port on portainer service overview. Did not mean to hijack the topic.

MetalArend on 23 Feb 2018

I'm running into this issue as well. Are there any updates on this?

christopher-coffin on 4 Jun 2019

@christopher-coffin This feature request can't be completed until the CLI supports the functionality.

If you go Thumbs Up this issue, it might get some traction?
https://github.com/docker/cli/issues/839

estenrye on 4 Jun 2019

This feature request can't be completed until the CLI supports the functionality.

I think the CLI does support it now

hugmoro on 12 Jun 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

stale[bot] on 9 Dec 2019

It'd be a bit sad to see this issue closed for inactivity, so here's a comment reasserting that the CLI supports the necessary syntax, the upstream issue should be closed to clarify this.

hugmoro on 9 Dec 2019

This issue has been automatically marked as not stale anymore due to the recent activity.

stale[bot] on 9 Dec 2019

I also ran into this issue today and wold love to see it resolved!

SchoolGuy on 16 Dec 2019

👍6

Still nothing?

chrisbecke on 8 Apr 2020

No updated on this yet? Encountered this problem as well.

smic-datalabs-von on 27 Apr 2020

This comment https://github.com/docker/cli/issues/839#issuecomment-500136821 seems to say that the problem is solved on the docker-cli side. The logical next step would be for compose to follow.

wbulot on 1 Jun 2020

👍3

I also have same problem. When I set endpoint_mode to dnsrr, it requires ports to be declared as host mode. So I need to write some port ranges long format but it doesn't allow to do that

ibotheperfect on 25 Aug 2020

Confirmed not fixed in 19.03.13, on Linux at least. It still creates two iptables rules for each port in the range, instead of using a ranged rule, and more than 1000 ports takes for.........ever to complete..

Here, I exposed ports 45000-45500 using the following:

docker-compose:

   ...
 ports:
      - "45000-45500:45000-45500/udp"
  ...

Resulting iptables ruleset... (Should be 2 lines, not 1000)

ACCEPT     udp  --  anywhere             anywhere             udp dpt:45500
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45500
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45499
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45499
...
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45006
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45005
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45005
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45004
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45004
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45003
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45003
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45002
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45002
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45001
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45001
ACCEPT     udp  --  anywhere             anywhere             udp dpt:45000
ACCEPT     udp  --  anywhere             anywhere             state RELATED,ESTABLISHED udp spt:45000

rwdim on 4 Nov 2020

One more thing, when trying to RUN the container using port ranges as follows (per the manual examples):

docker run -it --rm -p 8002:8888/tcp -p 45000-45500:45000/udp kurento:latest

Port mapping is not done at all. The first port of each set is added only to the DOCKER ruleset, and the range forwarding is not created in DOCKER-INGRESS:

Chain DOCKER (3 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             172.17.0.2           tcp dpt:45000
ACCEPT     tcp  --  anywhere             172.17.0.2           tcp dpt:8888


Chain DOCKER-INGRESS (1 references)
target     prot opt source               destination         
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:https
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:https
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:http
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:http
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:6379
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:6379
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8000
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:8000
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:8005
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:8005
ACCEPT     tcp  --  anywhere             anywhere             tcp dpt:2222
ACCEPT     tcp  --  anywhere             anywhere             state RELATED,ESTABLISHED tcp spt:2222
RETURN     all  --  anywhere             anywhere

UPDATE: using the following command DOES add the forwarding MAPPING, but not the state or source port... so it's still busted...

docker run -it --rm -p 8002:8888/tcp -p 45000-45500:45000-45500/udp kurento:latest

Gets:

Chain DOCKER (3 references)
target     prot opt source               destination   
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45011
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45010
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45009
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45008
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45007
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45006
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45005
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45004
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45003
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45002
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45001
ACCEPT     udp  --  anywhere             172.17.0.2           udp dpt:45000
ACCEPT     tcp  --  anywhere             172.17.0.2           tcp dpt:8888

rwdim on 4 Nov 2020

Was this page helpful?

0 / 5 - 0 ratings