Docker-py: Issue with port option in 2.4.0 version

Created on 28 Jun 2017  路  12Comments  路  Source: docker/docker-py

Hi,
I update to the 2.4 today and i got issue with docker-compose when i try to add the following line to my configuration file (docker-compose.yml) :
ports: - "127.0.0.1:9292:9090"

I got the following error:

ERROR: for ContainerName expected string or buffer Traceback (most recent call last): File "/usr/local/bin/docker-compose", line 11, in <module> sys.exit(main()) File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 68, in main command() File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 118, in perform_command handler(command, command_options) File "/usr/local/lib/python2.7/dist-packages/compose/cli/main.py", line 926, in up scale_override=parse_scale_args(options['--scale']), File "/usr/local/lib/python2.7/dist-packages/compose/project.py", line 424, in up get_deps File "/usr/local/lib/python2.7/dist-packages/compose/parallel.py", line 69, in parallel_execute raise error_to_reraise TypeError: expected string or buffer

I have no issue when i downgrade again to the 2.3 version of the package

To reproduce the issue, i use the following configuration ( it doesn't seem to depend on the image):

version: '2'

services :
    ContainerName:
        image: bae2d441e03a
        ports:
            - "127.0.0.1:9292:9090"

I run on Ubuntu 14.04.5 LTS with the following package:

docker==2.4.0
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1
Python 2.7.6
Client:
 Version:      17.05.0-ce
 API version:  1.29
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:06:06 2017
 OS/Arch:      linux/amd64

Server:
 Version:      17.05.0-ce
 API version:  1.29 (minimum version 1.12)
 Go version:   go1.7.5
 Git commit:   89658be
 Built:        Thu May  4 22:06:06 2017
 OS/Arch:      linux/amd64
 Experimental: false
kinbug
Source
picture erild
👍16

Most helpful comment

If you're on Arch and this bit you:

sudo pacman -U /var/cache/pacman/pkg/python-docker-2.3.0-1-any.pkg.tar.xz

assuming 2.3 is still in your pacman cache, confirmed the downgrade fixed it for me locally

picture gregflynn on 28 Jun 2017
👍6 ❤1

All 12 comments

I experienced a similar problem with docker-compose after upgrading docker-py to 2.4

Using both docker-compose 1.14 and 1.13 and the following configuration:

version: '3'

services:
  myservice:
    image: myimage
    ports:
      - "80:80"

I obtain the following error:

docker-compose up

ERROR: for myservice expected string or bytes-like object
Traceback (most recent call last):
  File "/usr/local/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 68, in main
    command()
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/usr/local/lib/python3.5/dist-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 424, in up
    get_deps
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
  File "/usr/local/lib/python3.5/dist-packages/compose/parallel.py", line 167, in producer
    result = func(obj)
  File "/usr/local/lib/python3.5/dist-packages/compose/project.py", line 410, in do
    rescale=rescale
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 460, in execute_convergence_plan
    self.show_scale_warnings(scale)
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 205, in show_scale_warnings
    if self.specifies_host_port() and desired_num > 1:
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in specifies_host_port
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 983, in <genexpr>
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/usr/local/lib/python3.5/dist-packages/compose/service.py", line 963, in has_host_port
    _, external_bindings = split_port(binding)
  File "/usr/local/lib/python3.5/dist-packages/docker/utils/ports.py", line 57, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or bytes-like object

Using compose 1.12 it still works

I added a couple of prints in split_port to confirm that compose calls the split_port function by passing a non-string argument:

def split_port(port):
    print(port)
    print(type(port))
    match = PORT_SPEC.match(port)

And this is the output:

80:80
<class 'str'>
80:80
<class 'str'>
ServicePort(target=80, published=80, protocol=None, mode=None, external_ip=None)
<class 'compose.config.types.ServicePort'>

ERROR: for myservice  expected string or bytes-like object
Traceback (most recent call last):
[...] (previous stack)
  File "/usr/local/lib/python3.5/dist-packages/docker/utils/ports.py", line 59, in split_port
    match = PORT_SPEC.match(port)
TypeError: expected string or bytes-like object

I think that the problem was introduced with this commit: 5dd91cd4aaa2e7cd8dde1dd316d53cab25ef9b78

Prior of this commit the cast to string of the input port (parts = str(port).split(':')) was robust enough to withstand this call from compose

mdantonio picture mdantonio on 28 Jun 2017
👍5

Same error here

aellwein picture aellwein on 28 Jun 2017

This is causing a pretty big issue on Circle CI right now

digitaldavenyc picture digitaldavenyc on 28 Jun 2017
👍5

If you're on Arch and this bit you:

sudo pacman -U /var/cache/pacman/pkg/python-docker-2.3.0-1-any.pkg.tar.xz

assuming 2.3 is still in your pacman cache, confirmed the downgrade fixed it for me locally

gregflynn picture gregflynn on 28 Jun 2017
👍6 ❤1

I used 

pip install docker-compose docker==2.3.0

to avoid the error with docker-compose "ports" notations.

aellwein picture aellwein on 28 Jun 2017
👍1

Yes same here

zikphil picture zikphil on 28 Jun 2017

My work around:

pip uninstall docker
pip uninstall docker-compose

pip install docker==2.3.0
pip install docker-compose==1.14.0

Referencing the docker-compose issue as well:

sono-bfio picture sono-bfio on 28 Jun 2017
👍1

Thanks everyone for the detailed reports. 2.4.1 should be out soon with the fix.

shin- picture shin- on 28 Jun 2017

I've just got this new error on the same condition with 2.4.1

ValueError: Invalid port "_ServicePort(target=8000, published=8001, protocol=None, mode=None, external_ip=None)", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]

EDIT:

Pip freeze

backports.ssl-match-hostname==3.5.0.1
cached-property==1.3.0
certifi==2017.4.17
chardet==3.0.4
colorama==0.3.9
docker==2.4.1
docker-compose==1.14.0
docker-pycreds==0.2.1
dockerpty==0.4.1
docopt==0.6.2
idna==2.5
jsonschema==2.6.0
PyYAML==3.12
requests==2.18.1
six==1.10.0
texttable==0.8.8
urllib3==1.21.1
websocket-client==0.44.0

Full trace back 

Traceback (most recent call last):
  File "/tmp/myvenv/bin/docker-compose", line 11, in <module>
    sys.exit(main())
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/cli/main.py", line 68, in main
    command()
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/cli/main.py", line 118, in perform_command
    handler(command, command_options)
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/cli/main.py", line 926, in up
    scale_override=parse_scale_args(options['--scale']),
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/project.py", line 424, in up
    get_deps
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/parallel.py", line 69, in parallel_execute
    raise error_to_reraise
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/parallel.py", line 167, in producer
    result = func(obj)
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/project.py", line 410, in do
    rescale=rescale
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/service.py", line 460, in execute_convergence_plan
    self.show_scale_warnings(scale)
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/service.py", line 205, in show_scale_warnings
    if self.specifies_host_port() and desired_num > 1:
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/service.py", line 983, in specifies_host_port
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/service.py", line 983, in <genexpr>
    return any(has_host_port(binding) for binding in self.options.get('ports', []))
  File "/tmp/myvenv/lib/python3.4/site-packages/compose/service.py", line 963, in has_host_port
    _, external_bindings = split_port(binding)
  File "/tmp/myvenv/lib/python3.4/site-packages/docker/utils/ports.py", line 60, in split_port
    _raise_invalid_port(port)
  File "/tmp/myvenv/lib/python3.4/site-packages/docker/utils/ports.py", line 43, in _raise_invalid_port
    'port[/protocol]' % port)
ValueError: Invalid port "ServicePort(target=8000, published=8001, protocol=None, mode=None, external_ip=None)", should be [[remote_ip:]remote_port[-remote_port]:]port[/protocol]
lindycoder picture lindycoder on 28 Jun 2017

Hmm, alright, should be good with 2.4.2

shin- picture shin- on 28 Jun 2017

Seems to be working, thank you!

lindycoder picture lindycoder on 28 Jun 2017

if you use localhost:7001:7001 ,the problem will be again ,however 127.0.0.1:7001:7001 is ok.
docker-compose version 1.15.0, build e12f3b9
docker-py version: 2.4.2
CPython version: 2.7.10

treemore picture treemore on 2 Aug 2017
Was this page helpful?
0 / 5 - 0 ratings

Related issues

uggla picture uggla  路  3Comments
paulcdejean picture paulcdejean  路  4Comments
ko-christ picture ko-christ  路  3Comments
AdamWRichardson picture AdamWRichardson  路  4Comments
jamesongithub picture jamesongithub  路  5Comments