Compose: ValueError: invalid literal for int() with base 0: '1g'

[fuelen]

$ docker-compose pull
Traceback (most recent call last):
  File "/usr/bin/docker-compose", line 11, in <module>
    load_entry_point('docker-compose==1.18.0', 'console_scripts', 'docker-compose')()
  File "/usr/lib/python3.6/site-packages/compose/cli/main.py", line 71, in main
    command()
  File "/usr/lib/python3.6/site-packages/compose/cli/main.py", line 121, in perform_command
    project = project_from_options('.', options)
  File "/usr/lib/python3.6/site-packages/compose/cli/command.py", line 37, in project_from_options
    override_dir=options.get('--project-directory'),
  File "/usr/lib/python3.6/site-packages/compose/cli/command.py", line 91, in get_project
    config_data = config.load(config_details)
  File "/usr/lib/python3.6/site-packages/compose/config/config.py", line 375, in load
    for config_file in config_details.config_files
  File "/usr/lib/python3.6/site-packages/compose/config/config.py", line 375, in <listcomp>
    for config_file in config_details.config_files
  File "/usr/lib/python3.6/site-packages/compose/config/config.py", line 506, in process_config_file
    environment)
  File "/usr/lib/python3.6/site-packages/compose/config/config.py", line 497, in interpolate_config_section
    environment
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 44, in interpolate_environment_variables
    for name, config_dict in config.items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 44, in <genexpr>
    for name, config_dict in config.items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 39, in process_item
    for key, val in (config_dict or {}).items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 39, in <genexpr>
    for key, val in (config_dict or {}).items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 54, in interpolate_value
    return recursive_interpolate(value, interpolator, get_config_path(config_key, section, name))
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 74, in recursive_interpolate
    for (key, val) in obj.items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 74, in <genexpr>
    for (key, val) in obj.items()
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 70, in recursive_interpolate
    return converter.convert(config_path, interpolator.interpolate(obj))
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 184, in convert
    return self.map[rexp](value)
  File "/usr/lib/python3.6/site-packages/compose/config/interpolation.py", line 141, in to_int
    return int(s, base=0)
ValueError: invalid literal for int() with base 0: '1g'

Arch Linux, docker-compose from official repos.

When I try to use binary docker-compose:

$ docker-compose logs
Traceback (most recent call last):
  File "bin/docker-compose", line 6, in <module>
  File "compose/cli/main.py", line 71, in main
  File "compose/cli/main.py", line 121, in perform_command
  File "compose/cli/command.py", line 37, in project_from_options
  File "compose/cli/command.py", line 91, in get_project
  File "compose/config/config.py", line 375, in load
  File "compose/config/config.py", line 506, in process_config_file
  File "compose/config/config.py", line 497, in interpolate_config_section
  File "compose/config/interpolation.py", line 44, in interpolate_environment_variables
  File "compose/config/interpolation.py", line 44, in <genexpr>
  File "compose/config/interpolation.py", line 39, in process_item
  File "compose/config/interpolation.py", line 39, in <genexpr>
  File "compose/config/interpolation.py", line 54, in interpolate_value
  File "compose/config/interpolation.py", line 74, in recursive_interpolate
  File "compose/config/interpolation.py", line 74, in <genexpr>
  File "compose/config/interpolation.py", line 70, in recursive_interpolate
  File "compose/config/interpolation.py", line 184, in convert
  File "compose/config/interpolation.py", line 141, in to_int
ValueError: invalid literal for int() with base 0: '1g'
Failed to execute script docker-compose

[fuelen]

The error was caused because I have the following line:

      mem_limit: 1g

I found it here: https://www.elastic.co/guide/en/elasticsearch/reference/5.4/docker.html

[shin-]

mem_limit only accepts int values. But yeah, it shouldn't blow up on you. Thanks for the report!

EDIT: It's unlikely mem_limit was the cause of the problem, but the error message for this issue going forward should help users diagnose the issue better.

[Nepoxx]

Shouldn't using units be a valid input? ('512m', '1g' etc.)

[jaseemabid]

@shin- Are you disallowing units in mem_limit now? The spec seems to allow it.

[shin-]

I don't think it was ever supported - but I'll take another look to make sure. Thanks for bringing it up.

[shin-]

Update on this: mem_limit used to accept literal strings like @Nepoxx 's examples, and it still does in 1.19.0-rc1. I believe the error reported by OP might have been caused by another field, like mem_swappiness, which only takes integers (but without a Compose file for reference, it's hard to say exactly).

Regardless, the original issue, which is the unhelpful stacktrace when an attempted conversion fails, has been fixed. So the following Compose file:

version: '2.3'
services:
  foo:
    image: busybox
    mem_swappiness: ${FOO:-1g}

produces the following error in 1.19.0-rc1:

$ docker-compose config
ERROR: Error while attempting to convert service.foo.mem_swappiness to appropriate type: "1g" is not a valid integer

as opposed to 1.18.0:

$ docker-compose config
Traceback (most recent call last):
[long stacktrace...]
    return int(s, base=0)
ValueError: invalid literal for int() with base 0: '1g'

Hope this clarifies things!

[jaseemabid]

@shin- Not really. The version 2 documentation says it accepts units for values[1]. You have effectively given a nice error message here rather than actually accept those values.

Why would 2 configuration parameters that accept memory values behave in different ways? If one accepts it, make it all accept it, or disable units for everything. Isn't consistency something users would expect here?

1. https://docs.docker.com/compose/compose-file/compose-file-v2/#specifying-byte-values

[shin-]

You might be thinking of something else, because mem_swappiness isn't a byte value, it's a number between 0 and 100.
https://docs.docker.com/engine/admin/resource_constraints/#--memory-swappiness-details

[jaseemabid]

@shin- If I docker-compose this file with version 1.18.0, build unknown installed from respositories on arch linux, I get the error ValueError: invalid literal for int() with base 0: '512m'.

Now if you grep the file, you see

   mem_limit: 512m
   mem_limit: 512m
   mem_limit: 512m
  mem_limit: 512m
  mem_limit: 512m

I'm suggesting that mem_limit should accept units because the spec says so and other similar config values suggests that it would.

[shin-]

Can you reproduce with the binary we provide here? https://github.com/docker/compose/releases/tag/1.18.0

I wasn't able to reproduce, so it may be an issue with the arch build. Otherwise, can you share your Compose file?

[jaseemabid]

@shin- I've linked to the docker compose file in the previous comment. https://github.com/zabbix/zabbix-docker/blob/3.4/docker-compose_v2_ubuntu_mysql_latest.yaml

[shin-]

@jaseemabid Your file has an indentation problem - you're declaring mem_limit as a sub-key of the ulimits declaration, which isn't valid.

[juanluisbaptiste]

I'm getting the same error as @jaseemabid mentioned with the same docker-compose. I'm confused as documentation says that mem_limit accepts the unit as part of the value, but I also get the same error. I'm using official build for ubuntu 18.0.3.1-ce.

So if that parameter really does not accept the units, shouldn't the documentation be updated to reflect this ?

[shin-]

The docs are correct. The format is supported. Your Compose file likely has an indentation error.

[juanluisbaptiste]

You are right, l missed that space, sorry.

On Fri, Aug 31, 2018, 11:28 AM Joffrey F [e-mail] wrote:

The docs are correct. The format is supported. Your Compose file likely
has an indentation error.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/docker/compose/issues/5527#issuecomment-417717776,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACBAMdGKLTnmpFM0rTeOwBAS5tMTqihEks5uWWQYgaJpZM4RR95B
.