Handson-ml: Docker-compose build fail

[lpatruno]

Hi Aurelion. Awesome book and great talk at AIConf last week...

Just tried building the docker image with compose and the build failed on

 RUN conda update -n base conda

with traceback

Step 3/33 : RUN conda update -n base conda
 ---> Running in e52f65e52a18
Traceback (most recent call last):
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 789, in __call__
    return func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/conda/cli/main.py", line 73, in _main
    from ..base.context import context
  File "/opt/conda/lib/python3.6/site-packages/conda/base/context.py", line 23, in <module>
    from ..common.configuration import (Configuration, LoadError, MapParameter, PrimitiveParameter,
  File "/opt/conda/lib/python3.6/site-packages/conda/common/configuration.py", line 34, in <module>
    from .serialize import yaml_load
  File "/opt/conda/lib/python3.6/site-packages/conda/common/serialize.py", line 28, in <module>
    yaml = get_yaml()
  File "/opt/conda/lib/python3.6/site-packages/conda/_vendor/auxlib/decorators.py", line 56, in _memoized_func
    result = func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/conda/common/serialize.py", line 17, in get_yaml
    import ruamel_yaml as yaml
ValueError: source code string cannot contain null bytes

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/opt/conda/bin/conda", line 11, in <module>
    sys.exit(main())
  File "/opt/conda/lib/python3.6/site-packages/conda/cli/main.py", line 126, in main
    return conda_exception_handler(_main, *args)
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 1025, in conda_exception_handler
    return_value = exception_handler(func, *args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 792, in __call__
    return self.handle_exception(exc_val, exc_tb)
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 832, in handle_exception
    return self.handle_unexpected_exception(exc_val, exc_tb)
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 844, in handle_unexpected_exception
    self.print_error_report(error_report)
  File "/opt/conda/lib/python3.6/site-packages/conda/exceptions.py", line 882, in print_error_report
    from .base.context import context
  File "/opt/conda/lib/python3.6/site-packages/conda/base/context.py", line 23, in <module>
    from ..common.configuration import (Configuration, LoadError, MapParameter, PrimitiveParameter,
  File "/opt/conda/lib/python3.6/site-packages/conda/common/configuration.py", line 34, in <module>
    from .serialize import yaml_load
  File "/opt/conda/lib/python3.6/site-packages/conda/common/serialize.py", line 28, in <module>
    yaml = get_yaml()
  File "/opt/conda/lib/python3.6/site-packages/conda/_vendor/auxlib/decorators.py", line 56, in _memoized_func
    result = func(*args, **kwargs)
  File "/opt/conda/lib/python3.6/site-packages/conda/common/serialize.py", line 17, in get_yaml
    import ruamel_yaml as yaml
ValueError: source code string cannot contain null bytes
ERROR: Service 'handson-ml' failed to build: The command '/bin/sh -c conda update -n base conda' returned a non-zero code: 1

Perhaps an issue in a new version of a package being installed?

[ageron]

Thanks @lpatruno , I'll take a look right away.

[ageron]

Mmh, this is a weird bug. @ziembla and @StevenBunkley, would you mind giving this a look please?

[ziembla]

Hi, I just spotted your problem and have no time to dig into now, but just off the top of my head: that's the line added by @drorata I think it was to address a change in anaconda base image. I suspect you have an older anaconda image (pulled some time ago into docker local image store) so you can try one of two things:

• just remove the offending line from the Dockerfile,
• "pull docker continuumio/anaconda3" to refresh it (if nothing else on your system depends on it's :latest)

Unfortunately, the way we're using docker now is letting it choose "all the latest versions" which is a mixed blessing. I've been thinking for some time about taking more control and preparing the curated image but that's a longer story... ;-D

Regards

[Raseth]

Hi, I've similar issue in the same step of docker building, however with slightly different error message:

sudo docker-compose build

Building handson-ml
Step 1/27 : FROM continuumio/anaconda3:2019.03
---> cb34e4508fdc
Step 2/27 : RUN apt-get update && apt-get upgrade -y && apt-get install -y libpq-dev build-essential git sudo cmake zlib1g-dev libjpeg-dev xvfb libav-tools xorg-dev libboost-all-dev libsdl2-dev swig unzip zip && rm -rf /var/lib/apt/lists/*
---> Using cache
---> b47f96939187
Step 3/27 : RUN conda update -n base conda
---> Using cache
---> 1b365fe0b133
Step 4/27 : RUN conda install -y joblib PyYAML==3.13
---> Running in ce69ab228072
WARNING conda.base.context:use_only_tar_bz2(632): Conda is constrained to only using the old .tar.bz2 file format because you have conda-build installed, and it is <3.18.3. Update or remove conda-build to get smaller downloads and faster extractions.
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed

UnsatisfiableError: The following specifications were found to be incompatible with each other:

• anaconda-navigator -> anaconda-client[version='>=1.6.14'] -> pyyaml
• anaconda==2019.03=py37_0 -> anaconda-client==1.7.2=py37_0 -> pyyaml
• anaconda==2019.03=py37_0 -> importlib_metadata==0.8=py37_0 -> zipp[version='>=0.3.2']
• bokeh -> pyyaml[version='>=3.10']
• conda-build -> pyyamldocker-compose version 1.24.0, build 0aa59064h
• conda-verify -> pyyaml
• dask -> bokeh[version='>=1.0.0'] -> pyyaml[version='>=3.10']
• pkgs/main/linux-64::importlib_metadata==0.8=py37_0 -> zipp[version='>=0.3.2']
• pkgs/main/linux-64::path.py==11.5.0=py37_0 -> importlib_metadata[version='>=0.5']
• pkgs/main/linux-64::zipp==0.3.3=py37_1
• pyyaml==3.13

ERROR: Service 'handson-ml' failed to build: The command '/bin/sh -c conda install -y joblib PyYAML==3.13' returned a non-zero code: 1

I use:

Docker version 18.09.6, build 481bc77
docker-compose version 1.24.0, build 0aa59064

And tried launching using:

docker build -t ageron .

[ageron]

Hi there,
I'm currently on vacation for 2 weeks. I'll check this when I get back, sorry for the delay (if anyone else wants to help, you're more than welcome).
Cheers,
Aurélien

[ageron]

Is the problem resolved?

[lvnilesh]

not yet unfortunately.

This step seemingly takes forever.

It is still running for over an hour now on a digital ocean box 8 vcpu / 32 GB Memory / 640 GB Disk / SFO2 - Debian 10 x64.

Step 5/27 : RUN conda install -y -c conda-forge         tensorflow         pyopengl         xgboost         nbdime
 ---> Running in 7821ab9bc9c1
Collecting package metadata (repodata.json): ...working... done
Solving environment: ...working... failed with initial frozen solve. Retrying with flexible solve.
Examining libssh2:   2%|▏         | 10/461 [00:00<00:00, 9175.90it/s]            
Comparing specs that have this dependency:  14%|█▍        | 1/7 [00:00<00:02,  2.72it/s]
Comparing specs that have this dependency:  29%|██▊       | 2/7 [10:30<26:15, 315.18s/it]                            
Finding shortest conflict path for libssh2:  25%|██▌       | 2/8 [00:11<00:33,  5.64s/it]