Moto: Travis Builds breaking due boto==2.49.0 trying to import google_compute_engine

[nackjicholson]

Reporting Bugs

Using moto==1.3.4

My pytest output in TravisCI is the following:

============================= test session starts ==============================
platform linux -- Python 3.6.3, pytest-3.7.1, py-1.5.4, pluggy-0.7.1
rootdir: /home/travis/build/CarbonLighthouse/intake, inifile:
plugins: cov-2.5.1
collected 0 items / 1 errors
==================================== ERRORS ====================================
______________________________ ERROR collecting  _______________________________
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config/__init__.py:372: in _getconftestmodules
    return self._path2confmods[path]
E   KeyError: local('/home/travis/build/CarbonLighthouse/intake/tests/integration')
During handling of the above exception, another exception occurred:
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config/__init__.py:403: in _importconftest
    return self._conftestpath2mod[conftestpath]
E   KeyError: local('/home/travis/build/CarbonLighthouse/intake/tests/integration/conftest.py')
During handling of the above exception, another exception occurred:
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config/__init__.py:409: in _importconftest
    mod = conftestpath.pyimport()
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/py/_path/local.py:668: in pyimport
    __import__(modname)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/assertion/rewrite.py:226: in load_module
    py.builtin.exec_(co, mod.__dict__)
tests/integration/conftest.py:12: in <module>
    from moto import mock_s3
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/moto/__init__.py:8: in <module>
    from .acm import mock_acm  # flake8: noqa
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/moto/acm/__init__.py:2: in <module>
    from .models import acm_backends
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/moto/acm/models.py:7: in <module>
    from moto.ec2 import ec2_backends
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/moto/ec2/__init__.py:2: in <module>
    from .models import ec2_backends
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/moto/ec2/models.py:13: in <module>
    import boto.ec2
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/boto/__init__.py:1216: in <module>
    boto.plugin.load_plugins(config)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/boto/plugin.py:93: in load_plugins
    _import_module(file)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/boto/plugin.py:75: in _import_module
    return imp.load_module(name, file, filename, data)
../../../virtualenv/python3.6.3/lib/python3.6/imp.py:235: in load_module
    return load_source(name, filename, file)
../../../virtualenv/python3.6.3/lib/python3.6/imp.py:172: in load_source
    module = _load(spec)
/usr/lib/python2.7/dist-packages/google_compute_engine/boto/compute_auth.py:19: in <module>
    from google_compute_engine import logger
E   ModuleNotFoundError: No module named 'google_compute_engine'
During handling of the above exception, another exception occurred:
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/py/_path/common.py:377: in visit
    for x in Visitor(fil, rec, ignore, bf, sort).gen(self):
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/py/_path/common.py:429: in gen
    for p in self.gen(subdir):
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/py/_path/common.py:418: in gen
    dirs = self.optsort([p for p in entries
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/py/_path/common.py:419: in <listcomp>
    if p.check(dir=1) and (rec is None or rec(p))])
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py:558: in _recurse
    ihook = self.gethookproxy(path)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/main.py:415: in gethookproxy
    my_conftestmodules = pm._getconftestmodules(fspath)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config/__init__.py:386: in _getconftestmodules
    mod = self._importconftest(conftestpath)
../../../virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/config/__init__.py:417: in _importconftest
    raise ConftestImportFailure(conftestpath, sys.exc_info())
E   _pytest.config.ConftestImportFailure: ModuleNotFoundError("No module named 'google_compute_engine'",)
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 226, in load_module
E       py.builtin.exec_(co, mod.__dict__)
E     File "/home/travis/build/CarbonLighthouse/intake/tests/integration/conftest.py", line 12, in <module>
E       from moto import mock_s3
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/moto/__init__.py", line 8, in <module>
E       from .acm import mock_acm  # flake8: noqa
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/moto/acm/__init__.py", line 2, in <module>
E       from .models import acm_backends
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/moto/acm/models.py", line 7, in <module>
E       from moto.ec2 import ec2_backends
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/moto/ec2/__init__.py", line 2, in <module>
E       from .models import ec2_backends
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/moto/ec2/models.py", line 13, in <module>
E       import boto.ec2
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/boto/__init__.py", line 1216, in <module>
E       boto.plugin.load_plugins(config)
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/boto/plugin.py", line 93, in load_plugins
E       _import_module(file)
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/boto/plugin.py", line 75, in _import_module
E       return imp.load_module(name, file, filename, data)
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/imp.py", line 235, in load_module
E       return load_source(name, filename, file)
E     File "/home/travis/virtualenv/python3.6.3/lib/python3.6/imp.py", line 172, in load_source
E       module = _load(spec)
E     File "/usr/lib/python2.7/dist-packages/google_compute_engine/boto/compute_auth.py", line 19, in <module>
E       from google_compute_engine import logger

It's really strange that it all of a sudden tries to lookup this google_compute_engine module out of python2.7 folder.

Upstream boto problem? I could try to pin my boto dependency to ==2.45.0 and see if it works.

Thanks for any help.

[nackjicholson]

Possibly related? Maybe this is happening because TravisCi is behind the scenes running on google cloud?

https://github.com/boto/boto/issues/3741

[nackjicholson]

Okay, I see this export BOTO_CONFIG=/dev/null in the moto .travis.yml.

I guess I have to do the same if I'm running a build on travis. If this isn't in your README or docs somewhere yet, it might be helpful to shout it out. This is a very wierd bug. Have you considered just dropping support for boto? 😆

[spulec]

Have you considered just dropping support for boto? <-- ha!

I think you are right that this is mostly a boto issue. I don't have a great place in the docs or README to add this, but hopefully people will find this issue going forward.