Boto: Problem importing boto in Python 3 running under Google Cloud

Created on 9 Jul 2015 · 6Comments · Source: boto/boto

Importing the module fails with the following message:

>>> import boto
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/boto/__init__.py", line 1216, in <module>
    boto.plugin.load_plugins(config)
  File "/usr/local/lib/python3.4/dist-packages/boto/plugin.py", line 93, in load_plugins
    _import_module(file)
  File "/usr/local/lib/python3.4/dist-packages/boto/plugin.py", line 75, in _import_module
    return imp.load_module(name, file, filename, data)
  File "/usr/lib/python3.4/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.4/imp.py", line 171, in load_source
    module = methods.load()
  File "/usr/share/google/boto/boto_plugins/compute_auth.py", line 64
    except (urllib2.URLError, urllib2.HTTPError, IOError), e:

After changing the line

except (urllib2.URLError, urllib2.HTTPError, IOError), e:

except (urllib2.URLError, urllib2.HTTPError, IOError) as e:

I got the error:

>>> import boto
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.4/dist-packages/boto/__init__.py", line 1216, in <module>
    boto.plugin.load_plugins(config)
  File "/usr/local/lib/python3.4/dist-packages/boto/plugin.py", line 93, in load_plugins
    _import_module(file)
  File "/usr/local/lib/python3.4/dist-packages/boto/plugin.py", line 75, in _import_module
    return imp.load_module(name, file, filename, data)
  File "/usr/lib/python3.4/imp.py", line 235, in load_module
    return load_source(name, filename, file)
  File "/usr/lib/python3.4/imp.py", line 171, in load_source
    module = methods.load()
  File "/usr/share/google/boto/boto_plugins/compute_auth.py", line 18, in <module>
    import urllib2
ImportError: No module named 'urllib2'

So I simply imported urllib3 as urllib2:

import urllib3 as urllib2

This seemed to solve the problem and I am now able to import boto. I haven't fully tested it to verify everything is working though (although I don't see why it shouldn't). For the record, my Python version is 3.4.2 and boto version is 2.38.0

Regards,

Sam

bug

Source

sbneto

Most helpful comment

The problem boils down to the fact that the travis CI machines have a global /etc/boto.cfg file that is automatically being loaded. One of the plugins called out in the /etc/boto.cfg file is not compatible with python3.

Given we don't actually want to load a boto config file, the easiest work around is to just set BOTO_CONFIG=/tmp/bogusvalue. I've commented on the tracking issue on travis that the ideal solution is to a) get that plugin compatible with python3 or b) remove that line from /etc/boto.cfg. I'll keep tracking that issue but I think for now I'd like to go ahead and just set this env var to get our builds passing in boto.

jamesls on 22 Dec 2015

👍3

All 6 comments

In Python 3.x, using as is required, hencewhy you got the error at the except (urllib2.URLError, urllib2.HTTPError, IOError), e: line. However, in Python 2.6+, the as is optional.

In the README.rst, Introduction section, this exists:

Currently, all features work with Python 2.6 and 2.7. Work is under way to support Python 3.3+ in the same codebase.

Thus, this is one of those things that needs to be updated for Py3 compatibility I would guess. The same goes for urllib2 - from its docs:

The urllib2 module has been split across several modules in Python 3.0 named urllib.request and urllib.error. The 2to3 tool will automatically adapt imports when converting your sources to 3.

You likely have urllib3 installed because you have Python 3, while boto is coded for urllib2 for Python 2.6 and 2.7.

TLDR: Boto is not 100% Python 3 ready yet, so YMMV.

jsanc623 on 9 Jul 2015

Reopening, this is causing py3 failures on travis. We need to look into this and figure out what's going on.

jamesls on 21 Dec 2015

This is also being investigated here:

https://github.com/travis-ci/travis-ci/issues/5246

ryansydnor on 21 Dec 2015

Ahh, that explains why google's in the stack trace. Interesting. The code that's failing is not part of boto, it's a plugin that boto's loading at runtime. I'll need to see what we can do about that.

jamesls on 21 Dec 2015