Incubator-superset: Can't create admin user on Python 3.6

Created on 31 Dec 2016 · 14Comments · Source: apache/incubator-superset

Make sure these boxes are checked before submitting your issue - thank you!

[x] I have checked the superset logs for python stacktraces and included it here as text if any
[x] I have reproduced the issue with at least the latest released version of superset
[x] I have checked the issue tracker for the same issue and I haven't found one similar

Superset version

(latest as of 12/31/2016)

Expected results

Create admin user

Actual results

Was unable to import superset Error: module 'xml.etree.ElementTree' has no attribute '_IterParseIterator'

Steps to reproduce

fabmanager create-admin --app superset
Add all requested info
Error

I don't need this solved, I just downgraded Python versions and it worked fine. However, I wanted it noted as it is definitely an issue with Python 3.6 - tracked down that error separately. Worth noting as it is the version that installs by default with brew install python3 so I think this will start coming up a lot.

Source

tomasienrbc

👍1

Most helpful comment

@tomasienrbc, if you just want to play around with superset using Python 3.6, the following worked for me:

git clone https://github.com/tiran/defusedxml.git
cd defusedxml
curl -O https://patch-diff.githubusercontent.com/raw/tiran/defusedxml/pull/4.patch
git apply 4.patch
python setup.py sdist

You can then use pip from your superset virtualenv to install the patched defusedxml package. E.g.
pip install -U /path/to/defusedxml/dist/defusedxml-0.4.1.tar.gz

rsxm on 2 Jan 2017

👍7

All 14 comments

Please post the full backtrace so we can see where's the code that explodes.

xrmx on 1 Jan 2017

@xrmx here you go!

Tommys-MBP:apps tommy$ fabmanager create-admin --app superset /usr/local/lib/python3.6/site-packages/flask_sqlalchemy/__init__.py:30: ExtDeprecationWarning: Importing flask.ext.sqlalchemy is deprecated, use flask_sqlalchemy instead. from flask.ext.sqlalchemy._compat import iteritems, itervalues, xrange, \ Username [admin]: redacted User first name [admin]: Thomas User last name [user]: Nicholas Email [[email protected]]: [email protected] Password: Repeat for confirmation: /usr/local/lib/python3.6/site-packages/flask_migrate/__init__.py:4: ExtDeprecationWarning: Importing flask.ext.script is deprecated, use flask_script instead. from flask.ext.script import Manager Was unable to import superset Error: module 'xml.etree.ElementTree' has no attribute '_IterParseIterator'

tomasienrbc on 1 Jan 2017

Hi, the problem seems to be with defusedxml as reported here: https://github.com/tiran/defusedxml/issues/3. This patch seems to fix the problem: https://github.com/tiran/defusedxml/pull/4.patch

~~Once this patched is merged and published it needs to make its way through the dependency list (see my stacktrace below): defusedxml -> openid -> flask_openid -> flask_appbuilder -> superset~~ (only defusedxml needs to be updated, see my comment below)

Stacktrace

File "superset/superset/bin/superset", line 32, in load
  from superset import app
File "superset/superset/__init__.py", line 25, in <module>
  app.config.from_object(CONFIG_MODULE)
File "superset/venv/lib/python3.6/site-packages/flask/config.py", line 168, in from_object
  obj = import_string(obj)
File "superset/venv/lib/python3.6/site-packages/werkzeug/utils.py", line 418, in import_string
  __import__(import_name)
File "superset/superset/config.py", line 17, in <module>
  from flask_appbuilder.security.manager import AUTH_DB
File "superset/venv/lib/python3.6/site-packages/flask_appbuilder/security/manager.py", line 6, in <module>
  from flask_openid import OpenID
File "superset/venv/lib/python3.6/site-packages/flask_openid.py", line 27, in <module>
  from openid.extensions import ax
File "superset/venv/lib/python3.6/site-packages/openid/extensions/ax.py", line 16, in <module>
  from openid.server.trustroot import TrustRoot
File "superset/venv/lib/python3.6/site-packages/openid/server/trustroot.py", line 21, in <module>
  from openid.yadis import services
File "superset/venv/lib/python3.6/site-packages/openid/yadis/services.py", line 3, in <module>
  from openid.yadis.filters import mkFilter
File "superset/venv/lib/python3.6/site-packages/openid/yadis/filters.py", line 14, in <module>
  from openid.yadis.etxrd import expandService
File "superset/venv/lib/python3.6/site-packages/openid/yadis/etxrd.py", line 31, in <module>
  SafeElementTree = importSafeElementTree()
File "superset/venv/lib/python3.6/site-packages/openid/oidutil.py", line 83, in importSafeElementTree
  return importElementTree(module_names)
File "superset/venv/lib/python3.6/site-packages/openid/oidutil.py", line 106, in importElementTree
  ElementTree = __import__(mod_name, None, None, ['unused'])
File "superset/venv/lib/python3.6/site-packages/defusedxml/cElementTree.py", line 16, in <module>
  from .ElementTree import DefusedXMLParser, _IterParseIterator
File "superset/venv/lib/python3.6/site-packages/defusedxml/ElementTree.py", line 62, in <module>
  _XMLParser, _iterparse, _IterParseIterator, ParseError = _get_py3_cls()
File "superset/venv/lib/python3.6/site-packages/defusedxml/ElementTree.py", line 56, in _get_py3_cls
  _IterParseIterator = pure_pymod._IterParseIterator

rsxm on 2 Jan 2017

👍2

python3-openid does not pin a specific defusedxml version, so defusedxml is the only package that needs to be updated:

https://github.com/necaris/python3-openid/blob/master/setup.py:

install_requires=['defusedxml']

https://github.com/mitsuhiko/flask-openid/blob/master/setup.py:

install_requires = ['Flask>=0.10.1', 'python3-openid>=2.0']

rsxm on 2 Jan 2017

👍1

@tomasienrbc, if you just want to play around with superset using Python 3.6, the following worked for me:

git clone https://github.com/tiran/defusedxml.git
cd defusedxml
curl -O https://patch-diff.githubusercontent.com/raw/tiran/defusedxml/pull/4.patch
git apply 4.patch
python setup.py sdist

You can then use pip from your superset virtualenv to install the patched defusedxml package. E.g.
pip install -U /path/to/defusedxml/dist/defusedxml-0.4.1.tar.gz

rsxm on 2 Jan 2017

👍7

@jr-minnaar thanks so much! I may do a new install with Python 3.6 soon and I'll let you know if I run into any trouble. For now, 2.7 is working and I'm just playing around there.

tomasienrbc on 2 Jan 2017

If that works I can close this I guess? Is this something that should be just fixed in the project?

tomasienrbc on 2 Jan 2017

@tomasienrbc until a new version of defusedxml is released we can't do much. Until this does not happen though i think keeping this bug open with hints on how to fix the better solution.

Thanks a lot @jr-minnaar for digging.

xrmx on 3 Jan 2017

New version is up on pypi https://pypi.python.org/pypi/defusedxml/0.5.0rc1 and it includes Python 3.6 compatibility fix.

ridha on 2 Feb 2017

👍2

@ridha shouldn't that now be implemented back into the master project? Should I implement that fix or tag this - sorry I'm not 100% familiar with how to proceed but seems obvious they should upgrade the pypi version right?

tomasienrbc on 27 Feb 2017

@tomasienrbc , Only defusedxml needs to be updated for fixing this issue. Latest version of python3-openid (3.1.0) includes updated defusedxml and 3.6 support. I can confirm that it fixes it.

ridha on 27 Feb 2017

I have a message while creating my superset account, when I write down all the information i have an error message as bellow:
Was unable to import superset Error: [Errno 2] No such file or directory: u'C:\Python27\Scripts\superset\static\assets\package.json'