Sphinx: Could not import extension sphinx.builders.linkcheck (exception: No module named html_parser)

[gr2m]

I’m new to Python, please excuse if I’m making something obviously wrong.

I’ve just installed sphinx via pip. My Python version is 2.7.10, pip version is 9.0.1

Problem

After running sphinx-quickstart I run make html

$ make html
Running Sphinx v1.5.1

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: No module named html_parser)
make: *** [html] Error 1

Environment info

• OS: Mac Sierra 10.12.2
• Python version: 2.7.10
• Sphinx version: 1.5.1

[tk0miya]

What version of "six" did you use?
Could you show me a result of "pip freeze" command?

[gr2m]

What version of "six" did you use?

I don’t even know what "six" is :/

Could you show me a result of "pip freeze" command?

alabaster==0.7.9
altgraph==0.10.2
appscript==1.0.1
Babel==2.3.4
bdist-mpkg==0.5.0
blinker==1.2
bonjour-py==0.3
cssmin==0.1.4
Cython==0.17.1
docutils==0.13.1
eventlet==0.12.1
Fiona==0.8
Flask==0.9
Flask-Assets==0.7
Flask-Babel==0.8
Flask-Login==0.1.3
Flask-Mail==0.7.3
Flask-SQLAlchemy==0.16
Flask-WTF==0.8
GeoAlchemy==0.7.1
git-remote-helpers==0.1.0
greenlet==0.4.0
gunicorn==0.17.2
html-parser==0.2
imagesize==0.7.1
Jinja2==2.6
macholib==1.5.1
MapProxy==1.5.0
matplotlib==1.3.1
modulegraph==0.10.4
numpy==1.7.0
PIL==1.1.7
Pillow==2.0.0
ply==3.9
psycopg2==2.4.5
py-bcrypt==0.2
py2app==0.7.3
Pygments==2.1.3
pyobjc-core==2.5.1
pyobjc-framework-Accounts==2.5.1
pyobjc-framework-AddressBook==2.5.1
pyobjc-framework-AppleScriptKit==2.5.1
pyobjc-framework-AppleScriptObjC==2.5.1
pyobjc-framework-Automator==2.5.1
pyobjc-framework-CFNetwork==2.5.1
pyobjc-framework-Cocoa==2.5.1
pyobjc-framework-Collaboration==2.5.1
pyobjc-framework-CoreData==2.5.1
pyobjc-framework-CoreLocation==2.5.1
pyobjc-framework-CoreText==2.5.1
pyobjc-framework-DictionaryServices==2.5.1
pyobjc-framework-EventKit==2.5.1
pyobjc-framework-ExceptionHandling==2.5.1
pyobjc-framework-FSEvents==2.5.1
pyobjc-framework-InputMethodKit==2.5.1
pyobjc-framework-InstallerPlugins==2.5.1
pyobjc-framework-InstantMessage==2.5.1
pyobjc-framework-LatentSemanticMapping==2.5.1
pyobjc-framework-LaunchServices==2.5.1
pyobjc-framework-Message==2.5.1
pyobjc-framework-OpenDirectory==2.5.1
pyobjc-framework-PreferencePanes==2.5.1
pyobjc-framework-PubSub==2.5.1
pyobjc-framework-QTKit==2.5.1
pyobjc-framework-Quartz==2.5.1
pyobjc-framework-ScreenSaver==2.5.1
pyobjc-framework-ScriptingBridge==2.5.1
pyobjc-framework-SearchKit==2.5.1
pyobjc-framework-ServiceManagement==2.5.1
pyobjc-framework-Social==2.5.1
pyobjc-framework-SyncServices==2.5.1
pyobjc-framework-SystemConfiguration==2.5.1
pyobjc-framework-WebKit==2.5.1
pyOpenSSL==0.13.1
pyparsing==2.0.1
python-dateutil==1.5
pytz==2016.10
PyYAML==3.10
requests==0.14.2
scipy==0.13.0b1
scriptine==0.2.0a4
Shapely==1.2.16
simplejson==3.0.7
six==1.10.0
snowballstemmer==1.2.1
speaklater==1.3
Sphinx==1.5.1
SQLAlchemy==0.7.9
vboxapi==1.0
virtualenv==15.1.0
webassets==0.7
Werkzeug==0.8.3.dev20130327
WTForms==1.0.2
wxPython==3.0.2.0
wxPython-common==3.0.2.0
xattr==0.6.4
zope.interface==4.1.1

[tk0miya]

The "six" is a compatibility library between python2 and 3.
https://pythonhosted.org/six/

It seems you are using current release (1.10.0).

six==1.10.0

But, strangly, 1.10.0 actually provides html_parser module...

Could you add this into your conf.py and run sphinx again?

def setup(app):
    import six
    print('Six version: %s' % six.__version__)

Then, you'll see the correct version of "six" module like following:

$ make html
sphinx-build -b html -d _build/doctrees . _build/html
Running Sphinx v1.5.2+
making output directory...
Six version: 1.10.0
loading translations [ja]... done
loading pickled environment... not yet created
...

[gr2m]

Thank you so much for your patience and help, I really appreciate it!

I have added your code snippet to the beginning and the end of the conf.py file, but did not see any additional logs.

Here is my full conf.py file: https://gist.github.com/c6f166ed60b2e43a96e826728d6deba4

[tk0miya]

Okay, how about this? This only adds print function barely.

import six
print('Six version: %s' % six.__version__)

[gr2m]

It now prints the Six version

$ make html
Running Sphinx v1.5.1
Six version: 1.10.0

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: No module named html_parser)
make: *** [html] Error 1

[ctrl-f5]

I've got the same issue:

$ make html
sphinx-build -b html -d build/doctrees   source build/html
Running Sphinx v1.5.1
Six version: 1.10.0

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: No module named packages.urllib3.exceptions)
make: *** [html] Error 1

We are using sphinx to build documentation in our vagrant, since a couple of weeks it just broke after a new provision (using debian image)

[ctrl-f5]

I temporarily fixed it by installing a previous version:

pip install sphinx==1.4.8

[gr2m]

pip install sphinx==1.4.8 made it work for me, too

[jputrino]

I got the same error as @ctrl-f5 when running 1.5.1. It's running on Ubuntu 14.04 in a Docker container. Downgrading to 1.4.8 fixed the issue

root@9b397e8d68e6:/Users/putrino/project/docs# sphinx-build docs/ docs/_build/ docs/index.rst
Running Sphinx v1.5.1

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: No module named packages.urllib3.exceptions)

[tk0miya]

http://stackoverflow.com/a/31671281 tells us loading html_parser with following import-clause.

from six.moves import html_parser

I could not understand why this works fine. But six is very complex library. So some technical tips might be needed...

Could you try to rewrite your linkcheck.py with following patch?

--- a/sphinx/builders/linkcheck.py
+++ b/sphinx/builders/linkcheck.py
@@ -16,11 +16,12 @@ import threading
 from os import path

 from requests.exceptions import HTTPError
-from six.moves import queue
+from six.moves import queue, html_parser
 from six.moves.urllib.parse import unquote
-from six.moves.html_parser import HTMLParser
 from docutils import nodes

+HTMLParser = html_parser.HTMLParser
+
 # 2015-06-25 [email protected].  This exception was deprecated in Python 3.3 and
 # removed in Python 3.5, however for backward compatibility reasons, we're not
 # going to just remove it.  If it doesn't exist, define an exception that will

@ctrl-f5 @jputrino It seems your errors are similar, but different to this issue.
Could you file a new issue please?

[tk0miya]

@gr2m could you try the patch above?
If confirmed, I will merge it into next minor release.

[gr2m]

sorry I missed your last message, I’ll have a look over the weekend

[jputrino]

@tk0miya - the patch is working for me. Thanks!

[jfbu]

Is this issue system specific ? I tried on mac os,

$ conda create --name tempenv python=2.7.10 sphinx
...
$ source activate tempenv
$ cd /to/temp/loc
$ sphinx-quickstart
... add to conf.py:
def setup(app):
    import six, sys
    print('Six version: %s' % six.__version__)
    print(sys.version)
...
$ make html
$ make clean html
Removing everything under '_build'...
Running Sphinx v1.5.1
making output directory...
Six version: 1.10.0
2.7.10 |Continuum Analytics, Inc.| (default, Oct 19 2015, 18:31:17) 
[GCC 4.2.1 (Apple Inc. build 5577)]
loading pickled environment... not yet created
...

and no error. I wanted to try with virtualenv, but for some reason, virtualenv invocation ends with an error when done from my Python27 conda environment. This seems to be https://github.com/conda/conda/issues/1367, so I downgraded to Python 2.7.9. Then I did virtualenv venv, source venv/bin/activate, pip install Sphinx, etc... pip freeze returns

$ pip freeze
alabaster==0.7.9
Babel==2.3.4
docutils==0.13.1
imagesize==0.7.1
Jinja2==2.9.4
MarkupSafe==0.23
Pygments==2.1.3
pytz==2016.10
requests==2.12.4
six==1.10.0
snowballstemmer==1.2.1
Sphinx==1.5.1
(venv) 

I then try the sphinx-quickstart and make install make html ;-) and I get no issue. Unfortunately the Python used is 2.7.5 because for some reason virtualenv installed the one found in /usr/bin not the conda installed one (which is 2.7.9).

Thus I could test with 2.7.10 in a conda env, and with 2.7.5 in a virtualenv env, without being able to reproduce issue on my mac os x.

edit: thanks to tip at https://github.com/conda/conda/issues/1367#issuecomment-217958169 (fixed the link) I could get my virtualenv to actually use the 2.7.9 Python (this is conda issue)

[jputrino]

@jfbu -- It could be system-specific. I actually didn't experience the error when I was using virtualenv. My system python version is 2.7.12; in the virtualenv, it's 2.7.10.

[jfbu]

@jputrino I now tested in a virtualenv with Python 2.7.12 and I can not reproduce the issue on my mac os x.

$ make html
Running Sphinx v1.5.1
making output directory...
Six version: 1.10.0
2.7.12 |Continuum Analytics, Inc.| (default, Jul  2 2016, 17:43:17) 
[GCC 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)]
loading pickled environment... not yet created
building
...

(I print Python version via:

def setup(app):
        import six, sys
        print('Six version: %s' % six.__version__)
        print(sys.version)

in conf.py.

pip freeze in the virtual env reports:

$ pip freeze
alabaster==0.7.9
Babel==2.3.4
docutils==0.13.1
imagesize==0.7.1
Jinja2==2.9.4
MarkupSafe==0.23
Pygments==2.1.3
pytz==2016.10
requests==2.12.4
six==1.10.0
snowballstemmer==1.2.1
Sphinx==1.5.1
(venv) 

[tk0miya]

Fixed at a0c2029.
Thank you for reporting and confirming!

It will be released as v1.5.2 at this weekend.
Thanks,

[GabrielDelepine]

I am still experiencing this issue with the version 1.5.2.
I had to keep using the version 1.4.8 (because 1.5.1 gives the same error).

I am using pyinstaller to package sphinx to a stand-alone binary.

This is the way I am using:

# Install pip with the distribution package manager
sudo apt-get install python pip

# Install a specific version of Sphinx (WARNING 1.5.2 REPRODUCES THE CURRENT BUG DESCRIBE ABOVE )
sudo -H pip install sphinx==1.5.2

# Create a python script
echo "
import sys
import sphinx
sys.exit(sphinx.main(sys.argv))" > ./sphinx-build.py

# Check if your python script is working
python ./sphinx-build.py --version

# Install and upgrade Pyinstaller with the distribution package manager
sudo pip install pyinstaller
sudo pip install --upgrade pyinstaller

# Package Sphinx
pyinstaller --onefile --clean sphinx-build.py

# The sphinx binary is available in ./dist/sphinx-build

# Check if the binary works
./dist/sphinx-build -- version

After following this way, I get a stand-alone binary and I can reproduce the bug describe above.

[GabrielDelepine]

Does anyone have a feedback about it ? Should I open a new issue ? Probably yes I guess

[tk0miya]

I don't know what the pyinstaller does inside. So I can't help you...

[jbarajas]

I was able to successfully address this issue by installing:

pip install docutils==0.12

[travisspencer]

I had the issue on 1.5.1. Installing 1.5.2 fixed it from me on Mac OS X with Python 2.7.10.

[MingfeiPan]

reinstall 1.5.2, work perfectly in macox 10.12.3 with python 2.7.13

[epogrebnyak]

The problem resurfaced in Sphinx v2.0.1 for me on Windows:

```λ sphinx-build -b html docs site -c .
Running Sphinx v2.0.1

Extension error:
Could not import extension sphinx.builders.linkcheck (exception: No module named 'chardet.enums')
```