Readthedocs.org: Building problems when switching from conda to pip

[astrojuanlu]

Details

• Project URL: https://readthedocs.org/projects/poliastro/, https://readthedocs.org/projects/anhiga-poliastro/
• Build URL (if applicable): https://readthedocs.org/projects/anhiga-poliastro/builds/6082645/
• Read the Docs username (if applicable): Anhiga

We were building docs normally using conda:

• https://readthedocs.org/projects/poliastro/builds/6066764/
• https://github.com/poliastro/poliastro/tree/f0c8cb702

However, then we tried to switch to pip:

https://github.com/poliastro/poliastro/pull/258/commits/481206315

And we got the error AttributeError: 'HTML5Translator' object has no attribute 'set_first_last'.

https://readthedocs.org/projects/anhiga-poliastro/builds/6082645/

 Running Sphinx v1.6.4
making output directory...
loading translations [en]... done
loading pickled environment... not yet created
loading intersphinx inventory from https://docs.python.org/3/objects.inv...
loading intersphinx inventory from http://matplotlib.org/objects.inv...
loading intersphinx inventory from http://docs.astropy.org/en/stable/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/numpy/objects.inv...
loading intersphinx inventory from https://docs.scipy.org/doc/scipy/reference/objects.inv...
building [mo]: targets for 0 po files that are out of date
building [epub]: targets for 19 source files that are out of date
updating environment: 19 added, 0 changed, 0 removed
reading sources... [  5%] about
reading sources... [ 10%] api
epoch True
state True
churi True
iss True
molniya True
soyuz_gto True
reading sources... [ 15%] changelog
reading sources... [ 21%] dastcom5 parameters
reading sources... [ 26%] examples/Catch that asteroid!
Executing notebook with kernel: python3
reading sources... [ 31%] examples/Comparing Hohmann and bielliptic transfers
Executing notebook with kernel: python3
reading sources... [ 36%] examples/Exploring the New Horizons launch
Executing notebook with kernel: python3
reading sources... [ 42%] examples/Going to Jupiter with Python using Jupyter and poliastro
Executing notebook with kernel: python3
reading sources... [ 47%] examples/Going to Mars with Python using poliastro
Executing notebook with kernel: python3
reading sources... [ 52%] examples/Propagation using Cowell's formulation
Executing notebook with kernel: python3
reading sources... [ 57%] examples/Revisiting Lambert's problem in Python
Executing notebook with kernel: python3
reading sources... [ 63%] examples/Studying Hohmann transfers
Executing notebook with kernel: python3
reading sources... [ 68%] examples/Using NEOS package
Executing notebook with kernel: python3
reading sources... [ 73%] getting_started
reading sources... [ 78%] index
reading sources... [ 84%] jupyter
reading sources... [ 89%] references
reading sources... [ 94%] success
reading sources... [100%] user_guide

looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
writing output... [  5%] about
writing output... [ 10%] api
writing output... [ 15%] changelog
writing output... [ 21%] dastcom5 parameters
writing output... [ 26%] examples/Catch that asteroid!
writing output... [ 31%] examples/Comparing Hohmann and bielliptic transfers
writing output... [ 36%] examples/Exploring the New Horizons launch
writing output... [ 42%] examples/Going to Jupiter with Python using Jupyter and poliastro
writing output... [ 47%] examples/Going to Mars with Python using poliastro
/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/importlib/_bootstrap.py:222: RuntimeWarning: numpy.dtype size changed, may indicate binary incompatibility. Expected 96, got 88
  return f(*args, **kwds)

Traceback (most recent call last):
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/cmdline.py", line 306, in main
    app.build(opts.force_all, filenames)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/application.py", line 339, in build
    self.builder.build_update()
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 331, in build_update
    'out of date' % len(to_build))
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 397, in build
    self.write(docnames, list(updated_docnames), method)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 434, in write
    self._write_serial(sorted(docnames))
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 443, in _write_serial
    self.write_doc(docname, doctree)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/_epub_base.py", line 359, in write_doc
    StandaloneHTMLBuilder.write_doc(self, docname, doctree)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/builders/html.py", line 555, in write_doc
    self.docwriter.write(doctree, destination)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/docutils/writers/__init__.py", line 80, in write
    self.translate()
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/sphinx/writers/html.py", line 56, in translate
    self.document.walkabout(visitor)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
    if child.walkabout(visitor):
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/docutils/nodes.py", line 174, in walkabout
    if child.walkabout(visitor):
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/docutils/nodes.py", line 166, in walkabout
    visitor.dispatch_visit(self)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/docutils/nodes.py", line 1882, in dispatch_visit
    return method(node)
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/nbsphinx.py", line 1262, in visit_admonition_html
    self.set_first_last(node)
AttributeError: 'HTML5Translator' object has no attribute 'set_first_last'

Exception occurred:
  File "/home/docs/checkouts/readthedocs.org/user_builds/anhiga-poliastro/envs/rtd-pip/lib/python3.5/site-packages/nbsphinx.py", line 1262, in visit_admonition_html
    self.set_first_last(node)
AttributeError: 'HTML5Translator' object has no attribute 'set_first_last'
The full traceback has been saved in /tmp/sphinx-err-ayurt3so.log, if you want to report the issue to the developers.
Please also report this if it was a user error, so that a better error message can be provided next time.
A bug report can be filed in the tracker at <https://github.com/sphinx-doc/sphinx/issues>. Thanks!

We had the suspicion that the Sphinx version was being incorrectly updated, so we restricted it to sphinx<1.6 in our own requirements. This seemed to fix the previous issue, but still the build never finishes and yields a timeout:

https://readthedocs.org/projects/anhiga-poliastro/builds/6083360/

Anything we are doing wrong?

[astrojuanlu]

The timeouts were due to the fact that we were running nbsphinx with some heavy notebooks, so I conditionally disabled it and now everything works. So I guess the problem is now reduced to: what happens when pip accidentally tries to update the Sphinx version to one different to the one pinned in RTD?

[humitos]

what happens when pip accidentally tries to update the Sphinx version to one different to the one pinned in RTD?

It _shouldn't_ be a problem if it's still compatible with some sphinx extensions (like https://github.com/rtfd/sphinx_rtd_theme and https://github.com/rtfd/readthedocs-sphinx-ext) that are used by RTD.

Regarding timeout, if I remember correctly it's 15 minutes per build.

Finally, are you able to build the docs with READTHEDOCS=True and those packages that I mentioned installed in your own virtualenv _and_ the latest sphinx version?

[gpagliuca]

I have the same AttributeError: 'HTML5Translator' object has no attribute 'set_first_last' for my project here https://github.com/gpagliuca/pyfas.
Fixed forcing sphinx<1.6 as well

[astrojuanlu]

I was hit by this today again, don't think it has to do with conda:

https://readthedocs.org/projects/telluric/builds/7029356/

I commented in https://github.com/spatialaudio/nbsphinx/issues/176 anyway.

[astrojuanlu]

I guess my particular problem is that I'm installing some extra requirements:

https://github.com/satellogic/telluric/blob/c62b757f5fed7789977a4c87f1cf31a181c59166/readthedocs.yml#L1-L5

And, among them, there's Sphinx:

https://github.com/satellogic/telluric/blob/c62b757f5fed7789977a4c87f1cf31a181c59166/setup.py#L44

The second pip install step in https://readthedocs.org/projects/telluric/builds/7029356/ is using pip install --ignore-installed, and it's overriding the Sphinx version with 1.7.2 and docutils with 0.14. Then I guess something breaks and we have this issue.

Now setting sphinx<1.7 is not enough (since I opened this issue, the default Sphinx version in RTD has changed I think, or at least that's what I understand from the build logs):

https://readthedocs.org/projects/juanlu001-telluric/builds/7030485/

and docutils<0.14 didn't work either:

https://readthedocs.org/projects/juanlu001-telluric/builds/7030502/

Only sphinx<1.6 works:

https://readthedocs.org/projects/juanlu001-telluric/builds/7030523/

I hope this gives enough information to debug the issue.

[humitos]

I'm not sure that this is the problem in RTD itself. It seems to be a problem of incompatibility between sphinx/docutils/nbsphinx packages to me.

The second pip install step in readthedocs.org/projects/telluric/builds/7029356 is using pip install --ignore-installed, and it's overriding the Sphinx version with 1.7.2 and docutils with 0.14. Then I guess something breaks and we have this issue.

RTD installs by default the minimum pip packages to build the documentation (suppose that the repo only has a index.rst without requirements or anything else) after the virtualenv is created. Then, we _need_ to call pip with --ignore-installed using the requirements.txt file provided by the user (if exists) to install extra packages but also to override to specific versions the ones that were installed by default by RTD.

That said, since that I'm not seeing a clear problem in RTD that I can think on how to solve it. What is the RTD issue or solution in your opinion? Thanks!

[astrojuanlu]

According to nbsphinx CI, it's tested against a variety of Sphinx versions,
so I'm still not sure what's happening. I will try to reproduce this issue
locally in a couple of weeks.

On Thu, Apr 12, 2018, 7:21 AM Manuel Kaufmann [e-mail]
wrote:

I'm not sure that this is the problem in RTD itself. It seems to be a
problem of incompatibility between sphinx/docutils/nbsphinx packages to me.

The second pip install step in
readthedocs.org/projects/telluric/builds/7029356 is using pip install
--ignore-installed, and it's overriding the Sphinx version with 1.7.2 and
docutils with 0.14. Then I guess something breaks and we have this issue.

RTD installs by default the minimum pip packages to build the
documentation (suppose that the repo only has a index.rst without
requirements or anything else) after the virtualenv is created. Then, we
need to call pip with --ignore-installed using the requirements.txt
file provided by the user (if exists) to install extra packages but also to
override to specific versions the ones that were installed by default by
RTD.

That said, since that I'm not seeing a clear problem in RTD that I can
think on how to solve it. What is the RTD issue or solution in your
opinion? Thanks!

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/rtfd/readthedocs.org/issues/3148#issuecomment-380822602,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AATUZcxEHF1YPjjQjJgdJA8NGHTSzUwkks5tn2LogaJpZM4PwFJP
.

[astrojuanlu]

The new release of nbsphinx fixed the issue: https://github.com/spatialaudio/nbsphinx/issues/176#issuecomment-384201244

It was a problem with the epub builder. Sorry for bothering you with this long standing bug report that I should have studied better, and thanks for building an awesome product!

[humitos]

Thanks for the update on this!