Sphinx: Sphinx 1.8.1 fails pickling environment

[NJDFan]

Subject: Sphinx 1.8.1 fails pickling environment.

Problem

I try to build the documentation for an internal project on which I've successfully built the documentation several times, but something has changed, which I recognize as one of the world's least helpful statements.

When I try to _make html_ or for that matter _make pickle_ I get

looking for now-outdated files... none found
pickling environment... 
Exception occurred:
  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 376, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <cyfunction __init__.<locals>.add_cdata at 0x7fdf3203d498>: attribute lookup add_cdata on lxml.builder failed

It doesn't seem to be a system-wide problem; I am able to rebuild another project of mine just fine with the same version of Python, Sphinx, etc.

Procedure to reproduce the problem

make html

Error logs / results

# Sphinx version: 1.8.1
# Python version: 3.5.2 (CPython)
# Docutils version: 0.14 
# Jinja2 version: 2.10
# Last messages:
#   reading sources... [100%] writingtests
#   
#   
#   
#   looking for now-outdated files...
#   
#   none found
#   
#   pickling environment...
#   
# Loaded extensions:
#   sphinx.ext.autodoc (1.8.1) from /home/rgaddi/.local/lib/python3.5/site-packages/sphinx/ext/autodoc/__init__.py
#   alabaster (0.7.12) from /home/rgaddi/.local/lib/python3.5/site-packages/alabaster/__init__.py
#   sphinx.ext.napoleon (1.8.1) from /home/rgaddi/.local/lib/python3.5/site-packages/sphinx/ext/napoleon/__init__.py
#   sphinx.ext.viewcode (1.8.1) from /home/rgaddi/.local/lib/python3.5/site-packages/sphinx/ext/viewcode.py
#   sphinx.ext.mathjax (1.8.1) from /home/rgaddi/.local/lib/python3.5/site-packages/sphinx/ext/mathjax.py
Traceback (most recent call last):
  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/cmd/build.py", line 304, in build_main
    app.build(args.force_all, filenames)
  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/application.py", line 341, in build
    self.builder.build_update()
  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 347, in build_update
    len(to_build))
  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 376, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <cyfunction __init__.<locals>.add_cdata at 0x7fdf3203d498>: attribute lookup add_cdata on lxml.builder failed

Expected results

Obviously I expect to make my HTML documentation. I assume that requires some intermediate pickling, though I don't quite know why.

Reproducible project / your project

Attached. Unfortunately I'm not able to try to build up a smaller test case so this is the entire whopping mess o' code.

Environment info

• OS: Linux - Ubuntu 16.04.5 LTS
• Python version: 3.5.2
• Sphinx version: 1.8.1

[NJDFan]

hti_test.tar.gz

Documentation sources and makefile is in the doc subdirectory, surprisingly to no one.

[NJDFan]

Same project builds fine on my machine at work.

• OS: Linux - Ubuntu 16.04.5 LTS
• Python version: 3.5.2
• Sphinx version: 1.6.2 (with alabaster 0.7.10)

[willingc]

Hi @NJDFan, It looks like the environment that you are building in does not have Cython installed and accessible. Perhaps give it a try in a virtual environment with Cython installed. I suspect that your work machine has Cython and that's why it is passing.

[NJDFan]

Hi @willingc, no luck. I installed Cython from apt (so it's system-wide), and it still fails with Can't pickle <cyfunction __init__.<locals>.add_cdata

[tk0miya]

@NJDFan Could you let me know about dependencies? I'd like to know it to reproduce the error. But it seems some extra packages are needed to build.

[tk0miya]

Any updates? I still need help to reproduce the error...

[willingc]

@tk0miya Perhaps close? Cython project look like they are building correctly.

[tk0miya]

@willingc Thanks, I'll wait a comment from @NJDFan for a while more.

[NJDFan]

Thanks for following up. I haven't had a chance to try to replicate, and I
won't have access to that computer again for two weeks or so. But I'm
definitely still interested in trying to get resolution on this.

On Sun, Dec 23, 2018, 08:52 Takeshi KOMIYA <[email protected] wrote:

@willingc https://github.com/willingc Thanks, I'll wait a comment from
@NJDFan https://github.com/NJDFan for a while more.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/sphinx-doc/sphinx/issues/5550#issuecomment-449645063,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AIIgCCyVBWBr4V-pUVshTdYO23FddPPAks5u76axgaJpZM4Xn6FW
.

[tk0miya]

Happy holidays!

[tk0miya]

ping.

[tk0miya]

I'm closing this now because of not reproduced.
Feel free to reopen if you're still in trouble.

Note: This error message says Sphinx tries to pickle local function on storing environment data. I guess your project set local function to config values. At present, it is not allowed.

  File "/home/rgaddi/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", line 376, in build
    pickle.dump(self.env, f, pickle.HIGHEST_PROTOCOL)
_pickle.PicklingError: Can't pickle <cyfunction __init__.<locals>.add_cdata at 0x7fdf3203d498>: attribute lookup add_cdata on lxml.builder failed

Thanks,