Spyder: Sphinx >=1.7.0 breaks help on Windows with a working dir on a drive different from the Python install drive

[VanitarNordic]

I faced this error while I was coding (typing) in the editor:

after closing it, the Spyder continues to its normal operation, I did not see any after effect.

Windows 10 - Python 3.6.4. The Spyder 3.2.7 was installed using Anaconda

[DanielAbdelNour]

I had to downgrade Sphinx to resolve this. It affected Spyder's help section rendering in my case.
This can be recreated by following the menu path Help > Tutorial.

[CAM-Gerlach]

Thanks for reporting. I was unable to reproduce this on Windows 8.1 with the same Python and Spyder versions when I selected Help > Tutorial, and haven't seen it yet. Do you have Spyder/Anaconda installed in a different drive than your boot disk? That's what the error messages seems to suggest...

[VanitarNordic]

@CAM-Gerlach
You're welcome. No I have the Anaconda on the boot drive which is C. This message appeared only after upgrading the Sphinx. in my case this message appeared when I start a new instance of the Spyder and start coding using autocomplete.

[CAM-Gerlach]

This message appeared only after upgrading the Sphinx

Upgrading from which to which version?

in my case this message appeared when I start a new instance of the Spyder and start coding using autocomplete.

So it only appears once per session, and never again? Is it consistant? WHat do you mean, "using autocomplete"? What sequence(s) of actions exactly triggers?

It seems the proximate cause might be related to this bit of windows nonsense, but I'm not quite sure yet what could actually be triggering it in your case. Is any of your code on anther drive?

[VanitarNordic]

as the picture says, upgrading the Sphinx to 1.7.1.

Let me add some information and update the previous post. @DanielAbdelNour is also correct.

I captured a video from the screen to how reproduce it. This proves you can reproduce the error more than once and it is consistent and it is related to the help.

[CAM-Gerlach]

as the picture says, upgrading the Sphinx to 1.7.1.

As my question says, from which version?

I captured a video from the screen to how reproduce it.

Thanks.

This proves you can reproduce the error more than once and it is consistent and it is related to the help.

More than once _in the same session_, or just generally?

Regardless, there seems to be something machine specific going on here, as again, I'm unable to reproduce it on my system with Python 3.6.4, Spyder 3.2.7, Sphinx 1.7.1 and Windows 8.1. Please post the full system information that was requested in the issue template (versions of Qt, PyQt, and dependencies) to help us narrow this down.

@DanielAbdelNour Can you do the same? Also, downgrade sphinx to what version? 1.7.0? 1.6.6? Something else?

Thanks.

[VanitarNordic]

@CAM-Gerlach

Sorry I did not get that question. The answer is that I remember that I upgraded the Sphinx two times. once from something to 1.7,0 and the last from 1.7.0 to 1.7.1. so what version it had was just before 1.7.0. Besides, this error was producing with 1.7.0 also. I'll put the whole Conda environment information.

[CAM-Gerlach]

@VanitarNordic Thanks. Can you try downgrading to 1.6.6 (with conda install sphinx=1.6.6 and see if the error reoccurs? If not, can you try to locate the version that stops causing the problem? If you can't find it, did you update anything else at the same time you updated Sphinx and first saw the problem?

The environment information is nice, but probably (hopefully) not necessary. What I asked for was what was stated in the new issue template that was apparently deleted/not followed, specifically your Qt and PyQt versions (from Help > About) and main dependencies (From Help > Dependencies > Copy to clipboard).

[VanitarNordic]

Dependencies:

IPython >=4.0    :  6.2.1 (OK)
cython >=0.21    :  None (NOK)
jedi >=0.9.0     :  0.11.1 (OK)
nbconvert >=4.0  :  5.3.1 (OK)
numpy >=1.7      :  1.14.1 (OK)
pandas >=0.13.1  :  0.22.0 (OK)
pycodestyle >=2.3:  2.3.1 (OK)
pyflakes >=0.6.0 :  1.6.0 (OK)
pygments >=2.0   :  2.2.0 (OK)
pylint >=0.25    :  1.8.2 (OK)
qtconsole >=4.2.0:  4.3.1 (OK)
rope >=0.9.4     :  0.9.4-1 (OK)
sphinx >=0.6.6   :  1.7.1 (OK)
sympy >=0.7.3    :  None (NOK)

QT and PyQT:

Can you try downgrading to 1.6.6 (with conda install sphinx=1.6.6 and see if the error reoccurs?

I'll try downgrading a.s.a.p. I remember that I updated some other components by Conda commands either, but I don't remember what they were. For now before testing the downgrading, I opened closed/re-opened the Spyder and the error does not appear by the same procedure. Therefore we can say it does not always appear, but when it appears, it is consistent.

[CAM-Gerlach]

Thanks. Let us know what you find.

Therefore we can say it does not always appear, but when it appears, it is consistent.

So again, just to be clear: This consistently you speak of is within the same Spyder session?

[VanitarNordic]

So again, just to be clear: This consistently you speak of is within the same Spyder session?

Yes, I always open just one Spyder session. When it appears like few minutes ago, it is consistent and reproducible in that session. I closed and opened the Spyder again. it did not appear so far nor reproducible.

[CAM-Gerlach]

Hmm, interesting. So it has happened over multiple previous sessions? Any pattern to that? Also, anything different in your current session—project location/working directory, files open, etc?

[VanitarNordic]

@CAM-Gerlach

Aha, now it appeared again in the same session which I told you it has not appeared so far,

Now, it can be produced by the same process.

I don't do anything different to create just this error. For the pattern, it appears after I run a code. if I don't run a code it does not appear. The code is on Drive D and Anaconda is installed on Drive C as Windows. Everything including the code is identical before and after I upgraded the Sphinx.

[CAM-Gerlach]

The code is on Drive D and Anaconda is installed on Drive C as Windows

Well there we go, which is why in my original followup, I asked

Is any of your code on another drive?

With that in hand, I was able to immediately reproduce it after switching my working directory (top right) to another drive, and then went to "Show tutorial" as you mentioned. Therefore, it seems likely the error has to do with use of a relative path (i.e. relative to the working dir) that attempts to access a location on another windows drive somewhere in the Sphinx call. From there, it shouldn't be too difficult for someone to trace and fix. I appreciate your help!

[CAM-Gerlach]

I can confirm that all sphinx help is in fact broken (not showing the tutorial, etc. at all, and only showing plain text of docstrings for Sphinx 1.7.0+ on Windows under the above stated conditions. I can confirm Sphinx <= 1.6.6 is unaffected, so the location for now is to downgrade to that with conda install sphinx=1.6.6 from the Anaconda prompt (or just don't use a working directory different from your Python/Anaconda install).

[VanitarNordic]

@CAM-Gerlach

I had to leave the discussion because it was midnight in my time. All parts of the code are in a single file on Drive D, which Anaconda and Windows are on Drive C. It is the identical situation as it was before upgrading the Sphinx. Now you reproduced it. Thank you very much for your follow up and patience.

[CAM-Gerlach]

No problem; sorry for the confusion. What I meant by

Is any of your code on anther drive?

is another drive relative to your Anaconda/Spyder install; I guess I assumed that would be the most straightforward interpretation (or at least a potential interpretation, such that it would have solicited a response if anything deviated from a "baseline" setup with all code and Anaconda on the same drive) but I guess should have been explicit specifying both.

In any case, thanks to your help, the problem is finally isolated so it can be solved.

[RicardoNoguerales]

This error message is fired in C:\Python27\Lib\site-packages\spyder\plugins\help.py
when an exception in method runof class SphinxThreadis handled by the try: except: blocks

This handling prevents the full traceback been shown in Spyder's Internal terminal
If you deactivate the try-except, the traceback appears:

>>> Traceback (most recent call last):
  File "c:\python27\lib\site-packages\spyder\plugins\help.py", line 301, in run
    html_text = sphinxify(doc['docstring'], context)
  File "c:\python27\lib\site-packages\spyder\utils\help\sphinxify.py", line 215, in sphinxify
    sphinx_app.build(None, [rst_name])
  File "c:\python27\lib\site-packages\sphinx\application.py", line 344, in build
    'outdir': path.relpath(self.outdir),
  File "c:\python27\lib\ntpath.py", line 528, in relpath
    % (path_prefix, start_prefix))
ValueError: path is on drive c:, start on drive E:     

The problem explained

• Help section rendering uses Sphinx to convert rst text help to nice HTML.Sphinx logs this activity.
• Part of this logging is in buildmethod of class Sphinx
  c:\Python27\Lib\site-packages\sphinx\application.py circa line 343
  logger.info(self.builder.epilog % { 'outdir': path.relpath(self.outdir), 'project': self.config.project })
  Calls path.relpath ( os.path.relpath, in Windows defined in c:\python27\lib\ntpath.py ) with a single argument, some temporary dir usually in C:\
• Being called with a single argument, relpathuses second argument's default '.', the python current working dir, say os.getcwd()
• relpathfires a ValueErrorif called with paths from different drives.
  So if Spyder's working dir (python current working dir) is not in drive C: , the exception is fired

Why once per session

This exception switches help display mode from _richt text_ to _plain text_

_plain text_ mode doesnt use Sphinx, so the problem dissapears unless you switch back to _richt text_ mode.

SOLUTIONS

Being involved Sphinx, Spyder and ntpath.py, there are various options

• Patch ntpath.py
  You could patch relpathin c:\python27\lib\ntpath.py , but this could be dangerous, other scripts can expect this exception been fired and handle it their own way.

• Patch Sphinx (my choose)
  c:\Python27\Lib\site-packages\sphinx\application.py this way

###                logger.info(self.builder.epilog % {
###                    'outdir': path.relpath(self.outdir),
###                    'project': self.config.project
###                })
                # PATCH----------------------------------------------
                try:
                    the_relpath = path.relpath(self.outdir)
                except ValueError:
                    the_relpath = self.outdir
                logger.info(self.builder.epilog % {
                    'outdir': the_relpath,
                    'project': self.config.project
                })
                # PACH----------------------------------------------

• Override relpath in Spyder
  In C:\Python27\Lib\site-packages\spyder\utils\help\sphinxify.py
  after line 32 , from sphinx.application import Sphinx
  insert this

#-----------------------------------------------------------#PATCH
real_osp_relpath = osp.relpath
def relpath_patched(path, start='.'):
    try:
        #print '**** relpath_patched OK ****',path
        return real_osp_relpath(path, start)
    except:
        #print '**** relpath_patched avoided exception ****',path
        return path
sphinx.application.os.path.relpath = relpath_patched
# or os.path.relpath = relpath_patched
#-----------------------------------------------------------#PATCH

Warning: Seems to override relpathin all Spyder app modules, not only in sphinx.application

[ccordoba12]

@RicardoNoguerales, thanks for investigating the source of the problem and giving us a solution. We'll come up with a fix for this in our next release.

[ccordoba12]

@dalthviz, please investigate this one to see if it's still a problem.

[dalthviz]

@ccordoba12 testing with Sphinx 1.7.5 seems like this is no more an issue:

[ccordoba12]

Thanks @dalthviz! I figured it was no longer a problem since no one else commented here.

Closing then.

[koneshgit]

I had Sphinx 1.8.2, and the same problem. Downgraded to 1.7.5 and now it is working