Sphinx: Sphinx 1.7.1 in Pycharm task seems to not parse parameters properly

[schtoom]

Subject: Sphinx 1.7.1 Pycharm task seems to not parse parameters properly - I get different results when running the Make.bat file versus Pycharm.

Problem

When I run my Sphinx task in Pycharm, I get the following error:
sphinx_runner.py: error: cannot find source directory (C:\Program Files\JetBrains\PyCharm Community Edition 2017.3.3\helpers\rest_runners\sphinx_runner.py)

When I perform the same action with the make.bat file, it works flawlessly.

Both methods share the same .RST file.

Procedure to reproduce the problem

Run the basic Sphinx build task from Pycharm.

I did some troubleshooting and traced the issue into the parsing and assigning of arguments in main in cmdline.py. When I run make html from the command line, I get the following value for args, the parsed arguments that main processes:
Namespace(builder='html', color='auto', confdir=None, define=[], doctreedir='_build\\doctrees', filenames=[], force_all=False, freshenv=False, htmldefine=[], jobs=1, nitpicky=False, noconfig=False, outputdir='_build\\html', pdb=False, quiet=False, really_quiet=False, sourcedir='.', tags=[], traceback=False, verbosity=0, warnfile=None, warningiserror=False)

When I run the Sphinx task, this is the command line that it generates in my console:
C:\Python36\python.exe "C:\Program Files\JetBrains\PyCharm Community Edition 2017.3.3\helpers\rest_runners\sphinx_runner.py" -b html C:\WORKSPACE\testtools\software_test_platform C:\WORKSPACE\testtools\software_test_platform\_build\html

However, this is what args parses out to be:
Namespace(builder='html', color='auto', confdir=None, define=[], doctreedir=None, filenames=['C:\\WORKSPACE\\testtools\\software_test_platform\\_build\\html'], force_all=False, freshenv=False, htmldefine=[], jobs=1, nitpicky=False, noconfig=False, outputdir='C:\\WORKSPACE\\testtools\\software_test_platform', pdb=False, quiet=False, really_quiet=False, sourcedir='C:\\Program Files\\JetBrains\\PyCharm Community Edition 2017.3.3\\helpers\\rest_runners\\sphinx_runner.py', tags=[], traceback=False, verbosity=0, warnfile=None, warningiserror=False)

Basically, it appears that the important directory values are not being assigned correctly. filenames has been assigned what should be the outputdir, outputdir has been assigned the sourcedir, and sourcedir has been assigned the location of sphinx_runner.py, the script that kicks off the whole process.

Error logs / results

This is the configuration of my Pycharm task:

I've assigned the correct values for Input (the location of my .RST file) and Output (where I want my HTML files generated) based on the instructions here:
https://www.jetbrains.com/help/pycharm/run-debug-configuration-sphinx-task.html

This is the error I receive when running the Sphinx task:
sphinx_runner.py: error: cannot find source directory (C:\Program Files\JetBrains\PyCharm Community Edition 2017.3.3\helpers\rest_runners\sphinx_runner.py)

Expected results

I expect make.bat and Pycharm to produce identical results.

Reproducible project / your project

My .RST and conf.py files are included in the attached ZIP file. This is sort of a Hello World project for me with Sphinx, so I don't believe there is anything crazy in my configuration.
basic_sample.zip

Environment info

• OS: Windows 10
• Python version: 3.6
• Sphinx version: 1.7.1
• Pycharm Community Edition 2017.3

[tk0miya]

I don't know what is the sphinx_runner.py doing.
But I guess the cause might be the change of interface of Sphinx's main function internally.
So please report this to JetBrains team.

[schtoom]

Thanks a ton! I wasn't sure on which side to consult first (here or JetBrains), and you guys seemed pretty quick and responsive. I'll fire this issue up with them in the morning.

[takahyon]

I get the same problem in InteliJ Ultimate and Pycharm. @schtoom Did u have any reply from Jetbrain? If u have pls share it plz...... thx

[vaskokj]

I've got this same problem. PyCharm 2017.1.3. Sphinx 1.7.2

[schtoom]

I have a solution, although not one given to me by JetBrains. I found it on the world wide interwebs and wish I could give proper attribution.

In sphinx_runner.py, replace this line:

cmdline.main(sys.argv)

with something like this:

try:
    cmdline.main(sys.argv[1:])
except:
        print('Failed at calling cmdline.main(sys.argv[1:]) in sphinx_runner.py.')

Obviously only the cmdline change is needed, but I'm a sucker for wrapping things up when I can. Hope that helps!

[tk0miya]

Ah, I understand the reason. It caused by the change of interface of our main function on 1.7...

If so, it should be updated like this:

import sphinx

if sphinx.__version_info__ > (1, 7):
    import sphinx.cmd.build import main
    main(sys.argv[1:])
else:
    import sphinx.cmdline import main
    main(sys.argv)

Did anybody report this to JetBrains? Please let them know about this.

[ericnchen]

I found this because I also had the same problems and the solution written here works for me as well, using sphinx 1.7.2.

I checked PyCharm's issue tracker and it has been posted there, but has not been fixed. It would be helpful if more registered users could upvote the bug :)

https://youtrack.jetbrains.com/issue/PY-28647

[cesarcruz]

Solutions proveded above doesn't works for me in PyCharm 2018, changing file pycharm/helpers/rest_runners/sphinx_runner.py to the following works:

__author__ = 'catherine'

if __name__ == "__main__":
    try:
        from sphinx.cmd import build
    except:
        raise NameError("Cannot find sphinx in selected interpreter.")

    import sys

    build.main(sys.argv[1:])

Reference https://gist.github.com/badge/f9fdd31687903e76b2281b05f8207d02

[tk0miya]

It seems the upstream issue is not resolved yet. But I'm closing this now because this is not a bug of Sphinx itself.
Thanks,

[jat255]

Just to close the book on this, it looks like the upstream issue was fixed in the 2018.2 release of PyCharm