Sphinx: Make html fails if torch (PyTorch DL framework) is imported

[JuanFMontesinos]

Describe the bug
I realized sphinxs fails to make html if torch is imported in any file using autodoc api.

Project Structure
Tree structure:
asd
+-asd
| +-asd
| | + __init__.py
| | + dummy.py
| +-docs
| +- sphinx-quickstart tree...

__init__.py is empty
dummy.py contains:

import torch
def dummy():
    """
    Dummy function

    :param str filename: str, Path to file to be evaluate
    :return: tuple(time, fps) in the mentioned format
    """

    pass
print('Successful')

conf.py generated by sphinx-quickstart

# Configuration file for the Sphinx documentation builder.
#
# This file only contains a selection of the most common options. For a full
# list see the documentation:
# http://www.sphinx-doc.org/en/master/config

# -- Path setup --------------------------------------------------------------

# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
sys.path.insert(0, os.path.abspath('.'))
sys.path.insert(0, os.path.abspath('..'))
sys.path.insert(0, os.path.abspath('../..'))



# -- Project information -----------------------------------------------------

project = 'dummy'
copyright = '2019, dummy'
author = 'dummy'

# The full version, including alpha/beta/rc tags
release = 'dummy'


# -- General configuration ---------------------------------------------------

# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = ['sphinx.ext.autodoc','sphinx.ext.viewcode',
]

# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']

# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path.
exclude_patterns = []


# -- Options for HTML output -------------------------------------------------

# The theme to use for HTML and HTML Help pages.  See the documentation for
# a list of builtin themes.
#
html_theme = 'sphinx_rtd_theme'

# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']

To Reproduce
Steps used to build (run from docs folder):

rm -r build
rm source/asd.rst
rm source/modules.rst
sphinx-apidoc -o source/ ../asd
make html

Running with and without importing torch leads to different results:
Without torch:

Creating file source/asd.rst.
Creating file source/modules.rst.
Running Sphinx v2.1.2
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 3 source files that are out of date
updating environment: 3 added, 0 changed, 0 removed
Successfulurces... [ 33%] asd                                                                                                                                                             
reading sources... [100%] modules                                                                                                                                                         
looking for now-outdated files... none found
pickling environment... done
checking consistency... /home/jfm/asd/docs/source/modules.rst: WARNING: document isn't included in any toctree
done
preparing documents... done
writing output... [100%] modules                                                                                                                                                          
generating indices... genindex py-modindex
highlighting module code... [100%] asd.dummy                                                                                                                                              
writing additional pages... search/home/jfm/.local/lib/python3.6/site-packages/sphinx_rtd_theme/search.html:20: RemovedInSphinx30Warning: To modify script_files in the theme is deprecated. Please insert a <script> tag directly in your theme instead.
  {{ super() }}

copying static files... done
copying extra files... done
dumping search index in English (code: en) ... done
dumping object inventory... done
build succeeded, 1 warning.

The HTML pages are in build/html.
Error: no DISPLAY environment variable specified

With torch:

Creating file source/asd.rst.
Creating file source/modules.rst.
Running Sphinx v2.1.2
making output directory... done
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 3 source files that are out of date
updating environment: 3 added, 0 changed, 0 removed
terminate called after throwing an instance of 'std::runtime_error'                                                                                                                       
  what():  expected ) but found 'ident' here:
quantized::fake_quantize_per_tensor_affine_forward(Tensor X, float scale, int zero_point, int num_bits = 8, int quant_delay = 0, int iter = 0) -> Tensor
                                                                                                            ~~~ <--- HERE

Aborted (core dumped)
Makefile:20: recipe for target 'html' failed
make: *** [html] Error 134
Error: no DISPLAY environment variable specified

Obviously dummy.py works, namely, torch perfectly works in the system

Environment info

• OS: Ubuntu 18
• Python version: 3.6.7
• Sphinx version: 2.1.2
• PyTorch version: 1.1.0 (GPU version, official binaries)

[KadKla]

Dear Juan,

I had the same problem and found a workaround using mocking:
http://www.sphinx-doc.org/en/master/usage/extensions/autodoc.html#confval-autodoc_mock_imports

After adding:

autodoc_mock_imports = ["torch"]

"make html" compiles without errors.

I hope that helps!

[JuanFMontesinos]

I will have a try but that would be awesome. Thanks ^^

[tk0miya]

Is dummy.py runnable standalone? Please check python dummy.py.

[JuanFMontesinos]

Hi, it is runnable yep. It's python3 not python.

[tk0miya]

With python:3.7 docker image, both running python3 dummy.py and building HTMLs are succeeded. I need to information to reproduce.

Note: It seems there are some reports similar to this error:
https://github.com/pytorch/pytorch/issues/20804

[JuanFMontesinos]

Hi,
As you mentioned it is probably the same error, however I have no clue about how to provide additional information as I'm not very familiarized with cuda and sphinx code.
As it looks it is a cuda-related problem:
nvcc: NVIDIA (R) Cuda compiler driver Copyright (c) 2005-2018 NVIDIA Corporation Built on Sat_Aug_25_21:08:01_CDT_2018 Cuda compilation tools, release 10.0, V10.0.130 NVIDIA-SMI 410.48
That's the version I'm using.

[tk0miya]

It seems the fix is not released yet. So please try with HEAD of pytorch.
I think there are nothing to do from sphinx side.

[tk0miya]

I'm closing this now because this is not a bug of Sphinx.
Thanks,