Sphinx: Cannot suppress autodoc: failed to import module

[peterjc]

Subject:

Problem

I am running Sphinx apidoc on a large codebase, with multiple soft dependencies which may not be installed (e.g. some are Python 2 only).

I would like to treat most warnings as errors via sphinx-build -W ... and ignore the ImportError (including ModuleNotFoundError under Python 3.6) warnings.

Currently http://www.sphinx-doc.org/en/stable/config.html#confval-suppress_warnings does not include any way to suppress this kind of warning.

Test case

Example Python file, sql.py

"""This is a silly module using an import."""

import MySQLdb

__version__ = "0.0.1"


class MySQLdbNULL(object):
    """Silly object."""

    def __str__(self):
        """Return the MySQLDb string for NULL."""
        return MySQLdb.NULL


if __name__ == "__main__":
    s = MySQLdbNULL()
    print(s)

Example RST file, sql.rst

$ more sql.rst 
Module contents
---------------

.. automodule:: sql
    :members:
    :undoc-members:
    :show-inheritance:

Minimal configuration file conf.py, set to suppress all documented warnings:

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
#
# Minimal configuration file, created by hand
extensions = ['sphinx.ext.autodoc']
source_suffix = '.rst'
master_doc = 'sql'
# Suppress all documented warnings:
suppress_warnings = ['app.add_node',
                     'app.add_directive',
                     'app.add_role',
                     'app.add_generic_role',
                     'app.add_source_parser',
                     'download.not_readable',
                     'image.not_readable',
                     'ref.term',
                     'ref.ref',
                     'ref.numref',
                     'ref.keyword',
                     'ref.option',
                     'ref.citation',
                     'ref.footnote',
                     'ref.doc',
                     'misc.highlighting_failure',
                     'toc.secnum',
                     'epub.unknown_project_files']

Test case output:

$ python3 -msphinx -W . html
Running Sphinx v1.7+
loading pickled environment... not yet created
building [mo]: targets for 0 po files that are out of date
building [html]: targets for 1 source files that are out of date
updating environment: 1 added, 0 changed, 0 removed
reading sources... [100%] sql                                                                                                                                     

Warning, treated as error:
/private/tmp/test_case/sql.rst:4: (WARNING/2) autodoc: failed to import module 'sql'; the following exception was raised:
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/Sphinx-1.7.dev20171023-py3.6.egg/sphinx/ext/autodoc/__init__.py", line 398, in import_object
    __import__(self.modname)
  File "/private/tmp/test_case/sql.py", line 3, in <module>
    import MySQLdb
ModuleNotFoundError: No module named 'MySQLdb'

[jenshnielsen]

Did you try to mock the imports or soft dependencies with http://www.sphinx-doc.org/en/stable/ext/autodoc.html#confval-autodoc_mock_imports

I find that the easiest way to resolve missing dependencies

[peterjc]

Thanks for the tip - I will explore autodoc_mock_imports = ['MySQLdb', ...] and report back.

[peterjc]

Yes, that seems to work for me.

Do you think the suppress_warnings documentation should reference using autodoc_mock_imports to for dealing with import warnings? I would have found that helpful.

[jenshnielsen]

Perhaps, I am just a Sphinx user like you so I don't have any special influence on this

[tk0miya]

+1 for improving suppress_warnings.

[tk0miya]

Note: The warning means failure of automodule directive. It means your document is broken. So I think suppressing it is not good way.

[tastyminerals]

Having the same issue with Python3.6, I successfully used sphinx with Python2.7 using the same dir structure and same conf.py. It does not import the modules that sphinx-build2 imports with the exact same sys.path.

So question is, does sphinx actually work for Python3.6?

[peterjc]

I'm using Sphinx successfully with Python 3.6, and autodoc_mock_imports = ['MySQLdb'] in my conf.py worked nicely for this specific dependency which was not available.

Your problem is likely due to a package which is not compatible with Python 3?

If I understood @tk0miya correctly, the import warnings mean the Sphinx output will be broken, so there is no point suppressing this warning. Instead if you cannot install/fix the non-imported module, you should use the mock feature.

[tastyminerals]

Ok, here is the project structure:

project/
    docs/conf.py
    project/main.py
    project/mods/mod1.py
                 mod2.py
    project/res/dic1.py
                blacklist.py

conf.py contains sys.path.insert(0, os.path.abspath(os.path.pardir)) at the top and with this setup I am getting import errors of mod1.py and mod2.py as well as dic1.py and blacklist.py. I actually tried all kind path variants and none worked.

[peterjc]

@tastyminerals I'm not sure what is wrong, but it is different from the goal of this issue which was about improving the configurability of sphinx to ignore certain kinds of warning.

I suggest you file a separate new issue with a complete test case.

Note that prior to Python 3.3 you would probably need to include project/mods/__init__.py and project/res/__init__.py (which can be empty files), so I am puzzled why things worked for you on Python 2.7

[tk0miya]

@peterjc Yes, this warning means Sphinx fails to import a python module and it makes the document broken. So, in this case, autodoc_mock_imports is much better to resolve the error.

I know suppress_warning is not resolution of error. It only hides warning messages.

[erinallard-infoscout]

Thanks so much for giving the autodoc_mock_imports suggestion. Worked for me!

[tk0miya]

I added autodoc.* type to suppress_warnings in #5357.
Thanks,

[Nighthyst]

Hello, I have a similar issue with a project I am currently working on. I create a small DummyPythonProject in my repositories to show the issue. It seems like whenever a module use an import, Sphinx is no longer able to document it and I have this "failed to import" error. However for me it's not an existing library that cause the error but one of the project's module : https://github.com/Nighthyst/DummyPythonProject

The solution proposed here with the autodoc_mock_imports list in conf.py doesn't solve the issue or I don't know how to use properly. I am using Python 3.6.9 with Anaconda. Any advice to solve this ?