Sphinx: doctests has no access to the module globals (c.f. running with python -m doctest)

[skirpichev]

Lets take this simple example:

$ cat docs/conf.py 
import a

extensions = ['sphinx.ext.doctest', 'sphinx.ext.autodoc']
$ cat docs/con
conf.py       contents.rst  
$ cat docs/contents.rst 
.. automodule:: a
    :members:
$ cat a/__init__.py 
def foo():
    """bar

    >>> foo()
    1
    """
    return 1

Testing with the plain doctest module works:

$ python3 -m doctest -v a/__init__.py 
Trying:
    foo()
Expecting:
    1
ok
1 items had no tests:
    __init__
1 items passed all tests:
   1 tests in __init__.foo
1 tests in 2 items.
1 passed and 0 failed.
Test passed.

py.test --doctest-modules works in the same way, it follows same execution context convention.

Yet, apparently doctest code has no access to the module globals (e.g. foo):

$ python3 -m sphinx -b doctest docs/ builds/
Running Sphinx v1.8.4
loading pickled environment... done
building [mo]: targets for 0 po files that are out of date
building [doctest]: targets for 1 source files that are out of date
updating environment: 0 added, 0 changed, 0 removed
looking for now-outdated files... none found
running tests...

Document: contents
------------------
**********************************************************************
File "../a/__init__.py", line ?, in default
Failed example:
    foo()
Exception raised:
    Traceback (most recent call last):
      File "/usr/lib/python3.7/doctest.py", line 1329, in __run
        compileflags, 1), test.globs)
      File "<doctest default[0]>", line 1, in <module>
        foo()
    NameError: name 'foo' is not defined
**********************************************************************
1 items had failures:
   1 of   1 in default
1 tests in 1 items.
0 passed and 1 failed.
***Test Failed*** 1 failures.

Doctest summary
===============
    1 test
    1 failure in tests
    0 failures in setup code
    0 failures in cleanup code
build finished with problems.

Sounds like a bug for me.

[tk0miya]

I don't think this is a bug. In this case, automodule reads the python module and translates it to following reST code:

.. py:module:: a

.. py:function::

   a

       >>> foo()
       1

It only describes about the module. After execution of autodoc, real python module is not related with the reST code. Therefore, doctest module does not import objects under a module.

[skirpichev]

Well, a lot of code must be adapted to use doctest extension. People tend
to rely on the doctest module behaviour. I will appreciate any workarounds.
On Jul 21, 2019 1:48 PM, "Takeshi KOMIYA" [e-mail] wrote:

I don't think this is a bug. In this case, automodule reads the python
module and translates it to following reST code:

.. py:module:: a

.. py:function::

a

   >>> foo()
   1

It only describes about the module. After execution of autodoc, real
python module is not related with the reST code. Therefore, doctest module
does not import objects under a module.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHub
https://github.com/sphinx-doc/sphinx/issues/6590?email_source=notifications&email_token=AAQOKGB5SS25WUWRMMTVFNTQAQ5IZA5CNFSM4IE6HMX2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD2OAY4A#issuecomment-513543280,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAQOKGCQFOVVDPHT4LEK7DDQAQ5IZANCNFSM4IE6HMXQ
.

[tk0miya]

Please add "import" statement to docstring or testsetup directive to load the target function. It is explicit way to do that.
Thanks,

[skirpichev]

Please add "import" statement

Please don't use this extensions would be more sane suggestion.

Well, @tk0miya, I do understand why you closed this issue. What I don't understand - it's classification. Definitely, this is not a question, it's a bug. It an important bug, as most likely it
will affect almost every project, which could try to use this extension.

I hope, you will reconsider your decision and at least reopen this issue as a documentation bug (this way, maybe, people will not waste time on this extension).

[tk0miya]

Okay, I can agree this is a lack of documentation.

[skirpichev]

Please add "import" statement to docstring or testsetup directive to load the target function.

@tk0miya, maybe there is some way to do this (i.e. import the _current_ module) once, in the doctest_global_setup? (I'm still hoping to find a workaround...)

[ankostis]

In order to see the effort for a possible fix, can some of the authors describe the differences between the sphinx.ext.DocTestBuilder against the standard doctest module?

[edit: repying to myself]

• Doctest & pytest scan & run examples directly from the modules where they are defined.
• Sphinx finds them if auto-documenters have assembled them into a DOM, outside of their module.

Q: is it possible to go back from an auto-documenter DOM --> defining module?

[tk0miya]

Sphinx's doctest is not related with autodoc directly. It can use alone. That is a large difference between other tools.

Q: is it possible to go back from an auto-documenter DOM --> defining module?

It might be able to do, but some hack will be needed.

[ankostis]

Q: is it possible to go back from an auto-documenter DOM --> defining module?

Standard sphinx-extension sphinx.ext.viewcode seems to do that, and it does'nt look hackish.
https://github.com/sphinx-doc/sphinx/blob/713bbf5cafa3fc5e143ced59dafe56f4b802ef80/sphinx/ext/viewcode.py#L86-L95

[ankostis]

So beyond any test-case adaptations, the above pasted code and the lines below would be all that is needed to fix this issue:

module = importlib.import_module(modname)
modglobs = vars(module).copy()
# Use `modglobs` as doctest's globs

And not to break TCs relying on existing behavior, a new configval would be required.

Would this scheme work, @tk0miya?

[tk0miya]

Standard sphinx-extension sphinx.ext.viewcode seems to do that, and it does'nt look hackish.

Thank you for letting me know. Indeed, the code will work!

And not to break TCs relying on existing behavior, a new configval would be required.

Would this scheme work, @tk0miya?

TBH, I'm not good at doctest extension. So I need to learn how it work and how implemented. It means you know about it deeper than me. I still don't understand what you're saying. But let's go forward. Please send a PR. I'll take a look at it.