Pytest: DeprecationWarning for imp module triggered when run with -Werror

[johnthagen]

Testing with -Werror is convenient for CI testing so that warnings can be quickly identified and corrected rather than staying dormant.

With Python 3.5.1 if you execute with -Werror you receive a PendingDeprecationWarning because imp has been deprecated since Python 3.4.

Edit: As of Python 3.6 this is now a full DeprecationWarning.

The code causing the issue is:

https://github.com/pytest-dev/pytest/blob/ecc5c84c1ec77ad43f0fe0cb34a097bb855c1423/src/_pytest/assertion/rewrite.py#L8

To reproduce:

$ python -Werror -m pytest test.py

Traceback (most recent call last):
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\runpy.py", line 170, in _run_module_as_main
    "__main__", mod_spec)
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\pytest.py", line 16, in <module>
    import pytest
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\pytest.py", line 27, in <module>
    _preloadplugins() # to populate pytest.* namespace so help(pytest) works
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\_pytest\config.py", line 75, in _preloadplugins
    _preinit.append(get_config())
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\_pytest\config.py", line 84, in get_config
    pluginmanager.import_plugin(spec)
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\_pytest\config.py", line 384, in import_plugin
    __import__(importspec)
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\site-packages\_pytest\main.py", line 6, in <module>
    import os, sys, imp
  File "C:\Users\John Hagen\AppData\Local\Programs\Python\Python35\lib\imp.py", line 33, in <module>
    PendingDeprecationWarning, stacklevel=2)
PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

$ pip -V
pip 8.0.2
$ pip list
...
pytest (2.8.7)

This was tested on Windows 10 x64.

[RonnyPfannschmidt]

closing this one,
the remaining import of the imp module is in a place that handles old python versions

[rpkilby]

@RonnyPfannschmidt - is there an alternative way to view deprecations with pytest? The -Wall and Wonce flags aren't sufficient, since they only report the last line of the stack trace. Usually, you would use -Werror, however this causes the test runner to break before the test suite can even run.

As a side note, This now affects the assertion.rewrite module.

[rpkilby]

the remaining import of the imp module is in a place that handles old python versions

If the code is only handling old python versions, why is it executing in python 3.5?

[RonnyPfannschmidt]

@rpkilby mainly because the code is in place and the import didnt hurt anyone before

[nicoddemus]

If the code is only handling old python versions, why is it executing in python 3.5?

It is used by all Python versions actually. In order to get rid of it we have to map the functionality used by imp to importlib (its replacement) and create a compatibility layer because importlib was introduced in 3.1 only.

[rpkilby]

Thanks @RonnyPfannschmidt & @nicoddemus.
For anyone else who runs into this, there is pytest-warnings which seems to work well enough.

[nicoddemus]

Closing this because I don't think there's any explicit action left to be done here

[leezu]

Shouldn't this stay open until the compatibility layer mentioned by @nicoddemus is introduced?

With

This is pytest version 3.2.2, imported from /usr/lib64/python3.6/site-packages/pytest.py
setuptools registered plugins:
  pytest-timeout-1.2.0 at /usr/lib64/python3.6/site-packages/pytest_timeout.py
  pytest-pep8-1.0.6 at /usr/lib64/python3.6/site-packages/pytest_pep8.py
  pytest-cov-2.5.1 at /usr/lib64/python3.6/site-packages/pytest_cov/plugin.py

_pytest/assertion/rewrite.py uses the imp module, triggering the following warning on every run of one of my programs:

/usr/lib64/python3.6/site-packages/_pytest/assertion/rewrite.py:7: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import imp            

which I think is annoying enough to at least leave the report open ;)?

[nicoddemus]

@leezu thanks that's a good point.

[Zac-HD]

For anyone else who wants to run with warnings as errors: in Hypothesis we found that there are too many warnings in tools to make environment-wide -Werror (let alone PYTHONWARNINGS=error) practical.

Instead, we configure warnings in conftest.py:

import warnings
warnings.simplefilter('error')

[tnir]

With Python 3.6, we will get the following warning instead of PendingDeprecationWarning:

DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

Traceback (most recent call last):
  File "/usr/local/lib/python3.6/runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "/usr/local/lib/python3.6/runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "/usr/local/lib/python3.6/site-packages/pytest.py", line 9, in 
    from _pytest.config import (
  File "/usr/local/lib/python3.6/site-packages/_pytest/config.py", line 18, in 
    import _pytest.assertion
  File "/usr/local/lib/python3.6/site-packages/_pytest/assertion/__init__.py", line 9, in 
    from _pytest.assertion import rewrite
  File "/usr/local/lib/python3.6/site-packages/_pytest/assertion/rewrite.py", line 6, in 
    import imp
  File "/usr/local/lib/python3.6/imp.py", line 33, in 
    DeprecationWarning, stacklevel=2)
DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

(tested with Python 3.6.5 (python:3.6-alpine3.7 Docker image))

[johnthagen]

The most narrow way of suppressing this I've been able to find is:

$ python -Werror -Wignore:::_pytest.assertion.rewrite -m pytest test.py

This will suppress all warnings in _pytest.assertion.rewrite, which is arguably a little brittle if pytest refactors internally but at least it avoids having to globally disable all DeprecationWarnings and miss the opportunity to catch the use of deprecated items in user code.

Having this fixed in pytest would still be very ideal and would allow downstream users to use -Werror in a much more straightforward way.