Pytest: Warnings now need additional pytest warning suppression because of UnicodeWarning

[dwaynebailey]

• [X] Include a detailed description of the bug or suggestion
• [X] pip list of the virtual environment you are using
• [X] pytest and operating system versions
• [ ] Minimal example if possible

I'm a past user pf pytest-warning as we want to keep our code warning free, we also expose and manage all warning from our dependencies by turning all warnings into errors.

With pytest 3.1.0 we added the following to setup.cfg to suppress warnings from an upstream Django dependency.

once:::sortedm2m.fields

With the upgrade to pytest 3.1.1 we get the following snippet traceback:

INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/warnings.py", line 81, in catch_warnings_for_item
INTERNALERROR>     UnicodeWarning)
INTERNALERROR> UnicodeWarning: This warning /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/sortedm2m/fields.py:333: RemovedInDjango20Warning: Usage of field.rel has been deprecated. Use field.remote_field instead.
INTERNALERROR>   if self.rel.to != RECURSIVE_RELATIONSHIP_CONSTANT:
INTERNALERROR>  is broken as it's message is not a str instance(after all this is a stdlib problem workaround)

This is unhelpful in our context because:

1. We've already suppressed the actual warning, now we get another warning
2. It's in a dependency that we don't control
3. Using Unicode messages here is what Django does (correctly or incorrectly)

So to suppress this we now additionally add:

ignore:This warning:UnicodeWarning

So the question from me would be:

1. Shouldn't such a warning be the responsibility of Python and pytest is merely the reporter of such warnings.
2. If it wants to make such reports it shouldn't in ones that have already been supressed
3. This code is potentially masking the actual Warning you care about (haven't tested that though)

pip list

alabaster (0.7.9)
appdirs (1.4.3)
appnope (0.1.0)
aspy.yaml (0.2.1)
astroid (1.5.2, /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages)
autopep8 (1.2.1)
Babel (2.4.0)
backports.functools-lru-cache (1.3)
backports.shutil-get-terminal-size (1.0.0)
bleach (2.0.0)
cached-property (1.3.0)
caniusepython3 (4.0.0)
cffi (1.8.3)
click (6.7)
codacy-coverage (1.3.1)
codecov (2.0.5)
coid (0.1.1)
colorama (0.3.6)
configparser (3.5.0)
cov-core (1.15.0)
coverage (4.3.1)
coveralls (1.1)
cryptography (1.5.2)
cssmin (0.2.0)
decorator (4.0.10)
diff-match-patch (20121119)
dirsync (2.2.2)
distlib (0.2.4)
Django (1.10.7)
django-allauth (0.32.0)
django-appconf (1.0.2)
django-assets (0.12)
django-avatar (3.0.0)
django-babel (0.5.1)
django-bulk-update (2.1.0)
django-contact-form (1.4.1)
django-contrib-comments (1.7.3)
django-debug-toolbar (1.5)
django-extensions (1.7.9)
django-lint (2.0.4)
django-overextends (0.4.3)
django-redis (4.8.0)
django-rq (0.9.5)
django-sortedm2m (1.3.3)
django-statici18n (1.3.0)
django-transaction-hooks (0.2)
docopt (0.6.2)
docutils (0.13.1)
elasticsearch (1.9.0)
enum34 (1.1.6)
factory-boy (2.8.1)
fake-factory (0.7.2)
Faker (0.7.7)
flake8 (3.3.0)
flake8-quotes (0.9.0)
funcsigs (1.0.2)
functools32 (3.2.3.post2)
futures (3.0.5)
gitdb (0.6.4)
gitdb2 (2.0.0)
GitPython (2.1.1)
glue (0.13)
gnureadline (6.3.3)
greenlet (0.4.10)
html5lib (0.999999999)
humanize (0.5.1)
idna (2.1)
imagesize (0.7.1)
ipaddress (1.0.18)
ipython (5.3.0)
ipython-genutils (0.1.0)
isort (4.2.5)
Jinja2 (2.9.4)
jsonfield (2.0.1)
jsonschema (2.5.1)
kazoo (1.3.1)
lazy-object-proxy (1.2.2)
logilab-astng (0.24.3)
logilab-common (1.2.2)
lxml (3.7.2)
Markdown (2.6.8)
MarkupSafe (0.23)
mccabe (0.6.1)
mock (1.3.0)
natural (0.2.0)
netaddr (0.7.18)
nodeenv (0.13.6)
oauthlib (2.0.1)
ohmr (0.1.0)
olefile (0.44)
ordereddict (1.1)
packaging (16.8)
path.py (8.1.2)
pathlib2 (2.2.0)
pbr (1.10.0)
pexpect (4.2.1)
pickleshare (0.7.4)
Pillow (4.0.0)
pilo (0.5.2)
pip (9.0.1)
pluggy (0.3.1)
Pootle (2.8.0rc5, /Users/dwayne/dev/pootle)
pootle-fs-git (0.0.1, /Users/dwayne/dev/envs/pootle/src/pootle-fs-git)
pre-commit (0.7.6)
pre-commit-hooks (0.4.2)
prompt-toolkit (1.0.9)
ptyprocess (0.5.1)
pudb (2015.4.1)
py (1.4.32)
pyasn1 (0.1.9)
pycodestyle (2.3.1)
pycparser (2.14)
pydocstyle (2.0.0)
pyflakes (1.5.0)
Pygments (2.1.3)
pylint (1.7.1)
pylint-django (0.7.2)
pylint-plugin-utils (0.2.4)
pyOpenSSL (16.1.0)
pyparsing (2.2.0)
pytest (3.1.1)
pytest-catchlog (1.2.2)
pytest-cov (2.5.1)
pytest-django (3.1.2)
python-dateutil (2.6.0)
python-etcd (0.3.3)
python-Levenshtein (0.12.0)
python-openid (2.2.5)
pytz (2017.2)
PyYAML (3.11)
readme-renderer (17.2)
redis (2.10.5)
requests (2.12.4)
requests-oauthlib (0.7.0)
resolver (0.2.1)
rq (0.8.0)
rump (0.2.1)
scandir (1.5)
selector (0.10.1)
setuptools (35.0.2)
simplegeneric (0.8.1)
simplejson (3.8.2)
singledispatch (3.4.0.3)
six (1.10.0)
smmap (0.9.0)
smmap2 (2.0.1)
snowballstemmer (1.2.1)
Sphinx (1.5.6)
sphinx-me (0.3)
sphinx-rtd-theme (0.1.9)
sphinxcontrib-websupport (1.0.1)
sqlparse (0.2.1)
stemming (1.0.1)
termcolor (1.1.0)
tox (2.5.0)
traitlets (4.3.1)
translate-toolkit (2.1.0)
typing (3.6.1)
urllib3 (1.19.1)
urwid (1.3.1)
virtualenv (14.0.5)
wcwidth (0.1.7)
webassets (0.12.1)
webencodings (0.5)
wrapt (1.10.8)
wsgim-rip (0.1.0)
zope.interface (4.3.2)

pytest --version

(pootle)~/dev/pootle (requirements/pytest-3.1.1)$ pytest --version
This is pytest version 3.1.1, imported from /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/pytest.pyc
setuptools registered plugins:
  pytest-django-3.1.2 at /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/pytest_django/plugin.py
  pytest-cov-2.5.1 at /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/pytest_cov/plugin.py
  pytest-catchlog-1.2.2 at /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/pytest_catchlog.py
  django-assets-0.12 at /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/django_assets/pytest_plugin.py
  Pootle-2.8.0rc5 at /Users/dwayne/dev/pootle/pytest_pootle/plugin.py

(pootle)~/dev/pootle (requirements/pytest-3.1.1)$ uname -a
Darwin XXXXXXX 16.6.0 Darwin Kernel Version 16.6.0: Fri Apr 14 16:21:16 PDT 2017; root:xnu-3789.60.24~6/RELEASE_X86_64 x86_64

[RonnyPfannschmidt]

it certainly should't raise a internalerror

there is 2 details we need to fix

a) dont trigger a internalerror
b) dont trigger our warning unless its non-ascii unicode

@dwaynebailey can you provide a full traceback?

[nicoddemus]

That reminds me: we really should not introduce new warnings in patch releases. We should watch out for that in the future (was introduced in #2437).

[dwaynebailey]

@RonnyPfannschmidt the full Traceback as requested:

INTERNALERROR> Traceback (most recent call last):
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/main.py", line 105, in wrap_session
INTERNALERROR>     session.exitstatus = doit(config, session) or 0
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/main.py", line 141, in _main
INTERNALERROR>     config.hook.pytest_runtestloop(session=session)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 614, in execute
INTERNALERROR>     res = hook_impl.function(*args)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/main.py", line 164, in pytest_runtestloop
INTERNALERROR>     item.config.hook.pytest_runtest_protocol(item=item, nextitem=nextitem)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 745, in __call__
INTERNALERROR>     return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 339, in _hookexec
INTERNALERROR>     return self._inner_hookexec(hook, methods, kwargs)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 334, in <lambda>
INTERNALERROR>     _MultiCall(methods, kwargs, hook.spec_opts).execute()
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 613, in execute
INTERNALERROR>     return _wrapped_call(hook_impl.function(*args), self.execute)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/vendored_packages/pluggy.py", line 250, in _wrapped_call
INTERNALERROR>     wrap_controller.send(call_outcome)
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/warnings.py", line 87, in pytest_runtest_protocol
INTERNALERROR>     yield
INTERNALERROR>   File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/contextlib.py", line 24, in __exit__
INTERNALERROR>     self.gen.next()
INTERNALERROR>   File "/Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/_pytest/warnings.py", line 81, in catch_warnings_for_item
INTERNALERROR>     UnicodeWarning)
INTERNALERROR> UnicodeWarning: This warning /Users/dwayne/dev/envs/pootle/lib/python2.7/site-packages/sortedm2m/fields.py:333: RemovedInDjango20Warning: Usage of field.rel has been deprecated. Use field.remote_field instead.
INTERNALERROR>   if self.rel.to != RECURSIVE_RELATIONSHIP_CONSTANT:
INTERNALERROR>  is broken as it's message is not a str instance(after all this is a stdlib problem workaround)

[RonnyPfannschmidt]

@dwaynebailey did you configure anything that makes UnicodeWarnings errors?

[dwaynebailey]

@RonnyPfannschmidt yes I did. By default I turn all warning into errors, and then manually override those as we choose to ignore them. So we follow a more intentional approach to being warning free.

The relevant parts of setup.cfg are as follows:

[tool:pytest]
filterwarnings =
    # Error by default
    error
    once:::sortedm2m.fields
    ignore:This warning:UnicodeWarning

Make everything error by default. Ignore a bunch of warnings from sortedm2m. Then ignore the new pytest warning.