Pytest: `monkeypatch.delattr(os, 'environ')` causes tests to be unrunnable

[asottile]

I was debugging the tests of procfs and noticed a very strange bit of code. I've boiled it down to a minimal reproduction.

The odd bit here is this code _works_ in pytest 3.1.3 (the newest version I could find that works).

sample code

import os

import pytest


@pytest.fixture(autouse=True)
def patch_os(monkeypatch):
    monkeypatch.delattr(os, 'environ')


def test(): pass

pytest 3.5.0 (current)

$ pip freeze --all
attrs==17.4.0
more-itertools==4.1.0
pip==9.0.3
pluggy==0.6.0
py==1.5.3
pytest==3.5.0
setuptools==39.0.1
six==1.11.0
wheel==0.30.0
$ py.test test.py
============================= test session starts ==============================
platform darwin -- Python 3.6.4, pytest-3.5.0, py-1.5.3, pluggy-0.6.0
rootdir: /private/tmp/test, inifile:
collected 1 item                                                               

test.py FE                                                               [100%]

==================================== ERRORS ====================================
__________________________ ERROR at teardown of test ___________________________

self = <CallInfo when='teardown' exception: module 'os' has no attribute 'environ'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x10e7c8730>
when = 'teardown'

    def __init__(self, func, when):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

venv/lib/python3.6/site-packages/_pytest/runner.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.6/site-packages/_pytest/runner.py:178: in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
venv/lib/python3.6/site-packages/pluggy/__init__.py:617: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
venv/lib/python3.6/site-packages/pluggy/__init__.py:222: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv/lib/python3.6/site-packages/pluggy/__init__.py:216: in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
venv/lib/python3.6/site-packages/_pytest/runner.py:122: in pytest_runtest_teardown
    _update_current_test_var(item, 'teardown')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

item = <Function 'test'>, when = 'teardown'

    def _update_current_test_var(item, when):
        """
        Update PYTEST_CURRENT_TEST to reflect the current item and stage.

        If ``when`` is None, delete PYTEST_CURRENT_TEST from the environment.
        """
        var_name = 'PYTEST_CURRENT_TEST'
        if when:
            value = '{0} ({1})'.format(item.nodeid, when)
            # don't allow null bytes on environment variables (see #2644, #2957)
            value = value.replace('\x00', '(null)')
>           os.environ[var_name] = value
E           AttributeError: module 'os' has no attribute 'environ'

venv/lib/python3.6/site-packages/_pytest/runner.py:138: AttributeError
=================================== FAILURES ===================================
_____________________________________ test _____________________________________

self = <CallInfo when='call' exception: module 'os' has no attribute 'environ'>
func = <function call_runtest_hook.<locals>.<lambda> at 0x10e652bf8>
when = 'call'

    def __init__(self, func, when):
        #: context of invocation: one of "setup", "call",
        #: "teardown", "memocollect"
        self.when = when
        self.start = time()
        try:
>           self.result = func()

venv/lib/python3.6/site-packages/_pytest/runner.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
venv/lib/python3.6/site-packages/_pytest/runner.py:178: in <lambda>
    return CallInfo(lambda: ihook(item=item, **kwds), when=when)
venv/lib/python3.6/site-packages/pluggy/__init__.py:617: in __call__
    return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
venv/lib/python3.6/site-packages/pluggy/__init__.py:222: in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
venv/lib/python3.6/site-packages/pluggy/__init__.py:216: in <lambda>
    firstresult=hook.spec_opts.get('firstresult'),
venv/lib/python3.6/site-packages/_pytest/runner.py:107: in pytest_runtest_call
    _update_current_test_var(item, 'call')
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 

item = <Function 'test'>, when = 'call'

    def _update_current_test_var(item, when):
        """
        Update PYTEST_CURRENT_TEST to reflect the current item and stage.

        If ``when`` is None, delete PYTEST_CURRENT_TEST from the environment.
        """
        var_name = 'PYTEST_CURRENT_TEST'
        if when:
            value = '{0} ({1})'.format(item.nodeid, when)
            # don't allow null bytes on environment variables (see #2644, #2957)
            value = value.replace('\x00', '(null)')
>           os.environ[var_name] = value
E           AttributeError: module 'os' has no attribute 'environ'

venv/lib/python3.6/site-packages/_pytest/runner.py:138: AttributeError
====================== 1 failed, 1 error in 0.16 seconds =======================

pytest 3.1.3 (older)

$ pip freeze --all
attrs==17.4.0
more-itertools==4.1.0
pip==9.0.3
pluggy==0.6.0
py==1.4.34
pytest==3.1.3
setuptools==39.0.1
six==1.11.0
wheel==0.30.0
$ py.test test.py
============================= test session starts ==============================
platform darwin -- Python 3.6.4, pytest-3.1.3, py-1.4.34, pluggy-0.4.0
rootdir: /private/tmp/test, inifile:
collected 1 item s

test.py .

=========================== 1 passed in 0.01 seconds ===========================

failure for procfs

The failure mode for procfs was much worse (probably due to deleting many more things from the os module) to the point where it was nearly impossible to know what was happening:

____________________ ERROR at teardown of test_missing_file ____________________
self = <CallInfo when='teardown' exception: 'module' object has no attribute 'environ'>
func = <function <lambda> at 0x7ffa4a45d9b0>, when = 'teardown'
>   ???
.tox/py27/lib/python2.7/site-packages/_pytest/runner.py:192: 
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
.tox/py27/lib/python2.7/site-packages/_pytest/runner.py:178: in <lambda>
    ???
.tox/py27/lib/python2.7/site-packages/pluggy/__init__.py:617: in __call__
    ???
.tox/py27/lib/python2.7/site-packages/pluggy/__init__.py:222: in _hookexec
    ???
.tox/py27/lib/python2.7/site-packages/pluggy/__init__.py:216: in <lambda>
    ???
.tox/py27/lib/python2.7/site-packages/_pytest/runner.py:122: in pytest_runtest_teardown
    ???
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
item = <Function 'test_missing_file'>, when = 'teardown'
>   ???
E   AttributeError: 'module' object has no attribute 'environ'
.tox/py27/lib/python2.7/site-packages/_pytest/runner.py:138: AttributeError
___________________ ERROR at setup of test_first_process_id ____________________
self = <pytest_cov.plugin.CovPlugin object at 0x7ffa4ade5350>
item = <Function 'test_first_process_id'>
>   ???
E   AttributeError: 'module' object has no attribute 'getpid'
.tox/py27/lib/python2.7/site-packages/pytest_cov/plugin.py:282: AttributeError

wat

I realize this example is super-contrived and probably not something that _should_ be supported. Honestly I was more surprised that it worked previously.

Here's a patch that "fixes" this, though I imagine any other module could be _broken_ in the same way:

--- runner.py   2018-03-28 09:53:48.000000000 -0700
+++ venv/lib/python3.6/site-packages/_pytest/runner.py  2018-03-28 09:54:14.000000000 -0700
@@ -2,8 +2,8 @@
 from __future__ import absolute_import, division, print_function

 import bdb
-import os
 import sys
+from os import environ
 from time import time

 import py
@@ -135,9 +135,9 @@
         value = '{0} ({1})'.format(item.nodeid, when)
         # don't allow null bytes on environment variables (see #2644, #2957)
         value = value.replace('\x00', '(null)')
-        os.environ[var_name] = value
+        environ[var_name] = value
     else:
-        os.environ.pop(var_name)
+        environ.pop(var_name)


 def pytest_report_teststatus(report):

Command I used to "bisect" the problem (I didn't actually bisect, it was fast enough to just try every version):

pip install "pytest<$(pip freeze | grep pytest | cut -d= -f3)" && py.test test.py

followup

• should something like this be supported? (I would wager: no)
• can the error message be improved? (probably not?)

[pytestbot]

GitMate.io thinks possibly related issues are https://github.com/pytest-dev/pytest/issues/1156 (os.getlogin() fails in tests), https://github.com/pytest-dev/pytest/issues/528 (Test causes segfault), https://github.com/pytest-dev/pytest/issues/2172 (Capturing output causes test to fail.), https://github.com/pytest-dev/pytest/issues/365 (py.test creates an environment that causes colorama to fail), and https://github.com/pytest-dev/pytest/issues/190 (Naming test methods "setup" causes different behavior).

[RonnyPfannschmidt]

its a kind of known issue we plan to eventually do imports from the stdlib in a way that protects pytest from 3rd party monkeypatching, as all kinds of libs do all kinds of fun issues

i dont have the other issue readyly at hand

[The-Compiler]

See #3290

[nicoddemus]

Should we close this as a duplicate?

[asottile]

yeah I think so -- this is a good concrete example, but that ticket definitely covers the expected behaviour here :D

[nicoddemus]

Thanks @asottile!