Pytest: Pytest 3.7.1 isn't keeping class variables set by autouse function

[s1113950]

Link to sample failure (it's a public project so hopefully you can see this?)
https://circleci.com/gh/IntelAI/mlt/1857

pytest env:

(.venv) root@2ca5ec29de7d:/usr/share/mlt# pytest --version
This is pytest version 3.7.1, imported from /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pytest.pyc
setuptools registered plugins:
  pytest-xdist-1.22.5 at /usr/share/mlt/.venv/local/lib/python2.7/site-packages/xdist/looponfail.py
  pytest-xdist-1.22.5 at /usr/share/mlt/.venv/local/lib/python2.7/site-packages/xdist/plugin.py
  pytest-forked-0.2 at /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pytest_forked/__init__.pyc
  pytest-cov-2.5.1 at /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pytest_cov/plugin.py

The vars get set as they used to, but then in between the variables being set by this class here and when the first test is called, the variables disappear:

class CommandTester(object):
    @classmethod
    @pytest.fixture(scope='class', autouse=True)
    def setup(self):
        self.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')
        self.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')

class TestConfig(CommandTester):
    def test_config_list(self):
        """
        Tests listing configs in an init directory
        """
        self.init()



md5-72168c5bed007bb4058823e61559ef44



def init(self, template='hello-world', template_repo=basedir(),
             enable_sync=False):
        self._set_new_mlt_project_vars(template)
>       init_options = ['mlt', 'init', '--registry={}'.format(self.registry),
                        '--template-repo={}'.format(template_repo),
                        '--namespace={}'.format(self.namespace),
                        '--template={}'.format(template), self.app_name]
E       AttributeError: 'TestConfig' object has no attribute 'registry'



md5-8a1c9bf71b970b7aa6e80a23a5ddc4d5



======================================================================== test session starts ===============================================================[277/816]
platform linux2 -- Python 2.7.12, pytest-3.7.1, py-1.5.4, pluggy-0.7.1 -- /usr/share/mlt/.venv/bin/python2
cachedir: .pytest_cache
rootdir: /usr/share/mlt, inifile: tox.ini
plugins: xdist-1.22.5, forked-0.2, cov-2.5.1
collected 1 item

tests/e2e/test_config_updates.py::TestConfig::test_update_config
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /usr/share/mlt/tests/test_utils/e2e_commands.py(46)setup()
-> self.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')
(Pdb) n
> /usr/share/mlt/tests/test_utils/e2e_commands.py(51)setup()
-> self.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')
(Pdb) n
--Return--
> /usr/share/mlt/tests/test_utils/e2e_commands.py(51)setup()->None
-> self.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(800)call_fixture_func()
-> return res
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(800)call_fixture_func()->None
-> return res
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(942)pytest_fixture_setup()
-> fixturedef.cached_result = (result, my_cache_key, None)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(943)pytest_fixture_setup()
-> return result
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(943)pytest_fixture_setup()->None
-> return result
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(181)_multicall()                                                                 [241/816]
-> if res is not None:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(163)_multicall()
-> for hook_impl in reversed(hook_impls):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(188)_multicall()
-> if firstresult:  # first result hooks return a single value
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(189)_multicall()
-> outcome = _Result(results[0] if results else None, excinfo)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(194)_multicall()
-> for gen in reversed(teardowns):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(195)_multicall()
-> try:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
StopIteration: None
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(198)_multicall()
-> except StopIteration:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(199)_multicall()
-> pass
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(194)_multicall()
-> for gen in reversed(teardowns):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(201)_multicall()
-> return outcome.get_result()
(Pdb) n                                                                                                                                                     [205/816]
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(201)_multicall()->None
-> return outcome.get_result()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/manager.py(61)<lambda>()->None
-> firstresult=hook.spec_opts.get('firstresult'),
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/manager.py(67)_hookexec()->None
-> return self._inner_hookexec(hook, methods, kwargs)
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/hooks.py(258)__call__()->None
-> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(896)execute()->None
-> return hook.pytest_fixture_setup(fixturedef=self, request=request)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(606)_compute_fixture_value()
-> self.session._setupstate.addfinalizer(
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(607)_compute_fixture_value()
-> functools.partial(fixturedef.finish, request=subrequest),
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(608)_compute_fixture_value()
-> subrequest.node,
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(608)_compute_fixture_value()->None
-> subrequest.node,
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(533)_get_active_fixturedef()
-> self._fixture_defs[argname] = fixturedef
(Pdb) n                                                                                                                                                     [169/816]
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(534)_get_active_fixturedef()
-> return fixturedef
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(534)_get_active_fixturedef()-><Fixture...ig::()' >
-> return fixturedef
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(509)getfixturevalue()->None
-> return self._get_active_fixturedef(argname).cached_result[0]
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(463)_fillfixtures()
-> for argname in fixturenames:
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(463)_fillfixtures()->None
-> for argname in fixturenames:
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/fixtures.py(294)fillfixtures()->None
-> request._fillfixtures()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/python.py(1435)setup()->None
-> fixtures.fillfixtures(self)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(367)prepare()
-> for col in needed_collectors[len(self.stack) :]:
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(367)prepare()->None
-> for col in needed_collectors[len(self.stack) :]:
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(104)pytest_runtest_setup()->None
-> item.session._setupstate.prepare(item)
(Pdb) n                                                                                                                                                     [132/816]
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(181)_multicall()
-> if res is not None:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(163)_multicall()
-> for hook_impl in reversed(hook_impls):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(164)_multicall()
-> try:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(165)_multicall()
-> args = [caller_kwargs[argname] for argname in hook_impl.argnames]
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(165)_multicall()
-> args = [caller_kwargs[argname] for argname in hook_impl.argnames]
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(172)_multicall()
-> if hook_impl.hookwrapper:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(180)_multicall()
-> res = hook_impl.function(*args)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(181)_multicall()
-> if res is not None:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(163)_multicall()
-> for hook_impl in reversed(hook_impls):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(188)_multicall()
-> if firstresult:  # first result hooks return a single value
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(191)_multicall()
-> outcome = _Result(results, excinfo)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(194)_multicall()
-> for gen in reversed(teardowns):
(Pdb) n                                                                                                                                                      [96/816]
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(195)_multicall()
-> try:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
StopIteration: None
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(198)_multicall()
-> except StopIteration:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(199)_multicall()
-> pass
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(194)_multicall()
-> for gen in reversed(teardowns):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(195)_multicall()
-> try:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
StopIteration: None
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(196)_multicall()
-> gen.send(outcome)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(198)_multicall()
-> except StopIteration:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(199)_multicall()
-> pass
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(194)_multicall()
-> for gen in reversed(teardowns):
(Pdb) n                                                                                                                                                      [58/816]
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(201)_multicall()
-> return outcome.get_result()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/callers.py(201)_multicall()->[]
-> return outcome.get_result()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/manager.py(61)<lambda>()->[]
-> firstresult=hook.spec_opts.get('firstresult'),
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/manager.py(67)_hookexec()->[]
-> return self._inner_hookexec(hook, methods, kwargs)
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/pluggy/hooks.py(258)__call__()->[]
-> return self._hookexec(self, self._nonwrappers + self._wrappers, kwargs)
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(183)<lambda>()->[]
-> lambda: ihook(item=item, **kwds),
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(210)__init__()
-> self.stop = time()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(210)__init__()->None
-> self.stop = time()
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(185)call_runtest_hook()-><CallInf...sult: []>
-> treat_keyboard_interrupt_as_exception=item.config.getvalue("usepdb"),
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(162)call_and_report()
-> hook = item.ihook
(Pdb) n                                                                                                                                                      [21/816]
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(163)call_and_report()
-> report = hook.pytest_runtest_makereport(item=item, call=call)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(164)call_and_report()
-> if log:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(165)call_and_report()
-> hook.pytest_runtest_logreport(report=report)
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(166)call_and_report()
-> if check_interactive_exception(call, report):
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(168)call_and_report()
-> return report
(Pdb) n
--Return--
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(168)call_and_report()-><TestRep...'passed'>
-> return report
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(76)runtestprotocol()
-> reports = [rep]
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(77)runtestprotocol()
-> if rep.passed:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(78)runtestprotocol()
-> if item.config.option.setupshow:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(80)runtestprotocol()
-> if not item.config.option.setuponly:
(Pdb) n
> /usr/share/mlt/.venv/local/lib/python2.7/site-packages/_pytest/runner.py(81)runtestprotocol()
-> reports.append(call_and_report(item, "call", log))
(Pdb) n
>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>> PDB set_trace (IO-capturing turned off) >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
> /usr/share/mlt/tests/test_utils/e2e_commands.py(126)init()
-> self._set_new_mlt_project_vars(template)
(Pdb) n
> /usr/share/mlt/tests/test_utils/e2e_commands.py(127)init()
-> init_options = ['mlt', 'init', '--registry={}'.format(self.registry),
(Pdb) self.registry
*** AttributeError: 'TestConfig' object has no attribute 'registry'



md5-d594ace1ad361fff1e5047be3d3a5f5a



(.venv) root@2ca5ec29de7d:/usr/share/mlt# pip list
Package        Version           Location
-------------- ----------------- ------------------------------------------------
apipkg         1.5
argh           0.26.2
atomicwrites   1.1.5
attrs          18.1.0
conditional    1.2
configparser   3.5.0
coverage       4.5.1
docopt         0.6.2
enum34         1.1.6
execnet        1.5.0
flake8         3.5.0
funcsigs       1.0.2
functools32    3.2.3.post2
jsonschema     2.6.0
mccabe         0.6.1
mlt            0.2.1+10.gfbc10bf /usr/share/mlt/.venv/lib/python2.7/site-packages
mock           2.0.0
more-itertools 4.3.0
packaging      17.1
pathlib2       2.3.2
pathtools      0.1.2
pbr            4.2.0
pip            18.0
pluggy         0.7.1
progressbar2   3.38.0
py             1.5.4
pycodestyle    2.3.1
pyflakes       1.6.0
pyparsing      2.2.0
pytest         3.7.1
pytest-cov     2.5.1
pytest-forked  0.2
pytest-xdist   1.22.5
python-utils   2.3.0
PyYAML         3.13
scandir        1.8
setuptools     40.0.0
six            1.11.0
tabulate       0.8.2
termcolor      1.1.0
tox            3.1.2
virtualenv     16.0.0
watchdog       0.8.3
wheel          0.31.1

This used to work with the prior version of pytest (we hadn't pegged pytest as we should have and tests all broke after the 8/2 release).

EDIT
Pegging to these versions made things work again:
https://github.com/IntelAI/mlt/commit/2d5ec3251da295051b0862fa0db3c6e71dbb22ec

[nicoddemus]

Thanks @s1113950 for the report. 👍

I bet this is caused by the same problem that is causing #3774.

Can you confirm that you don't have this problem in 3.6.4? And what about 3.7.0?

[ashahba]

We don't have this issue with 3.6.4 but we haven't tested 3.7.0. However since we didn't peg versions previously, and our tests passed yesterday, I'm assuming 3.7.0 is fine and only 3.7.1 is broken.

[s1113950]

@ashahba I'll test things locally really quick

[s1113950]

Can confirm 3.6.4 and 3.7.0 are fine, but 3.7.1 fails @nicoddemus

[nicoddemus]

Thanks @s1113950!

[blueyed]

I've bisected a problem myself to c6b11b9f - which is about missing methods in a decorated fixture:

def decorated_drf_client(f):
    @wraps(f)
    def wrapper(*args, **kwargs):
        drf_client = f(*args, **kwargs)

        def assert_status(url_or_response, expected_status,
                          expected_data=None, data=None, method=None,
                          format=None):
            __tracebackhide__ = True
            …

        return drf_client
    return wrapper


@decorated_drf_client
@pytest.fixture
def drf_client(db):
    return CustomAPIClient(HTTP_ACCEPT='application/json; version=1.0')

The assert_status method is not available in the fixture anymore due to c6b11b9f.
(i.e. AttributeError: 'CustomAPIClient' object has no attribute 'assert_400')

I've put it here a comment since it sounds very similar.

@s1113950
Can you check if c6b11b9f is causing your issue, too? (but very likely I would say)

Mine appears to be the same as https://github.com/pytest-dev/pytest/issues/3774 at least apparently.

[nicoddemus]

@blueyed #3780 should fix your issue, could you try it out just to make sure we are not missing anything?

@s1113950 #3780 should also fix your issue if you change the order of the decorators:

class CommandTester(object):
    @pytest.fixture(scope='class', autouse=True)
    @classmethod    
    def setup(self):
        self.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')
        self.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')

As discussed in #3780 we plan to eventually raise an error if fixtures are called directly, and applying @pytest.fixture last seems to be the only way to move this forward for class-scoped fixtures which live in a class.

[s1113950]

I tried using version pytest-3.7.2.dev16+g4d8903fd (should be master at the time I pulled it just now) and got this error @nicoddemus :

tests/e2e/test_update_template.py:22: in <module>
    from test_utils.e2e_commands import CommandTester
tests/test_utils/e2e_commands.py:40: in <module>
    class CommandTester(object):
tests/test_utils/e2e_commands.py:43: in CommandTester
    @classmethod
.venv3/lib/python3.6/site-packages/_pytest/fixtures.py:1007: in __call__
    function = wrap_function_to_warning_if_called_directly(function, self)
.venv3/lib/python3.6/site-packages/_pytest/fixtures.py:960: in wrap_function_to_warning_if_called_directly
    msg = FIXTURE_FUNCTION_CALL.format(name=fixture_marker.name or function.__name__)
E   AttributeError: 'classmethod' object has no attribute '__name__'

😞
I changed the fixture to be this:

    @pytest.fixture(scope='class', autouse=True)
    @classmethod
    def setup(self):
        # just in case tests fail, want a clean namespace always
        self.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')

        # ANY NEW TFJOBS NEED TO HAVE THEIR TEMPLATE NAMES LISTED HERE
        # TFJob terminates pods after completion so can't check old pods
        # for status of job completion
        self.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')

We're also running pytest-xdist if that matters at all?

[nicoddemus]

Ahh good to know. We need another fix, fortunately it is simple. Thanks for reporting!

[s1113950]

Our project is open source if you want to repro for yourself 😄 : https://github.com/IntelAI/mlt
To reproduce the error I did a pip3 install . of master of pytest into the .venv3 of mlt, and then ran e2e tests like so:

`EXTRA_ARGS=`$MLT_REGISTRY_AUTH_COMMAND` make test-e2e-no-docker`

(see developer_guide.md for a better explanation)
and had these env vars set:

MLT_REGISTRY=gcr.io/XXXXXX
MLT_REGISTRY_AUTH_COMMAND=gcloud container clusters get-credentials XXX --zone us-west1-a --project XXXXXX

It should repro itself without those env vars set, but just in case.

Otherwise I can try a different fix that you guys make, or try and submit one myself when I have time (probably after next Wednesday when sprint restarts)

[nicoddemus]

@s1113950 thanks, but fortunately this is trivial to reproduce in isolation. 👍

Unfortunately after giving this a go I think there's no easy fix for this, please read https://github.com/pytest-dev/pytest/pull/3781/comment#issuecomment-411897957. The bottom line is that fixtures decorated with @classmethod worked mostly by accident and were never meant to be supported.

Fortunately there's an easy workaround:

--- foo1.py     2018-08-09 18:12:46.211165800 -0300
+++ foo2.py     2018-08-09 18:12:58.771387100 -0300
@@ -1,6 +1,6 @@
 class CommandTester(object):
-    @classmethod
     @pytest.fixture(scope='class', autouse=True)
-    def setup(cls):
+    def setup(self):
+        cls = type(self)
         cls.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')
         cls.tfjob_templates = ('tf-dist-mnist', 'tf-distributed')

[s1113950]

👍 thanks for the workaround!

[blueyed]

@nicoddemus
Unfortunately my issue is not fixed on master.
Should I create a new one for it?
(btw: I also have to install/upgrade pluggy-0.7.1 manually - is pluggy involved here / causing this maybe?)

[nicoddemus]

@blueyed no, it is all related to that now we wrap the fixture function itself in order to generate a warning in case it is called directly (#3661). Really a shame that such a seemingly simple change has caused so much problems.

Before I mentioned:

As discussed in #3780 we plan to eventually raise an error if fixtures are called directly, and applying @pytest.fixture last seems to be the only way to move this forward for class-scoped fixtures which live in a class.

But it seems this might not be actually possible in the end, see https://github.com/pytest-dev/pytest/pull/3781#issuecomment-411897957.

Can you see if the workaround in https://github.com/pytest-dev/pytest/issues/3778#issuecomment-411899446 works for you? It should work for any pytest version.

[blueyed]

@nicoddemus
Not sure how to apply the workaround to my use case, where I am wrapping the pytest fixture? (https://github.com/pytest-dev/pytest/issues/3778#issuecomment-411775050)

[nicoddemus]

Oh sorry @blueyed I missed your comment! 😅

Does changing the order of the decorators still gives an error?

@pytest.fixture
@decorated_drf_client
def drf_client(db):
    return CustomAPIClient(HTTP_ACCEPT='application/json; version=1.0')

[s1113950]

@nicoddemus the workaround didn't work, getting original error again:

self = <test_deploy_flow.TestDeployFlow object at 0x1106ce2d0>, template = 'hello-world', template_repo = '/Users/robertso/Nervana/mlt', enable_sync = True

    def init(self, template='hello-world', template_repo=basedir(),
             enable_sync=False):
        self._set_new_mlt_project_vars(template)
>       init_options = ['mlt', 'init', '--registry={}'.format(self.registry),
                        '--template-repo={}'.format(template_repo),
                        '--namespace={}'.format(self.namespace),
                        '--template={}'.format(template), self.app_name]
E       AttributeError: 'TestDeployFlow' object has no attribute 'registry'

where it isn't called as expected.

@@ -38,9 +38,15 @@ from project import basedir


 class CommandTester(object):
-    @classmethod
+
     @pytest.fixture(scope='class', autouse=True)
     def setup(self):
+        # workaround for issue described here:
+        # https://github.com/pytest-dev/pytest/issues/3778
+             # issuecomment-411899446
+        # this will enable us to use pytest 3.7.1
+        cls = type(self)
+
         # just in case tests fail, want a clean namespace always
         self.registry = os.getenv('MLT_REGISTRY', 'localhost:5000')

my patch attempt

[nicoddemus]

@s1113950 sorry, I wasn't clear enough.

You need to set the attributes to the class explicitly instead of relying on the function receiving a class as first argument because of the @classmethod decorator.

Try this:

class CommandTester(object):
    @pytest.fixture(scope='class', autouse=True)
    def setup(self):
        type(self).registry = os.getenv('MLT_REGISTRY', 'localhost:5000')
        type(self).tfjob_templates = ('tf-dist-mnist', 'tf-distributed')

[s1113950]

That worked! Thanks so much

[nicoddemus]

Great, thanks for checking!

Let's leave this open for now until we see if @blueyed has the same problem. 👍

[blueyed]

@nicoddemus
I've tried switching decorators before, but only after ef8ec01e3979aff992d63540a0e36a957e133c76 it works now.
Thanks!

[xNinjaKittyx]

What exactly will be the intended behavior at the end of all this?

Will the original behavior be supported eventually? Or will there be a new way?

Just trying to think of the migration path I should take.

[nicoddemus]

IMHO we should just go ahead and say this is not supported because:

1. It worked in the past only by implementation accident rather than design.
2. It will probably get into the way of future refactorings we plan (change the fixture implementation into a proper class).
3. I did try already to fix the regression, but I was not successful (#3781).

@blueyed @asottile @RonnyPfannschmidt thoughts?

[asottile]

IMHO we should just go ahead and say this is not supported

I'm inclined to agree, @classmethod seems like an odd edgecase that'll be difficult to support properly (especially if it's only used to perform side-effecty global mutation)

[nicoddemus]

Well let's close this as "won't fix because it worked by accident, with an acceptable workaround" and also because we just couldn't make it work in #3781. If someone can come up with a reasonable patch in the future we can revisit this. 👍

[boweeb]

IMHO we should just go ahead and say this is not supported

I'm inclined to agree, @classmethod seems like an odd edgecase that'll be difficult to support properly

How does this relate to the documentation?
https://docs.pytest.org/en/stable/xunit_setup.html?highlight=class#class-level-setup-teardown

At face value it seems contradictory. Should the docs be updated or am I missing something?