Pytest: Possible bug with the 3.10.0 release

[fzarifian]

Hello,

Something went wrong with the 3.10.0 release

   ============================= test session starts ==============================
    platform linux2 -- Python 2.7.12, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
    rootdir: /builds/ansible-roles/json-inventory/molecule/default, inifile:
    plugins: testinfra-1.16.0
    collecting ... 
collected 3 items                                                              

Traceback (most recent call last):
  File "/usr/local/bin/py.test", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config/__init__.py", line 76, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/main.py", line 218, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/main.py", line 211, in wrap_session
    session=session, exitstatus=session.exitstatus
  File "/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/terminal.py", line 627, in pytest_sessionfinish
    outcome.get_result()
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/stepwise.py", line 102, in pytest_sessionfinish
    self.config.cache.set("cache/stepwise", [])
AttributeError: 'Config' object has no attribute 'cache'

The same nine hours ago with the 3.9.3 release :

    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.12, pytest-3.9.3, py-1.7.0, pluggy-0.8.0
    rootdir: /builds/ansible-roles/json-inventory/molecule/default, inifile:
    plugins: testinfra-1.16.0

collecting 0 items                                                             
collecting 3 items                                                             
collected 3 items                                                              

    tests/test_default.py ...                                                [100%]

    =============================== warnings summary ===============================
    /usr/lib/python2.7/dist-packages/cryptography/x509/__init__.py:32
      /usr/lib/python2.7/dist-packages/cryptography/x509/__init__.py:32: PendingDeprecationWarning: CRLExtensionOID has been renamed to CRLEntryExtensionOID
        from cryptography.x509.oid import (

    -- Docs: https://docs.pytest.org/en/latest/warnings.html
    ==================== 3 passed, 1 warnings in 38.20 seconds =====================

If someone have an idea ? ;)

[fzarifian]

PR #4297 with PR #4147 ?

[RonnyPfannschmidt]

did you by chance disable the cache plugin?

[fzarifian]

NOT OK, tested with a pytest.ini file located at the rootdir containing :

[pytest]
addopts = -p no:cacheprovider

And I get this :

--> Executing Testinfra tests found in /builds/ansible-roles/postgresql_server/molecule/default/tests/...
    ============================= test session starts ==============================
    platform linux2 -- Python 2.7.12, pytest-3.10.0, py-1.7.0, pluggy-0.8.0
    rootdir: /builds/ansible-roles/postgresql_server/molecule/default, inifile: pytest.ini
    plugins: testinfra-1.16.0
    collecting ... 
collected 2 items                                                              

Traceback (most recent call last):
  File "/usr/local/bin/py.test", line 11, in <module>
    sys.exit(main())
  File "/usr/local/lib/python2.7/dist-packages/_pytest/config/__init__.py", line 76, in main
    return config.hook.pytest_cmdline_main(config=config)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 208, in _multicall
    return outcome.get_result()
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/main.py", line 218, in pytest_cmdline_main
    return wrap_session(config, _main)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/main.py", line 211, in wrap_session
    session=session, exitstatus=session.exitstatus
  File "/usr/local/lib/python2.7/dist-packages/pluggy/hooks.py", line 284, in __call__
    return self._hookexec(self, self.get_hookimpls(), kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 67, in _hookexec
    return self._inner_hookexec(hook, methods, kwargs)
  File "/usr/local/lib/python2.7/dist-packages/pluggy/manager.py", line 61, in <lambda>
    firstresult=hook.spec.opts.get("firstresult") if hook.spec else False,
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 203, in _multicall
    gen.send(outcome)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/terminal.py", line 627, in pytest_sessionfinish
    outcome.get_result()
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 81, in get_result
    _reraise(*ex)  # noqa
  File "/usr/local/lib/python2.7/dist-packages/pluggy/callers.py", line 187, in _multicall
    res = hook_impl.function(*args)
  File "/usr/local/lib/python2.7/dist-packages/_pytest/stepwise.py", line 102, in pytest_sessionfinish
    self.config.cache.set("cache/stepwise", [])
AttributeError: 'Config' object has no attribute 'cache'
    tests/test_default.py ..                                                 [100%]
An error occurred during the test sequence action: 'verify'. Cleaning up.

For me, disabling the cache plugin (like i found on the documentation) doesn't work...

Another information : the runner is a docker ubuntu16.04 built like this :

FROM ubuntu:16.04

RUN echo "# Installing apt repositories..."
    && echo "## Ansible APT Packages" \
    && apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 7BB9C367 \
    && add-apt-repository "deb [arch=amd64] http://ppa.launchpad.net/ansible/ansible/ubuntu xenial main" \
    && apt-get update -yq \
    && DEBIAN_FRONTEND=noninteractive apt-get -y install ansible python-pip

RUN echo "## Install python requirements" \
    && pip install pytest molecule docker

[RonnyPfannschmidt]

this is a "regression" - the stepwise plugin must be disabled as well in as it depending on the cache plugin"

[jsf9k]

FWIW, I'm seeing this too with version 3.10.0 when running molecule test against some of my Ansible roles. See, for instance, dhs-ncats/ansible-role-htop.

[fzarifian]

@jsf9k as mentionned by @RonnyPfannschmidt, you should unregister pytest module with a file in your repository like "molecule/default/pytest.ini" like this one.

[pytest]
addopts = -p no:cacheprovider -p no:stepwise

[fzarifian]

Or use like I do in your requirements.txt :

pytest==3.9.3

[amustafa]

I'm experiencing the same issue.

Though, @fzarifian my test already had -p no:cacheprovider. By removing it, I was also able to remove the issue. I didn't need to add the -p no:stepwise part.

That being said, we ultimately left it in and opted to just set the pytest version like in your last comment.

[fzarifian]

@amustafa : on my runner cacheprovider and stepwise was enabled (by default), and i have the issue.
It should be related to a python 2.7 bug, could you confirm you use python3 ?

[fzarifian]

(set the version is easier as it should be defined in the test runner image and not in all repositories you have to test)

[amustafa]

@fzarifian Yes, the test failed when I was testing against python3.

[jsf9k]

Thanks for your help @fzarifian and @RonnyPfannschmidt! I added the pytest.ini file and now molecule test completes successfully.

[jsf9k]

I confirmed that with version 3.10.1 I no longer have the issue.

[nicoddemus]

Thanks @jsf9k for the follow up.