Pytest: Feature Request: Add handling of SystemError to importorskip

[further-reading]

With Python 3.8 some modules will now generate SystemError: bad call flags when importing due to a change to the C-API. More details: https://bugs.python.org/msg348804 I would like to suggest that the importerskip function be updated to handle SystemErrors too, so tests can be easily skipped when module imports fail for this reason.

Example:

import pytest

class LeveldbStorageTest:
    pytest.importorskip('leveldb')

    def test_fake():
        import leveldb
        assert True == False

Expectation - tests in class are skipped, as with an ImportError.
Actual - pytest.importorskip('leveldb') line generates a SystemError.

piplist:

Package        Version
-------------- -------
atomicwrites   1.3.0  
attrs          19.3.0 
filelock       3.0.12 
leveldb        0.194  
more-itertools 7.2.0  
packaging      19.2   
pip            19.3.1 
pluggy         0.13.0 
py             1.8.0  
pyparsing      2.4.2  
pytest         5.2.2  
setuptools     41.4.0 
six            1.12.0 
toml           0.10.0 
tox            3.14.0 
virtualenv     16.7.6 
wcwidth        0.1.7  
wheel          0.33.6 

OS details

Distributor ID: Ubuntu
Description:    Ubuntu 18.04.3 LTS
Release:    18.04
Codename:   bionic

Python 3.8.0
pytest 5.2.2

[RonnyPfannschmidt]

i'm -1 on pretending a severely broken instead of missing module is a good skip reason

implementing this would basically mean that suddenly ci on python3.8 would do the funky lie- and sneakishly painlessly skip when in fact the world is on fire

on top of that code in production sharing the setup wouldnt skip something, it more likely would utterly break flat out

[Zac-HD]

Also -1 on this change for the same reason - it's annoying to deal with, but SystemError indicates that the module is broken rather than not present. You can still

try:
    import leveldb
except (ImportError, SystemError):
    pytest.skip()

[further-reading]

Thanks for clarifying the purpose of importerskip and suggesting a better approach Zac. I'll close this issue.

[RonnyPfannschmidt]

@further-reading for sanity i would propose to have 2 expect clauses, and using pytest.xfail if the module is broken simply to have at least better visibility