pytest segfaults but python -m pytest does not
tl;dr - read the title of this issue.
Related information
- Related issue: #2564
- Is triggered by: diana-hep/pyhf#385
- Related documentation: https://docs.pytest.org/en/latest/pythonpath.html#invoking-pytest-versus-python-m-pytest
This occurs as part of a travis-CI job. This is very hard to provide a MWE, so I will instead provide enough background information.
Environment
OS
travis@travis-job-2326826f-1987-469e-80ed-956889b8a731:~/build/diana-hep/pyhf$ uname -a
Linux travis-job-2326826f-1987-469e-80ed-956889b8a731 4.4.0-101-generic #124~14.04.1-Ubuntu SMP Fri Nov 10 19:05:36 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
pip list
Package Version Location
---------------------------------- ----------- ---------------------------------
absl-py 0.6.1
adal 1.2.0
alabaster 0.7.12
ansiwrap 0.8.3
asn1crypto 0.24.0
aspy.yaml 1.1.1
astor 0.7.1
atomicwrites 1.2.1
attrs 18.2.0
awkward 0.7.1
azure-common 1.1.16
azure-datalake-store 0.0.40
azure-nspkg 3.0.2
azure-storage-blob 1.4.0
azure-storage-common 1.4.0
azure-storage-nspkg 3.1.0
Babel 2.6.0
backports-abc 0.5
backports.functools-lru-cache 1.5
backports.shutil-get-terminal-size 1.0.0
backports.shutil-which 3.5.2
backports.weakref 1.0.post1
bleach 3.1.0
boto3 1.9.77
botocore 1.12.77
bumpversion 0.5.3
cachetools 3.0.0
certifi 2018.11.29
cffi 1.11.5
cfgv 1.4.0
chardet 3.0.4
Click 7.0
colorama 0.4.1
configparser 3.5.0
contextlib2 0.5.5
coverage 4.5.2
cryptography 2.4.2
cycler 0.10.0
decorator 4.3.0
defusedxml 0.5.0
docutils 0.14
entrypoints 0.3
enum34 1.1.6
funcsigs 1.0.2
functools32 3.2.3.post2
future 0.17.1
futures 3.2.0
gast 0.2.1.post0
gitdb2 2.0.5
GitPython 2.1.11
graphviz 0.10.1
grpcio 1.17.1
h5py 2.9.0
identify 1.1.8
idna 2.6
imagesize 1.1.0
iminuit 1.3.3
importlib-metadata 0.8
importlib-resources 1.0.2
ipaddress 1.0.22
ipykernel 4.10.0
ipython 5.8.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
Jinja2 2.10
jmespath 0.9.3
jsonpatch 1.23
jsonpointer 2.0
jsonschema 3.0.0a4
jupyter 1.0.0
jupyter-client 5.2.4
jupyter-console 5.2.0
jupyter-core 4.4.0
Keras-Applications 1.0.6
Keras-Preprocessing 1.0.5
kiwisolver 1.0.1
latexcodec 1.0.5
m2r 0.2.1
Markdown 3.0.1
MarkupSafe 1.1.0
matplotlib 2.2.3
mistune 0.8.4
mock 2.0.0
more-itertools 5.0.0
mxnet 1.3.1
nbconvert 5.4.0
nbdime 1.0.4
nbformat 4.4.0
nbsphinx 0.4.1
nodeenv 1.3.3
nose 1.3.7
notebook 5.7.4
numpy 1.14.6
oset 0.1.3
packaging 18.0
pandas 0.23.4
pandocfilters 1.4.2
papermill 0.16.2
pathlib2 2.3.3
pbr 5.1.1
pexpect 4.6.0
pickleshare 0.7.5
pip 18.1
pkginfo 1.5.0.1
pluggy 0.8.1
pockets 0.7.2
pre-commit 1.14.1
prometheus-client 0.5.0
prompt-toolkit 1.0.15
protobuf 3.6.1
ptyprocess 0.6.0
py 1.7.0
py-cpuinfo 4.0.0
pybtex 0.22.0
pybtex-docutils 0.2.1
pycparser 2.19
pyflakes 2.0.0
pygal 2.4.0
pygaljs 1.0.1
Pygments 2.3.1
pyhf 0.0.15 /home/travis/build/diana-hep/pyhf
PyJWT 1.7.1
pyparsing 2.3.0
pyrsistent 0.14.9
pytest 3.10.1
pytest-benchmark 3.2.0
pytest-console-scripts 0.1.7
pytest-cov 2.6.1
pytest-mock 1.10.0
pytest-runner 4.2
python-coveralls 2.9.1
python-dateutil 2.7.5
pytz 2018.9
PyYAML 3.13
pyzmq 17.1.2
qtconsole 4.4.3
readme-renderer 24.0
requests 2.18.4
requests-toolbelt 0.8.0
s3transfer 0.1.13
scandir 1.9.0
scipy 1.2.0
Send2Trash 1.5.0
setuptools 40.6.3
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.12.0
smmap2 2.0.5
snowballstemmer 1.2.1
Sphinx 1.8.3
sphinx-issues 1.2.0
sphinx-rtd-theme 0.4.2
sphinxcontrib-bibtex 0.4.2
sphinxcontrib-napoleon 0.7
sphinxcontrib-websupport 1.1.0
statistics 1.0.3.5
subprocess32 3.5.3
tensorboard 1.12.2
tensorflow 1.12.0
tensorflow-probability 0.5.0
termcolor 1.1.0
terminado 0.8.1
testpath 0.4.2
textwrap3 0.9.1
toml 0.10.0
torch 1.0.0
tornado 5.1.1
tqdm 4.29.0
traitlets 4.3.2
twine 1.12.1
typing 3.6.6
uproot 3.3.2
uproot-methods 0.3.3
urllib3 1.22
virtualenv 16.2.0
wcwidth 0.1.7
webencodings 0.5.1
Werkzeug 0.14.1
wheel 0.32.3
widgetsnbextension 3.4.2
zipp 0.3.3
python
travis@travis-job-2326826f-1987-469e-80ed-956889b8a731:~/build/diana-hep/pyhf$ which python
/opt/pyenv/shims/python
travis@travis-job-2326826f-1987-469e-80ed-956889b8a731:~/build/diana-hep/pyhf$ python --version
Python 2.7.14
python path
>>> import sys
>>> sys.path
['', '/home/travis/virtualenv/python2.7.14/lib/python27.zip', '/home/travis/virtualenv/python2.7.14/lib/python2.7', '/home/travis/virtualenv/python2.7.14/lib/python2.7/plat-linux2', '/home/travis/virtualenv/python2.7.14/lib/python2.7/lib-tk', '/home/travis/virtualenv/python2.7.14/lib/python2.7/lib-old', '/home/travis/virtualenv/python2.7.14/lib/python2.7/lib-dynload', '/opt/python/2.7.14/lib/python2.7', '/opt/python/2.7.14/lib/python2.7/plat-linux2', '/opt/python/2.7.14/lib/python2.7/lib-tk', '/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages', '/home/travis/build/diana-hep/pyhf']
Background
pyhf is a statistics tool that uses multiple backends for performing tensor calculations. Some of these backends include tensorflow, mxnet, pytorch, etc... We initially observed a segfault:
Segmentation fault: 11
Stack trace returned 10 entries:
[bt] (0) /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1e945a) [0x7f4fec78545a]
[bt] (1) /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f891f6) [0x7f4fef5251f6]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0) [0x7f5061593cb0]
[bt] (3) /lib/x86_64-linux-gnu/libpthread.so.0(sem_wait+0) [0x7f5061934640]
[bt] (4) /opt/python/2.7.14/lib/libpython2.7.so.1.0(PyThread_acquire_lock+0x68) [0x7f5061c92578]
[bt] (5) /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(+0x610c0) [0x7f505e3140c0]
[bt] (6) /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(+0x61749) [0x7f505e314749]
[bt] (7) /lib/x86_64-linux-gnu/libssl.so.1.0.0(SSL_CTX_free+0x6e) [0x7f505bd1f8de]
[bt] (8) /home/travis/virtualenv/python2.7.14/lib/python2.7/lib-dynload/_ssl.so(+0xa06b) [0x7f505bf5106b]
[bt] (9) /opt/python/2.7.14/lib/libpython2.7.so.1.0(+0xba17a) [0x7f5061bfe17a]
which we initially believed to be due to changes in pathlib vs pathlib2 in ionelmc/pytest-benchmark#8469a5.
However, upon doing some debug session with travis, I've stumbled upon the fact that pytest invocation causes a segfault while python -m pytest does not, even including updating the PYTHONPATH to take into account the current directory as suggested by pytest documentation. In summary, the following commands produce the segfault:
pytest tests/test_notebooks.pyPYTHONPATH=. pytest tests/test_notebooks.pypython $(which pytest) tests/test_notebooks.py
The following commands do not produce a segfault:
python -m pytest tests/test_notebooks.pyPYTHONPATH=. python -m pytest tests/test_notebooks.py/usr/bin/env python -m pytest tests/test_notebooks.py/home/travis/virtualenv/python2.7.14/bin/python -m pytest tests/test_notebooks.py
Here's a quick checklist in what to include:
- [x] Include a detailed description of the bug or suggestion
- [x]
pip listof the virtual environment you are using - [x] pytest and operating system versions
- [ ] Minimal example if possible
kratsg
👍1
All 24 comments
The vonly difference I could imagine there is the setuptools entry point machinery somehow having a negative side effect. This one sounds like it's going to be pretty difficult to reproduce!
asottile
on 11 Jan 2019
👍2
@asottile - this is my suspicion. I've got access to the travis debug session in 30 minute blocks if people have anything they would like me to test or print information on. I had initially thought, because of the differences, that this had to do with the $PYTHONPATH, but given that PYTHONPATH=. pytest segfaults as well, I was hoping someone could provide alternative executions to narrow down what the differences in setuptools machinery might be.
As a fun-ish fact, this only affects python 2.7, as python 3.6 seems to be fine.
kratsg
on 11 Jan 2019
just to confirm, this should also trigger the problem? python $(which pytest) tests/test_notebooks.py
asottile
on 11 Jan 2019
Confirmed.
travis@travis-job-f0e8da58-1f5a-4958-8426-802246755422:~/build/diana-hep/pyhf$ which pytest
/home/travis/virtualenv/python2.7.14/bin/pytest
@asottile Hi, I'm another developer on the project that @kratsg opened this issue up on. As he mentioned this is happening being noticed only inside a Travis CI build environment and something interesting that I've just noticed is that if I enter into the debug mode interactive session and step through the stages of our tests using the bash functions Travis CI provides I can get the reported segfault with
travis_run_before_install
travis_run_install
travis_run_script
where in this case travis_run_script is pytest tests/test_notebooks.py, but if I then run
travis_run_script
again then the tests will pass. So this seems extra strange in that it is not reproducible with the same command.
I don't think that this information makes it any easier to figure out what is happening, but I thought I'd mention it just in case.
Sorry, a relevant edit: After checking more, I noticed that the same error in the same place is being seen on two commits at the moment in our project and @kratsg was reporting on one and I the other. There is one difference between the two commits which is that in the one @kratsg is referencing we require torch>=1.0.0. The resulting difference in output of pip list is the following:
diff pip_list_matthewfeickert_torch_lt_100.out pip_list_kratsg_torch_gt_100.out
57d56
< identify 1.1.8
110c109
< pre-commit 1.14.2
---
> pre-commit 1.14.1
129c128
< pytest-benchmark 3.2.1
---
> pytest-benchmark 3.2.0
169c168
< torch 0.4.1
---
> torch 1.0.0
180c179
< webencodings 0.5.1
---
> webencodings 0.5.1
and we get the same segfault, but what I describe about being able to not get the segfault on my second pass of travis_run_script does not hold if I login to the debug session for @kratsg's commit. There pytest tests/test_notebooks.py always segfaults.
So to be explicit, my pip list is
Package Version Location
---------------------------------- ----------- ---------------------------------
absl-py 0.6.1
adal 1.2.0
alabaster 0.7.12
ansiwrap 0.8.3
asn1crypto 0.24.0
aspy.yaml 1.1.1
astor 0.7.1
atomicwrites 1.2.1
attrs 18.2.0
awkward 0.7.1
azure-common 1.1.16
azure-datalake-store 0.0.40
azure-nspkg 3.0.2
azure-storage-blob 1.4.0
azure-storage-common 1.4.0
azure-storage-nspkg 3.1.0
Babel 2.6.0
backports-abc 0.5
backports.functools-lru-cache 1.5
backports.shutil-get-terminal-size 1.0.0
backports.shutil-which 3.5.2
backports.weakref 1.0.post1
bleach 3.1.0
boto3 1.9.77
botocore 1.12.77
bumpversion 0.5.3
cachetools 3.0.0
certifi 2018.11.29
cffi 1.11.5
cfgv 1.4.0
chardet 3.0.4
Click 7.0
colorama 0.4.1
configparser 3.5.0
contextlib2 0.5.5
coverage 4.5.2
cryptography 2.4.2
cycler 0.10.0
decorator 4.3.0
defusedxml 0.5.0
docutils 0.14
entrypoints 0.3
enum34 1.1.6
funcsigs 1.0.2
functools32 3.2.3.post2
future 0.17.1
futures 3.2.0
gast 0.2.1.post0
gitdb2 2.0.5
GitPython 2.1.11
graphviz 0.10.1
grpcio 1.17.1
h5py 2.9.0
identify 1.1.8
identify 1.1.8
idna 2.6
imagesize 1.1.0
iminuit 1.3.3
importlib-metadata 0.8
importlib-resources 1.0.2
ipaddress 1.0.22
ipykernel 4.10.0
ipython 5.8.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
Jinja2 2.10
jmespath 0.9.3
jsonpatch 1.23
jsonpointer 2.0
jsonschema 3.0.0a4
jupyter 1.0.0
jupyter-client 5.2.4
jupyter-console 5.2.0
jupyter-core 4.4.0
Keras-Applications 1.0.6
Keras-Preprocessing 1.0.5
kiwisolver 1.0.1
latexcodec 1.0.5
m2r 0.2.1
Markdown 3.0.1
MarkupSafe 1.1.0
matplotlib 2.2.3
mistune 0.8.4
mock 2.0.0
more-itertools 5.0.0
mxnet 1.3.1
nbconvert 5.4.0
nbdime 1.0.4
nbformat 4.4.0
nbsphinx 0.4.1
nodeenv 1.3.3
nose 1.3.7
notebook 5.7.4
numpy 1.14.6
oset 0.1.3
packaging 18.0
pandas 0.23.4
pandocfilters 1.4.2
papermill 0.16.2
pathlib2 2.3.3
pbr 5.1.1
pexpect 4.6.0
pickleshare 0.7.5
pip 18.1
pkginfo 1.5.0.1
pluggy 0.8.1
pockets 0.7.2
pre-commit 1.14.2
prometheus-client 0.5.0
prompt-toolkit 1.0.15
protobuf 3.6.1
ptyprocess 0.6.0
py 1.7.0
py-cpuinfo 4.0.0
pybtex 0.22.0
pybtex-docutils 0.2.1
pycparser 2.19
pyflakes 2.0.0
pygal 2.4.0
pygaljs 1.0.1
Pygments 2.3.1
pyhf 0.0.15 /home/travis/build/diana-hep/pyhf
PyJWT 1.7.1
pyparsing 2.3.0
pyrsistent 0.14.9
pytest 3.10.1
pytest-benchmark 3.2.1
pytest-console-scripts 0.1.7
pytest-cov 2.6.1
pytest-mock 1.10.0
pytest-runner 4.2
python-coveralls 2.9.1
python-dateutil 2.7.5
pytz 2018.9
PyYAML 3.13
pyzmq 17.1.2
qtconsole 4.4.3
readme-renderer 24.0
requests 2.18.4
requests-toolbelt 0.8.0
s3transfer 0.1.13
scandir 1.9.0
scipy 1.2.0
Send2Trash 1.5.0
setuptools 40.6.3
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.12.0
smmap2 2.0.5
snowballstemmer 1.2.1
Sphinx 1.8.3
sphinx-issues 1.2.0
sphinx-rtd-theme 0.4.2
sphinxcontrib-bibtex 0.4.2
sphinxcontrib-napoleon 0.7
sphinxcontrib-websupport 1.1.0
statistics 1.0.3.5
subprocess32 3.5.3
tensorboard 1.12.2
tensorflow 1.12.0
tensorflow-probability 0.5.0
termcolor 1.1.0
terminado 0.8.1
testpath 0.4.2
textwrap3 0.9.1
toml 0.10.0
torch 0.4.1
tornado 5.1.1
tqdm 4.29.0
traitlets 4.3.2
twine 1.12.1
typing 3.6.6
uproot 3.3.2
uproot-methods 0.3.3
urllib3 1.22
virtualenv 16.2.0
wcwidth 0.1.7
webencodings 0.5.1
Werkzeug 0.14.1
wheel 0.32.3
widgetsnbextension 3.4.2
zipp 0.3.3
I'm very sorry if this caused confusion earlier. From here on out I will only report things that correspond to @kratsg's original report unless I am asked to do otherwise.
matthewfeickert
on 11 Jan 2019
whats the content of the pytest script on the box`
RonnyPfannschmidt
on 11 Jan 2019
whats the content of the pytest script on the box
tests/test_notebooks.py is essentially just calls to papermill.execute_notebook to make sure that the Jupyter notebooks we give as examples run through without errors.
@RonnyPfannschmidt If those links are not what you were asking for then I apologize and I can get you whatever info you want.
matthewfeickert
on 11 Jan 2019
@matthewfeickert i meant the actuall file content of the pytest executable - just to see if its a pip made file or a easy_install made file
RonnyPfannschmidt
on 11 Jan 2019
i meant the actual file content of the
pytestexecutable
Ah! Sorry about that.
travis@travis-job-fea5769d-5c3f-4f18-a8aa-bc516c1574c8:~/build/diana-hep/pyhf$ which pytest
/home/travis/virtualenv/python2.7.14/bin/pytest
travis@travis-job-fea5769d-5c3f-4f18-a8aa-bc516c1574c8:~/build/diana-hep/pyhf$ cat $(which pytest)
#!/home/travis/virtualenv/python2.7.14/bin/python
# -*- coding: utf-8 -*-
import re
import sys
from pytest import main
if __name__ == '__main__':
sys.argv[0] = re.sub(r'(-script\.pyw?|\.exe)?$', '', sys.argv[0])
sys.exit(main())
i see, thanks - this is a pip made script and just fine
its really not clear to me what causes the segfault there
my initial proposal would be to try and make a own pytest invocation script that does the problematic imports, then defers to pytest.main
also please try with --assert=plain just to see, whether the assertion rewriting is affecting the rest
RonnyPfannschmidt
on 11 Jan 2019
👍1
also please try with
--assert=plainjust to see, whether the assertion rewriting is affecting the rest
$ pytest --assert=plain tests/test_notebooks.py
produces the same segfault
travis@travis-job-a8d217c4-e9d1-4546-a03e-3b6bf02bc639:~/build/diana-hep/pyhf$ pytest --assert=plain tests/test_notebooks.py
=========================================================================================== test session starts ===========================================================================================
platform linux2 -- Python 2.7.14, pytest-3.10.1, py-1.7.0, pluggy-0.8.1
benchmark: 3.2.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /home/travis/build/diana-hep/pyhf, inifile: pytest.ini
plugins: mock-1.10.0, cov-2.6.1, console-scripts-0.1.7, benchmark-3.2.1
collected 1 item
tests/test_notebooks.py
# skipping showing all the test output here to save room
Coverage HTML written to dir htmlcov
============================================================================================ warnings summary =============================================================================================
/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/nbconvert/exporters/exporter_locator.py:28
/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/nbconvert/exporters/exporter_locator.py:28: DeprecationWarning: `nbconvert.exporters.exporter_locator` is deprecated in favor of `nbconvert.exporters.base` since nbconvert 5.0.
DeprecationWarning)
/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/tornado/web.py:1747
/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/tornado/web.py:1747: DeprecationWarning: @asynchronous is deprecated, use coroutines instead
DeprecationWarning)
tests/test_notebooks.py::test_notebooks
/home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/jupyter_client/session.py:348: DeprecationWarning: Session._session_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
def _session_changed(self, name, old, new):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
================================================================================== 1 passed, 3 warnings in 43.52 seconds ==================================================================================
Segmentation fault: 11
Stack trace returned 10 entries:
[bt] (0) /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1e945a) [0x7f668ed3445a]
[bt] (1) /home/travis/virtualenv/python2.7.14/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f891f6) [0x7f6691ad41f6]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0) [0x7f6703ad2cb0]
[bt] (3) /lib/x86_64-linux-gnu/libpthread.so.0(sem_wait+0) [0x7f6703e73640]
[bt] (4) /opt/python/2.7.14/lib/libpython2.7.so.1.0(PyThread_acquire_lock+0x68) [0x7f67041d1578]
[bt] (5) /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(+0x610c0) [0x7f67008530c0]
[bt] (6) /lib/x86_64-linux-gnu/libcrypto.so.1.0.0(+0x61749) [0x7f6700853749]
[bt] (7) /lib/x86_64-linux-gnu/libssl.so.1.0.0(SSL_CTX_free+0x6e) [0x7f66fe2de8de]
[bt] (8) /home/travis/virtualenv/python2.7.14/lib/python2.7/lib-dynload/_ssl.so(+0xa06b) [0x7f66fe51006b]
[bt] (9) /opt/python/2.7.14/lib/libpython2.7.so.1.0(+0xba17a) [0x7f670413d17a]
i suspect a tricky cleanup related bug in one of the other libraries,
off hand its incredibly hard to figure how/when it happens
however as far as i can tell, pytest completes execution, and then the segfault happens
RonnyPfannschmidt
on 11 Jan 2019
I can reproduce this outside of travis-ci :tada:
dockerfile
FROM ubuntu:trusty
RUN : \
&& apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends \
ca-certificates \
curl \
g++ \
gcc \
gdb \
git-core \
libbz2-dev \
libreadline-dev \
libsqlite3-dev \
libssl-dev \
make \
python-dev \
python-virtualenv \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
RUN : \
&& git clone https://github.com/pyenv/pyenv ~/.pyenv \
&& bash -euxc ' \
: \
&& export PYENV_ROOT=/root/.pyenv \
&& export PATH=/root/.pyenv/bin:$PATH \
&& eval "$(pyenv init -)" \
&& pyenv install 2.7.14 \
&& pyenv global 2.7.14 \
&& test "$(python2.7 --version 2>&1)" = "Python 2.7.14" \
'
WORKDIR /pyhf
ENV PATH=/venv/bin:$PATH
RUN : \
&& git clone https://github.com/diana-hep/pyhf . \
&& git checkout c3dd4592e54392ea0af25132c029df855423211b \
&& virtualenv /venv -p /root/.pyenv/shims/python2.7 \
&& pip install --upgrade setuptools pip \
&& pip install --no-cache-dir -e .[complete]
build
docker build -t test .
runtime
$ docker run --rm --privileged -ti test bash
root@dacd63468da6:/pyhf# gdb $(which python)
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.3) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /venv/bin/python...done.
(gdb) run /venv/bin/pytest tests/test_notebooks.py
Starting program: /venv/bin/python /venv/bin/pytest tests/test_notebooks.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffee627700 (LWP 46)]
[New Thread 0x7fffede26700 (LWP 47)]
[New Thread 0x7fffeb625700 (LWP 48)]
[New Thread 0x7fffe8e24700 (LWP 49)]
[New Thread 0x7fffe6623700 (LWP 50)]
[New Thread 0x7fffe3e22700 (LWP 51)]
[New Thread 0x7fffe1621700 (LWP 52)]
[New Thread 0x7fff7c4db700 (LWP 53)]
[New Thread 0x7fff7bcda700 (LWP 54)]
[New Thread 0x7fff7b4d9700 (LWP 55)]
[New Thread 0x7fff7acd8700 (LWP 56)]
[New Thread 0x7fff7a4d7700 (LWP 57)]
[New Thread 0x7fff79cd6700 (LWP 58)]
[New Thread 0x7fff794d5700 (LWP 59)]
[New Thread 0x7fff78cd4700 (LWP 60)]
[New Thread 0x7fff5bfff700 (LWP 61)]
[New Thread 0x7fff5b7fe700 (LWP 62)]
[New Thread 0x7fff5affd700 (LWP 63)]
[New Thread 0x7fff5a7fc700 (LWP 64)]
[New Thread 0x7fff59ffb700 (LWP 65)]
[New Thread 0x7fff597fa700 (LWP 66)]
[New Thread 0x7fff58ff9700 (LWP 67)]
[New Thread 0x7fff3bfff700 (LWP 68)]
[New Thread 0x7fff3b7fe700 (LWP 69)]
[New Thread 0x7fff3affd700 (LWP 70)]
[New Thread 0x7fff1d42a700 (LWP 71)]
[New Thread 0x7fff1cc29700 (LWP 72)]
[New Thread 0x7fff1c428700 (LWP 73)]
[Thread 0x7fff1c428700 (LWP 73) exited]
[Thread 0x7fff1cc29700 (LWP 72) exited]
[New Thread 0x7fff1cc29700 (LWP 74)]
[New Thread 0x7fff1c428700 (LWP 75)]
[New Thread 0x7fff1bc27700 (LWP 76)]
[New Thread 0x7fff1b426700 (LWP 77)]
[New Thread 0x7fff1ac25700 (LWP 78)]
[New Thread 0x7fff1a424700 (LWP 79)]
[New Thread 0x7fff19c23700 (LWP 80)]
[Thread 0x7fff1a424700 (LWP 79) exited]
[Thread 0x7fff19c23700 (LWP 80) exited]
[Thread 0x7fff1ac25700 (LWP 78) exited]
[Thread 0x7fff1b426700 (LWP 77) exited]
[Thread 0x7fffe1621700 (LWP 52) exited]
[Thread 0x7fffe3e22700 (LWP 51) exited]
[Thread 0x7fffede26700 (LWP 47) exited]
[Thread 0x7fffe6623700 (LWP 50) exited]
[Thread 0x7fffe8e24700 (LWP 49) exited]
[Thread 0x7fffeb625700 (LWP 48) exited]
[Thread 0x7fffee627700 (LWP 46) exited]
[New Thread 0x7fffe1621700 (LWP 82)]
[New Thread 0x7fffe3e22700 (LWP 87)]
[New Thread 0x7fffe6623700 (LWP 88)]
[New Thread 0x7fffe8e24700 (LWP 89)]
============================= test session starts ==============================
platform linux2 -- Python 2.7.14, pytest-3.10.1, py-1.7.0, pluggy-0.8.1
benchmark: 3.2.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /pyhf, inifile: pytest.ini
plugins: mock-1.10.0, cov-2.6.1, console-scripts-0.1.7, benchmark-3.2.1
collecting ...
Program received signal SIGSEGV, Segmentation fault.
update_refs (containers=0x7cf080 <generations+96>) at Modules/gcmodule.c:332
332 Modules/gcmodule.c: No such file or directory.
(gdb) bt
#0 update_refs (containers=0x7cf080 <generations+96>)
at Modules/gcmodule.c:332
#1 collect (generation=2) at Modules/gcmodule.c:924
#2 0x0000000000503461 in _PyObject_GC_Malloc (basicsize=<optimized out>)
at Modules/gcmodule.c:1511
#3 _PyObject_GC_New (tp=tp@entry=0x7e1a00 <PyFunction_Type>)
at Modules/gcmodule.c:1521
#4 0x0000000000533a84 in PyFunction_New (code=code@entry=0x7ffff2376ab0,
globals=0x7ffff236ee88) at Objects/funcobject.c:12
#5 0x00000000004b3c8f in PyEval_EvalFrameEx (f=f@entry=0x27f4be0,
throwflag=throwflag@entry=0) at Python/ceval.c:3051
#6 0x00000000004b8c66 in fast_function (nk=<optimized out>,
na=<optimized out>, n=3, pp_stack=0x7fffffff37d0, func=0x7ffff1c0aa28)
at Python/ceval.c:4442
#7 call_function (oparg=<optimized out>, pp_stack=0x7fffffff37d0)
at Python/ceval.c:4377
#8 PyEval_EvalFrameEx (f=f@entry=0x7ffff17657b0, throwflag=throwflag@entry=0)
at Python/ceval.c:2994
#9 0x00000000004baf58 in PyEval_EvalCodeEx (co=<optimized out>,
globals=<optimized out>, locals=locals@entry=0x0,
args=args@entry=0x7fffedc7e248, argcount=3, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=0x0) at Python/ceval.c:3589
#10 0x0000000000533190 in function_call (func=0x7ffff1c0ab18,
arg=0x7fffedc7e230, kw=0x0) at Objects/funcobject.c:523
#11 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff1c0ab18,
arg=arg@entry=0x7fffedc7e230, kw=kw@entry=0x0) at Objects/abstract.c:2547
#12 0x00000000004276e5 in instancemethod_call (func=0x7ffff1c0ab18,
arg=0x7fffedc7e230, kw=0x0) at Objects/classobject.c:2602
#13 0x0000000000423de0 in PyObject_Call (kw=0x0, arg=0x7fffedc84e18,
func=0x7ffff16b7aa0) at Objects/abstract.c:2547
#14 PyObject_CallFunctionObjArgs (callable=0x7ffff16b7aa0)
at Objects/abstract.c:2774
#15 0x00007ffff1c8011d in CTracer_handle_call (frame=0x664b0e0,
self=0x7ffff173e9f0) at coverage/ctracer/tracer.c:398
#16 CTracer_trace (self=0x7ffff173e9f0, frame=0x664b0e0, what=<optimized out>,
arg_unused=<optimized out>) at coverage/ctracer/tracer.c:871
#17 0x00000000004b0aad in call_trace (arg=0x7ba070 <_Py_NoneStruct>, what=0,
frame=0x664b0e0, obj=0x7ffff173e9f0, func=0x7ffff1c7f810 <CTracer_trace>)
at Python/ceval.c:4028
#18 call_trace_protected (func=0x7ffff1c7f810 <CTracer_trace>,
obj=0x7ffff173e9f0, frame=frame@entry=0x664b0e0, what=what@entry=0,
arg=0x7ba070 <_Py_NoneStruct>) at Python/ceval.c:4004
#19 0x00000000004b1ae5 in PyEval_EvalFrameEx (f=f@entry=0x664b0e0,
throwflag=throwflag@entry=0) at Python/ceval.c:993
#20 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fffedc821b0,
globals=globals@entry=0x7fffedcd6d70, locals=locals@entry=0x7fffedcd6d70,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#21 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fffedc821b0,
globals=globals@entry=0x7fffedcd6d70, locals=locals@entry=0x7fffedcd6d70)
at Python/ceval.c:669
#22 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (
name=0x6649e70 "wcwidth.table_wide", co=0x7fffedc821b0,
pathname=<optimized out>) at Python/import.c:731
#23 0x00000000004d4de6 in load_source_module (
name=0x6649e70 "wcwidth.table_wide",
pathname=0x664de00 "/venv/lib/python2.7/site-packages/wcwidth/table_wide.pyc", fp=<optimized out>) at Python/import.c:1121
#24 0x00000000004d5c77 in import_submodule (mod=mod@entry=0x7fffedc7f210,
subname=subname@entry=0x6649e78 "table_wide",
fullname=fullname@entry=0x6649e70 "wcwidth.table_wide")
at Python/import.c:2725
---Type <return> to continue, or q <return> to quit---q
Quit
(gdb) run /venv/bin/pytest tests/test_notebooks.py
The program being debugged has been started already.
Start it from the beginning? (y or n) y
Starting program: /venv/bin/python /venv/bin/pytest tests/test_notebooks.py
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffee627700 (LWP 101)]
[New Thread 0x7fffede26700 (LWP 102)]
[New Thread 0x7fffeb625700 (LWP 103)]
[New Thread 0x7fffe8e24700 (LWP 104)]
[New Thread 0x7fffe6623700 (LWP 105)]
[New Thread 0x7fffe3e22700 (LWP 106)]
[New Thread 0x7fffe1621700 (LWP 107)]
[New Thread 0x7fff7c4db700 (LWP 108)]
[New Thread 0x7fff7bcda700 (LWP 109)]
[New Thread 0x7fff7b4d9700 (LWP 110)]
[New Thread 0x7fff7acd8700 (LWP 111)]
[New Thread 0x7fff7a4d7700 (LWP 112)]
[New Thread 0x7fff79cd6700 (LWP 113)]
[New Thread 0x7fff794d5700 (LWP 114)]
[New Thread 0x7fff78cd4700 (LWP 115)]
[New Thread 0x7fff5bfff700 (LWP 116)]
[New Thread 0x7fff5b7fe700 (LWP 117)]
[New Thread 0x7fff5affd700 (LWP 118)]
[New Thread 0x7fff5a7fc700 (LWP 119)]
[New Thread 0x7fff59ffb700 (LWP 120)]
[New Thread 0x7fff597fa700 (LWP 121)]
[New Thread 0x7fff58ff9700 (LWP 122)]
[New Thread 0x7fff33fff700 (LWP 123)]
[New Thread 0x7fff3bfff700 (LWP 124)]
[New Thread 0x7fff3b7fe700 (LWP 125)]
[New Thread 0x7fff38ffd700 (LWP 126)]
[New Thread 0x7fff317fe700 (LWP 127)]
[New Thread 0x7fff30ffd700 (LWP 128)]
[Thread 0x7fff317fe700 (LWP 127) exited]
[Thread 0x7fff30ffd700 (LWP 128) exited]
[New Thread 0x7fff30ffd700 (LWP 129)]
[New Thread 0x7fff317fe700 (LWP 130)]
[New Thread 0x7fff1b42a700 (LWP 131)]
[New Thread 0x7fff1ac29700 (LWP 132)]
[New Thread 0x7fff1a428700 (LWP 133)]
[New Thread 0x7fff19c27700 (LWP 134)]
[New Thread 0x7fff19426700 (LWP 135)]
[Thread 0x7fff19426700 (LWP 135) exited]
[Thread 0x7fff19c27700 (LWP 134) exited]
[Thread 0x7fff1ac29700 (LWP 132) exited]
[Thread 0x7fff1a428700 (LWP 133) exited]
[Thread 0x7fffe3e22700 (LWP 106) exited]
[Thread 0x7fffe1621700 (LWP 107) exited]
[Thread 0x7fffede26700 (LWP 102) exited]
[Thread 0x7fffe6623700 (LWP 105) exited]
[Thread 0x7fffee627700 (LWP 101) exited]
[Thread 0x7fffe8e24700 (LWP 104) exited]
[Thread 0x7fffeb625700 (LWP 103) exited]
[New Thread 0x7fffe1621700 (LWP 137)]
[New Thread 0x7fffe3e22700 (LWP 142)]
[New Thread 0x7fffe6623700 (LWP 143)]
[New Thread 0x7fffe8e24700 (LWP 144)]
============================= test session starts ==============================
platform linux2 -- Python 2.7.14, pytest-3.10.1, py-1.7.0, pluggy-0.8.1
benchmark: 3.2.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /pyhf, inifile: pytest.ini
plugins: mock-1.10.0, cov-2.6.1, console-scripts-0.1.7, benchmark-3.2.1
collecting ... [Thread 0x7fffe1621700 (LWP 137) exited]
[Thread 0x7fffe8e24700 (LWP 144) exited]
[Thread 0x7fffe6623700 (LWP 143) exited]
[Thread 0x7fffe3e22700 (LWP 142) exited]
[New Thread 0x7fffe8e24700 (LWP 146)]
[New Thread 0x7fffe6623700 (LWP 151)]
[New Thread 0x7fffe3e22700 (LWP 152)]
[New Thread 0x7fffe1621700 (LWP 153)]
collected 1 item
tests/test_notebooks.py [New Thread 0x7fff189be700 (LWP 154)]
[Thread 0x7fffe8e24700 (LWP 146) exited]
[Thread 0x7fffe6623700 (LWP 151) exited]
[Thread 0x7fffe1621700 (LWP 153) exited]
[Thread 0x7fffe3e22700 (LWP 152) exited]
[New Thread 0x7fffe1621700 (LWP 156)]
[New Thread 0x7fffe3e22700 (LWP 161)]
[New Thread 0x7fffe6623700 (LWP 162)]
[New Thread 0x7fffe8e24700 (LWP 163)]
[Thread 0x7fffe6623700 (LWP 162) exited]
[Thread 0x7fffe8e24700 (LWP 163) exited]
[Thread 0x7fffe3e22700 (LWP 161) exited]
[Thread 0x7fffe1621700 (LWP 156) exited]
[New Thread 0x7fffe8e24700 (LWP 165)]
[New Thread 0x7fffe6623700 (LWP 170)]
[New Thread 0x7fffe3e22700 (LWP 171)]
[New Thread 0x7fffe1621700 (LWP 172)]
[New Thread 0x7fff03fff700 (LWP 173)]
[New Thread 0x7fff037fe700 (LWP 174)]
[New Thread 0x7fff02ffd700 (LWP 175)]
[Thread 0x7fff02ffd700 (LWP 175) exited]
[Thread 0x7fff189be700 (LWP 154) exited]
[New Thread 0x7fff189be700 (LWP 194)]
[Thread 0x7fffe1621700 (LWP 172) exited]
[Thread 0x7fffe3e22700 (LWP 171) exited]
[Thread 0x7fffe8e24700 (LWP 165) exited]
[Thread 0x7fffe6623700 (LWP 170) exited]
[New Thread 0x7fffe1621700 (LWP 196)]
[New Thread 0x7fffe3e22700 (LWP 201)]
[New Thread 0x7fffe6623700 (LWP 202)]
[New Thread 0x7fffe8e24700 (LWP 203)]
[New Thread 0x7fff02ffd700 (LWP 204)]
[Thread 0x7fff02ffd700 (LWP 204) exited]
[Thread 0x7fff189be700 (LWP 194) exited]
[New Thread 0x7fff189be700 (LWP 235)]
[Thread 0x7fffe3e22700 (LWP 201) exited]
[Thread 0x7fffe8e24700 (LWP 203) exited]
[Thread 0x7fffe6623700 (LWP 202) exited]
[Thread 0x7fffe1621700 (LWP 196) exited]
[New Thread 0x7fffe8e24700 (LWP 237)]
[New Thread 0x7fffe6623700 (LWP 242)]
[New Thread 0x7fffe3e22700 (LWP 243)]
[New Thread 0x7fffe1621700 (LWP 244)]
[New Thread 0x7fff02ffd700 (LWP 245)]
[Thread 0x7fff02ffd700 (LWP 245) exited]
[Thread 0x7fff189be700 (LWP 235) exited]
[New Thread 0x7fff189be700 (LWP 264)]
[Thread 0x7fffe6623700 (LWP 242) exited]
[Thread 0x7fffe1621700 (LWP 244) exited]
[Thread 0x7fffe3e22700 (LWP 243) exited]
[Thread 0x7fffe8e24700 (LWP 237) exited]
[New Thread 0x7fffe1621700 (LWP 266)]
[New Thread 0x7fffe3e22700 (LWP 271)]
[New Thread 0x7fffe6623700 (LWP 272)]
[New Thread 0x7fffe8e24700 (LWP 273)]
[New Thread 0x7fff02ffd700 (LWP 274)]
[Thread 0x7fff02ffd700 (LWP 274) exited]
[Thread 0x7fff189be700 (LWP 264) exited]
[New Thread 0x7fff189be700 (LWP 293)]
[Thread 0x7fffe8e24700 (LWP 273) exited]
[Thread 0x7fffe6623700 (LWP 272) exited]
[Thread 0x7fffe1621700 (LWP 266) exited]
[Thread 0x7fffe3e22700 (LWP 271) exited]
[New Thread 0x7fffe8e24700 (LWP 295)]
[New Thread 0x7fffe6623700 (LWP 300)]
[New Thread 0x7fffe3e22700 (LWP 301)]
[New Thread 0x7fffe1621700 (LWP 302)]
[New Thread 0x7fff02ffd700 (LWP 303)]
[Thread 0x7fff02ffd700 (LWP 303) exited]
[Thread 0x7fff189be700 (LWP 293) exited]
. [100%]
---------- coverage: platform linux2, python 2.7.14-final-0 ----------
Name Stmts Miss Cover Missing
-----------------------------------------------------------------
pyhf/__init__.py 31 5 84% 54-58, 60, 79
pyhf/commandline.py 78 78 0% 1-137
pyhf/constraints.py 86 2 98% 127-128
pyhf/events.py 31 5 84% 11-12, 15, 91, 99
pyhf/exceptions/__init__.py 25 11 56% 20-34
pyhf/interpolators/__init__.py 27 19 30% 2-15, 27-37
pyhf/interpolators/code0.py 50 13 74% 51-54, 83-89, 92, 95-96
pyhf/interpolators/code1.py 55 41 25% 27-43, 46-57, 60-72, 75-93, 98-104, 107, 110-111
pyhf/interpolators/code2.py 60 46 23% 31-45, 48-56, 59-64, 67-108, 113-121, 124, 127-128
pyhf/interpolators/code4.py 85 70 18% 35-126, 129-146, 149-164, 167-226, 269-343, 346-347, 350-351
pyhf/modifiers/__init__.py 60 12 80% 20, 40, 48, 58, 135, 138, 158-171
pyhf/modifiers/histosys.py 36 0 100%
pyhf/modifiers/lumi.py 34 5 85% 14, 55-62
pyhf/modifiers/normfactor.py 34 1 97% 55
pyhf/modifiers/normsys.py 36 6 83% 15, 54, 69-76
pyhf/modifiers/shapefactor.py 39 0 100%
pyhf/modifiers/shapesys.py 69 0 100%
pyhf/modifiers/staterror.py 65 32 51% 14, 55-72, 85-90, 95-123, 129-142
pyhf/optimize/__init__.py 34 16 53% 14-22, 27-35, 47-48
pyhf/optimize/opt_minuit.py 38 25 34% 18-46, 50-53, 59-64
pyhf/optimize/opt_pytorch.py 44 44 0% 1-58
pyhf/optimize/opt_scipy.py 23 6 74% 18-20, 38-40
pyhf/optimize/opt_tflow.py 46 46 0% 1-76
pyhf/paramsets.py 54 4 93% 23, 75, 91, 97
pyhf/pdf.py 219 22 90% 26-32, 58-64, 79, 87, 124, 273-281, 296, 444-450, 453-454
pyhf/readxml.py 116 116 0% 1-257
pyhf/simplemodels.py 4 0 100%
pyhf/tensor/__init__.py 34 6 82% 21-22, 34-35, 47-48
pyhf/tensor/mxnet_backend.py 127 90 29% 39-40, 52-57, 70-82, 91, 94-95, 107-113, 126-130, 143-147, 150-151, 163, 175, 189-191, 203-204, 217-219, 231-232, 244-245, 262, 289-292, 308, 334-350, 356, 359, 374-375, 378-380, 406-412, 418-420, 449-456, 479-485
pyhf/tensor/numpy_backend.py 83 16 81% 40-43, 46-48, 80, 95, 98, 101, 143, 197-199, 235
pyhf/tensor/pytorch_backend.py 121 85 30% 34-35, 38-43, 46-48, 60-62, 65, 68-69, 72, 75, 78-79, 86-87, 90-91, 94, 97, 100-102, 105-106, 109-111, 114-115, 118-119, 122, 125-128, 142, 145, 168-190, 204-205, 208-210, 236-238, 241-245, 270-274, 295-299
pyhf/tensor/tensorflow_backend.py 143 106 26% 40-45, 48-71, 74-86, 89, 92, 95, 107-118, 121-122, 129-130, 137-138, 141, 144, 147-149, 152-153, 156, 159, 162-164, 167-168, 171-172, 175, 178-181, 195, 219-244, 261, 290-292, 321-323, 351-355, 383-387, 413-417
pyhf/utils.py 69 3 96% 40-41, 229
pyhf/version.py 1 0 100%
pyhf/writexml.py 30 30 0% 1-45
-----------------------------------------------------------------
TOTAL 2087 961 54%
Coverage HTML written to dir htmlcov
=============================== warnings summary ===============================
/venv/lib/python2.7/site-packages/nbconvert/exporters/exporter_locator.py:28
/venv/lib/python2.7/site-packages/nbconvert/exporters/exporter_locator.py:28: DeprecationWarning: `nbconvert.exporters.exporter_locator` is deprecated in favor of `nbconvert.exporters.base` since nbconvert 5.0.
DeprecationWarning)
/venv/lib/python2.7/site-packages/tornado/web.py:1747
/venv/lib/python2.7/site-packages/tornado/web.py:1747: DeprecationWarning: @asynchronous is deprecated, use coroutines instead
DeprecationWarning)
tests/test_notebooks.py::test_notebooks
/venv/lib/python2.7/site-packages/jupyter_client/session.py:348: DeprecationWarning: Session._session_changed is deprecated in traitlets 4.1: use @observe and @unobserve instead.
def _session_changed(self, name, old, new):
-- Docs: https://docs.pytest.org/en/latest/warnings.html
==================== 1 passed, 3 warnings in 37.19 seconds =====================
Program received signal SIGSEGV, Segmentation fault.
update_refs (containers=0x7cf080 <generations+96>) at Modules/gcmodule.c:332
332 in Modules/gcmodule.c
(gdb) bt
#0 update_refs (containers=0x7cf080 <generations+96>)
at Modules/gcmodule.c:332
#1 collect (generation=generation@entry=2) at Modules/gcmodule.c:924
#2 0x00000000005031c8 in PyGC_Collect () at Modules/gcmodule.c:1440
#3 0x00000000004ebe0b in Py_Finalize () at Python/pythonrun.c:448
#4 0x00000000004eb9c8 in Py_Exit (sts=sts@entry=0) at Python/pythonrun.c:1783
#5 0x00000000004149f3 in handle_system_exit () at Python/pythonrun.c:1151
#6 0x00000000004ebd25 in handle_system_exit () at Python/pythonrun.c:1192
#7 PyErr_PrintEx (set_sys_last_vars=<optimized out>)
at Python/pythonrun.c:1161
#8 0x00000000004ec783 in PyErr_Print () at Python/pythonrun.c:1064
#9 PyRun_SimpleFileExFlags (fp=<optimized out>, filename=<optimized out>,
closeit=1, flags=0x7fffffffe5f0) at Python/pythonrun.c:952
#10 0x0000000000415f11 in Py_Main (argc=<optimized out>, argv=<optimized out>)
at Modules/main.c:645
#11 0x00007ffff7107f45 in __libc_start_main ()
from /lib/x86_64-linux-gnu/libc.so.6
#12 0x00000000004150fe in _start ()
Note that this shows two different crashes -- the first crashes on a "fresh" run during collection, the second crashes during final-gc. Running a third time shows the same as the final-gc crash.
Interestingly enough, python -m pytest is also crashing for me:
root@dacd63468da6:/pyhf# python -m pytest tests/test_notebooks.py
============================= test session starts ==============================
platform linux2 -- Python 2.7.14, pytest-3.10.1, py-1.7.0, pluggy-0.8.1
benchmark: 3.2.1 (defaults: timer=time.time disable_gc=False min_rounds=5 min_time=0.000005 max_time=1.0 calibration_precision=10 warmup=False warmup_iterations=100000)
rootdir: /pyhf, inifile: pytest.ini
plugins: mock-1.10.0, cov-2.6.1, console-scripts-0.1.7, benchmark-3.2.1
collecting ... *** Error in `python': double free or corruption (!prev): 0x0000000001adc2a0 ***
Aborted (core dumped)
😕2
👍1
@asottile thanks very much for going through so much work to reproduce this. It is both nice to see it reproduced but perhaps even more confusing. I don't think I am able to offer any ideas on the additional crash, but I will give it more time later on.
matthewfeickert
on 11 Jan 2019
I can also reproduce without invoking pytest:
root@dacd63468da6:/pyhf# find -name "*.pyc" -delete && python -c 'import tests.conftest'
2019-01-11 18:19:00.744299: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Segmentation fault: 11
Stack trace returned 10 entries:
[bt] (0) /venv/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x1e945a) [0x7fc30f61445a]
[bt] (1) /venv/lib/python2.7/site-packages/mxnet/libmxnet.so(+0x2f891f6) [0x7fc3123b41f6]
[bt] (2) /lib/x86_64-linux-gnu/libc.so.6(+0x36cb0) [0x7fc3de7cdcb0]
[bt] (3) python() [0x502168]
[bt] (4) python() [0x445eab]
[bt] (5) python() [0x5023e5]
[bt] (6) python(PyGC_Collect+0x28) [0x5031c8]
[bt] (7) python() [0x4ebe0b]
[bt] (8) python(Py_Main+0x62f) [0x4159cf]
[bt] (9) /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7fc3de7b8f45]
And I can reproduce without involving pytest:
root@dacd63468da6:/pyhf# find -name "*.pyc" -delete && python -c 'import torch, pyhf, tensorflow as tf; pyhf.tensor.tensorflow_backend(session=tf.Session())'; echo "===== $? ====="
2019-01-11 18:22:48.392944: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
Segmentation fault (core dumped)
===== 139 =====
The full bt for that:
root@dacd63468da6:/pyhf# find -name "*.pyc" -delete && gdb $(which python)
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.3) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /venv/bin/python...done.
(gdb) run -c 'import torch, pyhf, tensorflow as tf; pyhf.tensor.tensorflow_backend(session=tf.Session())'
Starting program: /venv/bin/python -c 'import torch, pyhf, tensorflow as tf; pyhf.tensor.tensorflow_backend(session=tf.Session())'
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff349a700 (LWP 1589)]
[New Thread 0x7ffff2c99700 (LWP 1590)]
[New Thread 0x7ffff0498700 (LWP 1591)]
[New Thread 0x7fffedc97700 (LWP 1592)]
[New Thread 0x7fffeb496700 (LWP 1593)]
[New Thread 0x7fffe8c95700 (LWP 1594)]
[New Thread 0x7fffe6494700 (LWP 1595)]
[Thread 0x7ffff0498700 (LWP 1591) exited]
[Thread 0x7fffe6494700 (LWP 1595) exited]
[Thread 0x7fffe8c95700 (LWP 1594) exited]
[Thread 0x7fffedc97700 (LWP 1592) exited]
[Thread 0x7fffeb496700 (LWP 1593) exited]
[Thread 0x7ffff349a700 (LWP 1589) exited]
[Thread 0x7ffff2c99700 (LWP 1590) exited]
[New Thread 0x7fffe6494700 (LWP 1599)]
2019-01-11 18:24:03.028776: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[New Thread 0x7fffe8c95700 (LWP 1600)]
[New Thread 0x7fffeb496700 (LWP 1601)]
[New Thread 0x7fffedc97700 (LWP 1602)]
[New Thread 0x7fff7e774700 (LWP 1603)]
[New Thread 0x7fff7df73700 (LWP 1604)]
[New Thread 0x7fff7d772700 (LWP 1605)]
[New Thread 0x7fff7cf71700 (LWP 1606)]
[New Thread 0x7fff5ffff700 (LWP 1607)]
[New Thread 0x7fff5f7fe700 (LWP 1608)]
[New Thread 0x7fff5effd700 (LWP 1609)]
[New Thread 0x7fff5e7fc700 (LWP 1610)]
[New Thread 0x7fff5dffb700 (LWP 1611)]
[New Thread 0x7fff5d7fa700 (LWP 1612)]
[New Thread 0x7fff5cff9700 (LWP 1613)]
[New Thread 0x7fff3ffff700 (LWP 1614)]
[New Thread 0x7fff3f7fe700 (LWP 1615)]
[New Thread 0x7fff3effd700 (LWP 1616)]
[Thread 0x7fff5f7fe700 (LWP 1608) exited]
Program received signal SIGSEGV, Segmentation fault.
visit_decref (op=0x7ffff7e11200, data=0x0) at Modules/gcmodule.c:360
360 Modules/gcmodule.c: No such file or directory.
(gdb) bt
#0 visit_decref (op=0x7ffff7e11200, data=0x0) at Modules/gcmodule.c:360
#1 0x0000000000445eab in list_traverse (o=0x7ffff7e0f320,
visit=0x502160 <visit_decref>, arg=0x0) at Objects/listobject.c:2366
#2 0x00000000005023e5 in subtract_refs (containers=0x7cf080 <generations+96>)
at Modules/gcmodule.c:385
#3 collect (generation=generation@entry=2) at Modules/gcmodule.c:925
#4 0x00000000005031c8 in PyGC_Collect () at Modules/gcmodule.c:1440
#5 0x00000000004ebe0b in Py_Finalize () at Python/pythonrun.c:448
#6 0x00000000004ebf55 in Py_Finalize () at Python/pythonrun.c:409
#7 0x00000000004159cf in Py_Main (argc=<optimized out>, argv=<optimized out>)
at Modules/main.c:670
#8 0x00007ffff7107f45 in __libc_start_main ()
from /lib/x86_64-linux-gnu/libc.so.6
#9 0x00000000004150fe in _start ()
This rabbit hole keeps getting deeper and deeper -- I suspect either heap corruption or stack smashing happening somewhere, the backtraces don't seem entirely related other than "can crash during any import" and then "crashes during interpreter teardown":
root@dacd63468da6:/pyhf# find -name "*.pyc" -delete && gdb $(which python)
GNU gdb (Ubuntu 7.7.1-0ubuntu5~14.04.3) 7.7.1
Copyright (C) 2014 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from /venv/bin/python...done.
(gdb) run -c 'import tests.conftest; import tests.test_notebooks'
Starting program: /venv/bin/python -c 'import tests.conftest; import tests.test_notebooks'
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7ffff349a700 (LWP 1674)]
[New Thread 0x7ffff2c99700 (LWP 1675)]
[New Thread 0x7ffff0498700 (LWP 1676)]
[New Thread 0x7fffedc97700 (LWP 1677)]
[New Thread 0x7fffeb496700 (LWP 1678)]
[New Thread 0x7fffe8c95700 (LWP 1679)]
[New Thread 0x7fffe6494700 (LWP 1680)]
[Thread 0x7ffff0498700 (LWP 1676) exited]
[Thread 0x7fffe6494700 (LWP 1680) exited]
[Thread 0x7fffe8c95700 (LWP 1679) exited]
[Thread 0x7fffedc97700 (LWP 1677) exited]
[Thread 0x7ffff349a700 (LWP 1674) exited]
[Thread 0x7fffeb496700 (LWP 1678) exited]
[Thread 0x7ffff2c99700 (LWP 1675) exited]
[New Thread 0x7fffe6494700 (LWP 1685)]
2019-01-11 18:26:37.510783: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX2 FMA
[New Thread 0x7fffe8c95700 (LWP 1686)]
[New Thread 0x7fffeb496700 (LWP 1687)]
[New Thread 0x7fffedc97700 (LWP 1688)]
[New Thread 0x7fff7e32e700 (LWP 1689)]
[New Thread 0x7fff7db2d700 (LWP 1690)]
[New Thread 0x7fff7d32c700 (LWP 1691)]
[New Thread 0x7fff7cb2b700 (LWP 1692)]
[New Thread 0x7fff5ffff700 (LWP 1693)]
[New Thread 0x7fff5f7fe700 (LWP 1694)]
[New Thread 0x7fff5effd700 (LWP 1695)]
[New Thread 0x7fff5e7fc700 (LWP 1696)]
[New Thread 0x7fff5dffb700 (LWP 1697)]
[New Thread 0x7fff5d7fa700 (LWP 1698)]
[New Thread 0x7fff5cff9700 (LWP 1699)]
[New Thread 0x7fff37fff700 (LWP 1700)]
[New Thread 0x7fff377fe700 (LWP 1701)]
[New Thread 0x7fff36ffd700 (LWP 1702)]
[New Thread 0x7fff2142a700 (LWP 1703)]
[New Thread 0x7fff20c29700 (LWP 1704)]
[New Thread 0x7fff20428700 (LWP 1705)]
[Thread 0x7fff20428700 (LWP 1705) exited]
[Thread 0x7fff20c29700 (LWP 1704) exited]
[New Thread 0x7fff20c29700 (LWP 1706)]
[New Thread 0x7fff20428700 (LWP 1707)]
[New Thread 0x7fff1fc27700 (LWP 1708)]
Program received signal SIGSEGV, Segmentation fault.
visit_decref (op=0x7ffff7e13200, data=0x0) at Modules/gcmodule.c:360
360 Modules/gcmodule.c: No such file or directory.
(gdb) bt
#0 visit_decref (op=0x7ffff7e13200, data=0x0) at Modules/gcmodule.c:360
#1 0x0000000000445eab in list_traverse (o=0x7ffff7e11560,
visit=0x502160 <visit_decref>, arg=0x0) at Objects/listobject.c:2366
#2 0x00000000005023e5 in subtract_refs (containers=0x7cf080 <generations+96>)
at Modules/gcmodule.c:385
#3 collect (generation=2) at Modules/gcmodule.c:925
#4 0x0000000000503541 in _PyObject_GC_Malloc (basicsize=<optimized out>)
at Modules/gcmodule.c:1511
#5 _PyObject_GC_NewVar (tp=tp@entry=0x7bfba0 <PyTuple_Type>,
nitems=nitems@entry=7) at Modules/gcmodule.c:1531
#6 0x0000000000479c7d in PyTuple_New (size=size@entry=7)
at Objects/tupleobject.c:90
#7 0x00000000004d9a28 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:885
#8 0x00000000004d9636 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:1024
#9 0x00000000004d9a72 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:891
#10 0x00000000004d9636 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:1024
#11 0x00000000004d9a72 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:891
#12 0x00000000004d9636 in r_object (p=0x7fffffffc6a0) at Python/marshal.c:1024
#13 0x00000000004e02f6 in PyMarshal_ReadObjectFromString (len=<optimized out>,
str=0x5d38bb0 "c") at Python/marshal.c:1188
#14 PyMarshal_ReadLastObjectFromFile (fp=fp@entry=0x5c1d860)
at Python/marshal.c:1149
#15 0x00000000004d4cb7 in read_compiled_module (fp=0x5c1d860,
cpathname=0x5c887f0 "/venv/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc") at Python/import.c:823
#16 load_source_module (name=0x5c82de0 "IPython.core.interactiveshell",
pathname=0x5c0eed0 "/venv/lib/python2.7/site-packages/IPython/core/interactiveshell.py", fp=0x5c71620) at Python/import.c:1094
#17 0x00000000004d5c77 in import_submodule (mod=mod@entry=0x7fff5c218fa0,
subname=subname@entry=0x5c82ded "interactiveshell",
fullname=fullname@entry=0x5c82de0 "IPython.core.interactiveshell")
at Python/import.c:2725
#18 0x00000000004d6d7e in load_next (p_buflen=<synthetic pointer>,
buf=0x5c82de0 "IPython.core.interactiveshell", p_name=<synthetic pointer>,
altmod=0x7fff5c218fa0, mod=0x7fff5c218fa0) at Python/import.c:2539
#19 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7fff5c21a248, globals=<optimized out>, name=0x0)
at Python/import.c:2256
#20 PyImport_ImportModuleLevel (name=<optimized out>, globals=<optimized out>,
locals=<optimized out>, fromlist=0x7fff5c21a248, level=<optimized out>)
at Python/import.c:2312
#21 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#22 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7fff5c0ec158, kw=kw@entry=0x0) at Objects/abstract.c:2547
#23 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7fff5c0ec158, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#24 PyEval_EvalFrameEx (f=f@entry=0x7fff5c322c90, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#25 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fff5c0dcd30,
globals=globals@entry=0x7fff5c0e76e0, locals=locals@entry=0x7fff5c0e76e0,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#26 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fff5c0dcd30,
globals=globals@entry=0x7fff5c0e76e0, locals=locals@entry=0x7fff5c0e76e0)
at Python/ceval.c:669
#27 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (
name=0x5caa380 "IPython.terminal.embed", co=0x7fff5c0dcd30,
pathname=<optimized out>) at Python/import.c:731
#28 0x00000000004d4de6 in load_source_module (
name=0x5caa380 "IPython.terminal.embed",
pathname=0x5c0dec0 "/venv/lib/python2.7/site-packages/IPython/terminal/embed.pyc", fp=<optimized out>) at Python/import.c:1121
---Type <return> to continue, or q <return> to quit---
#29 0x00000000004d5c77 in import_submodule (mod=mod@entry=0x7fff5c225fd8,
subname=subname@entry=0x5caa391 "embed",
fullname=fullname@entry=0x5caa380 "IPython.terminal.embed")
at Python/import.c:2725
#30 0x00000000004d6d7e in load_next (p_buflen=<synthetic pointer>,
buf=0x5caa380 "IPython.terminal.embed", p_name=<synthetic pointer>,
altmod=0x7fff5c225fd8, mod=0x7fff5c225fd8) at Python/import.c:2539
#31 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7fff5c216e10, globals=<optimized out>, name=0x0)
at Python/import.c:2256
#32 PyImport_ImportModuleLevel (name=<optimized out>, globals=<optimized out>,
locals=<optimized out>, fromlist=0x7fff5c216e10, level=<optimized out>)
at Python/import.c:2312
#33 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#34 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7fff5c270b90, kw=kw@entry=0x0) at Objects/abstract.c:2547
#35 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7fff5c270b90, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#36 PyEval_EvalFrameEx (f=f@entry=0x7fff5c322ad0, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#37 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fff5c224130,
globals=globals@entry=0x7fff5c217910, locals=locals@entry=0x7fff5c217910,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#38 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fff5c224130,
globals=globals@entry=0x7fff5c217910, locals=locals@entry=0x7fff5c217910)
at Python/ceval.c:669
#39 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (name=0x5c07d8a "IPython",
co=0x7fff5c224130, pathname=<optimized out>) at Python/import.c:731
#40 0x00000000004d4de6 in load_source_module (name=0x5c07d8a "IPython",
pathname=0x5c0b370 "/venv/lib/python2.7/site-packages/IPython/__init__.pyc", fp=<optimized out>) at Python/import.c:1121
#41 0x00000000004d61cd in load_package (name=0x5c07d8a "IPython",
pathname=<optimized out>) at Python/import.c:1188
#42 0x00000000004d5c77 in import_submodule (
mod=mod@entry=0x7ba070 <_Py_NoneStruct>,
subname=subname@entry=0x5c07d8a "IPython",
fullname=fullname@entry=0x5c07d8a "IPython") at Python/import.c:2725
#43 0x00000000004d71d6 in load_next (p_buflen=<synthetic pointer>,
buf=0x5c07d80 "papermill.IPython", p_name=<synthetic pointer>,
altmod=0x7ba070 <_Py_NoneStruct>, mod=0x7fff5c218328)
at Python/import.c:2543
#44 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7ba070 <_Py_NoneStruct>, globals=<optimized out>, name=0x0)
at Python/import.c:2247
#45 PyImport_ImportModuleLevel (name=0x7fff8ca08f54 "IPython",
globals=<optimized out>, locals=<optimized out>,
fromlist=0x7ba070 <_Py_NoneStruct>, level=<optimized out>)
at Python/import.c:2312
#46 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#47 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7fff5c28bf70, kw=kw@entry=0x0) at Objects/abstract.c:2547
#48 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7fff5c28bf70, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#49 PyEval_EvalFrameEx (f=f@entry=0x7fff7e4dc8e8, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#50 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fff5c221e30,
globals=globals@entry=0x7fff5c219c58, locals=locals@entry=0x7fff5c219c58,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
---Type <return> to continue, or q <return> to quit---
#51 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fff5c221e30,
globals=globals@entry=0x7fff5c219c58, locals=locals@entry=0x7fff5c219c58)
at Python/ceval.c:669
#52 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (
name=0x5b734c0 "papermill.api", co=0x7fff5c221e30,
pathname=<optimized out>) at Python/import.c:731
#53 0x00000000004d4de6 in load_source_module (name=0x5b734c0 "papermill.api",
pathname=0x549fe50 "/venv/lib/python2.7/site-packages/papermill/api.pyc",
fp=<optimized out>) at Python/import.c:1121
#54 0x00000000004d5c77 in import_submodule (mod=mod@entry=0x7fff5c218328,
subname=subname@entry=0x5b734ca "api",
fullname=fullname@entry=0x5b734c0 "papermill.api") at Python/import.c:2725
#55 0x00000000004d6cb6 in load_next (p_buflen=<synthetic pointer>,
buf=0x5b734c0 "papermill.api", p_name=<synthetic pointer>,
altmod=0x7fff5c218328, mod=0x7fff5c218328) at Python/import.c:2539
#56 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7fff5c28bba8, globals=<optimized out>, name=0x0)
at Python/import.c:2247
#57 PyImport_ImportModuleLevel (name=0x7fff976cb644 "api",
globals=<optimized out>, locals=<optimized out>, fromlist=0x7fff5c28bba8,
level=<optimized out>) at Python/import.c:2312
#58 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#59 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7fff5c2b7cb0, kw=kw@entry=0x0) at Objects/abstract.c:2547
#60 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7fff5c2b7cb0, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#61 PyEval_EvalFrameEx (f=f@entry=0x7fff7e364050, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#62 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fff5c26dcb0,
globals=globals@entry=0x7fff5c2177f8, locals=locals@entry=0x7fff5c2177f8,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#63 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fff5c26dcb0,
globals=globals@entry=0x7fff5c2177f8, locals=locals@entry=0x7fff5c2177f8)
at Python/ceval.c:669
#64 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (
name=0x5497d66 "papermill", co=0x7fff5c26dcb0, pathname=<optimized out>)
at Python/import.c:731
#65 0x00000000004d4de6 in load_source_module (name=0x5497d66 "papermill",
pathname=0x5be3360 "/venv/lib/python2.7/site-packages/papermill/__init__.pyc", fp=<optimized out>) at Python/import.c:1121
#66 0x00000000004d61cd in load_package (name=0x5497d66 "papermill",
pathname=<optimized out>) at Python/import.c:1188
#67 0x00000000004d5c77 in import_submodule (
mod=mod@entry=0x7ba070 <_Py_NoneStruct>,
subname=subname@entry=0x5497d66 "papermill",
fullname=fullname@entry=0x5497d66 "papermill") at Python/import.c:2725
#68 0x00000000004d71d6 in load_next (p_buflen=<synthetic pointer>,
buf=0x5497d60 "tests.papermill", p_name=<synthetic pointer>,
altmod=0x7ba070 <_Py_NoneStruct>, mod=0x7ffff7eebb08)
at Python/import.c:2543
#69 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7ba070 <_Py_NoneStruct>, globals=<optimized out>, name=0x0)
at Python/import.c:2247
#70 PyImport_ImportModuleLevel (name=0x7fff5c2912c4 "papermill",
globals=<optimized out>, locals=<optimized out>,
fromlist=0x7ba070 <_Py_NoneStruct>, level=<optimized out>)
at Python/import.c:2312
#71 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#72 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7fff5c2675d0, kw=kw@entry=0x0) at Objects/abstract.c:2547
---Type <return> to continue, or q <return> to quit---
#73 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7fff5c2675d0, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#74 PyEval_EvalFrameEx (f=f@entry=0x7ffff7fd7c20, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#75 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7fff5c26d430,
globals=globals@entry=0x7fff5c288c58, locals=locals@entry=0x7fff5c288c58,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#76 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7fff5c26d430,
globals=globals@entry=0x7fff5c288c58, locals=locals@entry=0x7fff5c288c58)
at Python/ceval.c:669
#77 0x00000000004d4a88 in PyImport_ExecCodeModuleEx (
name=0x889e30 "tests.test_notebooks", co=0x7fff5c26d430,
pathname=<optimized out>) at Python/import.c:731
#78 0x00000000004d4de6 in load_source_module (
name=0x889e30 "tests.test_notebooks",
pathname=0x881860 "tests/test_notebooks.py", fp=<optimized out>)
at Python/import.c:1121
#79 0x00000000004d5c77 in import_submodule (mod=mod@entry=0x7ffff7eebb08,
subname=subname@entry=0x889e36 "test_notebooks",
fullname=fullname@entry=0x889e30 "tests.test_notebooks")
at Python/import.c:2725
#80 0x00000000004d6d7e in load_next (p_buflen=<synthetic pointer>,
buf=0x889e30 "tests.test_notebooks", p_name=<synthetic pointer>,
altmod=0x7ffff7eebb08, mod=0x7ffff7eebb08) at Python/import.c:2539
#81 import_module_level (locals=<optimized out>, level=<optimized out>,
fromlist=0x7ba070 <_Py_NoneStruct>, globals=<optimized out>, name=0x0)
at Python/import.c:2256
#82 PyImport_ImportModuleLevel (name=<optimized out>, globals=<optimized out>,
locals=<optimized out>, fromlist=0x7ba070 <_Py_NoneStruct>,
level=<optimized out>) at Python/import.c:2312
#83 0x00000000004afb2b in builtin___import__ (self=<optimized out>,
args=<optimized out>, kwds=<optimized out>) at Python/bltinmodule.c:49
#84 0x00000000004231fa in PyObject_Call (func=func@entry=0x7ffff7fe1fc8,
arg=arg@entry=0x7ffff7f99158, kw=kw@entry=0x0) at Objects/abstract.c:2547
#85 0x00000000004b5506 in PyEval_CallObjectWithKeywords (kw=0x0,
arg=0x7ffff7f99158, func=0x7ffff7fe1fc8) at Python/ceval.c:4226
#86 PyEval_EvalFrameEx (f=f@entry=0x7ffff7fd78c0, throwflag=throwflag@entry=0)
at Python/ceval.c:2628
#87 0x00000000004baf58 in PyEval_EvalCodeEx (co=co@entry=0x7ffff7f90f30,
globals=globals@entry=0x7ffff7f83168, locals=locals@entry=0x7ffff7f83168,
args=args@entry=0x0, argcount=argcount@entry=0, kws=kws@entry=0x0,
kwcount=kwcount@entry=0, defs=defs@entry=0x0, defcount=defcount@entry=0,
closure=closure@entry=0x0) at Python/ceval.c:3589
#88 0x00000000004bb082 in PyEval_EvalCode (co=co@entry=0x7ffff7f90f30,
globals=globals@entry=0x7ffff7f83168, locals=locals@entry=0x7ffff7f83168)
at Python/ceval.c:669
#89 0x00000000004ecba2 in run_mod (arena=0x8380c0, flags=0x7fffffffe5f0,
locals=0x7ffff7f83168, globals=0x7ffff7f83168,
filename=0x5513fe "<string>", mod=0x8408f8) at Python/pythonrun.c:1376
#90 PyRun_StringFlags (flags=0x7fffffffe5f0, locals=0x7ffff7f83168,
globals=0x7ffff7f83168, start=257, str=0x8408f8 "\001")
at Python/pythonrun.c:1339
#91 PyRun_SimpleStringFlags (
command=command@entry=0x80b010 "import tests.conftest; import tests.test_notebooks\n", flags=flags@entry=0x7fffffffe5f0) at Python/pythonrun.c:974
#92 0x0000000000415a9b in Py_Main (argc=3, argv=0x7fffffffe7a8)
at Modules/main.c:589
#93 0x00007ffff7107f45 in __libc_start_main ()
from /lib/x86_64-linux-gnu/libc.so.6
#94 0x00000000004150fe in _start ()
@kratsg @matthewfeickert since there's a reproduction without involving pytest, are you ok with me closing this issue?
asottile
on 11 Jan 2019
@kratsg @matthewfeickert since there's a reproduction without involving pytest, are you ok with me closing this issue?
I guess I don't fully understand as conftest.py still uses pytest. Are you saying that because there's no CLI pytest invocation, that it doesn't involve pytest?
kratsg
on 11 Jan 2019
This command here reproduces without conftest.py:
find -name "*.pyc" -delete
python -c 'import torch, pyhf, tensorflow as tf; pyhf.tensor.tensorflow_backend(session=tf.Session())'
👍1
Ok, I can confirm that this command crashes on Ubuntu (not RHEL, and not MacOSX) so I think we can close this and I'll continue to explore. Thanks!
kratsg
on 11 Jan 2019
@asottile Thanks very much for your work on this and for helping us understand this bug. We really appreciate it!
matthewfeickert
on 11 Jan 2019
No problem! Happy to help and good luck tracking down the _torch_ ;)
asottile
on 11 Jan 2019
👍1
@kratsg did you ever figure out what was going on? I'm having this issue when trying to run tests of tensorflow in pytest
lminer
on 6 Mar 2020
👍1
@kratsg did you ever figure out what was going on? I'm having this issue when trying to run tests of tensorflow in pytest
Nope. I think in general, the recommendation is to use python -m <module> whenever you need to invoke it.
kratsg
on 6 Mar 2020
Was this page helpful?
0 / 5 - 0 ratings
Related issues
Balky79
路
3Comments
wstrm
路
3Comments
thakkardharmik
路
3Comments
jurisbu
路
3Comments
dmtucker
路
3Comments
Most helpful comment
The vonly difference I could imagine there is the setuptools entry point machinery somehow having a negative side effect. This one sounds like it's going to be pretty difficult to reproduce!