Pytest: xunit2 format does not contain a root <testsuites> tag
junit_family=xunit2 option still generates an old xml format tests report.
- junit_family=xunit2 creates the .xml file in the legacy format
-
Package Version
---------------------------------- --------
alabaster 0.7.12
anaconda-client 1.7.2
anaconda-project 0.8.2
aniso8601 6.0.0
appdirs 1.4.3
appnope 0.1.0
appscript 1.0.1
asn1crypto 0.24.0
astroid 2.2.5
astropy 3.1.2
atomicwrites 1.3.0
attrs 19.1.0
Babel 2.6.0
backcall 0.1.0
backports.os 0.1.1
backports.shutil-get-terminal-size 1.0.0
beautifulsoup4 4.7.1
bitarray 0.8.3
bkcharts 0.2
black 19.3b0
bleach 3.1.0
blpapi 3.13.0
bokeh 1.0.4
boto 2.49.0
Bottleneck 1.2.1
certifi 2019.3.9
cffi 1.11.5
chardet 3.0.4
Click 7.0
cloudpickle 0.8.0
clyent 1.2.2
colorama 0.4.1
colour 0.1.5
contextlib2 0.5.5
coverage 4.5.3
cryptography 2.6.1
cycler 0.10.0
Cython 0.29.6
cytoolz 0.9.0.1
dask 1.1.4
decorator 4.4.0
defusedxml 0.5.0
distributed 1.26.0
dnspython 1.16.0
docutils 0.14
dominate 2.3.5
entrypoints 0.3
et-xmlfile 1.0.1
eventlet 0.24.1
fastcache 1.0.2
filelock 3.0.10
flake8 3.7.7
Flask 1.0.2
Flask-API 1.0
Flask-Bootstrap 3.3.7.1
Flask-Login 0.4.1
Flask-Negotiate 0.1.0
Flask-RESTful 0.3.6
Flask-SocketIO 3.0.1
Flask-WTF 0.14.2
gevent 1.3.4
glob2 0.6
gmpy2 2.0.8
greenlet 0.4.15
h5py 2.9.0
heapdict 1.0.0
html5lib 1.0.1
idna 2.6
imageio 2.5.0
imagesize 1.1.0
importanize 0.7.0
importlib-metadata 0.18
ipykernel 5.1.0
ipython 7.4.0
ipython-genutils 0.2.0
ipywidgets 7.4.2
isort 4.3.16
itsdangerous 1.1.0
jdcal 1.4
jedi 0.13.3
Jinja2 2.10
joblib 0.13.0
jsonschema 3.0.1
jupyter 1.0.0
jupyter-client 5.2.4
jupyter-console 6.0.0
jupyter-core 4.4.0
jupyterlab 0.35.4
jupyterlab-server 0.2.0
keyring 18.0.0
kiwisolver 1.0.1
lazy-object-proxy 1.3.1
libarchive-c 2.8
lief 0.9.0
llvmlite 0.28.0
locket 0.2.0
lxml 4.3.2
MarkupSafe 1.1.1
marshmallow 3.0.0b17
matplotlib 2.2.2
mccabe 0.6.1
mistune 0.8.4
mkl-fft 1.0.10
mkl-random 1.0.2
monotonic 1.5
more-itertools 6.0.0
mpmath 1.1.0
msgpack 0.6.1
multipledispatch 0.6.0
nbconvert 5.4.1
nbformat 4.4.0
networkx 2.2
nltk 3.4
nose 1.3.7
notebook 5.7.8
numba 0.43.1
numexpr 2.6.9
numpy 1.14.5
numpydoc 0.8.0
olefile 0.46
openpyxl 2.6.1
packaging 19.0
pandas 0.23.2
pandocfilters 1.4.2
parso 0.3.4
partd 0.3.10
path.py 11.5.0
pathlib2 2.3.3
patsy 0.5.0
pep8 1.7.1
pexpect 4.6.0
pickleshare 0.7.5
Pillow 5.4.1
pip 19.0.3
pkginfo 1.5.0.1
pluggy 0.12.0
ply 3.11
progressbar 2.5
prometheus-client 0.6.0
prompt-toolkit 2.0.9
psutil 5.6.1
psycopg2-binary 2.7.7
ptvsd 4.2.10
ptyprocess 0.6.0
py 1.8.0
pycodestyle 2.5.0
pycosat 0.6.3
pycparser 2.19
pycrypto 2.6.1
pycurl 7.43.0.2
pyflakes 2.1.1
Pygments 2.3.1
pymonetdb 1.1.1
pyodbc 4.0.26
pyOpenSSL 19.0.0
pyparsing 2.3.1
pyrsistent 0.14.11
PySocks 1.6.8
pytest 4.6.3
pytest-arraydiff 0.3
pytest-astropy 0.5.0
pytest-cov 2.6.1
pytest-doctestplus 0.3.0
pytest-dotenv 0.4.0
pytest-env 0.6.2
pytest-openfiles 0.3.2
pytest-remotedata 0.3.1
python-dateutil 2.7.3
python-dotenv 0.10.2
python-engineio 3.7.0
python-socketio 4.0.3
pytz 2018.9
PyWavelets 1.0.2
PyYAML 5.1
pyzmq 18.0.0
QtAwesome 0.5.7
qtconsole 4.4.3
QtPy 1.7.0
requests 2.18.4
rogues 0.5.0
rope 0.12.0
ruamel-yaml 0.15.46
scikit-image 0.14.2
scikit-learn 0.20.3
scipy 1.1.0
seaborn 0.9.0
Send2Trash 1.5.0
setuptools 40.8.0
simplegeneric 0.8.1
singledispatch 3.4.0.3
six 1.12.0
snakeviz 2.0.0
snowballstemmer 1.2.1
sortedcollections 1.1.2
sortedcontainers 2.1.0
soupsieve 1.8
sparseqr 1.0.0
Sphinx 1.8.5
sphinxcontrib-websupport 1.1.0
spyder 3.3.3
spyder-kernels 0.4.2
SQLAlchemy 1.3.1
statsmodels 0.9.0
sympy 1.3
tables 3.5.1
tblib 1.3.2
terminado 0.8.1
testpath 0.4.2
toml 0.10.0
toolz 0.9.0
tornado 6.0.2
tqdm 4.31.1
traitlets 4.3.2
unicodecsv 0.14.1
urllib3 1.22
visitor 0.1.3
wcwidth 0.1.7
webargs 5.1.2
webencodings 0.5.1
Werkzeug 0.14.1
wheel 0.33.1
widgetsnbextension 3.4.2
wrapt 1.11.1
WTForms 2.2.1
wurlitzer 1.0.2
xlrd 1.2.0
XlsxWriter 1.0.5
xlwings 0.15.4
xlwt 1.3.0
xmlrunner 1.7.7
zict 0.1.4
zipp 0.5.1
- OS:
MAC OS Mojave == 10.14.5
Sample output from the tests by running:
$pytest -o junit_family=xunit2 --junitxml=test_sum.xml backend/tests/
<?xml version="1.0" encoding="utf-8"?><testsuite errors="0" failures="0" name="pytest" skipped="0" tests="8" time="152.617"><testcase classname="tests.backend.app.core.XXX" name="test_run_from_csv" time="29.087"></testcase><testcase classname="tests.backend.app.core.XXX" name="test_run" time="19.895"><system-err>
....
</testsuite>
nazariydolfin
All 10 comments
also tried -o="junit_family=xunit2"
nazariydolfin
on 24 Jun 2019
Hi @nazariydolfin, thanks for posting.
Can you point out where that violates the schema? We are using https://github.com/jenkinsci/xunit-plugin/blob/master/src/main/resources/org/jenkinsci/plugins/xunit/types/model/xsd/junit-10.xsd IIRC.
nicoddemus
on 24 Jun 2019
Hi, I require a schema like this:
<?xml version="1.0" encoding="UTF-8"?>
<testsuites disabled="" errors="" failures="" name="" tests="" time="">
<testsuite disabled="" errors="" failures="" hostname="" id=""
name="" package="" skipped="" tests="" time="" timestamp="">
<properties>
<property name="" value=""/>
</properties>
<testcase assertions="" classname="" name="" status="" time="">
<skipped/>
<error message="" type=""/>
<failure message="" type=""/>
<system-out/>
<system-err/>
</testcase>
<system-out/>
<system-err/>
</testsuite>
<testsuite ...>
...
</testsuite>
...
</testsuites>
Is there a special set up that I need to implement in tests perhaps for it to generate the .xml report in the above format?
Because I can see that the <testsuites> is defined in your link.
nazariydolfin
on 24 Jun 2019
Is there a special set up that I need to implement in tests perhaps for it to generate the .xml report in the above format?
Unfortunately it seems we don't fully conform with the schema. Not sure how it works though, Jenkins seems to expect that schema and doesn't complain, but it is clear the root of the schema should be <testsuites>, not <testsuite>.
Perhaps @jhunkeler has a comment?
nicoddemus
on 25 Jun 2019
Found another related comment: https://github.com/pytest-dev/pytest/issues/1126#issuecomment-446906944, specially:
The main thing I find missing in the currently output XML is that the top-level testsuites (plural!) tag is missing. The Schema allows for that and ...
(emphasis mine)
This would explain why the current format is accepted by many tools then.
cc @ringods
nicoddemus
on 25 Jun 2019
@nazariydolfin took the liberty to update the issue title now that the problem is more clear, hope that's OK.
nicoddemus
on 25 Jun 2019
Is there an easy hack around this? A workaround on the user side.
nazariyv
on 25 Jun 2019
The modifications I made a while back only extended what was already there, so I'm not surprised the outer level <testsuites> isn't supported. As you can see above JUnit.testsuite(...) (no s) populates the <testsuite> element. So judging from the code I'd say this module never supported generating <testsuites>.
I'll take a closer look at this tonight.
jhunkeler
on 25 Jun 2019
Hi @jhunkeler, how is it looking with this?
nazariyv
on 4 Jul 2019
Related issues
Balky79
路
3Comments
bluenote10
路
3Comments
mika
路
3Comments
reaperhulk
路
3Comments
RonnyPfannschmidt
路
3Comments
Most helpful comment
https://github.com/pytest-dev/pytest/blob/64a63652278d43b99aec5b5a3f36afd220b01f90/src/_pytest/junitxml.py#L661
The modifications I made a while back only extended what was already there, so I'm not surprised the outer level
<testsuites>isn't supported. As you can see aboveJUnit.testsuite(...)(nos) populates the<testsuite>element. So judging from the code I'd say this module never supported generating<testsuites>.I'll take a closer look at this tonight.