Nightwatch: Nightwatch junit reporter putting system-err message for a testsuite everywhere except for the testsuite with the error

Created on 1 Mar 2017 · 9Comments · Source: nightwatchjs/nightwatch

Node: v7.5.0
NightwatchJs: version 0.8.2
OS: Windows 10 Enterprise
Selenium: selenium-server-standalone-3.0.1

I am currently having a problem with Nightwatch not saving the system error in the right junit xml file. I have one of the later testsuites during a multi testsuite run fail with a system error. It seems that the error does not appear in the testsuite xml that has the system error but instead saves to the other testsuite xmls instead. Here are two example xml files (replaced items with generic descriptions) from a 3 testsuite run where Testsuite 3 gets an error at Testcase 3. I am not using a custom reporter for these results, I am using the built in junit reporter.

Testsuite 3

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites errors="1"
            failures="0"
            tests="7">

  <testsuite name="Testsuite 3"
    errors="1" failures="0" hostname="" id="" package="Testsuite 3" skipped="4"
    tests="7" time="6.429" timestamp="Wed, 01 Mar 2017 17:27:02 GMT">

    <testcase name="Testcase 1" time="3.529" assertions="5">










    </testcase>

    <testcase name="Testcase 2" time="2.890" assertions="6">












    </testcase>

    <testcase name="Testcase 3" time="0.01000" assertions="0">
    </testcase>

         <!-- Expected a system-err because the error happens in Testcase 3, however it does not show here. Note: The errors count does reflect an error has occurred in this test -->

    <testcase
      name="Testcase 4">
      <skipped />
    </testcase>

    <testcase
      name="Testcase 5">
      <skipped />
    </testcase>

    <testcase
      name="Testcase 6">
      <skipped />
    </testcase>

    <testcase
      name="Testcase 7">
      <skipped />
    </testcase>


  </testsuite>
</testsuites>

Testsuite 1

<?xml version="1.0" encoding="UTF-8" ?>
<testsuites errors="0"
            failures="0"
            tests="6">

  <testsuite name="Testsuite 1"
    errors="0" failures="0" hostname="" id="" package="Testsuite 1" skipped="0"
    tests="6" time="13.09" timestamp="Wed, 01 Mar 2017 17:26:35 GMT">

    <testcase name="Testcase 1" time="5.181" assertions="5">










    </testcase>


        <!-- Do not expect the system-err because the system-err happens in Testsuite 3, not here in Testsuite 1. Note: Error count here is 0, yet we have system-err elements within this file. -->


      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>

    <testcase name="Testcase 2" time="2.813" assertions="7">














    </testcase>
      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>

    <testcase name="Testcase 3" time="1.113" assertions="6">












    </testcase>
      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>

    <testcase name="Testcase 4" time="0.5870" assertions="5">










    </testcase>
      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>

    <testcase name="Testcase 5" time="1.480" assertions="7">














    </testcase>
      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>

    <testcase name="Testcase 6" time="1.913" assertions="5">










    </testcase>
      <system-err>
          Error while running [Testsuite 3  / Testcase 3]:

  Error: aTag was not found in &quot;pageobject&quot;. Available elements: favorite,addToFavorites,removeFromFavorites,albums,albumInput,createAlbum,removeFromFirstAlbum,addToOnlyAlbum,addToFirstAlbum,tag,tagInput,createTag,makePublic,firstUserPrivateTag,firstUserPublicTag,header
    at getElement (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:13:13)
    at Page.waitForElementPresent (C:\projects\node_modules\nightwatch\lib\page-object\command-wrapper.js:81:32)
    at Object.Testcase 3 (C:\projects\tests\Testsuite 3.js:52:5)
    at Module.call (C:\projects\node_modules\nightwatch\lib\runner\module.js:62:34)
    at C:\projects\node_modules\nightwatch\lib\runner\testcase.js:70:29
    at _fulfilled (C:\projects\node_modules\q\q.js:834:54)
    at self.promiseDispatch.done (C:\projects\node_modules\q\q.js:863:30)
    at Promise.promise.promiseDispatch (C:\projects\node_modules\q\q.js:796:13)
    at C:\projects\node_modules\q\q.js:556:49
    at runSingle (C:\projects\node_modules\q\q.js:137:13)
      </system-err>


  </testsuite>
</testsuites>

I'm not really sure what the problem is, but I think it is a bug with Nightwatch.

bug stale

Source

Harrison-Jue

Most helpful comment

@Harrison-Jue Yes it is, although not very active recently. Issues are piling up here but I am working on a major 1.0 release of nightwatch, and also working on nightcloud.io as well.

beatfactor on 10 Mar 2017

🎉2 👍2

All 9 comments

I don't think nightwatch works with node v7 as some dependencies need updating:
https://github.com/nightwatchjs/nightwatch/issues?utf8=%E2%9C%93&q=node%20v7
https://github.com/beatfactor/mocha/issues/5
https://github.com/beatfactor/mocha/pull/2

peter-mouland on 9 Mar 2017

@peter-mouland I tried using older versions of node, node 6.7, 6.4, 5.7, 5.4, and 4.7, that didn't change the output though.

On a somewhat related note - does anyone know if Nightwatch is being actively maintained? The project doesn't seem very active and my company is obviously concerned about committing to using Nightwatch long term if it is not going to be maintained \ further developed.

Harrison-Jue on 10 Mar 2017

@Harrison-Jue Yes it is, although not very active recently. Issues are piling up here but I am working on a major 1.0 release of nightwatch, and also working on nightcloud.io as well.

beatfactor on 10 Mar 2017

🎉2 👍2

@beatfactor Thanks for the quick response, I can't wait to see nightwatch 1.0.

Harrison-Jue on 10 Mar 2017

This is just a bug in the junit reporter, nothing to do with node v7, as far as I can tell.

beatfactor on 10 Mar 2017

Related: I, too, am getting funky junit reports. However, Instead of the system error appearing in the wrong testsuite.xml, the system error appears in ALL testsuite.xmls, whether the testsuite passed or not.

mvpmvh on 23 Jun 2017

@mvpmvh which nightwatch version?

beatfactor on 24 Jun 2017

Did anybody find a fix or workaround for this?

felipeloha on 12 Oct 2018

This issue has been automatically marked as stale because it has not had any recent activity.
If possible, please retry using the latest Nightwatch version and update the issue with any relevant details. If no further activity occurs, it will be closed. Thank you for your contribution.