Ava: Is there a way to get test status on afterEach() method ?

[subash-canapathy]

mocha has a way of getting the test status on the after* hooks to do customized teardowns. currentTest.status
this is required for UI tests running on saucelabs and that feature in mocha was born out of the same need: https://github.com/mochajs/mocha/issues/797

It would be super useful to have the test status exposed as metadata OR contextual information

Proposal A:
t.context.result - to contain the result object (the one which goes to the reporter)

Proposal B:
t.getTestStatus() - a method available only on the afterEach hook to get status of the test.

---

IssueHunt Summary

bunysae has been rewarded.

Backers (Total: $40.00)

• issuehunt ($40.00)

Submitted pull Requests

- #2429 Add t.passed to execution contexts

Tips

• Checkout the Issuehunt explorer to discover more funded issues.
  

  - Need some help from other developers? Add your repositories on IssueHunt to raise funds.

IssueHunt has been backed by the following sponsors. Become a sponsor

[jamestalmage]

If we do this, it has to be B. You are allowed to completely replace the context:

t.context = someThing()

Also, it would have to be for after.always hooks (we don't run after on failures without that modifier). Note that feature not shipped yet (but should soon).

this is required for UI tests running on saucelabs and that feature in mocha was born out of the same need: mochajs/mocha#797

Three stars for describing a real world need for the feature! ⭐ ⭐ ⭐

[novemberborn]

@subash-canapathy could you provide an example (in Mocha) of how you use this information with your Saucelabs tests? The issue you linked to doesn't provide that kind of context.

[subash-canapathy]

in mocha, when you run UI tests, you typically have the afterEach block handle the session closeouts with sauceLabs depending on the status of the test. For eg:

afterEach(function(){
  // propagate test status/result to sauceLabs
  this.driver.sauceJobStatus(this.currentTest.state)  
  if (this.currentTest.state == 'failed') {
    // do some additional testOpsCleanup if test failed.
  }
  this.driver.quit()
});

Currently we have a work around for this on AVA

test.beforeEach(async t => {
  t.context.testStatus = false;
}

// Flip the flag if test passes - thats only when afterEach gets called
test.afterEach(async t => {
  t.context.testStatus = true;
});

test.afterEach.always(async t => {
  await t.context.driver.quit()
    .finally(function() {
      if (isSauceSuite) {
        return t.context.driver.sauceJobStatus(t.context.testStatus);
      }
    });
});

I mean this works and gets the job done, but as a test runner, i'd rather let AVA tell me if the test has passed or failed, and be able to get that info on the after* hooks. This is essential for conditional teardowns and cleanups.

[novemberborn]

@subash-canapathy great example, thank you. I agree would be good if AVA could provide the plumbing for this.

[cgcgbcbc]

it would have to be for after.always hooks

Since hooks are tests in fact, is there an easy way to achieve this ..?

In addition, this feature means access test "history" in a test, seems that Sequence.run must be modified for passing previous result to hooks run ?

[rhysd]

I want this feature.

Three stars for describing a real world need for the feature!

I'm also currently using ava for E2E testing.
When test failed, I want to take a screenshot of the page before closing browser window. I'm thinking afterEach.always is suitable but there is no way to know the test result.

[jamestalmage]

@rhysd - That is a really cool idea!

[subash-canapathy]

@cgcgbcbc im not sure about Sequence.run, looking at the runner code I thought we could utilize the report callback to additionally put the status/result of the test in the context so that it still works reliably on concurrent tests (sort of like a thread-local result context)

[ngavrish]

Any updates on this guys? Will it be done in some visible perspective?

[frantic1048]

Same question here, I need to log noisy debug information if a test is failing.

Because .afterEach() and afterEach.always() hooks behave differently on test failing, I found a workaround:

import test from 'ava'

test.afterEach('this only runs when test succeed', t => {
  // set a success status in t.context
  t.context = 'success'
})

test.afterEach.always('this always runs', t => {
  if (t.context === 'success') {
    console.log('this test is succeed')
  } else {
    console.log('this test is failed')
    // log my noisy debug info
  }
})

test('success', t => {
  t.is(true, true, '')
})

test('fail', t => {   
  t.is(true, false, '')
})

You can initiate t.context as an Object in test.beforeEach() hook to carry more information. Here I just want to determine whether the test is failing.

NOTE: This workaround is still not reliable if criterias mentioned #928 are met, try to avoid them:

• There are test failures and --fail-fast is used.
• There are uncaught exceptions thrown.

[fruch]

I'm also looking for this option, basicly cause I'm trying to write a plugin to capture my app log, and only print it out when test is failing.

since ava is runing test in paraller, it's even more importent since logging to a file would be a bit more complex

[novemberborn]

@fruch and others, PRs or implementation specs are very much welcome for this feature. I'm happy to give feedback as you familiarize yourself with the problem and the code.

[fruch]

I have something working, based on @frantic1048 idea, and that was good enough for me.

it would be nicer to get the actual status in the test context, so have visibility similar to the test reporter, like test.duration and more.

I understand most of it is internal, and can easily break between version.

is afterEach.always being called also for skipped and todo ? (more ideas come in my head, but most of them are in the reporter category)

[novemberborn]

is afterEach.always being called also for skipped and todo ? (more ideas come in my head, but most of them are in the reporter category)

No it's not.

[lotosotol]

I have a similar issue retrieving the status of the test suite, I am talking about the full test suite's status which I need to pass to BrowserStack (which accepts passed or failed) - would be great to have the ability to retrieve this before closing browser's window either before calling the .quit() method.

Oh, and I am running my tests in Mocha, not AVA and could not find a solution for the issue there as well.

[IssueHuntBot]

@issuehunt has funded $40.00 to this issue.

---

• Submit pull request via IssueHunt to receive this reward.
• Want to contribute? Chip in to this issue via IssueHunt.
• Checkout the IssueHunt Issue Explorer to see more funded issues.
• Need help from developers? Add your repository on IssueHunt to raise funds.

[tryzniak]

Maybe it's worth introducing a new hook? For instance afterFail:

test.afterFail('only runs if a test failed', t => {
  // log, do screenshots
})

[bunysae]

@novemberborn
Could you transfer the bounty?

[novemberborn]

Could you transfer the bounty?

@sindresorhus knows how to do that 😄

[issuehunt-app[bot]]

@sindresorhus has rewarded $36.00 to @bunysae. See it on IssueHunt

• :moneybag: Total deposit: $40.00
• :tada: Repository reward(0%): $0.00
• :wrench: Service fee(10%): $4.00