Cypress: Add failed screenshot to test information for custom reporter

[Frozenfire92]

Current behavior:

When writing a custom reporter you have access to some information from the test, but no indication of the screenshot taken automatically on failure.

runner.on('test end', function(test) {
  console.log(test);
});

Desired behavior:

the test object to have the screenshot path

runner.on('test end', function(test) {
  console.log(test.screenshot);
  // or
  console.log(test.failedScreenshot);
});

Steps to reproduce: (app code and test code)

simple runner reporters/test.js

module.exports = function(runner, config) {
  runner.on('test end', function(test) {
    console.log('test done');
  });
}

use in cypress.json

{
  "reporter": "reporters/test.js"
}

Versions

Cypress 3.3.1
OS: Linux (Ubuntu)
Browser: Any

Workaround

You can access the files with fs and path no problem in the end hook. This is less than ideal as you have to try and map file names to your failures

runner.on('end', function() {
  //getFilePaths is a helper to list all files in a directory
  console.log('end:', getFilePaths('cypress/screenshots'));
});

[jennifer-shehane]

Have you taken a look at the after:screenshot api? https://on.cypress.io/after-screenshot-api#Usage

Also, could you explain a bit about what you want to achieve in the end? What do you want to do with the screenshots? We might have a recommendation if we can understand the end result you want.

[Frozenfire92]

@jennifer-shehane From my understanding that event is only available in the cypress context (i.e. a plugin) and not a reporter?

The end goal is to upload it to something like TestRail and tag it to the test case / test run. I can do this as described above, but its a decent amount of manual work trying to tag the failed screenshot to the failed test instance

[Shelex]

@jennifer-shehane Yeah, have added screenshots to mochawesome using mapping filename approach and it was a long road of defining how screenshot names behave with special symbols (some are supported, other deleted) and after switch from 3.2 to 3.3 it has changed ><
So it would be great to have it accessible inside test object.

[KevinHerklotz]

Yes I'm struggling with the same issue. I want to map the screenshot to the failed test inside my mochawesome report, but it seems to be impossible.
Either I need the screenshot filenames inside the test object (like you proposed) or I need access to the test object inside the after:screenshot callback.

In the meantime I'm using a workaround described here: https://github.com/cypress-io/cypress/issues/1910#issuecomment-514067968

[mateatslc]

this is how I improved on the workaround mentioned above:

const addContext = require('mochawesome/addContext');

Cypress.Screenshot.defaults({
    onAfterScreenshot($el, props) {
        document.querySelector('html').setAttribute('screenshot-path', props.path);
    },
});

Cypress.on('test:after:run', (test) => {
    if (test.state === 'failed') {
        const screenshotPath = document
            .querySelector('html')
            .getAttribute('screenshot-path');
        const reporterDirPath = Cypress.config()['reporterOptions']['reportDir'];

        addContext({ test }, `${screenshotPath.split(`${reporterDirPath}/`)[1]}`);
    }
});

[irfancharania]

@mateatslc: I'm not able to get your solution working in my support/index.js. Inspecting the HTML element after a failed run shows no attribute has been added, and the screenshotPath returns null.

Currently, I'm doing this (but this doesn't work for failures in before/after hooks...):

const addContext = require('mochawesome/addContext');

Cypress.on('test:after:run', (test, runnable) => {
    if (test.state === 'failed') {
        const specName = Cypress.spec.name.replace(':', '').replace('\\', '/');
        const contextName = runnable.parent.title.replace(':', '').replace('\\', '/');
        const specifyName = test.title.replace(':', '').replace('/', '');
        const screenshotFileName = `${contextName} -- ${specifyName} (failed).png`;
        addContext({ test }, `screenshots/${specName}/${screenshotFileName}`);
    }
});

Using versions

"cypress": "^4.0.2",
"mocha": "^5.2.0",
"mocha-multi-reporters": "^1.1.7",
"mochawesome": "^4.1.0",
"mochawesome-merge": "^4.0.0",
"mochawesome-report-generator": "^4.1.0",

[hemche]

@irfancharania Here is the one that works even if there are failures in any of the hooks as well.

Cypress.on('test:after:run', (test, runnable ) => {
if (test.state === 'failed') {
  let item = runnable
  const nameParts = [runnable.title]
  while (item.parent) {
    nameParts.unshift(item.parent.title)
    item = item.parent
  }
  if(runnable.hookName) {
    nameParts.push(`${runnable.hookName} hook`)
    }
  const fullTestName = nameParts.filter(Boolean).join(' -- ')
  const screenshotPath = `cypress/screenshots/${Cypress.spec.name}/${fullTestName} (failed).png`.replace("  "," ")

  addContext({ test }, screenshotPath)
}
})

[irfancharania]

@chemanth: Thanks! I'll give it a shot.

[BERRYL-zlh]

@irfancharania Here is the one that works even if there are failures in any of the hooks as well.

Cypress.on('test:after:run', (test, runnable ) => {
if (test.state === 'failed') {
  let item = runnable
  const nameParts = [runnable.title]
  while (item.parent) {
    nameParts.unshift(item.parent.title)
    item = item.parent
  }
  if(runnable.hookName) {
    nameParts.push(`${runnable.hookName} hook`)
    }
  const fullTestName = nameParts.filter(Boolean).join(' -- ')
  const screenshotPath = `cypress/screenshots/${Cypress.spec.name}/${fullTestName} (failed).png`.replace("  "," ")

  addContext({ test }, screenshotPath)
}
})

@chemanth
that can get hook name but seems could not get hook title if before hook like below
beforeEach('test',()=>{ })

do you know how can get title that can push to nameParts, thanks a lot

[bhagatparwinder]

@irfancharania Here is the one that works even if there are failures in any of the hooks as well.

Cypress.on('test:after:run', (test, runnable ) => {
if (test.state === 'failed') {
  let item = runnable
  const nameParts = [runnable.title]
  while (item.parent) {
    nameParts.unshift(item.parent.title)
    item = item.parent
  }
  if(runnable.hookName) {
    nameParts.push(`${runnable.hookName} hook`)
    }
  const fullTestName = nameParts.filter(Boolean).join(' -- ')
  const screenshotPath = `cypress/screenshots/${Cypress.spec.name}/${fullTestName} (failed).png`.replace("  "," ")

  addContext({ test }, screenshotPath)
}
})

Runnable does not have a hookName property when I am using it.