Cypress-cucumber-preprocessor: Screenshots added to cucumber.json

[adrianpaterson]

Hi,

I am able to generate the cucumber.json without any problem and have used multiple-cucumber-html-reporter to generate a great cucumber report off the back of this json.

The only thing i am missing is the ability to attach screenshots to the cucumber.json. I am looking for something similar to the mochawsome addContext functionality.

Is this currently possible? (All other reporters dont appear to generate json in the correct format for using to generate cucumber reports)

Cheers.

[foxhound91]

Hi @adrianpaterson, please check this comment https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/issues/155#issuecomment-508904086 from @jcundill I know he was working on implementing something similar

[DanielStoica85]

Hi @adrianpaterson,

Any chance you might provide an example on how you got multiple-cucumber-html-reporter to work with cypress-cucumber-preprocessor, even without screenshots? I am struggling with it and would appreciate the help.

[Lukaszli]

@DanielStoica85 If you already have cypress-cucumber-preprocessor you need to add to your package.json additional code to generate JSON output from cypress-cucumber-preprocessor.

"cypress-cucumber-preprocessor": { "cucumberJson": { "generate": true, "outputFolder": "results/cucumber-json", "filePrefix": "", "fileSuffix": ".cucumber", } },

than you need to create a file with the runner, you can find it in docs of multiple-cucumber-html-reporter:

const report = require('multiple-cucumber-html-reporter');

report.generate({
  jsonDir: 'results/cucumber-json/',
  reportPath: 'results/report.html',
  openReportInBrowser: true,
  metadata: {
    browser: {
      name: 'chrome',
      version: '60',
    },
    device: 'Local test machine',
    platform: {
      name: 'ubuntu',
      version: '16.04',
    },
  },
  customData: {
    title: 'Run info',
    data: [
      { label: 'Project', value: 'Custom project' },
      { label: 'Release', value: '1.2.3' },
      { label: 'Cycle', value: 'B11221.34321' },
      { label: 'Execution Start Time', value: 'Nov 19th 2017, 02:31 PM EST' },
      { label: 'Execution End Time', value: 'Nov 19th 2017, 02:56 PM EST' },
    ],
  },
});

and after finishing tests you need to execute runner using node
node <name_of_your_runner_file.js

[mohitsingla1]

Skip Scenarios Data Missing in cucumber.json
Anybody having an idea

[nathfw]

someone who managed to insert the screenshot next to the multiple cucumber html report?

[webnexusmobile]

@nathfw Hi, I built a working example based on comment https://github.com/TheBrainFamily/cypress-cucumber-preprocessor/issues/155#issuecomment-508904086 from @jcundill. There might be more elegant solutions, but it does the job. You can find it here: https://github.com/webnexusmobile/cypress-cucumber-getting-started

[Phenomite]

Hi, I built a working example based on comment #155

Great solution, however the overarching regex fails when you perform cy.screenshot() and have non (failed) etc screenshots present.

Ill try adding this support myself otherwise just letting you know so maybe you'd want to add it to yours yourself.

Regards.

[pvishnu991]

Is there a workaround to include vedios and screenshots for the test cases which are passed as well to the report????

Report used: "multiple-cucumber-html-reporter": "^1.16.2"
cucumber-report.js file which contains the current code to attach the screenshots on failures
-----------------Code----------------------------------------------------------------------------------
const report = require('multiple-cucumber-html-reporter')
const fs = require('fs-extra')
const path = require('path')
const chalk = require('chalk')
const args = process.argv;

const cucumberJsonDir = './cypress/cucumber-json'
const cucumberReportFileMap = {}
const cucumberReportMap = {}
const jsonIndentLevel = 2
const ReportDir = './cypress/reports/cucumber-report'
const screenshotsDir = './cypress/screenshots'

getCucumberReportMaps()
addScreenshots()
generateReport()

//Mapping cucumber json files from the cucumber-json directory to the features
function getCucumberReportMaps() {
const files = fs.readdirSync(cucumberJsonDir).filter(file => {
return file.indexOf('.json') > -1
})
files.forEach(file => {
const json = JSON.parse(
fs.readFileSync(path.join(cucumberJsonDir, file))
)
if (!json[0]) { return }
const [feature] = json[0].uri.split('/').reverse()
cucumberReportFileMap[feature] = file
cucumberReportMap[feature] = json
})
}

//Adding screenshots to the respective failed test steps in the feature files
function addScreenshots() {

const prependPathSegment = pathSegment => location => path.join(pathSegment, location)

const readdirPreserveRelativePath = location => fs.readdirSync(location).map(prependPathSegment(location))

const readdirRecursive = location => readdirPreserveRelativePath(location)
.reduce((result, currentValue) => fs.statSync(currentValue).isDirectory()
? result.concat(readdirRecursive(currentValue))
: result.concat(currentValue), [])
const screenshots = readdirRecursive(path.resolve(screenshotsDir)).filter(file => {
return file.indexOf('.png') > -1
})

const featuresList = Array.from(new Set(screenshots.map(x => x.match(/[\w-_.]+.feature/g)[0])))
featuresList.forEach(feature => {
screenshots.forEach(screenshot => {

  const regex = /(?<=--\ ).+?((?=\ \(example\ #\d+\))|(?=\ \(failed\))|(?=\.\w{3}))/g
  const [scenarioName] = screenshot.match(regex)
  console.info(chalk.blue('\n    Adding screenshot to cucumber-json report for'))
  console.info(chalk.blue(`    '${scenarioName}'`))

  const myScenarios = cucumberReportMap[feature][0].elements.filter(
    e => scenarioName.includes(e.name)
  )
  if (!myScenarios) { return }
  let foundFailedStep = false
  myScenarios.forEach(myScenario => {
    if (foundFailedStep) {
      return
    }
    let myStep
    if (screenshot.includes('(failed)')) {
      myStep = myScenario.steps.find(
        step => step.result.status === 'failed'
      )
    } else {
      myStep = myScenario.steps.find(
        step => step.name.includes('screenshot')
      )
    }
    if (!myStep) {
      return
    }
    const data = fs.readFileSync(
      path.resolve(screenshot)
    )
    if (data) {
      const base64Image = Buffer.from(data, 'binary').toString('base64')
      if (!myStep.embeddings) {
        myStep.embeddings = []
        myStep.embeddings.push({ data: base64Image, mime_type: 'image/png' })
        foundFailedStep = true
      }
    }
  })
  //Write JSON with screenshot back to report file.
  fs.writeFileSync(
    path.join(cucumberJsonDir, cucumberReportFileMap[feature]),
    JSON.stringify(cucumberReportMap[feature], null, jsonIndentLevel)
  )
})

})
}

//Report generation with customized meta data included
function generateReport() {
if (!fs.existsSync(cucumberJsonDir)) {
console.warn(chalk.yellow(WARNING: Folder './${cucumberJsonDir}' not found. REPORT CANNOT BE CREATED!))
} else {
report.generate({
jsonDir: cucumberJsonDir,
reportPath: ReportDir,
saveCollectedJSON: true,
displayDuration: true,
reportName: Regression Test Report - ${new Date().toLocaleString()},
metadata: {
browser: {
name: 'chrome'
},
device: 'LM',
platform: {
name: 'Windows'
}
}
})
}
}

[panraj01]

Hi, I built a working example based on comment #155

Great solution, however the overarching regex fails when you perform cy.screenshot() and have non (failed) etc screenshots present.

Ill try adding this support myself otherwise just letting you know so maybe you'd want to add it to yours yourself.

Regards.

Hi - I am facing the same issue. Did you find a way around this ?

Many thanks.

[jradom]

@panraj01 Did you find any way to attach non-failed (on-demand added) screenshots to a report?

[panraj01]

@panraj01 Did you find any way to attach non-failed (on-demand added) screenshots to a report?

Actually I did not for a short time and then i got into something else. I will get back to this when i get back to working on reporting, in the mean time if someone from this great community gets a solution, please share.

[gretherMG]

Anyone know how to generate cucumber-json reports with screenshot and also cucumber html reports with screenshot please, I would be very grateful?
of the suggestions that I have read, none have generated the cucumber-json report, nor the html with screenshots.

please

[panraj01]

@gretherMG Are you not able to

1 - generate cucumber.json or

2 - not able to generate html reports with cucumber.json or

3 - not able to add screenshot to failed test cases on the html report?

[gretherMG]

Hi @panraj01
The cucumber-json report is generated without screenshots, just like the html report. I have tried the solutions that they have indicated, but I can't do it, I don't know if someone has a clearer example, please,
I would be very grateful.

[gretherMG]

any solution please, someone managed to add screenshot to cucumber json, any ideas?

[panraj01]

Paste it as is - this works for screenshot on failure, i have modified it per my need, you can modify for yours.
Note: I was modifying it further to capture screenshot at will (not only failure) but i moved on to something else, you might find some lines that aren't needed to capture screenshot on failure.

const report = require('multiple-cucumber-html-reporter')
const fs = require('fs-extra')
const path = require('path')
const chalk = require('chalk')

const cucumberJsonDir = path.resolve(process.cwd(), "cypress/cyreport/cucumber-json") //'./cypress/test-results/cucumber-json'
const cucumberReportFileMap = {}
const cucumberReportMap = {}
const jsonIndentLevel = 2
const htmlReportDir = path.resolve(process.cwd(), "cypress/cyreport/htmlReport") //'./cypress/test-results/cucumber-json'./src/cyreport/html'
const screenshotsDir = path.resolve(process.cwd(), "cypress/cyreport/screenshots")
//const snapshotDir = './cypress/snapshots'

getCucumberReportMaps()
addScreenshots()
//addSnapshots()
generateReport()

function getCucumberReportMaps() {
filenames = fs.readdirSync(cucumberJsonDir);
const files = fs.readdirSync(cucumberJsonDir).filter(file => {
return file.indexOf('.json') > -1
})
files.forEach(file => {
const json = JSON.parse(
fs.readFileSync(path.join(cucumberJsonDir, file))
)
if (!json[0]) { return }
const [feature] = json[0].uri.split('/').reverse()
cucumberReportFileMap[feature] = file
cucumberReportMap[feature] = json
})
}

function addScreenshots() {
if (fs.existsSync(screenshotsDir)) { //only if screenshots exists
const prependPathSegment = pathSegment => location => path.join(pathSegment, location)
const readdirPreserveRelativePath = location => fs.readdirSync(location).map(prependPathSegment(location))
const readdirRecursive = location => readdirPreserveRelativePath(location)
.reduce((result, currentValue) => fs.statSync(currentValue).isDirectory()
? result.concat(readdirRecursive(currentValue))
: result.concat(currentValue), [])
const screenshots = readdirRecursive(path.resolve(screenshotsDir)).filter(file => {
return file.indexOf('.png') > -1
})
const featuresList = Array.from(new Set(screenshots.map(x => x.match(/[\w-_.]+.feature/g)[0])))
featuresList.forEach(feature => {
screenshots.forEach(screenshot => {
const regex = /(?<=--\ ).+?((?=\ (example\ #\d+))|(?=\ (failed))|(?=.\w{3}))/g
const [scenarioName] = screenshot.match(regex)
const myScenarios = cucumberReportMap[feature][0].elements.filter(
e => scenarioName.includes(e.name)
)
if (!myScenarios) {
return
}
let foundFailedStep = false
myScenarios.forEach(myScenario => {
if (foundFailedStep) {
return
}
let myStep
if (screenshot.includes('(failed)')) {
myStep = myScenario.steps.find(
step => step.result.status === 'failed'
)
}
else {
myStep = myScenario.steps.find(
step => step.result.status === 'passed'
)
}
if (!myStep) {
return
}
const data = fs.readFileSync(
path.resolve(screenshot)
)
if (data) {
const base64Image = Buffer.from(data, 'binary').toString('base64')
if (!myStep.embeddings) {
myStep.embeddings = []
myStep.embeddings.push({ data: base64Image, mime_type: 'image/png' })
foundFailedStep = true
}
}
})
//Write JSON with screenshot back to report file.
fs.writeFileSync(
path.join(cucumberJsonDir, cucumberReportFileMap[feature]),
JSON.stringify(cucumberReportMap[feature], null, jsonIndentLevel)
)
})
})
}
}

function generateReport() {
if (!fs.existsSync(cucumberJsonDir)) {
console.warn(chalk.yellow(WARNING: Folder './${cucumberJsonDir}' not found. REPORT CANNOT BE CREATED!))
} else {
report.generate({
jsonDir: cucumberJsonDir,
reportPath: htmlReportDir,
displayDuration: true,
useCDN: true,
pageTitle: 'Your App Name here',
reportName: App Name - ${new Date().toLocaleString()},
metadata: {
app: {
name: '103-Mobile',
version: '1.70.x'
},
browser: {
name: 'chrome'
},
device: 'EMULATOR',
platform: {
name: 'windows'
}
},
customData: {
title: 'Run info',
data: [
{ label: 'Project', value: 'CyMobileE2E' },
{ label: 'Release', value: '20.2' },
{ label: 'Cycle', value: 'I1' },
{ label: 'Execution Start Time', value: 'Sept 19th 2020, 02:31 PM EST' },
{ label: 'Execution End Time', value: 'Sept 19th 2020, 02:56 PM EST' }
]
}
})
}
}

[gretherMG]

thanks for the prompt reply but it shows me this error

[panraj01]

It is saying you have no screenshot captured to iterate through the array. Remember this utility attaches no screenshot for passed scenario. It only attaches screenshot to the report when there is a failure. Do you have screenshot in your screenshot folders for the utility to attach it to html report?

[gretherMG]

If I have a screenshot, there is something strange because after re-executing the command to generate the report, it tells me Error: ENOTDIR: not a directory, scandir 'D: \ Users, however the address of the folder where it is the report is correct. I'll find a way to see why this error. Thanks

[gretherMG]

Hi @panraj01 Thanks for your help, I modified the code a bit and it worked for me, here I leave my code in case you need another person. Greetings

const report = require("multiple-cucumber-html-reporter");
const fs = require("fs-extra");
const path = require("path");

const cucumberJsonDir = path.resolve(process.cwd(), "cypress/cucumber-json");
const cucumberReportFileMap = {};
const cucumberReportMap = {};
const jsonIndentLevel = 2;
const htmlReportDir = path.resolve(process.cwd(), "cypress/cucumber-json");
const screenshotsDir = path.resolve(process.cwd(), "cypress/screenshots");

getCucumberReportMaps();
addScreenshots();
generateReport();

function getCucumberReportMaps() {
  filenames = fs.readdirSync(cucumberJsonDir);
  const files = fs.readdirSync(cucumberJsonDir).filter((file) => {
    return file.indexOf(".json") > -1;
  });
  files.forEach((file) => {
    const json = JSON.parse(fs.readFileSync(path.join(cucumberJsonDir, file)));
    if (!json[0]) {
      return;
    }
    const [feature] = json[0].uri.split("/").reverse();
    cucumberReportFileMap[feature] = file;
    cucumberReportMap[feature] = json;
  });
}

function addScreenshots() {
  if (fs.existsSync(screenshotsDir)) {
    //only if screenshots exists
    const prependPathSegment = (pathSegment) => (location) =>
      path.join(pathSegment, location);

    const readdirPreserveRelativePath = (location) =>
      fs.readdirSync(location).map(prependPathSegment(location));

    const readdirRecursive = (location) =>
      readdirPreserveRelativePath(location).reduce(
        (result, currentValue) =>
          fs.statSync(currentValue).isDirectory()
            ? result.concat(readdirRecursive(currentValue))
            : result.concat(currentValue),
        []
      );

    const screenshots = readdirRecursive(path.resolve(screenshotsDir)).filter(
      (file) => {
        return file.indexOf(".png") > -1;
      }
    );

    const featuresList = Array.from(
      new Set(screenshots.map((x) => x.match(/[\w-_.]+.feature/g)[0]))
    );

    featuresList.forEach((feature) => {
      screenshots.forEach((screenshot) => {
        const regex = /(?<=--\ ).+?((?=\ (example\ #\d+))|(?=\ (failed))|(?=.\w{3}))/g;
        const [scenarioName] = screenshot.match(regex);

        var filename = screenshot.replace(/^.*[\\\/]/, "");

        const featureSelected = cucumberReportMap[feature][0];

        let myScenarios = [];

        cucumberReportMap[feature][0].elements.forEach((item) => {
          let fullFileName = featureSelected.name + " -- " + item.name;
          if (filename.includes(fullFileName)) {
            myScenarios.push(item);
          }
        });

        if (!myScenarios) {
          return;
        }
        let foundFailedStep = false;
        myScenarios.forEach((myScenario) => {
          if (foundFailedStep) {
            return;
          }
          let myStep;
          if (screenshot.includes("(failed)")) {
            myStep = myScenario.steps.find(
              (step) => step.result.status === "failed"
            );
          } else {
            myStep = myScenario.steps.find(
              (step) => step.result.status === "passed"
            );
          }
          if (!myStep) {
            return;
          }
          const data = fs.readFileSync(path.resolve(screenshot));
          if (data) {
            const base64Image = Buffer.from(data, "binary").toString("base64");
            if (!myStep.embeddings) {
              myStep.embeddings = [];
              myStep.embeddings.push({
                data: base64Image,
                mime_type: "image/png",
                name: myStep.name,
              });
              foundFailedStep = true;
            }
          }
        });
        //Write JSON with screenshot back to report file.
        fs.writeFileSync(
          path.join(cucumberJsonDir, cucumberReportFileMap[feature]),
          JSON.stringify(cucumberReportMap[feature], null, jsonIndentLevel)
        );
      });
    });
  }
}

function generateReport() {
  if (!fs.existsSync(cucumberJsonDir)) {
    console.warn("REPORT CANNOT BE CREATED!");
  } else {
    report.generate({
      jsonDir: cucumberJsonDir,
      reportPath: htmlReportDir,
      displayDuration: true,
      useCDN: true,
      pageTitle: "Simulacion de Credito Online",
      reportName: `Simulacion de Credito Online - ${new Date().toLocaleString()}`,
      metadata: {
        app: {
          name: "Simulacion de Credito Online",
          version: "1",
        },
        browser: {
          name: "electron",
        },
        device: "EMULATOR",
        platform: {
          name: "linux",
        },
      },
      customData: {
        title: "Run info",
        data: [
          { label: "Project", value: "Simulacion de Credito" },
          { label: "Release", value: "1" },
          {
            label: "Execution Start Time",
            value: `${new Date().toLocaleString()}`,
          },
          {
            label: "Execution End Time",
            value: `${new Date().toLocaleString()}`,
          },
        ],
      },
    });
  }
}

[panraj01]

Way to go, Awesome. Happy to help and thanks for sharing it for anyone else who has to go through the challenges you've had.

[redReno]

@gretherMG This really helped me out. Thanks! I had to adjust it a bit to fit our folders and feature names but it wasn't too much trouble.

[tongilcoto]

Hi @panraj01 Thanks for your help, I modified the code a bit and it worked for me, here I leave my code in case you need another person. Greetings

Thanks a lot for both @panraj01 and @gretherMG

I just needed to update .json files but I also got the .html report seamlessly

[grethermendezgomez]

Now a rather interesting query, how to use a CSV file as a scenario schema data source? Have you had that experience? I've been trying to implement it for weeks.

[panraj01]

I dont know what do you mean by scenario schema data source but if you want your test to look into certain data file (csv in your case) then you need to use a csv/xl reader library and then create a task that uses this 3rd party library, write it to another location that cypress can read.

I use a JSON format txt file that can be uploaded by the tester for testing a client, when the run starts, i look for that specific file if present my task copies the content to a know location in cypress fixture that i use in my test downstream.

Again, i did not understand your ask, i just gave you an example of a use case i had worked on.

Happy testing.

[grethermendezgomez]

Hi, @panraj01

This is what I mean where it says examples instead of burning the data, consume it directly from a json or from a csv

Scenario Outline: buy a product x and simulate its value
When submitting the form filled with "", "", "" required
And in section x reflected in the table the value is usd ""
And in table x is reflected "" and a value ""

Examples:
| prodname | subproduct | cost | interest | quota |
| iphone | 12 pro | 2000 | 1.4 | 12.50 |
| nokia | 1 plus | 350 | 10.2 | 25 |

greetings, hopefully we can find a solution among all ☺

[grethermendezgomez]

[panraj01]

Ah i get it now. You dont want to keep changing the data table entries - Well, although i understand you want to read the table off of a csv and feed csv data for each run (did i get that right?) the best way i would handle this is - Not with csv, you want to test your app logic and not specific numbers.

Keep the prod name and subproduct in the table as it is.
You want to test the calculation logic your front end or back end has - its the cost, interest and quota that would keep changing per your app calculation logic. To not burn the same information, i would try to keep the numbers dynamic for each run - in your step definition you can use something like faker lib, feed fresh numbers every time in your step definition (you can save them in alias as well if you want to access them any where downstream) and calculate what should show up in your UI's interest fields say for ex and assert the value in that field is per what you calculated with your alias numbers.

Not sure if it helps but this is my understanding of your issue.

[grethermendezgomez]

@panraj01
I finally succeeded, hehe here I leave the solution in case someone needs it, it is not the most elegant but it works.
Scenario Outline: buy a product x and simulate its value
When submitting the form filled with prodname, subproduct and cost required
|urljson|
|data/dataclient|
And in section x reflected in the table the value is usd value
|urljson|
|data/dataclient|
And in table x is reflected interest and a value quota
|urljson|
|data/dataclient|

Examples:
|urljson |
||
||

in the sequence of steps you get
Given("A client accessing the credit simulation page", () => {
cy.VisitUrl(500);
});
When("submitting the form filled with prodname, subproduct and cost required",(dataTable) => {
cy.ValuesToEnter(dataTable);
});

then in the code it is called like this

Cypress.Commands.add("ValuesToEnter", (dataTable) => {
dataTable.hashes().forEach(elem =>{
cy.fixture(elem.urljson).as("data");
cy.get("@data").then((valor) => {
valor.forEach(element => {
cy.get(Simulation.nameproduct).select(element.nameProduct);
cy.get(Simulation.subproduct).select(element.subProduct);
});
});
});
});

and so I can read from a json file using outline scenarios. I hope it will be of great help to others and if you have a better way of doing it, leave it here. ☺

[grethermendezgomez]