Serenity-core: Generate Serenity aggregate reports without executing maven goal

[Prasadsimha]

Hi ,

Is it possible to generate Serenity aggregate reports without executing maven goal. What I mean is , when I execute my Serenity runner class ( not Maven goal ) from command line it should create aggregate reports .

Please advise

Regards
Prasad Banala

[wakaleo]

As the name indicates, the "aggregate" command aggregates a collection of test results into a final report. It is very common to have multiple Serenity runner classes, and if aggregation were done after each test, it would be highly inefficient.

[Prasadsimha]

But when we deploy test jar file along with application jar in test
environment , a single class need to be executed ,which will execute all
the stories and produce the report . I agree there could be many story
files and step definition classes , But i don't see a reason to have many
serenity runner classes which will extend serenitystories class ...

like jbehave ,Junitstories class , if serenitystories is able to produce
aggregate reports after the class execution , that will be more help to
deploy the jars in test environments .

Please advise .

Regards
Prasad Banala

On Mon, Jan 4, 2016 at 5:16 PM, John Ferguson Smart <
[email protected]> wrote:

As the name indicates, the "aggregate" command aggregates a collection of
test results into a final report. It is very common to have multiple
Serenity runner classes, and if aggregation were done after each test, it
would be highly inefficient.

—
Reply to this email directly or view it on GitHub
https://github.com/serenity-bdd/serenity-core/issues/244#issuecomment-168828413
.

_Thanks & RegardsPrasad Banala_
_+1-7038685272_

[sudheerpujar]

Hi John,
This looks to me a genuine requirement, can this be done on demand. Like a property in serenity.properties or command line which forces the aggregate report, be default it will be false.

Why this is required

• When I want to run a single or only two three features using JunitCore like calling

        JUnitCore engine = new JUnitCore();
        engine.addListener(new TextListener(System.out));
        engine.run(featureClass);

Where featureClass is fully qualified name of class.

• Relaying features using Jenkins pipe line using following mvn command in each pipe line stages

mvn -pl ${moduleName}  -Dwebdriver.remote.url=${webDriverRemoteUrl} -Dwebdriver.remote.driver=iexplorer  -Dwebdriver.base.url=${webDriverBaseUrl} -Dit.test=VerifyPageTitleForLoginFeature clean verify

[wakaleo]

Generating the reports after each test run would be inefficient and potentially very slow. Why not just bind the aggregate task to the Maven verify or post-integration phase or (if you are doing it programmatically) call the HtmlAggregateStoryReporter directly?

[sudheerpujar]

Thanks @wakaleo , I will try this.

[savita20]

Hi @wakaleo,

How can we see aggregated report when I run my test with '@RunWith(CucumberWithSerenity.class)'. I used @regression tag for my all Test cases and ran my test suite I can't see aggregated report for all test.

Like when we get index.html when we run 'mvn serenity:aggregate' command. And this is possible only when we run our test through Maven.

[wakaleo]

You need to run the maven aggregate goal, which you can do from in your IDE

[hanymorcos]

I did some research. You can run the aggregate goal within a special hook. I struggled with @AfterClass hook, it wouldn't work for me because I wasn't using a Junit runner. Also @AfterStories didn't work for me. I stumbled on the solution below and it works for me now.

https://github.com/cucumber/cucumber-jvm/issues/515#issuecomment-388409511

`package a.b.c;

import cucumber.api.event.EventHandler;
import cucumber.api.event.EventPublisher;
import cucumber.api.event.TestRunFinished;
import cucumber.api.event.TestRunStarted;
import cucumber.api.formatter.Formatter;
import net.thucydides.core.Thucydides;
import net.thucydides.core.reports.html.HtmlAggregateStoryReporter;

import java.io.File;
import java.io.IOException;

public class Initialization implements Formatter {

private EventHandler<TestRunStarted> setup = event -> {
   //  System.out.println("setup!!");
};

private EventHandler<TestRunFinished> teardown = event -> {
    File sourceDirectory = new File("target/site/serenity/");
    HtmlAggregateStoryReporter reporter = new HtmlAggregateStoryReporter(Thucydides.getDefaultProjectKey());
    reporter.setOutputDirectory(sourceDirectory);
    try {
        reporter.generateReportsForTestResultsFrom(sourceDirectory);
    }
    catch (IOException e)
    {
        System.out.println("generateReportsForTestResultsFrom exception ");
        e.printStackTrace();
    }
};

@Override
public void setEventPublisher(EventPublisher publisher) {
    publisher.registerHandlerFor(TestRunStarted.class, setup);
    publisher.registerHandlerFor(TestRunFinished.class, teardown);
}

}`

[wakaleo]

This will work but will be slow for test suites of any size.

[hanymorcos]

It's fine for me. I have 30 tests.

[lcamiloch]

Hi @wakaleo
I try to generate a report with serenity BDD and gradle, when I execute the test an refresh the reports, the index.html file isn't visible in the reports, I change the configuration in gradle with the command "clean test aggregate" but this isn't working.
My question is:
how can I generate the index.html file when the test ends?

[wakaleo]

Could you describe exactly what you are doing to see this behaviour, and attach your build.gradle file?

[lcamiloch]

Ok , I execute the runner class with this configuration

And gradle execute the test normal, when the test ends, so I refresh the reports and the index.html file isn't visible, gradle doesn't generate the report except if I execute in gradle task the option "aggregate", but I need to see the report when the test ends, because in other moment implements jenkins for the execution for all test, this is my build.gradle file:

buildscript {
repositories {
mavenLocal()
jcenter()
mavenCentral()
maven {url = 'http://repo.maven.apache.org/maven2'}
}

dependencies {
    classpath "net.serenity-bdd:serenity-gradle-plugin:2.0.49"
    classpath "net.serenity-bdd:serenity-emailer:2.0.49"

}

}

plugins {
// Apply the java-library plugin to add support for Java Library
id 'java-library'
}

repositories {
// Use jcenter for resolving your dependencies.
// You can declare any Maven/Ivy/file repository here.
mavenLocal()
jcenter()
}

apply plugin: 'net.serenity-bdd.aggregator'

defaultTasks = ['clean', 'build']

tasks.withType(Test) {
reports.html.destination = new File('IVR', name)
println(it.name + '\t' + reports.html.destination)
scanForTestClasses = false
include "*/Runner.class" // whatever Ant pattern matches your test class files
}

dependencies {
// This dependency is exported to consumers, that is to say found on their compile classpath.
api 'org.apache.commons:commons-math3:3.6.1'

// This dependency is used internally, and not exposed to consumers on their own compile classpath.
implementation 'com.google.guava:guava:26.0-jre'

// Use JUnit test framework
testImplementation 'junit:junit:4.12'    


compile group: 'org.apache.poi', name: 'poi-ooxml-schemas', version: '4.1.0'    
compile group: 'org.apache.poi', name: 'poi-ooxml', version: '4.1.0'
compile group: 'org.apache.poi', name: 'poi', version: '4.1.0'
compile group: 'net.serenity-bdd', name: 'serenity-reports', version: '2.0.49'
compile group: 'net.serenity-bdd', name: 'serenity-model', version: '2.0.49'
compile group: 'net.serenity-bdd', name: 'serenity-junit', version: '2.0.49'
compile group: 'net.serenity-bdd', name: 'serenity-core', version: '2.0.49'
compile group: 'net.serenity-bdd', name: 'serenity-cucumber', version: '1.9.31'
compile group: 'org.slf4j', name: 'slf4j-api', version: '1.8.0-beta4'    
testCompile group: 'org.slf4j', name: 'slf4j-log4j12', version: '1.8.0-beta4'
compile group: 'net.thucydides', name: 'thucydides-core', version: '0.9.275'  


compile group: 'net.serenity-bdd', name: 'serenity-gradle-plugin', version: '1.9.36'
compile group: 'net.serenity-bdd', name: 'serenity-emailer', version: '1.9.36'

}

serenity {
reports = ["email"]
}

[jonathar2605]

Hi @wakaleo and @lcamiloch I have the same problem, but I can't resolve I change the version of the gradle and changes configurations in the serenity.config file, I need help with this problem.

[wakaleo]

"gradle clean test aggregate" will not generate the reports if the tests fail (Gradle will stop at "test"). To ensure that the report is always generated when you run "gradle test", add the following to your build.gradle file:

test.finalizedBy(aggregate)

[wakaleo]

(Also, remove 'thucydides-core' from your dependencies - see https://github.com/serenity-bdd/serenity-core#what-is-the-latest-stable-version-i-should-use for the latest versions)

[jonathar2605]

Thanks @wakaleo