Testng: Need a way to tweak the execution order so that the method annotated by @AfterMethod runs before any of the listener methods (e.g. onTestSuccess/onTestFailure).

[akshayamaldhure]

TestNG Version 6.9.9

Expected behavior

The method annotated by @AfterMethod should run before the onTestSuccess/onTestFailure
methods.

Actual behavior

The method annotated by @AfterMethod runs after the onTestSuccess/onTestFailure
methods.

Is the issue reproductible on runner?

Yes

• [ ] Shell
• [c ] Maven
• [ ] Gradle
• [ ] Ant
• [ ] Eclipse
• [ ] IntelliJ
• [ ] NetBeans

[krmahadevan]

@akshayamaldhure - Can you please elaborate as to what you are trying to achieve ? Maybe that will help us suggest an apt way of solving your problem.

[akshayamaldhure]

I would like to know if there's some way to tweak TestNG so that the method annotated by @AfterMethod gets executed first and then onTestSuccess/onTestFailure methods in the test listener run.In my project, the method annotated by @AfterMethod has the logic to print the response body for each test method. And the onTestSuccess/onTestFailure methods have logic to print a message (e.g. "Test completed"). I do not want to put the code to print this message in the @AfterMethod of every test class in my project.The problem is, the message "Test completed" is shown first and then the response body gets printed, which might be correct as per the TestNG design, but this is something I want to tweak.

[krmahadevan]

@akshayamaldhure - Isn't it easier for you to just print both the response and the message in just the onTestSuccess() message ? You could still have the @AfterMethod to compute the response and add it as an attribute to the ITestResult object via native injection, which can be popped out by the listener no ?

[akshayamaldhure]

I do not have access to my response object in onTestSuccess(). How do I possibly inject it there in order to access it and do things with it?

[krmahadevan]

Every @AfterMethod method can basically be provided with an ITestResult object as a parameter. When TestNG sees it, it resorts to natively injecting the current @Test method's result (ITestResult stands for the result associated with a TestNG method). So you can persist your response object into the ITestResult object via the ITestResult.setAttribute() and then pop out this value via ITestResult.getAttribute() in your listener.

Refer to this matrix to understand what all can be natively injected by TestNG : https://github.com/cbeust/testng/wiki/Allowed-parameters-injection-in-methods

[akshayamaldhure]

At result.getAttribute("response"), I get a null. Here's how I tried it.

TestClass.java

package test;

import static io.restassured.RestAssured.given;

import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;

import io.restassured.response.Response;

public class TestClass {

    Response response;

    @Test
    public void test() {
        response = given().baseUri("http://google.com").when().get();
    }

    @AfterMethod
    public void tearDown(ITestResult result) {
        result.setAttribute("response", response);
    }

}

TestListener.java

public class TestListener implements ITestListener, ISuiteListener {

    @Override
    public void onTestSuccess(ITestResult result) {
        System.out.println("Testing attribute injection: "+result.getAttribute("response"));
    }
}

[krmahadevan]

@akshayamaldhure - I understood what you are after. My apologies for not getting it earlier!

So you have basically two queries.

1. How does one ensure that configuration methods (teardown in specific) get executed before listener implementations.

The short answer is, there is no way in TestNG using which you can do this re-ordering.

1. How do I get the attribute that I set via a configuration method (@AfterMethod in this case) be made available in my listeners for extraction.

Here's how you do it.

Test class

import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Listeners;
import org.testng.annotations.Test;

@Listeners(TestListener.class)
public class TestClass {
    static final String DATA = "data";

    @Test
    public void test() {
    }

    @AfterMethod
    public void teadDown(ITestResult result) {
        result.setAttribute(DATA, new Object());
    }
}

Test listener

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

public class TestListener implements IInvokedMethodListener {

    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult testResult) {

    }

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult result) {
        if (!method.isConfigurationMethod()) {
            //We are interested only in configuration methods. So if its a test method lets skip it.
            return;
        }
        ITestResult testMethodResult = extractResultObject(result);
        if (testMethodResult == null) {
            //We are looking for only those ITestResult objects which was passed to a configuration method
            //as a parameter. This happens only when TestNG resorts to doing a native injection of an ITestResult
            //object into a configuration method.
            return;
        }
        System.out.println("Testing attribute injection: " + testMethodResult.getAttribute(TestClass.DATA));
    }

    private static ITestResult extractResultObject(ITestResult result) {
        if (result.getParameters() == null || result.getParameters().length == 0) {
            return null;
        }
        for (Object param : result.getParameters()) {
            if (param instanceof ITestResult) {
                return (ITestResult) param;
            }
        }

        return null;
    }
}

Execution output:

Testing attribute injection: java.lang.Object@6ae40994

===============================================
Default Suite
Total tests run: 1, Failures: 0, Skips: 0
===============================================


Process finished with exit code 0

[akshayamaldhure]

Thanks @krmahadevan for the detailed explanation. I have two questions.

1. I'm unable to write something similar for a non-string object (e.g. response). I see that the setAttribute() method accepts only a string as the first object. I did not understand how I can inject by response object from there.
2. Can I modify the statement testMethodResult.getAttribute(TestClass.DATA)by replacing TestClass by something more generic since I have lots of test classes?

[krmahadevan]

@akshayamaldhure

I'm unable to write something similar for a non-string object (e.g. response). I see that the setAttribute() method accepts only a string as the first object. I did not understand how I can inject by response object from there.

The setAttribute() takes two arguments. The key is always a String and the value is an Object. Imagine it to be a map. So the key can very well be a string and the value can be your Response object.
See here for javadocs references.

Can I modify the statement testMethodResult.getAttribute(TestClass.DATA)by replacing TestClass by something more generic since I have lots of test classes?

Yes you can. The example that I shared was merely for illustrative purposes ONLY.

[akshayamaldhure]

@krmahadevan Passing response as the second parameter to the setAttribute() method seems to be working. But I still lack the clarity as to how I could possibly make the getAttribute() generic. Could you please throw some light on the same?

[krmahadevan]

@akshayamaldhure - The fact that the value for the key that you pass via setAttribute() is of type Object itself makes it generic enough. Which means the following is perfectly valid

testResult.setAttribute("name", "krishnan");
testResult.setAttribute("response", response);
testResult.setAttribute("status", false);

from within the same method. As you can see I am making 3 consecutive calls to setAttribute() method, wherein all my keys are String but the values are completely different types.

• First call stores a String
• Second call stores your Response object and
• Third call stores a boolean

Is that what you are after ? I didnt quite understand what other generic behavior are you looking for here.

[akshayamaldhure]

@krmahadevan Sorry, that's not what I'm looking for. I actually took my other question further? (Can I modify the statement testMethodResult.getAttribute(TestClass.DATA) by replacing TestClass by something more generic since I have lots of test classes?). I would like to know how to do this.

[krmahadevan]

Yes you can. Assuming that you are pushing in the class name itself (or) the method name itself as the key. Both of these, you should be able to extract out of the ITestResult object itself. Does that make sense ?

Something like this

        //@Test method name serves as the key
        result.setAttribute(result.getMethod().getMethodName(), new Object());
        //Class to which @Test method belongs to as the key
        result.setAttribute(result.getInstance().getClass().getName(), new Object());

[akshayamaldhure]

Sorry @krmahadevan , that's not what I meant to ask. My question in my earlier comment was about the getAttribute() method and not about setAttribute() method. How do I make the statement testMethodResult.getAttribute(TestClass.DATA) generic by making the TestClass.DATA generic? e.g. Something like getTestClassWhichHasAnAfterClassThatSetsResponseAttribute().response.

[krmahadevan]

@akshayamaldhure - Pretty easy. Define a new interface called

interface TestAttribute {
   void String getAttribute();
}

Have all of your test classes implement this interface [ similar to how you work with the ITest interface ] and then within your afterInvocation() do something like below

String attributeName = "";
Object instance = testResult.getInstance();
if (instance instances TestAttribute){
   attributeName = ((TestAttribute)instance).getAttribute();
}

System.out.println("Testing attribute injection: " + testMethodResult.getAttribute(attribute));

[akshayamaldhure]

So what is the implemented getAttribute() method in my test class supposed to contain? Also, with reference to your code example, how does the afterInvocation() method in my listener know that it needs to get the response attribute from my test class? It would be great if you could put the complete picture in a code example. Thanks!

[krmahadevan]

So what is the implemented getAttribute() method in my test class supposed to contain?

Whatever you would like to use as the key in the call to ITestResult.getAttribute()

Also, with reference to your code example, how does the afterInvocation() method in my listener know that it needs to get the response attribute from my test class?

Its your test code and your test listener. So its your responsibility to let your listener know that it has to query the test class to retrieve the key to be used when it queries the ITestResult.getAttribute() to get hold of the response object

It would be great if you could put the complete picture in a code example. Thanks!

@akshayamaldhure - I have given you all the required information in bits and pieces. Why not you try to weave all of them together as an end-to-end sample and let me know where you are stuck ? That way I would exactly know where you need help!

[akshayamaldhure]

After weaving it all together, this is how it looks:

MyTest.java

import static io.restassured.RestAssured.given;

import org.testng.ITestResult;
import org.testng.annotations.AfterMethod;
import org.testng.annotations.Test;

import io.restassured.response.Response;

public class MyTest implements TestAttribute {

    Response response;

    @Test
    public void pingGoogle() {
        response = given().baseUri("http://google.com").when().get();
    }

    @AfterMethod
    public void flushResponse(ITestResult result) {
        result.setAttribute("response", response);
    }

    @Override
    public String getAttribute() {
        // not sure what do I put here
        return null;
    }
}

TestAttribute.java

public interface TestAttribute {

    public String getAttribute();
}

TestListener.java

import org.testng.IInvokedMethod;
import org.testng.IInvokedMethodListener;
import org.testng.ITestResult;

import test.TestAttribute;

public class TestListener implements IInvokedMethodListener {

    @Override
    public void afterInvocation(IInvokedMethod method, ITestResult result) {
        String attributeName = "";
        Object instance = result.getInstance();
        if (instance instanceof TestAttribute){
           attributeName = ((TestAttribute)instance).getAttribute();
        }
        System.out.println("Testing attribute injection via listener: "+result.getAttribute(attributeName));
    }

    @Override
    public void beforeInvocation(IInvokedMethod method, ITestResult result) {
        // Do nothing
    }
}

This is what I get after running the code:
Testing attribute injection via listener: null
(Naturally, because the getAttribute() method currently returns null).

[krmahadevan]

You have got everything right.

Just update your test class to something like this

public class MyTest implements TestAttribute {

    Response response;
        private static final String KEY = "response";

    @Test
    public void pingGoogle() {
        response = given().baseUri("http://google.com").when().get();
    }

    @AfterMethod
    public void flushResponse(ITestResult result) {
        result.setAttribute(KEY, response);
    }

    @Override
    public String getAttribute() {
        return KEY;
    }
}

[akshayamaldhure]

Even with that, I get Testing attribute injection: null.

I see that the attributeName gets resolved properly to "response" inside the listener, but result.getAttribute(attributeName) i.e. result.getAttribute("response") still returns null for some reason.

[krmahadevan]

That cant be. Are you sure you are trying this out on TestNG 6.13.1 (the latest released version as of today) ?

[akshayamaldhure]

I'm on TestNG version 6.9.9, but I get the same issue with TestNG version 6.13.1.

[akshayamaldhure]

I even tried making response as a public field inside my test class, but that does not help either.

[krmahadevan]

It works absolutely fine for me. Please ensure that you are using my version of the listener and not yours, because your version doesnt distinguish between a @Test and a @AfterXXX configuration method. The attribute is available only for a configuration method.

Please go through the sample I shared, and enhance it with the TestAttribute

[krmahadevan]

My sample retrieves the ITestResult from the testResult's parameters. Your version directly works with what is being passed via the afterInvocation().

[akshayamaldhure]

@krmahadevan Ah! I completely forgot about your version of the listener. With the same, it works perfectly fine. Thanks a lot for all the help!