Junit5: Cannot create a @ParameterizedTest that uses @MethodSource in a @Nested test class

[franz-see]

Overview

Bug report.
Version : org.junit.jupiter:junit-jupiter-params:5.0.2

Cant create a nested parameterized test that uses @MethodSource. @MethodSource requires that it points to a static method. But you cannot have a static method from a non-static inner class. And you cant have a nested class that is a static inner class

Scenario 1

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;

import java.util.Arrays;
import java.util.stream.Stream;

import static org.junit.jupiter.api.Assertions.assertEquals;

class NestedParameterizedTest {

    @Nested
    class TheNesting {

        @ParameterizedTest
        @MethodSource("getTestData")
        void shouldAdd(int a, int b, int c) {
            assertEquals(c, a + b);
        }

        Stream<Arguments> getTestData() {
            return Arrays.stream(new Arguments[]{
                    Arguments.of(1, 2, 3)
            });
        }
    }
}

This code results to a org.junit.platform.commons.util.PreconditionViolationException: Cannot invoke non-static method [java.util.stream.Stream<org.junit.jupiter.params.provider.Arguments> NestedParameterizedTest$TheNesting.getTestData()] on a null target.

Scenario 2

Making getTestData() static though results to a compile error Inner classes cannot have static declarations.

Scenario 3

Making getTestData() and class TheNesting static though results in Empty test suite.

[marcphilipp]

Have you tried adding @TestInstance(PER_CLASS) on the nested class?

[franz-see]

@marcphilipp Interesting. That works! :D can you explain why adding that would work?

FYI, here's the resulting test class that works

import org.junit.jupiter.api.Nested;
import org.junit.jupiter.api.TestInstance;
import org.junit.jupiter.params.ParameterizedTest;
import org.junit.jupiter.params.provider.Arguments;
import org.junit.jupiter.params.provider.MethodSource;

import java.util.Arrays;
import java.util.stream.Stream;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.TestInstance.Lifecycle.PER_CLASS;

class NestedParameterizedTest {

    @TestInstance(PER_CLASS)
    @Nested
    class TheNesting {

        @ParameterizedTest
        @MethodSource("getTestData")
        void shouldAdd(int a, int b, int c) {
            assertEquals(c, a + b);
        }

        Stream<Arguments> getTestData() {
            return Arrays.stream(new Arguments[]{
                    Arguments.of(1, 2, 3)
            });
        }
    }
}

And here's how it looks like in Intellij IDEA

[marcphilipp]

Interesting. That works! :D can you explain why adding that would work?

Sure! By default, JUnit creates a test class instance for each test invocation. Therefore, the method that provides arguments must be static. Using @TestInstance(PER_CLASS) changes the default behavior. Now, a test class instance is created only once and used for all test invocations of that class. Therefore, it can also be used to call getTestData(). Makes sense?

[sbrannen]

FYI: this is actually documented in the JavaDoc for @MethodSource: http://junit.org/junit5/docs/current/api/org/junit/jupiter/params/provider/MethodSource.html

I am therefore closing this issue as "works as designed".

[stalkerg]

This case not working correctly: @BeforeAll/@AfterAll methods calls only for the first argument from getTestData().

Because here https://github.com/junit-team/junit5/blob/master/junit-jupiter-engine/src/main/java/org/junit/jupiter/engine/descriptor/ClassTestDescriptor.java#L163 you set beforeAllMethodsExecuted even for Lifecycle.PER_CLASS.

[sbrannen]

@stalkerg,

@BeforeAll/@AfterAll methods are only supposed to be executed "once per test class" regardless of what the test instance lifecycle is.

That's by design.

If you need to do something before or after each method, you need to implement @BeforeEach/@AfterEach methods.

[sbrannen]

@franz-see,

By the way, you can simplify your getTestData() method by simply using Stream.of(...).

Stream<Arguments> getTestData() {
    return Stream.of(Arguments.of(1, 2, 3));
}

[stalkerg]

methods are only supposed to be executed "once per test class"

yep, it's logical but in my case, @AfterAll call immediately after the first argument (1).
And I can't use BeforeEach because I should use some instancing objects in getTestData().

[stalkerg]

I understand why call @BeforeAll before 1 but in my opinion, @AfterAll should call only after 3 from getTestData(). Because it's AfterAll

[kordzik]

Old topic, but haven't seen this mentioned: you can also put the static argument source method on your outer test class and then refer to it via FQN in your @Nested tests, like this: @MethodSource("x.y.z.NestedParameterizedTest#getTestData")

[nikitaax3]

I try to set up parameterized tests for an interface. The test classes are abstract to provide custom source methods. Now, I want to use nested classes to structure my tests.
Is it possible to use non-static argument source methods?

[sbrannen]

Is it possible to use non-static argument source methods?

Yes, see the answer in https://github.com/junit-team/junit5/issues/1229#issuecomment-355741728.

[nikitaax3]

@sbrannen This did help a little bit, but produced another error, due that my method sources are not static. I try to explain:

public abstract class AbstractOuterClass {

    @TestInstance(Lifecycle.PER_CLASS)
    @Nested
    class NestedTest{ 
        @ParameterizedTest(name = "{displayName} \nGiven arguments: {arguments}")
        @MethodSource("AbstractOuterClass#methodSource")
        protected void testGetBORelsAllAttributes(List<String> pks, String attributeNames, boolean objectHasAttributes)
                throws NotLoggedInException, InternalErrorException, ExternalErrorException, MultipleFailuresError {
            // test something
        }
    }

    protected Stream<Arguments> methodSource() {
        List<Arguments> listOfArguments = new LinkedList<>();

        // do things

        return listOfArguments.stream();
    }
}

The error I got then was this:
java.lang.IllegalArgumentException: object is not an instance of declaring class

This Version would work totally fine, if my method sources were static. But it is crucial to my test concept that they aren't.

My solution:

public abstract class AbstractOuterClass {

    @TestInstance(Lifecycle.PER_CLASS)
    @Nested
    class NestedTest{ 
        @ParameterizedTest(name = "{displayName} \nGiven arguments: {arguments}")
        @MethodSource("AbstractOuterClass#methodSource")
        protected void testGetBORelsAllAttributes(List<String> pks, String attributeNames, boolean objectHasAttributes)
                throws NotLoggedInException, InternalErrorException, ExternalErrorException, MultipleFailuresError {
            // test something
        }

        protected Stream<Arguments> methodSource() {
            return AbstractOuterClass.this.methodSource();
        }
    }

    protected Stream<Arguments> methodSource() {
        List<Arguments> listOfArguments = new LinkedList<>();

        // do things

        return listOfArguments.stream();
    }
}

I hope that explains my problem and my solution. It also works with my specific use in making the methodSource function abstract and implementing it in a different class.

Thank you for your response.