Karate: support JUnit 5

[ptrthomas]

should be straightforward, will try for 0.9.0

[ptrthomas]

there's this fluent api thingy that is used like this. one improvement over current cucumber / karate options, is now it is easier to load files relative to test class in JUnit 5.

EDIT: in "relative path" mode you can even omit the .feature file-extension so it is nice and DRY

EDIT: and you can have multiple features in the same single test-class which is a huge improvement, and reduces clutter in dev-mode

import com.intuit.karate.junit5.Karate;
import org.junit.jupiter.api.TestFactory;

public class SampleTest {

    @TestFactory
    public Object testSample() {
        return Karate.feature("sample").relativeTo(getClass()).run();
    }

    @TestFactory
    public Object testTags() {
        return Karate.feature("tags").tags("@second").relativeTo(getClass()).run();
    }

    @TestFactory
    public Object testFullPath() {
        return Karate
                .feature("classpath:com/intuit/karate/junit5/tags.feature")
                .tags("@first").run();
    }

}

[sormuras]

@ptrthomas nice approach!

Just keep in mind, that dynamic tests generated by @TestFactory don't participate in the default lifecycle of regular tests. That's why I refer to them as "testlets" or see them as "assertAll()-on-steroids", because you get a report entry for each generated testlets.

[ptrthomas]

@sormuras thanks, that helps ! just had a look. from experience, the @BeforeEach and @AfterEach only for the whole @TestFactory is fine and in fact I expect most users to stick to @BeforeAll and @AfterAll - and yes - we need the report entry for each 'unit' and that way this approach works great.

one question - it is mentioned that:

Dynamic tests are currently an experimental feature

I hope they are on the path to be "core" - it seems to be perfect for our use case.

[sormuras]

I hope they are on the path to be "core" - it seems to be perfect for our use case.

They are here to stay. Only minor API tweaks may happen to realize some features, that's why dynamic tests are still considered an experimental feature.

[ptrthomas]

@sormuras thank you !

[sormuras]

Having Object declared as the return type looks a bit odd. How about letting class Karate implements Iterable<DynamicNode>? That would yield:

    @TestFactory
    Karate testSample() {
        return Karate.feature("sample").relativeTo(getClass()).build();
    }

    @TestFactory
    Karate testTags() {
        return Karate.feature("tags").tags("@second").relativeTo(getClass()).build();
    }

    @TestFactory
    Karate testFullPath() {
        return Karate
                .feature("classpath:karate/tags.feature")
                .tags("@first")
                .build();
    }

If you like that approach I have a PR almost ready. ;-)

[ptrthomas]

@sormuras I love it. please proceed !

[ptrthomas]

@sormuras quick question - is this taking things too far ?

package karate;

import com.intuit.karate.junit5.Karate;

class SampleTest {

    @Karate.Test
    Karate testSample() {
        return Karate.feature("sample").relativeTo(getClass()).build();
    }

    @Karate.Test
    Karate testTags() {
        return Karate.feature("tags").tags("@second").relativeTo(getClass()).build();
    }

    @Karate.Test
    Karate testFullPath() {
        return Karate
                .feature("classpath:karate/tags.feature")
                .tags("@first")
                .build();
    }

}

[sormuras]

"Far", yes... especially reading those lines in com.intuit.karate.junit5.Karate:

    @TestFactory
    public @interface Test {

...but not "too far", imho.

Now, get rid of the terminating .build() call by letting the builder implement Iterable<DynamicNode> or similar as well. 🙂

[ptrthomas]

@sormuras thank you - this has been super-educational ! how does this look ?

package karate;

import com.intuit.karate.junit5.Karate;

class SampleTest {

    @Karate.Test
    Karate testSample() {
        return new Karate().feature("sample").relativeTo(getClass());
    }

    @Karate.Test
    Karate testTags() {
        return new Karate().feature("tags").tags("@second").relativeTo(getClass());
    }

    @Karate.Test
    Karate testFullPath() {
        return new Karate()
                .feature("classpath:karate/tags.feature")
                .tags("@first");
    }

}

[sormuras]

Killer Karate Style.

One more duplication new Karate().feature(... could be turned into a static method in Karate.

public static Karate.Builder Karate.testFeature(...) { return new Karate().feature(...); }

Too far?

Or just feature?

    @Karate.Test
    Karate testSample() {
        return Karate.feature("sample").relativeTo(getClass());
    }

[ptrthomas]

@sormuras heh yep - I think there is less typing now :) but will think through a little more, this is fun ^_^

I actually like how it is clear we are returning a Karate object now.

[ptrthomas]

@sormuras was thinking - can I suggest an enhancement for the @TestFactory annotation. if it is at class-level and any method's return type is Iterable<DynamicNode> that method will be treated as a dynamic-test factory method ?

[sormuras]

Feel free to propose this feature over at https://github.com/junit-team/junit5/issues/new/choose for discussion.

To me, it feels like too much "magic". You (and the IDE, for example) don't see the test entry-points no longer explicitly.

[ptrthomas]

@sormuras agreed, thanks for your inputs !

[ptrthomas]

0.9.0 released

[sormuras]

Cheers! 🍻

[dobogdas]

Question - I added below methods to my runner class. When I try to run the testTags() test, I get 'No tests found for given includes: ....' and 'Test events were not received' (in my IDE) errors. I have ensured that the "@test1" tag is a valid tag in my feature file.

The testFullPath() method runs my entire feature, as expected.

I'm using karate-junit5:0.9.5.RC5

@Karate.Test
Karate testFullPath() {
    return new Karate().feature("regressionTest.feature").relativeTo(getClass());

}

@Karate.Test
Karate testTags() {
    return new Karate().feature("regressionTest.feature").tags("@test1").relativeTo(getClass());
}

[ptrthomas]

@dobogdas no point commenting here, follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue