Karate: Feature Request: JUnit 5 TestSource (IDE "jump to source" feature)

[bkahlert]

Karate has a nice JUnit 5 support. IntelliJ has a nice JUnit 5 support. What is missing is support for JUnit 5 TestSource (https://junit.org/junit5/docs/current/user-guide/#writing-tests-dynamic-tests-uri-test-source) which would allow clicking on a failed test...

to jump to the corresponding scenario:

Currently we implement that feature on our own:

    fun ScenarioResult.asTestSourceUri(): URI {
        val workingDir = System.getProperty("user.dir")
        val featurePath = scenario.feature.relativePath.replace("classpath:", "")
        val line = scenario.line

        return URI.create(File("$workingDir/src/test/resources/$featurePath").toURI().toString() + "?line=$line")
    }

Though it works, implementation could be more robust and also for that to work we virtually had to turn Karate upside down as the URI is required during the DynamicTest instantiation.

[ptrthomas]

@bkahlert sure, I was not aware of this - also cc @kirksl to look out for similar possibilities on the VS Code plugin side

for now I'll tag this as hacktoberfest to see if anyone steps up

[ptrthomas]

@bkahlert BTW I'm re-writing the core of karate ref #1281 - so if you have any more things you'd like to improve, this is a good time, feel free to add comments. here is one example of upcoming improvements: https://github.com/intuit/karate/issues/1311#issuecomment-706094978

[chaudharydeepak]

@ptrthomas - this looks an interesting enhancement- if its ok I would like to take a initial jab at it? Thank you.

any pointers to get started wrt karate to implement this will be appreciated (though not mandatory) 😀 !

[ptrthomas]

@chaudharydeepak go for it, any research findings will help I still don't understand what OP meant by we virtually had to turn Karate upside down

[chaudharydeepak]

@ptrthomas - let me know if any documentation needs to be updated to highlight this. Thank you.

[ptrthomas]

@chaudharydeepak this is great, I think the IDE auto experience is good enough

[bkahlert]

@ptrthomas wish list:

• a builder would be great (see attached)
• a way to configure that JSON / Cucumber reports should be generated as well
• or at least a way to find out different paths easier

I already worked on these features and attached what I got far :)
Archive.zip

@chaudharydeepak I'm tagging you too in the hope, that the attached code is of use to you.

[ptrthomas]

@bkahlert great, (2) is certainly happening in 1281 - I thought the Runner and JUnit5 helpers were already builder-like - but we'll try improve those. since I'm not that great at kotlin you can request any specifics. maybe best thing is when we start releasing RC builds, you can try them and provide feedback. may be a month or two out

[bkahlert]

@ptrthomas Can't wait to see how you're going to solve the problem of finding the original feature file. I'm using this heuristic approach

feature.relativePath.replace("classpath:", "").let { featureFilePath ->
    return KarateDirectories.src.search { dir -> featureFilePath in dir }?.resolve(featureFilePath)?.toUri()
}

which works in my IntelliJ but not guaranteed to work everywhere.

Concerning the builder pattern:
It might also be a misunderstanding. It would be helpful to also be able to set system properties for the scope of the test execution because I like to set everything from code (as an alternative to provide everything from command line).

My background: I'm working on a tiny lib to make using Karate for E2E testing as easy as thinkable for me, my colleagues and other developers. The easiest I came up with are the KarateRunner you find unfinished in the attachment and a template pattern based JUnit test class that becomes active if extended:

@E2E
class E2ETests : KarateTestTemplate() {

    override fun KarateConfig.tests() {
        val application = SpringApplication.run(NotificationApp::class.java)
        val port = application.environment["local.server.port"]

        withServerPort(port)
        withFeaturesFoundAt {
            classPath("e2e")
        }
        // TODO set num threads
    }

    override fun KarateReporter.reports() {
        generateCucumberHtmlReport("Notification API") {
            addClassifications("Project", "You name it")
            ...
        }
    }
}

I hope that helps: What to me is the biggest difference (and advantage) of Kotlin to Java:
1) there are extensions functions that allow functions of the style myFun(obj1, obj2, obj3) to be called like obj1.myFun(obj2, obj3) which allows for amazing APIs.
2) If the last parameter of a function is a lambda, the function can not only be called like myFun({ LAMBDA }) but also like myFun { LAMBDA } which again allows for amazing APIs (e.g. see here https://try.kotlinlang.org/#/Kotlin%20Koans/Builders/Html%20builders/Task.kt)
Would have someone told be earlier, that would have helped me learning Kotlin way faster.

[ptrthomas]

able to set system properties for the scope of the test

yes will take care of this in 1281 - another reason is so that you can even run multiple "suites" in parallel, which we are designing for

tiny lib to make using Karate for E2E testing

sounds like you are using spring boot, please do be aware of #751 and maybe you can advise / help test - also there is this classpath scanner lib that may help with your first point: https://github.com/intuit/karate/pull/1317#issuecomment-706912304

[ptrthomas]

problem of finding the original feature file

yes. one option is to have the user specify an optional config file or system property which will make karate look in e.g. src/test/java for whatever is found in target/test-classes

[chaudharydeepak]

problem of finding the original feature file

yes. one option is to have the user specify an optional config file or system property which will make karate look in e.g. src/test/java for whatever is found in target/test-classes

Hi @ptrthomas - regarding configuration - i think it will make sense to have a global or module specific config file (kafka.props etc) pre-configured to load on runtime - & as well supports different profiles - for ex test env1 / test env 2 etc. Let me know if I am on wrong track altogether on this requirement.

[ptrthomas]

@chaudharydeepak yes I haven't thought this through - actually we may be overthinking this - what if this was just yet another param we can pass into the Runner / RunnerOptions etc and even CLI