Karate: Intellij tests result

[LeJeanbono]

Hello,
with 0.9.2 I've got a different test report in Intellij

with 0.9.1 :

Token keycloak and * ressource feature are executed in background and have @ignore.

I run tests directly with Intellij on right click java folder, Run all tests

Thanks for this amazing tool :)

[ptrthomas]

@LeJeanbono yes - this is a result of #690 cc @xltb

it is hard for me to understand what exactly is the issue - do you think you can provide a simplified example ?

[ptrthomas]

@LeJeanbono and it will be great if you follow this process: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue

[LeJeanbono]

All steps in background section (authentification, reset, create) are shown in the result and not link to the corresponding feature file.
And what is this hierarchy ? authentification > reset > create > no name ?
Will make an example project ;)

[LeJeanbono]

Example of a feature :

Feature: Stocks

Background:
* url baseUrl
* def keycloak = call read('classpath:common/authentification.feature') { username: 'user', password: 'pass' }
* configure headers = { 'Authorization' : '#(keycloak.authorization)'}
* call read('classpath:common/reset.feature')

Scenario: Pas de stock
* def medocs = call read('classpath:common/create.feature') { path: 'medicaments', json: {}, number: 2}
Given path 'sites', '22LVX', 'stocks'
When method get
Then status 200
And match response == '#[2]'

[LeJeanbono]

By the way, by clicking on test, I've got all corresponding execution log (#690)
And that's nice !!

[ptrthomas]

@LeJeanbono yep - we may have broken something, I'm not too worried because I prefer relying on the HTML report instead of IntelliJ. You just open it once and re-fresh it in the browser: https://twitter.com/KarateDSL/status/935029435140489216

[ptrthomas]

and hey @xltb I consider you responsible :P

but seriously I'd like someone to create a sample project with these edge cases that we can fix in the next version. sorry @LeJeanbono I can't use your example

[LeJeanbono]

myproject.zip

[ptrthomas]

@LeJeanbono @sudgarre can you please validate that the fix works for you

[sudgarre]

@ptrthomas : Doesn't seem to work for me.

You can please use the example i attached in #712 to reproduce the Issue #705

[ptrthomas]

@xltb - see above

[xltb]

@ptrthomas @sudgarre @LeJeanbono
Im a little bit confused about the specification.

Is this a correct way to describe:
If there is a call to a separate feature in the background steps or as part of a scenario there should be no separate node on the junit runner results. But the logs for this call should be found in the scenario of where it was found.
I already implemented this, but i need to test it with my project still and then clean up for the PR.

I also noticed a mention of the @ignore option should i also omit these scenarios from the junit results? Can you @ignore a scenario?

[sudgarre]

@xltb - Your description of the issue i faced in #705 is right. I did a test with @ignore scenarios in both my features (Main & Called features). Here is the result :

We should omit the @ignore scenarios as well in the report. Also @ignore scenario only in called.feature still executes (Found same issue on Karate 0.9.2 Junit4 test as well ; see console logs for info)

FYI/thoughts please - @ptrthomas @LeJeanbono

[xltb]

@sudgarre I don't have the standalone test in the project you zipped, could you either upload the project again or test with the develop branch of the karate fork on my page before i make the PR tomorrow? I think it should be fixed there.

As for the fact that ignored scenario's are not ignored if feature that contains them are called from a karate.call i think this is a new issue, or maybe not an issue at all. Currently ignored features are ran if directly called, why wouldn't the scenario's.

[sudgarre]

TestSud_kar_archetype 2.zip

@xltb : Attached my test project, please use UsersRunner class to run the test.
Agree on your point w.r.t @ignore scenarios in Called.feature

[sudgarre]

@xltb : Below is the result

[xltb]

@sudgarre can you confirm that this is the result that you are expecting? or is something still wrong in your opinion

[sudgarre]

@xltb : Strictly speaking the @ignore scenarios must not show in the Junit but i think we must not focus too much into it.

[xltb]

Currently it is read but goes yellow(which means ignored) and is not executed. I think this is nice.

[ptrthomas]

@xltb @sudgarre can you try now - I think I finally understood the problem and fixed it.

@xltb really appreciate your PR - but maybe it wasn't needed after all

and if any of you use slack and are interested in joining the karate slack group - send an e-mail to (my git username) at gmail dot com

[xltb]

@ptrthomas

No worries at all, your solution is much more clear & neat.

[sudgarre]

@ptrthomas : Tried and below is the result

[ptrthomas]

@sudgarre thanks - I'm assuming it is fixed.

[sudgarre]

@ptrthomas : Yes.

[ptrthomas]

released 0.9.3