Karate: [karate-gatling] explore ways of composing existing tests better for specific flows

[ptrthomas]

Triggered by this question on Stack Overflow: https://stackoverflow.com/q/51504264/143475 and others who have discussed this with me as well.

Please comment and add asks / opinion.

I see the following themes:

• when there already exists a bunch of Karate tests, it should be possible to "pick and choose" some of them, treat as a unit and then use that unit in a Gatling simulation.
• I think the above is best solved by tags. so you can declare that only some tags should participate in a Gatling run
• ideally your existing features should be split into fine-grained flows.
• but I get that people want to have feature files with multiple scenarios
• proposal: a tag of the form @name=foo will be considered special in Karate
• this is how you can control WHICH scenarios to isolate in a given feature. for e.g. in Gatling scenarioNames( "createUser,deleteUser")
• if we have a Karate-native means of calling a feature BUT only invoke a certain Scenario then for Gatling all you need to do is write one extra "wrapper" feature, which would be "thin" that can "delegate" to or "orchestrate" all the other features and scenarios that need to be called. this needs more thinking and will evolve I am guessing. it might be better to have this "orchestration" definable in Gatling itself to avoid one extra layer

[ptrthomas]

additional thought, should be possible to refer to the feature+scenario directly in one short-cut notation:

karateFeature("classpath:mock/cats-create.feature@name=foo")

would we need any other meta info on top of this ?

[ssailappan]

@ptrthomas could we then do something like this -> karateFeature("classpath:mock/*.feature@name=foo")

[ptrthomas]

@ssailappan yeah, that would be cool. maybe we should look at AOP point-cut syntax for inspiration !

I'm thinking v1 should keep it simple and have each feature explicit.
the other issue I see is if the above resolves to multiple features, do we run each in parallel or sequential, and in which order. and mapping it to the Gatling setUp refer demo

other possibilities:

karateFeature("classpath:mock/cats.feature@name=create,readMany,delete")

[manoj404]

@ptrthomas Just thinking here whether it would be better if we can read the variables from one feature file in "gatling simulation class" and pass it to the corresponding feature file as well.

[ptrthomas]

@manoj404 sorry i’m confused again. can you kindly provide an example.

[ptrthomas]

@manoj404 also please read this section of the docs carefully: https://github.com/intuit/karate#script-structure

hint: each Scenario should be independently runnable.

[manoj404]

@ptrthomas Let's say, we have all the reusable feature files for CRUD operation and we want to run performance on Get a record test case. So something like below steps i can write in simulation class,

1. Create a gatling simulation class like PerformanceRunOnGetRecord.scala

2. Read create record feature file and store the record id in some variable to pass next feature file.
   val createRecord = scenario("create").exec(karateFeature("classpath:mock/record-create.feature@name=record_name"))
   Here let's say the out of the api as {"record_id" : "12345"}

3. Here i am expecting some way to store the created record Id in a variable like below,
   val record_Id = some way to get the record ID

4. Read the get record feature file like below
   val getRecord = scenario("Read").exec(karateFeature("classpath:mock/record-read.feature@id=record_Id"))

5. Setup performance configuration on getRecord
   setUp( getRecord.inject(rampUsers(1) over (10 seconds))).maxDuration(1 minutes)

Again I might be thinking wrong here, but it's just a thought process. I think, if we can do something like this, then it might be a best way to reuse the functional feature files for performance test case. Please let me know if you have any queries.

[ptrthomas]

@manoj404

store the record id in some variable to pass next feature file.

No. you have to combine a flow into one scenario (or feature). Looks like you are simply not able to accept this concept. did you even read the link I posted in my previous comment ? any comments on that ?

[manoj404]

@ptrthomas Yeah i understand. That was just a thought process. Thank you

[ptrthomas]

@manoj404 a couple of extra points. just so that I can explain this to others later:

• remember that this is a perf test. gatling will run each unit in parallel. there's no way you can inter-depend
• if you already have a CRUD test for say DELETE, you will naturally have a CREATE as a "pre-requisite" for that flow. isn't this how most teams do testing. if you are really smart about tests you may modularize so that you have a re-usable C flow - but still you will have a single feature that will run C first and then D

hope that makes sense !

[ptrthomas]

implemented this, but decided to keep it simple. only one tag can feature in the "tag selector"

classpath:mock/cats-create.feature@name=foo

And it does not need to be in the @name=value form, any tag can be used:

classpath:mock/cats-create.feature@someTagName

The good thing is that if multiple Scenario-s match, they will all be executed - but in the order they appear in the Feature.

[ptrthomas]

0.9.0 released