Artillery: Allow multiple scripts to be given to "artillery run"

[hassy]

Use case: when testing a complex application (an ecommerce website backend in my case), it would be nice to keep scenario definitions in separate files for readability.

It's already possible to extract parts of a flow (e.g. checking out) into their own "subflows" using YAML anchors/references to reduce repetition. The ability to separate scenarios into different files would be another step towards making large test definitions easier to read and maintain.

Mechanism:

The run command can accept a list of files:

artillery run main.yaml scenario1.yaml scenario2.yaml scenario3.yaml...

Where scenario1.yaml etc have the following structure:

config:
  # optional payload definition - gets added to the global payload definitions
  payload:
    path: "./somefile.csv"
    fields: ["field1", "field2"]
  # optional variable definition - as above
  variables:
    somevar: ["value1", "value2"]
  # optional processor definition - as above
  processor: "./functionsForTheScenarios.js"
# Scenarios defined here get added to the global list
scenarios:
  - name: "Select a product and add it to cart"
     flow:
       # flow definition goes here

All paths are to be resolved relative to the path to the scenario file.
Any fields other than defined above will be ignored (e.g. config.environments - those should be set in main.yaml).

[steveschnepp]

Letting the main.ymlhaving special rights is counterintuitive. A much better way would be to have an #include mechanism, which then will just fit in.

[Offtree]

Have you considered allowing glob patterns instead of having to list out scenarios? Seems like part of the goal with this enhancement would be to allow for easier integration with a full team and not really have to think about what tests your testing pipeline is running. This would align with common practices with other testing frameworks.

Ex.

artillery run scenario.*.yaml
artillery run ./tests/**/*.scenario.yaml

[vichybaby]

Is anyone having a working example?

For me, only the first scenario got executed successfully.
artillery run scenario1.yaml scenario2.yaml...