Newman: Run Multiple Collections within folder

[shicky4]

Currently I have a number of Postman collections which we wish to use as a set of smoke tests. I anticipated I would be able to point at a directory containing postman collections and for all of these to be executed one by one. However, this is not possible. My expectation was incorrect and I have worked around the issue by having multiple newman run commands, though this seems like waste/duplication.

1. Newman Version (can be found via newman -v): 3.3.1
2. OS details (type, version, and architecture): Windows 7-x64
3. Are you using Newman as a library, or via the CLI? CLI
4. Did you encounter this recently, or has this bug always been there: No bug, potential feature request
5. Expected behaviour: Expected to point newman at a folder containing a number of collections and for it to iterate over them. Clearly this expectation was incorrect but could be helpful? At the moment I'm just repeating the same newman run command but pointing to a different collection within the same folder each time.
6. Command / script used to run Newman: newman run C:\PostmanCollections\requests\ -e C:\PostmanCollections\environment\postman_environment.json
7. Sample collection, and auxilliary files (minus the sensitive details): N/A
8. Screenshots (if applicable): N/A

[shamasis]

This is not something that is there in the roadmap. As such, use Newman programmatically to achieve this.

https://github.com/postmanlabs/newman/blob/develop/examples/run-collections-in-directory.js

[bradschafer]

Shamasis - Thanks for the example on reading a folder form disk..
I plan to use it for smoke tests as well.

How might one define the folder for the output directory in node?
I've tried :
export: 'path to place output'

and I can't use dashes in a object key ( cli-export, json-export etc..)
So I am not sure what to use to set the path.

[Ne4istb]

I solved this issue in a bit another way: have written a tool for combining several collections into one.
https://www.npmjs.com/package/postman-combine-collections

We successfully use it in CI run.

[caddac]

I solved this by having a Command Line build step in TeamCity and running:

for collection in ./tests/*; do newman run "$collection" --environment ./envs/dev.postman_environment.json' -r cli,teamcity; done

[kavigopal]

@Ne4istb - if you combine collection how to specify the data file for each of the collection while combining?

if its single we are specifying lik example below
"newman run collection.json -e QA_environmentvariable.json -g globalvariable.json -d Data.csv"

can you give some example code? like if i have to combine 2 collection with data like below whats the sample code format ?
Collection1.json - data1.json - environ1.json
collection2.json - data2.json - environ2.json

[steckdev]

So for our solution we ended up using the Newman Ubuntu docker image which we created with
docker run -v %cd%/../:/etc/newman --name postman_test --entrypoint /bin/bash -t postman/newman_ubuntu1404 "./Scripts/ci-dev-test.sh"
then ran the tests using this bash script.
`rm -r "./Results"; mkdir -p "./Results"

newman run './Collection.postman_collection.json' -e ./Environment/Dev.postman_environment.json --folder "Folder 1" -d "TestData/Data1.json" -r cli,junit --reporter-cli-no-assertions --reporter-junit-no-assertions --reporter-junit-export "Results/report1.xml"

newman run './Collection.postman_collection.json' -e ./Environment/Dev.postman_environment.json --folder "Folder 1" -d "TestData/Data2.json" -r cli,junit --reporter-cli-no-assertions --reporter-junit-no-assertions --reporter-junit-export "Results/report2.xml"

newman run './Collection.postman_collection.json' -e ./Environment/Dev.postman_environment.json --folder "Folder 2" -d "TestData/Data3.json" -r cli,junit --reporter-cli-no-assertions --reporter-junit-no-assertions --reporter-junit-export "Results/report3.xml"
`
I will be updating this to auto scrape the files and run the testing that way... but for now manual beats nothing. Hopefully this helps someone.