Newman: Variables set with pm.collectionVariables.set get converted from numbers to strings

[pavellishin]

I've encountered an odd problem that doesn't manifest in postman, but does in Newman. When I set collection variables in the pre-request script tab, they appear to be converted into strings.

1. Newman Version (can be found via newman -v): 4.5.7
2. OS details (type, version, and architecture): OS X High Sierra 10.13.6 (although also manifests in Jenkins)
3. Are you using Newman as a library, or via the CLI? CLI
4. Did you encounter this recently, or has this bug always been there: No idea!
5. Expected behaviour: When comparing against collection variables, preserve type.
6. Command / script used to run Newman: newman run https://www.getpostman.com/collections/a66a2b24bf49a0126bdc
7. Sample collection, and auxiliary files (minus the sensitive details): https://www.getpostman.com/collections/a66a2b24bf49a0126bdc
8. Screenshots (if applicable): See below

Reproducing the problem:

1. Open Postman app.
2. Create a new request, doesn't matter what kind. (I used GET http://example.org)
3. Set up a numeric collection variable in the Pre-request Script tab:

pm.collectionVariables.set('aNumber', 137);

1. Add tests verifying that the variable is a number:

pm.test("Is this a number?", () => {
    const aNumber = pm.variables.get("aNumber");
    pm.expect(aNumber).to.equal(137);
    pm.expect(typeof aNumber).to.equal("number");
});

1. Execute request in Postman app, and watch them succeed.

2. Execute request in newman CLI, and watch them fail:

# newman run https://www.getpostman.com/collections/a66a2b24bf49a0126bdc
newman: Newman v4 deprecates support for the v1 collection format
  Use the Postman Native app to export collections in the v2 format

newman

Newman Bug 20191209

→ Bug demo
  GET http://example.org/ [200 OK, 1014B, 97ms]
  1. Is this a number?

┌─────────────────────────┬──────────────────┬──────────────────┐
│                         │         executed │           failed │
├─────────────────────────┼──────────────────┼──────────────────┤
│              iterations │                1 │                0 │
├─────────────────────────┼──────────────────┼──────────────────┤
│                requests │                1 │                0 │
├─────────────────────────┼──────────────────┼──────────────────┤
│            test-scripts │                1 │                0 │
├─────────────────────────┼──────────────────┼──────────────────┤
│      prerequest-scripts │                1 │                0 │
├─────────────────────────┼──────────────────┼──────────────────┤
│              assertions │                1 │                1 │
├─────────────────────────┴──────────────────┴──────────────────┤
│ total run duration: 174ms                                     │
├───────────────────────────────────────────────────────────────┤
│ total data received: 648B (approx)                            │
├───────────────────────────────────────────────────────────────┤
│ average response time: 97ms [min: 97ms, max: 97ms, s.d.: 0µs] │
└───────────────────────────────────────────────────────────────┘

  #  failure                                                                          detail

 1.  AssertionError                                                                   Is this a number?
                                                                                      expected '137' to equal 137
                                                                                      at assertion:0 in test-script
                                                                                      inside "Bug demo"

[carolmirakove]

@pavellishin I encountered this problem too and am getting around it with this workaround:
https://stackoverflow.com/a/57172160/3903984

[pavellishin]

Yup, I'm using something similar, but I'd rather see this fixed. Right now, we don't care about the string-vs-number distinction, but in the future the type may matter.

[im-Amitto]

@pavellishin It's not that pm.collectionVariables.set is converting the int into a string but when you initialized the aNumber in the variable key of your collection you gave it a value "" which is a string.

In sort, you initialized your variable with a string value and therefore the pm.collectionVariables.set is treating it as a string.

If you initialize your variable with 0 instead of "". your test should work fine.

.
.
.
"variables": [
    {
      "key": "aNumber",
-     "value": "",
+     "value": 0,
      "disabled": false
    }
  ],
.
.
.

[pavellishin]

In sort, you initialized your variable with a string value and therefore the pm.collectionVariables.set is treating it as a string.

But I didn't do that. I'm not sure how those lines get generated by Postman itself, but I didn't take any explicit steps to initialize that data. Plus, those tests work fine in Postman, but fail in Newman.

To me this suggests that whatever is generating that JSON has a bug in it; it shouldn't be initializing a variable to any value at all.

Also not sure why it magically casts my explicit value of 137 into a string...

[shamasis]

Hi. Giving some context to the issue here so that we can decide how to proceed. All Postman variables are supposed to be strings. Though we internally support a number of types, it is not yet completely baked into the product (both app and newman needs to build this in tandem in order to ensure it works seamlessly.)

As such, a number of issues manifest due to JavaScript quirks as we try to cast non string get/set into strings. There have also been cases where we missed putting casting logic in certain flows and instead handled it by mistake in app UI flows (as such creating disparity between how app and newman works.) There have also been cases where it was very surprising to users, who did not know about the string type aspect, to suddenly see their numbers become string and such.

Hope this helps.

[pavellishin]

All Postman variables are supposed to be strings.

Oof, I didn't know that. I assumed they could be anything, since the GUI app supports this.

Does this mean that variables can only be simple scalar values, and not objects?

[shamasis]

True. Variables can be scalar values and even Postman GUI supports only scalar values.

But if you’re using the variables programmatically, you can apply casting shenanigans to convert objects to string and vice versa.