Newman: POST requests not sending files on newman v3?

Created on 10 Aug 2016 · 13Comments · Source: postmanlabs/newman

Got a collection exported from postman (4.4.3) which makes a GET request for a URL and then POSTs to that URL with a file.
The combo works in postman, the file appearing at the destination.

The exported collection works in newman 2.1.2, again the file appearing at the destination.

However in newman 3.0.0 no file appears. Looking in the summary object i can see a bad multipart request object.

Let me know if you need any further information, hope this helps!

Attaching collection, v3 runner, v2 runner. The summary json for v3 is below:

Object moved to here.

\r\n\r\n","responseTime":24,"responseSize":345,"update":{},"reason":{},"text":{},"json":{},"mime":{},"dataURI":{},"size":{},"describe":{},"toObjectResolved":{},"toJSON":{}},"id":"72957952-1662-477d-aa9b-6a7bb89dfedc"}],"transfers":{"responseTotal":595},"failures":[],"error":null}}
NewmanBug.zip

pending-close v3

Source

samcayford

😕4

Most helpful comment

This issue should have been fixed in Newman v3.1.0, which can be installed with npm i -g newman@beta. Additionally, please change the key value in the formdata array object to src, like so:

"body": {
    "mode": "formdata",
    "formdata": [
        {
            "key": "file",
            "src": "file1.txt",
            "type": "file"
        }
    ]
}

A fix to correct the issue with collection exports will also be added in the near future.

kunagpal on 19 Aug 2016

👍5

All 13 comments

Thanks for reporting. Will get back to you shortly.

shamasis on 10 Aug 2016

I have the same issue.
POST "form-data" body fails: multipart request has not attached file

"requests": [
{
  "id": "056dd3a9-ba88-85d3-a29f-d1962079cba9",
  "method": "POST",
  "data": [
    {
      "key": "order",
      "value": "{   \"products\": [] }",
      "type": "text",
      "enabled": true
    },
    {
      "key": "contract",
      "value": "./contract.pdf",
      "enabled": true,
      "type": "file",
      "mimeType": "application/pdf"
    }
  ],
  "dataMode": "params",
  "name": "Create order",
  "description": "Response is validated to ensure that all necessary links are present etc.",
  "collectionId": "ebbcca65-e781-7897-016f-76f651809092",
  "responses": []
},

valmol on 11 Aug 2016

I also am experiencing this problem. I've replaced the relative paths with absolute ones, as suggested by an earlier tutorial, but still no cigar.
Here is the request in question:

    {
            "name": "ImportSet Files [204]",
            "event": [
                {
                    "listen": "test",
                    "script": {
                        "type": "text/javascript",
                        "exec": "tests[\"Status code is 201\"] = responseCode.code === 201\n\nif (tests[\"Status code is 201\"])\n{\n    //Check for a Valid Schema\n    var schema = JSON.parse(postman.getEnvironmentVariable(\"importSetSchema\"));\n    var data = JSON.parse(responseBody);\n    tests[\"Valid ImportSet Schema\"] = tv4.validateResult(data, schema).valid;\n    \n    //Check for an Invalid Schema match by modifying the schema in the responseBody\n    var invalidschema = JSON.stringify(data).replace(\"state\",\"province\");\n    invalidschema = JSON.parse(invalidschema);\n    tests[\"Invalid ImportSet Schema\"] = tv4.validateResult(invalidschema, schema).valid === false ;\n    \n    //Check for an Invalid Schema Data match by modifying the data in the responseBody\n    var invaliddata = JSON.parse(responseBody);\n    invaliddata.state = \"ZZZzzz\";\n    tests[\"Invalid ImportSet Data\"] = tv4.validateResult(invaliddata, schema).valid === false ;\n    \n    tests[\"State is New\"] = data.state == \"New\";\n    tests[\"Enough GTFS Files\"] = data.allFiles.length > 3\n    tests[\"Has agency.txt\"] = data.allFiles.indexOf(\"agency.txt\") > -1;\n}\n\n"
                    }
                }
            ],
            "request": {
                "url": "{{WriterUrl}}/api/importsets/{{importSetId}}/Files",
                "method": "POST",
                "header": [
                    {
                        "key": "Authorization",
                        "value": "{{BearerToken}}",
                        "description": ""
                    }
                ],
                "body": {
                    "mode": "formdata",
                    "formdata": [
                        {
                            "key": "files",
                            "value": "E:\\Git\\WriterApi\\test\\Postman\\testfeed.zip",
                            "type": "file",
                            "enabled": true
                        }
                    ]
                },
                "description": ""
            },
            "response": []
        },

ncthbrt on 12 Aug 2016

Hi, same problem here !
The same request works on Newman 2.1.2, but since Newman 3, file-POST won't work. The collection is exported from Postman for Mac 4.6.1.

MacBook-Pro:api eric$ node node_modules/newman/bin/newman.js --version
3.0.1

Following, my request.

{
    "name": "PostToS3",
    "event": [
        {
            "listen": "test",
            "script": {
                "type": "text/javascript",
                "exec": "tests[\"Status code is 204\"] = responseCode.code === 204;\n\nif (responseCode.code !== 204) {\n    console.log(responseBody);\n    console.log(\"s3bucket: \" + environment['s3bucket']); \n    console.log(\"s3key: \" + environment['s3key']); \n    console.log(\"s3acl: \" + environment['s3acl']); \n    console.log(\"s3amzalgorithm: \" + environment['s3amzalgorithm']); \n    console.log(\"s3amzcredential: \" + environment['s3amzcredential']); \n    console.log(\"s3amzdate: \" + environment['s3amzdate']); \n    console.log(\"s3amzsignature: \" + environment['s3amzsignature']); \n    console.log(\"s3policy: \" + environment['s3policy']); \n}\n"
            }
        }
    ],
    "request": {
        "url": "http://{{s3bucket}}.s3.amazonaws.com/",
        "method": "POST",
        "header": [],
        "body": {
            "mode": "formdata",
            "formdata": [
                {
                    "key": "all",
                    "value": "{{s3acl}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "key",
                    "value": "{{s3key}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "policy",
                    "value": "{{s3policy}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "x-amz-algorithm",
                    "value": "{{s3amzalgorithm}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "x-amz-credential",
                    "value": "{{s3amzcredential}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "x-amz-date",
                    "value": "{{s3amzdate}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "x-amz-signature",
                    "value": "{{s3amzsignature}}",
                    "type": "text",
                    "enabled": true
                },
                {
                    "key": "file",
                    "type": "file",
                    "enabled": true,
                    "value": "test/assets/cuisine1.jpg"
                }
            ]
        },
        "description": ""
    },
    "response": []
}

DeadEye0112 on 16 Aug 2016

Same problem here.
This is the file form-data (which should get submitted):

"body": {
    "mode": "formdata",
    "formdata": [
        {
            "key": "file",
            "value": "file1.txt",
            "type": "file",
            "enabled": true
        }
    ]
}

But server responds with: Form field with name=file required
Meaning: File (key) doesn't get passed along.

max-preuschen on 16 Aug 2016

This issue should have been fixed in Newman v3.1.0, which can be installed with npm i -g newman@beta. Additionally, please change the key value in the formdata array object to src, like so:

"body": {
    "mode": "formdata",
    "formdata": [
        {
            "key": "file",
            "src": "file1.txt",
            "type": "file"
        }
    ]
}

A fix to correct the issue with collection exports will also be added in the near future.

kunagpal on 19 Aug 2016

👍5

One quick update on this - we are working to ensure that when you export a collection from postman app - the file name should be stored along with it. As such, in Newman, if those files are found relative to the PWD, it will be uploaded.

Thoughts?

shamasis on 1 Sep 2016

@shamasis SGTM.

kunagpal on 1 Sep 2016

I'd be quite happy.

ncthbrt on 2 Sep 2016

@shamasis Could you guys update your Postman documentation here: https://www.getpostman.com/docs/run_file_post_requests to reflect the current state of file posts? It took me quite a while of testing and googling to eventually find this github post.

Also, I'm struggling a bit with how to properly format my CSV and Postman Collection for multiple files. I was trying to have multiple columns in my CSV for "Attachment1", "Attachment2", etc. That works if each row has an attachment for each of those columns, but if a row only has one attachment and I leave the "Attachment2" column blank, it errors out.

Any thoughts on how to deal with this so that I can allow for a dynamic number of files to be uploaded depending on the row in the CSV?

newlandj on 24 Nov 2016

Hey @newlandj, using CSV data files would be a a bit awkward here, as we need fine grained CRUD control over sequence control variables. Allow me to present a configurable alternative, using environment variables, and postman.setNextRequest.

Workflow:

Maintain a starting environment with a variable called allFiles (say). The value of this variable will be a stringified JSON, of the following form:

[
  { "files": [ "file.txt" ] },
  { "files": [ "file.txt", "file.txt" ] },
  { "files": [ "file.txt", "file.txt", "file.txt" ] }
]

Within pre-request scripts, include logic to sequentially pop each element of the parsed JSON value of allFiles.
Next, pop off file values from individual elements obtained in the previous step.
Use the individual popped values in step 3 to set environment variables (The name of this environment variable is pre-used within the collection file, to upload files from.
If validation checks fail, use postman.setNextRequest(null); to prematurely terminate the collection run. Else, use postman.setNextRequest(<upload-request-name>) to keep the collection run going.
Done :tada:!

Sample files:

Collection:

{
    "variables": [],
    "info": {
        "name": "file-uploads",
        "_postman_id": "beb7d901-c9c1-a9e9-762c-9565ebabd161",
        "description": "A set of `POST` requests to upload files as form data fields",
        "schema": "https://schema.getpostman.com/json/collection/v2.0.0/collection.json"
    },
    "item": [
        {
            "name": "Form data file upload",
            "event": [
                {
                    "listen": "test",
                    "script": {
                        "type": "text/javascript",
                        "exec": [
                            "var response = JSON.parse(responseBody).files[\"file.txt\"];",
                            "",
                            "tests[\"Status code is 200\"] = responseCode.code === 200;",
                            "tests[\"File was uploaded correctly\"] = /^data:application\\/octet-stream;base64/.test(response);"
                        ]
                    }
                },
                {
                    "listen": "prerequest",
                    "script": {
                        "type": "text/javascript",
                        "exec": [
                            "var file,",
                            "",
                            "    allFiles = postman.getEnvironmentVariable(\"allFiles\"),",
                            "    currentFiles = postman.getEnvironmentVariable(\"currentFiles\"),",
                            "    ",
                            "    onError = function (error) {",
                            "        console.error(error);",
                            "        return postman.setNextRequest(null);",
                            "    };",
                            "",
                            "console.log('file: ', postman.getEnvironmentVariable(\"file\"));",
                            "console.log('all: ', postman.getEnvironmentVariable(\"allFiles\"));",
                            "console.log('current: ', postman.getEnvironmentVariable(\"currentFiles\"));",
                            "",
                            "if (currentFiles) {",
                            "    try {",
                            "        currentFiles = JSON.parse(currentFiles);",
                            "    }",
                            "    catch (err) {",
                            "        onError(err);",
                            "    }",
                            "}",
                            "else if (allFiles) { // the currentFiles variable does not exist, or is empty",
                            "    try {",
                            "        allFiles = JSON.parse(allFiles);",
                            "",
                            "        if (!_.isArray(allFiles) || _.isEmpty(allFiles)) {",
                            "            return postman.setNextRequest(null);",
                            "        }",
                            "        else {",
                            "            currentFiles = allFiles.pop();",
                            "            postman.setEnvironmentVariable(\"allFiles\", JSON.stringify(allFiles));",
                            "        }",
                            "    }",
                            "    catch (err) {",
                            "        onError(err);",
                            "    }",
                            "}",
                            "else {",
                            "    return postman.setNextRequest(null);",
                            "}",
                            "",
                            "if (!_.isObject(currentFiles) || !_.isArray(currentFiles.files) || _.isEmpty(currentFiles.files)) {",
                            "    if (allFiles) { // the currentFiles variable does not exist, or is empty",
                            "        try {",
                            "            allFiles = JSON.parse(allFiles);",
                            "    ",
                            "            if (!_.isArray(allFiles) || _.isEmpty(allFiles)) {",
                            "                return postman.setNextRequest(null);",
                            "            }",
                            "            else {",
                            "                currentFiles = allFiles.pop();",
                            "                postman.setEnvironmentVariable(\"allFiles\", JSON.stringify(allFiles));",
                            "",
                            "                return postman.setNextRequest(\"Form data file upload\");",
                            "            }",
                            "        }",
                            "        catch (err) {",
                            "            onError(err);",
                            "        }",
                            "    }",
                            "    else {",
                            "        return postman.setNextRequest(null);",
                            "    }",
                            "}",
                            "",
                            "file = currentFiles.files.pop();",
                            "",
                            "postman.setEnvironmentVariable(\"file\", file);",
                            "postman.setNextRequest(\"Form data file upload\");",
                            "postman.setEnvironmentVariable(\"currentFiles\", JSON.stringify(currentFiles));",
                            ""
                        ]
                    }
                }
            ],
            "request": {
                "url": "https://echo.getpostman.com/post",
                "method": "POST",
                "header": [],
                "body": {
                    "mode": "formdata",
                    "formdata": [
                        {
                            "key": "file",
                            "type": "file",
                            "enabled": true,
                            "src": "{{file}}",
                            "value": ""
                        }
                    ]
                },
                "description": "Uploads a file as a form data field to `https://echo.getpostman.com/post` via a `POST` request."
            },
            "response": []
        }
    ]
}

Environment:

{
  "id": "84f2eecb-933d-6880-e91f-5264269a8597",
  "name": "dynamic-uploads",
  "values": [
    {
      "enabled": true,
      "key": "allFiles",
      "type": "text",
      "value": "[{\"files\":[\"file.txt\"]},{\"files\":[\"file.txt\",\"file.txt\"]},{\"files\":[\"file.txt\",\"file.txt\",\"file.txt\"]}]"
    }
  ],
  "timestamp": 1479996222649,
  "_postman_variable_scope": "environment",
  "_postman_exported_at": "2016-11-24T14:03:45.406Z",
  "_postman_exported_using": "Postman/4.8.3"
}

Note: file.txt should be placed relative to the collection file's location. For the above example to work, file.txt should be in the same directory as the collection file.
Feel free to get back to me regarding any questions that you might have.

kunagpal on 24 Nov 2016

Thanks @kunagpal I appreciate the detailed response. I'm a bit too much of an amateur to be able to implement that solution I think, and I have some work-arounds that will get me there. I was hoping I could just put a variable in my CSV to say whether an attachment block was enabled with a "true" or "false" variable for a particular row in the CSV, but it doesn't appear that the "enabled" key does anything with the attachments. It still tries to upload even when I have it set to False.

Last question I suppose. I could easily concatenate the file names in Excel if there was a particular way to format them so that I could pass one to many file names into a single upload call. I was hoping there was a way to do this since you can select multiple files to upload from a single Post call inside of Postman.

Any chance I could do something fancy in Excel to make this work?

newlandj on 26 Nov 2016

Hey kunagpal,
I have been really trying hard to do a file upload using the newman collection,and i tried the above code that you had given for a try and when I execute the collection from newman,I got the following ...can you please look in to it,(It seems the file is not getting uploaded based on the total data received )

G:>newman run 1.json -e 2.json
newman

file-uploads

→ Form data file upload
┌
│ 'file: ', null
│ 'all: ', '[{"files":["file.txt"]},{"files":["file
│ .txt","file.txt"]},{"files":["file.txt","file.txt","file.txt"]}]'
│ 'current: ', null
└
POST https://echo.getpostman.com/post [404 Not Found, 418B, 4s]
1? JSONError in test-script

→ Form data file upload
┌
│ 'file: ', 'file.txt'
│ 'all: ', '[{"files":["file.txt"]},{"files":["file
│ .txt","file.txt"]}]'
│ 'current: ', '{"files":["file.txt","file.txt"]}'
│ [39m
└
POST https://echo.getpostman.com/post [404 Not Found, 292B, 510ms]
2? JSONError in test-script

→ Form data file upload
┌
│ 'file: ', 'file.txt'
│ 'all: ', '[{"files":["file.txt"]},{"files":["file
│ .txt","file.txt"]}]'
│ 'current: ', '{"files":["file.txt"]}'
└
POST https://echo.getpostman.com/post [404 Not Found, 292B, 507ms]
3? JSONError in test-script

→ Form data file upload
┌
│ 'file: ', 'file.txt'
│ 'all: ', '[{"files":["file.txt"]},{"files":["file
│ .txt","file.txt"]}]'
│ 'current: ', '{"files":[]}'
└
POST https://echo.getpostman.com/post [404 Not Found, 292B, 549ms]
4? JSONError in test-script

→ Form data file upload
┌
│ 'file: ', 'file.txt'
│ 'all: ', '[{"files":["file.txt"]}]'
│ 'current: ', '{"files":[]}'
└
POST https://echo.getpostman.com/post [404 Not Found, 292B, 555ms]
5? JSONError in test-script

→ Form data file upload
┌
│ 'file: ', 'file.txt'
│ 'all: ', '[]'
│ 'current: ', '{"files":[]}'
└
POST https://echo.getpostman.com/post [404 Not Found, 292B, 597ms]
6? JSONError in test-script

┌─────────────────────────┬──────────┬──────────┐
│ │ executed │ failed │
├─────────────────────────┼──────────┼──────────┤
│ iterations │ 1 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ requests │ 6 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ test-scripts │ 6 │ 6 │
├─────────────────────────┼──────────┼──────────┤
│ prerequest-scripts │ 6 │ 0 │
├─────────────────────────┼──────────┼──────────┤
│ assertions │ 0 │ 0 │
├─────────────────────────┴──────────┴──────────┤
│ total run duration: 7.3s │
├───────────────────────────────────────────────┤
│ total data received: 0B (approx) │
├───────────────────────────────────────────────┤
│ average response time: 1117ms │
└───────────────────────────────────────────────┘

# failure detail

JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""
JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""
JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""
JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""
JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""
JSONError No data, empty input at 1:1
^
at test-script
inside "Form data file upload" of ""