Aws-sdk-js: S3.getObject "Converting circular structure to JSON" error in node 8.10

[SamWSoftware]

const AWS = require('aws-sdk');
const s3 = new AWS.S3();

exports.handler = async (event) => {
    // TODO implement

    var params = {
        Bucket: 'bucketName',
        Key: `Key`
    };

    return s3.getObject(params, function(err, data) {
        if (err) { // an error occurred
        console.log(err)
            return 'Hello from Lambda!, error';
        } else { // successful response
            console.log(data);
            return 'Hello from Lambda! data';
        }
    });

};

{
  "errorMessage": "Converting circular structure to JSON",
  "errorType": "TypeError",
  "stackTrace": []
}

```
START RequestId: f66642cd-6561-11e8-9101-45144114db6b Version: $LATEST
Unable to stringify response body as json: Converting circular structure to JSON: TypeError
at JSON.stringify ()
2018-06-01T06:06:46.750Z f66642cd-6561-11e8-9101-45144114db6b { AcceptRanges: 'bytes',
LastModified: 2018-05-13T08:32:08.000Z,
ContentLength: 198,
ETag: '"6c437a16c722e7f90b76dc3198355248"',
ContentType: 'application/json',
Metadata: {},
Body: END RequestId: f66642cd-6561-11e8-9101-45144114db6b
REPORT RequestId: f66642cd-6561-11e8-9101-45144114db6b Duration: 936.36 ms Billed Duration: 1000 ms Memory Size: 128 MB Max Memory Used: 32 MB
````

[SamWSoftware]

This code works fine when you change to node 6.10 using callbacks

[AllanZhengYP]

Hey @SamWSoftware
I cannot reproduce this issue. I notice that you actually get the Body back. Can you try the log the data you get like this:

exports.handler = (event, context, callback) => {
    // TODO implement

    var params = {
        Bucket: 'bucketName',
        Key: `Key`
    };

    return s3.getObject(params, function(err, data) {
        if (err) { // an error occurred
            console.log(err)
            callback('Hello from Lambda!, error')
        } else { // successful response
            console.log(data.Body.toString('utf8'));
            callback(null, 'Hello from Lambda! data')
        }
    });
};

I don't think node 6.10 supports async methods. Can you make sure this the snippet you are using?

[SamWSoftware]

I had to promisify the .getObject to make it work with async

[Vadorequest]

I have a similar error, migrating an AWS Lambda (that is runnin Next.js powered website) from 6 to 8, and it's failing with the same error and nothing else in the logs. No way to know what's causing this.

2018-06-19 20:01:50.691 (+02:00)        d6dadfff-73ea-11e8-8235-83b997ef5a85    2018-06-19 18:01:50 req from http://staging.hep.loan-advisor.studylink.fr/
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

Unable to stringify response body as json: Converting circular structure to JSON: TypeError

2018-06-19 20:01:50.912 (+02:00)        d7041355-73ea-11e8-804b-f5627adbf026    2018-06-19 18:01:50 req from http://staging.hep.loan-advisor.studylink.fr/
2018-06-19 20:01:50.952 (+02:00)        d70c292b-73ea-11e8-860b-e33ea957669f    2018-06-19 18:01:50 req from http://staging.hep.loan-advisor.studylink.fr/
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

2018-06-19 20:01:51.232 (+02:00)        d73e0f5e-73ea-11e8-8a97-49f136cb780a    2018-06-19 18:01:51 req from http://staging.hep.loan-advisor.studylink.fr/
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

2018-06-19 20:01:51.150 (+02:00)        d7400ac6-73ea-11e8-b19b-8dd89a20a874    2018-06-19 18:01:51 req from http://staging.hep.loan-advisor.studylink.fr/
END RequestId: d45bd68f-73ea-11e8-96ae-97c810255855
REPORT RequestId: d45bd68f-73ea-11e8-96ae-97c810255855  Duration: 6003.99 ms    Billed Duration: 6000 ms        Memory Size: 128 MB     Max Memory Used: 48 MB  

2018-06-19T18:01:51.757Z d45bd68f-73ea-11e8-96ae-97c810255855 Task timed out after 6.00 seconds
TypeError
END RequestId: d4992e1a-73ea-11e8-80ae-bfc369787d66
REPORT RequestId: d4992e1a-73ea-11e8-80ae-bfc369787d66  Duration: 6005.77 ms    Billed Duration: 6000 ms        Memory Size: 128 MB     Max Memory Used: 43 MB  

2018-06-19T18:01:52.130Z d4992e1a-73ea-11e8-80ae-bfc369787d66 Task timed out after 6.01 seconds
TypeError
START RequestId: d87a3803-73ea-11e8-b8dd-d3ff652bc1cc Version: $LATEST
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

2018-06-19 20:01:53.033 (+02:00)        d87a3803-73ea-11e8-b8dd-d3ff652bc1cc    2018-06-19 18:01:53 req from http://staging.hep.loan-advisor.studylink.fr/
START RequestId: d9622768-73ea-11e8-99f4-4d4070fb60ab Version: $LATEST
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

2018-06-19 20:01:54.736 (+02:00)        d9622768-73ea-11e8-99f4-4d4070fb60ab    2018-06-19 18:01:54 req from http://staging.hep.loan-advisor.studylink.fr/
END RequestId: d635515f-73ea-11e8-931c-792352da3d2a
REPORT RequestId: d635515f-73ea-11e8-931c-792352da3d2a  Duration: 6003.08 ms    Billed Duration: 6000 ms        Memory Size: 128 MB     Max Memory Used: 48 MB  

2018-06-19T18:01:54.829Z d635515f-73ea-11e8-931c-792352da3d2a Task timed out after 6.00 seconds
TypeError
START RequestId: da461f49-73ea-11e8-931c-792352da3d2a Version: $LATEST
END RequestId: d6dadfff-73ea-11e8-8235-83b997ef5a85
REPORT RequestId: d6dadfff-73ea-11e8-8235-83b997ef5a85  Duration: 6006.63 ms    Billed Duration: 6000 ms        Memory Size: 128 MB     Max Memory Used: 41 MB  

2018-06-19T18:01:55.918Z d6dadfff-73ea-11e8-8235-83b997ef5a85 Task timed out after 6.01 seconds
TypeError
Unable to stringify response body as json: Converting circular structure to JSON: TypeError

Logs are a mess, I took the first and last reference of d6dadfff-73ea-11e8-8235-83b997ef5a85 and everything in between.

Works fine when deploying using node6. If you have any idea how to get better logs it'd be great, just saying! :)

[chrisradek]

@Vadorequest
Can you share an example of the code you're calling? What are you doing with the data object returned by the service? For example, are you passing it to console.log, the lambda handler's callback function, or using JSON.stringify?

[Vadorequest]

What I'm calling is full-featured website powered by https://github.com/zeit/next.js, therefore I can't really share any relevant code, it's way too complex.

Also, as far as I know, it's not using aws-sdk, but since it's a framework I cannot be sure either. (there is no db connection/fetching for sure, but the sdk could be used to do other things)

I mean, the issue could be related to aws-sdk, or could not be, I don't know. And the lack of stacktrace sure doesn't help.

Here is something that looks like what I've got, it was a POC of serverless + next.js frameworks, to power next.js on AWS lambda: https://github.com/Vadorequest/serverless-with-next

[taylor-zr]

Internal customer here. The code doesn't have to be complex to reproduce:

var aws = require('aws-sdk')
var s3 = new aws.S3();
var params = {Bucket: 'dms-taylorzr', Key: 'testlocal'};

exports.handler = async (event) => {
    const response = await s3.getObject(params, function(err, data){
        if(err) {
            console.log(err)
        } else {
            console.log(data);
        }
    });

    return response.Body.toString();
};

Haven't tested it locally, but I am getting this occurring 100% of the time from a Lambda invocation.

[srchase]

@taylor-zr

The 'response' you're attempting to return is the actual request, which has a circular reference to itself, resulting in the error you're seeing.

Could you try this instead?

exports.handler = async (event) => {
    return await s3.getObject(params).promise()
    .then((res) => {
        return res.Body.toString('utf-8');
    })
    .catch((err) => {
        return err;
    });
};

[taylor-zr]

Thanks! Still very new to node, I appreciate the help.

[lock[bot]]

This thread has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs and link to relevant comments in this thread.