Aws-sdk-js: TypeError: Cannot read property 'push' of undefined in amazon sdk file

Created on 6 Jun 2019  路  4Comments  路  Source: aws/aws-sdk-js

TypeError: Cannot read property - Hitting this error with both scan and query functions
This error is hit often when a scan or query function is called. Perhaps sometimes it works well too. And the trace shows that it's been hit in the library file. Copying the trace below.

{
  TypeError: Cannot read property 'push' of undefined
    at Request.HTTP_DATA (/var/runtime/node_modules/aws-sdk/lib/event_listeners.js:376:35)
    at Request.callListeners (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/runtime/node_modules/aws-sdk/lib/request.js:683:14)
    at IncomingMessage.onReadable (/var/runtime/node_modules/aws-sdk/lib/event_listeners.js:276:32)
    at emitNone (events.js:106:13)
    at IncomingMessage.emit (events.js:208:7)
    at emitReadable_ (_stream_readable.js:513:10)
    at emitReadable (_stream_readable.js:507:7)
    at addChunk (_stream_readable.js:274:7)
  message: 'Cannot read property \'push\' of undefined',
  code: 'TypeError',
  time: 2019-06-06T05:54:33.513Z,
  statusCode: 200,
  retryable: false,
  retryDelay: 47.396740262614536
}

EDIT(trivikr): Put Error inside console block

third-party
Source
picture bisjo

Most helpful comment

I used to call scan as follows:
retrievedData = await dynamoDB.scan(params, scanDynamo).promise();
And this produced the error in my previous post.

However, the following does not produce the same error as above:

retrievedData = new Promise((resolve, reject) => {
  dynamoDB.scan(params, (err, data) => {
    if (err) {
      reject(err);
    } else {
      resolve(data);
    }
  });
});

Also see:
https://github.com/aws/aws-sdk-js/issues/1453#issuecomment-375509554

picture mobilfid on 17 Jul 2019
👍5 🎉1

All 4 comments

I am getting this issue as well when using callbacks on the uploadPart function.

I am using andrewrk/node-s3-client code, and it keeps erroring out with this call stack and error pointing to Request.HTTP_DATA() function resp.httpResponse.buffers as being undefined.

function makeUploadPartFn(start, end, part, uploadId) {
  return function(cb) {
    doWithRetry(tryUploadPart, self.s3RetryCount, self.s3RetryDelay, function(
      err,
      data
    ) {
      if (fatalError) return;
      if (err) return handleError(err);
      uploader.emit("part", data);
      cb();
    });
  };

  function tryUploadPart(cb) {
    if (fatalError) return;
    self.s3Pend.go(function(pendCb) {
      if (fatalError) {
        pendCb();
        return;
      }
      var inStream = localFileSlicer.createReadStream({
        start: start,
        end: end
      });
      var errorOccurred = false;
      inStream.on("error", function(err) {
        if (fatalError || errorOccurred) return;
        handleError(err);
      });
      s3Params.ContentLength = end - start;
      s3Params.PartNumber = part.PartNumber;
      s3Params.UploadId = uploadId;

      var multipartETag = new MultipartETag({
        size: s3Params.ContentLength,
        count: 1
      });
      var prevBytes = 0;
      var overallDelta = 0;
      var pend = new Pend();
      var haveETag = pend.hold();
      multipartETag.on("progress", function() {
        if (fatalError || errorOccurred) return;
        var delta = multipartETag.bytes - prevBytes;
        prevBytes = multipartETag.bytes;
        uploader.progressAmount += delta;
        overallDelta += delta;
        uploader.emit("progress");
      });
      multipartETag.on("end", function() {
        if (fatalError || errorOccurred) return;
        var delta = multipartETag.bytes - prevBytes;
        uploader.progressAmount += delta;
        uploader.progressTotal += end - start - multipartETag.bytes;
        uploader.emit("progress");
        haveETag();
      });
      inStream.pipe(multipartETag);
      s3Params.Body = multipartETag;

      self.s3.uploadPart(extend({}, s3Params), function(err, data) {
        pendCb(); // ---- being called twice right here  ----  error stack points at resp.httpResponse.buffers as being undefined
        if (fatalError || errorOccurred) return;
        if (err) {
          errorOccurred = true;
          uploader.progressAmount -= overallDelta;
          cb(err);
          return;
        }
        pend.wait(function() {
          if (fatalError) return;
          if (!compareMultipartETag(data.ETag, multipartETag)) {
            errorOccurred = true;
            uploader.progressAmount -= overallDelta;
            cb(new Error("ETag does not match MD5 checksum"));
            return;
          }
          part.ETag = data.ETag;
          cb(null, data);
        });
      });
    });
  }
}

If there is an error or HTTP_DONE is called shouldn't it remove all other eventlisteners that where added to that event so they no longer get called even if there are outstanding HTTP_DATA listeners that haven't come back yet? This is the only place I can see that is deleting the key in question.

EDIT(trivikr): Added javascript block for code

PsyTae picture PsyTae on 5 Jul 2019

Having the same issue with Scan:

Error [TypeError]: Cannot read property 'push' of undefined
    at Request.HTTP_DATA (projectPath/node_modules/aws-sdk/lib/event_listeners.js:406:35)
    at Request.callListeners (projectPath/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (projectPath/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (projectPath/node_modules/aws-sdk/lib/request.js:683:14)
    at IncomingMessage.onReadable (projectPath/node_modules/aws-sdk/lib/event_listeners.js:289:32)
    at IncomingMessage.emit (events.js:200:13)
    at IncomingMessage.EventEmitter.emit (domain.js:471:20)
    at emitReadable_ (_stream_readable.js:558:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:9) {
  message: "Cannot read property 'push' of undefined",
  code: 'TypeError',
  time: 2019-07-17T09:02:24.782Z,
  statusCode: 200,
  retryable: false,
  retryDelay: 57.894922684251135
}
  Error: TypeError: Cannot read property 'push' of undefined
      at Request.HTTP_DATA (projectPath/node_modules/aws-sdk/lib/e  vent_listeners.js:406:35)
      at Request.callListeners (projectPath/node_modules/aws-sdk/l  ib/sequential_executor.js:106:20)
      at Request.emit (projectPath/node_modules/aws-sdk/lib/sequen  tial_executor.js:78:10)
      at Request.emit (projectPath/node_modules/aws-sdk/lib/reques  t.js:683:14)
      at IncomingMessage.onReadable (projectPath/node_modules/aws-  sdk/lib/event_listeners.js:289:32)
      at IncomingMessage.emit (events.js:200:13)
      at IncomingMessage.EventEmitter.emit (domain.js:471:20)  at emitReadable_ (_stream_readable.js:558:12)
      at processTicksAndRejections (internal/process/task_queues.js:84:9)

Error [TypeError]: Cannot read property 'push' of undefined
    at Request.HTTP_DATA (projectPath/node_modules/aws-sdk/lib/event_listeners.js:406:35)
    at Request.callListeners (projectPath/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
    at Request.emit (projectPath/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
    at Request.emit (projectPath/node_modules/aws-sdk/lib/request.js:683:14)
    at IncomingMessage.onReadable (projectPath/node_modules/aws-sdk/lib/event_listeners.js:289:32)
    at IncomingMessage.emit (events.js:200:13)
    at IncomingMessage.EventEmitter.emit (domain.js:471:20)
    at emitReadable_ (_stream_readable.js:558:12)
    at processTicksAndRejections (internal/process/task_queues.js:84:9) {
  message: "Cannot read property 'push' of undefined",
  code: 'TypeError',
  time: 2019-07-17T09:02:24.782Z,
  statusCode: 200,
  retryable: false,
  retryDelay: 57.894922684251135
}

(Dirty) Fix/Solution: -- In event_listener.js, Line 400 -- Change the following:

add("HTTP_DONE", "httpDone", function HTTP_DONE(resp) {
  // convert buffers array into single buffer
  if (resp.httpResponse.buffers && resp.httpResponse.buffers.length > 0) {
    var body = AWS.util.buffer.concat(resp.httpResponse.buffers);
    resp.httpResponse.body = body;
  }
  delete resp.httpResponse.numBytes;
  delete resp.httpResponse.buffers;
});

to:

add("HTTP_DONE", "httpDone", function HTTP_DONE(resp) {
  // convert buffers array into single buffer
  if (resp.httpResponse.buffers && resp.httpResponse.buffers.length > 0) {
    var body = AWS.util.buffer.concat(resp.httpResponse.buffers);
    resp.httpResponse.body = body;
  }
  delete resp.httpResponse.numBytes;
  resp.httpResponse.buffers = [];
});

Changing Line 400:
delete resp.httpResponse.buffers;
to:
resp.httpResponse.buffers = [];
fixed my randomly appearing "Cannot read property 'push' of undefined" problem.

EDIT(trivikr): Added console and javascript block, and prettified code

mobilfid picture mobilfid on 17 Jul 2019
👍1

I used to call scan as follows:
retrievedData = await dynamoDB.scan(params, scanDynamo).promise();
And this produced the error in my previous post.

However, the following does not produce the same error as above:

retrievedData = new Promise((resolve, reject) => {
  dynamoDB.scan(params, (err, data) => {
    if (err) {
      reject(err);
    } else {
      resolve(data);
    }
  });
});

Also see:
https://github.com/aws/aws-sdk-js/issues/1453#issuecomment-375509554

mobilfid picture mobilfid on 17 Jul 2019
👍5 🎉1

I tried the above method on how to change in event_listener.js line 400. There is the following code I wrote for updating DynamoDB. It was successful all the time!

https://github.com/aws/aws-sdk-js/issues/2700#issuecomment-512175014

module.exports.updateExtermination = async (instanceId, cveNum) => {
  const params = {
    TableName: tableName,
    Key: { 'instanceid': instanceId, 'cveid': cveNum },
    UpdateExpression: 'set extermination = :e',
    ExpressionAttributeValues: {
      ':e': new Date().toISOString()
    },
    ReturnValues: 'UPDATED_NEW'
  }
  return await dynamodb.update(params).promise()
}
komayama picture komayama on 17 Mar 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

ataraxus picture ataraxus  路  4Comments
pihish picture pihish  路  4Comments
moltar picture moltar  路  3Comments
adolfosrs picture adolfosrs  路  3Comments
lreher picture lreher  路  3Comments