Aws-sdk-js: [Request] Improve stack traces

[awerlang]

Please, how can we have more helpful information in stack traces for DynamoDB (haven't confirmed for other services, like S3 or SES, but perhaps that's the same thing), to make it easier to make sense of it?

This is a sample stack trace for ProvisionedThroughputExceededException, but it's the same thing for invalid requests like invalid values sent over to DynamoDB.

I installed longjohn package but it didn't captured any trace of my application code.

Stack trace: ProvisionedThroughputExceededException: The level of configured provisioned throughput for the table was exceeded. Consider increasing your provisioning level with the UpdateTable API.
    at Request.extractError (/var/app/current/node_modules/aws-sdk/lib/protocol/json.js:43:27)
    at Request.callListeners (/var/app/current/node_modules/aws-sdk/lib/sequential_executor.js:105:20)
    at Request.emit (/var/app/current/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/app/current/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/var/app/current/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/app/current/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/app/current/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/app/current/node_modules/aws-sdk/lib/request.js:38:9)
    at Request.<anonymous> (/var/app/current/node_modules/aws-sdk/lib/request.js:670:12)
    at Request.callListeners (/var/app/current/node_modules/aws-sdk/lib/sequential_executor.js:115:18)
    at Request.emit (/var/app/current/node_modules/aws-sdk/lib/sequential_executor.js:77:10)
    at Request.emit (/var/app/current/node_modules/aws-sdk/lib/request.js:668:14)
    at Request.transition (/var/app/current/node_modules/aws-sdk/lib/request.js:22:10)
    at AcceptorStateMachine.runTo (/var/app/current/node_modules/aws-sdk/lib/state_machine.js:14:12)
    at /var/app/current/node_modules/aws-sdk/lib/state_machine.js:26:10
    at Request.<anonymous> (/var/app/current/node_modules/aws-sdk/lib/request.js:38:9)

[jeskew]

The stack traces from the DynamoDB client would be significantly longer than stack traces from SES or S3 due to the higher number of retries enabled by default for that service.

I haven't used longjohn, but their documentation states that stack trace length is configurable. You should be able to set that configuration parameter to -1 to get a full trace.

[awerlang]

By default it limits on 10 frames, but it doesn't makes any difference, as a single frame is captured.

[jeskew]

So is the issue that the message provided by the service could be clearer?

[awerlang]

I'd like to see on the stack trace a trace to the originating call site, something that points to user code. It is okay to achieve this by requiring use of longjohn, as a regular stack trace usually points to library code anyways.

[wmonk]

Any updates on this issue? In certain situations it's hard to work out where the error actually originates from in my own code.

[yuta-imai]

I'm also struggling here.

[philiiiiiipp]

We have been experimenting with a couple of solutions to give us stack traces with the focus of getting the original place in our code when using dynamodb. We came up with is something along the lines of:

const __client = new DynamoDB.DocumentClient()

const methods = [
  'batchGet',
  'batchWrite',
  'createSet',
  'delete',
  'get',
  'put',
  'query',
  'scan',
  'transactGet',
  'transactWrite',
  'update',
];

// @ts-ignore
export const Client = methods.reduce((prev, method) => {
  // @ts-ignore
  prev[method] = (...args) => {
    return {
      promise: async () => {
        const error = new Error();

        return __client[method](...args)
          .promise()
          .catch(async (e) => {
            e.message +=
              '\nExtended stack\n' +
              error.stack +
              '\n' +
              util.inspect(args, {
                showHidden: false,
                depth: null,
              });
          });
      },
    };
  };

  return prev;
}, {});

We currently only use this when running our tests locally, so we have not tested this in production or under load, but it has been very helpful there.

[ajredniwja]

Hey everyone, this needed a major version change which has been out now. V3 of the SDK should fix this.

The version 3.x of the AWS SDK for JavaScript is generally available. For more information see the Developer Guide or API Reference.

[philiiiiiipp]

Hey @ajredniwja I tried it and it seems to work there 🎉. The API's changed quite a bit, but the moment it has support for the DocumentClient I would happily switch.

This is probably not the right place to ask this, but while on the topic,

1. Do you know what the idea is to make v3 available inside the Lambda runtime?
2. Will v2 become deprecated ?