Aws-xray-sdk-node: HTTPsCaptureGlobal captures sampling API calls from X-Ray SDK

[awssandra]

Problem:

Currently, when using the captureHTTPsGlobal function, this patches every instance of the http built-in library. This means that the AWS SDK proxied calls the X-Ray SDK makes to getSamplingRules and getSamplingTargets are attempting to be traced.

Because there is no context to resolve the segment, this fails, however, this creates the log entries as follows:

INFO - Options for request [ host: 127.0.0.1, method: POST, path: /SamplingTargets ] is missing the sub/segment context for automatic mode. Ignoring.

Repro Case:

Configure a basic sample app that utilizes the sampling APIs. Use xray.captureHTTPsGlobal() as a part of the initialization logic, and view logs on the running sample app. The logs should display the text above.

Proposed Solution A:

The Node SDK already resolves a similar issue on the customer's AWS SDK calls by ensuring the HTTP raw call is not recorded (again) further down the stack, by checking if the 'X-Amzn-Trace-Id' header is on the request. If so, it is ignored entirely. https://github.com/aws/aws-xray-sdk-node/blob/master/packages/core/lib/patchers/http_p.js#L59

Simple fix, inject an empty 'X-Amzn-Trace-Id' header on the outgoing request(s):
https://github.com/aws/aws-xray-sdk-node/blob/master/packages/core/lib/middleware/sampling/service_connector.js#L24
https://github.com/aws/aws-xray-sdk-node/blob/master/packages/core/lib/middleware/sampling/service_connector.js#L37

Most likely easiest to utilize the 'customizeRequests' function for the X-Ray service client.
Sample implementation here:
https://github.com/aws/aws-xray-sdk-node/blob/master/packages/core/lib/patchers/aws_p.js#L56

Proposed Solution B:

Add a new "uncapture" function. Construct a new HTTP object and "uncapture" the HTTP client. Uncapture would work by checking the HTTP client object for the base "__get" and "__request" functions (which were moved by the original capture function patching). If present, replace the "get" and "request" functions with the original "__get" and "__request" functions.

https://github.com/aws/aws-xray-sdk-node/blob/master/packages/core/lib/patchers/http_p.js#L150

Edit: This implementation may be trickier than it seems, given we don't have control or an intercept for the http client from the AWS SDK's end. May be more work than solution A.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[willarmiros]

Commenting to prevent closure.

[dt-atkinson]

Have this issue also

[willarmiros]

Thank you for letting us know. We are prioritizing a fix for this issue as soon as we can.

[willarmiros]

Resolved in 3.0.0-alpha.2.