Dd-trace-js: Error in span does not cause entire request to report error in APM

[alexrhogue]

I am having an issue with getting an error in a span to propagate up and cause the entire trace to be marked as a failure.

I want to provide my team with the ability to notice a critical error in an http request (even if the code catches that error and handles it graceful). To achieve this I am create a span and causing it to error out. This works. However, the trace created by the express plugin (and I assume the http plugins as well) does not seem to care about a failed.

I think I must be missing something. But I can't find any clear examples or documentation on how to achieve this.

example code:

  tracer.trace('noticeError', (span, onError) => {
        onError(new Error('notice error'));
    });

Environment

• Operation system:
  OSX 10.14.3
• Node version:
  v11.15.0
• Tracer version:
  [email protected]
• Agent version:
  6.14.1

[rochdev]

I'm not sure to understand exactly the expected behavior. Are you trying to add an error to a specific span, or add an error to the root span of the tracer? Or are you trying to add an error to the root span whenever there is another span that has an error?

[alexrhogue]

Thanks for the quick reply. I am trying to add an error to the root span of the tracer.

[rochdev]

Are errors reported as 200 status code? You can control which status codes result in a trace-level error. Otherwise, I'm afraid I'll have to provide a workaround since the tracer considers a handled error as a success from the request perspective.

[rochdev]

Another question: do you expect any error on any span to flag the trace with an error, or only specific ones?

[alexrhogue]

1) The errors are reported as 200 status code
2) Just a specific one

[rochdev]

Ok, then it should be possible to flag the request and use that on the request span in a hook. It would look like this:

// right after tracer.init()
tracer.use('express', {
  hooks: {
    request: (span, req, res) => {
      // assuming somewhere in your app you set req.error = true
      if (req.error) {
        span.setTag('error', req.error)
      }
    }
  }
})

You could also set the error to an actual Error object instead of true but I wouldn't recommend it. This would give the impression that the error was thrown by the request (i.e. actually bubbled up) and would be redundant since you will be able to find the real error on one of the spans in the trace. It's up to you however what works best for your use case.

Please let me know if that helps!

[alexrhogue]

Trying that right now. Thank you.

[alexrhogue]

Seems to be working great. Request is showing up as an error. And I can inspect the spans to find the actual error. Only issue I see is that some code paths where I may want to 'noticeError' may not currently have access to the req.

I'm currently attempting to search through the spans in the express middleware to find my error span. I'll keep you posted!

Thanks again.

[rochdev]

It's also possible to access the request span from any span at any time during the request lifecycle, but there is no public API to do it yet so you would have to rely on private properties.

If this is something you are comfortable with, it would look like this:

// anywhere during the request lifecycle
const span = tracer.scope().active()

if (span) {
  span.context()._trace.started[0].setTag('error', true)
}

[alexrhogue]

I think that is exactly what I need. Having to use private props is not ideal but I think I can work with it for now. Should I open another issue to request that as a first party feature?

[rochdev]

Sure thing! This is definitely something that is on our radar but we want to make sure the API will cover all uses cases, which is why it hasn't landed yet.