Itemizing issues for discussion:
In browser JavaScript, we fetch the sources remotely, or they are uploaded as a part of a release. For raven-node, we don't do any of these steps since we can read the source code from disk. So we get lines of source, and it'd be impossible to discover the source the //sourceMappingUrl from the server. So either: a) discover this by the client, and send this over as a new attribute, or b) require uploading minified code as a part of releases, which we currently don't do for anything except browser js today.
In similar vain to discovery, how do we apply these? Would we require a user to upload all minified and unminifed code to us and basically treat it like browser js? If so, would the client opt out of sending any lines of context? We'd need some way to signal to the server, "hey, you should look for sources that were uploaded". These flows are going to need to be a bit different for node, since we can't fetch via HTTP and files are no longer going to be urls, they'll be file system paths.
OR, we could apply the sourcemap in raven-node, but my hunch is that in the scenarios that people are using minified code in node, the sourcemap and original source is unavailable, so this probably won't be as compatible.
/cc @benvinegar
I think raven-node should apply the sourcemap. My docker build has the full source in it, and wouldn't mind giving raven-node the directory of the build (with the compiled .js and .map files), as well as the directory with the source files.
The one weird thing about my build is I have multiple build folders, so accepting an array of folders would be nice.
Can you confirm that this still isn't supported? I just spent half a day trying to figure out why sourcemaps uploaded as release artifacts aren't applied to errors - could it be because I'm using the raven-node client?
This definitely is not yet supported.
Ouch, to say this is frustrating would be an understatement - would you be able to put a big fat warning on the node integration docs until this is implemented?
To be fair, nothing in the documentation suggests that they do work. All of our sourcemap documentation is under the raven-js docs.
It might be obvious to your team, less so to an outsider - node is just javascript after all.
I just assumed it would work as the processing is happening on sentry's servers - you have the map artifacts, the original source files and the sourceMapUrl - it's not clear why the client the error is sent from matters.
@tommor to be fair everything in the JavaScript docs points at the browser and we have entirely separate docs for Node.
There's a lot of reasons why we only do this for browser right now. We wouldn't blindly allow lookup of source maps which don't have URLs so without uploading them the concept wouldn't even work. This is extremely different to how sourcemaps we're originally built and generally used.
It's something we'll likely add in the future but we are very careful about how we support things.
Hi I am using https://github.com/evanw/node-source-map-support . That way I get a partial sourcemap stacktrace inside sentry.
"hey, you should look for sources that were uploaded"
Is this something that needs to be supported server side in addition to some lib changes? I would be happy to extend raven-node, but it seems like there is some server side (Sentry) support for this to work. I compared different payloads between browser (raven-js) and node (raven-node), the payload is equivalent, in both cases the sourcemaps are published to sentry, the main difference in the payload is the sentry_client used and the sentry_version.
Is the spec defined? Meaning, if I publish a release from node and use a path for it, what do I need to push to api/{id}/store/ in order for this to work?
My scenario is a bit different, but somewhat related. We're have a logger-service written in node that logs information from the broswer like a/b testing, analytics, errors. All the error information goes to sentry, sumologs and our analytics platform. So we don't really have sourcemap information client side and raven-node doesn't have an easy way to get them in order to map the frames. We would rather have this done server side.
The following is an example stack trace logged by raven-node, proxying a browser error:
## Body
{
"extra":{
"event":"Error",
"category":"fe_error_development",
"timestamp":"2016-11-30T03:38:57.301Z",
"appName":"client-app",
"userId":3,
"traits":{
"anonymousId":"1cec0457-217a-4e8b-8457-d44874557fee",
"employeeId":1,
"companyId":1,
"userId":3
},
"context":{
"ip":"192.168.21.12",
"userAgent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36",
"screenData":{
"availWidth":1916,
"availHeight":1177,
"width":1920,
"height":1200,
"colorDepth":24,
"pixelDepth":24,
"availLeft":4,
"availTop":23,
"orientation":{
"angle":0,
"type":"landscape-primary",
"onchange":null
}
},
"referer":"http://fonebone.foo/",
"currentRoute":"",
"targetRoute":"",
"href":"http://localhost:4300/"
},
"properties":{
"loggerVersion":"0.2.11",
"appVersion":"0.0.0+3aefae02",
"message":"miguel-test-from-main-navigation",
"stack":"Error: miguel-test-from-main-navigation\\n at Class.raiseError (http://localhost:4300/assets/client-app.js:5242:11)\\n at Class.send (http://localhost:4300/assets/vendor-stable.js:58050:54)\\n at runRegisteredAction (http://localhost:4300/assets/vendor-stable.js:34693:25)\\n at Backburner.run (http://localhost:4300/assets/vendor-stable.js:10528:25)\\n at Object.run [as default] (http://localhost:4300/assets/vendor-stable.js:31349:27)\\n at Object.handler (http://localhost:4300/assets/vendor-stable.js:34685:39)\\n at HTMLButtonElement.<anonymous> (http://localhost:4300/assets/vendor-stable.js:57006:27)\\n at HTMLBodyElement.dispatch (http://localhost:4300/assets/vendor-stable.js:4647:9)\\n at HTMLBodyElement.elemData.handle (http://localhost:4300/assets/vendor-stable.js:4315:28)"
},
"environment":"development",
"receivedAt":"2016-11-30T15:35:43.501Z",
"messageId":"45c45c43-eab7-494b-8d79-df3b40339eea"
},
"tags":{
"browser":"Chrome",
"route":"",
"targetRoute":"",
"href":"http://localhost:4300/",
"userId":3,
"employeeId":1,
"companyId":1,
"app":"client-app",
"appVersion":"0.0.0+3aefae02",
"event":"Error"
},
"message":"Error: miguel-test-from-main-navigation",
"exception":[
{
"type":"Error",
"value":"miguel-test-from-main-navigation",
"stacktrace":{
"frames":[
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":4315,
"colno":28,
"function":"HTMLBodyElement.elemData.handle",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":4647,
"colno":9,
"function":"HTMLBodyElement.dispatch",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":57006,
"colno":27,
"function":"HTMLButtonElement.<anonymous>",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":34685,
"colno":39,
"function":"Object.handler",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":31349,
"colno":27,
"function":"Object.run [as default]",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":10528,
"colno":25,
"function":"Backburner.run",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":34693,
"colno":25,
"function":"runRegisteredAction",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":58050,
"colno":54,
"function":"Class.send",
"in_app":false,
"module":"vendor-stable"
},
{
"filename":"http://localhost:4300/assets/client-app.js",
"lineno":5242,
"colno":11,
"function":"Class.raiseError",
"in_app":false,
"module":"client-app"
}
]
}
}
],
"modules":{
"uuid":"3.0.0",
"utilities":"1.0.4",
"aws-sdk":"2.7.7",
"xmlbuilder":"2.6.2",
"jmespath":"0.15.0",
"xml2js":"0.4.15",
"useragent":"2.1.9",
"lru-cache":"2.2.4",
"raven":"0.12.3",
"json-stringify-safe":"5.0.1",
"cookie":"0.3.1",
"lsmod":"1.0.0",
"stack-trace":"0.0.9",
"signalfx":"2.0.4",
"winston":"1.1.2",
"pkginfo":"0.3.1",
"cycle":"1.0.3",
"colors":"1.0.3",
"request":"2.79.0",
"extend":"3.0.0",
"tough-cookie":"2.3.2",
"hawk":"3.1.3",
"boom":"2.10.1",
"hoek":"2.16.3",
"sntp":"1.0.9",
"cryptiles":"2.0.5",
"aws-sign2":"0.6.0",
"aws4":"1.5.0",
"http-signature":"1.1.1",
"assert-plus":"0.2.0",
"sshpk":"1.10.1",
"asn1":"0.2.3",
"jsprim":"1.3.1",
"extsprintf":"1.0.2",
"verror":"1.3.6",
"json-schema":"0.2.3",
"mime-types":"2.1.13",
"mime-db":"1.25.0",
"stringstream":"0.0.5",
"caseless":"0.11.0",
"forever-agent":"0.6.1",
"form-data":"2.1.2",
"combined-stream":"1.0.5",
"delayed-stream":"1.0.0",
"asynckit":"0.4.0",
"isstream":"0.1.2",
"is-typedarray":"1.0.0",
"qs":"6.3.0",
"har-validator":"2.0.6",
"pinkie-promise":"2.0.1",
"is-my-json-valid":"2.15.0",
"generate-object-property":"1.2.0",
"is-property":"1.0.2",
"generate-function":"2.0.0",
"jsonpointer":"4.0.0",
"xtend":"4.0.1",
"oauth-sign":"0.8.2",
"tunnel-agent":"0.4.3",
"promise":"7.1.1",
"asap":"2.0.5",
"protobufjs":"4.1.3",
"bytebuffer":"4.1.0",
"long":"2.4.0"
},
"server_name":"MAC-10002601",
"environment":"",
"logger":"",
"event_id":"f997fea9b8624696ad1234ef5c3dac02",
"timestamp":"2016-11-30T15:35:54",
"project":"118579",
"platform":"node"
}
The following is an example stack trace logged by raven-js of the same error:
{
"project":"118420",
"logger":"javascript",
"platform":"javascript",
"request":{
"headers":{
"User-Agent":"Mozilla/5.0 (Macintosh; Intel Mac OS X 10_11_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/54.0.2840.98 Safari/537.36",
"Referer":"http://localhost:4300/"
},
"url":"http://localhost:4300/"
},
"exception":{
"values":[
{
"type":"Error",
"value":"miguel-test-from-main-navigation",
"stacktrace":{
"frames":[
{
"filename":"https://cdn.ravenjs.com/3.8.1/raven.min.js",
"lineno":2,
"colno":6160,
"function":"HTMLBodyElement.d",
"in_app":false
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":4315,
"colno":28,
"function":"HTMLBodyElement.elemData.handle",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":4647,
"colno":9,
"function":"HTMLBodyElement.dispatch",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":57006,
"colno":27,
"function":"HTMLButtonElement.<anonymous>",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":34685,
"colno":39,
"function":"Object.handler",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":31349,
"colno":27,
"function":"Object.run [as default]",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":10539,
"colno":25,
"function":"Backburner.run",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":34693,
"colno":25,
"function":"runRegisteredAction",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/vendor-stable.js",
"lineno":58050,
"colno":54,
"function":"Class.send",
"in_app":true
},
{
"filename":"http://localhost:4300/assets/client-app.js",
"lineno":5242,
"colno":11,
"function":"Class.raiseError",
"in_app":true
}
]
}
}
]
},
"culprit":"http://localhost:4300/assets/client-app.js",
"extra":{
"session:duration":144304
},
"breadcrumbs":{
"values":[
{
"timestamp":1480475278.369,
"category":"ui.click",
"message":"body.v1-zenefits > input[type='checkbox']"
},
{
"timestamp":1480475278.379,
"category":"ui.click",
"message":"body.v1-zenefits > input[type='checkbox']"
},
{
"timestamp":1480475281.105,
"type":"http",
"category":"xhr",
"data":{
"method":"POST",
"url":"/pusher_auth",
"status_code":0
}
},
{
"timestamp":1480475281.106,
"message":"Pusher : Couldn't get auth info from your webapp : 0",
"level":"warning",
"category":"console"
},
{
"timestamp":1480475284.403,
"message":"Deprecation warning: moment construction falls back to js Date. This is discouraged and will be removed in upcoming major release. Please refer to https://github.com/moment/moment/issues/1407 for more info.",
"level":"warning",
"category":"console"
},
{
"timestamp":1480475304.014,
"category":"sentry",
"message":"Error: miguel-test-from-main-navigation",
"event_id":"26aa3daf8f844de48a36ba876a8c26db",
"level":"error"
},
{
"timestamp":1480475304.03,
"category":"ui.click",
"message":"button"
},
{
"timestamp":1480475328.608,
"category":"sentry",
"message":"Error: miguel-test-from-main-navigation",
"event_id":"022798320e1940d99f33921fd50fbce1",
"level":"error"
},
{
"timestamp":1480475328.62,
"category":"ui.click",
"message":"button"
},
{
"timestamp":1480475368.587,
"category":"sentry",
"message":"Error: miguel-test-from-main-navigation",
"event_id":"8e6e125b91ea4830be354b1fb1932522",
"level":"error"
},
{
"timestamp":1480475368.596,
"category":"ui.click",
"message":"button"
},
{
"timestamp":1480475388.387,
"category":"sentry",
"message":"Error: miguel-test-from-main-navigation",
"event_id":"1400ffc4785b40229481a429e633a106",
"level":"error"
},
{
"timestamp":1480475388.395,
"category":"ui.click",
"message":"button"
}
]
},
"event_id":"c9706ccdb3004c498ab983766a13d946"
}
The only "significant" differences around the stack traces is that raven-node adds a modulename but raven-js doesn't have it. Other likely relevant differences are the logger (javascript, "") and platform ("javascript", "node").
As an aside, IMO, doing this server side is also a better approach than getting raven-node doing this. While sourcemaps may be available in some cases on the server with the minified code, processing this sourcemaps seems like a different concern, this could easily add IO and memory concerns that the server doesn't have, for large applications I don't think it's a good idea to do the mapping and that's something that would be better to do on a server that is already optimized for this. Also, sentry already does this for browser errors, so it may be better not to have to duplicate this logic there.
@mattrobenolt it would be awesome if the node docs explicitly stated that source maps are not supported.
I have a node project running transpiled ES6 code using babel.
In order to get clarity for the traceback, is there a way to support sourcemaps for raven-node?
@llonchj not at the moment, but I have to ask - if you're only using ES2015 features, do you need babel? Node has had essentially full ES2015 support since 6.5.0 (PTC and modules excluded).
@LewisJEllis There are other reasons to use Babel (or another transpiler) than ES2015 features, such as async / await. Another big use case is server side rendering React / JSX. Or if your codebase is written in Typescript or Flow. All of these need a transpiler that can (and should) output source-maps.
I think this should be a high priority issue to resolve.
@chrisblossom +1. There is a solid requirement to include sourcemap to raven-node.
We're just butting up against this at the moment. I think the implicit understanding is that source maps are supported (because of support in the raven-js). Having it made explicit in these docs that it is not working would have saved my team some not inconsiderable amount of resource trying to get them to work.
We have the same setup as @chrisblossom and definitely have a requirement for sourcemaps on the server side. For implementation I think it makes sense to have them pushed up in the release object (as opposed to being grabbed from some externally accessible endpoint).
Is there anything we can do, as users, to aid with the development of this?
@bfritscher you mention using https://github.com/evanw/node-source-map-support to get partial support working. We already have source maps generated, how were you able to get these in to Sentry itself?
Our scenario is a bit different, but it may work for some of you. We have a node server that aggregates JS errors, as far as sentry was concerned, it was a node error, not a JS error, so what we did is we changed the "platform" on the payload to send it as a "javascript" error not a "node" error.

We had to upload the sourcemaps and this just worked for us. Now it may be different for you, but give it a go, it may just work.
@dcramer kindly reconsider sourcemap support for nodejs.
@llonchj to be clear, we've never said we won't support it. We just don't today and don't have immediate plans. For the meantime, I'd recommend just leveraging JavaScript tooling that can fix it on your side before we get it. For instance, that's how we support CoffeeScript since they've done this automatically since forever ago.
@mattrobenolt thanks for your comment.
As I mentioned earlier, i have the frontend (raven-js) sourcemaps working in sentry.
I transpile JS using babel and generate the sourcemaps, node app uses raven-node and my understanding is that sentry does not process sourcemaps if platform is "node".
Please help me understand your suggested tooling/workflow for raven-node.
The fundamental difference here is that from raven-node, the running process would have access to the source files and can read them from disk. Whereas with raven-js, that's not possible. The browser doesn't have access to the raw source code, so we have to do special processing.
Now, I don't personally work in node world to have a proper/tested suggestion here, but https://github.com/evanw/node-source-map-support seems like it'd do the trick.
My point is that simply node has the ability to apply this information on the client before sending it to us. We will definitely support sourcemaps on the server for node, but we're not there yet.
And yes, we explicitly would block if the language is "node" since we have a whitelist of languages that'll go through this extra processing.
@mattrobenolt i am happy to upload the sourcemap to the release as with the frontend(node-js).
Just by changing src/sentry/utils/javascript.py:35 to if event.platform not in ['javascript', 'node']: i think will work.
What are your thoughts?
Very unlikely that'll be enough, which is why we haven't done it. At least historically. There are lots of assumptions down the processing pipeline that assume URLs, not file paths. So I wouldn't want to just trivial flip that switch unless we can invest the time to actually testing this.
Thanks for your clarification
2017-03-29 8:44 GMT+11:00 Matt Robenolt notifications@github.com:
Very unlikely that'll be enough, which is why we haven't done it. At least
historically. There are lots of assumptions down the processing pipeline
that assume URLs, not file paths. So I wouldn't want to just trivial flip
that switch unless we can invest the time to actually testing this.—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/getsentry/sentry/issues/2632#issuecomment-289914269,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAVtPpLk7bZEB0B3h3kr08b63bZ5SdK4ks5rqX87gaJpZM4HRwQl
.
@llonchj that trick worked for us because we're logging browser errors using the raven-node. With node errors the location will be different, but getting it to work it may be as simple as rewriting the location with something like location.replace('/path/to/app', ''), assuming you have a path like /path/to/node_modules/server.js and you uploaded the sourcemaps as ~/server.js.map...
Regarding:
Now, I don't personally work in node world to have a proper/tested suggestion here, but https://github.com/evanw/node-source-map-support seems like it'd do the trick.
Echoing what @MiguelMadero already said:
While sourcemaps may be available in some cases on the server with the minified code, processing this sourcemaps seems like a different concern, this could easily add IO and memory concerns that the server doesn't have, for large applications I don't think it's a good idea to do the mapping and that's something that would be better to do on a server that is already optimized for this.
I’d really like to _not_ spend memory on source-maps and CPU cycles on symbolicating traces in our app process, especially for those cases where cascading failures could lead to many errors and eventually starving the server.
In similar vain to discovery, how do we apply these? Would we require a user to upload all minified and unminifed code to us and basically treat it like browser js? If so, would the client opt out of sending any lines of context? We'd need some way to signal to the server, "hey, you should look for sources that were uploaded". These flows are going to need to be a bit different for node, since we can't fetch via HTTP and files are no longer going to be urls, they'll be file system paths.
@mattrobenolt Correct me if I’m wrong, but it looks like the main issue is finding the sources? If so, have you considered the sourcesContent attribute of the source-maps spec to make source-maps self-contained?
Bump... want source maps for Node.js :(
:+1:
@alloy i believe that sourcesContent would work here.
+1 to getting this working 👍
+1
+1 please implement this
+1; webpacking JS for node deployments is getting increasingly popular.
@iEchoic You are referring to server side apps ? If that is the case, can you share any resource that would give a detailed explanation ?
Thanks!
@bilby91 sure: http://jlongster.com/Backend-Apps-with-Webpack--Part-I
I'm doing it to share the same build pipeline with my frontend, and to use some useful Webpack plugins (like circular-dependency-plugin).
+1
+1
+1
+100
@kamilogorek can we close this issue now?
If this is resolved by https://github.com/getsentry/sentry/pull/6626 can we have some docs before this is closed, please.
@kamilogorek @mitsuhiko Ooh, nice! Whenever this is available to use, would you mind notifying us here?
@netproteus docs are already in place :) https://docs.sentry.io/clients/node/sourcemaps/#raven-node-sourcemaps
cc @tybro0103
@mitsuhiko yes, go ahead
@kamilogorek Is there similar solution from @sentry/node?
@sheerun it works exactly in the same way in the new SDK (just use beforeSend instead of dataCallback).
https://docs.sentry.io/platforms/javascript/sourcemaps/
https://docs.sentry.io/clients/node/sourcemaps/
It turns out not exactly because different object is provided to beforeSend (not exception). Here's the code I use:
beforeSend (event) {
if (
!event.exception ||
!event.exception.values ||
!event.exception.values[0]
) {
return event
}
const value = event.exception.values[0]
if (value.stacktrace && value.stacktrace.frames) {
const root = process.cwd()
value.stacktrace.frames.forEach(function (frame) {
if (frame.filename.startsWith('/')) {
frame.filename = 'app:///' + relative(root, frame.filename)
}
})
}
return event
}
@kamilogorek Do you think SDK could do it automatically (maybe as a flag), instead documenting this hack again?
@sheerun WIP: https://github.com/getsentry/sentry-javascript/pull/1611
Most helpful comment
@LewisJEllis There are other reasons to use Babel (or another transpiler) than ES2015 features, such as async / await. Another big use case is server side rendering React / JSX. Or if your codebase is written in Typescript or Flow. All of these need a transpiler that can (and should) output source-maps.
I think this should be a high priority issue to resolve.