Sentry: Clarify Sourcemap Matching

[mitsuhiko]

This is something that comes up in different forms when users start messing around with filenames because we're not supporting all URLs correctly.

We should document somewhere what we support and what we do not support. In particular relative filenames were never supported and definitely broke a bit more at one point (#4718) but elsewhere snippets fly around that propagate the use.

I think we should support these formats:

• /absolute-path.js (matches on absolute paths only. This would be for file usage, see below)
• ~/wildcard-path.js (matches on any URL)
• http://full-url.invalid/path.js (matches on exact URL)

The absolute path usage would be for clients that do not have a good URL to work with. In particular we encounter this with react-native and electron which uses file:// or some made up paths that are not stable and hard to match on.

I propose we document this somewhere and maybe validate the artifact names on upload and error in the API. The tools we provide (eg: sentry-cli) should also follow those rules then.

[mattrobenolt]

Would it make sense to impose a protocol for files with file:// as well to not be ambiguous? That way we always handle "urls" which would eliminate the /absolute-path.js path which would become file:///absolute-path.js or ~/absolute-path.js.

[mitsuhiko]

The issue with at least electron and react-native is that those paths are arbitrary. Eg: file:///var/lib/containers/some-uuid-foo/static/index.js etc. For electron I think it's the actual FS path or some other stuff if you use an archive.

[mitsuhiko]

@mattrobenolt oh also, the reason @HazAT and I decided against using file:/// but / is that in that case the UI does not actually show a URL link to click on for the source filename if relative references are used.

[HazAT]

I think we should go with

/absolute-path.js
~/wildcard-path.js
http://full-url.invalid/path.js

But I am not sure what to do if a user uploads a file without any prefix like absolute-path.js should the cli enforce it?

[mitsuhiko]

The CLI should effectively force it to be absolute at least I think.

[mitsuhiko]

(Currently it forces it to be ~/ by default, but if the prefix is set to empty it should become /)

[steverandy]

I have an electron app and not very sure how to integrate sentry. It will be helpful if there is a documentation on how to make sourcemap support works.

I have seen in some discussions on github, where dataCallback is used to modify the path to match the sourcemap, but still not sure if this is the good/recommended way of doing it.

Is there any plan for official support for an electron app?

[mitsuhiko]

@steverandy we're looking into it but I don't have anything to share for now. However we will document somewhere what our matching behavior is to make the temporary workarounds more reliable.

[oyshan]

If I understand it correctly, ~ only removes host and protocol?
E.g. for the URL http://myapp.com/dataset (where the dataset can vary) with sourceMappingUrl=mapfile.map, an artifact named ~/mapfile.map will not be found, as Sentry will try to find an artifact named either http://myapp.com/dataset/mapfile.map or ~/dataset/mapfile.map.

In our case, as "myapp" can run on multiple datasets (i.e. with the exact same source code), we cannot upload artifacts for each of these datasets. Therefore it would be nice to have a wildcard artifact matching.
Is it possible to introduce a wildcard/asterisk (*) matching pattern that matches everything before *? E.g. for the example above, an artifact named */mapfile.map will be found. Is something like this possible, or could it be added for an upcoming version?

[mitsuhiko]

@oyshan if you are at /dataset it will find it at ~/mapfile.map. If however you are at /dataset/ (note the trailing slash) then it will look at ~/dataset/mapfile.map.

[mitsuhiko]

And no, we do not support wildcards and we won't make attempts to do so. However the SDK can trivially rewrite filenames to strip all prefixes if you want. That's what react-native for instance does.

[oyshan]

Thanks for the fast reply, @mitsuhiko ! :)

We are at /dataset/. The challenge we have is that there can be a multitude of datasets which we don't have any knowledge of. We only upload source map files that are common for all the dataset implementations, i.e. we can only upload artifacts named ~/mapfile.map, not ~/dataset/mapfile.map, as we don't know dataset. Is there a workaround for this that you know of?

[oyshan]

And btw, is it possible to rename artifacts that are uploaded to Sentry? Or do we have to delete them and reupload with new file names?

[mitsuhiko]

@oyshan you can rewrite the urls before the event is sent. Then you can change the path to anything you desire. Example from react-native: https://github.com/getsentry/react-native-sentry/blob/9001c5c0dc9314a264650704b90141a50fe5e999/lib/raven-plugin.js#L243-L264

[oyshan]

Thanks, @mitsuhiko !
I've looked into the React native code, but I'm still a bit unsure which part of the data that should be rewritten. Which properties of the data object should be rewritten (in raven.dataCallback) to make Sentry look for artifacts at ~/mapfile.com instead of ~/dataset/mapfile.map?

data.culprit?
data.request.url?
the filename in the exception stacktrace frames?

Sorry for being stupid:)

[kamilogorek]

@oyshan culprit and filename of all the frames. You can basically modify the same code that Mitsuhiko mentioned above.

function yourNormalizeFunction (name) {
  // do some magic here
  return updatedName;
}

Raven.config('__DSN__', {
  dataCallback (data) {
    if (data.culprit) {
      data.culprit = yourNormalizeFunction(data.culprit);
    }

    var stacktrace = data.stacktrace || (data.exception && data.exception.values[0].stacktrace);
    if (stacktrace) {
      stacktrace.frames.forEach(function(frame) {
        frame.filename = yourNormalizeFunction(frame.filename);
      });
    }

    return data;
  }
}).install()

[oyshan]

Cheers! @kamilogorek

[BYK]

Closing this issue due to staleness. Feel free to comment here if you think we should still work on this.