Applicationinsights-js: Add support for mapping minified Javascript to source

[MaxShehovtsov]

From https://visualstudio.uservoice.com/forums/357324/suggestions/12292965

Currently Application Insights does not resolve minified Javascript to it's matching original source. This makes solving production issues impossible - and who doesn't minify? Please add support for map files (or some other method) for making this possible.

[mttmccb]

Having this line in the AppInsightsModule.js

//# sourceMappingURL=AppInsightsModule.js.map

is also causing unnecessary errors as the map file isn't in the directory

[shkrishms]

Hi,
This feature is very fundamental to our product too. We would like to be notified of user errors and without source mapping, its very unusable. I would like to know what are the chances this will be supported and when it will be released?

[StefanKert]

Are there any news on this? This is really necessary because otherwise the search for exceptions would be really painful.

[shkrishms]

any news on this?

[KamilSzostak]

Source map is finally supported. Enjoy and please let me know if you're experiencing any issues.

[SergeyKanzhelev]

@KamilSzostak is there UI feature to upload the map file? I cannot find it

[defr0zen]

@KamilSzostak Could you please shed a little light on what exactly is supported now?
We are using AppInsights in the browser (https://az416426.vo.msecnd.net/scripts/a/ai.0.js) and exceptions are still reported for minified source. The source maps are published together with our app (checked in google chrome).
How exactly does AppInsights SDK read the source maps? Should it happen in the browser prior to sending exceptions to the server?

[denious]

Anyone figure this out? Happy to hear AppInsights supports source maps now but where are the docs?

[KamilSzostak]

I'll take a closer look at this issue in next milestone.

[jimbuck]

Being able to see original source filenames and line numbers in the exception stack trace is extremely important. If this is supported then can just have one example of how to enable it @KamilSzostak?

[sylvanaar]

The single most important feature to any javascript developer. What is the due date for 1.0.16?

[robrohan]

Is there anything I can do to help this along? This makes appInsights unusable for us (from a front end perspective), but we'd like to keep everything in one place.

[brauliodiez]

Any update on this issue? About @KamilSzostak announcement:

_Source map is finally supported. Enjoy and please let me know if you're experiencing any issues._

Is there any blog post, or official documentation that we can take as reference? Or was something announced but not working / released? If that's the case is in scope for 1.0.16? When could be the approx release date for this minor release?

[denious]

Got some feedback today from MSFT claiming the work is complete but still no links to anything official. Can anyone confirm, please?

https://feedback.azure.com/forums/357324-application-insights/suggestions/9751743-support-source-maps-in-javascript-exceptions

[josundt]

I am not able to see stack traces of JavaScript errors using source maps.
Been waiting for this for some time now, we are using TrackJS to accomplish this until Application Insights support it.

I also noticed that the work is allegedly complete, but I can't find anything indicating this to be true.
Ref: https://feedback.azure.com/forums/357324-application-insights/suggestions/9751743-support-source-maps-in-javascript-exceptions

How can I link source maps to stack traces in Application Insights exception viewer?

[sahilanguralla]

I think this has been moved to milestone 1.1.0. It's still not available.

[shayanc]

Any updates?

[jpiyali]

@shayanc - originally when Kamil added support, he generated sourcemap for our script. We dont have support for showing stack for instrumented sites. We'll plan for this in next version of SDK which is currently in beta.

[tommck]

Can't believe it's been nearly 3 years and this still isn't there

[Nickman87]

This is one of the major features needed to use insights in any modern frontend... How can this take so long? :O

[Nickman87]

@markwolff Did this feature seriously just get delayed even more? How can this library be of any use in modern applications???

[aldarund]

So is it going to be moved to next milestone each time previous one is released ? :) Another 3 years till its ready ? :)

[dagoltz]

I really don't think you appreciate how important this feature is...

[mderazon]

Talked with Azure support who talked with the engineers

Source map feature is currently as internal preview and it will be available to everyone on following weeks, we will update Github open issues related to it once is ready

MS could have been more communicative and active in this thread. Like previously mentioned, there is no practical use for this sdk without source maps.

Is there any way to opt in to the preview version meanwhile until it's released in the "following weeks" ?

[markwolff]

A first iteration of this feature is now in prod. See PR #948 for brief summary. That PR hasn't been completed yet as I'm waiting for a few more updates to rollout which resolve some QOL issues with the feature (basically by end of this week). I plan on closing this issue and opening a new one at that point with a general roadmap of how this feature will be built out in the future, and look forward to feedback to guide how we build it out (both in UX and in how we can consume your source maps). You can use the Feedback button in the Portal or just post a comment in the issue here and I'll see it.

The gif shows pretty well how this initial iteration works. You can drag your source maps to the callstack and it will unminify it. For now, the source map file must match the minified source file name (foo.min.js --> foo.min.js.map), but this will be addressed in a rollout coming later this week. Afterwards, the source map can be named whatever and a source map will only work on its corresponding minified file. If a source map does not work for you as you think it should, please post details in this repo or use the Feedback button in the Portal.

This is only a first iteration, and eventually it will progress towards allowing you to host your source maps somewhere and automatically unminify or view the original source files, without needing to manually drag and drop. I will provide details on a timeline/roadmap in the new issue.

[mderazon]

Great news. I am not sure what is the recommended strategy of using source maps in production.
Obviously you don't want to include them and tell your browser about them, so what did you have in mind for such "auto discovery" ? Webpack usually generates source maps with the same hash in the filename as the original file. For example, here's an output of my dist folder:

1.df0836e3bbcdbabf6975.js
1.df0836e3bbcdbabf6975.js.map
10.f583723cb584708c7a1a.js
10.f583723cb584708c7a1a.js.map
11.4867cc7417d0d39a8ba7.js
11.4867cc7417d0d39a8ba7.js.map
...
common.d9f822e476f1b33f2a3b.js
common.d9f822e476f1b33f2a3b.js.map
main.544c76488fd4f86aaf67.js
main.544c76488fd4f86aaf67.js.map
polyfills.38e7c9da5cd7eb18db35.js
polyfills.38e7c9da5cd7eb18db35.js.map
runtime.f255f948c9bbb79c25cd.js
runtime.f255f948c9bbb79c25cd.js.map
scripts.c126dd109a9329b9e689.js
scripts.c126dd109a9329b9e689.js.map
styles.08c9dd2727d4aaba1fd1.css
styles.08c9dd2727d4aaba1fd1.css.map

So maybe we can host them on the same server as the original file and Azure could look for them there ? For some people it will not be "secure" enough as people can guess the sourcemap files exist there

Also, found one Angular article about this:
https://blog.angularindepth.com/debug-angular-apps-in-production-without-revealing-source-maps-ab4a235edd85

[josundt]

I know that what I'm about to write is way outside the scope for this issue, but I still need to share my thoughts because I think it might be a good idea...

We host our web apps in Azure, and we use Azure DevOps for build/release management.
While working with CI builds lately, I started thinking about how source+maps in many ways fill the same purpose as .NET .pdb symbol files.

For .NET, Azure DevOps allow you to publish symbols (.pdb) from CI builds and DevOps works as .NET symbol server. Would it not be possible to have a similar approach to source+maps?

If the Azure DevOps CI build allowed you to publish source+maps and supported working as a "source map symbol server", it could theoretically feed the Application Insights portal and thereby remove the need to drag files into the browser (if you configure the connection to the server).

The limitation with the "drag-maps-into-the-browser" approach is that when f.ex. investigating a production issue, you normally don't have the files to drag on the computer you are using.

Is this crazy thinking or could I be onto something?

[markwolff]

@josundt The sounds like a very smooth experience and pretty much the route I was planning on taking. The eventual goal is for this to integrate with your CI/CD process. One way would be for a CD step to upload them to blob storage (either via Azure devops or curling it via cli), and providing a way to connect your App Insights instance to that storage.

[aldarund]

Yes, thats basically how other handles this. E.g. sentry. They allow to upload source map and they even have a webpack plugin for that

[Nickman87]

@markwolff any news on the sourcemap roadmap? When can we expect automated uploading and linking of sourcemaps?

Awesome news that this has finally been implemented! Will set it up further in the coming days to see how good it works and provide feedback where needed 👍

[markwolff]

@Nickman87 I will probably start working on it around Sept/Oct time frame. It may initially just be automatically work on browser-side only -- the underlying telemetry won't be auto unminified and it will only work on this panel. Ideally I would like it to work everywhere, e.g when running your queries, but it is a stretch goal at this point.

[aviatrix]

@markwolff or whoever ends up implementing this: please consider that people deploy frequently.
If i could upload source maps for specific version and mark those files for that version only as i mark AI releases would be great. Also if i browsing a certain exception it knows for what version it is based on the release marks (time) and should pickup the proper source maps.

[dudewad]

Is there a roadmap or ETA for integration of the auto-consumption of source map files? This feature would be invaluable. Presently we are building and deploying from devops which means manually going and finding source map files. For our team this can be difficult because not eveyone is familiar with (nor should they be) the front end deployment architecture. The less hands in there the better.

I also see this as a security concern. We are using Angular and "hiding" source maps (i.e. not adding the source map references to the minified files or adding source map headers). This is better because it doesn't just allow automatic perusal of unminified source code on the client side.

[markwolff]

I expect it to start rolling out in a few weeks (by end of Feb). Just curious, once unminification can be done automatically, do you still envision you or your team still using the drag&drop behavior?

[dagoltz]

We will probably only use the automatic option, assuming we can somehow secure access to the sourcemaps (using blob storage maybe?)

[markwolff]

Yes the source maps would be held by a blob storage account. The plan is for this feature to simply "link" your AI resource to a blob container.

[aviatrix]

Would this work for both node.js and browser source maps?
For example, Sentry is sending the node.js source maps as well as fetching browser source maps.

[markwolff]

Yes it will work for telemetry sent by this SDK and the node.js SDK. Can you clarify what you mean by "sending" and "fetching" source maps? In either case, you would need to upload source maps to a linked storage account.

[tommck]

It scares me that people are worried about publishing map files for JS code. Security through obscurity is not a valid approach to making your script code secure.

It's script code. It's delivered to the user. Your APIs should be coded securely and there should be no way for people to penetrate your systems even if they have the entire source code to your Angular app

[dudewad]

@tommck I presume you're referring to my comment.

I disagree with your statement in that obfuscation is _another barrier_ to penetration. The company I work at has a value proposition that is security-related. Every step we can take to make it harder to penetrate our security is a step we will take. Minification and obfuscation is not an impenetrable security mechanism and I don't think anyone is arguing that. But we also would prefer not to willy nilly have our source code available on the internet in such an easy fashion.

Github offers private repos precisely to keep source code hidden. There's nothing wrong with wanting your code to be obfuscated and stay that way.

I'd like to add that "there should be no way for people to penetrate your system even if they have the entire source code" - that's a wonderful premise but it doesn't really apply to agile teams that push out a lot of software that also don't have security experts to catch everything. Most, if not all apps, are penetrable.

[tommck]

Github offers private repos because not all apps are downloaded to people's web browsers.

If your app is securing something worth taking, obfuscating the code won't stop people.
You should also not tell clients that hiding your unobfuscated code is a security benefit. It's a dubious claim at best

[dudewad]

I feel like you're hyper-focusing on one small aspect of what I said that 1) I don't actually think is that important and 2) you're making assumptions about what we tell clients, none of which is accurate.

Anyways. I don't really care to hyjack this thread over an unimportant discussion - I would still like to know when this feature (the Azure automated pickup of the source maps) will become available because I think a bunch of us want it.

[aviatrix]

Yes it will work for telemetry sent by this SDK and the node.js SDK. Can you clarify what you mean by "sending" and "fetching" source maps? In either case, you would need to upload source maps to a linked storage account.

The Sentry SDK automatically uploads the sourcemaps to wherever it's needed, and we don't have to configure anything for it to do so. In AppIns' case, would we have to manually upload to blob storage our source maps after each release? and if so, would the association between different releases <-> sourcemaps be manual or automatic?

Sorry, if i'm asking too many questions, i'm just trying to clear in my head how will this work compared to what we already use.

[xigolle]

@markwolff I think our team will still use the drag & drop feature especially in the transfer phase from manual to automatic.

Just a general question about the source maps and how they work now.
Could I just publish my production code (without sourcemap) to the production environment and when I need a source map do a build of that exact version of the code?
Or are there things that are specific to a build that makes the mapping work ?

Thanks in advance if somebody can help me out!

[markwolff]

@markwolff I think our team will still use the drag & drop feature especially in the transfer phase from manual to automatic.

Just a general question about the source maps and how they work now.
Could I just publish my production code (without sourcemap) to the production environment and when I need a source map do a build of that exact version of the code?
Or are there things that are specific to a build that makes the mapping work ?

Thanks in advance if somebody can help me out!

In order to make it backwards compatible with all versions of the Javascript SDK, we will search for sourcemaps based on the filename in the call stack. It is assumed the file is named in a standard way, e.g a stack frame at foo.js will search for foo.js.map

To answer your question, each time you build your app, it will generate a new source map. This new source map won't work on previous builds, even if the code has not changed at all. I think you could force this source map to work on other builds (by keeping its file naming constant), but you would have to ignore all of the filename info. The App Insights feature isn't built for this currently!

[markwolff]

Just adding an announcement to this thread -- automatic source map support should be rolled out completely and available for everyone to use. As previously discussed in this thread, you can link your AI resource with a storage account which contains your application's source maps.

Each time you view an exception you can click "unminify" and it will try and fetch the corresponding source maps from your blob storage (regardless of folder structure!). Please try it out and let me know if this doesn't cover your particular application's scenario. Either open a new issue or reply back in this one with more details if isn't working for you.

I am closing this issue since I believe this feature is now finally available.

Docs: https://go.microsoft.com/fwlink/?linkid=2118047
Azure Announcement: https://azure.microsoft.com/en-us/updates/azure-monitor-source-map-integration-with-azure-blob-storage-enables-faster-debugging/

[RobARichardson]

@markwolff I've followed every step in the Docs. I've got my Deployment Pipeline in Azure Dev Ops configured to copy my source map files to Azure Storage
(using Azure File Copy) and my Application Insights Account is configured with the correct Storage Account and Container. All my file names match, but I keep getting this error message when I check the 'Unminify' checkbox. Any ideas? Is this a known issue?

[markwolff]

Hey @RobARichardson can you post what the errors look like in your browser's F12 menu (and also the callstack itself if that is okay)?

[RobARichardson]

@markwolff Wasn't sure if you were asking for the minified or unminified version so I did both. Let me know if I can provide anything else. Very eager to get this working.
Unminified:

Minified:

[markwolff]

Oops sorry I meant the call stack and F12 when you try to unminify from the Portal

[RobARichardson]

Ahh, that makes more sense. So, when I try to unminify from the Azure Portal, I'm seeing no errors in the Dev Tools Console at all.

[chebum]

@markwolff I see the same error as @RobARichardson
DevTools aren't empty though. I have the following warnings:

CO6RvAHl1NAR.js:9 [AppInsightsExtension]  11:37:48 AM MsPortalFx/MsPortalFxEnd MsPortalFxEnd: Module "MsPortal.Azure" wasn't defined explicitly or anonymously in downloaded file "https://afd.hosting.portal.azure.net/appinsights/Content/1.0.20200410.1714/Scripts/MsPortal.Azure.js".  Will proceed assuming it loaded correctly.

DevTools failed to parse SourceMap: chrome-extension://hdokiejnpimakedhajhdlcegeplioahd/sourcemaps/onloadwff.js.map
DevTools failed to parse SourceMap: https://afd.hosting.portal.azure.net/appinsights/Content/1.0.20200410.1714/Scripts/Oss/applicationinsights-web-basic.min.js.map
DevTools failed to parse SourceMap: https://afd.hosting.portal.azure.net/appinsights/Content/1.0.20200410.1714/Scripts/Oss/applicationinsights-common.min.js.map

[mirkolugano]

@markwolff thank you, that's a wonderful feature. I'd like to point out a couple of scenarios though, where I think it
can be improved.

1. I am embedding scripts into my page like this <script type="text/javascript" src="~/Scripts/bundled/register.min.js" asp-append-version="true"></script> which results in the final HTML in having a "version" appended to the file url like this: https://mysite.com/Scripts/bundled/register.min.js?v=HgCD8hP3kcnJPwImmu8nky2zGX3hPNKSoyT6rTyEfuc. This is to avoid caching issues on the client. It seems to me, that the "unminify" feature does not recognize the query string (which should be stripped out), thus looking for a .map file named exactly like the full path (including query string) plus .map. (see image below).

1. Some scripts I use are from external libraries, which I do not need to unminify and debug. A source map is therefore not available. It would be best, if the corresponding line would be ignored, and directly pass on to the following lines, for which sourcemaps are available. Instead, it gives an error and no unminification takes place. (see image below)

[jawn]

Since, the issue is closed, maybe log a new issue for that with a reference to this one?

[mirkolugano]

@jawn thank you, I opened 2 separate issues (the second could be considered maybe a feature request, but I think it is also a bug). Feel free to remove my previous comment.

[bjuraga]

The way this feature is implemented it is not scaling beyond a single app per application insights instance, so it is not very useful.
If you have more than one client app, it is likely the apps will have the same js file names for some core parts, i.e. app.js.
Because we are forced to flatten the file structure there would be filename collision and the latest deployed app will overwrite the matching files, making it harder and more confusing to debug.
Also in our case we have the same filenames in folders on different level in the folder structure.

I dont think this implementation is working at all. Can you remove the need to flatten the folder structure and potentially support more than one app?

[markwolff]

@bjuraga There is no need to flatten folder structure. It will search for map files matching your folder structure, and so if you are able to tweak your folder names it should pick out the correct map file.

[bjuraga]

Thanks @markwolff i will give the folder structure a bit more testing.
Is there also a way to support multiple apps?
At least without the need for per app application insights and blob storage?

[markwolff]

@bjuraga Multiple apps should be supported through folder name differences. If your js files are consumed something like https://example.com/app1/static/js/main.js, https://example.com/app2/static/js/main.js, then your scenario should work out of the box.

Another potential solution is to name your files with some unique hash each time you build, main.abc123.js -- Webpack does this by default.

[bjuraga]

sure if the folder structure pans out and i host all my apps in a single app service as you stated with https://example.com/ then its just an extension of the folder structure example.
However i am hosting multiple apps but one per app service, meaning i have:
https://app1.com/js/main.js
https://app2.com/js/main.js
i believe this is a typical use case
in my case the different apps are actually the same app but different versions of it so i can support legacy versions.
But even without this complexity, i believe the example i shared above is not supported?

[markwolff]

Yes the example you posted above wouldn't work

[bjuraga]

Thank you for clarifying @markwolff

Is there a plan to support this? Or should i file a suggestion/bug/defect?

[markwolff]

There are currently no plans to support this. You can submit a feature request on our user voice page

[bjuraga]

suggestion filed
https://feedback.azure.com/forums/357324-azure-monitor-application-insights/suggestions/40845535-unminify-add-support-for-multiple-apps