Sdk: Publish Single-file: Smart settings for IncludeSymbolsInSingleFile

Created on 27 Sep 2019 · 11Comments · Source: dotnet/sdk

When publishing an app as a single file, the default setting is to not include PDB files in the output bundle by default (and instead leave them as separate files).

However, this setting is unsuitable for some apps, where the deps.json file includes a dependency on the PDB file. This causes runtime-crash of the app when the apphost validates the contents against the deps.json manifest. For example:
https://github.com/PowerShell/PowerShell/issues/10266
https://github.com/Viir/bots/tree/adapt-for-single-file-exe-publish

The inclusion of PDB files as a dependency is almost always an error, but an app (and its dependencies/nuget packages) are not required to associate the same semantics with file-extensions.

So, if the deps.json writer finds a PDB file in the manifest, the SDK should default the IncludeSymbolsInSingleFile setting to true. The SDK should also issue a warning about this change, so that PDBs are not silently included -- unintentionally increasing the size of published apps.

Source

swaroop-sridhar

👍7

Most helpful comment

Great points. At the end of the day, we have to uniformly decide if something is a symbol file or not. If we decide it is not a symbol file, but an arbitrary native asset, then it should be in deps.json AND deployed irrespective of IncludeSymbolsInSingleFile. And if we decide it is a symbol file, then it should not be in deps.json AND included in single file if and only if IncludeSymbolsInSingleFile is true.

NuGet and the assets files should be the source of truth for what files in package are what kind of thing. If we decide to use extension heuristics for this, then they should go in nuget. SDK should respect what NuGet tells us.

I will combine this issue and the issue of not copying managed PDBs into one feature request for nuget.

nguerrera on 27 Sep 2019

👍2

All 11 comments

CC: @Viir

swaroop-sridhar on 27 Sep 2019

CC: @dasMulli

swaroop-sridhar on 27 Sep 2019

@lpereira In .net 5, will the apphost validate the dependencies listed in the deps.json file at startup?
This PDB inclusion issue will be moot for most apps, but for the apphost checking.

CC: @vitek-karas

swaroop-sridhar on 27 Sep 2019

I'm still not sure if the pdb in the deps.json isn't a bug that should be fixed in the first place. Though this likely needs Nuget involvement.. @nguerrera @dsplaisted thoughts?

dasMulli on 27 Sep 2019

I think it is a bug. We have related discussion about being able to query for pds so that we can copy them. It's ironic that we copy them for native, but not managed. Feels like there's an exclusion of pdbs only in managed case.

nguerrera on 27 Sep 2019

I believe that technically it's a bug to put a pdb in the native files of a package. NuGet interprets anything in the folder as a native file, which is how it flows through to the deps.json.

It seems like it is natural to put pdbs in the native folder though, so if we want to support that we could update the NuGet convention, or apply a PDB convention on top of it in the SDK.

dsplaisted on 27 Sep 2019

I think it is perfectly natural for authors to include artefacts that help for debugging or error tracking.

For managed code, I usually suggest deploying PDBs with everything to get "good" stack traces. Tools like Sentry even help tracking down the GIT commit that caused error to occur based on stack trace symbolificatiton.
So authors may want to include pdbs / dsym / etc.

But "native" really means native assets so that may also mean runtime-specific scripts and assets..
I don't think there should be much introspection of the file type going on here. PDB is one special case for windows but I don't think it is good to make assumptions on files in native folders.

dasMulli on 27 Sep 2019

To double down: Does IncludeSymbolsInSingleFile also apply to dsym files?
I think there should be parity for each runtime if that is the case, so leaving native pdb / dsym / * files alone and not applying filters to that or encoding the knowledge about all kinds of symbol files.

dasMulli on 27 Sep 2019

👍1

I will combine this issue and the issue of not copying managed PDBs into one feature request for nuget.

nguerrera on 27 Sep 2019

👍2

@lpereira In .net 5, will the apphost validate the dependencies listed in the deps.json file at startup?

I've been thinking about this, and I might open the deps in apphost so the validation is performed right at the beginning, but keep them open -- the runtime can then just use the open file descriptors instead. In a way, the check now becomes more robust while still improving the performance, by cutting a lot of unneeded stat(2) calls. (Need to sleep on this for a little while longer, though.)