Sdk: PDBs fail to generate when referencing netstandard project from UWP app

[Thealexbarney]

Steps to reproduce

1. Create a solution with a UWP app and netstandard library using VS2017 RTW.
2. Reference the library from the UWP app.
3. Build an appx package in debug mode, or build an appxupload package, including symbol packages.

Results

The pdb files aren't generated for the debug build or the appxupload. Release builds generate pdb files without apparent issue.
Pdb files aren't included in the appxupload package, even though the build succeeds.

This does not happen when referencing a PCL.

Update: MSBuild 15.1 gives the error Failed to convert FastLink symbols to full PDBs for '....pdb' due to failure to find the tool '\mspdbcmf.exe'.

Here is the diagnostic output from the msbuild task:

Target "_GenerateAppxSymbolPackage: (TargetId:237)" in file "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\Microsoft.AppXPackage.Targets" from project "C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\App1.csproj" (target "_CreateAppxBundlePlatformSpecificArtifacts" depends on it):
Building target "_GenerateAppxSymbolPackage" completely.
Output file "C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\AppPackages\App1_1.1.7.0_Debug_Test\App1_1.1.7.0_x86_Debug.appxsym" does not exist.
Set Property: StripPrivateSymbols=false
Set Property: ReconstituteFastlinkPdbs=true
Set Property: MsPdbCmfExeFullpath=\mspdbcmf.exe
Using "GenerateAppxSymbolPackage" task from assembly "C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\Microsoft.Build.AppxPackage.dll".
Task "GenerateAppxSymbolPackage" (TaskId:488)
  Task Parameter:PdbCopyExeFullPath=C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\PDBCopy.exe (TaskId:488)
  Task Parameter:MsPdbCmfExeFullpath=\mspdbcmf.exe (TaskId:488)
  Task Parameter:
      PdbFiles=
          C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\bin\x86\Debug\App1.pdb
                  OutputGroup=DebugSymbolsProjectOutputGroup
                  ProjectName=App1
                  TargetPath=App1.pdb
          C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\bin\Debug\netstandard1.4\ClassLibrary1.pdb
                  BuildReference=true
                  CopyLocal=true
                  FusionName=
                  MSBuildSourceProjectFile=C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\ClassLibrary1.csproj
                  MSBuildSourceTargetName=Build
                  Name=ClassLibrary1
                  OriginalItemSpec=C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\bin\Debug\netstandard1.4\ClassLibrary1.dll
                  OriginalProjectReferenceItemSpec=..\ClassLibrary1\ClassLibrary1.csproj
                  OutputGroup=CopyLocalFilesOutputGroup
                  OutputItemType=
                  Project={265dbec3-4830-48b8-88cc-6965bd5fec75}
                  ProjectName=App1
                  ReferenceOutputAssembly=true
                  ReferenceSourceTarget=ProjectReference
                  ResolvedFrom=C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\bin\Debug\netstandard1.4\ClassLibrary1.dll
                  SetTargetFramework=TargetFramework=netstandard1.4;ProjectHasSingleTargetFramework=false;ProjectIsRidAgnostic=true
                  TargetPath=ClassLibrary1.pdb
                  Targets=
                  UndefineProperties=;ProjectHasSingleTargetFramework;RuntimeIdentifier;ProjectIsRidAgnostic
                  Version= (TaskId:488)
  Task Parameter:ProjectName=App1 (TaskId:488)
  Task Parameter:StrippedDirectory=obj\x86\Debug\Stripped (TaskId:488)
  Task Parameter:StripPrivateSymbols=False (TaskId:488)
  Task Parameter:ReconstituteFastlinkPdbs=True (TaskId:488)
  Task Parameter:AppxSymbolPackageOutput=C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\AppPackages\App1_1.1.7.0_Debug_Test\App1_1.1.7.0_x86_Debug.appxsym (TaskId:488)
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\PDBCopy.exe "C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\bin\x86\Debug\App1.pdb" obj\x86\Debug\Stripped\App1.pdb  (TaskId:488)
  Stripping private symbols from 'C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\bin\x86\Debug\App1.pdb'. (TaskId:488)
  C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\PDBCopy.exe "C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\bin\Debug\netstandard1.4\ClassLibrary1.pdb" obj\x86\Debug\Stripped\ClassLibrary1.pdb  (TaskId:488)
  Can't open pdb file C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\ClassLibrary1\bin\Debug\netstandard1.4\ClassLibrary1.pdb (TaskId:488)
STRIPPRIVATESYMBOLS : error : EC_FILE_SYSTEM [C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\App1.csproj]
  The command exited with code 3. (TaskId:488)
C:\Program Files (x86)\Microsoft Visual Studio\2017\Enterprise\MSBuild\Microsoft\VisualStudio\v15.0\AppxPackage\Microsoft.AppXPackage.Targets(2670,5): error APPX0502: File '\mspdbcmf.exe' not found. [C:\Users\Develop\Documents\Visual Studio 2017\Projects\App1\App1\App1.csproj]
Done executing task "GenerateAppxSymbolPackage". (TaskId:488)

Workaround

In the .NET Standard project, set the DebugType to pdbonly
e.g.:

<PropertyGroup>
    <DebugType>pdbonly</DebugType>
</PropertyGroup>

Or in the GUI, Project properties -> Build -> Advanced -> Debugging Information -> pdb-only

Issue also filed here.

[andriysavin]

Try using "pdb only" instead of "portable" in the .NET Standard library symbol generation settings.

[jmvillagra]

where is that setting? i'm facing the same issue here

[andriysavin]

In project settings, build->advanced if not mistaking (sorry, writing from a phone)

[jmvillagra]

I ended up changing the project file by hand to:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netstandard1.4</TargetFramework>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Debug|AnyCPU'">
    <DebugType>pdbonly</DebugType>
    <DebugSymbols>True</DebugSymbols>
  </PropertyGroup>

  <PropertyGroup Condition="'$(Configuration)|$(Platform)'=='Release|AnyCPU'">
    <DebugType>pdbonly</DebugType>
    <DebugSymbols>True</DebugSymbols>
  </PropertyGroup>

</Project>

This seems to work

[andriysavin]

Just for completeness:

[Thealexbarney]

I've added the workaround to the top-comment.

[dsplaisted]

It looks to me like the issue here is probably that the .NET Native toolchain doesn't handle portable PDBs.

[dsplaisted]

Based on the comment here, it looks like this has been fixed.

[wpeter-sfdc]

@dsplaisted I know that the thread you referenced says this should be fixed, but it doesn't seem like it is, in fact check the latest comment. Someone else still hitting this and so am I, I'm using dotnet sdk2.0 and VS2017 15.3 and UWP 5.4.0... can we re-open this issue?

[adamhewitt627]

I'm seeing this with VS2017 15.3.5, UWP5.4.0, and the UWP Community Toolkit:

C:\Program Files (x86)\Microsoft Visual Studio\2017\Professional\MSBuild\
  Microsoft\VisualStudio\v15.0\AppxPackage\Microsoft.AppXPackage.Targets(2846,5): warning : 
  Failed to convert FastLink symbols to full PDBs for 
  'C:\Users\adamhewitt\.nuget\packages\Microsoft.Toolkit\2.0.0\lib\netstandard1.4\
    Microsoft.Toolkit.pdb' due to failure to find the tool '\mspdbcmf.exe'

[wpeter-sfdc]

ping @dsplaisted - the workaround here only works if you wrote the netstandard projects that your UWP project is referencing, if you are pulling netstandard libraries from nuget then you are stuck.

[adamhewitt627]

@wpeter-sfdc Understood, but shouldn't the various declarations of it being "fixed" (links above, and the .NET Native 1.7 release notes supporting portable PDBs) mean the workaround isn't necessary?

[wpeter-sfdc]

@adamhewitt627 Right, my point is that this bug should have some urgency to it as the workaround only works in some scenarios - and as more nuget libraries move to netstandard the workaround will be less applicable (since netstandard uses portable PDB by default).

[adamhewitt627]

Ah good, we're on the same page. I completely missed that you're who already said it doesn't look fixed. I thought you were suggesting that nothing more needed done on this. Maybe a squeaky wheel can get some input on why this doesn't appear to be properly addressed in 1.7.

[trylek]

@adamhewitt627 @Thealexbarney I would love to better understand the issue you're hitting as I was involved in the portable PDB implementation in .NET Native and this is the first time I'm hearing about any issues with that. Thealexbarney pointed out that release build worked fine for him but that's the mode using .NET Native, debug build normally uses CoreCLR - are you hitting the issue in debug mode with the "Use .NET Native" switch turned on?

I must admit I don't fully understand the repro instructions above, the description almost exactly matches what I and others have been testing multiple times; I guess there must be something interestingly special about your code that's tripping this. The PDB format is pretty complex in its entirety, I can easily imagine we missed some corner case somewhere - that's exactly where a simple repro would be super helpful.

Thanks

Tomas

[adamhewitt627]

It turns out reproducing this is harder than I expected. I forgot to mention that I have only really seen this error when trying to build at command line, while Visual Studio works.
This project reproduces the error on the _first_ build, but not on a second build.

msbuild PdbRepro.csproj -t:Clean,Restore,Build -p:Configuration=Release -p:UapAppxPackageBuildMode=StoreUpload

EDIT: I didn't think about the build optimizing outputs. Including Clean in the list of targets prompts the error each time.

[trylek]

Thanks @adamhewitt627 for sharing the repro instructions, I've been able to reproduce the issue on my local installation of VS2017 15.4 preview 2. I believe the problem is an internal bug in VS build scripts. I have made a simple attempt to fix this but I wasn't successful, it will require a bigger scripting guru; I'll follow up on this with the VS folks. For now I have a very simple workaround that might get you unblocked in the short term: please set the environment variables

set VC_ExecutablePath_x64_x64=%VCToolsInstallDir%bin\HostX64\x64
set VC_ExecutablePath_x86_x86=%VCToolsInstallDir%bin\HostX86\x86

in the VS developer prompt prior to running msbuild to build your project. I was able to build your PdbRepro project successfully with this hack, please let me know if it fixes your issue.

Thanks

Tomas

[Thealexbarney]

Since @adamhewitt627 already posted a repro repo, here are somewhat more detailed steps to reproduce the error from the GUI in VS 15.3.4.

1. Create a new Blank UWP App.
2. Use version 5.4.0 of Microsoft.NETCore.UniversalWindowsPlatform.
3. Add a new .NET Standard 1.4 library project to the solution.
4. Reference the library project from the UWP project.
5. On the UWP project, right click -> Add -> Reference.
6. Select the .NET Standard project.
7. On the UWP project, right click -> Store -> Create App Packages. Don't build packages to upload to the Windows Store.
8. Create Debug configuration packages, and make sure include full PDB symbol files is selected.
9. The error will show up during the build.

At this point, associating the UWP app with the store and doing a build for upload to the store will also produce the error.

Setting those two environment variables and rebuilding from the command line gives
fatal error CMF1106: failed to open input PDB file for reading (PDB error code = 11)

[adamhewitt627]

I am also getting build errors, even with those environment variables set:
STRIPPRIVATESYMBOLS : error : EC_FILE_SYSTEM
and fatal error CMF1106: failed to open input PDB file for reading (PDB error code = 3)

[wpeter-sfdc]

@trylek since you've confirmed this bug still exists can you re-open this issue? Thanks!

[trylek]

Thanks @wpeter-sfdc and @adamhewitt627 for your additional feedback. There is an ongoing internal discussion on the subject and I have been given the following additional guidance on the subject:

_Basically, if a PDB was created with /fastlink, it is not included in symbols package. It should be fixable by making the following project change: In link.exe switches for C++ component(s), use /DEBUG:FULL or /DEBUG instead of /DEBUG:FASTLINK when intention is to produce Debug APPXBUNDLE and APPXSYM packages._

Please let me know if this additional info finally helps you overcome the problems you're hitting. My understanding of the internal discussion is that conversion of FASTLINK PDB's is simply not supported by design. Based on your feedback with respect to the simple workaround we can continue discussing whether additional changes might be desirable at the VS side.

[jmorei]

Hi @trylek -- thank you for the update. I'm unable to comment on the newest detail regarding C++ debug linking options, as I have little experience there. However, in case it is helpful, I've attempted the workaround you suggested above with environment variables and am encountering the same errors that @adamhewitt627 described. This build failure is blocking the consumption of the UWP Community Toolkit as a dependency in my UWP Store app project, and I've been unable to find any local workaround; it looks like we're dependent either on an update on the SDK side or the application of a workaround at the library dependency level in order to unblock.

[adamhewitt627]

@trylek I'm unclear what "simple workaround" you mean since setting the environment variables doesn't work. I'm also not sure how C++ build flags would impact a .NET app consuming a .NETStandard project. Also, that project file suggests it's building portable, not fastlink. (At least, I think that's the default?)

[wpeter-sfdc]

@trylek That workaround does not help I don't believe... if I'm understanding it correctly, that workaround still puts the onus on making changes to the way netstandard libraries are generated. However, our UWP app consumes 3rd party nuget packages that don't know or care about this workaround, and therefore we get build errors. This seems like a pretty core scenario: a UWP app consumes netstandard libraries via nuget.

Is there any workaround we can do on the UWP side (without needing to modify all 3rd party nugets we consume)?

Why is this issue still closed? Many folks (including yourself) have confirmed it still repros, can you re-open the issue? I keep having to explain to management/colleagues "this bug is blocking us, I know it says closed but it's not".

This bug is really hurting us, would appreciate some traction on it.

Thanks!

[trylek]

@wpeter-sfdc @adamhewitt627 Sorry Peter and Adam for the delay, we're really trying to find the rightful owner of this issue to prevent the situation when we reopen the issue and someone closes it again because it looks like the pre-existing situation was caused by a misunderstanding concerning the nature of the problem. I'm being told that someone with better understanding of VS scripts than I have should take over the investigation shortly and follow up on this thread as appropriate.

In the meantime, could you please let me know what version of Visual Studio you're using? I'm being told that in the latest version the error has been downgraded to a warning so that it no longer prevents packaging.

It also looks like I incorrectly interpreted the PDB flavor issue: indeed fastlink PDB's are irrelevant in the UWP case - it turns out that it's the case that the portable PDB's aren't supported either by the conversion tool. I understand you cannot affect the PDB flavor for nuget packages used by your app - that's probably where the new VS logic should hopefully provide some relief - however if you happen to be in control of the components you're using, it should be also possible to eliminate the problem by switching over to using "full" (non-portable) PDB's. [Modulo the fact that you'll still need to manually set the path to the convertor using the environment variables as I described above.]

Thanks for your patience

Tomas

[trylek]

FYI I have just created an internal bug tracking the issue that is now being triaged on the VS team.

[adamhewitt627]

@trylek Glad to hear. I'm on VS 15.3.5, and my experience is that _inside_ VS it is a warning. However, I've still been getting an error building from command line, which is preferable for an automated build.

[dotMorten]

Thank you for this. I've spent all day trying to get my commandline build working, and turns out I'm also hitting this bug.

[brad0000]

Same problem here, VS 15.3.5 building UWP app on Jenkins with Microsoft.Toolkit.Uwp.UI.Controls dependency.

[haefele]

Same bug, but using VS 15.4.1
But strangely only happening on my local machine. The build on AppVeyor is working for some reason.

[steingran]

Getting this bug on VS 15.4.1.

@trylek Any update on the progress on this issue?

[trylek]

@steingran I apologize, this is completely beyond me for now. I see that an internal VS task #504917 has been created to track this but that's about it. @tmat, do you have any idea how to continue getting a traction on this issue?

[vitek-karas]

I verified that the app described by the repro steps from @Thealexbarney above is fixed with recent build of VS 15.5. The fix should be in VS 15.5 RTM. Not sure if it made it to 15.5 Preview 4 (once it's actually released which should be any day now), but it might.
Investigating this issue we also found another small one where .NET Native build will not copy a .pdb file with no matching binary into the resulting app. Fix for that will come at some later time.

[MelbourneDeveloper]

I am currently not able to debug with .NET Standard 2.0 on UWP. It's not all solutions, but one particular solution I cannot get working. In my case, I am seeing the PDB files. I have logged the issue here: https://github.com/dotnet/standard/issues/621 . I wonder if there are still general issues with Visual Studio debugging .NET Standard dlls on UWP...

I am using Visual Studio 15.5.3

[drewdz]

I realize this is an older thread but I would like to add a "me too" to this: I have created a brand new UWP project as part of a Xamarin.Forms application. I am unable to set any breakpoints in code that I have written.

Breakpoints are disabled with the message that "No symbols have been loaded..."

Here is some of my version info...

Microsoft Visual Studio Enterprise 2017
Version 15.7.1
VisualStudio.15.Release/15.7.1+27703.2000
Microsoft .NET Framework
Version 4.7.02556
.
.
.
Xamarin 4.10.0.442 (396b18cef)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer 4.12.264 (fc37cd02e)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK 8.3.0.19 (HEAD/342b2ce96)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 11.10.1.177 (7e782c1)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

[TommasoScalici]

Me too, I'm experiencing the same behavior. Breakpoints are disabled while doing debug, and not triggered.

My Visual Studio info:

Microsoft Visual Studio Community 2017
Versione 15.7.5
VisualStudio.15.Release/15.7.5+27703.2042
Microsoft .NET Framework
Versione 4.7.03056

Edizione installata: Community

Visual C++ 2017 00369-60000-00001-AA619
Microsoft Visual C++ 2017

Gestione pacchetti NuGet 4.6.0
Gestione pacchetti NuGet in Visual Studio. Per altre informazioni su NuGet, visitare il sito Web http://docs.nuget.org/.

GitHub.VisualStudio 2.5.4.3349
A Visual Studio Extension that brings the GitHub Flow into Visual Studio.

Mono Debugging for Visual Studio 4.10.5-pre (ab58725)
Support for debugging Mono processes with Visual Studio.

Msiler 2.2.1
MSIL Code Viewer Extension for Visual Studio 2012/2013/2015

Multilingual App Toolkit 4.0
Toolkit di applicazioni multilingue consente di localizzare l'applicazione Windows Store fornendo gestione file, pseudo e traduzione automatica, editor di traduzione e integrazione di compilazione. http://aka.ms/matinstall
Xamarin Designer 4.12.1 (f3257e429)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK 8.3.3.2 (HEAD/dffc59120)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK 11.12.0.4 (64fece5)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

[eberkund]

I am experiencing this issue when building on appcenter.ms with the Quartz scheduling library.

[vitek-karas]

/cc @tommcdon