Arcade: Symbol Publishing Failure

[vatsan-madhavan]

• Is this issue blocking: yes

Please see https://dev.azure.com/dnceng/internal/_build/results?buildId=507786&view=results

This is an internal-only prototype branch. The DLL and the corresponding symbol that's failing to publish haven't been altered in anyway, although the branch itself has seen extensive changes.

To build, checkout the branch and use:

build -pack -configuration Release -platform x64 -ci /p:DotNetUseShippingVersions=true

/cc @JohnTortugo, @mikem8361
/cc @rladuca

[vatsan-madhavan]

From a local build, I built the binaries that are failing to upload symbols as an experiment(PresentationNative_cor3.pdb), and then created a WPF app with privates from this build and tried loading this PDB in WinDbg.

This is what I got (successful loading of symbols):

DBGHELP: C:\Program Files\dotnet\shared\Microsoft.WindowsDesktop.App\5.0.0-alpha1.19530.2\PresentationNative_cor3.pdb cached to C:\ProgramData\Dbg\sym\PresentationNative_cor3.pdb\F4D2A6EDC2B3450C9292B70DCCAFCDCE1\PresentationNative_cor3.pdb

DBGHELP: PresentationNative_cor3 - private symbols & lines 
      C:\ProgramData\Dbg\sym\PresentationNative_cor3.pdb\F4D2A6EDC2B3450C9292B70DCCAFCDCE1\PresentationNative_cor3.pdb

Based on this, it doesn't look like there is anything wrong with the PDB's. Also based on the history and given there has been no change to the way this DLL is being built, the failures are somewhat surprising.

[mikem8361]

Looks like the uploader is failing on runtimes\win-x64\native\bilinearspan-compile.pdb in the runtime.win-x64.Microsoft.DotNet.Wpf.DncEng.5.0.0-alpha.1.20104.15.symbols.nupkg package. I'm investigating. The pdb seems valid but I'm not sure why the uploader is failing.

[JohnTortugo]

Thanks @mikem8361

[vatsan-madhavan]

That pdb is _intetesting_ in that it belongs to a .lib file and not a .dll.

[vatsan-madhavan]

The DLL and the corresponding symbol that's failing to publish haven't been altered in anyway

This is only true for PresentationNative_cor3.pdb/dll.

bilinearspan-compile.pdb has never been published before - it doesn't have any history of success/failure until now.

[mikem8361]

Is there anyway not to include bilinearspan-compile.pdb in these packages. It has an empty “Dbi” stream which isn’t that usual for a PDB. At least we haven’t hit this with other PDBs.

[vatsan-madhavan]

Let me play around with /Z7 and see if I can embed that pdb.

[vatsan-madhavan]

@mikem8361 Is there a way for me to include the PDB but prevent it from being published ?

I think I can use /Z7/oldStyle to embed the PDB in the .lib file, but when linking against the .lib, the PCH compilation usually looks for the PDB's corresponding to the libs. I could probably try changing everything to /Z7 (so that when PCH compilation happens on the side that consumes bilenarspan.lib, it wouldn't look for bilinearspan-compile.pdb), but then we wouldn't be able to produce separate PDB's for shipping DLL's - and that's not good.

[mikem8361]

There is an exclusion file that arcade provides the symbol uploader called “eng\SymbolPublishingExclusionsFile.txt”. It contains the relative path in the nuget package which would be something like

runtimes/win-x64/native/bilinearspan-compile.pdb
runtimes/win-x86/native/bilinearspan-compile.pdb

From: Vatsan Madhavan [e-mail]
Sent: Wednesday, February 5, 2020 1:17 PM
To: dotnet/arcade [e-mail]
Cc: Mike McLaughlin [e-mail]; Mention [e-mail]
Subject: Re: [dotnet/arcade] Symbol Publishing Failure (#4763)

@mikem8361https://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fmikem8361&data=02%7C01%7Cmikem%40microsoft.com%7Cd0681d4f11b7425b704308d7aa80bf65%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165342251735706&sdata=Al05cGA9crn1D%2F46wxieVQyMvTF48LpW6Uvidb%2BP79c%3D&reserved=0 Is there a way for me to include the PDB but prevent it from being published ?

I think I can use /Z7/oldStyle to embed the PDB in the .lib file, but when linking against the .lib, the PCH compilation usually looks for the PDB's corresponding to the libs. I could probably try changing everything to /Z7 (so that when PCH compilation happens on the side that consumes bilenarspan.lib, it wouldn't look for bilinearspan-compile.pdb), but then we wouldn't be able to produce separate PDB's for shipping DLL's - and that's not good.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHubhttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fdotnet%2Farcade%2Fissues%2F4763%3Femail_source%3Dnotifications%26email_token%3DACPYK22FJMZUPQRB3UNNTBDRBMUE7A5CNFSM4KQB2LEKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEK5AM3I%23issuecomment-582616685&data=02%7C01%7Cmikem%40microsoft.com%7Cd0681d4f11b7425b704308d7aa80bf65%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165342251745662&sdata=em3CjiIXmcwBC%2FdEceVJPieQAtszQF45k%2BF6C%2Fo0IG4%3D&reserved=0, or unsubscribehttps://nam06.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FACPYK26STQLQB6KJHTRSTV3RBMUE7ANCNFSM4KQB2LEA&data=02%7C01%7Cmikem%40microsoft.com%7Cd0681d4f11b7425b704308d7aa80bf65%7C72f988bf86f141af91ab2d7cd011db47%7C1%7C0%7C637165342251745662&sdata=Nz%2FQT%2F9cqwQMrBn5ODYToyMRw2OYeSiNS4pNjUhzR3E%3D&reserved=0.

[JohnTortugo]

@vatsan-madhavan - were you able to workaround this? I'm not sure what the action step is for this issue.

[vatsan-madhavan]

Yes-ish. The workaround I have now was very expensive to write (took over a week to build right) and the only reason I went down that path was because my work was time-sensitive. Building one library that way has ripple effects on other builds, which requires careful adjustments...

I’d like to undo use of /oldStyle actually since it’s outmoded - so I’d like the issue to be kept open for a fix to be made to the symbol publishing library.

[markwilkie]

Do you know what the right fix looks like @JohnTortugo ?

[JohnTortugo]

@mikem8361: Is there anyway not to include bilinearspan-compile.pdb in these packages. It has an empty “Dbi” stream which isn’t that usual for a PDB. At least we haven’t hit this with other PDBs.

I think the next step would be to improve the symbol uploader written by @mikem8361 to be able to gracefully handle files with empty "Dbi". I'm not sure how much work that would be. @mikem8361 can you please elaborate?

[mikem8361]

I can change the uploader to skip PDBs that don't have a DbiStream. It wouldn't take much to do that.

[mikem8361]

Is that good enough?

[vatsan-madhavan]

I think so. For now, I've only see it in in PDB's for object files. When a DLL, LIB or EXE is produced, the final PDB seems fine, and that final PDB would upload to the sym-server without problems.

In the future if we encounter a situation where uploading these PDB's becomes ncecessary, this could be revisited. I would recommend opening an issue that describes the problem (about the DbiStream) clearly (or modifying this issue's description) so that someone who encounters this problem in the future could find it and report back.

[riarenas]

@mikem8361 Are we using a symbol uploader with this fix yet?

[mikem8361]

I haven’t got this fix into symstore yet so it isn’t in the symuploader either. I try to get it in soon. Then it will take a while to get it into symloader after that.

[riarenas]

Thanks!

[mikem8361]

Version 1.0.0-beta-65224-01 SymUploader and Microsoft.SymbolUploader.Build.Task have been published to buildtools feed (https://dotnet.myget.org/gallery/dotnet-buildtools).

They are also in:

\bvtsrv2\team\DotNetCore\DotNet-SymUploader

[garath]

Triage: Looks like this is merged. Closing.