Ef6: Source-stepping

[jnm2]

Providing PDBs with SourceLink could have a great effect on your team’s day jobs and side projects. It will make it easy for anyone to drop into Entity Framework code any time they want to understand what's going on, and EF users may return the courtesy by taking some initiative in diagnosing bugs they observe. Creating a repro is much easier when you aren't dealing with a black box!

When searching for prior issues I found https://data.uservoice.com/forums/72025-entity-framework-core-feature-suggestions/suggestions/8622391-ship-symbols-pdb-for-entityframework-6-x:

@divega responded:

This is something we have spent many hours attempting but symbolsource would not accept our package. We would try something else if something else works.

I’m sorry about those hours lost and offer to donate mine. What many open source projects do, and I see guidance on this now too, is to include the PDBs in the main NuGet.org package. All you have to do as a consumer is turn off Just My Code and you get the full debugging experience. There is almost no barrier to entry, unlike setting up symbol servers.

In my experience if you use portable PDBs + SourceLink, the size overhead in the final nupkgs ranges from an additional 10% to 50%. Either way hard disk space is cheap, and ‘just working’ is a terrific tradeoff.

[jnm2]

See also https://github.com/dotnet/sourcelink#sourcelink, available to the SDK-based csproj format in the .NET Core 2.1 timeframe.

[richlander]

Cool! Any plans to start working on this now that .NET Core 2.1 has shipped?

[ajcvickers]

This will be covered by the ASP.NET Core build.

[MagicAndre1981]

@ajcvickers what about user who don't care about EF Core? We want this for EF6

[ajcvickers]

@MagicAndre1981 I didn't mention EF Core. This issue is about EF6, which in the future will be handled by the ASP.NET Core build system.

[MagicAndre1981]

this is confusing. Why can't this be done now by simply adding a Directory.Build.props ? Why must this be done via a new Build system? 🤔🤷‍♂️

[ajcvickers]

@MagicAndre1981 Is there value in doing this without releasing it?

[jnm2]

@MagicAndre1981 He's saying that the next release of EF6 is already going to have source-stepping capabilities for free without doing any additional work on this, because of recent changes they've made to their build process for other reasons.

[MagicAndre1981]

@jnm2 ok, thanks for this good explanation.

@ajcvickers simply release a 6.2.1 with source stepping 🙄

[jnm2]

@MagicAndre1981 A release purely containing a single enhancement doesn't make sense unless no other development work is underway. Imagine if everyone asked for their favorite enhancement to get its own special release?

[MagicAndre1981]

the purpose of nuget is to get fixes/improvements faster to users and not wait months for a new installer. Helping users to debug EF6 is a HUGE improvement with high priority. I'm out here and unsubscribe now from that issue. Bye ...

[jnm2]

@MagicAndre1981 For reference, in all the open-source and proprietary projects I've worked on, source-stepping was considered a very minor enhancement. I think we should be fair to the Entity Framework team.

If you need something now, I'd use the free dotPeek as a symbol server.

[MagicAndre1981]

I just tested with 6.3.0-preview9-19423-04 and PDBs are NOT available, not in nuget package, nor on nuget symbol server. So there is something wrong with the so cool buildsystem when final bits are about to be released end of September and we should expect that the last preview represents to 99% the final bits and dev experience 🤦‍♂️

[jnm2]

@MagicAndre1981 Welcome back :)

@ajcvickers Should we expect previews to have symbols contained in the package or uploaded to symbols.nuget.org?

[MagicAndre1981]

@MagicAndre1981 Welcome back :)

Should we expect previews to have symbols contained in the package or uploaded to symbols.nuget.org?

for .net core preview they are on nuget symbol server

[bricelam]

@leecow Are you publishing the packages during Andrew's leave? Do you know if we just forgot to publish the symbols during past releases?

[leecow]

We've brought ASP into the umbrella .NET Core release pipeline but don't know if EF was a part of that work. I suspect not so, this will need to be covered by Andrew's proxies or Arthur.

[MagicAndre1981]

still no code stepping in 6.3 RC1. Now we can hope this so cool build system gets fixed for 6.4 release so that this simple but powerful feature is enabled there 🙄

[bricelam]

@anurse Do you know what we're missing here? We produce the *.symbols.nupkg files...

[MagicAndre1981]

We produce the *.symbols.nupkg files...

this is wrong format. NuGet Symbol server wants snupkg files.

NuGet.org's symbol server only accepts the new symbol package format - .snupkg.

[bricelam]

@MagicAndre1981 That's covered by https://github.com/dotnet/arcade/issues/1959

[anurse]

this is wrong format. NuGet Symbol server wants snupkg files.

We don't publish symbols to NuGet's symbol server, we publish to the official Microsoft Symbol Server and our process does use .symbols.nupkg.

It looks like the release pipeline for EF6 doesn't publish the symbols, just the packages (our other pipelines publish symbols from the .symbols.nupkg packages). It should be a matter of adding the necessary tasks to the pipeline and running a release but I don't have context on what those tasks need to be. @dougbu and @JunTaoLuo might be able to help guide you here @bricelam. Comparing with the other pipelines (like the one for AspNetCore) would be a good start.

[jnm2]

Could you publish to NuGet's server too, like .NET Core does? That's what I have enabled.

[anurse]

Could you publish to NuGet's server too, like .NET Core does? That's what I have enabled.

Is that the case? It's my understanding we publish .NET Core symbols to the Microsoft Symbol Server as well (not the NuGet server). The Microsoft one is enabled by default in our debuggers. Have you explicitly disabled it?

[anurse]

Yeah, it doesn't look to me that .NET Core packages are published to NuGet's servers. It's generally our policy to use the official Microsoft symbol server (since we have that infrastructure in place). The NuGet symbol server is primarily for third-party usage.

> C:\Debuggers\SymChk.exe /s "SRV*https://symbols.nuget.org/download/symbols" .\System.Text.Json.dll
SYMCHK: System.Text.Json.dll FAILED  - System.Text.Json.ni.pdb mismatched or not found
SYMCHK: FAILED files = 1
SYMCHK: PASSED + IGNORED files = 0
> C:\Debuggers\SymChk.exe /s "SRV*https://msdl.microsoft.com/download/symbols" .\System.Text.Json.dll
SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

We should definitely get the EF6 symbols published to the Microsoft symbol server though. We have the infrastructure, we just need to incorporate these packages.

[dougbu]

It looks like the release pipeline for EF6 doesn't publish the symbols

EF6 does publish symbols for official builds as part of the usual post-build stages. This is true for the 'release/6.3', 'release/6.4' and 'master' branches as of @smitpatel's merges into 'master' yesterday. And, the same is true across ASP.NET Core repos.

We don't use the NuGet symbol servers anywhere in ASP.NET Core.

Could you publish to NuGet's server too, like .NET Core does?

This would duplicate the pushes and cause extra special cases when we push to NuGet. Further, the .snupkg requirement is extra work we have no requirement for at the moment.

I'm not sure anything in .NET Core still pushes symbols to the NuGet symbol server. How did you confirm this?

---

Bottom line, please enable the default Microsoft symbol servers to enable stepping into ASP.NET Core code for EF6 daily builds as well as official releases.

[anurse]

It does indeed look like the symbols are on the Microsoft symbol server:

ps1 $ C:\Debuggers\SymChk.exe /s "SRV*https://msdl.microsoft.com/download/symbols" .\entityframework.6.3.0-rc1-19458-04.nupkg\lib\netstandard2.1\EntityFramework.dll
SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

[jnm2]

@anurse @dougbu I must be wrong, I apologize. I always keep Microsoft Symbol Server unchecked as far as I know. I must have been stepping into corefx source code by some other means.

[MagicAndre1981]

It does indeed look like the symbols are on the Microsoft symbol server:

I have https://msdl.microsoft.com/download/symbols in symbol config in VS2017, but can't get PDB for net45 version of EF6.3 RC1. I get PDBs for Microsoft dlls, but no chance of getting PDB for EF6.

What is wrong for me?

C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>symchk.exe "C:\Users\USERNAME\.nuget\packages\entityframework\6.3.0-rc
1-19458-04\lib\net45\EntityFramework.dll" /s https://msdl.microsoft.com/download/symbols
SYMCHK: EntityFramework.dll  FAILED  - /_/artifacts/obj/EntityFramework/Release/net45/EntityFramework.pdb mismatched or not found

SYMCHK: FAILED files = 1
SYMCHK: PASSED + IGNORED files = 0

//Edit

ok, PDB for .netstandard 2.1 can be found, but not normal .net framework which I use:

C:\Program Files (x86)\Windows Kits\10\Debuggers\x64>symchk.exe  "C:\Users\USERNAME\.nuget\packages\entityframework\6.3.0-rc1-19458-04\lib\netstandard2.1\EntityFramework.dll" /s https://msdl.microsoft.com/download/symbols

SYMCHK: FAILED files = 0
SYMCHK: PASSED + IGNORED files = 1

I checked the PDB with dotPeek and it has indeed source link enabled:

<SourceLink>
    <Conversion FilePath="/_/*" Uri="https://raw.githubusercontent.com/aspnet/EntityFramework6/7d269be1011313b730c36c51b8ab4920076b627d/*" />
</SourceLink>

So I did everything correctly, but there is still an issue that PDBs for net40 and net45 are not uploaded only ns2.1 which I don't care at all as I will not use VS2019 and .net core 3.x until the broken VS2019 UI gets repaired.

@bricelam @anurse

So reopen the issue until net4x PDBs are uploaded

[anurse]

Good catch! I will admit that I just checked the netstandard2.1 DLL 🤦‍♂, I can also repro a symchk failure for the net45 DLL.

[ajcvickers]

@bricelam What is actionable here?

[bricelam]

Investigating why the net45 symbols are still missing

[MagicAndre1981]

Why is this removed from 6.4? With entityframework.6.4.0-preview3-19553-01 I can get the 4.5 PDB, but 4.0 and NS2.1 fail, for preview2, the 40 and 45 fail but ns21 was downloadable.

Your so cool buildsystem only randomly picks 1 dll/pdb combination. Still again shows that symbol servers suck and the original sourcelink idea to embed the PDB with source data to the nugets was best.

[jnm2]

@MagicAndre1981 Rudeness is not an effective motivational strategy. Or rather, it motivates people in directions that hurt the point you're making.