Runtime: Corefx package update in corefxlab results in free before malloc error during restore on OSX

[ahsonkhan]

From https://github.com/dotnet/corefxlab/pull/2140
Update Cli, CoreFx, CoreSetup to preview2-008266, preview2-26228-03, preview2-26227-03, respectively

https://ci.dot.net/job/dotnet_corefxlab/job/master/job/osx10.12_debug_prtest/1613/consoleFull#7440085258bb52e59-de52-4f1c-b131-470db4c594cd

22:09:53 Restoring all packages
22:09:53 Error:
22:09:53   An assembly specified in the application dependencies manifest (dotnet.deps.json) was not found:
22:09:53     package: 'Microsoft.ApplicationInsights', version: '2.0.0'
22:09:53     path: 'lib/dotnet5.4/Microsoft.ApplicationInsights.dll'
22:09:53 dotnet(51167,0x7fffcf7163c0) malloc: *** error for object 0xffffffffffffffff: pointer being freed was not allocated
22:09:53 *** set a breakpoint in malloc_error_break to debug
22:09:53 ./build.sh: line 84: 51167 Abort trap: 6           ./$dotnetExePath restore $myFile /p:VersionSuffix="$BuildVersion"
22:09:53 
Failed to restore packages.

cc @danmosemsft, @livarcocc, @joshfree

[pakrym]

We see this after updating to 2.1.0-preview2-26225-03 runtime while on 2.1.300-preview2-008251 CLI.

All ASP.NET Core, OS X CI agents are failing with this error.

Moving to newer shared runtime is required to get System.IO.Piplines from corefx and we've already made a lot of reaction changes to accommodate the move.
/cc @muratg

[pakrym]

Removing new hostfxr helps.

[pakrym]

/cc @steveharter

[steveharter]

Does the malloc assert occur only in the deps.json failure case?

I'll try to get a local repro.

[steveharter]

I assume the deps.json failure is not expected, and the malloc issue is secondary. Is this correct?

[pakrym]

Yes

[anurse]

:eyes: This issue is causing all our macOS builds to fail. We're getting close to a standstill. Are there any tests that can prevent this from happening in the future? This repros when running any SDK command using the new hostfxr.

[muratg]

👁

[eerhardt]

This repros when running any SDK command using the new hostfxr.

I'm surprised that all the CLI tests passed on macOS here: https://github.com/dotnet/cli/pull/8704. That used 2.1.0-preview2-26227-01 of the runtime.

/cc @nguerrera @livarcocc @wli3

[steveharter]

On my mac, I wasn't able to repro a simple dotnet new console failure with that build

[steveharter]

But was able to repro by building https://github.com/aspnet/kestrelhttpserver/

[anurse]

We're seeing the problem with hostfxr build 2.1.0-preview2-26225-03 I believe. Deleting that host/fxr folder fixes the issue (even while still using the matching runtime) since it falls back to a previous hostfxr build.

[steveharter]

An update: at the root level I believe this is an issue with the way we dynamically load and invoke hostpolicy; it could be our bug or an OSX thing. More on that later.

Running https://github.com/aspnet/kestrelhttpserver/build.sh installs two version of the host/fxr and shared/Microsoft.NETCore.App:

• 2.1.0-preview2-26131-06
• 2.1.0-preview2-26225-03

I'm not sure why two versions are installed. If only one set is loaded there doesn't appear to be an issue (at least not this current issue).

When running dotnet.exe it picks the newer hostfxr (2.1.0-preview2-26225-03) and using the settings from the SDK (/Users/steveharter/.dotnet/sdk/2.1.300-preview2-008251/dotnet.runtimeconfig.json) which is targeting Microsoft.NETCore.App, version '2.1.0-preview2-26131-06' hostfxr loads and executes the hostpolicy from that Microsoft.NETCoreApp folder. So the hostpolicy version is older than the hostfxr version.

Normally loading a newer hostfxr and an older hostpolicy is fine and a supported scenario. The contract between the two is a binary pack\unpack which flows information from hostfxr to hostpolicy in a backwards-compatible way. That contract part appears to be working fine and hostfxr appears to load and invoke the hostpolicy entry point successfully in the same process. Hostpolicy and hostfxr actually cross-compile some of the same code (not all) and when hostpolicy is invoked, it is actually finding and using the previously loaded code\module for hostfxr, which is newer in this case, instead of the code that was compiled for hostpolicy. Since some cross-compiled types have different memory layouts, that is likely causing issues. This bug surfaced when a commit was made that changed a bunch of this cross-compiled code. I'll look at this closer on Friday.

[steveharter]

Here's some steps to verify what I'm seeing:
1) On OSX and with a https://github.com/aspnet/kestrelhttpserver repo
2) export COREHOST_TRACE=1
3) Run build.sh 2>out1.txt
4) Remove\rename the folder for the newer fxr at /Users/<user>/.dotnet/fxr/2.1.0-preview2-26225-03
5) Run build.sh 2>out2.txt

Diff out1.txt and out2.txt

You should see lines like this diffing:
<< Adding runtime asset runtimes/osx-x64/lib/netcoreapp2.1/Microsoft.CSharp.dll assemblyVersion= fileVersion= from runtime.osx-x64.Microsoft.NETCore.App/2.1.0-preview2-26131-06
>> Adding runtime asset runtimes/osx-x64/lib/netcoreapp2.1/Microsoft.CSharp.dll from runtime.osx-x64.Microsoft.NETCore.App/2.1.0-preview2-26131-06

Notice the different formatting (assemblyVersion, fileVersion) in the first. The newer code has different formatting, but the code that is outputting this information is run from hostpolicy, not hostfxr. Both the old and new hostfxr are loading the same hostpolicy (I verified through lldb and image list). However, both hostfxr and hostpolicy contain that code (cross-compiled). So I believe due to (required) compilation with Position Independent Code (PIC) it is finding and using the previously loaded code in hostfxr.

I'm going to see if adding namespaces in the hostfxr project fixes the issue. I may also refactor the hostfxr project to not include unused types.

[steveharter]

@pakrym others - are you going to ignore this issue and commit anyway?

I'm working on a fix, but need verification from other(s) before I send the PR, which will consist of adding namespaces to hostfxr. I'm shooting for end of day...

[jkotas]

I think the best way to deal with the C/C++ name collisions is to limit what your module depends on using linker scripts. We do that in CoreCLR: https://github.com/dotnet/coreclr/blob/master/src/dlls/mscoree/coreclr/CMakeLists.txt#L25 @janvorli put this together for CoreCLR, so he can help you with any details if you need.

[pakrym]

@pakrym others - are you going to ignore this issue and commit anyway?

Commit what?

[steveharter]

Are you still blocked?

[pakrym]

@steveharter we added a temporary workaround that removes latest hostfxr to dotnet-install script.

[steveharter]

Final update: the issue has been fixed and I verified on my mac that the latest core-setup build 2.1.0-preview2-26305-07 doesn't have this issue (i.e. I can run SDK commands without a crash\malloc error).

This issue has always existed AFAIK, but didn't trigger until non-trivial changes were made to types shared between hostfxr and hostpolicy and an attempt was made to use a hostpolicy that had the older versions of these types. I created an issue to add automated tests which should catch future cases.