Xamarin-macios: Could not AOT the assembly System.Runtime.CompilerServices.Unsafe.dll (MT3001)

[juripyykko]

Steps to Reproduce

1. Create new Cross-Platform Xamarin.Forms .NET Standard 2.0 project in Visual Studio (i used 2017 PC).
2. Open this project in Visual Studio for Mac.
3. Add NuGet Microsoft.EntityFrameworkCore.Sqlite to the iOS project.
4. Add the following for example to AppDelegate.cs in the iOS project:
   var temp = Microsoft.EntityFrameworkCore.PropertyAccessMod.Field;
5. Try to run this on an iPhone (which I believe requires some config, e.g. bundle identifier etc.)

Expected Behavior

App starting on device showing the default MainPage.

Actual Behavior

Error:

Could not AOT the assembly '/Users/masystem/Dev/TestProjects/TestReproduceError/TestReproduceError/TestReproduceError.iOS/obj/iPhone/Debug/device-builds/iphone9.3-11.1.2/mtouch-cache/Build/System.Runtime.CompilerServices.Unsafe.dll' (MT3001)

Environment

Visual Studio on PC

Microsoft Visual Studio Professional 2017 
Version 15.5.1
VisualStudio.15.Release/15.5.1+27130.2003
Microsoft .NET Framework
Version 4.7.02556

Installed Version: Professional

Visual Basic 2017   00370-20004-56726-AA880
Microsoft Visual Basic 2017

Visual C# 2017   00370-20004-56726-AA880
Microsoft Visual C# 2017

Visual F# 4.1   00370-20004-56726-AA880
Microsoft Visual F# 4.1

Application Insights Tools for Visual Studio Package   8.10.01106.1
Application Insights Tools for Visual Studio

ASP.NET and Web Tools 2017   15.0.31125.0
ASP.NET and Web Tools 2017

ASP.NET Core Razor Language Services   1.0
Provides languages services for ASP.NET Core Razor.

ASP.NET Web Frameworks and Tools 2017   5.2.51007.0
For additional information, visit https://www.asp.net/

Azure App Service Tools v3.0.0   15.0.31106.0
Azure App Service Tools v3.0.0

Azure Data Lake Node   1.0
This package contains the Data Lake integration nodes for Server Explorer.

Azure Data Lake Tools for Visual Studio   2.2.9000.1
Microsoft Azure Data Lake Tools for Visual Studio

Azure Data Lake Tools for Visual Studio   2.2.9000.1
Microsoft Azure Data Lake Tools for Visual Studio

Common Azure Tools   1.10
Provides common services for use by Azure Mobile Services and Microsoft Azure Tools.

Fabric.DiagnosticEvents   1.0
Fabric Diagnostic Events

JavaScript Language Service   2.0
JavaScript Language Service

JavaScript Project System   2.0
JavaScript Project System

JavaScript UWP Project System   2.0
JavaScript UWP Project System

Merq   1.1.17-rc (cba4571)
Command Bus, Event Stream and Async Manager for Visual Studio extensions.

Microsoft Azure HDInsight Azure Node   2.2.9000.1
HDInsight Node under Azure Node

Microsoft Azure Hive Query Language Service   2.2.9000.1
Language service for Hive query

Microsoft Azure Service Fabric Tools for Visual Studio   1.8
Microsoft Azure Service Fabric Tools for Visual Studio

Microsoft Azure Stream Analytics Language Service   2.2.9000.1
Language service for Azure Stream Analytics

Microsoft Azure Stream Analytics Node   1.0
Azure Stream Analytics Node under Azure Node

Microsoft Azure Tools   2.9
Microsoft Azure Tools for Microsoft Visual Studio 2017 - v2.9.51120.3

Microsoft Continuous Delivery Tools for Visual Studio   0.3
Simplifying the configuration of continuous build integration and continuous build delivery from within the Visual Studio IDE.

Microsoft JVM Debugger   1.0
Provides support for connecting the Visual Studio debugger to JDWP compatible Java Virtual Machines

Microsoft MI-Based Debugger   1.0
Provides support for connecting Visual Studio to MI compatible debuggers

Microsoft Visual Studio Tools for Containers   1.1
Develop, run, validate your ASP.NET Core applications in the target environment. F5 your application directly into a container with debugging, or CTRL + F5 to edit & refresh your app without having to rebuild the container.

Microsoft Visual Studio VC Package   1.0
Microsoft Visual Studio VC Package

Mono Debugging for Visual Studio   4.8.4-pre (3fe64e3)
Support for debugging Mono processes with Visual Studio.

NuGet Package Manager   4.5.0
NuGet Package Manager in Visual Studio. For more information about NuGet, visit http://docs.nuget.org/.

SQL Server Data Tools   15.1.61710.120
Microsoft SQL Server Data Tools

ToolWindowHostedEditor   1.0
Hosting json editor into a tool window

TypeScript Tools   15.5.11025.1
TypeScript Tools for Microsoft Visual Studio

WebJobs Tools v1.0.0   15.0.31201.0
WebJobs Tools v1.0.0

Visual Studio Code Debug Adapter Host Package   1.0
Interop layer for hosting Visual Studio Code debug adapters in Visual Studio

Visual Studio Tools for Universal Windows Apps   15.0.27128.01
The Visual Studio Tools for Universal Windows apps allow you to build a single universal app experience that can reach every device running Windows 10: phone, tablet, PC, and more. It includes the Microsoft Windows 10 Software Development Kit.

VisualStudio.Mac   1.0
Mac Extension for Visual Studio

Xamarin   4.8.0.753 (6575bd113)
Visual Studio extension to enable development for Xamarin.iOS and Xamarin.Android.

Xamarin Designer   4.8.183 (2577c82ea)
Visual Studio extension to enable Xamarin Designer tools in Visual Studio.

Xamarin.Android SDK   8.1.0.24 (HEAD/9cfa7836b)
Xamarin.Android Reference Assemblies and MSBuild support.

Xamarin.iOS and Xamarin.Mac SDK   11.6.1.2 (6857dfc)
Xamarin.iOS and Xamarin.Mac Reference Assemblies and MSBuild support.

Visual Studio on Mac

=== Visual Studio Community 2017 for Mac ===

Version 7.3.3 (build 5)
Installation UUID: 53fae928-0326-4001-840c-b7d7eff46e01
Runtime:
Mono 5.4.1.7 (2017-06/e66d9abbb27) (64-bit)
GTK+ 2.24.23 (Raleigh theme)

Package version: 504010007

=== NuGet ===

Version: 4.3.1.4445

=== .NET Core ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Version: 2.0.0
SDK: /usr/local/share/dotnet/sdk/2.0.0/Sdks
SDK Version: 2.0.0
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/5.4.1/lib/mono/msbuild/15.0/bin/Sdks

=== Xamarin.Profiler ===

Version: 1.6.0
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Xamarin.Android ===

Version: 8.1.3.0 (Visual Studio Community)
Android SDK: /Users/masystem/Library/Developer/Xamarin/android-sdk-macosx
Supported Android versions:
7.1 (API level 25)
8.0 (API level 26)

SDK Tools Version: 25.2.5
SDK Platform Tools Version: 25.0.5
SDK Build Tools Version: 25.0.3

Java SDK: /usr
java version "1.8.0_131"
Java(TM) SE Runtime Environment (build 1.8.0_131-b11)
Java HotSpot(TM) 64-Bit Server VM (build 25.131-b11, mixed mode)

Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Xamarin Inspector ===

Version: 1.3.2
Hash: 461f09a
Branch: 1.3-release
Build date: Tue, 03 Oct 2017 18:26:57 GMT
Client compatibility: 1

=== Apple Developer Tools ===

Xcode 9.2 (13772)
Build 9C40b

=== Xamarin.iOS ===

Version: 11.6.1.3 (Visual Studio Community)
Hash: f70a1348
Branch: xcode9.2
Build date: 2017-12-18 14:47:16-0500

=== Xamarin.Mac ===

Version: 4.0.0.215 (Visual Studio Community)

=== Build Information ===

Release ID: 703030005
Git revision: b1c2982e201e71ef758866c9ade05f253a8c6f47
Build date: 2017-12-21 11:04:40-05
Xamarin addins: f397ddfbacfb39e60c9cc8d9e410f73faf8c2cbc
Build lane: monodevelop-lion-d15-5

=== Operating System ===

Mac OS X 10.12.6
Darwin 16.7.0 Darwin Kernel Version 16.7.0
    Thu Jun 15 17:36:27 PDT 2017
    root:xnu-3789.70.16~2/RELEASE_X86_64 x86_64

=== Enabled user installed addins ===

Internet of Things (IoT) development (Preview) 7.1

Build Logs

[spouliot]

Thanks for the report. This should be already fixed in the current previews (i.e. upgrade to the beta channel).
Duplicate of https://bugzilla.xamarin.com/show_bug.cgi?id=60771

[juripyykko]

I've changed to beta channel on the Mac, but I still get the same error. Do I need to recreate the whole project on the beta channel versions? @spouliot

[spouliot]

@juripyykko make sure you do a clean on your solution before building with the beta. If it still happens attach your test case (clean, zip and drag'n'drop into a comment text box) and re-ope the issue.

[juripyykko]

TestReproduceError.zip

Same problem persists after clean build with the beta.
@spouliot I don't seem to be able to re-open an issue I didn't close, could you do that please?

[timrisi]

@juripyykko I reopened the issue for you. Do you have a test case you can provide?

[juripyykko]

@timrisi Thanks for opening it. I'm unable to build it for the iPhone. That's the test I guess.

[MAlshehri]

Same problem Xamarin Forums

[juripyykko]

@spouliot @timrisi Any update on this? We're getting closer to release and haven't been able to run the app on an iPhone yet. Is someone looking at it? Can I assist in any way?

[aritchie]

Is there any workaround to this issue. It is going to blocking a release for me fairly soon. I have tried the following:
1) latest beta on VS4Mac and from build host on windows
2) System.Runtime.CompilerServices.Unsafe v4.3 (this fixes android side incidentally) and v4.4
3) Remove linking altogether

Nothing is working. An update on any workaround would be fantastic.

[aritchie]

For a test case, just add Microsoft.EntityFrameworkCore.Sqlite - it fails with any project in release.

[spouliot]

@VincentDondain there was a similar discussion (about ref and lib iirc) on another bug. Can you check this up please ?

[VincentDondain]

This issue seems like a duplicate of https://bugzilla.xamarin.com/show_bug.cgi?id=59184. It should be fixed here: https://github.com/mono/mono/pull/6826

[aritchie]

So should this fix already be in the beta channel or VS 15.6p2? If so, I've tried both and I'm being hit with this issue.

[VincentDondain]

Alright looks like I closed this a little too fast.

You hit in this issue the same blocker as https://bugzilla.xamarin.com/show_bug.cgi?id=59184 but it's not the same fix (59184 was about a mono bug: https://github.com/mono/mono/pull/6826 that is totally unrelated).

However the workaround mentioned here: https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c49 should unblock you.

move the System.Runtime.CompilerServices.Unsafe.dll from:
/Users/vidondai/.nuget/packages/system.runtime.compilerservices.unsafe/4.4.0/lib/netstandard2.0/ to -> /Users/vidondai/.nuget/packages/system.runtime.compilerservices.unsafe/4.4.0/ref/netstandard2.0/

We shouldn't pass system.runtime.compilerservices.unsafe/4.4.0/ref/netstandard2.0/System.Runtime.CompilerServices.Unsafe.dll to the mtouch. Nothing can be done if the reference (no IL) is given to the AOT compiler.

Need to investigate a little more to know why we're gettingSystem.Runtime.CompilerServices.Unsafe.dll from the wrong place.

[VincentDondain]

Discussed this with @mrward and as far as we can tell system.runtime.compilerservices.unsafe is the only package (needed for the test case) that has a ref/netstandard2.0 folder.

It seems that the Nuget/msbuild target choses the dll from ref over lib for some reason.
Removing the ref folder entirely is also a way to force nugget to use the dll from lib.

Matt suggested that having a ref/xamarin.ios folder (in addition to the others) with the working dll (the one that can be AOTed) would work.

@spouliot I'm thinking this it more a Nuget / package issue than a Xamarin.iOS and tooling one. What do you think?

[spouliot]

@VincentDondain not sure where the issue is - but we're not getting the right thing from msbuild.

Can you try another type of project (maybe netcore ?), use the nugget and see what msbuild resolves and deploy with the app ?

Thanks!

[mrward]

There are some differences in the build behaviour when a PackageReference is used versus using a packages.config file. PackageReferences will cause msbuild/NuGet to use ref assemblies. Using a packages.config file will result in the lib assemblies being used at compile time. The TestReproduceError.zip is using PackageReferences.

If the project uses PackageReferences then MSBuild/NuGet will use the ref to compile against if it exists and is compatible with the target framework of the project. If you add a PackageReference to System.runtime.compilerservices.unsafe in an Xamarin.iOS project, or .NET Framework project, or .NET Core project, then the project.assets.json file has:

      "System.Runtime.CompilerServices.Unsafe/4.4.0": {
        "type": "package",
        "dependencies": {
          "NETStandard.Library": "1.6.1"
        },
        "compile": {
          "ref/netstandard1.0/System.Runtime.CompilerServices.Unsafe.dll": {}
        },
        "runtime": {
          "lib/netstandard1.0/System.Runtime.CompilerServices.Unsafe.dll": {}
        }
      }

So MSBuild compiles against the assembly in the ref directory. The ref assembly is passed to Csc for all project types. As far as I am aware, from MSBuild's point of view, the correct assembly is being used. It will not compile against the lib if there is an assembly in the ref folder that is compatible with the project's target framework.

About the only thing I have managed to do to affect this is to exclude it using:

<PackageReference Include="System.Runtime.CompilerServices.Unsafe" Version="4.4.0">
        <ExcludeAssets>all</ExcludeAssets>
 </PackageReference>

After restoring with the above PackageReference the project.assets.json file has:

   "System.Runtime.CompilerServices.Unsafe/4.4.0": {
        "type": "package",
        "compile": {
          "ref/netstandard2.0/_._": {}
        },
        "runtime": {
          "lib/netstandard2.0/_._": {}
        }
      }

However presumably this is not much use if any types from the assembly are used. Not sure if there is another way to affect this with some MSBuild properties/targets that could be added to the Xamarin.iOS targets - maybe a question for @radical

One solution would be to copy the lib/netstandard2.0 assembly and add it to a ref/xamarinios directory in the NuGet package, as suggested by Vincent.

If the project uses a packages.config then the lib/netstandard2.0 assembly is compiled against for .NET Framework and Xamarin.iOS.

[VincentDondain]

Seems like https://github.com/Microsoft/msbuild/issues/2776 is identical to this issue.

We diagnosed the same thing as https://github.com/Microsoft/msbuild/issues/2776#issuecomment-358594438

[mrward]

Just wondering if perhaps a better way to fix this would be to have this handled in MSBuild by the iOS targets.

With a .NET Core project, for example, there are two steps: build and publish. The build target compiles against the ref assembly. The publish target will result in the lib assembly being copied to the publish directory so it can be used at runtime. Not sure if this fits with how the iOS MSBuild targets work. If they are expecting a non-ref assembly at compile time then perhaps there is a way to run some sort of target, after or before NuGet has done or is doing its assembly resolution, and switch to the lib when compiling an iOS project.

[VincentDondain]

As @mrward described above .NET Core projects compile against the ref assembly and use the lib assembly when publishing.

It's also important to note that at runtime the lib assembly isn't _copied_ inside the app (for a .NET Core console project for instance) but there is a file called MyApp.deps.json that contains:

"System.Runtime.CompilerServices.Unsafe/4.4.0": {
        "runtime": {
          "lib/netstandard2.0/System.Runtime.CompilerServices.Unsafe.dll": {}
        }
},

so it knows where to find the lib assembly at runtime.

Dean has been working on a fix for Xamarin.Android, see: https://github.com/xamarin/xamarin-android/pull/1356 and it sounds to me that Xamarin.iOS needs a similar fix.
For net standard packages we need a mechanism to pass the path to the lib assembly to mtouch so it can be AOTed.

[dellis1972]

@VincentDondain that PR xamarin/xamarin-android#1356 has been updated to use the Nuget.ProjectModel. There is also a test task I wrote to replace ref assemblies with lib ones. Might give you some ideas.

[1] https://gist.github.com/dellis1972/b1f8838c52200c1fa7e61f7ad8bd268f

[VincentDondain]

@dellis1972 awesome thanks, I'm definitely taking inspiration from your PR (:

[gmarbury]

Not sure I know how to fix this. I can't compile IOS on mac but can compile on Windows

[ryl]

Is there a recommended workaround to use on the iOS side in the meantime?

[juripyykko]

The workaround mentioned above worked for me:

move the System.Runtime.CompilerServices.Unsafe.dll from:
/Users/vidondai/.nuget/packages/system.runtime.compilerservices.unsafe/4.4.0/lib/netstandard2.0/ to -> /Users/vidondai/.nuget/packages/system.runtime.compilerservices.unsafe/4.4.0/ref/netstandard2.0/

[VincentDondain]

Fixed in https://github.com/xamarin/xamarin-macios/commit/e297e99378868e219c88a23a04e25388ce194978