Efcore: Xamarin iOS : Crash when i add row to table with nullable type or with a lot of columns

[NitrofCG]

Update

According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c57, the more complete fix in Mono 2017-12 is now merged and has been verified on Xamarin.iOS.

According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c58, the fix should be included in the 15.7 release of Xamarin, which I believe maps to Xamarin.iOS 11.9 (at least I found that on https://github.com/xamarin/xamarin-macios/wiki).

Original issue

EntityFramework crash when i insert a row inside table. it crash only on iOS Device (iPod Touch and iPad mini 4).

I have try successfully onAndroid Device and iOS Simulator.
But when i use on iOS device, it fall to Xamarin iOS limitation.

First way : when i have a nullable type inside TrainingDayRow table (exemple : int? unit)

Exception message: 
Stack trace:

{System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_int_int_int_int_DateTime_DateTime_DateTime_Nullable`1<int> (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
at (wrapper managed-to-native) System.Reflection.MonoCMethod:InternalInvoke (System.Reflection.MonoCMethod,object,object[],System.Exception&)
at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.12.0.14/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:661 
 --- End of stack trace from previous location where exception was thrown ---

Second way : Have a table with a lot of column (here on TrainingExerciseRow with 14 columns)

Exception message: 
Stack trace:

{System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) <Module>:runtime_invoke_void__this___object_int_int_int_int_DateTime_DateTime_DateTime_int (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
at (wrapper managed-to-native) System.Reflection.MonoCMethod:InternalInvoke (System.Reflection.MonoCMethod,object,object[],System.Exception&)
 at System.Reflection.MonoCMethod.InternalInvoke (System.Object obj, System.Object[] parameters) [0x00002] in /Library/Frameworks/Xamarin.iOS.framework/Versions/10.12.0.14/src/mono/mcs/class/corlib/System.Reflection/MonoMethod.cs:661 
--- End of stack trace from previous location where exception was thrown ---

Steps to reproduce

Clone my Github source and just click to "+".

Source code
https://github.com/Thetyne/XamarinTodo

Further technical details

EntityFrameworkCore 2.0.0-preview2-final
NetStandard 2.0.0.0-preview2-25401-01
Database Provider: Microsoft.EntityFrameworkCore.Sqlite
Operating system: iOS/Macosx
IDE: Visual Studio Community 2017 for Mac Version 7.1 Preview (7.1 build 1281)

[divega]

@Thetyne thanks for reporting this.

I have created an issue for this for Xamarin iOS at https://bugzilla.xamarin.com/show_bug.cgi?id=58424 (this issue has been resolved as a duplicate of https://bugzilla.xamarin.com/show_bug.cgi?id=59184). Hopefully we will figure out if this is something that can be addressed by either an improvement in Xamarin, a chance in EF Core or a workaround in the application code.

cc @marek-safar as he has helped with similar issues in the past.

[divega]

Note for triage: This is due to a limitation in the number of parameters for methods that are invoked dynamically in the implementation of Xamario.iOS.

From @vargaz on the Xamarin issue:

I can reproduce, it tries to do a method invoke on this method:
"Microsoft.EntityFrameworkCore.ChangeTracking.Internal.Snapshot`20:.ctor (int,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string,string)"

xamarin.ios currently has some limits on what methods can be dynamically invoked, this method has too arguments.

The limit for parameters is about 14.

It will take some time to fix this since it is low level code and it needs to work on multiple platforms. It might be better to try to rewrite the c# code to use less arguments.

So, it seems that using a smaller value for Snapshot.MaxGenericTypes (about 14 instead of 30) could help run EF Core in Xamarin.iOS with larger entities.

More details about the limitation:

On arm64, its 8+6=14, or arm its 4+10=14. This includes the this pointer as well.

[ajcvickers]

We will add a quirk mode to workaround this issue until it is fixed in Xamarin.iOS.

[ns-wgilmour]

For anyone here looking for a quick fix on iOS, I managed to get my project working by pulling EFCore, changing Snapshot.MaxGenericTypes to 2 (I tried 14 and 5, but they didn't work, which may just be my model), building and packaging it, and using the new NuGet packages in my project.

[ajcvickers]

@ns-wgilmour When you changed MaxGenericTypes to 14 and 5, did you still see the same failure as reported above? Can you post the exception message and stack trace if there is one?

@divega It may not be feasible to go all the way down to 2 generic types. Also, at this level there may be other generics we create that have similar problems.

[ns-wgilmour]

@ajcvickers I did, although I only tested it on my current project rather than with the test project I created here to showcase the errors (I catch the errors in that test project, so don't expect it to throw). My current project would crash when trying to do anything related to a particular model with 3 optional relations in it, the keys for which are represented in the model as 3 Nullable<long>. The model has several other properties that bring its total count to 12 (that are stored in the database).

I think there is slightly more to this problem than meets the eye, or perhaps a misunderstanding of it. @divega, when commenting on the Xamarin issue, said that up to 30 generic arguments could be used, as specified by Snapshot.MaxGenericTypes, and when I first saw that, I thought it referred to the total number of properties on the model, but that did not seem to be the case with my model of 12 properties, as setting the value to 14 and 5 seemed to have no effect. I then thought it might refer to the number of properties on the model that were generically-typed, such as Nullable<>, and set it to a number that was below the number of generic types my problem model had, in this case 2, and it then worked.

Also, whilst that particular case seems to be related to generic types, in my test project there is a model with roughly 20 string properties, no Nullables or generic types, and that also fails with the same error.

I'm not sure if the circumstances are just coincidental but to me there seems to be two different causes. I am unfortunately swamped at the moment so haven't really had time to properly investigate with my test project. I must also confess I know very little about how all this reflection and dynamic invoking generically typed methods works, so my apologies if what I am saying seems nonsensical, I am just trying to put down my observations and thought process.

[divega]

I agree we don't understand the limitation in Xamarin.iOS well enough.

According to @vargaz it is about the number of arguments on a method that is invoked dynamically. The original repro hit this on the constructor of one of the Snapshot<T,...Tn> types. I.e. it is only indirectly related to the number of generic arguments, and there are other ways to hit it.

Hi hope @vargaz has more suggestions here, or information on when the limitation could be lifted.

[divega]

@ns-wgilmour maybe you can help us by providing a new repro here and in the Xamarin issue at https://bugzilla.xamarin.com/show_bug.cgi?id=58424.

[vargaz]

The limitation is on the number of arguments to the method which is invoked dynamically. There is no need for another testcase right now.

[divega]

@vargaz thanks for responding. We are trying to understand why (according to https://github.com/aspnet/EntityFrameworkCore/issues/9249#issuecomment-329648999), limiting the number of generic arguments to 14, or even 5 doesn't help.

[divega]

Update: https://github.com/mono/mono/pull/5612 will fix this for ARM64 devices in an upcoming release of Xamarin.iOS.

[MasanoriOnuki]

I tested dynamic invocation on iOS devices.

Test code is here.

The number of arguments that doesn't throw exception is 3 on ARM64 or 1 on ARMv7.
If the number of arguments is over 13, always throw exception.

I think SnapShot instance itself is one of the arguments. so 4-1=3 and 14-1=13.
I think whether dynamic invocation throws an exception depends on the size of the argument type.

To quick fix this problem for ARM64, SnapShot.MaxGenericTypes is set to 3.

[divega]

The fixes for this issue have been incorporated in Mono. Not sure what builds of Xamarin will contain it but they are in https://github.com/mono/mono/tree/2017-10.

I believe there is nothing actionable on the EF Core side other than validating the fix.

Thanks @vargaz!

[divega]

Removed the area-external label to repurpose the issue just to validate the fix on our side.

[mikaelkolkinn]

I would just like to add to @MasanoriOnuki's response that iPhone 5S can give exceptions with Snapshot.MaxGenericTypes set to 3. Snapshot.MaxGenericTypes = 2 seems to work well.

Just in case anyone has to build EntityFrameworkCore from the sources and set the Snapshot.MaxGenericTypes to a lower value on iOS to work around the issue until the Mono update is part of a new Xamarin release.

On a sidenote, such a workaround would be much easier to implement for others if Snapshot.MaxGenericTypes was public static int instead of public const int.

[Kilowhisky]

FYI still appears to not be present in the latest Xamarin or EF Core. Fun times!

Core v2.0.1
Forms v2.5.0.122203

I was able to use it by doing the MaxGenericTypes = 3 fix. Very bad hack though as the repo tags don't match up to the releases on nuget.org so i had to take my best guess as to where 2.0.1 actually was. I had to use a localPackages nuget.config in order to get it to work too. I'm up and running but it is uuuuuuuggggggllllyyyy.

[NitrofCG]

Me when i check this issue date : 👎.
Why can't you fix this critical to your current mono branch?

Nobody can't use Entity Framework on iOS ...

[Reshiru]

@Thetyne seems so^^ I'm not 100% sure but is this also related to this issue?

https://stackoverflow.com/questions/48600391/xamarin-forms-sqlite-code-first-add-element-to-an-foreign-element-with-list-of-t

I just can't use my app on an apple device because of this... and I couldn't even use any workaround...

If anyone could help, would be great. Thanks^^

[NitrofCG]

@Reshiru Yes, I think it's the same issue.

I gave up the idea that i use EF Core in my mobile apps :(

[Reshiru]

@Thetyne Well^^ I have to keep it currently (also i can't find any solid alternative) and I'm mostly used to EF :/ ) But as I heard this issue is only caused on Ad-Hoc Debug mode. I didn't try out to publish the App at the moment (Need to wait until I have In-House release possibilities) but as I heard after a release it should work. I hope for the best c: ^^

[Reshiru]

@Kilowhisky and @mikaelkolkinn hey there. Where did you exactly set this? In the database context? Did you override any method?

[mikaelkolkinn]

@Reshiru What I did was to clone/fork this repository (https://github.com/aspnet/EntityFrameworkCore) to a repository of my own and add it as a git submodule to my main git repository. I then added the EFCore projects I needed to my app's solution, which in my case were EFCore, EFCore.Relational and EFCore.Sqlite.Core. In order to build the projects I also had to add the Nuget repositories in the NuGet.config file found in the EntityFrameworkCore submodule, one can simply tell NuGet to use this config file.

Once my solution could build successfully, I converted Microsoft.EntityFrameworkCore.ChangeTracking.Internal.Snapshot.MaxGenericTypes found in the EFCoreproject (https://github.com/aspnet/EntityFrameworkCore/blob/dev/src/EFCore/ChangeTracking/Internal/Snapshot.cs), from a public const int to a public static int. This way I can change the value from outside the EFCore projects and only modify it when running on iOS.

Right now I have this simple snippet in the constructor of my DbContext class:

if (Device.RuntimePlatform == Device.iOS)
{
      Snapshot.MaxGenericTypes = 2;
}

[msangtarash]

@ajcvickers Could you please change Snapshot.MaxGenericTypes from const to static? Or may we know the idea behind this? Why you don't use static readonly for example?

[Reshiru]

@mikaelkolkinn Thanks, I tried to do it like you described but somehow I can't AOT the implemented EFCore Projects. I tried (mainly) this methods and some abstractions:

1. Change and compile the project in the given solution (EFCore.Runtime), Results: Could not AOT the assembly 'Path/....dll'
2. Import needed projects (same as you had) in my main project, added the NuGet packages and tried to recompile it. Results: NotNullAttribute not compilable because of the class type.

Would be great if you could send me somehow the compiled dlls with the issue fix (workaround).

[ajcvickers]

@msangtarash It is not intended that the value for MaxGenericTypes be changed. Setting it to pathologically low numbers also largely defeats the purpose of the design. Also, there are likely other places that will fail for similar reasons if the underlying bug is not fixed, so I think there needs to be a very strong case to change this as oppose to getting the underlying bug fixed.

[divega]

Update: According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c57, the more complete fix in Mono 2017-12 is now merged and has been verified on Xamarin.iOS.

According to https://bugzilla.xamarin.com/show_bug.cgi?id=59184#c58, the fix should be included in the 15.7 release of Xamarin, which I believe maps to Xamarin.iOS 11.9 (at least I found that on https://github.com/xamarin/xamarin-macios/wiki).

I am not sure about the timing of that release. @marek-safar do you have any schedule information you can share?

[marek-safar]

15.7 preview 1 should go out today with the final version (RTW) to be somewhere around 10th of May

[ZohreMoradi]

@marek-safar I'm facing the same issue with vs 2017.7 prev 2 + xam.ios 11.10.0.32
Is it fixed or not?

[aritchie]

@marek-safar This was fixed in one of the earlier 15.7 betas, but seems to be rebroken in the latest builds. Xamarin.iOS v11.10.0.50

[marek-safar]

That's interesting. Are you sure it's the same failure?

@brendanzagaeski could you help us to verify this?

[aritchie]

It appears to be. I use long & DateTimeOffset with nullables in this one particular model. I switched everything to int & datetime, now everything works. I'll setup a sample for you hopefully in the next couple of days.

[ajcvickers]

@divega Can this be moved out of the 2.1 milestone. (We are trying to make 2.1 only include issues that we must fix for 2.1)

[aritchie]

If you guys open up Snapshot.MaxGenericTypes to allow external setting, this issue can be moved entirely to the xamarin iOS team. It is framework bug with them, not with EFCore.

[divega]

As @ajcvickers already explained in https://github.com/aspnet/EntityFrameworkCore/issues/9249#issuecomment-370515031 we won’t be making changes for this. We only kept this open for verification purposes, but we have at least one report indicating that this may not be fixed in the 2.1 timeframe.

@aritchie I recommend you follow up in the Xamarin issue tracker. I will move this to our backlog for now.

[ghost]

I tried today with Visual Studio 15.7.2, Xamarin.Forms 3.0.0.482510, Xamarin.iOS 11.10.1.178 - still no luck...

[0:] System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> System.ExecutionEngineException: Attempting to JIT compile method '(wrapper runtime-invoke) :runtime_invoke_void__this___Guid_Nullable1<Guid>_object_object_double_int_single_object_object_object_object_Nullable1_object_DateTime_Nullable`1 (object,intptr,intptr,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.

;-(

[Juansero29]

Got this problem as well when using a Devart.SQLite ADO library. To be exact:

````
Unhandled Exception:

System.ExecutionEngineException: Attempting to JIT compile method '(wrapper native-to-managed)   : (intptr,int,intptr)' while running in aot-only mode. See https://developer.xamarin.com/guides/ios/advanced_topics/limitations/ for more information.
occurred
````
This error only comes when I test my app on a physical device though. On emulators everything works like a charm.

[vargaz]

Thats a different problem.

[Kelsch]

Still seeing this issue on visual studio version 15.8.1 with xamarin 4.11.0.732 and xamarin.ios 11.14.0.13

[cfdelaune]

Like @Kelsch, I am still seeing the problem with the latest VS and Xamarin versions. I am using EF Core 2.1.2, and would be willing to try the EF Core 2.2 preview, but at this point, it is unclear to me where the problem resides. Is this a VS issue, Xamarin.iOS issue, or an EF Core issue? Until today, I have not been able to test because I kept running a compile error (described here) that prevented me from running on a physical iOS device. It appears that was resolved with VS 15.8.x.

Any information is much appreciated!

[divega]

@Kelsch @cfdelaune I am sorry to hear that this isn't fixed for you in recent releases of Xamarin. If you are seeing the exact same symptoms described here, it is mostly likely still a Xamarin issue. I know that the Xamarin team has made several fixes trying to address this, but perhaps there is one more case that they need to look at.

If you haven't already, please report it by creating a new issue on Xamarin's new issue tracker at https://github.com/xamarin/xamarin-macios/issues. You can mention that you believe this is similar to https://bugzilla.xamarin.com/show_bug.cgi?id=58424 and https://bugzilla.xamarin.com/show_bug.cgi?id=59184, but please also include a repro project so that they can investigate why the fixes already included in the product aren't helping.

Also, feel free to @ mention me in the issue so I can follow up with the Xamarin team.

cc @marek-safar, @vargaz , @mandel-macaque

[vargaz]

Also, could you paste the _exact_ error message you are getting, i.e. the {System.ExecutionEngineException: Attempting to JIT compile method
message.

[cfdelaune]

Thank you for following up. I will post a new issue asap. For now, here is a screenshot of the error I am getting on an iPhone 7 when attempting to add a record to a table with the the following column types (8 total):

• Guid
• string
• DateTime
• Guid?
• DateTime
• DateTime?
• int?
• string

[cfdelaune]

After digging in a bit more, I have been able to isolate that the problem only occurs when there is a column of type Nullable Guid (Guid?). With this configuration I was able to get it working on a physical device with a table consisting of 20+ columns and including Nullable DateTime? and int? types, but throwing a Guid? column into the mix causes this error.

As @divega suggests, I will submit this as a new issue. In the meantime if anyone has ideas for workarounds, I would love to hear them.

[cfdelaune]

I just submitted this as a new issue here.