Efcore: Xamarin Android: Exceptions thrown if SDK Assemblies are linked, after updating all NuGet-Packages

[daniels7]

If I build a Xamarin Android App with linked SDK Assemblies it will crash after EF Core is Updated to newest version.

Exception message:
Stack trace:
06-25 16:48:31.244  9279  9279 I MonoDroid: UNHANDLED EXCEPTION:
06-25 16:48:31.257  9279  9279 I MonoDroid: System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator' threw an exception. ---> System.ArgumentException: An item with the same key has already been added. Key: Int32 Abs(Int32)
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Collections.Generic.Dictionary`2[TKey,TValue].TryInsert (TKey key, TValue value, System.Collections.Generic.InsertionBehavior behavior) <0xceeb8b14 + 0x0045c> in <c099d544051e40b89a67a87a43581f01>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Collections.Generic.Dictionary`2[TKey,TValue].Add (TKey key, TValue value) <0xceeb7fd0 + 0x00013> in <c099d544051e40b89a67a87a43581f01>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator..cctor () <0xcd753afc + 0x00207> in <135a24207446461281bfc2c90f71e590>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:    --- End of inner exception stack trace ---
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.RelationalCompositeMethodCallTranslator+<>c__DisplayClass5_0.<Translate>b__0 (Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.IMethodCallTranslator translator) <0xcd61f2dc + 0x00044> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable+SelectListIterator`2[TSource,TResult].MoveNext () <0xce42d194 + 0x000ff> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable.TryGetFirst[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate, System.Boolean& found) <0xce415f5c + 0x000f3> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Enumerable.FirstOrDefault[TSource] (System.Collections.Generic.IEnumerable`1[T] source, System.Func`2[T,TResult] predicate) <0xce415c6c + 0x0003f> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionTranslators.RelationalCompositeMethodCallTranslator.Translate (System.Linq.Expressions.MethodCallExpression methodCallExpression, Microsoft.EntityFrameworkCore.Metadata.IModel model) <0xcd61ef6c + 0x002bf> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.VisitMethodCall (System.Linq.Expressions.MethodCallExpression methodCallExpression) <0xcd614aac + 0x00393> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Expressions.MethodCallExpression.Accept (System.Linq.Expressions.ExpressionVisitor visitor) <0xce4a5000 + 0x0001b> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Expressions.ExpressionVisitor.Visit (System.Linq.Expressions.Expression node) <0xce49f890 + 0x00027> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.Parsing.ThrowingExpressionVisitor.Visit (System.Linq.Expressions.Expression expression) <0xce62b070 + 0x0006b> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.ExpressionVisitors.SqlTranslatingExpressionVisitor.Visit (System.Linq.Expressions.Expression expression) <0xcd613b00 + 0x0010f> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitWhereClause (Remotion.Linq.Clauses.WhereClause whereClause, Remotion.Linq.QueryModel queryModel, System.Int32 index) <0xcd5f7064 + 0x000ef> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.Clauses.WhereClause.Accept (Remotion.Linq.IQueryModelVisitor visitor, Remotion.Linq.QueryModel queryModel, System.Int32 index) <0xce63c7cc + 0x00093> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.QueryModelVisitorBase.VisitBodyClauses (System.Collections.ObjectModel.ObservableCollection`1[T] bodyClauses, Remotion.Linq.QueryModel queryModel) <0xce62e964 + 0x00167> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Remotion.Linq.QueryModelVisitorBase.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xce62e468 + 0x0006b> in <1081ac98085b4c0b96c42f3872e13655>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xce6e2e08 + 0x00047> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.RelationalQueryModelVisitor.VisitQueryModel (Remotion.Linq.QueryModel queryModel) <0xcd5f5da0 + 0x0005f> in <b4c4c93ef655443999d2211e0c2e61cd>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.EntityQueryModelVisitor.CreateQueryExecutor[TResult] (Remotion.Linq.QueryModel queryModel) <0xce6e0fcc + 0x00177> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Storage.Database.CompileQuery[TResult] (Remotion.Linq.QueryModel queryModel) <0xce6c8964 + 0x000bb> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.CompileQueryCore[TResult] (System.Linq.Expressions.Expression query, Microsoft.EntityFrameworkCore.Query.Internal.IQueryModelGenerator queryModelGenerator, Microsoft.EntityFrameworkCore.Storage.IDatabase database, Microsoft.EntityFrameworkCore.Diagnostics.IDiagnosticsLogger`1[TLoggerCategory] logger, System.Type contextType) <0xce70f75c + 0x0028f> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler+<>c__DisplayClass13_0`1[TResult].<Execute>b__0 () <0xce71037c + 0x0005b> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQueryCore[TFunc] (System.Object cacheKey, System.Func`1[TResult] compiler) <0xce6f70ac + 0x000c3> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.CompiledQueryCache.GetOrAddQuery[TResult] (System.Object cacheKey, System.Func`1[TResult] compiler) <0xce6f7024 + 0x0003b> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.QueryCompiler.Execute[TResult] (System.Linq.Expressions.Expression query) <0xce70f47c + 0x001a7> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at Microsoft.EntityFrameworkCore.Query.Internal.EntityQueryProvider.Execute[TResult] (System.Linq.Expressions.Expression expression) <0xce6f7b58 + 0x00047> in <1a963d6bdb8d435e8dcdc28935e050ad>:0
06-25 16:48:31.257  9279  9279 I MonoDroid:   at System.Linq.Queryable.FirstOrDefault[TSource] (System.Linq.IQueryable`1[T] source) <0xce4096f4 + 0x000cb> in <dd44b4af145c4bd6a0a60da9e4b5131f>:0

Steps to reproduce

I'll try to append a sample project tomorrow, though I somehow have the thought this might be somehow related to Issue #10207 ?

Does anyone have an idea on how to modify the linking behaviour so that the Application will work again? Without linking the resulting apk sadly is 3 times as big as usual.

Further technical details

EF Core version: (found in project.csproj or packages.config) 2.1.1
Database Provider: (e.g. Microsoft.EntityFrameworkCore.SqlServer) Microsoft.EntityFrameworkCore.Sqlite
Operating system: Windows 10 Pro x64
IDE: (e.g. Visual Studio 2017 15.4) VS 2017 15.7.4

[marfh]

I have the same issue, works fine in 2.0.3 but not 2.1.0 or 2.1.1.

[ajcvickers]

@daniels7 @dwaskel It would be great if you could post a sample project/solution that demonstrates this.

@cwrea Does this look like something you have run into?

[mfeingol]

I'm seeing the same thing. I don't have a simple repro on hand, but willing to provide access to a private VSTS repo to the right person.

[BarretHTW]

I have this problem as well, resulting in a too big apk...

[daniels7]

@ajcvickers

Issue12460.zip

Here is a sample project as requested. Just build it in Release-Mode to install on an Android-Phone and it crashes instantly.

Please fix this asap, thank you very much

[michaelrinderle]

im running into the same issue. release apk for google play ends up at ~90mb. eek!

proposed solutions didnt work for me, i.e. linkdescription.txt, --skip-link, etc

[daniels7]

@ajcvickers
Any reaction from you or any other member of the team would be very nice.
This Bug is critical and fucks up every new release of our App because it is three times as big as it should be.

Come on, any reaction....

It's been 2 days since you got a sample project by me and more than two weeks since the Bug has been reported by multiple users.

[BarretHTW]

???

[divega]

Everyone, sorry for the delay in answering. I have assigned this to @smitpatel and myself. I would like to ask from the people that repro if just running this code throws if you copy it inside your app:

``` C#
var supportedMethods = new Dictionary
{
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(double) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(float) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(int) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(long) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(sbyte) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(short) }), "abs" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(byte), typeof(byte) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(double), typeof(double) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(float), typeof(float) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(int), typeof(int) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(long), typeof(long) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(sbyte), typeof(sbyte) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(short), typeof(short) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(uint), typeof(uint) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Max), new[] { typeof(ushort), typeof(ushort) }), "max" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(byte), typeof(byte) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(double), typeof(double) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(float), typeof(float) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(int), typeof(int) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(long), typeof(long) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(sbyte), typeof(sbyte) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(short), typeof(short) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(uint), typeof(uint) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Min), new[] { typeof(ushort), typeof(ushort) }), "min" },
{ typeof(Math).GetMethod(nameof(Math.Round), new[] { typeof(double) }), "round" },
{ typeof(Math).GetMethod(nameof(Math.Round), new[] { typeof(double), typeof(int) }), "round" }
};

This is a copy of the code from which the exception is coming. It is creating a dictionary where the key is a MethodInfo. Since it is saying that Abs(int32) is a duplicate key, that means that either `typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(double) })` or `typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(float) })` returned the same method info as `typeof(Math).GetMethod(nameof(Math.Abs), new[] { typeof(int) })`, which seems very unexpected. 

Even if the linker is removing some members, that doesn't seem to be an excuse for reflection to start doing fuzzy matching. That said, I am not super familiar with the technology and there could be a reason for this. In any case, I would say that part is an issue in Xamarin/Mono and I would be grateful if you can report it at https://github.com/xamarin/xamarin-android/issues.

Re how to workaround this, I would guess that using the options in the linker to skip linking certain assemblies. E.g. as documented in https://docs.microsoft.com/en-us/xamarin/ios/deploy-test/linker?tabs=vsmac, you could either use:

--linkskip=NameOfAssemblyToSkipWithoutFileExtension

Or a custom link configuration as described at https://docs.microsoft.com/en-us/xamarin/cross-platform/deploy-test/linker. 

I am not sure what assembly/assemblies you should be excluding to make sure all overoads of Math.Abs are preserved. It could be mscorlib, in which case your LinkDescription.xml file woudl look something like this (I am basing this on @cwrea's sample app [LinkDescription.xml file](https://github.com/cwrea/XamarinTodo/blob/master/Todo.Android/LinkDescription.xml)):

``` xml
  <assembly fullname="mscorlib">
    <type fullname="System.String">
      <method name="Compare"></method>
      <method name="CompareTo"></method>
      <method name="ToUpper"></method>
      <method name="ToLower"></method>
    </type>
     <type fullname="System.Math">
      <method name="Abs"></method>
      <method name="Max"></method>
      <method name="Min"></method>
      <method name="Round"></method>
    </type>
 </assembly>
  <assembly fullname="System.Core">
    <type fullname="System.Linq.Expressions.Expression`1"></type>
    <type fullname="System.Linq.Queryable"></type>
  </assembly>
</linker>

Also, I haven't tried this workaround myself on Xamarin Android, so I am not sure with just specifying the names of the methods in the file is enough to preserve all overloads. Worst case, you may need to specify all the signatures (which is also documented in the previous link).

[divega]

@smitpatel could you try to repro this and try the workaround when you have a chance?

[michaelrinderle]

these solutions don't work. trying to skip assemblies is the worst, you keep compiling, marking an assembly/namespace in ef core until you run up against an assembly that just work link correctly and you're stuck after hours of work for nothing.

as i recall xamarin pushed this back on ef core team since they have to mark reflection that needs to be kept in source so this doesnt happen.

ef core and xamarin are a ms flagship now, and you can't link with the ef assm and end up with a bloated apk on top of the mono framework because of it? this kind of stinks!

[divega]

@ainsophical just to double-check, when you say that these solution's don't work, do you mean that you don't think this is the right long term solution or that you tried the workaround and it really didn't work?

FWIW, we also don't think that this is the ideal long term solution and that is why https://github.com/aspnet/EntityFrameworkCore/issues/10963 is still open. However all the proposals from Xamarin and experts in that issue are about moving the workarounds to EF Core and potentially all its dependencies. For example, we would ship our own LinkDescription.xml and/or annotate members in EF Core, Microsoft.Extensions.* and .NET Core BCL with PreserveAttribute.

If the workaround I suggested doesn't work at all, that is very important to know because it means that this particular issue is caused by a different problem and moving the workarounds into EF Core and its dependencies wouldn't help.

[michaelrinderle]

i've tried everything suggested on the topic from multiple sources including the above. I included my own linkdescription.xml file and started the compile process to see which assem. where getting stripped out during the linking process; the conclusion was that at a certain point i couldnt get ef core to work on my xam. app with linking (sdk / sdk & user assem.) since I was getting an exception for a reference to an assembly despite it being on my xml file. i dont recall the referenced assembly at the moment but it was in the ef namespace.

[stddd]

I rolled back to 2.03 and work fine with <AndroidLinkMode>SdkOnly</AndroidLinkMode>
->
<PackageReference Include="Microsoft.EntityFrameworkCore"> <Version>2.0.3</Version> </PackageReference> <PackageReference Include="Microsoft.EntityFrameworkCore.Sqlite"> <Version>2.0.3</Version> </PackageReference>

Waiting for the fix now !

[divega]

I found that the issue hasn't been reported to Xamarin so I created https://github.com/xamarin/xamarin-android/issues/1990. I would suggest everyone that reported the issue here to track that one.

[divega]

From https://github.com/xamarin/xamarin-android/issues/1990#issuecomment-408250461

_@JonDouglas commented_
I can confirm the behavior outlined in the reproduction steps on Xamarin.Android 8.3.3.2:

07-26 15:56:50.903 29686-29686/? E/AndroidRuntime: FATAL EXCEPTION: main
   Process: Issue12460.Issue12460, PID: 29686
   android.runtime.JavaProxyThrowable: System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteCompositeMethodCallTranslator' threw an exception. ---> System.TypeInitializationException: The type initializer for 'Microsoft.EntityFrameworkCore.Sqlite.Query.ExpressionTranslators.Internal.SqliteMathTranslator' threw an exception. ---> System.ArgumentException: An item with the same key has already been added. Key: Double Abs(Double)

However, the behavior does not exhibit on Xamarin.Android 9.0.0.15. Perhaps the bump to Mono 5.12 resolved this issue upstream @jonpryor ?
@divega Have you tried 15.8 Previews(4 or 5) with this issue? I can't seem to reproduce there.

@daniels7 @dwaskel @mfeingol @BarretHTW @stddd @ainsophical and anyone that has hit the issue, could you please answer @JonDouglas's question on whether this repros for you using 15.8 Previews 4 or 5?

[divega]

Per the previous comment, upgrading to Xamarin.Android 9 fixes this issue. In the meanwhile, downgrading to a previous build of EF Core can serve as a workaround.

Closing the issue, since there doesn't seem to be anything actionable on the EF Core side.

[mister-giga]

@divega I am targeting Android 9, but this error still appears for me.

[michaelrinderle]

what IDE and version are you using? this problem went away for me a long time ago.
more importantly, what version of EF core are you on?

[mister-giga]

@ainsophical I am using visual studio 2019 for mac. I was using 2.2.4 version of packages.
I am stuck with version 2.0.3 for months already.

[divega]

@mister-giga just in case it helps, there was a very similar exception reported becasue the linker removes System.Date. The workaround is described at https://github.com/xamarin/xamarin-android/issues/2620#issuecomment-456447322.