Xamarin-android: [Bug] Crash with Android 5.x and x64 devices

[stevecluk]

Description

I have an app that's been in the Play Store for a few years now. It was recently updated with the latest nugets and Forms, and the Pre-Launch report started showing crashes in the Huawei p8 Lite.
The crash only seems to happen on x64 devices (and emulators) that are running Android 5.0 or 5.1. Android 5.x 32-bit devices, and all 6+ devices are fine.

The main crash I was seeing was a result of trying to check if a folder exists from the MainActivity's OnCreate(), showing a Dll issue:

android.runtime.JavaProxyThrowable: System.DllNotFoundException: libmono-native.so assembly:<unknown assembly> type:<unknown type> member:(null)
  at (wrapper managed-to-native) Interop+Sys.Stat(byte&,Interop/Sys/FileStatus&)
  at Interop+Sys.Stat (System.ReadOnlySpan`1[T] path, Interop+Sys+FileStatus& output) [0x00028] in <ac2cc6a1f7c64bf5b08e0682cda293fb>:0 
  at System.IO.FileSystem.FileExists (System.ReadOnlySpan`1[T] fullPath, System.Int32 fileType, Interop+ErrorInfo& errorInfo) [0x00007] in <ac2cc6a1f7c64bf5b08e0682cda293fb>:0 
  at System.IO.FileSystem.DirectoryExists (System.ReadOnlySpan`1[T] fullPath, Interop+ErrorInfo& errorInfo) [0x00000] in <ac2cc6a1f7c64bf5b08e0682cda293fb>:0 
  at System.IO.FileSystem.DirectoryExists (System.ReadOnlySpan`1[T] fullPath) [0x00000] in <ac2cc6a1f7c64bf5b08e0682cda293fb>:0 
  at System.IO.Directory.Exists (System.String path) [0x0001e] in <ac2cc6a1f7c64bf5b08e0682cda293fb>:0 

To reproduce the problem, I'd created an empty Xamarin Forms app using the latest Visual Studio, and tried to run it (debug or release) on the Android x64 emulator, which also crashed on startup but with a different library error this time.
As with the Interop issue, this crash only happens on 64-bit builds of 5.0/5.1 emulator (and devices), where as 32-bit builds work fine. Android 6+ devices run either x86/x64 fine.

05-08 11:55:05.178: E/AndroidRuntime(7531): Process: com.companyname.crashtest, PID: 7531
05-08 11:55:05.178: E/AndroidRuntime(7531): java.lang.UnsatisfiedLinkError: dlopen failed: could not load library "libmonosgen-2.0.so" needed by "/data/app/com.companyname.crashtest-1/lib/x86_64/libmonodroid.so"; caused by library "libmonosgen-2.0.so" not found
05-08 11:55:05.178: E/AndroidRuntime(7531):     at java.lang.Runtime.loadLibrary(Runtime.java:371)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at java.lang.System.loadLibrary(System.java:989)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at mono.MonoPackageManager.LoadApplication(MonoPackageManager.java:80)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at mono.MonoRuntimeProvider.attachInfo(MonoRuntimeProvider.java:48)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread.installProvider(ActivityThread.java:4964)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread.installContentProviders(ActivityThread.java:4559)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4499)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread.access$1500(ActivityThread.java:144)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1339)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.os.Handler.dispatchMessage(Handler.java:102)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.os.Looper.loop(Looper.java:135)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at android.app.ActivityThread.main(ActivityThread.java:5221)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at java.lang.reflect.Method.invoke(Native Method)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at java.lang.reflect.Method.invoke(Method.java:372)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:899)
05-08 11:55:05.178: E/AndroidRuntime(7531):     at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:694)

I had also tried updating to the latest Forms 4.6.0.726, and Xamarin.Essentials 1.5.3.1, but still crashes.

Steps to Reproduce

1. Create a new Xamarin.Forms project (or use the zipped template project provided)
2. Build and deploy as-is on an Android 5.x 64-bit emulator (or Huawei P8 Lite device)

Expected Behavior

The app doesn't crash.

Actual Behavior

The app crashes complaining about libraries

Basic Information

• Version with issue:
• Last known good version:
• IDE: Visual Studio 2019 v16.5.4
• Platform Target Frameworks:
  
  
  
  • iOS: Xamarin.iOS 13.16.0.13
  
  
  • Android: Xamarin.Android 10.2.0.100
  
  
• Android Support Library Version: N/A
• Nuget Packages:
  NETStandard.Library v2.0.3
  Xamarin.Forms 4.6.0.726
  Xamarin.Essentials 1.5.3.1
• Affected Devices: 64-bit Android 5.0 or 5.1 emulators (or Huawei P8 Lite device at least)

Screenshots

N/A

Reproduction Link

N/A

Workaround

N/A

[stevecluk]

Adding a dropbox link to the test project

https://www.dropbox.com/s/7l8ci0s61r556jy/CrashTest.zip?dl=0

[PureWeen]

Time    Device Name Type    PID Tag Message
    pixel_2_lollipop_5_0_-_api_21_x64   Error   3987    AndroidRuntime  java.lang.UnsatisfiedLinkError: dlopen failed: could not load library "libmonosgen-2.0.so" needed by "/data/app/com.companyname.Xamarin.Forms.Sandbox-1/lib/x86_64/libmonodroid.so"; caused by library "libmonosgen-2.0.so" not found
    at java.lang.Runtime.loadLibrary(Runtime.java:371)
    at java.lang.System.loadLibrary(System.java:989)
    at mono.MonoPackageManager.LoadApplication(MonoPackageManager.java:80)
    at mono.MonoRuntimeProvider.attachInfo(MonoRuntimeProvider.java:48)
    at android.app.ActivityThread.installProvider(ActivityThread.java:4964)
    at android.app.ActivityThread.installContentProviders(ActivityThread.java:4559)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4499)
    at android.app.ActivityThread.access$1500(ActivityThread.java:144)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1339)
    at android.os.Handler.dispatchMessage(Handler.java:102)
    at android.os.Looper.loop(Looper.java:135)
    at android.app.ActivityThread.main(ActivityThread.java:5221)
    at java.lang.reflect.Method.invoke(Native Method)
    at java.lang.reflect.Method.invoke(Method.java:372)
    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:899)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:694)
  Force finishing activity com.companyname.Xamarin.Forms.Sandbox/crc645b466b186780a260.MainActivity

[PureWeen]

@stevecluk I've poked the Android team to see if they have any thoughts

[stevecluk]

Thanks - this used to work with older builds, I'd just come across it when updating an app I'd released a few years ago to the latest packages. I'd wondered if a workaround might have been to upload an APK/AAB for 5.x devices only that just had 32-bit support (which worked locally), but Google Play requires 64-bit now for all updates so that's a non-starter, and I'm seeing quite a few crashes for people on those legacy devices.

[PureWeen]

It looks like this is a known issues with pre 6.0 android devices and it's related to using shared runtime/fast deployment

I tested and for me when I turn off shared runtime then it's able to deploy and run without issues

For your debug builds you can set

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <EmbedAssembliesIntoApk>False</EmbedAssembliesIntoApk>
    <AndroidUseSharedRuntime>False</AndroidUseSharedRuntime>
  </PropertyGroup>

This will allow some of the benefits of the shared runtime such as faster dev turnaround when an assembly changes.

[martinstoeckli]

I can reproduce the same error with my project https://github.com/martinstoeckli/SilentNotes/tree/master/src/SilentNotes.Android, with:

• Android emulator x86_64 - Lollipop 5.0 - API 21 - Phone M-DPI 5.1''.
• VisualStudio 16.6.1
• Xamarin.Android.SDK 10.3.1.4

With PureWeen's workaround it does not abort immediately, instead it throws an exception when calling GalaSoft.MvvmLight.Ioc.SimpleIoc.Default.Reset():

System.DllNotFoundException
  Message=libmono-native.so assembly:<unknown assembly> type:<unknown type> member:(null)

Interestingly the same error occurs when compiling in release mode. Other devices, e.g. Android 10.0 -API 29 do not have this problem. Really hope it is an emulator problem only.

[grendello]

@PureWeen @martinstoeckli @stevecluk do you see the problem with Release builds as well? Shared runtime is used just in (some) Debug builds.

[grendello]

Potential duplicate of https://github.com/xamarin/xamarin-android/issues/4772

[martinstoeckli]

Yes, it occured even in release builds, but at a different position in the code (see comment above). I uninstalled the app from the emulator first, then did a "Rebuild Solution" in "Release mode" and finally deployed it to the emulator.

[grendello]

@martinstoeckli I managed to reproduce the issue with Release using your app, thanks! In both Debug and Release cases the problem is buggy dynamic linker in Android 5. I will open a PR to fix both issues, hopefully today.

[grendello]

OK, so the above PR fixes the Release builds, as explained in
its commit message, but it does not fix the Debug w/shared
runtime scenario, the explanation why follows.

As explained in the PR #4861 message, the dynamic linker on
Android 5.x (for some reason only for 64-bit processes) fails to
correctly resolve dynamic dependencies by looking up already
loaded libraries by their SONAME. That is the cause of the
error in both Debug and Release scenarios, but the Debug
configuration with shared runtime is a bit more complicated and
bears more explanation here.

Xamarin.Android has been using the so-called shared runtime
nearly from the very beginning. Shared runtime is basically an
APK which contains the Mono runtime as well as the BCL and the
reason it was originally introduced was because devices 10 years
ago had really small storage and packaging Mono runtime + BCL
with every application would very quickly exhaust the storage
space on your device. Thus the idea to share those assemblies and
dynamic libraries between all the apps being developed on the
device. Shared runtime being a separate APK uses a different data
directory than the application being worked on and, thus, it is
necessary to find that location and explicitly load both the
managed assemblies and native libraries from there during
application startup. This is done, mostly, in Xamarin.Android's
native runtime which knows how to find the assemblies, shared
libraries and how to load them. Xamarin.Android used to contain,
basically, just three shared libraries - Mono runtime,
Xamarin.Android runtime and BTLS (TLS 1.2 implementation) - and
all of them were loaded with the C library function dlopen by
the Xamarin.Android runtime which, in turn, was loaded by Java
JNI using its internal mechanisms.

At some point, when we bumped the lowest Xamarin.Android platform
dependency to 19, we were able to switch to directly linking
against the Mono runtime since the dynamic loader in Android
handled the dependencies correctly - or so we thought :) It
turned out that it worked most of the time but not always and
therefore we started to preload the Mono runtime using a Java API
and everything worked pretty well until this issue popped up.

The problem here is that with shared runtime, the Mono
runtime library (libmonosgen-2.0.so) lives in the shared
runtime's data directory and not where the other shared
libraries, installed with the application, are located. And so
the 64-bit Android 5.x dynamic loader is unable to find
libmonosgen-2.0.so because of the bug described at the top.

The fix is simple - we should copy the Mono runtime (or symlink
it) during Xamarin.Android startup to the same location where the
other shared libraries are. But... we can't! The reason is that
the data directory where the application shared libraries live is
not owned by the application but by the system and, thus, we
have no permission to modify the directory in any way. So, this
solution is off the board.

Another solution would be to always place the Mono runtime in the
APK of the application, even during the shared runtime Debug
builds but... we don't want to do that! Why? Because sometime in
the (hopefully) near future, the shared runtime will be no more
and the hacks will no longer be necessary.

Instead of the fix I propose a workaround - if you need to test
on Android 5.x 64-bit emulators, disable the shared runtime in
your Debug configuration.

[brendanzagaeski]

_Release status update_

A new Preview version of Xamarin.Android has now been published that includes the fix for this item (for apps built in the Release configuration). The fix is not yet included in a Release version of Xamarin.Android. I will update this again when a Release version is available that includes the fix.

Fix included in Xamarin.Android 11.0.0.3.

Fix included on Windows in Visual Studio 2019 version 16.7 Preview 4. To try the Preview version that includes the fix, check for the latest updates in Visual Studio Preview.

Fix included on macOS in Visual Studio 2019 for Mac version 8.7 Preview 4. To try the Preview version that includes the fix, check for the latest updates on the Preview updater channel.

[brendanzagaeski]

_Release status update_

A new Release version of Xamarin.Android has now been published that includes the fix for this item (for apps built in the Release configuration).

Fix included in Xamarin.Android SDK version 11.0.0.3.

Fix included on Windows in Visual Studio 2019 version 16.7. To get the new version that includes the fix, check for the latest updates or install the most recent release from https://visualstudio.microsoft.com/downloads/.

Fix included on macOS in Visual Studio 2019 for Mac version 8.7. To get the new version that includes the fix, check for the latest updates on the Stable updater channel.

[danardelean]

I think there is a regression because this problem is still present in the latest Xamarin.Android release: 11.1.0.26