Xamarin-android: App linked using d8 fails at runtime with java.lang.InstantiationException

[mfeingol]

Steps to Reproduce

I decided to experiment with D8 as a replacement for Proguard. To that end, I added the following two lines to my working debug build:

    <AndroidLinkTool>r8</AndroidLinkTool>
    <AndroidR8ExtraArguments>--lib &quot;C:\Program Files (x86)\Android\android-sdk\platforms\android-28\optional\org.apache.http.legacy.jar&quot;</AndroidR8ExtraArguments>

cf https://github.com/xamarin/xamarin-android/issues/2693 if you're wondering about the second line.

This is the debug build's property group:

  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>portable</DebugType>
    <Optimize>false</Optimize>
    <OutputPath>bin\Debug</OutputPath>
    <DefineConstants>DEBUG;</DefineConstants>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <AndroidLinkMode>SdkOnly</AndroidLinkMode>
    <AotAssemblies>false</AotAssemblies>
    <EnableLLVM>false</EnableLLVM>
    <BundleAssemblies>false</BundleAssemblies>
    <EmbedAssembliesIntoApk>false</EmbedAssembliesIntoApk>
    <JavaMaximumHeapSize>1G</JavaMaximumHeapSize>
    <AndroidSupportedAbis />
    <AndroidUseSharedRuntime>true</AndroidUseSharedRuntime>
    <NoWarn>AvoidAsyncVoid</NoWarn>
    <AndroidEnableMultiDex>true</AndroidEnableMultiDex>
    <EnableProguard>false</EnableProguard>
    <LangVersion>latest</LangVersion>
    <AndroidUseAapt2>true</AndroidUseAapt2>
    <AndroidLinkTool>r8</AndroidLinkTool>
    <AndroidR8ExtraArguments>--lib &quot;C:\Program Files (x86)\Android\android-sdk\platforms\android-28\optional\org.apache.http.legacy.jar&quot;</AndroidR8ExtraArguments>
  </PropertyGroup>

After this, my project compiles, but fails at launch with the following exception:

Time    Device Name Type    PID Tag Message
04-07 18:08:58.350  android_8_1 Error   11764   AndroidRuntime  java.lang.RuntimeException: Unable to get provider android.support.v4.content.FileProvider: java.lang.InstantiationException: java.lang.Class<android.support.v4.content.FileProvider> cannot be instantiated
    at android.app.ActivityThread.installProvider(ActivityThread.java:6242)
    at android.app.ActivityThread.installContentProviders(ActivityThread.java:5805)
    at android.app.ActivityThread.handleBindApplication(ActivityThread.java:5722)
    at android.app.ActivityThread.-wrap1(Unknown Source:0)
    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1656)
    at android.os.Handler.dispatchMessage(Handler.java:106)
    at android.os.Looper.loop(Looper.java:164)
    at android.app.ActivityThread.main(ActivityThread.java:6494)
    at java.lang.reflect.Method.invoke(Native Method)
    at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

I should note the build has one new warning... and not the most useful one.

4>R8 : warning : The rule `-keep public class *extends java.lang.annotation.Annotation {

Expected Behavior

A working app, with r8 as a drop-in replacement for proguard and multidex.

Actual Behavior

App crashes on launch.

Version Information

VS 2019 RTM

[brendanzagaeski]

Thanks for the report!

To leave a first quick cross-reference for the Xamarin.Android team to look into, I suspect this might be related to https://github.com/xamarin/xamarin-android/issues/2931, where the issue is that the version of R8 currently shipped in Xamarin.Android version 9.2 (the current version included in Visual Studio 2019) handles some types from the Android Support Library differently than ProGuard. In that case, the newer version of the upstream R8 tool that's now included in the master and d16-1 development branches restores the expected ProGuard behavior.

[brendanzagaeski]

For @mfeingol, if you're up for a little local experiment, you can try backing up the r8.jar file in your current installation, found by default at a location similar to C:\Program Files (x86)\Microsoft Visual Studio\2019\Enterprise\MSBuild\Xamarin\Android\r8.jar, depending on Visual Studio edition. And then you can replace it with the new r8.jar version unzipped from one of the recent development .vsix builds from the open source continuous builds server. If you're interested in trying that, I can also help unzip the r8.jar file from the .vsix and send that over to you, in case that would be helpful.

If you'd prefer not to experiment with your local environment, if it's easy to list a few quick steps to create a project that hits this issue (or zip up and attach one), that would be perfect. I took a first quick attempt to reproduce the issue by building and running a sample that uses android.support.v4.content.FileProvider, but it looks like that sample is not sufficient to replicate the issue, so any quick tips you can share about setting up an app to use FileProvider in the problematic way would be perfect. Thanks!

[jonathanpeppers]

@mfeingol could you attach a diagnostic build log?

I would like to see all the messages that r8 is printing:

4>R8 : warning : The rule `-keep public class *extends java.lang.annotation.Annotation {

It seems like maybe this rule is getting ignored by r8.

I suspect your issue with org.apache.http.legacy.jar will be fixed by https://github.com/xamarin/xamarin-android/pull/2736. This will be coming in the VS 2019 16.1 preview that will be out soon.

[mfeingol]

@jonathanpeppers: here's a diagnostic build log:

build.zip

@brendanzagaeski: the startup crash still repros with the r8 from Xamarin.Android.Sdk-OSS-9.2.99.68_d16-1_9e3d37f.vsix. I can give you access to my Azure DevOps org; that might be the simplest repro.

[jonathanpeppers]

Hmm, looking at the log, it seems like R8 is just stripping out what is required to call into:

• java.lang.Class<android.support.v4.content.FileProvider>

It seems proguard isn't doing this. R8 is generally more aggressive, but there could be a bug here.

@mfeingol email me at the email listed on my Github account. We can work out what you can send me to help figure this out, thanks!

[jonathanpeppers]

@mfeingol sent me his .dex files (and other info) over email.

The current problem is a java.lang.InstantiationException at runtime:

Java.Lang.RuntimeException: Unable to get provider android.support.v4.content.FileProvider: 
    java.lang.InstantiationException: java.lang.Class<android.support.v4.content.FileProvider> cannot be instantiated

Looking at a diff of the dex files (dexdump output), r8 is on the left, proguard on the right:

r8 is marking the class abstract???

Still looking into this...

[jdluzen]

I wanted to try out R8/D8 as well. D8 seems to be working on the default Shell app. But R8 fails at runtime with Error inflating class android.support.v7.widget.ContentFrameLayout ... Can't instantiate abstract class android.support.v7.widget.ContentFrameLayout. This seems to be related to what you're already looking in to.

[jonathanpeppers]

@jdluzen your issue is just some general rules needed for a Xamarin.Forms app. You would have the same problem using proguard:

Can you add a proguard.cfg file that looks like this? https://github.com/xamarin/Xamarin.Forms/blob/master/.nuspec/proguard.cfg

[mfeingol]

@jonathanpeppers: curious if you ever made progress on this issue. I'm downloading 16.1.0 right now and will be giving the updated r8/d8 a spin again soon.

[jonathanpeppers]

16.1 has some general d8/r8 fixes, but I'm not sure it will solve this issue.

I think you could try adding the rule:

-keep class android.support.v4.content.FileProvider { *; }

This issue is still on my list to look into further at some point.

[jonathanpeppers]

@mfeingol I think I just figured out what your issue may be, can you turn off multidex and try again?

<!--Or just remove this completely-->
<AndroidEnableMultiDex>False</AndroidEnableMultiDex>

_Also, see if you can use VS 2019 16.1 that just hit stable._

I think there is an issue the way we are running r8 when you have all of the options on:

• d8
• multidex
• r8

You shouldn't need multidex anyway, if you have r8 turned on. r8 should make the Java code small enough, where you won't need multidex.

[mfeingol]

I don't believe I was using multidex, but I'll give it another try over the weekend.

[jonathanpeppers]

The original post at the top has:

 <AndroidEnableMultiDex>true</AndroidEnableMultiDex>

But maybe that changed?

Also if you update to 16.1, you shouldn't need the AndroidR8ExtraArguments workaround.

[mfeingol]

Sorry, you're right, the debug build needs it, but the release build doesn't.

I think I was assuming that if I had r8 running, multidex wouldn't run. But now I'm not sure what was running and what wasn't. :-) I'll get this sorted out over the weekend with 16.1.0 and update with a definitive answer.

[mfeingol]

Revisiting this with 16.1.1 and the following settings for debug builds, things appear to be working:

    <AndroidLinkMode>SdkOnly</AndroidLinkMode>
    <AotAssemblies>false</AotAssemblies>
    <EnableLLVM>false</EnableLLVM>
    <BundleAssemblies>false</BundleAssemblies>
    <EmbedAssembliesIntoApk>false</EmbedAssembliesIntoApk>
    <AndroidUseSharedRuntime>true</AndroidUseSharedRuntime>
    <AndroidEnableMultiDex>true</AndroidEnableMultiDex>
    <AndroidUseAapt2>true</AndroidUseAapt2>
    <AndroidDexTool>d8</AndroidDexTool>
    <AndroidEnableSGenConcurrent>true</AndroidEnableSGenConcurrent>

I do see the following warnings during build. Are those worth worrying about?

1>R8 : warning : The rule `-keep public class *extends java.lang.annotation.Annotation {
1>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
1>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
1>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
[...]

My proguard.cfg file is the following:

-keep class android.support.v7.widget.FitWindowsFrameLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsFrameLayout

-keep class android.support.v7.widget.FitWindowsLinearLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsLinearLayout

-keep class android.support.design.** { *; }
-keep class android.support.multidex.MultiDexApplication { *; }

[jonathanpeppers]

This one is something about the default multi-dex rules in the Android SDK:
R8 : warning : The rule `-keep public class *extends java.lang.annotation.Annotation {

r8 seems like it wants extends to say implements, but we can ignore it.

This one means there were multiple jar files containing:
R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.

We can also ignore this one, but I should probably figure out a way to make that not show a warning for Xamarin.Android builds.

@mfeingol does a Release build work? If you have <AndroidLinkTool>r8</AndroidLinkTool>, make sure you have <AndroidEnableMultiDex>False</AndroidEnableMultiDex>.

[tipa]

I had the same issue and disabling multidex resolved it for me

[mfeingol]

@jonathanpeppers: well, now that you mention it, I'm having a devil of a time getting my release builds to work. They work on x86 in the emulator, but when pushed to my ARM device it crashes on startup.

Would appreciate any advice. I've fiddled with my proguard.cfg, but I haven't come up with the right incantation yet.

Here's what I see in Device Monitor on app launch (heavily filtered):

05-28 08:00:20.750: I/zygote64(27669): Rejecting re-init on previously-failed class java.lang.Class<android.support.v4.view.ViewCompat$OnUnhandledKeyEventListenerWrapper>: java.lang.NoClassDefFoundError: Failed resolution of: Landroid/view/View$OnUnhandledKeyEventListener;
05-28 08:00:20.750: I/zygote64(27669):   at void android.support.v4.view.ViewCompat.setOnApplyWindowInsetsListener(android.view.View, android.support.v4.view.OnApplyWindowInsetsListener) (:2203)
05-28 08:00:20.750: I/zygote64(27669):   at android.view.ViewGroup android.support.v7.app.AppCompatDelegateImpl.createSubDecor() (:637)
05-28 08:00:20.750: I/zygote64(27669):   at void android.support.v7.app.AppCompatDelegateImpl.ensureSubDecor() (:518)

05-28 08:00:20.750: I/zygote64(27669): Caused by: java.lang.ClassNotFoundException: Didn't find class "android.view.View$OnUnhandledKeyEventListener" on path: DexPathList[[zip file "/data/app/com.backroads.android-lhU_3UuEhhgD-UKQMTkZmA==/base.apk"],nativeLibraryDirectories=[/data/app/com.backroads.android-lhU_3UuEhhgD-UKQMTkZmA==/lib/arm64, /data/app/com.backroads.android-lhU_3UuEhhgD-UKQMTkZmA==/base.apk!/lib/arm64-v8a, /system/lib64, /system/vendor/lib64]]
05-28 08:00:20.750: I/zygote64(27669):   at java.lang.Class dalvik.system.BaseDexClassLoader.findClass(java.lang.String) (BaseDexClassLoader.java:93)
05-28 08:00:20.750: I/zygote64(27669):   at java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String, boolean) (ClassLoader.java:379)
05-28 08:00:20.750: I/zygote64(27669):   at java.lang.Class java.lang.ClassLoader.loadClass(java.lang.String) (ClassLoader.java:312)

05-28 08:00:22.756: A/libc(27669): Fatal signal 11 (SIGSEGV), code 1, fault addr 0x20 in tid 27695 (Thread Pool Wor)
05-28 08:00:22.863: A/DEBUG(27781): signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x20
05-28 08:00:22.863: A/DEBUG(27781): Cause: null pointer dereference
05-28 08:00:22.865: A/DEBUG(27781):     #00 pc 0000000000177678  /data/app/com.backroads.android-lhU_3UuEhhgD-UKQMTkZmA==/lib/arm64/libmonosgen-2.0.so (mono_jit_info_get_method+8)
05-28 08:00:22.865: A/DEBUG(27781):     #01 pc 00000000000be654  /data/app/com.backroads.android-lhU_3UuEhhgD-UKQMTkZmA==/lib/arm64/libmonosgen-2.0.so

My proguard.cfg:

-keep class android.support.v7.widget.FitWindowsFrameLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsFrameLayout

-keep class android.support.v7.widget.FitWindowsLinearLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsLinearLayout

-keep class android.support.design.** { *; }
-keep class android.support.multidex.MultiDexApplication { *; }

# workarounds for Xamarin.Android & r8 in VS 2019 16.0
-keep class android.runtime.** { <init>(...); }
-keep class assembly_mono_android.android.runtime.** { <init>(...); }
-keepclassmembers class * extends android.view.View {
   *** set*(...);
}

csproj:

  <PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>portable</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <AndroidManagedSymbols>true</AndroidManagedSymbols>
    <AndroidUseSharedRuntime>false</AndroidUseSharedRuntime>
    <AotAssemblies>true</AotAssemblies>
    <EnableLLVM>true</EnableLLVM>
    <BundleAssemblies>true</BundleAssemblies>
    <AndroidSupportedAbis>armeabi-v7a;arm64-v8a</AndroidSupportedAbis>
    <JavaMaximumHeapSize>1G</JavaMaximumHeapSize>
    <AndroidLinkMode>SdkOnly</AndroidLinkMode>
    <EmbedAssembliesIntoApk>true</EmbedAssembliesIntoApk>
    <NoWarn>AvoidAsyncVoid</NoWarn>
    <AndroidEnableMultiDex>false</AndroidEnableMultiDex>
    <AndroidEnableSGenConcurrent>true</AndroidEnableSGenConcurrent>
    <AndroidCreatePackagePerAbi>true</AndroidCreatePackagePerAbi>
    <LangVersion>latest</LangVersion>
    <AndroidUseAapt2>true</AndroidUseAapt2>
    <AndroidLinkTool>r8</AndroidLinkTool>
  </PropertyGroup>

[tipa]

That SIGSEGV error might be caused by these issues: https://github.com/mono/mono/issues/14170 + https://github.com/xamarin/xamarin-android/issues/3112

[mfeingol]

It's a great thought, but I think those are happening only with a debugger attached? This is a release build with no debugger.

[tipa]

It's not only related to debugging. You can easily generate such a crash with these lines (on a 64bit device I believe), no matter if a debugger is attached or not:

try { throw new FileLoadException(); }
catch (System.Exception e) when (e is FileLoadException) { }

That bug was introduced with the Mono version that shipped with VS19 16.1
Also happens when a library you are using is rethrowing exceptions internally: https://github.com/xamarin/xamarin-android/issues/3112#issuecomment-495354178

[jonathanpeppers]

Followed up with @mfeingol over email, he was able to get a Release build of his app using VS 2019 16.1:

• <AndroidDexTool>d8</AndroidDexTool>
• <AndroidLinkTool>r8</AndroidLinkTool>

_NOTE: for Debug builds, remove <AndroidLinkTool> and use multidex if needed_

The proguard rules were simply:

-keep class android.support.v7.widget.FitWindowsLinearLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsLinearLayout

Using Xamarin.Forms 3.6.0.344457.

Closing for now, I'll follow up to see if we need to update Xamarin.Forms proguard file. Thanks @mfeingol for the report and diagnosis!

[tipa]

Why should AndroidLinkTool be removed for Debug builds?

[jonathanpeppers]

@tipa just to keep your Debug build as fast as possible. This is why you would turn off the linker for Debug builds as well.

[mfeingol]

For completeness, this is my release configuration. With the proguard.cfg Jonathan lists above, I'm able to successfully build with 16.1.1:

    <Optimize>true</Optimize>
    <AndroidManagedSymbols>true</AndroidManagedSymbols>
    <AndroidUseSharedRuntime>false</AndroidUseSharedRuntime>
    <AotAssemblies>true</AotAssemblies>
    <EnableLLVM>true</EnableLLVM>
    <BundleAssemblies>true</BundleAssemblies>
    <JavaMaximumHeapSize>1G</JavaMaximumHeapSize>
    <AndroidLinkMode>SdkOnly</AndroidLinkMode>
    <EmbedAssembliesIntoApk>true</EmbedAssembliesIntoApk>
    <AndroidEnableMultiDex>false</AndroidEnableMultiDex>
    <AndroidEnableSGenConcurrent>true</AndroidEnableSGenConcurrent>
    <AndroidCreatePackagePerAbi>true</AndroidCreatePackagePerAbi>
    <AndroidUseAapt2>true</AndroidUseAapt2>
    <AndroidLinkTool>r8</AndroidLinkTool>

[tipa]

@tipa just to keep your Debug build as fast as possible. This is why you would turn off the linker for Debug builds as well.

Shouldn't there be an option in the VS GUI to disable that "Code shrinker" step? Right now in the drop-down there are only options for R8 and ProGuard, both slowing down Debug builds if I understand correctly...?

[jonathanpeppers]

Yes they are fixing that in hopefully 16.1.2. To remove it, you’ll have to edit the csproj.

[KennyDizi]

This one is something about the default multi-dex rules in the Android SDK:
R8 : warning : The rule `-keep public class *extends java.lang.annotation.Annotation {

r8 seems like it wants extends to say implements, but we can ignore it.

This one means there were multiple jar files containing:
R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.

We can also ignore this one, but I should probably figure out a way to make that not show a warning for Xamarin.Android builds.

@mfeingol does a Release build work? If you have <AndroidLinkTool>r8</AndroidLinkTool>, make sure you have <AndroidEnableMultiDex>False</AndroidEnableMultiDex>.

VS 16.2.5 still have many warnings:
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/maven/com.squareup.okio/okio/pom.xml' already exists.
3>R8 : warning : Resource 'META-INF/maven/com.squareup.okio/okio/pom.properties' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.
3>R8 : warning : Resource 'META-INF/MANIFEST.MF' already exists.

[jonathanpeppers]

@xamarindevelopervietnam I filed an issue here about the warnings to follow up later: https://github.com/xamarin/xamarin-android/issues/3622

[nschoenberg]

Had this issue with VS 16.4.5 on Emulator and my Android Phone, can confirm the proguad.cfg solved my issue:

-keep class android.support.v7.widget.FitWindowsLinearLayout { *; }
-dontwarn android.support.v7.widget.FitWindowsLinearLayout

For reference, my release settings:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Release|AnyCPU' ">
    <DebugSymbols>true</DebugSymbols>
    <DebugType>pdbonly</DebugType>
    <Optimize>true</Optimize>
    <OutputPath>bin\Release</OutputPath>
    <ErrorReport>prompt</ErrorReport>
    <WarningLevel>4</WarningLevel>
    <AndroidManagedSymbols>true</AndroidManagedSymbols>
    <AndroidUseSharedRuntime>false</AndroidUseSharedRuntime>
    <AndroidDexTool>d8</AndroidDexTool>
    <AndroidLinkTool>r8</AndroidLinkTool>
    <AndroidHttpClientHandlerType>Xamarin.Android.Net.AndroidClientHandler</AndroidHttpClientHandlerType>
    <AndroidKeyStore>false</AndroidKeyStore>
    <MandroidI18n />
  </PropertyGroup>