Azure-sdk-for-net: [BUG] Xamarin linking issues when using this library

[msioen]

Describe the bug
After updating the Microsoft.Azure.ServiceBus package from 3.4.0 to 4.1.1 our release builds don't work properly anymore. This is in a Xamarin.Android application.
Linking is necessary to keep our application size small, in previous versions of the library this worked without problem.

I've linked two separate stacktraces here. One for normal usage where only sdk assemblies are linked away, this is the standard setting for new Xamarin projects.

The second one is happening in our usecase. We have a more complex setup where some libraries are referenced which aren't linkersafe and need to be excluded from linking. Simply ignoring some libraries during the link fase starts failing the build as well.

Exception or Stack Trace (Link SDK Assemblies only)
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: The "LinkAssemblies" task failed unexpectedly.
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: Mono.Linker.MarkException: Error processing method: 'System.Void Microsoft.Azure.Services.AppAuthentication.AdalAuthenticationContext/d__0::MoveNext()' in assembly: 'Microsoft.Azure.Services.AppAuthentication.dll' ---> Mono.Cecil.ResolutionException: Failed to resolve System.Threading.Tasks.Task`1 Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContextIntegratedAuthExtensions::AcquireTokenAsync(Microsoft.IdentityModel.Clients.ActiveDirectory.AuthenticationContext,System.String,System.String,Microsoft.IdentityModel.Clients.ActiveDirectory.UserCredential)
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.HandleUnresolvedMethod (Mono.Cecil.MethodReference reference) [0x00013] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.MarkMethod (Mono.Cecil.MethodReference reference) [0x0004a] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.MarkInstruction (Mono.Cecil.Cil.Instruction instruction) [0x00040] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.MarkMethodBody (Mono.Cecil.Cil.MethodBody body) [0x000c2] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessMethod (Mono.Cecil.MethodDefinition method) [0x00186] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessQueue () [0x0001b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: --- End of inner exception stack trace ---
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessQueue () [0x00047] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue () [0x0000a] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.Process () [0x00102] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.Process (Mono.Linker.LinkContext context) [0x0000d] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.MonoDroidMarkStep.Process (Mono.Linker.LinkContext context) [0x0000b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Pipeline.ProcessStep (Mono.Linker.LinkContext context, Mono.Linker.Steps.IStep step) [0x0000d] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Pipeline.Process (Mono.Linker.LinkContext context) [0x0000f] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.Linker.Run (Mono.Linker.Pipeline pipeline, Mono.Linker.LinkContext context) [0x00000] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.Linker.Process (MonoDroid.Tuner.LinkerOptions options, Mono.Linker.ILogger logger, Mono.Linker.LinkContext& context) [0x00071] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Xamarin.Android.Tasks.LinkAssemblies.Execute (Java.Interop.Tools.Cecil.DirectoryAssemblyResolver res) [0x002d4] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Xamarin.Android.Tasks.LinkAssemblies.Execute () [0x0001b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute () [0x00023] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/msbuild-15/src/Build/BackEnd/TaskExecutionHost/TaskExecutionHost.cs:574
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask (Microsoft.Build.BackEnd.ITaskExecutionHost taskExecutionHost, Microsoft.Build.BackEnd.Logging.TaskLoggingContext taskLoggingContext, Microsoft.Build.BackEnd.TaskHost taskHost, Microsoft.Build.BackEnd.ItemBucket bucket, Microsoft.Build.BackEnd.TaskExecutionMode howToExecuteTask) [0x002a9] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/msbuild-15/src/Build/BackEnd/Components/RequestBuilder/TaskBuilder.cs:805

Exception or Stack Trace (Link all assemblies)
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: The "LinkAssemblies" task failed unexpectedly.
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: Mono.Linker.MarkException: Error processing method: 'System.Void Microsoft.Azure.Services.AppAuthentication.ClientCertificateAzureServiceTokenProvider/d__8::MoveNext()' in assembly: 'Microsoft.Azure.Services.AppAuthentication.dll' ---> Mono.Cecil.ResolutionException: Failed to resolve Microsoft.IdentityModel.Clients.ActiveDirectory.ClientAssertionCertificate
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.HandleUnresolvedType (Mono.Cecil.TypeReference reference) [0x00013] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x0002c] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.MonoDroidMarkStep.MarkType (Mono.Cecil.TypeReference reference) [0x00000] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.MarkMethodBody (Mono.Cecil.Cil.MethodBody body) [0x00047] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessMethod (Mono.Cecil.MethodDefinition method) [0x00186] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessQueue () [0x0001b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: --- End of inner exception stack trace ---
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessQueue () [0x00047] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.ProcessPrimaryQueue () [0x0000a] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.Process () [0x00102] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Steps.MarkStep.Process (Mono.Linker.LinkContext context) [0x0000d] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.MonoDroidMarkStep.Process (Mono.Linker.LinkContext context) [0x0000b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Pipeline.ProcessStep (Mono.Linker.LinkContext context, Mono.Linker.Steps.IStep step) [0x0000d] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Mono.Linker.Pipeline.Process (Mono.Linker.LinkContext context) [0x0000f] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.Linker.Run (Mono.Linker.Pipeline pipeline, Mono.Linker.LinkContext context) [0x00000] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at MonoDroid.Tuner.Linker.Process (MonoDroid.Tuner.LinkerOptions options, Mono.Linker.ILogger logger, Mono.Linker.LinkContext& context) [0x00071] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Xamarin.Android.Tasks.LinkAssemblies.Execute (Java.Interop.Tools.Cecil.DirectoryAssemblyResolver res) [0x002d4] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Xamarin.Android.Tasks.LinkAssemblies.Execute () [0x0001b] in <610526b6785240868e600b8f8b229a30>:0
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Microsoft.Build.BackEnd.TaskExecutionHost.Microsoft.Build.BackEnd.ITaskExecutionHost.Execute () [0x00023] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/msbuild-15/src/Build/BackEnd/TaskExecutionHost/TaskExecutionHost.cs:574
/Library/Frameworks/Mono.framework/External/xbuild/Xamarin/Android/Xamarin.Android.Common.targets(2105,5): error MSB4018: at Microsoft.Build.BackEnd.TaskBuilder.ExecuteInstantiatedTask (Microsoft.Build.BackEnd.ITaskExecutionHost taskExecutionHost, Microsoft.Build.BackEnd.Logging.TaskLoggingContext taskLoggingContext, Microsoft.Build.BackEnd.TaskHost taskHost, Microsoft.Build.BackEnd.ItemBucket bucket, Microsoft.Build.BackEnd.TaskExecutionMode howToExecuteTask) [0x002a9] in /Users/builder/jenkins/workspace/build-package-osx-mono/2019-06/external/bockbuild/builds/msbuild-15/src/Build/BackEnd/Components/RequestBuilder/TaskBuilder.cs:805

To Reproduce
Stacktrace 1: (link sdk assemblies only)

• create a new Xamarin.Android project
• add Microsoft.Azure.ServiceBus nuget package
• add some code which uses it - simply instantiating a subscriptionclient seems to be enough
• build the project in release mode (default settings)
• build fails on a linking issue

Stacktrace 2: (full sdk link)

• start from repro stacktrace 1
• change linking settings to 'link all'
• ignore Microsoft.Azure.Amqp from being linked either through LinkConfig or project settings
• build the project in release mode
• build fails on another linking issue

The sample project can be found here: https://github.com/msioen/ServicebusLinkingIssue

Expected behavior
Default mono linking options should not fail the build.

Setup (please complete the following information):

• OS: target: Android
• IDE : VS4Mac
• Version of the Library used: 4.1.1

Additional context
Note that in the second repro I ignored the Microsoft.Azure.Amqp assembly from being linked by mono. The linking settings in our actual project are somewhat different but it fails with the same exact stacktrace so as a minimal reproduction this seems viable.

Information Checklist
Kindly make sure that you have added all the following information above and checkoff the required fields otherwise we will treat the issuer as an incomplete report

• [ x] Bug Description Added
• [ x] Repro Steps Added
• [ x] Setup information Added

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[msftbot[bot]]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @jfggdl

[jsquire]

@nemakam and @binzywu: Would you be so kind as to offer your thoughts?

[nemakam]

@msioen , is it always failing on Microsoft.Azure.Services.AppAuthentication.dll? That's a new package that was added in 4.0.0
Also, I'm not very familiar with xamarin and its linking process. Do you have an understanding of what causes this and how to mitigate this?

[msioen]

@nemakam
I'm no expert on the linker but I should be able to somewhat explain what it does. In order to keep applications as small as possible the mono team created this. The linker basically makes a map of all the methods/classes/properties which are used in a project. Everything that's not needed and is not marked as critical will be removed resulting in smaller dll's and thus smaller apps.

I'm presuming what's happening here is that the ClientAssertionCertificate doesn't exist for mono Android/iOS. Previously the application never linked to any bits which used that class so it didn't complain about it. Now it traverses the code and finds something it fails on.
This probably can be solved by multitargeting the library and, for example, stripping out some code for certain targets.

But I'm sure there will be several Microsoft folks who might have both better explanations and better solutions.

[serkantkaraca]

Does your sample fail the same way on Windows OS build?

[msioen]

I can confirm the same exceptions happen in Visual Studio for Windows - used the same sample project as in original issue to reproduce.

[serkantkaraca]

Can you try skipping linking of failing assembly as described here - https://docs.microsoft.com/en-us/xamarin/android/deploy-test/building-apps/build-process

<AndroidLinkSkip>Assembly1;Assembly2</AndroidLinkSkip>

[msioen]

My sample project is currently skipping that way - see https://github.com/msioen/ServicebusLinkingIssue/blob/master/ServicebusLinkingIssue/ServicebusLinkingIssue.csproj#L51
Same issue happens both when skipping via csproj or via a linkconfiguration file.

[msioen]

@serkantkaraca do you have an update on this issue?

[aparent-emgs]

Any news on this? I'm getting the same exact issues as well...

[serkantkaraca]

Seems I lost track of this issue. Can you try skipping Microsoft.Azure.Services.AppAuthentication assembly? You are skipping AMQP library in your sample.

[aparent-emgs]

I don't know about if it works in his project, but I can say that skipping it in my project did not make any difference at all. In fact I was already skipping all of these:

Microsoft.Azure.Services.AppAuthentication
Microsoft.Azure.ServiceBus
Microsoft.IdentityModel.Clients.ActiveDirectory
Microsoft.IdentityModel.Clients.ActiveDirectory.Platform
Microsoft.Azure.Amqp

What we did for now to workaround this problem is just to set the Linking to None in the project's Android Options (didn't know there was an option to simply turn it off before posting my first comment here). Not ideal but at least I can have a working deployment onto my tablet/emulator.

[SHuygh]

@serkantkaraca Skipping the Microsoft.Azure.Services.AppAuthentication during linking results in the same stack trace as mentioned in the initial report.

The issue is caused by the fact that Microsoft.IdentityModel.Clients.ActiveDirectory.ClientAssertionCertificate is not available for Xamarin due to multi targeting of the Microsoft.IdentityModel.Clients.ActiveDirectory library.
As can be seen in here, CertificateAssertion is not included for both MonoAndroid and Xamarin.iOS10.
https://github.com/AzureAD/azure-activedirectory-library-for-dotnet/blob/68d7dea3643e075be85abbca1ab88ba614465541/src/Microsoft.IdentityModel.Clients.ActiveDirectory/Microsoft.IdentityModel.Clients.ActiveDirectory.csproj

In the servicebus library seems (at least for us) to originate from Microsoft.Azure.ServiceBus.ServiceBusConnection.InitializeConnection(ServiceBusConnectionStringBuilder builder) due to presence of the ManagedIdentityTokenProvider. Thus even though we do not use the ManagedIdentity path in the code, the linker detects it as a possible pathway.

[eichbaum]

I'm facing the same problem with Xamarin.iOS. Is there any solution for this issue on iOS?
Disabling linking is not an option for iOS, as the linker needs to remove unused deprecated stuff which otherwise blocks submitting the app to the App Store when using with Xamarin.Forms.

[oscarjaergren]

Just posting in this thread to get update about when this is fixed, got the same issue.

[Nullstr1ng]

Any updates? My .apk file way is way too big without linker.

[serkantkaraca]

@Nullstr1ng a PR is in the works which will provide device builds to avoid linking issues. This PR didn't make to 4.3.0 and will be part of next release in a month.

[ShubhamGargCTH]

Any updates ? we are having same issue with our ios build. Thanks

[nelutzu91]

@serkantkaraca do you have an specific date when this fix will be available? We cannot release ios to the apple store without it.

[eichbaum]

@nelutzu91 You might downgrade Microsoft.Azure.ServiceBus package to version 2.0.0 until Microsoft fixed the issue. Version 2.0.0 works fine and let you build with linking enabled (which is required for uploading to iOS App Store).

I also needed to add the package Microsoft.Azure.Amqp with version 2.4.8 to have it working.

As management features were added in a later version it is not possible to create topics in version 2.0.0. Anyway subscribing and receiving messages works fine.