Xamarin-macios: Frequent app crashes from crashlytics report starts with xamarin_initialize.cold.1 + 40252

[hot33331]

Stacktrace from Crashlytics:

Crashed: tid_407
0 libsystem_kernel.dylib 0x1bef89d88 __pthread_kill + 8
1 libsystem_pthread.dylib 0x1beea21e8 pthread_kill$VARIANT$mp + 136
2 libsystem_c.dylib 0x1bedf5644 abort + 100
3 Application.iOS 0x108000df8 xamarin_initialize.cold.1 + 40252
4 Application.iOS 0x107ff6034 xamarin_printf + 2468 (runtime.m:2468)
5 Application.iOS 0x107ebba40 mono_invoke_unhandled_exception_hook + 1299 (exception.c:1299)
6 Application.iOS 0x107e50a78 mono_handle_exception_internal + 2783 (mini-exceptions.c:2783)
7 Application.iOS 0x107e5146c mono_resume_unwind + 3532 (mini-exceptions.c:3532)
8 Application.iOS 0x107e45398 mono_arm_resume_unwind + 421 (exceptions-arm64.c:421)
9 Application.iOS 0x105047818 llvm_resume_unwind_trampoline + 1999688
10 Application.iOS 0x105d7abe0 Xamarin_iOS_UIKit_UIApplication_Main_string___intptr_intptr + 15841040 (UIApplication.cs:15841040)
11 Application.iOS 0x107e62678 mono_jit_runtime_invoke + 3165 (mini-runtime.c:3165)
12 Application.iOS 0x107f1db54 mono_runtime_invoke_checked + 3052 (object.c:3052)
13 Application.iOS 0x107f23dac mono_runtime_exec_main_checked + 5178 (object.c:5178)
14 Application.iOS 0x107e40c30 mono_jit_exec + 1383 (driver.c:1383)
15 Application.iOS 0x107ffda34 xamarin_main + 485 (monotouch-main.m:485)
16 Application.iOS 0x104e0a508 main + 188 (main.m:188)
17 libdyld.dylib 0x1bef94800 start + 4

Steps to Reproduce

Since this is from Crashlytics, I do not have repro steps.

Expected Behavior

App does not crash

Actual Behavior

App crashes

[whitneyschmidt]

@hot33331 Thank you for filing this issue!

It's hard to diagnose what's going on here without more information. __pthread_kill indicates that something killed the app, but it's not clear why this happened. A few questions:

1. Does the application crash consistently? Is the crash before or after launch?
2. Are there any steps you are completing in the application before it crashes? For example, you might be clicking on a certain UI element.
3. Is the application a Xamarin.iOS, Xamarin.Mac, or Xamarin Forms application?
4. Are you running VSWin, VSMac or a different development platform? You're using Crashlytics so I'm wondering whether you are developing with Firebase :)
5. Is the application running in a simulator or on a physical device, and have you tried running it on both with different results?
6. Are you using any third-party libraries or external software?

For us to investigate this further, could you please provide your full build logs, crash reports (if any), test case (to reproduce) and all your version information.

To get full build logs just set the log verbosity to diagnostic at the following locations:

• On Visual Studio for Mac: Preferences > Projects > Build
• On Visual Studio for Windows: Tools > Options > Projects and Solutions > Build and Run

On Visual Studio for Windows you also want to add -v -v -v -v to the mtouch additional arguments by right-clicking the project in the solution explorer and selecting Properties.

Note: this is done automatically on Visual Studio for Mac when the log verbosity is set to diagnostic.

The easiest way to get exact version information:

• On Visual Studio for Mac: "Visual Studio" menu, "About Visual Studio" item, "Show Details" button.
• On Visual Studio for Windows: "Help menu", "About Microsoft Visual Studio" item.

Then copy/paste the version information (you can use the "Copy Information" button).

We look forward to hearing from you!

[itoledo]

I'm getting this exact same crash but on store certification. On device/simulator works fine.
Attached is the symbolicated crash. App was built on AppCenter with Xamarin.iOS 13.18.
crash2.txt

[whitneyschmidt]

@itoledo Thank you for letting us know that you are encountering this issue as well.

Would it be possible for you to share a sample project that reproduces the issue? This is the best way for us to move forward with investigation.

Also, are you using any Google APIs/extensions? We have had similar looking issues in the past related to Firebase.

In addition, could you share the version information from your IDE?

The easiest way to get exact version information:

On Visual Studio for Mac: "Visual Studio" menu, "About Visual Studio" item, "Show Details" button.
On Visual Studio for Windows: "Help menu", "About Microsoft Visual Studio" item.
Then copy/paste the version information (you can use the "Copy Information" button).

[itoledo]

I can share the source code of our app. Problem is I can't replicate the problem, other than sending newer binaries to the App Store and waiting for certification to reject them.

We are using Xamarin.Firebase.iOS.Admob 7.57.0, should we try with an older version?

VS 16.6.2.
version-info.txt

[spouliot]

@itoledo the stack trace is not identical, even if some pieces are similar, so the problem can be different.

Can you open a different issue (so we don't risk confusing them) and also share your code with us ?

Even if we can't crash it we might be able to figure out what's running (looks like some async code) for a similar stack trace. Thanks!

Side note: you might want to try starting the app without any network connectivity. That's part of Apple's tests (along with IPv6 only networks) and it's easy to have code (even more 3rd party) that depends on the network.

[muellnes]

@hot33331 Thanks Tobias, for committing this crash. It was from one of our Apps. where we both work on as a team.
At the moment this crash is the top scorer in our current Xamarin.iOS application.
We do not see anything comparable on Xamarin.Android.

So to answer your questions in short Whitney:

Does the application crash consistently? Is the crash before or after launch?

Yes we have that consistently with our customers.
Actually on a 7 day base over 1000 Customers experience this crash.
I did check our analytics logs before asking Tobias for help and given they are correct it does not happen at a certain place or point in time after the user starts the application.
Over the last 10 crashes it has been either direct on startup, after using the application for a minute or even over several sessions.

Are there any steps you are completing in the application before it crashes? For example, you might be clicking on a certain UI element.

The traces also do not lead to any specific UI element.

Is the application a Xamarin.iOS, Xamarin.Mac, or Xamarin Forms application?

Xamarin.iOS

Are you running VSWin, VSMac or a different development platform?

Mainly VSMac or the product is compiled with the build tools of the current 8.6 version of VSMac and Xcode 11.5
But the problem did already also exist before that with Xcode 11.4 and VS 8.5

You're using Crashlytics so I'm wondering whether you are developing with Firebase :)

We use Firebase only for telemetry and did keep any other Firebase component like database, push notifications, etc. out.

Is the application running in a simulator or on a physical device, and have you tried running it on both with different results?

For simulator I can truly say I did not experience this crash. The first time I did see it was with the cash logs from AppStore and firebase. So we are hit by it in the field at the customer level - making things quite ugly for us.

I will collect the rest of the data as per your request and try to get it to you in the next week.

Thanks a lot for your help,
@whitneyschmidt

[muellnes]

Version Info

All of our systems (developer/CI) are using the same versions of the build tools, which are:

=== Visual Studio Professional 2019 for Mac ===

Version 8.6 (build 4520)
Installation UUID: 670cbdbb-d368-46f6-a55d-5638d32752cf
GTK+ 2.24.23 (Raleigh theme)
Xamarin.Mac 6.18.0.23 (d16-6 / 088c73638)

Package version: 610000104

=== Mono Framework MDK ===

Runtime:
Mono 6.10.0.104 (2019-12/5d03a6fe116) (64-bit)
Package version: 610000104

=== Roslyn (Language Service) ===

3.6.0-3.20210.9+4eafdcb1bcbd8d3573f2ba6065e56d9b9ce4f8a3

=== NuGet ===

Version: 5.6.0.6591

=== .NET Core SDK ===

SDK: /usr/local/share/dotnet/sdk/3.1.300/Sdks
SDK Versions:
3.1.300
2.1.802
MSBuild SDKs: /Library/Frameworks/Mono.framework/Versions/6.10.0/lib/mono/msbuild/Current/bin/Sdks

=== .NET Core Runtime ===

Runtime: /usr/local/share/dotnet/dotnet
Runtime Versions:
3.1.4
2.1.18
2.1.13

=== Xamarin.Profiler ===

Version: 1.6.12.29
Location: /Applications/Xamarin Profiler.app/Contents/MacOS/Xamarin Profiler

=== Updater ===

Version: 11

=== Apple Developer Tools ===

Xcode 11.5 (16139)
Build 11E608c

=== Xamarin.Mac ===

Version: 6.18.1.31 (Visual Studio Professional)
Hash: b3eedfed9
Branch: d16-6
Build date: 2020-04-09 10:22:36-0400

=== Xamarin.iOS ===

Version: 13.18.1.31 (Visual Studio Professional)
Hash: b3eedfed9
Branch: d16-6
Build date: 2020-04-09 10:22:36-0400

=== Xamarin Designer ===

Version: 16.6.0.329
Hash: d4f8bcd13
Branch: remotes/origin/d16-6
Build date: 2020-04-24 02:16:02 UTC

=== Xamarin.Android ===

Version: 10.3.1.0 (Visual Studio Professional)
Commit: xamarin-android/d16-6/3fe860a
Android SDK: /Users/Shared/Development/Android/sdk
Supported Android versions:
8.0 (API level 26)
8.1 (API level 27)

SDK Tools Version: 26.1.1
SDK Platform Tools Version: 29.0.4
SDK Build Tools Version: 29.0.2

Build Information:
Mono: 165f4b0
Java.Interop: xamarin/java.interop/d16-6@2cab35c
ProGuard: xamarin/proguard/master@905836d
SQLite: xamarin/sqlite/3.31.1@49232bc
Xamarin.Android Tools: xamarin/xamarin-android-tools/d16-6@bfb66f3

=== Microsoft Mobile OpenJDK ===

Java SDK: /Users/demuesa/Library/Developer/Xamarin/jdk/microsoft_dist_openjdk_8.0.25
1.8.0-25
Android Designer EPL code available here:
https://github.com/xamarin/AndroidDesigner.EPL

=== Android SDK Manager ===

Version: 16.6.0.50
Hash: 5901879
Branch: remotes/origin/d16-6
Build date: 2020-04-30 04:01:22 UTC

=== Android Device Manager ===

Version: 16.6.0.95
Hash: 45d17b5
Branch: remotes/origin/d16-6
Build date: 2020-04-30 04:01:42 UTC

=== Build Information ===

Release ID: 806004520
Git revision: 9c92c7baceef7771f8dcf5c86e1e1a66fdc26182
Build date: 2020-05-18 10:05:00-04
Build branch: release-8.6
Xamarin extensions: 9c92c7baceef7771f8dcf5c86e1e1a66fdc26182

=== Operating System ===

Mac OS X 10.15.5
Darwin 19.5.0 Darwin Kernel Version 19.5.0
Thu Apr 30 18:25:59 PDT 2020
root:xnu-6153.121.1~7/RELEASE_X86_64 x86_64

=== Enabled user installed extensions ===

DeepClean 1.2.5
Xamarin.Forms HotReload extension 1.4.0
Template Creator 0.5
Stack Overflow Search 0.7.1
Log Monitor 0.2
.NET Core support for Mono.Debugging 8.0.5
Paket 0.5.1
NuGet Package Management Extensions 0.21
Mutatio 1.0.3
Open With 0.2
Android Signature Tool 2.2.0
StyleCop Support 1.1.0.4
NuGet Package Explorer 0.4
MSBuild Editor 2.4.0
HotReloading 0.3.1

[Rybjerg]

Thread 0 Crashed:
0 libsystem_kernel.dylib 0x000000019a500df0 0x19a4da000 + 159216
1 libsystem_pthread.dylib 0x000000019a420930 0x19a41e000 + 10544
2 libsystem_c.dylib 0x000000019a3aec24 0x19a338000 + 486436
3 libsystem_c.dylib 0x000000019a3aebb0 0x19a338000 + 486320
4 Tools.iOS 0x0000000106d79200 xamarin_initialize.cold.1 + 0
5 Tools.iOS 0x0000000106d6e43c xamarin_printf + 40805436 (runtime.m:2468)
6 Tools.iOS 0x0000000106c33e48 mono_invoke_unhandled_exception_hook + 39517768 (exception.c:1299)
7 Tools.iOS 0x0000000106bc8e80 mono_handle_exception_internal + 39079552 (mini-exceptions.c:2783)
8 Tools.iOS 0x0000000106bc9874 mono_resume_unwind + 39082100 (mini-exceptions.c:3532)
9 Tools.iOS 0x0000000106bbd7a0 mono_arm_resume_unwind + 39032736 (exceptions-arm64.c:421)
10 Tools.iOS 0x00000001048bc428 llvm_resume_unwind_trampoline + 168
11 Tools.iOS 0x00000001059669c4 Xamarin_iOS_UIKit_UIApplication_Main_string___intptr_intptr + 19802564 (UIApplication.cs:0)
12 Tools.iOS 0x0000000106bdaa80 mono_jit_runtime_invoke + 39152256 (mini-runtime.c:3165)
13 Tools.iOS 0x0000000106c95f5c mono_runtime_invoke_checked + 39919452 (object.c:3220)
14 Tools.iOS 0x0000000106c9c1b4 mono_runtime_exec_main_checked + 39944628 (object.c:5284)
15 Tools.iOS 0x0000000106bb9038 mono_jit_exec + 39014456 (driver.c:1328)
16 Tools.iOS 0x0000000106d75e3c xamarin_main + 40836668 (monotouch-main.m:0)
17 Tools.iOS 0x0000000104759944 main + 874820 (main.m:170)
18 libdyld.dylib 0x000000019a50b2dc 0x19a50a000 + 4828

Is this same stacktrace?

[Rybjerg]

This came from Apple review. I could not reproduce it on our mix of physical devices / iOS versions or simulator.

This was caused by calling Xamarin.Essentials.Geolocation.GetLastKnownLocationAsync() without have optained the LocationWhenInUse permission. (cause of an error in the code)

So could this simple be, calling protected data without the proper permission only crashes on some devices?

[chamons]

Yes, if you call protected data without the proper permission Apple's API will hard crash you, no question asked. I've seen that many times.

If there is a race condition between you asking for permissions and trying to use them, then maybe that could be the root cause.

[muellnes]

Thanks for the input. We definitely always request permissions from the user before accessing them. Also we never access permissions the user did deny.
Like camera, wifi, location or any other permission-able request.

[whitneyschmidt]

@muellnes @hot33331 Have you been able to find a resolution to this issue or are your users still encountering it?

[muellnes]

@muellnes @hot33331 Have you been able to find a resolution to this issue or are your users still encountering it?

Hello Withney, yes the issue is even more persistent than before getting an even higher impact on our customers. As we can not provide you the build logs due to data protection issues I have also filed an internal ticket with MS.

[chamons]

@muellnes Can you provide a link or ticket number so we can link the two?

[muellnes]

@muellnes Can you provide a link or ticket number so we can link the two?

Of course @chamons. Here is the linked ticket:

https://developercommunity.visualstudio.com/content/problem/1134781/frequent-app-crashes-from-crashlytics-report-start.html

[mandel-macaque]

@muellnes Can I ask you for some information regarding the device and the application? The crash might be related to some apple security checks that are done at boot time, for example, if your app is using Bluetooth, missing the keys NSBluetoothPeripheralUsageDescription and NSBluetoothAlwaysUsageDescription on iOS 13 will crash the application in this way.

In order to understand everything better can you please let us know:

• The iOS version of the crashing devices. Is it always a specific version, is it any one we support?
• The requirements you have added for the application in the plist. Is it using bluetooth like in the example I provided?

[muellnes]

@mandel-macaque Thank you for asking.
The crash appears on a wide variety of iOS Versions:

• iOS 13
• iOS 12
• iOS 11
• iOS 10

Due to internal tests also on iOS 14.

We do not use bluetooth communication and as such do not require that permission.
All the permissions we require have the necessary UsageDescription in the Info.plist.

[VolceOntra]

We are also seeing this crash reported.

Our app uses NO bluetooth OR location APIs and OS versions span iOS 12-14b4. Like @muellnes reported, we are not able to reproduce via the simulator or our devices even though we have a high frequency report of crashes in Crashlytics due to this issue.

[whitneyschmidt]

@VolceOntra The crash you are seeing with xamarin_initialize.cold.1 doesn't give us very much information, and could be coming from a large variety of sources and issues. It's unlikely that you are seeing the crash for the same reason as @muellnes.

From from Chris's response in the DevCom thread, looking at the stack shows:

2 libsystem_c.dylib 0x1bedf5644 abort + 100
3 Application.iOS 0x108000df8 xamarin_initialize.cold.1 + 40252
4 Application.iOS 0x107ff6034 xamarin_printf + 2468 (runtime.m:2468)
5 Application.iOS 0x107ebba40 mono_invoke_unhandled_exception_hook + 1299 (exception.c:1299)
6 Application.iOS 0x107e50a78 mono_handle_exception_internal + 2783 (mini-exceptions.c:2783)
7 Application.iOS 0x107e5146c mono_resume_unwind + 3532 (mini-exceptions.c:3532)
8 Application.iOS 0x107e45398 mono_arm_resume_unwind + 421 (exceptions-arm64.c:421)
9 Application.iOS 0x105047818 llvm_resume_unwind_trampoline + 1999688
10 Application.iOS 0x105d7abe0 Xamarin_iOS_UIKit_UIApplication_Main_string___intptr_intptr + 15841040 (UIApplication.cs:15841040)
11 Application.iOS 0x107e62678 mono_jit_runtime_invoke + 3165 (mini-runtime.c:3165)
12 Application.iOS 0x107f1db54 mono_runtime_invoke_checked + 3052 (object.c:3052)
13 Application.iOS 0x107f23dac mono_runtime_exec_main_checked + 5178 (object.c:5178)
14 Application.iOS 0x107e40c30 mono_jit_exec + 1383 (driver.c:1383)
15 Application.iOS 0x107ffda34 xamarin_main + 485 (monotouch-main.m:485)
16 Application.iOS 0x104e0a508 main + 188 (main.m:188)
17 libdyld.dylib 0x1bef94800 start + 4

From this is looks like we are calling managed main, which calls UIApplication Main. Somewhere after that we hit an exception, that gets unhandled, we try to print something and crash printing.

That exception could be anything, a third party library, your own code, something in the SDK.

If the uncaught exception "escapes" the managed stack, it can do bad things to your runtime state. See

https://docs.microsoft.com/en-us/xamarin/ios/platform/exception-marshaling for details.

I'd consider working the problem by figuring out what exception is being thrown, or modifying your code to catch it and try to log it for more information. See

https://docs.microsoft.com/en-us/xamarin/ios/platform/exception-marshaling for details on that as well.

@muellnes From the version information you posed, it would seem that the issue is from a component that's shared by your app across different iOS versions. Without knowing more about your application, third party libraries that it uses, and any recent changes to the source code or resources used, it's very difficult to know what the source of the problem might be.

[whitneyschmidt]

@muellnes Closing out this issue since it's unfortunately hard to tell what the cause of the crash is without a reproducible case, and difficult to know whether the root problem is something in the Xamarin.MaciOS SDK or another tool or library that's used.

If more information becomes available, feel free to reopen this for investigation.

[muellnes]

@whitneyschmidt We did add the proposed additional logging via using the exception-marshaling callbacks and did get some interesting stack traces out of this. Main issue we have at the moment with this is that the numbers between logged stack traces from the Runtime.MarshalManagedException and Runtime.MarshalObjectiveCException event handlers do not equal up with the appearance of the xamarin_initialize.cold.1 issue.

[hot33331]

Hey @whitneyschmidt , would it be an option that someone from Microsoft takes a look into the code under NDA? Unfortunately we're not at liberty to disclose our codebase.

[chamons]

As noted on the forums (which I believe is still up to date):

If you want one on one support via email (business hours only) or need to provide private code or other sensitive info, then please submit a support request to the Xamarin support team at this link.