Firebase-android-sdk: Crash in Android 7 and 8 with FirebaseInstallations

Created on 8 Jun 2020 · 25Comments · Source: firebase/firebase-android-sdk

Step 2: Describe your environment

Android Studio version: 4.0.0
Firebase Component: Might be any of Firebase performance, config, messaging dynamic links or analytics
Component version: Following the order of the last sentence: 19.0.7, 19.1.4, 20.2.0, 19.1.0 and 17.4.2

Step 3:

Getting a crash in both Android 8 and 7 with the following message in crashlytics:

Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.Object com.android.org.conscrypt.SSLParametersImpl.clone()' on a null object reference

Steps to reproduce:

Full stacktrace:

Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.Object com.android.org.conscrypt.SSLParametersImpl.clone()' on a null object reference
       at com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket(OpenSSLSocketFactoryImpl.java:106)
       at com.android.okhttp.Connection.connectTls(Connection.java:1494)
       at com.android.okhttp.Connection.connectSocket(Connection.java:1458)
       at com.android.okhttp.Connection.connect(Connection.java:1413)
       at com.android.okhttp.Connection.connectAndSetOwner(Connection.java:1700)
       at com.android.okhttp.OkHttpClient$1.connectAndSetOwner(OkHttpClient.java:133)
       at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:466)
       at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:371)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:503)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:130)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:261)
       at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
       at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeRequestBodyToOutputStream(FirebaseInstallationServiceClient.java:208)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeFIDCreateRequestBodyToOutputStream(FirebaseInstallationServiceClient.java:190)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.createFirebaseInstallation(FirebaseInstallationServiceClient.java:160)
       at com.google.firebase.installations.FirebaseInstallations.registerFidWithServer(FirebaseInstallations.java:464)
       at com.google.firebase.installations.FirebaseInstallations.doNetworkCall(FirebaseInstallations.java:341)
       at com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0(FirebaseInstallations.java:331)
       at com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run(FirebaseInstallations.java)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1133)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:607)
       at java.lang.Thread.run(Thread.java:762)

firebase-installations p2

Source

franvis

All 25 comments

I found a few problems with this issue:

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.
This issue does not seem to follow the issue template. Make sure you provide all the required information.

google-oss-bot on 8 Jun 2020

Thanks for reaching out. I found that this could happen if you set System properties out of band on your Android App. See: https://github.com/simpligility/ksoap2-android/issues/110#issue-231347582

Is that something you are doing?

Also, here's where the error is throw. Let us know if you see something that stands out
https://github.com/firebase/firebase-android-sdk/blob/455e48371ef627b1aa387126f1ff5bd91544fe3c/firebase-installations/src/main/java/com/google/firebase/installations/remote/FirebaseInstallationServiceClient.java#L209

ashwinraghav on 8 Jun 2020

@ashwinraghav Thank you for the quick reply! Sadly enough we are not doing anything with the system properties and we cannot see anything that stands out. I can tell you that before this release we updated all our dependencies from firebase (the ones I previously mentioned) and that according to crashlytics there are no logs or keys so it might be that this happens at the very beginning of the app, but not totally sure.

Anything else I can provide as info to help solving this crash?

franvis on 10 Jun 2020

@ankitaj224 @andirayo While I don't have a great idea of what could cause this, it seems like we could model this as a failure to fetch tokens, ultimately resulting in an auth error bubbled up through the firebase product. Thoughts?

@franvis Everything I see points to this being a crash that we live with on Android, and supposedly fixed in Gingerbread!, clearly not the case here. How frequent are these crashes in your case ?

ashwinraghav on 10 Jun 2020

@franvis also are you seeing this only for specific device models? or is it a pretty wide spread of devices?

ciarand on 10 Jun 2020

@ashwinraghav We have had this crash roughly around 90 times in 5 days. Just so you know I just noticed it happening in Android 6 as well, so my thoughts about having it only in Android 7 and 8 can be dismissed.

@ciarand We do not see a pattern for a certain brand. If I take android 8 cases for example, 68% happens on Samsung devices but the remaining 32% is quite spread.

franvis on 11 Jun 2020

👍1

We have the same crashes in our project after update to the last Firebase versions.
firebase-analytics 17.3.0 -> 17.4.3
firebase-crashlytics 17.0.0-beta03 -> 17.0.1

Stacktrace 1:

com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket (OpenSSLSocketFactoryImpl.java:106)
com.android.okhttp.Connection.connectTls (Connection.java:224)
com.android.okhttp.Connection.connectSocket (Connection.java:199)
com.android.okhttp.Connection.connect (Connection.java:172)
com.android.okhttp.Connection.connectAndSetOwner (Connection.java:367)
com.android.okhttp.OkHttpClient$1.connectAndSetOwner (OkHttpClient.java:130)
com.android.okhttp.internal.http.HttpEngine.connect (HttpEngine.java:330)
com.android.okhttp.internal.http.HttpEngine.sendRequest (HttpEngine.java:247)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:457)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect (HttpURLConnectionImpl.java:126)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream (HttpURLConnectionImpl.java:257)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream (DelegatingHttpsURLConnection.java:218)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeFIDCreateRequestBodyToOutputStream (FirebaseInstallationServiceClient.java:180)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.createFirebaseInstallation (FirebaseInstallationServiceClient.java:154)
com.google.firebase.installations.FirebaseInstallations.registerFidWithServer (FirebaseInstallations.java:433)
com.google.firebase.installations.FirebaseInstallations.doNetworkCall (FirebaseInstallations.java:333)
com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0 (FirebaseInstallations.java:323)
com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run (FirebaseInstallations.java)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1133)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:607)
java.lang.Thread.run (Thread.java:760)

Devices 1:
Android 6.X, 7.X, 8.0

Stacktrace 2:

com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket (OpenSSLSocketFactoryImpl.java:135)
com.android.okhttp.internal.io.RealConnection.connectTls (RealConnection.java:1463)
com.android.okhttp.internal.io.RealConnection.connectSocket (RealConnection.java:1423)
com.android.okhttp.internal.io.RealConnection.connect (RealConnection.java:1367)
com.android.okhttp.internal.http.StreamAllocation.findConnection (StreamAllocation.java:219)
com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection (StreamAllocation.java:142)
com.android.okhttp.internal.http.StreamAllocation.newStream (StreamAllocation.java:104)
com.android.okhttp.internal.http.HttpEngine.connect (HttpEngine.java:410)
com.android.okhttp.internal.http.HttpEngine.sendRequest (HttpEngine.java:343)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:489)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect (HttpURLConnectionImpl.java:131)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream (HttpURLConnectionImpl.java:262)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream (DelegatingHttpsURLConnection.java:218)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeFIDCreateRequestBodyToOutputStream (FirebaseInstallationServiceClient.java:180)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.createFirebaseInstallation (FirebaseInstallationServiceClient.java:154)
com.google.firebase.installations.FirebaseInstallations.registerFidWithServer (FirebaseInstallations.java:433)
com.google.firebase.installations.FirebaseInstallations.doNetworkCall (FirebaseInstallations.java:333)
com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0 (FirebaseInstallations.java:323)
com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run (FirebaseInstallations.java:4)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1162)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:636)
java.lang.Thread.run (Thread.java:764)

Devices 2:
Only Android 8.0

Stacktrace 3:

com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket (OpenSSLSocketFactoryImpl.java:102)
com.android.okhttp.internal.http.SocketConnector.connectTls (SocketConnector.java:89)
com.android.okhttp.Connection.connect (Connection.java:143)
com.android.okhttp.Connection.connectAndSetOwner (Connection.java:185)
com.android.okhttp.OkHttpClient$1.connectAndSetOwner (OkHttpClient.java:128)
com.android.okhttp.internal.http.HttpEngine.nextConnection (HttpEngine.java:342)
com.android.okhttp.internal.http.HttpEngine.connect (HttpEngine.java:331)
com.android.okhttp.internal.http.HttpEngine.sendRequest (HttpEngine.java:249)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:437)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect (HttpURLConnectionImpl.java:114)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream (HttpURLConnectionImpl.java:245)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream (DelegatingHttpsURLConnection.java:218)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java:25)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeFIDCreateRequestBodyToOutputStream (FirebaseInstallationServiceClient.java:180)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.createFirebaseInstallation (FirebaseInstallationServiceClient.java:154)
com.google.firebase.installations.FirebaseInstallations.registerFidWithServer (FirebaseInstallations.java:433)
com.google.firebase.installations.FirebaseInstallations.doNetworkCall (FirebaseInstallations.java:333)
com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0 (FirebaseInstallations.java:323)
com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run (FirebaseInstallations.java)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1113)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:588)
java.lang.Thread.run (Thread.java:818)

Devices 3:
Android 5.X, 6.X

Stacktrace 4:

com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket (OpenSSLSocketFactoryImpl.java:153)
com.android.okhttp.internal.io.RealConnection.connectTls (RealConnection.java:181)
com.android.okhttp.internal.io.RealConnection.connectSocket (RealConnection.java:149)
com.android.okhttp.internal.io.RealConnection.connect (RealConnection.java:112)
com.android.okhttp.internal.http.StreamAllocation.findConnection (StreamAllocation.java:184)
com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection (StreamAllocation.java:126)
com.android.okhttp.internal.http.StreamAllocation.newStream (StreamAllocation.java:95)
com.android.okhttp.internal.http.HttpEngine.connect (HttpEngine.java:281)
com.android.okhttp.internal.http.HttpEngine.sendRequest (HttpEngine.java:224)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute (HttpURLConnectionImpl.java:461)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect (HttpURLConnectionImpl.java:127)
com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream (HttpURLConnectionImpl.java:258)
com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream (DelegatingHttpsURLConnection.java:218)
com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream (HttpsURLConnectionImpl.java:26)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeFIDCreateRequestBodyToOutputStream (FirebaseInstallationServiceClient.java:180)
com.google.firebase.installations.remote.FirebaseInstallationServiceClient.createFirebaseInstallation (FirebaseInstallationServiceClient.java:154)
com.google.firebase.installations.FirebaseInstallations.registerFidWithServer (FirebaseInstallations.java:433)
com.google.firebase.installations.FirebaseInstallations.doNetworkCall (FirebaseInstallations.java:333)
com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0 (FirebaseInstallations.java:323)
com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run (FirebaseInstallations.java:4)
java.util.concurrent.ThreadPoolExecutor.processTask (ThreadPoolExecutor.java:1187)
java.util.concurrent.ThreadPoolExecutor.runWorker (ThreadPoolExecutor.java:1152)
java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:641)
java.lang.Thread.run (Thread.java:784)

Devices 4:
Only Android 9

Prichman on 15 Jun 2020

👍1

@ashwinraghav Thanks for checking in. Regarding your question - this failure is unrelated to auth token fetch. Its runtime error caused on a background thread trying to connect to FIS servers to register FID.

@ankitaj224 @andirayo While I don't have a great idea of what could cause this, it seems like we could model this as a failure to fetch tokens, ultimately resulting in an auth error bubbled up through the firebase product. Thoughts?

@franvis Everything I see points to this being a crash that we live with on Android, and supposedly fixed in Gingerbread!, clearly not the case here. How frequent are these crashes in your case ?

ankitaj224 on 30 Jun 2020

@franvis Are these app crashes or background thread crashes seen in crashllytics?

ankitaj224 on 30 Jun 2020

@ankitaj224 they are both. We see the crash in our crashlytics multiple times and every time close or around 50% of the devices are in background state and 50% not.

franvis on 30 Jun 2020

@ankitaj224 Can I help with something else? We keep seeing this as one of the biggest crashes in our logs :(

franvis on 3 Jul 2020

@franvis Extremely sorry for my delayed response due to other tasks at hand and also for this being the biggest crash.

First - this crash seems mostly due to Android okhttp error then Firebase-Installations SDK error. I will raise a bug internally with the Android team. However, I will still try to see how firebase-installations SDK can fail-safe that should avoid these crashes.

I tried but couldn't reproduce this locally. Did you or @Prichman able to reproduce this crash locally? If so, please provide the steps below on this issue.

Also, can you confirm that you are transitively using firebase-installations:16.3.1 version? If different, which version of firebase-installations is your app using.

ankitaj224 on 6 Jul 2020

Hello @ankitaj224, no problem at all! I can confirm that we have 16.3.1 for firebase-installations and also just in case we are having 16.0.0 of firebase-installations-interop. Unfortunately I couldn't reproduce it in my side. The only exception I see that most likely is not related is this one for crashlytics:

Caused by: java.net.SocketTimeoutException: failed to connect to settings.crashlytics.com/172.217.168.195 (port 443) from /192.168.1.103 (port 50248) after 10000ms
        at libcore.io.IoBridge.connectErrno(IoBridge.java:185)
        at libcore.io.IoBridge.connect(IoBridge.java:129)
        at java.net.PlainSocketImpl.socketConnect(PlainSocketImpl.java:137)
        at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:391)
        at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:231)
        at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:213)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:436)
        at java.net.Socket.connect(Socket.java:621)
        at com.android.okhttp.internal.Platform.connectSocket(Platform.java:145)
        at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:141)
        at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
        at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:184)
        at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:126)
        at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:95)
        at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:281)
        at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:224)
        at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:461)
        at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponse(HttpURLConnectionImpl.java:407)
        at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getResponseCode(HttpURLConnectionImpl.java:538)
        at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getResponseCode(DelegatingHttpsURLConnection.java:105)
        at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getResponseCode(HttpsURLConnectionImpl.java:26)
        at com.google.firebase.perf.network.zzf.getResponseCode(Unknown Source:81)
        at com.google.firebase.perf.network.zzc.getResponseCode(Unknown Source:15)

Would you confirm me that this is not related? Anything else or other info I can provide?

franvis on 8 Jul 2020

👍1

@ankitaj224 Alas. We can't reproduce it locally. I can only see these crash reports in console.

Prichman on 8 Jul 2020

👍1

@franvis The stack trace you posted doesn't seem related to crashes seen above. I have had no luck either reproducing it locally.

To rule out the possibilities, can you & @Prichman please attach stack traces of all threads as seen in Crashlytics. It will help me understand what's going on better.

ankitaj224 on 9 Jul 2020

Hello @ankitaj224,
Here is one crash with the first 5 Threads:

Fatal Exception: java.lang.NullPointerException: Attempt to invoke virtual method 'java.lang.Object com.android.org.conscrypt.SSLParametersImpl.clone()' on a null object reference
       at com.android.org.conscrypt.OpenSSLSocketFactoryImpl.createSocket(OpenSSLSocketFactoryImpl.java:135)
       at com.android.okhttp.internal.io.RealConnection.connectTls(RealConnection.java:1463)
       at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:1423)
       at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:1367)
       at com.android.okhttp.internal.http.StreamAllocation.findConnection(StreamAllocation.java:219)
       at com.android.okhttp.internal.http.StreamAllocation.findHealthyConnection(StreamAllocation.java:142)
       at com.android.okhttp.internal.http.StreamAllocation.newStream(StreamAllocation.java:104)
       at com.android.okhttp.internal.http.HttpEngine.connect(HttpEngine.java:410)
       at com.android.okhttp.internal.http.HttpEngine.sendRequest(HttpEngine.java:343)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.execute(HttpURLConnectionImpl.java:489)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.connect(HttpURLConnectionImpl.java:131)
       at com.android.okhttp.internal.huc.HttpURLConnectionImpl.getOutputStream(HttpURLConnectionImpl.java:262)
       at com.android.okhttp.internal.huc.DelegatingHttpsURLConnection.getOutputStream(DelegatingHttpsURLConnection.java:218)
       at com.android.okhttp.internal.huc.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeRequestBodyToOutputStream(FirebaseInstallationServiceClient.java:208)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.writeGenerateAuthTokenRequestBodyToOutputStream(FirebaseInstallationServiceClient.java:248)
       at com.google.firebase.installations.remote.FirebaseInstallationServiceClient.generateAuthToken(FirebaseInstallationServiceClient.java:369)
       at com.google.firebase.installations.FirebaseInstallations.fetchAuthTokenFromServer(FirebaseInstallations.java:495)
       at com.google.firebase.installations.FirebaseInstallations.doNetworkCall(FirebaseInstallations.java:343)
       at com.google.firebase.installations.FirebaseInstallations.lambda$doRegistrationInternal$0(FirebaseInstallations.java:331)
       at com.google.firebase.installations.FirebaseInstallations$$Lambda$5.run(FirebaseInstallations.java:4)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
       at java.lang.Thread.run(Thread.java:764)

       at java.lang.Object.wait(Object.java)
       at java.lang.Thread.parkFor$(Thread.java:2135)
       at sun.misc.Unsafe.park(Unsafe.java:358)
       at java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:230)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2101)
       at java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467)
       at java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1086)
       at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1147)
       at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
       at java.lang.Thread.run(Thread.java:764)



md5-8a956b4b8c1bc3c5445da74614159264



firebase-installations-executor-2
       at java.lang.Object.wait(Object.java)
       at java.lang.Thread.parkFor$(Thread.java:2135)
       at sun.misc.Unsafe.park(Unsafe.java:358)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:190)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.parkAndCheckInterrupt(AbstractQueuedSynchronizer.java:868)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1021)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer.acquireSharedInterruptibly(AbstractQueuedSynchronizer.java:1328)
       at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:232)
       at com.google.firebase.crashlytics.internal.common.Utils.awaitEvenIfOnMainThread(Utils.java:185)
       at com.google.firebase.crashlytics.internal.common.CrashlyticsController.handleUncaughtException(CrashlyticsController.java:447)
       at com.google.firebase.crashlytics.internal.common.CrashlyticsController$5.onUncaughtException(CrashlyticsController.java:365)
       at com.google.firebase.crashlytics.internal.common.CrashlyticsUncaughtExceptionHandler.uncaughtException(CrashlyticsUncaughtExceptionHandler.java:54)
       at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1068)
       at java.lang.ThreadGroup.uncaughtException(ThreadGroup.java:1063)
       at java.lang.Thread.dispatchUncaughtException(Thread.java:1953)



md5-8a956b4b8c1bc3c5445da74614159264



Thread-6
       at java.lang.Object.wait(Object.java)
       at java.lang.Thread.parkFor$(Thread.java:2135)
       at sun.misc.Unsafe.park(Unsafe.java:358)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:190)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2059)
       at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:548)
       at com.android.volley.CacheDispatcher.processRequest(CacheDispatcher.java:118)
       at com.android.volley.CacheDispatcher.run(CacheDispatcher.java:97)



md5-8a956b4b8c1bc3c5445da74614159264



Thread-7
       at java.lang.Object.wait(Object.java)
       at java.lang.Thread.parkFor$(Thread.java:2135)
       at sun.misc.Unsafe.park(Unsafe.java:358)
       at java.util.concurrent.locks.LockSupport.park(LockSupport.java:190)
       at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2059)
       at java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:548)
       at com.android.volley.NetworkDispatcher.processRequest(NetworkDispatcher.java:110)
       at com.android.volley.NetworkDispatcher.run(NetworkDispatcher.java:90)

Do you need me to post all the rest of them?

franvis on 10 Jul 2020

@ankitaj224 sorry to bother you again, but do you have any update on this matter? Do you need me to put the rest of the threads in the previous comment?

franvis on 17 Jul 2020

@franvis Please don't be. I am really sorry for the delay in my responses. I looked at the thread stack trace you posted but that dint help in identifying the root cause. I performed some testing on firebase-installations apis with different network settings across Android version mentioned in this bug and the results were as expected.

Also, I tried searching for similar reported issues but found just 1 issue as Ashwin previously pointed out in this comment.
To which you mentioned that your app doesn't update any System properties.

I am still talking to other Engineers in Firebase to see if they have any ideas about how to reproduce or handle this issue (if there are any reasonable workarounds). Also, I am trying to identify the Android team that maintains this library and raise a bug with that team as well.

I will keep you posted as I make progress with finding the root cause/ a workaround.

Previously, you mentioned the crashes for this issue had increased. Can you please share the stats on that?

ankitaj224 on 20 Jul 2020

Hello @ankitaj224. I'm sorry to say but, it is hard to see a trend in crashlytics since this same crash is reported as different ones for each version of Android. If I sum up those crashes and compare them to our previous version there's no significant movement in the trend. Quick update on the firebase dependencies just in case: We removed dynamic links (We didn't need it anymore), but that didn't changed anything related to this. Please keep me updated if you find something, I will try to investigate a bit more and do the same.

franvis on 20 Jul 2020

Any movement on this? It’s been the highest cause of crashes in my app by a large margin for the past month.

StainlessStlRat on 15 Aug 2020

After updating our google-services.json by removing an old SHA-256 that we had in the configuration of our firebase project and downloading the json file again, this crash went away. For now, we don't see it anymore. Unfortunately I'm not sure I can recommend this solution because thanks to doing that we increase terribly our firebase config errors when fetching our config. We use to have ~400 errors per day, now we have ~11K per day. The follow-up of that problem can be found in https://github.com/firebase/firebase-android-sdk/issues/1776

franvis on 25 Aug 2020

I will leave this open unless @ankitaj224 believes that what I did should be the recommended solution.

franvis on 25 Aug 2020

Thanks @franvis for reporting the steps you followed and how that helped your app with this crash. I am not sure how your fix translates to fixing this crash exactly.

Before recommending updating SHA-256, I want to find out from other developers who are seeing this issue if they have multiple SHA256 configured for their projects as well.

@StainlessStlRat What % of your requests are crashing with this error? And can you please verify if you have one/multiple SHA-256 hash configured for your Firebase project.

ankitaj224 on 25 Aug 2020

Android team responded that they are working on a fix for conscrypt lib which will be available in latest builds but it wont get backported to Android 7 & 8.

For a potential fix: please refer to https://github.com/firebase/firebase-android-sdk/issues/1626#issuecomment-679868746

As there haven't been any new reports, I will be closing this issue. Please feel free to reopen if anyone continues to run into this issue.

Thanks.

ankitaj224 on 21 Oct 2020

I did everything that was mentioned as a potential fix but nothing helped.
Then I found that all similar crashes happen because of changed "BC" security provider with external BouncyCastle.
Our application uses BouncyCastle library to support ECDSA that is not available in native android BC provider, that why we had something like
Security.removeProvider("BC"); Security.addProvider(new BouncyCastleProvider());

When I refactored code to use BouncyCastleProvider instance instead of registering it as a system provider issue has gone.
Hope it helps others. Just check you are not replacing native BC provider with external.