Apps-android-commons: android.app.RemoteServiceException crash reports on Play Store

[misaochan]

This crash has 80 reports on the Play Store, and is our highest crash besides #1876 . Unfortunately, I have no idea what is causing it because the stack trace below is all we have. :/

android.app.RemoteServiceException: 
  at android.app.ActivityThread$H.handleMessage (ActivityThread.java:1867)
  at android.os.Handler.dispatchMessage (Handler.java:106)
  at android.os.Looper.loop (Looper.java:164)
  at android.app.ActivityThread.main (ActivityThread.java:6753)
  at java.lang.reflect.Method.invoke (Native Method)
  at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run (RuntimeInit.java:482)
  at com.android.internal.os.ZygoteInit.main (ZygoteInit.java:807)

[misaochan]

Extra info: This crash only started on 21 Aug, which is when 2.8.2 was rolled out to production. 100% of reports are from v2.8.2. Bad luck that we never saw it in beta I guess.

Android version:
Android 8.1 | 123 | 69.5%
28 | 54 | 30.5%

Devices:
Pixel (sailfish) | 38 | 21.5%
Nokia 6.1 (PL2_sprout) | 21 | 11.9%
OnePlus 6 (OnePlus6) | 17 | 9.6%
Mi A1 (tissot_sprout) | 15 | 8.5%
Others | 86 | 48.6%

[misaochan]

This is really strange, the only 2 PRs for 2.8.2 were https://github.com/commons-app/apps-android-commons/commit/4c476e7a064102599daad0b59ce538de752ec77a and https://github.com/commons-app/apps-android-commons/commit/11c3772dd003287cd93ba0d35201e7b6cd759ff5 . I really wonder where the issue lies. Do we want to try removing Traceur and seeing if that works? @neslihanturan @maskaravivek

[neslihanturan]

Removing other commit wouldn't be that easy, because it already solves a visible problem. Fastest solution would be removing Traceur indeed.

[maskaravivek]

Traceur only makes the logs more verbose. Without Traceur these logs would not be decipherable. Can we check the number of active installations for 2.8.1 and 2.8.0. 2.8.2 was released quickly after 2.8.1/2.8.0 so many people might not have used 2.8.0. Also was 2.8.0/2.8.1 ever pushed to prod or was it just pushed to beta channel?

[misaochan]

@maskaravivek Hmm, good point. Unfortunately the Play console does not give me active installs for every version, we can only see the total number of install events for every version.

2.8.1 was released to beta on 19 Aug and prod on 20 Aug, so it is indeed possible that 2.8.1 was not tested substantially. It has 594 install events.

2.8.0 could potentially be the culprit as it was never released to prod. However, it was released to beta on 8 Aug. Could just be bad luck that it never showed up on beta though despite having 600+ beta testers and 2 weeks of availability. :/

That is actually really bad news because it widens the search substantially, and the stack trace itself provides almost no useful information.

[neslihanturan]

Maybe this crash was already there before Traceur too, but after we started to get this log?

[maskaravivek]

I just came across a Reddit thread and it seems that Traceur could cause some crashes in edge scenarios.

Maybe we could revert it for now. The same thread suggests the usage of RxJava2Debug but we can consider using it later.

@dbrant Do you have any suggestions here? Basically we want verbose stacktraces for RxJava exceptions.

[misaochan]

OK, I'll set Traceur to only work in debug perhaps? And then we can reenable later or switch to another one?

[ashishkumar468]

Can someone confirm, if this happening only post Nougat or do we have instances pre oreo as well? (Android has updated the way the services are to be handled starting from oreo) and similar crashes are thrown in such scenarios.

[misaochan]

@ashishkumar0207 Thanks for the insight. 100% of the cases are from Android 8.1 and above, so only Oreo. Is there a way to find out which line of code is triggering this crash? The stack trace is supremely unhelpful. :/

[ashishkumar468]

@misaochan I will try to reproduce it and if Its related to what I am suspecting, I should be able to, will update if something comes up. Meanwhile, it will be great if someone else can also try to reproduce it in oreo devices.

[misaochan]

Sounds good. I wonder why Android 8.0 (also Oreo) never seems to experience this crash though. As seen above only users with 8.1 and Pie are experiencing it. I also use the app a fair bit on my 8.0 device and have never gotten this crash.

[misaochan]

I can confirm that removing Traceur did not solve the problem. I guess we can reenable it while we investigate further.

[misaochan]

Has anyone had any luck with this? I have a bit more information from the dev console now:

Reports
60 on last 7 days
Reports total
87
Impacted users
22

Android 8.1 | 46 | 76.7%
Android 9 | 14 | 23.3%

Redmi Note 5 Pro (whyred) | 14 | 23.3%
Mi A1 (tissot_sprout) | 12 | 20.0%
Pixel 2 XL (taimen) | 11 | 18.3%
Nokia 6 (PLE) | 5 | 8.3%
OnePlus 6 (OnePlus6) | 4 | 6.7%
P20 Pro (HWCLT) | 4 | 6.7%
OnePlus5 (OnePlus5) | 4 | 6.7%
Pixel 2 (walleye) | 3 | 5.0%
Redmi S2 (ysl) | 3 | 5.0%

[misaochan]

Based on this review:

Not at all a single picture getting uploaded.

Device Mi A1 (tissot_sprout)
App version 90 (2.8.3)
OS Android 8.1

and this one:

Auto-translated from Spanish - show original review
ERROR TERRIBLE !!!! Every time I try to upload a photo the application closes again and again ... So what is the application for if you can not upload files? My phone is an Android 8.1
Device Mi A2 (jasmine_sprout)
App version 89 (2.8.2)
OS Android 8.1

I think a reasonable assumption would be that the crash happens while uploading on an Android 8.1 and above. Will try to confirm soon.

[misaochan]

Another report:

Through gallery when I upload, after writing name and description once I click on upload, it crashes 100% of time. Just tested this only with one file.

[misaochan]

I have been unable to reproduce this on an API 27 (Android 8.1.0) Nexus S emulator running prodDebug of the current master. :/ I tested uploading from the app and uploading from the gallery (Google Photos), both succeeded.

Might need to test with prodRelease next to see if that is the issue, or if the problem just happens to specific users.

[misaochan]

I have finally been able to reproduce this - it needs to be run on prodRelease, prodDebug does not encounter similar issues. The crash occurs when you tap the arrow in the action bar of ShareActivity to submit the image after filling in the title/desc, and it is 100% reproducible unfortunately! It also happens regardless of the upload method, whether from gallery or from in-app.

09-27 01:42:19.772 8206-8206/fr.free.nrw.commons E/AndroidRuntime: FATAL EXCEPTION: main
    Process: fr.free.nrw.commons, PID: 8206
    android.app.RemoteServiceException: Bad notification for startForeground: java.lang.RuntimeException: invalid channel for service notification: Notification(channel=CommonsNotificationAll pri=0 contentView=null vibrate=null sound=null tick defaults=0x0 flags=0x52 color=0x00000000 vis=PRIVATE)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1768)
        at android.os.Handler.dispatchMessage(Handler.java:106)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6494)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

[misaochan]

Changing target SDK to 26 solves the problem temporarily. Obviously we should target the latest SDK, but I will release a hotfix with this for the time being, so that users will stop crashing.

This error in the logs is probably related:

09-27 01:51:17.278 1615-1630/system_process E/NotificationService: No Channel found for pkg=fr.free.nrw.commons, channelId=CommonsNotificationAll, id=1, tag=null, opPkg=fr.free.nrw.commons, callingUid=10080, userId=0, incomingUserId=0, notificationUid=10080, notification=Notification(channel=CommonsNotificationAll pri=0 contentView=null vibrate=null sound=null tick defaults=0x0 flags=0x52 color=0x00000000 vis=PRIVATE)

[misaochan]

Reopening due to issues with getting fixes released in #1907 .

[misaochan]

The status of this issue is that we currently have the ideal fix working in master, and the target SDK hotfix pushed to 2.8.4. Either way, users should hopefully not crash anymore. Master should be released in 2.9 in the coming weeks.