Fenix: [Bug] [Crash] Nightly: Incompatible megazord version : library "places" was compiled expecting app-services version "0.52.0", but the megazord "megazord" provides version "0.53.0"

[agashlin]

Steps to reproduce

Install build 20200228092428, currently offered from the Play Store

Actual behavior

Crash when attempting to navigate or enter text in the address bar.

Device information

• Android device: Google Pixel 3a
• Fenix version: Nightly 200229 18:01 (Build # 20601807) 35.0.0, 0dd4e8df2, GV: 75.0a1-20200228092428

Here's a stack from a crash report. It seems to be the same as issue #8878, but this is the currently shipping build.

java.lang.NoClassDefFoundError: mozilla.appservices.places.LibPlacesFFI
    at mozilla.appservices.places.PlacesApi.<init>(PlacesConnection.kt:6)
    at mozilla.components.browser.storage.sync.RustPlacesConnection.init(Connection.kt:3)
    at mozilla.components.browser.storage.sync.PlacesStorage$places$2.invoke(PlacesStorage.kt:3)
    at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
    at mozilla.components.browser.storage.sync.PlacesStorage.getPlaces$browser_storage_sync_release(Unknown Source:7)
    at mozilla.components.browser.storage.sync.PlacesStorage$reader$2.invoke(PlacesStorage.kt:1)
    at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
    at mozilla.components.browser.storage.sync.PlacesStorage.getReader$browser_storage_sync_release(Unknown Source:7)
    at mozilla.components.browser.storage.sync.PlacesBookmarksStorage$searchBookmarks$2.invokeSuspend(PlacesBookmarksStorage.kt:3)
    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:15)
    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:15)
    Suppressed: java.lang.NoClassDefFoundError: mozilla.appservices.places.LibPlacesFFI
        at mozilla.appservices.places.PlacesApi.<init>(PlacesConnection.kt:6)
        at mozilla.components.browser.storage.sync.RustPlacesConnection.init(Connection.kt:3)
        at mozilla.components.browser.storage.sync.PlacesStorage$places$2.invoke(PlacesStorage.kt:3)
        at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
        at mozilla.components.browser.storage.sync.PlacesStorage.getPlaces$browser_storage_sync_release(Unknown Source:7)
        at mozilla.components.browser.storage.sync.PlacesHistoryStorage.getSuggestions(PlacesHistoryStorage.kt:1)
        at mozilla.components.feature.awesomebar.provider.HistoryStorageSuggestionProvider.onInputChanged(HistoryStorageSuggestionProvider.kt:3)
        at mozilla.components.browser.awesomebar.BrowserAwesomeBar$onInputChanged$1.invokeSuspend(BrowserAwesomeBar.kt:2)
        at mozilla.components.browser.awesomebar.BrowserAwesomeBar$onInputChanged$1.invoke(Unknown Source:10)
        at mozilla.components.browser.awesomebar.BrowserAwesomeBar$queryProvidersForSuggestions$1$invokeSuspend$$inlined$forEach$lambda$1$1.invokeSuspend(BrowserAwesomeBar.kt:2)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:15)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1167)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:641)
        at java.lang.Thread.run(Thread.java:919)
    Caused by: java.lang.ExceptionInInitializerError
        at mozilla.appservices.places.LibPlacesFFI.<clinit>(Unknown Source:0)
        at mozilla.appservices.places.PlacesApi.<init>(PlacesConnection.kt:6)
        at mozilla.components.browser.storage.sync.RustPlacesConnection.init(Connection.kt:3)
        at mozilla.components.browser.storage.sync.PlacesStorage$places$2.invoke(PlacesStorage.kt:3)
        at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
        at mozilla.components.browser.storage.sync.PlacesStorage.getPlaces$browser_storage_sync_release(Unknown Source:7)
        at mozilla.components.browser.storage.sync.PlacesHistoryStorage.getHandle(PlacesHistoryStorage.kt:1)
        at mozilla.components.service.fxa.sync.WorkManagerSyncWorker.doSync(WorkManagerSyncManager.kt:8)
        at mozilla.components.service.fxa.sync.WorkManagerSyncWorker$doWork$2.invokeSuspend(WorkManagerSyncManager.kt:2)
        at mozilla.components.service.fxa.sync.WorkManagerSyncWorker$doWork$2.invoke(Unknown Source:10)
        at mozilla.components.service.sync.logins.SyncableLoginsStore.unlocked(AsyncLoginsStorage.kt:6)
        at mozilla.components.service.sync.logins.SyncableLoginsStore$unlocked$1.invokeSuspend(Unknown Source:12)
        at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
        at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:15)
        at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
        at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:15)
    Caused by: mozilla.appservices.support.native.IncompatibleMegazordVersion: Incompatible megazord version: library "places" was compiled expecting app-services version "0.52.0", but the megazord "megazord" provides version "0.53.0"
        at mozilla.appservices.support.native.HelpersKt.lookupMegazordLibrary(Helpers.kt:13)
        at mozilla.appservices.support.native.HelpersKt.findMegazordLibraryName(Helpers.kt:4)
        at mozilla.appservices.places.LibPlacesFFI$Companion.<clinit>(LibPlacesFFI.kt:2)
        ... 16 more
    [CIRCULAR REFERENCE:java.lang.ExceptionInInitializerError]

[djc]

On my Pixel 2 I also see very quick crashing when trying to open a new tab or navigating in the current one.

[agashlin]

Current release seems ok:

Nightly 200301 06:01 (Build #20610607)
35.0.0, 0dd4e8df2
GV: 75.0a1-20200228092428
Sunday 3/1 @ 6:06 AM

Leaving open in case someone wants to look into the root cause.

[lnicola]

It's not, for me it still crashes when loading a webpage. I checked that it's the same version. EDIT: my bad, I'm running the older version. I checked the git tag (0dd4e8df2) and the build id, but didn't notice the difference (20601807 vs. 20610607).

This comes up time and time again: #7608, #6916, #6411, #4532. It should have been fixed by #8742, but apparently not. I'm not sure if there's any CI for Fenix, it should normally catch such issues.

Stack trace showing the same error:

03-01 09:20:06.133  5102  5102 E AndroidRuntime: FATAL EXCEPTION: main
03-01 09:20:06.133  5102  5102 E AndroidRuntime: Process: org.mozilla.fenix.nightly, PID: 5102
03-01 09:20:06.133  5102  5102 E AndroidRuntime: java.lang.NoClassDefFoundError: mozilla.appservices.places.LibPlacesFFI
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.places.PlacesApi.<init>(PlacesConnection.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.RustPlacesConnection.init(Connection.kt:3)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage$places$2.invoke(PlacesStorage.kt:3)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage.getPlaces$browser_storage_sync_release(Unknown Source:7)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage$reader$2.invoke(PlacesStorage.kt:1)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage.getReader$browser_storage_sync_release(Unknown Source:7)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesBookmarksStorage$getBookmarksWithUrl$2.invokeSuspend(PlacesBookmarksStorage.kt:3)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlin.coroutines.jvm.internal.BaseContinuationImpl.resumeWith(ContinuationImpl.kt:2)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlinx.coroutines.DispatchedTask.run(DispatchedTask.kt:15)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlinx.coroutines.scheduling.CoroutineScheduler.runSafely(CoroutineScheduler.kt:1)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlinx.coroutines.scheduling.CoroutineScheduler$Worker.run(CoroutineScheduler.kt:15)
03-01 09:20:06.133  5102  5102 E AndroidRuntime: Caused by: java.lang.ExceptionInInitializerError
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.places.LibPlacesFFI.<clinit>(Unknown Source:0)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.places.PlacesApi.<init>(PlacesConnection.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.RustPlacesConnection.init(Connection.kt:3)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage$places$2.invoke(PlacesStorage.kt:3)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at kotlin.SynchronizedLazyImpl.getValue(LazyJVM.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesStorage.getPlaces$browser_storage_sync_release(Unknown Source:7)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.browser.storage.sync.PlacesHistoryStorage.getHandle(PlacesHistoryStorage.kt:1)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.service.fxa.sync.WorkManagerSyncWorker.doSync(WorkManagerSyncManager.kt:8)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.service.fxa.sync.WorkManagerSyncWorker$doWork$2.invokeSuspend(WorkManagerSyncManager.kt:2)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.service.fxa.sync.WorkManagerSyncWorker$doWork$2.invoke(Unknown Source:10)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.service.sync.logins.SyncableLoginsStore.unlocked(AsyncLoginsStorage.kt:6)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.components.service.sync.logins.SyncableLoginsStore$unlocked$1.invokeSuspend(Unknown Source:12)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    ... 4 more
03-01 09:20:06.133  5102  5102 E AndroidRuntime: Caused by: mozilla.appservices.support.native.IncompatibleMegazordVersion: Incompatible megazord version: library "places" was compiled expecting app-services version "0.52.0", but the megazord "megazord" provides version "0.53.0"
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.support.native.HelpersKt.lookupMegazordLibrary(Helpers.kt:13)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.support.native.HelpersKt.findMegazordLibraryName(Helpers.kt:4)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    at mozilla.appservices.places.LibPlacesFFI$Companion.<clinit>(LibPlacesFFI.kt:2)
03-01 09:20:06.133  5102  5102 E AndroidRuntime:    ... 16 more

[kinghat]

update just came through(Nightly 200301 06:01) and I'm back to normal.

[djc]

200301 is not available for me from the Play Store. 200229 still crashes.

[kinghat]

200301 is not available for me from the Play Store. 200229 still crashes.

just wait longer 👌

[ekager]

We recently removed the direct AS dependency in Fenix (trying to avoid these mismatches) and subsequently updated to AS53 in AC.
https://github.com/mozilla-mobile/fenix/pull/8742
https://github.com/mozilla-mobile/android-components/pull/6110

Even though it fixed itself in the next nightly, we should investigate how we were getting the mismatch still.

[sflorean]

@agashlin this is still reproducible for you on the latest Nightly? Wasn't able to reproduce on Nightly 10/29 with Pixel 3 (Android 11).

[agashlin]

No, I haven't seen this since it was fixed by the next Nightly back in February. I think this was only left open to investigate the root cause of the mismatches.