Bazel: Android app cannot load NDK native library on API 22 with Bazel 0.19.0

[Steve-Munday]

Description of the problem:

Our Android app heavily uses C++ 11 code, with the native library built using Bazel and the NDK 16b. The final app binary is built using Android Studio. Our minSDK is 22. We found that when we upgraded Bazel from 0.16.0 to 0.19.0, that devices running our API 22 could not load native libraries, and would crash on launch, with the following stack trace:

12-17 17:57:58.183 29937-29937/com.company.appname E/art: dlopen("/data/app/com.company.appname-1/lib/arm/libdorothy.so", RTLD_LAZY) failed: dlopen failed: cannot locate symbol "dladdr" referenced by "libdorothy.so"...
12-17 17:57:58.195 29937-29937/com.company.appname E/AndroidRuntime: FATAL EXCEPTION: main
    Process: com.company.appname, PID: 29937
    java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "dladdr" referenced by "libdorothy.so"...
        at java.lang.Runtime.loadLibrary(Runtime.java:371)
        at java.lang.System.loadLibrary(System.java:988)
        at com.company.app.appApplication.<clinit>(appApplication.java:51)
        at java.lang.reflect.Constructor.newInstance(Native Method)
        at java.lang.Class.newInstance(Class.java:1606)
        at android.app.Instrumentation.newApplication(Instrumentation.java:1000)
        at android.app.Instrumentation.newApplication(Instrumentation.java:985)
        at android.app.LoadedApk.makeApplication(LoadedApk.java:567)
        at android.app.ActivityThread.handleBindApplication(ActivityThread.java:4831)
        at android.app.ActivityThread.access$1500(ActivityThread.java:178)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1531)
        at android.os.Handler.dispatchMessage(Handler.java:111)
        at android.os.Looper.loop(Looper.java:194)
        at android.app.ActivityThread.main(ActivityThread.java:5637)
        at java.lang.reflect.Method.invoke(Native Method)
        at java.lang.reflect.Method.invoke(Method.java:372)
        at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:959)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:754)

This error only occurs on devices running Lollipop. I thought it might be related to dropping support for armeabi architectures, but our test device is a Lenovo Tab 2, which has a 64-bit CPU and should be armeabi-v7a. Maybe this is related to changes in the STL from 0.17.0?

Bugs: what's the simplest, easiest way to reproduce this bug? Please provide a minimal example if possible.

I don't have a minimal example, but I believe it should be reproducible on any Android app with minSDK=22 and bazel 0.19.0.

What operating system are you running Bazel on?

MacOS Mojave 10.14
Test Android Device: Lenovo Tab 2

What's the output of bazel info release?

0.19.0

Have you found anything relevant by searching the web?

This NDK doc has some tips as to what might be going on:

https://developer.android.com/training/articles/perf-jni#faq-why-do-i-get-unsatisfiedlinkerror

but I don't think this is the answer. The library is there as it loads successfully in API 23+.
Other than that I can't find ANYTHING where the runtime cannot load symbol dladdr.

Any other information, logs, or outputs that you want to share?

Relevant bits from .bazelrc:

build --copt=-Wall --copt=-Wextra --cxxopt=-std=c++11
build --spawn_strategy=standalone
build --genrule_strategy=standalone
build --android_crosstool_top=@androidndk//:toolchain-libcpp
build --fat_apk_cpu=armeabi,armeabi-v7a

[jin]

Reproduced with Bazel 0.20.0, NDK r16b and the NDK app in Bazel's examples repository on an Lollipop x86 emulator.

E/art     ( 3658): dlopen("/data/app/com.example.android.bazel-1/lib/x86/libapp.so", RTLD_LAZY) failed: dlopen failed: cannot locate symbol "dl_iterate_phdr" referenced by "libapp.so"...
E/AndroidRuntime( 3658): FATAL EXCEPTION: main
E/AndroidRuntime( 3658): Process: com.example.android.bazel, PID: 3658
E/AndroidRuntime( 3658): java.lang.UnsatisfiedLinkError: dlopen failed: cannot locate symbol "dl_iterate_phdr" referenced by "libapp.so"...
E/AndroidRuntime( 3658):    at java.lang.Runtime.loadLibrary(Runtime.java:371)
E/AndroidRuntime( 3658):    at java.lang.System.loadLibrary(System.java:988)
E/AndroidRuntime( 3658):    at com.example.android.bazel.MainActivity.<clinit>(MainActivity.java:10)
E/AndroidRuntime( 3658):    at java.lang.reflect.Constructor.newInstance(Native Method)
E/AndroidRuntime( 3658):    at java.lang.Class.newInstance(Class.java:1606)
E/AndroidRuntime( 3658):    at android.app.Instrumentation.newActivity(Instrumentation.java:1066)
E/AndroidRuntime( 3658):    at android.app.ActivityThread.performLaunchActivity(ActivityThread.java:2226)
E/AndroidRuntime( 3658):    at android.app.ActivityThread.handleLaunchActivity(ActivityThread.java:2387)
E/AndroidRuntime( 3658):    at android.app.ActivityThread.access$800(ActivityThread.java:151)
E/AndroidRuntime( 3658):    at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1303)
E/AndroidRuntime( 3658):    at android.os.Handler.dispatchMessage(Handler.java:102)
E/AndroidRuntime( 3658):    at android.os.Looper.loop(Looper.java:135)
E/AndroidRuntime( 3658):    at android.app.ActivityThread.main(ActivityThread.java:5254)
E/AndroidRuntime( 3658):    at java.lang.reflect.Method.invoke(Native Method)
E/AndroidRuntime( 3658):    at java.lang.reflect.Method.invoke(Method.java:372)
E/AndroidRuntime( 3658):    at com.android.internal.os.ZygoteInit$MethodAndArgsCaller.run(ZygoteInit.java:903)
E/AndroidRuntime( 3658):    at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:698)

[jin]

Maybe this is related to changes in the STL from 0.17.0?

Just verified that 0.18.0 produces an APK that triggers the UnsatisfiedLinkError. 0.17.2 is fine.

[jin]

Bisected to 2f281960b829e964526a9d292d4c3003e4d19f1c. This commit stops hardcoding -static-libgcc in the link actions. What's curious here is that if I manually set it as a linkopt, it still doesn't work.

[jin]

What happens if you build the app with --linkopt=-ldl, or have a linkopts = ["-ldl"] attribute on your cc_library?

You may also need to set android_ndk_repository.api_level to 22, on top of the -ldl linkopt:

android_ndk_repository(
    name = "androidndk",
    api_level = 22,
)

Using these, I managed to install the Android NDK example app onto an x86 Lollipop emulator.

[Steve-Munday]

Ok thanks @jin I will try that and report back soon.

[Steve-Munday]

I was also able to install and run our app on an API 22 device successfully. Thanks again @jin!

[jin]

Cool! Looks like this is a case of a missing link option for libdl. I'll close this now.