Aws-sdk-cpp: InitiAPI() getting crashed in C++ AWS Android SDK when consumed in Android apk (Need help)

[salunkhesagar1386]

While executing the Aws::InitAPI(Aws::SDKOptions) on Android the application is getting crashed.
Same code works fine for Windows.

I am simplifying the explanation by mentioning the exact steps I followed.

_Note: The SWIG part for generating the JNI Wrapper is test OK and its working fine. SO you can ignore that part while analysis._

Following are the steps that I have followed to create my own cross platform static library project that consumes required AWS SDK libraries.

• Built the AWS C++ SDK libraries using CMake for Android and Windows
• Created a cross platform C++ static library project in VS2017.
• My static lib project exposes only one API e.g. 'MyAWSWrapper::listS3Bucket()'.
• Added all required dependent AWS Libraries dependencies in project setting for each platform in VS
• Built the project for Android and Windows which resulted into a '.so' and '.lib' files.

Following is what I did for Android on top of above steps:

• Generated JNI Wrapper files for my exposed C++ static library interface using 'SWIG'
• Created a Android wrapper project which consumes the AWS Android SDK '.so' files and one mine '.so' files
• Result of this Android wrapper project is a '.aar'
• Created a single butoon Android UI project which makes calls to this C++ library through Wrapper
• The implementation class of C++ interface gets called successfully and when reached at line wher I have made call to Aws::InitAPI(Aws::SDKOptions), application crashesh. The crash log is pasted at bottom.

Following is what I did for Windows on top of steps mentioned on first bullet:

• Created a C++ Windows console application
• Linked the required AWS SDK '.lib' and my '.lib' libraries in 'Linker -> Input ->Additional dependencies'
• Project got built successfully.
• While launch it was expecting the 'dll' files for all libraries that I had added in 'Linker -> Input ->Additional dependencies' VS project.
• Then I copied all the dll's where the C++ console application's executable was residing.
• Built the console application again
• Ran the application and call for Aws::InitAPI(Aws::SDKOptions) succeeded with no error.
• Further to this wrote code to List the S3 buckets inside same library which used AWS S3 SDK, and succeeded without any error.

_Why is Android failing to initialize? Have I missed anything while building the Android SDK libraries?_

What platform/OS are you using?

Android

What compiler are you using? what version?

Clang 5.0

What's your CMake arguments?

cmake -DCUSTOM_MEMORY_MANAGEMENT=0 -DBUILD_ONLY="cognito-identity;core;identity-management;s3;sts" -DTARGET_ARCH=ANDROID -DANDROID_ABI=armeabi-v7a ~/git/aws-sdk-cpp

Can you provide a TRACE level log? (sanitize any sensitive information)

--------- beginning of crash
2019-03-01 17:10:23.029 22640-22640/com.example.sample.awstester A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x1 in tid 22640 (sample.awstester)
2019-03-01 17:10:23.078 22718-22718/? E/propClient: PropClient failed to load
2019-03-01 17:10:23.110 22719-22719/? I/crash_dump32: obtaining output fd from tombstoned
2019-03-01 17:10:23.111 1191-1191/? I//system/bin/tombstoned: received crash request for pid 22640
2019-03-01 17:10:23.113 22719-22719/? I/crash_dump32: performing dump of process 22640 (target tid = 22640)
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: * *
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: Build fingerprint: 'Verizon/hero2qltevzw/hero2qltevzw:8.0.0/R16NW/G935VVRS8CRK2:user/release-keys'
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: Revision: '15'
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: ABI: 'arm'
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: pid: 22640, tid: 22640, name: sample.awstester >>> com.example.sample.awstester <<<
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x1
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: Cause: null pointer dereference
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: r0 00000001 r1 00000001 r2 d13ea1a4 r3 00000041
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: r4 ef9c9234 r5 00000000 r6 00000000 r7 ffd0d7f0
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: r8 0000004a r9 eef74000 sl ffd0d850 fp ffd0d548
2019-03-01 17:10:23.114 22719-22719/? A/DEBUG: ip d13dcbe4 sp ffd0d540 lr d102ddbc pc d114eac8 cpsr 600f0010
2019-03-01 17:10:23.153 22719-22719/? A/DEBUG: backtrace:
2019-03-01 17:10:23.153 22719-22719/? A/DEBUG: #00 pc 001f9ac8 /data/app/com.example.sample.awstester-K9sGZKePIlELp6UedDMOPw==/lib/arm/libaws-cpp-sdk-core.so (_ZN3Aws5Utils6Memory25InitializeAWSMemorySystemERNS1_21MemorySystemInterfaceE+84)
2019-03-01 17:10:23.153 22719-22719/? A/DEBUG: #01 pc 000d8db8 /data/app/com.example.sample.awstester-K9sGZKePIlELp6UedDMOPw==/lib/arm/libaws-cpp-sdk-core.so (_ZN3Aws7InitAPIERKNS_10SDKOptionsE+44)
2019-03-01 17:10:23.153 22719-22719/? A/DEBUG: #02 pc 00069b95 /data/app/com.example.sample.awstester-K9sGZKePIlELp6UedDMOPw==/lib/arm/libAWSServicesProviderD.so (_ZN19AWSServicesProviderC1Ev+56)
2019-03-01 17:10:23.153 22719-22719/? A/DEBUG: #03 pc 000695f5 /data/app/com.example.sample.awstester-K9sGZKePIlELp6UedDMOPw==/lib/arm/libAWSServicesProviderD.so (Java_com_testdomain_AWSServicesProviderWrapper_AWSServicesProviderWrapperJNI_new_1AWSServicesProvider+40)

[marcomagdy]

The stack trace points to this function:

void InitializeAWSMemorySystem(MemorySystemInterface& memorySystem)
{
    #ifdef USE_AWS_MEMORY_MANAGEMENT
        if(AWSMemorySystem != nullptr)
        {
            AWSMemorySystem->End();
        }

        AWSMemorySystem = &memorySystem;
        AWSMemorySystem->Begin();
    #else
        AWS_UNREFERENCED_PARAM(memorySystem);
    #endif // USE_AWS_MEMORY_MANAGEMENT
}

Which is only called if you specify a memory manager in the SDKOptions. So, try disabling that and see if that's the problem.
If you're not using a memory manager, then I suspect something is broken in your packaging, as in the wrong lib/so is getting loaded.

[salunkhesagar1386]

Thank you so much @marcomagdy for your inputs. I made sure that no custom options are set.

Verified following,

1. My CMake file CUSTOM_MEMORY_MANAGEMENT was not set, so I take that it as FALSE by default.
2. This what I have done in my code related to SDKOptions.
   _Aws::SDKOptions m_SDKOptions;
   InitAPI(m_SDKOptions);_

I want to build the Android SDKs for ARMv7s and ARM abi's. Took care of this while building the SDKs too.

What am I missing?

[salunkhesagar1386]

After using this -DCUSTOM_MEMORY_MANAGEMENT=0 and re-generating the Android libraries, the error got changed. Now the application is getting crashed in

/data/app/com.example.sample.awstester-euMjrYwTpQTNqxTzSww1zQ==/lib/arm/libaws-cpp-sdk-core.so (_ZNKSt6__ndk18functionIFNS_10shared_ptrIN3Aws5Utils6Crypto22SymmetricCipherFactoryEEEvEEclEv+24)

following is the backtrace of crash,

--------- beginning of crash

2019-02-14 14:49:21.433 5775-5775/com.example.sample.awstester A/libc: Fatal signal 11 (SIGSEGV), code 1, fault addr 0x7 in tid 5775 (sample.awstester)
2019-02-14 14:49:21.467 6395-6395/? E/propClient: PropClient failed to load
2019-02-14 14:49:21.492 6396-6396/? I/crash_dump32: obtaining output fd from tombstoned
2019-02-14 14:49:21.496 959-959/? I//system/bin/tombstoned: received crash request for pid 5775
2019-02-14 14:49:21.497 6396-6396/? I/crash_dump32: performing dump of process 5775 (target tid = 5775)
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: * *
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: Build fingerprint: 'samsung/star2qlteue/star2qlteue:8.0.0/R16NW/G965U1UEU3ARL1:user/release-keys'
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: Revision: '14'
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: ABI: 'arm'
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: pid: 5775, tid: 5775, name: sample.awstester >>> com.example.sample.awstester <<<
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: signal 11 (SIGSEGV), code 1 (SEGV_MAPERR), fault addr 0x7
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: Cause: null pointer dereference
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: r0 ffab9084 r1 00000007 r2 00000004 r3 00000000
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: r4 ffab9138 r5 00000000 r6 00000000 r7 ffab9270
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: r8 0000004a r9 ed27b000 sl ffab92d0 fp ffab8fc8
2019-02-14 14:49:21.498 6396-6396/? A/DEBUG: ip d002e22c sp ffab8fc0 lr cfc8b210 pc cfc8b65c cpsr 200f0010
2019-02-14 14:49:21.512 6396-6396/? A/DEBUG: backtrace:
2019-02-14 14:49:21.512 6396-6396/? A/DEBUG: #00 pc 000c665c /data/app/com.example.sample.awstester-euMjrYwTpQTNqxTzSww1zQ==/lib/arm/libaws-cpp-sdk-core.so (_ZNKSt6__ndk18functionIFNS_10shared_ptrIN3Aws5Utils6Crypto22SymmetricCipherFactoryEEEvEEclEv+24)
2019-02-14 14:49:21.512 6396-6396/? A/DEBUG: #01 pc 000c620c /data/app/com.example.sample.awstester-euMjrYwTpQTNqxTzSww1zQ==/lib/arm/libaws-cpp-sdk-core.so (_ZN3Aws7InitAPIERKNS_10SDKOptionsE+1380)
2019-02-14 14:49:21.512 6396-6396/? A/DEBUG: #02 pc 00069b45 /data/app/com.example.sample.awstester-euMjrYwTpQTNqxTzSww1zQ==/lib/arm/libAWSServicesProviderD.so (_ZN19AWSServicesProviderC1Ev+72)
2019-02-14 14:49:21.512 6396-6396/? A/DEBUG: #03 pc 00069595 /data/app/com.example.sample.awstester-euMjrYwTpQTNqxTzSww1zQ==/lib/arm/libAWSServicesProviderD.so (Java_com_testdomain_AWSServicesProviderWrapper_AWSServicesProviderWrapperJNI_new_1AWSServicesProvider+40)

Can anyone help in understanding the cause of this?

[salunkhesagar1386]

@marcomagdy I read on other GITHub issue post #342. A question was asked 'is anywhere else in your application using libcurl or openssl?' and answer to this was 'if you are, you have to tell the SDK not to via the SDKOptions.'.

My application is getting crashed while InitAPI is getting called on Android, the same code works fine on Windows. My application also uses liburl internally for other REST calls. Reading this post, I modified following in my code, but no luck,

_HttpOptions httpOptions;
httpOptions.initAndCleanupCurl = false;
httpOptions.installSigPipeHandler = false;
m_SDKOptions.httpOptions = httpOptions;_

_CryptoOptions cryptoOptions;
cryptoOptions.initAndCleanupOpenSSL = false;_
_/*Added CryptoOptions because in log I saw app crashed @ line# 75 (inside if condition below) in Aws.cpp file, but no luck
if(options.cryptoOptions.sha256Factory_create_fn)
{
Aws::Utils::Crypto::SetSha256Factory(options.cryptoOptions.sha256Factory_create_fn());
}
*/_

_sdkOptions.cryptoOptions = cryptoOptions;_

I added '_curl_global_init(CURL_GLOBAL_ALL);_' where I am using the libCurl library calls in other area of my application.

[marcomagdy]

Make Aws::InitAPI(..) the first thing your application calls when it starts. And similarly Aws::ShutdownAPI() the last thing your application calls.

In other words, don't call init right before you need to call some AWS functionality. Do it at the very beginning.

[salunkhesagar1386]

Thank you @marcomagdy . The calls of Aws::InitAPI(..) is the first to be called in my C++ library and Aws::ShutdownAPI() the last one. Initialization of classes which uses libCurl is after the AWS initialization.

[marcomagdy]

The SDK also initializes the crypto libs (in addition to libcurl). It should be called early in your application not just your library.
Also, make sure that you are not linking two different versions of the crypto/ssl libs. Ones brought in by the SDK and others directly by your application.

[salunkhesagar1386]

@marcomagdy Thanks again. I might be writing too much, but its just to help understand the details.

Application is divided in 3 layers,

• UI (Android)
• Wrapper (Bridging between Android and C++ code)
• Static library where my business logic is written and consumes libCurl and AWS SDK.

The very first call I make to my static lib project is to initialize the AWS using InitAPI(..) method and till then nothing is instantiated from my static library project.

Once its initialized, I make call to APIs like _authenticateUser(...), listS3Bucket(...)_ of my library.

Somehow, everything works fine on Windows, but Android its crashing.

So here's how I have designed and implemented entire application:

• I have created a VS solution in which one shared static library project exists (contains Win and Android)
• This C++ library has just two classes AWSServicesProvider and CUserAuthentication.
• AWSServicesProvider uses/includes AWS SDK APIs and abstracts the actual AWS APIs from user.
• CUserAuthentication just uses libCurl and abstracts the actual libCurl APIs from user. I havent used openSSL anywhere in this class
• Both the above classes expose 1 APIs each which takes std::string and returns std::string.
• Created JNI Wrapper files using SWIG
• Added the C++ director classes files created by SWIG in VS Solution (with .cpp and .h extension)
• Created a Android project which wraps the all AWS .so, libcurls .so and my static lib's .so file in it and loads these libraries using ' System.loadLibrary()'
• Consumed the corresponding .java files created by SWIG in this Android project which bridges between C++ and Android java files.
• Created on simple Android application to call the API's of my C++ static lib project.

Did you meant that the libCurl is internally initializing or using OpenSSL libraries?
I haven't deployed any other library (.so or .a) apart from AWS SDK libs , libCurl and my own business logic library in apk package.

Just For your knowledge, the intention behind having this architecture is to avoid repetitive business logic on multiple platforms.

[martinkbrown]

I don't know much about the SDK, but can you try setting the following option?

cryptoOptions.sha256Factory_create_fn = nullptr;

To see what happens if that if statement is not entered.

[salunkhesagar1386]

@martinkbrown I already tried setting up the value to nullptr just considering if by default it's holding some value then this would reset it back. But no luck.

[salunkhesagar1386]

@marcomagdy : Further to this, today I tried by removing entire libCurl dependency from my project and rebuilding the application, but same place its getting crashed still.

00 pc 000c665c /data/app/com.example.sample.awstester-sSm2mcRoRNMwMUssrRyNQw==/lib/arm/libaws-cpp-sdk-core.so (_ZNKSt6__ndk18functionIFNS_10shared_ptrIN3Aws5Utils6Crypto22SymmetricCipherFactoryEEEvEEclEv+24)

Anything you can help in?

[salunkhesagar1386]

As suggested by @marcomagdy I made changes in my application to make the 'Aws::InitAPI(sdkOptions);' at the very beginning before making any call (be it AWS or any other library) and the call got succeeded. In my application I was using making the call to get the user signed-in to Azure AD using Microsoft's IDP log-in page and then was initializing the AWS library.

So conclusion is, 'InitAPI' has to be called at very beginning of application launch and 'ShutdownAPI' just before closure of application.

P.S. Thanks @marcomagdy :)