Aws-sdk-android: Submit Events is not working properly

[abdullahmehmood]

Describe the bug
After Investigation I have found that the sdk is not storing the offline data, when device is offline and then if we call the submit event method just after the record event then crash occurs and because of this crash it is unable to store the records in the DB. When I check the awspinpint.db file its empty.

So, I have to add the check network availability whenever I call the submit method. If we don't do this then our data is lost.

To Reproduce
Disconnect the device from Internet and then record and submit events then check the db file. Or again connect the device to internet it will not send the previous recorded data.

Your code
Same like in the documentation:
https://aws-amplify.github.io/docs/android/analytics#custom-events

public void logEvent() {
   final AnalyticsEvent event =
       pinpointManager.getAnalyticsClient().createEvent("EventName")
           .withAttribute("DemoAttribute1", "DemoAttributeValue1")
           .withAttribute("DemoAttribute2", "DemoAttributeValue2")
           .withMetric("DemoMetric1", Math.random());
   pinpointManager.getAnalyticsClient().recordEvent(event);
   pinpointManager.getAnalyticsClient().submitEvents();
}

Expected behavior
SDK should handle the submitEvent crash method when device is offline and also record the events in DB properly.

[kvasukib]

@abdullahmehmood Sorry for the inconvenience caused. I am able to reproduce the issue. I am working on a fix for it and will have a PR for you to test.

[kvasukib]

@abdullahmehmood Submitted PR #776 to fix the issue. Please feel free to build from the PR for testing. before the fix can be released.

[abdullahmehmood]

@kvasukib Thanks. I'll check it out.

[abdullahmehmood]

@kvasukib Can I get the aar file? So, that i can check this on my side. As I have added the Gradle dependency of aws sdk in my project. or When can I have the release of this fix?

[Noks2808]

Hi,

Any updates on the release of this fix?

[msesma]

Hi. Having the same issue. Any news about a release date for this fix?

[msesma]

I can confirm https://github.com/aws-amplify/aws-sdk-android/pull/776 fixes the issue in my case. I built it from the PR and it works like a charm. Please release it as soon as possible.

[abdullahmehmood]

@kvasukib I am also waiting for the release. Can you give ETA of this release?

[kvasukib]

@msesma Thank you for confirming that. I am working on to get a closure by end of this week. I will try to include in one of the upcoming releases.

[rohandubal]

Hello . @abdullahmehmood @msesma

We have fixed this issue with SDK release 2.13.1. Please try it out and let us know if it works for you.

Best,
Rohan

[msesma]

Hello. unfortunately it is not working.

• Using 2.12.7 with a jar file compiled from PR https://github.com/aws-amplify/aws-sdk-android/pull/776 works ok and sends events when returning to online.
• Using 2.13.1 from repos it does not send the events recorded while offline.

Checked using exactly same code and test steps.
@kvasukib Could you please test this again with 2.13.1 version?

EDIT:

• The above behaviour is happening in a Pixel 2 with android 9.
• In a Motorola G1 with Android 5.1 SDK 2.13.1 works ok offline.

Weird. Does it makes sense to you?

[kvasukib]

@msesma Thank you for the quick response. The different in the behavior when you build from the PR vs consuming the JAR could be due to the change I made while handling client exception.I am summarizing the behavior as follows:

When putEvents fails with an exception:

• AmazonServiceException:
  
  
  
  • Retryable exception
    
    do not delete events from database
  
  
  • No-retryable exception (ValidationException, BadRequestException, SerializationException)
    
    delete events from database
  
  

• AmazonClientException
  
  
  
  • UnknownHostException
    
    do not delete events from database
  
  
  • All other client exceptions
    
    delete events from database
  
  

The difference you are noticing is due to the fact that different Android API levels may throw different exceptions when network is not available.

Can you post the exception that you are getting when the network is not available?
Are you seeing the log statement from https://github.com/aws-amplify/aws-sdk-android/blob/master/aws-android-sdk-pinpoint/src/main/java/com/amazonaws/mobileconnectors/pinpoint/internal/event/EventRecorder.java#L428?

Ref: https://github.com/aws-amplify/aws-sdk-android/blob/master/aws-android-sdk-pinpoint/src/main/java/com/amazonaws/mobileconnectors/pinpoint/internal/event/EventRecorder.java#L416

[abdullahmehmood]

@kvasukib I have tested it. Its seems to be working fine now. The events are recorded in the db file while the device is offline. And its also submitting the events when device come online.

For log statement:
I am seeing the below error in the logs when device is offline:
"UnknownHostException: Unable to successfully deliver events to server. Events will be saved, error likely recoverable....."

[kvasukib]

@abdullahmehmood Thank you for the feedback. That's the expected behavior. Currently we check for UnknownHostException when there is a AmazonClientException. For all other exceptions under AmazonClientException, we clear the events stored in the local database.

We can add more exceptions to the list of exceptions which are recoverable based on your feedback and further testing.

[msesma]

Hi @kvasukib those are the logs. As you can see, both the locally compiled from the PR library and the released running in Android 5 save the events, but the release running in Android 9 deletes them:

SDK 2.12.7 Fix PR compiled locally. Executed in Android 9 (Pixel 2)

2019-04-10 09:23:53.773 11496-11613/com.touchsurgery.dev E/EventRecorder: Unable to successfully deliver events to server. Events will be saved, error likely recoverable.Unable to execute HTTP request: Failed to connect to /192.168.6.95:8888
    com.amazonaws.AmazonClientException: Unable to execute HTTP request: Failed to connect to /192.168.6.95:8888
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:441)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:212)
        ......
     Caused by: java.net.ConnectException: Failed to connect to /192.168.6.95:8888
        at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:143)
        ......



SDK 2.13.1 Executed in Android 9 (Pixel 2)

2019-04-10 09:36:12.053 12214-12338/com.touchsurgery.dev E/EventRecorder: Failed submission of 1 events, events will be removed from the local database. 
    com.amazonaws.AmazonClientException: Unable to execute HTTP request: Failed to connect to /192.168.6.95:8888
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:441)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:212)
        ......
     Caused by: java.net.ConnectException: Failed to connect to /192.168.6.95:8888
        at com.android.okhttp.internal.io.RealConnection.connectSocket(RealConnection.java:143)
        at com.android.okhttp.internal.io.RealConnection.connect(RealConnection.java:112)
        ......


SDK 2.13.1 Executed in Android 5.1 (Moto G1)

04-10 09:43:48.689 8615-9084/com.touchsurgery.dev E/EventRecorder: UnknownHostException: Unable to successfully deliver events to server. Events will be saved, error likely recoverable.Unable to execute HTTP request: Unable to resolve host "pinpoint.us-east-1.amazonaws.com": No address associated with hostname
    com.amazonaws.AmazonClientException: Unable to execute HTTP request: Unable to resolve host "pinpoint.us-east-1.amazonaws.com": No address associated with hostname
        at com.amazonaws.http.AmazonHttpClient.executeHelper(AmazonHttpClient.java:441)
        at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:212)
        ......
     Caused by: java.net.UnknownHostException: Unable to resolve host "pinpoint.us-east-1.amazonaws.com": No address associated with hostname
        at java.net.InetAddress.lookupHostByName(InetAddress.java:427)
        at java.net.InetAddress.getAllByNameImpl(InetAddress.java:252)
        ......

[msesma]

Seems that Android 5 returns UnknownHostException while Android 9 returns ConnectException and causes the database event drop.

In order to check if the events are being sent, I'm using Charles proxy. That is why Failed to connect to /192.168.6.95:8888 is logged and the exception is ConnectException.
Seems that Android 5.1 logs the real server not being DNS resolved instead the proxy and the exception is UnknownHostException. I have tested 2.13.1 in Android 9 without Charles and the log is:

2019-04-10 10:12:25.573 12214-12338/com.touchsurgery.dev E/EventRecorder: UnknownHostException: Unable to successfully deliver events to server. Events will be saved, error likely recoverable.Unable to execute HTTP request: Unable to resolve host "pinpoint.us-east-1.amazonaws.com": No address associated with hostname

So it will work 👏 in non proxied networks.

But what happens if the user is in a proxied network or a version of Android starts to cache DNS resolution? It will fail. What do you think about saving events for both UnknownHostException and ConnectException?

[kvasukib]

@msesma Thank you for the detailed analysis. We can add more exceptions to the list of exceptions where the events will be saved in the local database. Do you have a list of exceptions where the events will be saved in the local database?

I will investigate the ConnectException and see if we can add it to the list.

[kvasukib]

@msesma I have submitted a PR #893 to add SocketException (ConnectException derives from SocketException) to the list.

[msesma]

Great! I think that will be enough.
I've just seen that the PR is already merged so next version will contain it. I'll test it as soon as it get released and confirm if it is working.

Thanks a lot for your help.

[msesma]

Verified with release version 2.13.2. Everything is working in Android 5.11 and android 9 for proxied and non-proxied networks.
Thanks again for your help @kvasukib

[abdullahmehmood]

@kvasukib I have noticed some of the events are dropping again. I am facing the following error messages in the log:
1st Error msg: "Failed submission of 1 events, events will be removed from the local database."
2nd Error msg: "AmazonClientException: Unable to getCredentials....."

Only on the above error messages it will not store the offline data in the local db. And After sometime when I run the app again. Then it is storing the offline data and also the error messages was changed. Can you please look why this is happening.

[kvasukib]

@abdullahmehmood Sorry for the inconvenience caused. For the 1st error message, you have encountered an error for which SDK decided it to be non-retryable, so the events are dropped.

For the 2nd error message, can you post the complete error so I can infer if it's retryable or not.

The logic is as follows:

• AmazonServiceException:

  • Retryable exception
    do not delete events from database
  • No-retryable exception (ValidationException, BadRequestException, SerializationException)
    delete events from database

• AmazonClientException

  • UnknownHostException, SocketException
    do not delete events from database
  • All other client exceptions
    delete events from database

[stale[bot]]

This issue has been automatically closed because of inactivity. Please open a new issue if are still encountering problems.