Aws-sdk-android: Using startService for a background event is causing crashes in Android O

[linusmartensson]

https://github.com/aws/aws-sdk-android/blob/67d34accb74ce4ae66d87a1a4dbd5ced3bec5322/aws-android-sdk-s3/src/main/java/com/amazonaws/mobileconnectors/s3/transferutility/TransferUtility.java#L761

Android rules for background services have changed, the above line is causing our application to crash irrecoverably.
See the android migration guide for reference: https://developer.android.com/about/versions/oreo/background.html#migration

[cbommas]

Hello linusmartensson,

Thank you for reporting this issue to us and sorry about any inconvenience caused. We are looking into this and will reply back shortly.

[cbommas]

Also, appreciate it if you could you post any logs or stack traces to help with the analysis of this issue.

[linusmartensson]

Sorry, I reported this whilst traveling with work and our bandaid-solution was to disable all uploads to s3, since we were in an environment with patchy network availability.

The issue appears at any time when wifi is disconnected on an android O device and the application is in the background. I can try to provide logs when I find time if this is not enough for you to reproduce the issue.

[raghunandankavi2010]

This is certainly due to behaviour changes in android O. service cannot be started when app is in background.

The logs

 Caused by java.lang.IllegalStateException: Not allowed to start service Intent { act=add_transfer cmp=com.fabianenos.drawabletest/com.amazonaws.mobileconnectors.s3.transferutility.TransferService (has extras) }: app is in background uid UidRecord{f07ac u0a333 RCVR bg:+1m46s815ms idle change:uncached procs:1 seq(0,0,0)}
   at android.app.ContextImpl.startServiceCommon(ContextImpl.java:1521)
   at android.app.ContextImpl.startService(ContextImpl.java:1477)
   at android.content.ContextWrapper.startService(ContextWrapper.java:650)
   at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.sendIntent(TransferUtility.java:432)
   at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:216)
   at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:186)
   at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:154)
   at com.fabianenos.drawabletest.services.AlarmReceiver.uploadPendingBills(AlarmReciever.kt:118)
   at com.fabianenos.drawabletest.services.AlarmReceiver.onReceive(AlarmReciever.kt:203)
   at android.app.ActivityThread.handleReceiver(ActivityThread.java:3187)
   at android.app.ActivityThread.-wrap17(Unknown Source)
   at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1672)
   at android.os.Handler.dispatchMessage(Handler.java:106)
   at android.os.Looper.loop(Looper.java:164)
   at android.app.ActivityThread.main(ActivityThread.java:6494)
   at java.lang.reflect.Method.invoke(Method.java)
   at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:438)
   at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:807)

Waiting for a fix. Amazon sdk version 2.3.3

[kvasukib]

Hi @raghunandankavi2010, Sorry for the inconvenience caused. Does this crash happen for every upload/download operation? Can you give us some information on the device that you are running your app against? Also, can you paste a code snippet for further debugging?

[raghunandankavi2010]

The sample code is in kotlin. The code below is called in alarm receiver which is a broadcast receiver. This is called regularly at 1 minute interval. This happens with the latest os android oreo.

            transferUtility = TransferUtility(s3, context)
            observer = transferUtility.upload(
                    bucket,
                    fileIdentifier,
                    File(bill)
            )

Amazon sdk version 2.3.3. Let me know if you need any other code snippet.

We can get over this problem ( with no fixes on amazon sdk) by not scheduling a service repeatedly when the app is in background.

[kvasukib]

Can you try upgrading to 2.6.16 and observe if there are any changes in the behavior? Also do you see the above mentioned crash when you invoke upload from a activity?

[raghunandankavi2010]

No this cash does not appear when we start a service from activity ( both 2.3.3 and the latest versions). My bad!. I thought I was replying to some one less.

On Mar 4, 2018 1:27 PM, "Raghunandan Kavi" [e-mail]
wrote:

I have upgraded to the latest version already. You need to change the
implementation for upload. Instead of doing this upload task every few
minutes with alarm receiver switch to job scheduler.

However jobshceduler does not work in exact time period.

On Mar 4, 2018 1:23 PM, "Karthikeyan" [e-mail] wrote:

Can you try upgrading to 2.6.16 and observe if there are any changes in
the behavior? Also do you see the above mentioned crash when you invoke
upload from a activity?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-sdk-android/issues/400#issuecomment-370209204,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEBXqfO25b3vUPTayZwlozyAi9RG5j7Fks5ta51dgaJpZM4RpihQ
.

[kvasukib]

Thank you @linusmartensson for the quick reply. Can I know what's the use-case for invoking upload for the same file from a alarm receiver? When you call upload, we send an intent to the TransferService in the SDK which takes care of uploading this file in the foreground/background.

[svvashishtha]

Hey @kvasukib, facing the same issue here. I'm using a sync adapter to make background data upload calls.
Crash logs point to this particular line
final TransferObserver observer = transferUtility.upload(Constants.BUCKET_NAME, destination, currentFile);
Also tested with 2.6.16 version, crash is persistent.

[svvashishtha]

Hey @kvasukib,
Is there any update regarding this issue?
Our app invariably crashes on devices running android oreo with the same log.
FATAL EXCEPTION: SyncAdapterThread-1 Process: app.example.upload, PID: 24228 java.lang.IllegalStateException: Not allowed to start service Intent { act=add_transfer cmp=app.example.upload/com.amazonaws.mobileconnectors.s3.transferutility.TransferService (has extras) }: app is in background uid UidRecord{f80276c u0a300 TRNB idle procs:1 seq(0,0,0)} at android.app.ContextImpl.startServiceCommon(ContextImpl.java:1538) at android.app.ContextImpl.startService(ContextImpl.java:1484) at android.content.ContextWrapper.startService(ContextWrapper.java:663) at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.sendIntent(TransferUtility.java:871) at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:540) at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:493) at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:462) at com.amazonaws.mobileconnectors.s3.transferutility.TransferUtility.upload(TransferUtility.java:403)

[faisal-vf]

Same problem here. Also using 2.6.16 version. We have to schedule uploads of data periodically and the app mostly remains in the background causing crashes on android O.

What we need is a TransferJobService and a TransferUtility.scheduleUpload() :)

[linusmartensson]

@kvasukib @cbommas please note again, as implied in my initial message, this crash is intended from Android, as a background service in Android O is not allowed to invoke startService due to stricter battery- and resource preservation rules in the new Android version. That means this library is currently broken for any and all applications running on android O devices, and the migration guidelines i linked you to in my original message provides direct instructions you need to resolve that.

[raghunandankavi2010]

We could use our own implementation or use use jobshceduler if you want to
run background jobs to upload something to AWS in the background

On Mon, Mar 26, 2018, 2:51 PM linusmartensson [e-mail]
wrote:

@kvasukib https://github.com/kvasukib @cbommas
https://github.com/cbommas please note again, as implied in my initial
message, this crash is intended from Android, as a background service in
Android O is not allowed to invoke startService due to stricter battery-
and resource preservation rules in the new Android version. That means this
library is currently broken for any and all applications running on android
O devices, and the migration guidelines i linked you to in my original
message provides direct instructions you need to resolve that.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-sdk-android/issues/400#issuecomment-376101605,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEBXqUb-Y_nE9tqd5oHQ8yzTIH0709qfks5tiLMpgaJpZM4RpihQ
.

[faisal-vf]

@raghunandankavi2010 You would still need to call TransferUtility.upload() method which ultimately calls context.startService() consequently resulting in a crash.

Unless, is there a way to upload to AWS without calling that method ?

[raghunandankavi2010]

I meant to say you need not use the service at all. Implement your own
mechanism for uploading.

On Mon, Mar 26, 2018, 3:40 PM faisal-vf [e-mail] wrote:

@raghunandankavi2010 https://github.com/raghunandankavi2010 You would
still need to call TransferUtility.upload() method which ultimately calls
context.startService() consequently resulting in a crash.

Unless, is there a way to upload to AWS without calling that method ?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-sdk-android/issues/400#issuecomment-376115714,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AEBXqan_31943gN6CLIC8M1OLD6T3Khgks5tiL6hgaJpZM4RpihQ
.

[svvashishtha]

@raghunandankavi2010 That is irrelevant to this issue. The purpose of this discussion here is to fix this issue. If you have an alternate solution, you should post it on StackOverflow.

[kvasukib]

Sorry for the inconvenience caused. We acknowledge the importance of the issue. We are working towards a fix. We will update this thread once we release.

[mohitmahi]

The only possible solution is to use JobIntentService available in android support v4 library.

But make sure it uses AsynTask not HandlerThread, so it is not as same as IntentService
but works on Pre-O as IntentService and Post-O as JobService

[markacoss]

@kvasukib Is there an update to when this issue will be fixed and made available? Currently getting this issue in production.

[h4ck3rviv3k]

@Kvasukib any updates on this issue?

[martron]

In case it helps anyone else, our solution was to implement the deprecated TransferManager for Post-O devices.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[markacoss]

@kvasukib Do you have an ETA when this issue might be resolved? Still having crashes in production.

[maskaravivek]

@kvasukib Any update on this? My app is experiencing crashes on Android 8.0 devices because of this issue.

[kvasukib]

@markacoss @maskaravivek Sorry for the delayed response. We are still working on to prioritize it. I will have an update on this next week.

[quadrant2]

@kvasukib Working to prioritize? You offered an update in a week two weeks ago. With great power comes great responsibility. Many many devs are affected by this.

[kvasukib]

@quadrant2 Sorry for the delayed response. Can you try the following and see if the transfer service starts? If you are invoking TransferUtility from a service context, please do the following in your AndroidManifest.xml file. The following example uses MainService from which TransferUtility is invoked.

<service
            android:name=".MainService"
            android:enabled="true"
            android:launchMode="singleTop"
            android:process=":main_service"
            android:stopWithTask="false" />
        <service
    android:name="com.amazonaws.mobileconnectors.s3.transferutility.TransferService"
            android:enabled="true"
            android:process=":main_service" />

Are you invoking TransferService from an activity context or service context?

[dennyabrain]

we are invoking it from a JobService.. so i guess Service context.

[maskaravivek]

I have already moved my file hosting needs to Firebase as this issue was affecting more than 3-4% of the users. The app was crashing for these many sessions.

[markacoss]

We are running TransferUtility from a Job Scheduler Service. How will using these settings fix the issue when the TransferUtility runnings while app is background and Oreo 8 does not allow normal services to do so?

[quadrant2]

Two questions:

1.) Does AWS understand the issue at this point and its impact on the
developers that rely on it? (ETA on fix would be appreciated).

2.) If there is no clear answer to #1, and we use the depreciated transfer
utility as a stop gap, can AWS at least guarantee that you will not phase
out the old transfer utility until this issue is resolved?

On Sun, Jul 22, 2018 at 6:25 PM, markacoss [e-mail] wrote:

We are running TransferUtility from a Job Scheduler Service. How will
using these settings fix the issue when the TransferUtility runnings while
app is background and Oreo 8 does not all normal services to do so?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/aws/aws-sdk-android/issues/400#issuecomment-406901863,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ABVYJSNnhCy7Sn0sadiCXI85COgbs7ttks5uJPvggaJpZM4RpihQ
.

--
Jason Van Anden
Founder and Technologist
718-388-5063

[vladostaci]

@kvasukib
12% of phones using Android 8.0 and above are crashing because AWS is starting a background service. Can someone in your team read this guide https://developer.android.com/about/versions/oreo/background and implement something that works on all Android versions. I don't thing there can be a more critical issue than this one.

[silvialifecake]

There is an ETA about the resolution of this bug? Starting from November will not be anymore possible to upload a build to Play Store without targeting at least Oreo. An approximation would help my team to decide how to proceed.

[desokroshan]

Everyone, Our apologies for the inconvenience caused and the delay in response. We have duly prioritized the issue and are presently working on a fix.

@linusmartensson @vladostaci Thanks for pointers to the migration guide.

I will update the thread once we have released the fix.

[kanikaverma7]

@desokroshan Any updates ?

[vladostaci]

I'm just using a background service with a TransferManager like suggested by https://github.com/aws/aws-sdk-android/issues/400#issuecomment-391713028
The only side-effect of this is that a notification should be displayed while the service is running. You can also use a JobScheduler in case you don't want the notification to be displayed.

[knollori]

@desokroshan Any updates on a fix?

[kvasukib]

Hello all,

Apologies for the delay.

We have made some changes to TransferUtility in 2.7.0 version of the SDK. The summary of the changes can be found in CHANGELOG or Documentation.

Summary:

Starting version 2.7.0 of the SDK, TransferService logic has been refactored. This service now will be responsible only for monitoring network connectivity changes. When the network goes offline, the transfers that are in progress will be paused. When the network comes back online, the transfers that are paused will be resumed. If you expect your app to perform long-running transfers in the background, you need to initiate the transfers from a background service of your choice.

The TransferService will not be started or stopped by TransferUtility anymore. You have to start TransferService manually from your application. A recommended way is to start the service upon Application startup. One way you can do this is to include the following line in the onCreate method of your app's Application class.

    getApplicationContext().startService(new Intent(getApplicationContext(), TransferService.class));

[jesusdanielx15]

Im still having these problems on Android O, in Huawei devices and some Samsung.. its Still the same error with SDK 2.7.3

[kvasukib]

@jesusdanielx15 Can you share the code snippet and the crash that you are getting? TransferUtility will no longer use TransferService in order to start a transfer. Instead TransferUtility will directly queue the transfer to the thread pool. Now, TransferService has become an optional component which is used for listening to network connectivity changes. If you need the transfers to be paused/resumed based on network availability then you can use the TransferService. Otherwise you don't have to use the TransferService.

https://docs.aws.amazon.com/aws-mobile/latest/developerguide/how-to-transfer-files-with-transfer-utility.html#long-running-transfers

[jesusdanielx15]

This is the stack trace of the crash that i am getting:

Caused by java.lang.IllegalStateException: Not allowed to start service Intent { cmp=com.drivin.mobile.app/com.amazonaws.mobileconnectors.s3.transferutility.TransferService }: app is in background uid UidRecord{d1478e8 u0a29 RCVR idle procs:1 seq(0,0,0)}
at android.app.ContextImpl.startServiceCommon(ContextImpl.java:1538)
at android.app.ContextImpl.startService(ContextImpl.java:1484)
at android.content.ContextWrapper.startService(ContextWrapper.java:663)
at com.inzpiral.drivin.app.activities.MainApplication.onCreate(MainApplication.java:20)
at android.app.Instrumentation.callApplicationOnCreate(Instrumentation.java:1125)
at android.app.ActivityThread.handleBindApplication(ActivityThread.java:6062)
at android.app.ActivityThread.-wrap1(Unknown Source)
at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1764)
at android.os.Handler.dispatchMessage(Handler.java:105)
at android.os.Looper.loop(Looper.java:164)
at android.app.ActivityThread.main(ActivityThread.java:6944)
at java.lang.reflect.Method.invoke(Method.java)
at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:327)
at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:1374)

and as i said im working with sdk 2.7.3 and have to start service from the Apps Aplications' class like this:
getApplicationContext().startService(new Intent(getApplicationContext(), TransferService.class));

this is my UtilAWS class:

public class UtilS3 {

private final AWSMobileClient awsMobileClient;
private AmazonS3Client amazonS3Client;
private TransferUtility transferUtility;
private Context mContext;


public UtilS3(Context context) {
    this.mContext = context;
    awsMobileClient = AWSMobileClient.getInstance();
    awsMobileClient.initialize(context).execute();
}

/**
 * Gets an instance of the TransferUtility which is constructed using the
 * given Context
 *
 * @return a TransferUtility instance
 */
public TransferUtility getTransferUtility() {
    if (transferUtility == null) {
        transferUtility = TransferUtility.builder()
                .context(mContext)
                .awsConfiguration(awsMobileClient.getConfiguration())
                .s3Client(getS3Client())
                .build();
    }

    return transferUtility;
}

/**
 * Gets an instance of a S3 client which is constructed using the given
 * Context.
 *
 * @return A default S3 client.
 */
public AmazonS3Client getS3Client() {
    if (amazonS3Client == null) {

        ClientConfiguration clientConfiguration = new ClientConfiguration();
        clientConfiguration.setSocketTimeout(30000);
        clientConfiguration.setConnectionTimeout(30000);
        clientConfiguration.setMaxErrorRetry(5);
        amazonS3Client = new AmazonS3Client(awsMobileClient.getCredentialsProvider(), clientConfiguration);
    }
    return amazonS3Client;
}

}

and the implementation to upload a file from a Fragment is:

    UtilS3 utilS3 = new UtilS3(getActivity());
    transferUtility = utilS3.getTransferUtility();
   uploadObserver = transferUtility.upload(mFinalPath, file);
    uploadObserver.setTransferListener(new TransferListener() {
        @Override
        public void onStateChanged(int id, TransferState state) {
            // do something

            /*if(state == TransferState.COMPLETED){
                mIntents = 0;
                if(mProgressDialogFragment.isAdded()){
                    mProgressDialogFragment.dismissAllowingStateLoss();

                }
                generateSigned(mFinalPath);
            }else if(state == TransferState.WAITING_FOR_NETWORK){
                Toast.makeText(mContext,R.string.dialog_local_mode, Toast.LENGTH_SHORT).show();

                if(mAddress != null){
                    updateAddressScenario();
                }else if(mOrden != null){
                    updateOrder();
                }
            }*/
        }

        @Override
        public void onProgressChanged(int id, long bytesCurrent, long bytesTotal) {
            float percentDonef = ((float) bytesCurrent / (float) bytesTotal) * 100;
            int percentDone = (int) percentDonef;
            Log.i(TAG, "ID: " + id + " bytesCurrent: " + bytesCurrent + " bytesTotal: " + bytesTotal + " percentDone: %" + percentDone);

        }

        @Override
        public void onError(int id, Exception ex) {
            // do something
            if (mContext != null)
                Toast.makeText(mContext, mContext.getString(R.string.s3_upload_error), 
               Toast.LENGTH_SHORT).show();
            Answers.getInstance().logCustom(new CustomEvent("Upload Photo Error").putCustomAttribute("description", ex.getLocalizedMessage()).putCustomAttribute("vehicle_id", SessionCache.getCache(mContext).getActiveVehicle().getVehicleId()));
            if (mProgressDialogFragment.isAdded()) {
                mProgressDialogFragment.dismissAllowingStateLoss();
            }
        }
    });

and yes i need the transfers to be paused/resumed beacause many of our clientes work offline when there is no network availability. thank you for your time

[kvasukib]

@jesusdanielx15 Thank you for the response. Apologies for the delayed reply. Android limits the app to start a service when the app is in background. See https://developer.android.com/about/versions/oreo/background. Can you invoke the service in onResume of the Application class and make sure the service is started only when the app is in foreground?

[jesusdanielx15]

Dont worry, I will give it a try, thank you for your support

@jesusdanielx15 Thank you for the response. Apologies for the delayed reply. Android limits the app to start a service when the app is in background. See https://developer.android.com/about/versions/oreo/background. Can you invoke the service in onResume of the Application class and make sure the service is started only when the app is in foreground?

[stephenruda]

My solution was to run my own foreground a service for the duration of the upload and stopping the service when the upload is complete. This is still using the older version of the library so I haven't tried the changes in 2.7.0+.

My application uploads a file of logs every 24 hours, so when this happened in the background it was crashing. The app mostly runs in the background so waiting until the app was opened in order for it to make an upload wasn't an option for me.

Running your own foreground service essentially puts your app in the foreground so even though the TransferService was still not a foreground service, my own foreground service promoted the app to the foreground and allows other services to run (because the app is in the foreground).

Hope this helps someone here.

[arams]

Dont worry, I will give it a try, thank you for your support

@jesusdanielx15 Thank you for the response. Apologies for the delayed reply. Android limits the app to start a service when the app is in background. See https://developer.android.com/about/versions/oreo/background. Can you invoke the service in onResume of the Application class and make sure the service is started only when the app is in foreground?

Is this suggestion works fine? @jesusdanielx15

[arams]

@kvasukib In our project we run the background sync using the new work manager. We give the control to the system. It decides when to start our background service. During that time, we can use internet and we are able to upload few updates to our server. Same time we are trying to upload few contents to S3. But S3 transfer service is doing it in a separate upload service and it crashes. Why? Isn't there a simple upload method like synchronous request?

TransferObserver downloadObserver =
transferUtility.download(
"s3Folder/s3Key.txt",
new File("/path/to/file/localFile.txt"));

Will this line start the upload? If I don't want to track the upload can I just ignore the TransferService?

[jesusdanielx15]

Dont worry, I will give it a try, thank you for your support

@jesusdanielx15 Thank you for the response. Apologies for the delayed reply. Android limits the app to start a service when the app is in background. See https://developer.android.com/about/versions/oreo/background. Can you invoke the service in onResume of the Application class and make sure the service is started only when the app is in foreground?

Is this suggestion works fine? @jesusdanielx15

Yes it worked fine for me

[kvasukib]

@arams If you need to upload synchronously, you can refer to AmazonS3Client -> putObject where you can pass in the bucket, key and file and do an upload. However you need to wrap the call to putObject in a background thread. See https://github.com/aws/aws-sdk-android/blob/master/aws-android-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java

[arams]

@arams If you need to upload synchronously, you can refer to AmazonS3Client -> putObject where you can pass in the bucket, key and file and do an upload. However you need to wrap the call to putObject in a background thread. See https://github.com/aws/aws-sdk-android/blob/master/aws-android-sdk-s3/src/main/java/com/amazonaws/services/s3/AmazonS3Client.java

Thanks a lot. It worked.

[frankmuellr]

This issue has been closed because all questions appear to be answered. Please open a new issue if are still encountering problems.