Android: App closed, phone sensors are not updating

[freemann]

Home Assistant Android version:
2.3.0-306-full

Android version:
10.0.7

Phone model:
OnePlus 7pro

Home Assistant version:
0.114.3

Last working Home Assistant release (if known):
Unknown

Description of problem:
When I close the HA companion app on my phone, non of the app related sensors are updating in HA.

Traceback (if applicable):

Screenshot of problem:

Additional information:

[dshokouhi]

This is expected you need to keep the app running in the background so the background services continue to run. Closing the app also essentially stops these services from running. Try these steps to make sure the app has proper permissions: https://companion.home-assistant.io/docs/troubleshooting/faqs/#location-is-not-updating-in-android-app

[anyuta1166]

As I already said in the comments to the location tracking issue, this seems to be Android 9+ specific behavior.

If the phone is left intact and enters the doze mode, sensors update may be delayed by up to 2 hours even if the app is running in the background.

You still don't want to create a service with notification? ^_^ However I'm not sure that it will help.

Currently I'm sending location tracking request intent from Tasker to the Home Assistant app every 5 minutes as a workaround. This doesn't let the phone enter doze mode and another sensors are getting updated too as a side effect.
But ideally I want to get rid of Tasker, because it is a crutch and also draining the battery a bit fast (2% per hour in idle mode).

[freemann]

Totally agree with @anyuta1166
It's a bit against the automation principal to work with old data. I was working with "Automate" to send battery and location data. 2 weeks ago I moved from Automate to companion data, but this is just not working as it should.

Thinking to go back to Automate, that just worked like a charm.

Hope you could consider to change this behavior :)

[freemann]

@dshokouhi I love the foreground service!
Please "push" it through!! ❤️

[freemann]

Should this also fix #671 or make the updates more frequent evenly spread of time?

[dshokouhi]

@freemann the foreground service just needs to be tested a lil longer to make sure things are still good. It won't fix #671 because the periodic work request that we use has a minimum interval of 15 minutes to remain battery friendly so we won't get each and every percentage change. This change will make sure that when the work request starts the updates will be pulled with a foreground service to ensure they actually get pulled.

[dshokouhi]

So somethings to consider when the app is closed and how work manager works: https://stackoverflow.com/questions/50682061/android-is-workmanager-running-when-app-is-closed/52605503#52605503

Notice how it says some devices closing the app (or removing from recent apps) is also the same as Force Stop so in those cases foreground service won't help.

@freemann look into https://dontkillmyapp.com/ for your OnePlus device as those devices are more susceptible to killing the app which will stop the worker.

[freemann]

I always set "import" apps to "not optimise" for battery use.
And mentioned before I have very(!!) good and stable results with Automate;
https://forum.xda-developers.com/showpost.php?p=78588761&postcount=7
Head "other apps"

And the battery status is in companion running pretty stable, but I was hoping for a more flexible way of reporting the battery data with the new service overhaul.
If that's running ok, i was hoping it's possible to change this.

[anyuta1166]

Fortunately on my devices closing the app (or removing from recent apps) is not the same as Force Stop, so the periodic work continues to run. I'll test your PR tonight to check if it makes any differences. I wonder if it will bypass the doze mode on Android 9+ or not.

[anyuta1166]

@dshokouhi Unfortunately that doesn't work for me and my phone :-/

Tonight everything worked as expected until 02:30 (sensors update every 15 minutes and location subscription).
After that we see update_sensor_states and update_location at irregular intervals. Sensors update doesn't happen every 15 minutes and location subscription doesn't work too.

Everything starts to work automagically as long as I start using the phone this morning and it exits the doze mode (after 09:30, not included in the log).

2020-09-03 01:45:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 01:45:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 01:45:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 01:45:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 01:45:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 01:45:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 01:46:13 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 01:48:22 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 01:54:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:00:00 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:00:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:00:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:00:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:00:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:04:07 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:10:01 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:15:02 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:15:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:15:09 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:15:09 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:15:09 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:16:20 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:16:20 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:20:04 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:23:08 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:25:37 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:29:36 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:29:39 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:30:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:30:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 02:30:39 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 02:30:39 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 03:21:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 03:21:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 03:21:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 03:21:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 03:21:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 03:29:36 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 04:29:37 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 05:17:16 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 05:17:16 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 05:17:16 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 05:17:16 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 05:29:37 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 06:29:37 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:03:47 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:03:47 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:03:47 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 07:03:47 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 07:29:37 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:46:55 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:46:55 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 07:46:55 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 07:46:55 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 08:29:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:07:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:07:17 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:07:21 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 09:07:32 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 09:07:32 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 09:07:32 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:07:32 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:26:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:26:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:26:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 09:26:38 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_location:
2020-09-03 09:30:00 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:31:15 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:
2020-09-03 09:31:15 DEBUG (MainThread) [homeassistant.components.mobile_app.webhook] Received webhook payload from Anna for type update_sensor_states:

[anyuta1166]

So I still have to use Tasker..

Oh, wait, I have an idea. I'll just enable my SIP client again, it won't let the device sleep ;-o

But I still want a reliable solution without crutches,

[anyuta1166]

May be a service with a permanent foreground notification could help?

Or abuse alarms as other applications do? ;-)

[freemann]

Testing Beta version 'app_version': 'beta-346-833534f-full (346)'
Here's the HA logfile;
mobile_app.webhook.data.log

Here's a screenshot of LoveLace;

Datapoint 1 = 21:33
Datapoint 2 = 21:48 (nice 15 min)
Datapoint 3 = 22:00 (13 min)
The next points are at:
22:20 (interval 20min)
22:35 (interval 15min)
22:50 (interval 15min)
22:53 (interval 3min)
23:05 (interval 12min)
23:20 (interval 15min)
The intervals are, so it seams, all over the place and from 00:20 its a flatliner... Lovelace shows 70% until this morning, the widgets in Android show 70% and my phones statusbar shows 55%

Here the Android screenshot;

[dshokouhi]

@freemann your device is very aggressive at killing the app and sensor worker, you will need to see what is causing it to get killed. On my pixel devices here that experienced the issue the foreground notification made things better and the app is no longer killed in the background. Also to properly investigate this we need to see the ADB logs around when this issue happens. The mobile app debug logs won't be helpful, we need to see what is causing your system to kill it.

[anyuta1166]

@dshokouhi As I already said, I have the similar issue on my device. And nothing is killing the app. Sensors and location updates continues right at the moment when I wake up the phone by grabbing it and unlocking it. No need to launch the app etc.. It's just the doze mode.

[freemann]

I'm only trying to help.
I'm doing standard stuff, because you can ask every dumb person to make special settings to use this app.

OnePlus is a very populair brand, I think especially with domotica users. But I think you guys have the data about the "marked share" of OnePlus and companion.

Sorry and good luck, Automate us working flawlessly so going back to that solution.