Fenix: [Investigate] Abnormally high device-commands traffic

[grigoryk]

There are concerns from @rfk that Fenix is generating an abnormally high level of device-commands traffic (e.g. checking for send tabs). This issue is to investigate what's going on.

Acceptance Criteria

• [ ] Communicate with AS to determine how frequently we should be syncing (update these ACs with result)
• [ ] Ensure that we are only syncing as frequently as expected

[jonalmeida]

@grigoryk Where my knowledge is lacking: are the "device-commands traffic" the same as if someone is pressing the "sync now" button?

If so, it might just be this one call in onResume. We had this here originally because send tab landed before push, and for usability we sync'd every time. We could remove it now.

[rfk]

are the "device-commands traffic" the same as if someone is pressing the "sync now" button?

I believe it's part of what happens when you press "sync now", although other stuff also happens that is not related to device-commands.

+1 to trying to remove the onResume trigger and seeing how much of a difference that makes.

[rfk]

The fix in #7908 definitely seems to have helped with the traffic load here. It's still significantly higher than what we see on other clients, but about half what it used to be.

I don't have any other theories for why the rate should be higher on Fenix than on other clients. AFAICT there are three ways that Fenix could trigger a request to GET /v1/account/device/commands:

• By receiving and handling a "command received" push message.
• By the user clicking the "sync now" button, of which we do not have very many.
• By invoking the postAuthenticated handler on the FxaAccountManager.

IIUC the postAuthenticated handler may be invoked on application startup thanks to the Event.AccountRestored event, though I'm not 100% sure I follow the logic there completely. If we're polling for commands on every startup that might explain the higher volume?

[rfk]

By invoking the postAuthenticated handler on the FxaAccountManager.

I instrumented a local Fenix build to print out all its network requests, and from what I can tell, this event is the main remaining source of calls to pollForEventsAsync. It seems pretty easy to trigger the AccountRestored event by flicking around between apps, closing and re-opening the app, etc, and this in turn triggers a call to postAuthenticated which polls for device commands.

I'd like to understand whether this polling on AccountRestored is really necessary. @jonalmeida if FxA sends a push notification but the app is closed, what happens? Does it wake up the app code and give us an opportunity to fetch the commands at that point? (I tried to test this out locally, but it looks like I need to add some firebase config to my build to get it to work).

If we do like polling on AccountRestored, I wonder if we can add a bit of debouncing to ensure that we don't poll too frequently.

[jonalmeida]

@rfk pollForEventsAsync was already removed in #7908 so I'm guess you mean the syncNowAsync one.

I can prepare a patch to remove that line as well in that case.

@jonalmeida if FxA sends a push notification but the app is closed, what happens? Does it wake up the app code and give us an opportunity to fetch the commands at that point?

Yes, that should work provided the app a cold boot has taken placed or the app wasn't force-closed.

[rfk]

@rfk pollForEventsAsync was already removed in #7908 so I'm guess you mean the syncNowAsync one.

To clarify, I was referring to the pollForEventsAsync that happens implicitly inside the FxAAccountManager on application startup.

I'm fine with polling explicit on "sync now", that seems like the right thing from the user's perspective of "something didn't happen, let's try hitting sync now and see if it works".

that should work provided [...] the app wasn't force-closed.

What happens if we receive a push notification after the app as been force-closed?

[jonalmeida]

What happens if we receive a push notification after the app as been force-closed?

They get dropped entirely; never delivered. Grisha mentioned a while ago they saw this happen on Fennec as well (there's an old bugzilla somewhere that this is documented).

[jonalmeida]

@rfk were there any noticeable changes in traffic after the a-c change above landed?

_wonders if we can close this issue now_

[rfk]

I re-ran my bigquery queries, and latest versions of a-c seem to be producing noticeable fewer device-commands requests per active user than earlier versions. I think we can safely close this. Thanks again!

[ebalazs-sv]

@grigoryk @jonalmeida Hi! Can you please provide details to verify this manually? Thanks!

[jonalmeida]

I re-ran my bigquery queries, and latest versions of a-c seem to be producing noticeable fewer device-commands requests per active user than earlier versions. I think we can safely close this. Thanks again!

@ebalazs-sv Ryan Kelly has already done the verification, thanks. :)