Cht-core: Tasks and Targets disappear in Android 4.4.2 after upgrading to 2.6.2 and clicking Reload

[sglangevin]

This is a blocker for the 2.6.2 release #2240

Note that this cannot be reproduced in the web browser, only in the Android app on Tecno Y4. After updating and submitting a form from a task, the task cleared properly. Then I submitted another form and the task didn't clear as expected. I then viewed that recently submitted form on the web app, logged in as admin. After that I reloaded the app on my phone and tasks and targets were disabled. Not sure which part of that process cause them to become disabled, but we need to fix this before we update.

[SCdF]

That is really weird! So you're on your phone, as a user that is _not_ an admin, and you can't see either tasks or targets?

It sounds like it worked but then you reloaded and it didn't work. What happens if you reload again?

[SCdF]

(FWIW we block these things for admins but unless you're an admin it should be fine)

[sglangevin]

@SCdF I reloaded several times and no dice. I know they are blocked for admins (thanks for implementing that, v. helpful!) but for some reason that is being carried over to my phone.

[sglangevin]

I've been able to reproduce this on two devices running Android 4.4.2 but NOT on Android 4.4.4 or higher - those devices worked fine. If you have a Tecno on version 2.6.1, update to 2.6.2 and notice that Tasks and Targets are still there. Then go to About and click Reload. Notice that tasks and targets no longer load. In your console, you'll see Error getting tasks Object {error: "ReferenceError", name: "ReferenceError", reason: "console is not defined", message: "console is not defined", status: 500}

From there, hard-close the app and open it again. Tasks should load (they loaded most of the time for me). If you do a Reload again, you'll get the console errors. Here are a couple of screen shots to help define the error. Also, note that I am seeing that the error is on line 5 of inbox.js but when I go to un-minify it, it changes to line 4426, or something like that.

[sglangevin]

Reloading from Tasks I get an error loading Tasks, reloading from Contacts I get error loading Contacts but Tasks and Targets load successfully. This is only an issue on 2.6.2, not on 2.6.1.

[garethbowen]

So it looks like an issue with old versions of the webview, combined with some code change between 2.6.1 and 2.6.2.

The line that's throwing the error is from the angular log provider. It has the feel of a race condition that somehow the service is executing before the LogProvider or console is ready.

[garethbowen]

It would be interesting to test on the Android browser on the Tecno which should use the same engine as the webview and see if you can replicate it there.

[estellecomment]

Not seeing it happen on Tecno on local env, with latest testing.
Tasks are fine, contacts are fine, targets are fine. Reloading from contacts and from tasks doesn't do anything bad.
Fulfilling a task clears the task.
Gonna try on lg-testing and lg.

I see @sglangevin 's testing was on lg-testing. Has lg-testing gotten the same config changes as lg? We made changes to the tasks rules after #2179, right @abbyad ?

[mandric]

You can check and compare settings if you want:

curl https://admin:secret@hostname/medic/_design/medic/_rewrite/app_settings/medic | jq .settings > hostname-settings.json

There is also a json-diff npm module that comes in handy if you want to compare two bits of JSON.

[estellecomment]

My Tecno is android 4.4.2, not sure about webview :
"Mozilla/5.0 (Linux; Android 4.4.2; TECNO-Y4 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36"

[estellecomment]

thanks @mandric !
lg-testing and lg have identical settings. So that's not it.

[estellecomment]

Not seeing it on lg-testing. Tasks are there, reloading doesn't change anything, fulfilling a task clears it.

Next, trying to see if it's specific to upgrades 2.6.1 -> 2.6.2.

[estellecomment]

Commit for 2.6.1 : d6bd90224041048ba86f1f28cf561bf3cdde4aee
Commit for 2.6.2 (latest) : 4a0f6a90144916c2042d368bfa6c1628ccd324c0

[estellecomment]

Note : I'm doing all this with the demo user. Maybe it doesn't have enough data or something.

[sglangevin]

Just did some additional testing on the Y4's browser. During initial loading I saw a similar error but it was in reference to "read status". Same error message though - console not defined. I wasn't able to test the upgrade in the Y4 browser, though.

[garethbowen]

To get the diff between 2.6.1 and 2.6.2 use: https://github.com/medic/medic-webapp/compare/2.6.1...2.6.2

[garethbowen]

I can reproduce this on my Tecno against lg.dev. Tracing it back caused a migraine, but also it looks like the root cause is angular attempting to complete outstanding requests as part of the $defer service. I'm still looking but my guess currently is we're doing something in InboxCtrl which angular tries to clear because we immediately navigate on load. There is code that changed between 2.6.1 and 2.6.2 which could do something like this. Much more investigation required...

[garethbowen]

There was a $timeout added in InboxCtrl for initialising select2 elements which is my prime suspect.

[estellecomment]

wow and whaaaaat

[garethbowen]

I'm still finding this really hard to reproduce but I've made a patch which eliminates my prime suspect and released it as 2.6.3.beta. Assigning to @sglangevin to install the patch and test to confirm.

[sglangevin]

@garethbowen I synced a phone to lg.dev, then updated and now I'm seeing the console not defined error more consistently. It shows up in my console every time I click Reload. I also tried syncing a different phone, also running Android 4.4.2 to lg.dev after I ran the update and it loaded properly and has no issues when I try Reloading it. Dev tools isn't working with that phone, so I wasn't able to see the console.

Then I cleared app data on my Tecno and then synced to lg.dev (2.6.3-beta). I got `error fetching read status: console not defined and a new range error that I haven't seen before:

However, tasks and targets still loaded during that sync and there were no errors when I clicked Reload. If someone can help me, (maybe @abbyad ?) I'd like to test a phone that is synced to version 2.6.1, then upgraded to 2.6.3-beta to see if I notice any issues. Once we see that working, we can move this to ready.

The app also seems slower than normal, like we've lost some of the performance gains that we'd made a few months ago. Since lg.dev has only a small amount of data, I'm wondering what's going on there. Data is downloading normally (time-wise) but the UI is sluggish and I'm seeing lots of spinners. There's also a longer than usual delay between when I see forms download and when the '+' sign illuminates in History. Seems like app "warm up" is slower or something. I'm also noticing that the spinner stays up far beyond the initial sync (during reload). Probably need a different issue for that, just wanted to jot my thoughts while they are top of mind.

[garethbowen]

RangeError means it was trying to parse an invalid date. Not sure why that would be but probably an invalid date somewhere on lg.dev.

@sglangevin So after the first reload on 2.6.3.beta you couldn't get any more console not defined errors?

I've noticed the slow down too but only on the dev app. I've been switching it between different hosts so it's possible it's caused by that. We can do more performance investigation on lg.dev and lg-testing.app.

[sglangevin]

@garethbowen I had to clear the app data and start again to get rid of the console not defined errors. I'd like to test the upgrade from 2.6.1 to 2.6.3-beta to see what happens. If that looks good, we could release and I'd be ready to update lg.app.

[sglangevin]

I've done more testing, loading a phone on 2.6.3-beta. I still see one console not defined error (error fetching read status), but tasks and targets load normally and persist after a reload. Here's what I get on initial load:

After a reload, no errors:

I've reproduced this reliably on my Tecno using both lg.dev and brac.dev, both of which are on version 2.6.3-beta. @garethbowen you may want to investigate the error further, but I'm only seeing the 'error fetching read status' one and have not had any issues loading tasks and targets. The weird thing is that when I updated from 2.6.2 to 2.6.3-beta, I had more errors than loading 2.6.3-beta directly, so I'm curious to see what would happen during an update from 2.6.1 to 2.6.3.

[garethbowen]

@sglangevin I've pushed a new 2.6.3-beta.3308 which has a patch for the "error fetching read status" bug and also replication performance improvements. Could you please update and try again when you have some time?

[sglangevin]

@garethbowen I'm getting error loading tasks again when I update. This seems to be step back from the previous beta, where I got the error fetching read status but had no problem loading tasks and targets. Reopening.

[estellecomment]

From a technical standpoint, I think it's fair to say, if you will, AAAAARRRRRRRRRRGGGGGGGHHH

[garethbowen]

I've been looking at another "that can't possibly happen" issue this week also related to race conditions at startup. I'm wondering if there's a bug in Chrome's web worker implementation. I'm tempted to remove the pouch worker plugin and see if that fixes it.

[garethbowen]

That did indeed seem to fix it, but caused a small performance regression. I'll spend a little more time debugging this to see if we can have both worker-pouch _and_ fix these errors.

[garethbowen]

Some thoughts... old web worker implementations didn't have access to the console so it's possible it's code inside the web worker (inside pouch?) that's causing this. For example having debug mode on spits out lots of console log lines.

[estellecomment]

👀 :popcorn:

[garethbowen]

Root cause: https://github.com/nolanlawson/worker-pouch/issues/16

I'll hold off releasing 2.6.3 and hopefully get a solution soon.

[sglangevin]

Nice @garethbowen - so glad to hear you found the root cause! Thanks for helping see this through.

[garethbowen]

Assigning to @sglangevin for testing.

[sglangevin]

After a very long saga, this is working well! Thanks to @garethbowen for tracking down this bug.