Cht-core: Support Tecno W4

[estellecomment]

A partner is running a training with 139 CHWs next week, on android 6, on Tecno W4, and they're not able to launch the app :

Figure out if android 6 breaks the app. (maybe a container issue?)
(for that, we need android 6 phones...)
@abbyad not sure what priority is, assigning to the milestone.

[estellecomment]

cc @Enock1990

[henokgetachew]

I have tested with Android 6.0.1 and it works fine.

[estellecomment]

Thanks!
So it's the W4 maybe... I've sent them instructions through @Enock1990 for debugging phones, to see if they can report anything more informative.
http://medicmobile.cloud.answerhub.com/answers/177/post.html

[estellecomment]

There's changes in how permissions should be requested...
https://developer.android.com/about/versions/marshmallow/android-6.0-changes.html

[Enock1990]

Looks like Android 6.0.1 works well - I have tested with Samsung A8. I am trying to look for Tecno W4 or any other phone with just V6.0. @estellecomment on the priority issue, They are planning to do the scale up next week.

[SCdF]

NB: @Enock1990 mentioned on Slack that he got a W4 and it works. Enock unless I misunderstood you IMO we should close this ticket as wontfix, as there doesn't seem to be a code issue.

[alxndrsn]

Looks like the user who logged in doesn't have permission to view tasks. Worth checking:

1. which URL the webview is pointed at when the screenshot was taken
2. what result you get for the same user on the same server in a normal browser

[Enock1990]

@SCdF I have gone through the process of downloading, installing and login in and still the same result from there end and a different result on my end. It looks like the forms are not loading. @alxndrsn do you mean the base url? it's brac-ug.[email protected]. Using the same user on the browser it works well.

[Enock1990]

Anyone here who can help with this @estellecomment @SCdF @mandric @abbyad @browndav @alxndrsn @sglangevin @ngamita . I have attached all the error snapshots here.

[SCdF]

So this fails only on Android 6.0.0 and not 6.0.1? To confirm, these logs are from your phone? What version of the web view? Have you tried directly on Chrome on the phone? If so, what version of Chrome?

FWIW, all errors shown except those in the final screenshot are normal. Are all of the errors Error fetching form definitions? Or were there more before that? Are there any 404s / 500s in the network log?

If you type var db = PouchDB('medic-user-XXXX') into the console, replacing XXXX with the username you logged in with, you can hopefully get a reference to the DB. Then you can do things like this:

db.info(console.log); // general information about the local DB
db.allDocs({include_docs: true}, console.log); // every single document you have locally, including the actual document
db.query('medic-client/forms', console.log); // a list of all forms you have

[Enock1990]

yes @SCdF this only happens on V 6.0. We got this logs from @ngamita and they are from the phones(Tecno W4) that they are using in Uganda. My Tecno W4(same make&model) works just fine - this makes this issue very weird!

When you click on the link on the webview it opens the application very well on the web browser

There is the 404 ERROR which is distinct and to clarify I'll attach a screenshot of the about page showing that it's not allowing loading of the forms.

[SCdF]

yes @SCdF this only happens on V 6.0. .. My Tecno W4(same make&model) works just fine

Same make and model, same version of Android? Why is your version different from their version?

Unfortunately those screenshots of the about page don't help, though it does indicate that it hasn't got translations working, though that could just be bootstrap failing due to lack of forms.

To re-post from my previous comment, getting the following questions answered would be helpful:

• What version of the web view?
• Have you tried directly on Chrome on the phone?
• If so, what version of Chrome?
• Are all of the errors Error fetching form definitions? Or were there more before that?
• Are there any 404s / 500s in the network log?

[SCdF]

Oh to be clear, when I say version of the webview and Chrome, I mean by going into the Android Settings -> Apps, and then tapping on the Android System Webview or Chrome. For example, on my Nexus 5 both my Android System Webview version and my Chrome version is 53.0.2785.124.

[Enock1990]

@SCdF I'll be getting one of the phones that they are using in Uganda tomorrow morning. I will get you the exact details.

[Enock1990]

Hey @SCdF
What version of the web view?
* 53.0.2785.124*
Have you tried directly on Chrome on the phone?
yes
If so, what version of Chrome?
* 53.0.2785.124*
Are all of the errors Error fetching form definitions? Or were there more before that?
could you expound on this, please?

Are there any 404s / 500s in the network log?
I can't find 500 and 404 on 'Network' but only 404s on 'Console'

[SCdF]

What version of the web view?
* 53.0.2785.124*
Have you tried directly on Chrome on the phone?
yes
If so, what version of Chrome?
* 53.0.2785.124*

And on the phone that does work that is exactly the same except it's 6.0.1 instead of 6.0.0, what are those versions?
Also, do you know why their phones are 6.0.0 and yours is 6.0.1? Is there an update they can run, or something like that?

Anyway, from your output, it's really weird, it looks like you don't even have the medic-client ddoc!

You're not seeing any 404s in the network because the failure to get those documents has already happened, and it's not trying again. Can you wipe your phone's data for the app (so we start from the very start) and start again?

I'm interested in:

• The first real error you get (ignoring the errors that have documentation around them that they are normal)
• The 404s / 500s etc that you get in the network tab.
• How many documents you end up with, and which documents those are. If you run that allDocs comment I talked about above and then expand the reply by clicking on the little > arrow, you can see what it's returning in more detail.

[SCdF]

OK, @Enock1990 and I spent a few hours on hangouts trying to debug this.

Summary of things we learnt:

• There are no significantly obvious differences between the two phones except the security patch level: The bad phone is on May 2016, and the good phone is on July 2016.
• Both phones have the same Chrome version and Webview version, which in turn are the same as each other.
• The app fails to work both for BRAC, and also for LG end points. We tested this by using a debug APK I build of master of medic-android and attempting logging in from someone on both sites
• While the app fails, Chrome on the device works fine. So, even though the Chrome version and the webview versions are identical, something is different between them. Presumably hidden in the security patch changes (since the identical webview version works on the good phone). Potentially they could do training with Chrome?
• They appear to have all the documents you might expect downloaded: _medic/design, resources, their user doc and a bunch of forms
• However, _medic/design downloads twice, the first time correctly with a nonce, the second time incorrectly without, which is really really weird. From the network log (over hangouts, apologies for the grain)
  

The first one looks to have downloaded correctly, and probably is a full JSON document. The second one looks like a completely legitimate URL, but it returns "no response", which is the most I could get out of looking at Chrome over desktop sharing. This could be important, though it's hard to tell. EDIT: looking at this locally, I noticed that one is GET and one is HEAD. Crisis averted.

All of the errors were just as useless to look at over hangouts than they were over screenshots, lots of failures to get authentication, due to not being able to correctly pull settings and user settings out of various places (like in Enock's screenshot above). To really debug it I'd want to walk through it locally with a build that allows for useful debugging.

I believe the troublesome phone is going to be sent to one of the devs (probably me) to look at in more detail ASAP.

Some notes about the phones:

• BRAC is going to be buying a huge number of these phones, and the old ones are clearly weird and broken, so it's not unlikely that they may just be able to ask tecno for replacements and have that pan out great for them
• I hope the W4s we've bought devs that are in San Fran aren't these bad versions ( @estellecomment do you know if they got bought? We should get someone to check because we potentially might want to send them back if that's the only solution )

[sglangevin]

@SCdF BRAC has already bought a huge number of these phones. We are waiting for them to tell us whether or not they can replace them with a different phone. Even if they are able to get another phone for the first few training sessions, they might be stuck with a lot of these broken W4s, so if we can find a solution, that would be ideal.

Also, I don't think we bought W4s for anyone, we had bought L5s for storage space testing, and those are in Nairobi.

[SCdF]

@sglangevin ideally they'd be able to just ship them back to Tecno and either get Tecno to flash them to the newer code level or just give them phones with new builds on them

[SCdF]

One of the broken phones is being shipped to me, to arrive on Tuesday. I'll pick this up then.

[SCdF]

I have the phone. It could be something to do with web workers?

When attempting to boot the app the Settings service attempts to load the ddoc, and it 404s. When you do a DB().info() in place you get:

When I try it myself later outside of that scope (using PouchDB('db-name').get('_design/medic').then(console.log) directly in the console) I get the ddoc fine. My DB.info() looks like this:

I'm going to try reverting the web worker stuff and see if that helps.

[SCdF]

Yep patching web workers out seems to fix it. Need to double check that everything is working by comparing on Chrome (ie this user has no contacts or tasks, but that might be expected).

Also not sure why this is happening: replication deals with web workers fine, but us getting documents doesn't.

[garethbowen]

Interesting that your second case is using websql not indexeddb. Maybe the problem is with web workers and websql?

Try requesting idb explicitly and see what happens. You'll need to wipe the db to get it to create it with the right db adapter. Did the security patch do anything to idb?

We have some custom code to shim things like console into the webworker which isn't needed in later versions of Chrome but maybe there's a bug in the custom code. You could try removing it to see if a stock implementation just works.

I'm happy to help if you want to pair or just bounce ideas off me.

[SCdF]

Cheers @garethbowen I'll muck around with that in a moment.

Another interesting note: I tried it against master running my local dev db, which is a modified LG setup. Initially it failed in a similar way, but more violently: same inability to load settings, and then it started spamming huge numbers of errors about translations, and various other things, in an infinite loop (which made it hard to work out what was going on). I threw in a location.reload() to see if it would be more obvious the second time around, and then it actually loaded correctly!

So either we've changed something in later builds (8.0.0->master) or there is some kind of race condition that I got really lucky with. I'll also be taking a look at changes to the DB connect code between those two codebases to see if there is anything relevant.

[SCdF]

So in the android app, PouchDB is unable to find the IndexedDb adapter. _Sort of_. If I try to use PouchDB from the console: PouchDB('any-db').info().then(console.log) it works, but it uses the websql adapter. If I try to force idb, it throws an error because it can't find the idb adapter.

Going Deeper

Android app

If I disable web workers, nuke and run the app in the android app, the app works fine but it uses websql. I am left with an app with idb empty, and websql containing the data.

If I enable web workers, nuke and run the app in the android app:

• when DB().replicate(.. runs it uses the worker adapter, seems to not fail, and writes to both websql and idb (seriously), with the exception that the ddoc only gets written to websql
• when DB().get(.. is invoked it uses the worker adapter, and works for all docs in idb. Since the ddoc is not in IDB everything breaks

Chrome

If I disable web workers, nuke and run the app in Chrome, the app works fine and uses idb. I am left with an app with websql empty, and idb containing the data.

If I enable web workers, nuke and run the app in the android app, the app works fine and uses idb. I am left with an app with websql empty, and idb containing the data.

[garethbowen]

Wow. So something to do with the pre-patch 6.0.0 webview is breaking idb.

I'm ok with saying idb is required and trying to get Tecno to install the security patch on all devices if you think that'd work...

[SCdF]

New facts:

• When web workers is enabled in the android app, idb and websql both get populated, _except_ that idb doesn't get _design/medic stored in it. This is the only document that is missing from IDB.
• This means that the 404 in testing is not because DB().get(.. looks somewhere else, it fails because it can't find the ddoc in idb.
• This also explains why everything else fails, because the app can't properly bootstrap if it can't find the ddoc

[SCdF]

@garethbowen Yeah so it's really weird:

• We can't directly access idb, but clearly the web worker can
• Somehow PouchDB is saving things twice, possibly because if you do DB() without an adapter you get websql by default, and so maybe there are some assumptions in the code that are having it leak into websql
• Except when it comes to the ddoc, which is _only_ stored in this leaky way and never makes it into idb.

Or _something_.

Next steps IMO would be:

• Looking into direct Chrome usage, perhaps even folding PWA code in to 2.8.x (or having BRAC use 2.9.x if it comes out in time)
• Looking into them sending these phones back to Tecno
• ... something else, if my or anyone else's brain thinks of anything over night

[garethbowen]

WRT backporting PWA functionality to 2.8.x this is the change. It's mostly the manifest but I cleaned up some icons while I was in there.

[SCdF]

Cool. I've also attached a google account to it, and am going to attempt to update it via the play store as much as possible

[SCdF]

So PWAs aren't going to pan out on these phones:

• Tecno's homescreen skin / app / whatchamacallit doesn't support Chrome's "Add to Home Screen' functionality, which means that you can't use PWAs.
• Even ignoring PWAs, the best I got get setup was a bookmark link on the home screen by bookmarking it in the Android Browser (the default one that is gone in modern installs) and using the bookmark widget to add it to the home screen. Except
  
  
  
  • It's still just opening in Chrome as a tab
  
  
  • So the URL bar is there
  
  
  • And if you close Chrome you lose the session
  
  
  • And sometimes it will decide it should open it in a separate tab, so you might get multiple tabs open instead of just one
  
  
  • Etc
  
  

[SCdF]

I liiiiiieeeeeeeeed.

We can actually install the Google Now launcher (the default Google launcher that is on Nexus devices) on these phones from the play store. This launcher correctly understands Chrome's add to home screen command.

[SCdF]

So Gareth and I did some more debugging, and we've found a problem with idb support detection based on User Agent strings (!) in PouchDB that we're confident that we can patch out.

Here is the bad Tecno's user agent string for the web view.
Mozilla/6.0 (Linux; U; Android 6.0; zh-cn;TECNO-W4 Build/ AppleWebKit/534.30 (KHTML, like Gecko) Version/6.0 Mobile Safari/534.30
In contrast, here is an example of a different Tecno's user agent string:
Mozilla/5.0 (Linux; Android 4.4.2; TECNO-Y4 Build/KOT49H) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/30.0.0.0 Mobile Safari/537.36

The bad tecno's user agent causes issues with PouchDB's idb detection because it get's flagged as Mobile Safari: It says Safari (which most everyone does) but it doesn't also mention it's Chrome. Safari has apparently proven to be so buggy for the PouchDB team that they've completely drop support for it.

Potentially PouchDB could make their user agent detection more robust (though that is its own never ending nightmare), but what we can do ourselves is simply patch out the Safari check to always return false, and just make it clear that our software is not supported on Safari.

[SCdF]

@garethbowen can you review the following:

• Branch to create a 2.8.2 build, which is 2.8.1 + this patch: https://github.com/medic/medic-webapp/compare/2.8.1...2.8.2_br
• A slightly different PR to get this change into master: https://github.com/medic/medic-webapp/pull/2827

[garethbowen]

Looks good.

Because this is going to be a long lived patch I wonder if we'd be better off using the other style of patch on master. After getting the original you could overwrite the valid method with your implementation. Up to you...

[SCdF]

I think if it breaks in the future we can move it, but I'll leave it there for now.

[SCdF]

@Enock1990 I've assigned this to you to acceptance test. I've updated brac's dev site to be on the 2.8.2 beta. Chat to me on Slack. Once you're cool with it we can put out a final release and get brac's prod box upgraded!

[SCdF]

Released in 2.8.2.