Collect: Separate form preloaded values for username, email from server settings

[lognaturel]

Software and hardware versions

Collect v1.4.12

Problem description

Values for preloaded username and email address fields in forms are pulled from the settings for username and google account, respectively. This page has more information about adding preloaded fields to forms. This comment explains why the duplicate fields are there and this code sets the values of the preloaded fields.

Preloaded values explain why Google account / username / password are shown on the settings screen in addition to in the Configure platform settings submenu. Having these settings values in two places makes the settings screen confusing. Additionally, the different configuration fields point to the same value which means that it's not possible to set identifying values for data collectors that are separate from server auth values. This is problematic because organizations often use the same username to connect to their servers across all data collectors.

Steps to reproduce the problem

Go to General Settings. See the username, password and google account fields that are also available in Configure Server Settings.

Expected behavior

Server settings should only be configurable from Configure Server Settings. Metadata values should be set separately.

I propose to:

1. Remove Google account, username and password from the Server Settings settings group
2. Create a new settings group titled Form Metadata (other name suggestions welcome) below the Server Settings group. Create a link to a new activity with text "Form Metadata" or something like it.
3. Add hint text to the new settings group activity that says something like "These values will be added to forms that have username, email and/or telephone preloaded fields to identify the person submitting data."
4. Add new username, email and telephone fields to the Form Metadata group activity. These values should be separate from server authentication and used in PropertyManager
5. Remove the code in PropertyManager that attempts to get a phone number from the device because it doesn't actually work.

This should probably be done as several smaller pull requests.

[smoyth]

I like where this is headed. I agree the email, username, and password should be removed from Server Settings.

It would also be nice to be able to define custom fields here. What if the form metadata list started out blank and you could add new ones, choosing keys from the predefined list (username, email, telephone) or adding your own?

ODK would prefer to take username/email/phone from the metadata if found, but would fall back to server settings if not.

What would the UI look like for adding "hint text" to a settings group? I don't see an example of that in the app currently.

[lognaturel]

@hooverlunch The keys would need to be unique, right? I think it would be easier to have a fixed UI for the most common fields (username, email, etc) and then have some option to define custom keys. Either way, this seems like a good reason to make "Form metadata" a link to a new activity rather than having the fields right there in the settings.

For the hint, I was imagining something like the current hints/subtitles for automatically send over wifi, enable analytics, etc. If we have a separate activity for form metadata, the explanation can be at the top of that activity.

[smoyth]

Sounds fantastic.

Would need to make sure this works with the config-via-file functionality, whatever that's properly called...

[lognaturel]

I believe you are talking about "bulk configuration" which I think is only documented at the bottom of this blog post.

[smoyth]

That's the one! Would you agree we'd want to support that with form
metadata?

On Tue, Feb 14, 2017 at 5:28 AM, Hélène Martin [e-mail]
wrote:

I believe you are talking about "bulk configuration" which I think is only
documented at the bottom of this blog post
https://opendatakit.org/2013/04/nafundi-adds-auto-send-navigation-buttons-and-bulk-configuration-to-collect/
.

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/opendatakit/collect/issues/311#issuecomment-279668873,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AAq3cBgsSPbDTGsyOIG5OuUz6Y6jurcVks5rcYHQgaJpZM4LTMOC
.

--
Tom Smyth

Worker-Owner, Sassafras Tech Collective
Specializing in innovative, usable tech for social change
sassafras.coop · @sassafrastech

[lognaturel]

I think so. Now that I've thought through this, I do think that the custom fields piece should be filed as a separate issue. It's great to have this discussion here to influence the preloaded values UI redesign but I think it adds too much functionality to this one issue! @hooverlunch if you could file your original description of what you want incorporating what we've discussed as a separate issue that points to this one, that would be super! 😊

[smoyth]

@lognaturel done. See https://github.com/opendatakit/collect/issues/443

[smoyth]

@lognaturel I wonder if you were envisioning the email and phone fields as being validated and/or restricted to certain keyboards? Just wondering if we could use them for things other than emails and or phone numbers. #443 would be nice to have but in case funding or time is tight, considering this as a backup (seeing as it is a prereq anyway).

[yanokwa]

I think they should be somewhat validated. Emails should have @ and be greater than 3 characters. Phone numbers should be numbers with support for +, -, (), and the like.

[dcbriccetti]

Android Studio is telling me it’s not recommended to use device ID. Google says this: https://developer.android.com/training/articles/user-data-ids.html

[smoyth]

Actually that's fine. I forgot that the username field will be distinct from the authentication username so we could just use that. So validation will not cramp my style at all. Thanks!

[smoyth]

@dcbriccetti Are you implementing this issue?

[dcbriccetti]

@hooverlunch @lognaturel asked me to take it, yes. Not the custom fields piece, just the original issue.

[smoyth]

Super!

[dcbriccetti]

Is any more discussion needed on @lognaturel’s proposed solution in the initial description of this issue?

[lognaturel]

One thing I've been thinking about is users who have been using those field. Unlinking the metadata values from the login values could mean people who thought they were collecting username / email information suddenly wouldn't be. I wonder if we should do a "migration" such that on first launch after this change values are copied over?

[dcbriccetti]

I don’t fully understand the above yet. Can I proceed with some other steps and return to that later?

Also, this page may be out of date: https://opendatakit.org/help/form-design/examples/#Property_values
The sections at the end.

[lognaturel]

Yes, that can be added on later. I think my proposal is fine and you can go ahead. 😄

[yanokwa]

This is looking good to me. I do think we should see both mutable and immutable data. The latter is important because then it's easy for an enumerator/supervisor to go to that screen and report those Ids rather than digging through Android's menus.

So the list would be

mutable

• username
• phone number
• email

immutable

• device_id
• subscriber_id
• sim_serial

[dcbriccetti]

Are you envisioning something like this?

[yanokwa]

@dcbriccetti I don't think the indentation should be there. I'd also drop the headings and put the current values as summaries so they are always visible. See Configure platform settings for an example.

[dcbriccetti]

Here’s what the same screen looks like on a smaller emulated device, for what it’s worth:

[dcbriccetti]

Is this a step in the right direction?

[lognaturel]

Looks great to me!

[dcbriccetti]

Is this what you mean by “hint text” in #3?

[dcbriccetti]

Remove the code in PropertyManager that attempts to get a phone number from the device because it doesn't actually work.

It’s working for me. Maybe I fixed it without being aware of it.

[dcbriccetti]

This should now look more like what Yaw was describing in https://github.com/opendatakit/collect/issues/311#issuecomment-285998064

[yanokwa]

@dcbriccetti I preferred the split you had previously. Maybe we should call them User defined and Device defined.

Phone numbers are not registered on all SIM so it's best if the phone number is left as user entered data.

[dcbriccetti]

OK, I’ll remove the code from PropertyManager that gets the phone number from the device, and make it user-entered like the username and email.

How’s this?

[smoyth]

Great work on this so far! Perhaps if the system returns a valid-looking phone number we could pre-set the field to that, but make it editable?

[lognaturel]

@hooverlunch @dcbriccetti The challenge with that is knowing when to set it. Whenever the field is blank? But then how does the user indicate a blank phone number? Only the first time that the menu is viewed? If you have a good way around that, then it might be nice to have.

I really think that in practice very few physical phones will have phone numbers readable in this way. @dcbriccetti you're on an emulator, right?

[smoyth]

Yes, I think the default value, when ODK is installed, is the one read from the SIM. Once it's edited, it's gone.

My phone seems to know its phone number too.

[dcbriccetti]

I’ve run on an HTC One, a Nexus tablet, and two emulated devices. I got the phone number from the HTC One smartphone, but I didn’t note on which of the others I got a phone number via

android.telephony.TelephonyManager
public String getLine1Number()
Returns the phone number string for line 1, for example, the MSISDN for a GSM phone. Return null if it is unavailable.
Requires Permission: READ_PHONE_STATE OR android.Manifest.permission.READ_SMS
The default SMS app can also use this.

[dcbriccetti]

[dcbriccetti]

The coding is done for the migration, if the scheme shown in the spreadsheet above is acceptable. I plan to test more Friday. Maybe somebody wants to pair with me on testing.

[smoyth]

Wow, nice work!