Collect: Move to minSdkVersion:16 and targetSdkVersion/compileSdkVersion:22

[lognaturel]

The current configuration is:

minSdkVersion(9)
targetSdkVersion(9)
compileSdkVersion(22)

A minSdkVersion of 9 means that it is not possible to install the application on any device running Android API level < 9. It's important not to use API features above this version or to include alternate code paths for this level or there will be runtime crashes for older devices.

A targetSdkVersion of 9 means that when the application is installed on a device running Android API level > 9, forward-compatibility behaviors may be enabled so that the application behaves and looks as it would in API level 9 (Gingerbread 2.3 - 2.3.2). The application can still be installed on systems running older Android versions (down to minSdkVersion). targetSdkVersion is supposed to be the version the app is most tested on and is recommended to be the latest API leveI. More info in the Android docs.

A compileSdkVersion of 22 means that the app is compiled with version 22 of the SDK which determines the latest APIs available for use.

Here is a good article with more details and the following recommendation:

minSdkVersion (lowest possible) <= 
    targetSdkVersion == compileSdkVersion (latest SDK)

Having such a low minSdkVersion means relying on deprecated APIs. A low targetSdkVersion has strange side effects such as the one described in #285 because of the forward-compatibility features.

The overall Android stats suggest that moving the minSdkVersion up to 16 or even 19 would still support most devices.

According to Google Play statistics, only 1.3% of ODK users use an Android version under 4.03 (API 15). Android 4.4 (API 19) is the most popular Android version (33%). That said, a lot of organizations set up their phones without going through the Play Store so it's not clear whether this can be trusted. How can we get more reliable information on what devices _new versions_ of ODK will need to run on?

With the information we have, I'd recommend moving minSdkVersion to 14, targetSdkVersion to 22 and leaving compileSdkVersion at 22. I also recommend enabling lint errors for APIs > minSdkVersion or that value is meaningless (the app can't actually run on those older devices if newer APIs are used -- this is done by #279).

[IanUme]

I like the idea if moving up min sdk versions. Unfortunately, other than the dev console statistics, you would have to add analytics to the app to determine which devices are actually being used.

Lint errors are also nice! It would be a lot of work to clean them all up though!

[lognaturel]

I agree with you that analytics are needed to get a more precise sense of what Android versions are used. There's been some thought given to analytics and there's even #212 for adding Firebase analytics but the problem is that the oldest devices likely don't have Google Play Services installed and wouldn't be counted that way. If you have alternate ideas, please comment there.

Lint-wise, I think we're actually ok! #279 proposes a configuration and fixes all related failures.

[jwishnie]

A proposal:

1. Rename the existing ODK Collect App to "ODK Collect Classic" (or something similar)--and leave in the Play store.

2. Bump the minimum API level to 14 or 19 as proposed above (I'll argue for 19).

3. Publish a version with the bumped API level to the Play Store as "ODK Collect" and note in the description that users who need an older version for their older phones can download "ODK Collect Classic"

4. All future development goes into the new "ODK Collect" with higher API level.

5. If needed, high priority fixes (e.g. bad security problems that are solved) can be pushed, at API level 9, to "ODK Collect Classic"

Reasoning:

• We want a path forward to using newer APIs while supporting these older devices.
• Over time, we can expect fewer older devices to be in use, but without comprehensive data, we should assume that there are old phones that will never receive on OS update (e.g. Huawei IDEOS) in the field. We want to make sure these devices that are currently used with ODK Collect continue to work fine.

[mitchellsundt]

I think the ODK community organization should create a poll for organizational users to provide info on what OS versions they are using, and what quantity of devices they have. That would help balance the stats from Google Play.

==============

The description is not quite correct. The code is littered with tests for Build.version being > 11 or > 16, and for appropriate code to be executed. I.e., as long as the compileSdkVersion is greater than whatever API level functionality you need (e.g., 16), you can make use of the newer functionality, you just need to add these runtime guards.

Yes, you do need to decide whether target == compile or if it should be frozen at less than this. Since target affects UI look-and-feel, you likely need to review the theme and UI appearances.

For those curious, here are the ODK Collect APK's stats from the play store that Helene refers to:

Android 7.0 232 0.31%
Android 6.0 12,669 16.79%
Android 5.1 14,183 18.80%
Android 5.0 7,961 10.55%
Android 4.4 25,161 33.35%
Android 4.3 1,440 1.91%
Android 4.2 7,731 10.25%
Android 4.1 3,589 4.76% API 16
Android 4.0.3 - 4.0.4 1,428 1.89% API 15
Android 2.3.3 - 2.3.7 466 0.62% API 10
Others 580 0.77%

======================
And no, you don't need to fork the Google Play store.

We currently have 5 active APKs for ODK Collect on the play store:

1.4.12 API 9+ OpenGL 2.0+
1.4.10 API 9+ OpenGL 1.0+
1.4.7 API 8+ OpenGL 1.0+
1.4.5 API 7+ OpenGL 1.0+
1.1.7 API 4+ OpenGL 1.0+

Devices see the highest of these for which they are compatible. I.e., if you have a device that only supports OpenGL 1.0, you are restricted to 1.4.10 or earlier.

i.e., there is already a mechanism for handling deprecating older devices like this -- you would just release a new version that would only be seen on, e.g., API 14+.

If you think there might be a need to update two or more of these branches, you should add a gap to the versionNumber of the build. Visibility of builds is ranked by versionNumber not the version String. So if you release a 1.4.13 that targets API 14+, and you might still maintain the branch for API 9+, you should jump the version number for 1.4.13 so you have a range of version numbers that can be used for the API 9+ branch.

If you don't do that, then your API 14+ devices will see the API 9+ builds as newly available builds when they come out, which is definitely not desireable.

[lognaturel]

Thanks for the clarifications and suggestions, @mitchellsundt!

@yanokwa do you think that there is a useful list of organizational user contacts that we could ask about Android versions? From what I understand, polling hasn't been super successful in the past but I like the idea of targeting large organizations that we know don't install through the Play Store rather than sending out a general blast to the community list.

[yanokwa]

I will gin up a list over the next few days and reach out to them. I think that this stage, we really need to analytics in the app that work without Play services. The sooner we have those, the less blind we'll be...

[yanokwa]

Just to update the thread, I have emails out to a bunch of folks to see what versions they have. In a week or so, I'll follow up with an email to the community.

My guess is that it's pretty safe the following.

minSdkVersion(16) - Android 4.1, Jelly Bean - This hits 97% of active Google Play users.
targetSdkVersion(19) - Android 4.4, KitKat - This the most popular version of Google play.
compileSdkVersion(19) - Android 4.4, KitKat

[yanokwa]

I've spoken to the project management committee, a handful of largish Collect users and a couple of the commercial organizations that build on Collect. I think we can safely go up to Android 4.1 (16, JellyBean).

My question is if we should go up to Android 4.4 (KitKat, 19). Some of the differences are spelled out at http://socialcompare.com/en/comparison/android-versions-comparison.

@mitchellsundt @jnordling @meletis @dorey @grzesiek2010 @jasonrogena @srsudar @ianume @carstendev Are there APIs in Android 4.4 (KitKat, 19) that you think are a must have?

[mitchellsundt]

When a 4.4 API is needed (when running on a 4.4 or newer device), there are runtime checks to ensure that it is used, so there is no gain to moving to a newer targetSdk version w.r.t. code behavior.

The biggest gain / change is to the look-and-feel of the app, as the update would move you to a new default style and theme (which is where the testing burden in a targetSdk update lies).

W.r.t. APIs, the biggest screw-up in Android API updates have centered around the media store API. This broke several times during the 4.x OSes. If you are intending to pro-actively verify OS compatibility, this is a must-verify area of the code.

When moving beyond 4.x, the big challenge would be the transition to 6.x and 7.x where users get to manage individual permissions -- there would need to be extensive guards added everywhere to catch permissions refusals; I don't think anything major changed in 5.x.

[lognaturel]

Thanks @mitchellsundt! It's really good to know ahead of time about the potential issues around the media store API. We should explore whether we can put any automated testing in place on parts of the code that touch those APIs before making the change.

Do you have a sense of whether we're better off moving targetSdkVersion incrementally or more aggressively? My thinking in suggesting going to 22 now is that each change will require significant UI verification and that the fewer times we have to do it the better.

[mitchellsundt]

I generally only look at jar and SDK updates and eliminating uses of deprecated APIs annually; Google updates libraries and SDKs so frequently that it would require someone working about 25% time to keep up.

Prior to transitioning, if the code is still using the deprecated managedQuery() logic, and hasn't transitioned to the new LoaderManager architecture, that should be updated. This probably requires moving to a Fragment architecture. Also worth looking at any other deprecated API usage (and perhaps updating Google Play Services and eliminating gdata library usage).

Jumping to 22 seems reasonable, because the big change in 5.x was moving to the new material design UI. The shift from DEX to ART largely seemed to impact the dev environment but not the operation of the code. I don't think there are issues with any of the libraries due to that change.

Moving to 23 or beyond adds even more complexity with runtime permissions and 7.0 likely has breaking changes in the area of the media store w.r.t. file:// URLs; I'd do that as a separate transition.

[yanokwa]

Alrightly, I think we've reached consensus. We'll be moving to

• minSdkVersion(16) - Android 4.1, Jelly Bean
• targetSdkVersion(22) - Android 4.4, KitKat
• compileSdkVersion(22) - Android 4.4, KitKat

As far as sequencing, we'll need to get some testing in place first (Issue #392)