Qksms: F-Droid AntiFeatures

[IzzySoft]

An update of QKSMS brought in several non-free dependencies, including tracking. So as a quick action, we needed to mark your app with NonFreeDep (PlayStore/Market), NonFreeNet (Billing, Install Referrer) and Tracking (Install Referrer opening a connection right at update, without the user's permission):

Offending libs:
---------------
* BillingClient (/com/android/billingclient): NonFreeNet
* Play Install Referrer Library (/com/android/installreferrer): NonFreeNet,Tracking
* Google Play (/com/android/vending): NonFreeNet
* Android Market (/com/google/android/finsky): NonFreeNet

For reference:

• https://gitlab.com/fdroid/fdroiddata/-/issues/2260
• https://gitlab.com/fdroid/fdroiddata/-/merge_requests/8052

We assume this was an "oversight" at your end, and those dependencies should not go into the noAnalytics flavor. So once you've fixed it at your end (excluding them from that flavor) and that update reached the F-Droid repo, we can (and will) remove those AntiFeatures again.

Thanks for taking care!

[moezbhatti]

definitely an oversight - I'll get this fixed later this week!

[IzzySoft]

Thanks! I'll try to be as quick with removing the AntiFeatures then as I had to be now adding them (apologies for that, I hope you understand the reasoning).

[moezbhatti]

no problem, completely understandable!

[IzzySoft]

@moezbhatti any progress? Which week did you refer to? Users start asking and getting-scared-away.

[moezbhatti]

@IzzySoft sorry for the delay - I haven't had much free time lately. I just pushed up ff4170af324eff7446f6ae41800c245f9fef1e0d which resolves the issue

I still need to fix a bug that was introduced in the 3.9.0 release, then I can go ahead and publish 3.9.1 which will include these changes

[IzzySoft]

Thanks @moezbhatti! But don't you want to keep the issue open until solved? Whichever you chose: Please let us know when we should check to remove the AntiFeature. Thanks!

[moezbhatti]

Generally I close issues once the relevant changes have been merged into master and are ready to deploy - as this makes it easier for me to write my changelogs. I suppose this is pretty misleading though, since it could confuse people who want to check on the status of an issue like this, where the changed have been made but not deployed

In either case - will update you! I think I've solved the bug, just waiting for some users to test and confirm. Hopefully I can have an updated release for you tomorrow

[moezbhatti]

Alright @IzzySoft, we should be good to go with v3.9.1!

[IzzySoft]

Great! Let me know once it arrived in the index, as before that I cannot verify. Thanks!

[moezbhatti]

Looks like it's up! https://gitlab.com/fdroid/fdroiddata/-/commit/f90fb3a8b78097cce35ffae0a665c6d9e24ff510

[IzzySoft]

That's a commit, not a link to the repo. A metadata commit doesn't mean there's a successful build that was published – it rather means there hopefully will be one soon™. So let me quote myself with emphasis:

Let me know once it arrived _in the index,_ as before that I cannot verify.

That commit was just 6h ago. It's not even in the currently running build queue yet. So how am I supposed to verify the APK? :wink:

[moezbhatti]

Ah I see. Sorry, much of the F-Droid process is lost on me. I saw that it took about a day for the new release to show up in the Build Metadata, so I figured it had probably been built in that time

I'm not sure what _the index_ refers to - presumably you want to know once the build shows up here? Or is there somewhere else I should be looking?

[IzzySoft]

Press the A key: yes to all :smile: Exactly that. "The Index" is not the "speaking one" from Orson Scott Card's Homecoming Saga – but rather the index-v1.json resp. index.xml file distributed to the clients and our website. When the APK is listed in that index, it is available (new APKs get synced with the resp. "index release").

[ghost]

@moezbhatti The app is definitely beautiful, waiting for the anti features to get removed to install it
Keep up the good work

[moezbhatti]

@IzzySoft this is maybe a dumb question, but how long does it generally take for build to be added to the index?

[IzzySoft]

Not as long as it currently does (build process got stuck at an obstacle, obstacle meanwhile identified and in the process of being fixed). Usually up to a week from your tagging the release. Let's see if we'll have a "waterfall" when that pipe has been cleaned :mask:

[moezbhatti]

Good to know, thanks!

[ghost]

@moezbhatti the anti features still exist after the update 🤔

[IzzySoft]

Which is strange:

$ scanapk com.moez.QKSMS_2216.apk

Libraries detected:
-------------------
* Android Support v4 (/android/support/v4): Development Framework
* Arch (/androidx/arch): Utility
* AppCompat (/androidx/appcompat): Utility
* Constraint Layout Library (/androidx/constraintlayout): Utility
* Androidx Core (/androidx/core): Utility
* Lifecycle (/androidx/lifecycle): Utility
* Loader (/androidx/loader): Utility
* Media (/androidx/media): Utility
* Transition (/androidx/transition): UI Component
* Vectordrawable (/androidx/vectordrawable): UI Component
* Conductor (/com/bluelinelabs/conductor): Development Framework
* Glide (/com/bumptech/glide): Utility
* Rx Preferences (/com/f2prateek/rx/preferences): Utility
* ReLinker (/com/getkeepsafe/relinker): Utility
* vinnie (/com/github/mangstadt/vinnie): Utility
* ExoPlayer (/com/google/android/exoplayer2): Utility
* FlexboxLayout (/com/google/android/flexbox): Utility
* Google Material Design (/com/google/android/material): Utility
* RxBinding (/com/jakewharton/rxbinding): Utility
* Android SMS/MMS Sending Library (/com/klinker/android/send_message): Utility
* Moshi JSON library (/com/squareup/moshi): Utility
* OkHttp (/com/squareup/okhttp): Utility
* AutoDispose (/com/uber/autodispose): Utility
* RxDogTag (/com/uber/rxdogtag): Utility
* Dagger (/dagger): Utility
* ez-vcard (/ezvcard): Utility
* libphonenumber-android (/io/michaelrocks/libphonenumber/android): Utility
* RxJava (/io/reactivex): Utility
* realm (/io/realm): Utility
* JavaX Dependency Injection (/javax/inject): Utility
* Kotlin (/kotlin): Utility
* kotlinx.coroutines (/kotlinx/coroutines): Utility
* ShortcutBadger (/me/leolin/shortcutbadger): UI Component
* OkHttp okio Framework (/okio): Utility
* Reactive Streams (/org/reactivestreams): Utility
* org.w3c.dom (/org/w3c/dom): Utility
* Timber (/timber/log): Utility
* PhotoView (/uk/co/senab/photoview): UI Component

No offending libs found.

As you can see, those 4 "offending libs" are gone. So at least NonFreeDep can be removed without question I'd say. But it was reported to me that:

in latest available build from fdroid it still promotes n one free(google) […] after installing it asked to rate it on playstore.

So I wonder what's going on here? An app installed from F-Droid shouldn't ask to be rated at Play when started. May I ask you for clarification, @moezbhatti? Are we missing something here?

[ghost]

@moezbhatti something shady going on here? 😏

[moezbhatti]

@moezbhatti the anti features still exist after the update 🤔

@hefty-fading-culprit-epiphany-drew Which ones are you referring to?

So I wonder what's going on here? An app installed from F-Droid shouldn't ask to be rated at Play when started. May I ask you for clarification, @moezbhatti? Are we missing something here?

@IzzySoft Thanks – I just checked and it looks like the rating prompt is agnostic to the build variant. I'll update this so that it only prompts for ratings in the Google Play release

[IzzySoft]

@hefty-fading-culprit-epiphany-drew no reason to think of bad intent. Google can be a bitch at times (sneaking in proprietary stuff as dependency to something they declared FOSS and such), and mistakes happen as well. I didn't think this behavior was intended (and was correct with that as @moezbhatti just confirmed). The only irritating thing is that the call was indeed sent to that Google address, even without the library – but I think the devs are as puzzled about that as we are :wink:

TL;DR: I'm very confident the next release will finally solve the issue. As soon as the APK is available at F-Droid, please verify and let me know, I'll then remove all AntiFeatures. I'll remove NonFreeDep right now as those 4 libs are gone – NonFreeNet and Tracking, even if unintended, will remain until that "ping" is gone.

[moezbhatti]

@IzzySoft with those libraries gone, I believe Tracking should also be removed – unless someone still seeing attempts to connect to Google servers after the update?

[IzzySoft]

@moezbhatti that's why I left it in for now: someone saw connections to *.1e100.net.

[moezbhatti]

@IzzySoft gotcha, thanks. Did they provide any other details? I'm not totally sure where that would be coming from, so any info would be helpful in the investigation

[moezbhatti]

Actually, maybe I have an idea. I store the QKSMS changelog on Firebase, since it makes it easy for me to structure the data for presentation in the app, and host it for free

On app startup, the app pings https://firestore.googleapis.com/v1/projects/qksms-app/databases/(default)/documents/changelog to check if there are any changes to display to the user

I'm not using their SDK for this, I'm just hitting that url with a GET request. There's no authentication, and I'm not attaching any headers to this request, so I'd imagine this should be totally fine – but it might be bad for reasons I'm unaware of. Let me know what you think

[IzzySoft]

Urgs. That would mean NonFreeNet has to stay.

So actually we've got two independent issues here which coincidentally looked related (Play Rating form and a connection to Google servers) but actually are not. But yeah, this perfectly explains things – and I'd strongly recommend adjusting both. That rating thingy you're already on. Why not simply including the changelogs with your assets? For updates, maybe simply put a Changelog.md here in your source repo and link it from your app (making clear it's loaded from Github to avoid complaints)?

No auth & no data sent is one thing. But the IP will always be shown, together with some device characteristics (UA, protocol specifics etc). Firebase is a Google product, they can easily combine that with other data coming from the same device. And even if this might be "just a feather", if multiple apps do so the chicken is naked :wink:

[moezbhatti]

Got it, that makes perfect sense

I used to include it in the assets, but there were a few benefits to hosting it online that made me switch

• If I have a typo to fix or I want to change something, I can do that
• I can support cumulative changelogs for users who jump multiple versions at once, without having to maintain and bundle a file with the entire history of changelogs

And storing it as raw, structured data instead of a webpage means I can easily format it to fit in the app's native UI

All this in mind, I think the solution is just to host it somewhere else. I'll get that sorted out

[IzzySoft]

That's why so many apps request network permissions although the apps themselves would not even need it to do their jobs. Is it really worth it? What else does your app need internet access for? If there's nothing else but the changelog, I'd seriously consider offlining it. You can link to an online version from the about screen, and users can follow the link if they choose to – while not being bothered by the fact of an internet permission (which they cannot deny usually) being present. On the linked site, you can have all your convenience again, preferably using your self-hosted database and not sitting behind Cloudflare or some GAFAM.

Not saying you have to – but that's how I'd approach that. I'd not want to sacrifice privacy for convenience here (and I especially avoid things like Firebase and other Google services whenever I can). All my personal opinion of course – no blaming, framing or anything along such lines :wink:

[moezbhatti]

Unfortunately it wouldn't be possible to get rid of the internet permission, since internet is required for MMS transmission

I'll consider making this feature offline though

[IzzySoft]

Ah, I thought about that for a moment but then thought some system app would take care for this (isn't there something like mmssmsprovider?) OK, in that case you'll need to keep the permission. Still, changelog without AntiFeature would be nice :smile: Thanks for considering!

[giorgosspam]

This is probably not the place for this so apologies in advance for asking here. If indeed so, I'd be very thankful for a hint about where to place my query:

Apparently it is the case that f-droid allowed at least one automatic update of qksms after offending libraries/behaviour were introduced into the source. Judging by the date this issue was opened, I'd assume that this was build 3.9.0, added on f-droid on January 5th, 2021.

Could you please confirm that builds prior to 3.9.0 (and now only available on the f-droid archive repository) were free of any triggers for "Anti-Features"?

Thanks!

PS.: While I am very thankful to @moezbhatti and his work on qksms (and not to mention the entire f-droid team!), I find it distressing to learn that an automatic update on a previously "clean" app introduced tracking and other privacy concerns.

[moezbhatti]

The google billing library has been packaged with the app ever since the beginning – so these I believe the triggers for these anti features were always there

[IzzySoft]

The Billing library would have accounted for NonFreeNet only in the past – only recent changes of Google made this NonFreeDep as well. And Tracking was only due to the InstallReferrer – which AFAIR was only added with the now previous-to-last version and removed again with the now last version.

All else is already explained above, none of it was "bad intent": some libraries slipped in due to "missed dependencies" (already cleaned up), remains of the InstallReferrer were not removed as expected (aka "slipped through despite of other measures") – and the next release will have everything cleaned up. Once that has been confirmed, remaining AntiFeatures will be removed.

As for "F-Droid allowed": Thanks to this incident we noticed an error in the build process (it skipped parts of the scan if I understand correctly; you can ask the fdroidserver team for details), and we're working to fix that as well as to further improve it – which will certainly not happen overnight, but we're on it. So apologies from our end as well.

[giorgosspam]

Thank you both for the quick and exhaustive answers!

@IzzySoft: No need to apologize, I'm glad this has been picked up and is being dealt with.
:-)

[ghost]

@hefty-fading-culprit-epiphany-drew no reason to think of bad intent. Google can be a bitch at times (sneaking in proprietary stuff as dependency to something they declared FOSS and such), and mistakes happen as well

Blaming Google for everything 😏

[IzzySoft]

Blaming Google for everything :smirk:

At times, yes. Whenever they deserve it – like here. It's there strategy, and was from the start: embrace, extend, … drive out. Just take a look at all the AOSP apps. In the beginning, Android was entirely open source. Meanwhile, one by one, components got replaced by Google's proprietary things, and development on the original ones is stopped (eg calendar, keyboard, mail, …). And they still advertize Android as being "open" (which is true only to a degree now). First lure them all in – then stick them in – then they can't leave easily. So yes, blaming Google here. Absolutely. And rightfully.

Not saying other corps are better. And no, I won't expand on this one :rofl:

[halis300]

Not saying other corps are better. And no, I won't expand on this one 🤣

Maybe Pinephone and Pine64 will be one day an alternative with anbox

[mechanic41turk]

The anti-features are still persistent on the fdroid page, even with today's update. @moezbhatti would you consider removing them?

[IzzySoft]

Having checked the APK, I can confirm the AntiFeatures are indeed gone with today's release. I've hence removed them from metadata (and, while being on it, also fixed formatting in the description). Should "go live" with the next index deployment.

Speaking of description: would you be open to establishing Fastlane structures here in your app's repo – so you not only could customize and update it according to your "gusto" without needing merge requests, but also provide screenshots and more? Say yes, and I'll open a PR with initial data to get you started.

[moezbhatti]

Thanks @IzzySoft! However, I haven't moved the changelog offline yet – so the app will still be attempting to hit a Google URL whenever the app updates. Should we include another antifeature / reopen this ticket in the meantime?

[IzzySoft]

You mean we need to add back NonFreeNet? Why not simply disable that check for now in the fdroid build (if possible) until you finally fixed it?

[moezbhatti]

Yeah – that's right. I'm planning on getting the changelog thing addressed in the next update though so it shouldn't be long

[IzzySoft]

Then let's not be more catholic than the pope – or act like politicians in a pandemic crisis (fence up-down-up-down-up…). Your next update fixes it in a couple of days, and that's it. You've kept your word until now, I see no reason for doubt here. Thanks!

[ghost]

@moezbhatti don't close this issue yet, your practice of saying it's fixed while it's not and closing this issue repeatedly raises suspicion 🕵️

[moezbhatti]

@hefty-fading-culprit-epiphany-drew I didn't close the issue, and I didn't claim that it was fixed

[IzzySoft]

@hefty-fading-culprit-epiphany-drew It was I closing the issue, at the point where my check of the APK confirmed the last of the "offending libs" was gone. So if anyone should be blamed for "suspicious behavior" it's rather me. As for @moezbhatti it's just the other way around: despite of the issue being obviously solved and all suspicion being removed, he pointed out a "hidden culprit". To me, this speaks much in favor of him and the project: being transparent and open (instead of "rubbing the hands in glee" that something went through unnoticed). Tapping my head (well, no hat on it at the moment) in respect!

TL;DR: let's avoid "politicians Covid management" (close-open-close-open…) here. I fully trust @moezbhatti keeping his word (quote: "I'm planning on getting the changelog thing addressed in the next update though so it shouldn't be long"). So with the next release being published at F-Droid, this issue should definitely belong to the past.

[ghost]

@moezbhatti anti features are gone.

[ghost]

I am astonished, how you managed to do it without a new release?

[IzzySoft]

AntiFeatures are usually not bound to a specific release. The last release solved the issue, I verified they were gone – and then removed them from the metadata. Next index release reflected those changes. All fine – and a big thanks to @moezbhatti for the prompt help!

[ghost]

Thanks @moezbhatti for patiently listening to my accusations and solving this issue now I can switch back to this app