Apps-android-commons: Migrate all APIs to retrofit

[maskaravivek]

Summary:

This is a subtask to track network layer refactoring as part of #1092.

• [x] getAuthCookie > PR raised
• [x] login > PR raised
• [x] getEditToken > PR raised
• [x] getWikidataCsrfToken > PR raised
• [x] getCentralAuthToken > PR raised
• [x] fileExistsWithName
• [x] pageExists
• [x] findThumbnailByFilename
• [x] getCategoryImages
• [x] getSubCategoryList
• [x] getParentCategoryList
• [x] searchCategory
• [x] uploadFile > PR raised
• [x] edit > PR raised
• [x] prependEdit > PR raised
• [x] appendEdit > PR raised
• [x] wikidatCreateClaim > PR raised
• [x] addWikidataEditTag > PR raised
• [x] fetchMediaByFilename
• [x] searchCategories
• [x] allCategories
• [x] getNotifications
• [x] markNotificationAsRead > PR raised
• [x] searchTitles
• [x] revisionsByFilename
• [x] existingFile
• [x] logEvents
• [x] isUserBlockedFromCommons

Suggested test flows (by misaochan) for backend-overhaul branch:

• [x] Log in, upload a picture. Make sure the upload does not fail and that it is associated with the correct account (check this in Commons). Log out, then log in to a different account. Repeat the process. Log out, then log back in to the original account. Repeat the process for the last time.
• [x] Test with 2FA account if you have one
• [x] Try uploading a file that already exists or a title that already exists
• [x] Test notifications
• [x] Test Achievements
• [x] (After Vivek reports this is fixed): Test Wikidata edits from Nearby

[ilgazer]

Can I help with this or should I refrain from doing grant tasks?

[maskaravivek]

You are most welcome to contribute. Our grant doesn't define the APIs specifically and given that we are supposed to overhaul the entire app, some help would be great. :)

You can pick any of the pending APIs. I have updated the comment to reflect which ones have a PR raised already. You can pick from the remaining ones.

It would be best to pick just 1 API at a time and raise a PR for it separately so that it can be merged quickly. Base your PR to backend-overhaul branch

[ilgazer]

That's nice to hear!

[ilgazer]

I also completed searchCategory :)

[misaochan]

Added suggested test flows - if anyone wants to help with testing, we would greatly appreciate that, as more eyes are better. :) I will start testing soon.

[misaochan]

Does the backend-overhaul branch work for anyone else at the moment? I tested the branch with a real device running Android 8.1. After logging in I cannot even test any of the upload flows, because my main screen is completely empty (as in, there isn't even the upload FAB, lol).

Achievements screen loads but thinks that I have no uploads.

Logging out and logging in to a different account had the same problem.

I submitted logs to the forum. The relevant parts are posted below:

2019-07-10 23:33:57,888     [DEBUG] [file-logging-thread-1] [LoginActivity] : Login to start!
2019-07-10 23:34:01,147     [ERROR] [file-logging-thread-1] [SessionManager] : User is not logged in
2019-07-10 23:34:01,181     [INFO ] [file-logging-thread-1] [OkHttpJsonApiClient] : Url https://tools.wmflabs.org/urbanecmbot/commonsmisc/feedback.py?user=misaochan
2019-07-10 23:34:01,396     [DEBUG] [file-logging-thread-1] [MainActivity] : Contributions activity notifications menu item is visible
2019-07-10 23:34:01,776     [DEBUG] [file-logging-thread-1] [MainActivity] : Number of notifications is 2
2019-07-10 23:34:01,985     [DEBUG] [file-logging-thread-1] [MainActivity] : Number of notifications is 2
2019-07-10 23:34:01,998     [DEBUG] [file-logging-thread-1] [CustomApiResult] : API response for method https://commons.wikimedia.org/w/api.php is
 <?xml version="1.0" encoding="UTF-8"?><api batchcomplete=""><query><userinfo id="0" name="IP address redacted" anon=""/></query></api>
2019-07-10 23:34:02,728     [DEBUG] [file-logging-thread-1] [OkHttpJsonApiClient] : Response for achievements is Content-type: application/json

[maskaravivek]

@misaochan Can you test on a clean install?

[misaochan]

@maskaravivek This is a clean install.

[neslihanturan]

Beta debug, API 26 emulator
-I was already logged in an uploaded an image successfully. But having no progressbar during upload seemed a little weird. I checked uploaded image page, everything seemed well to me.
Then I logged out and in, what I see is same with @misaochan :

Here are the logs I see on clicking home from navbar:

*.wiktionary.org *.wmflabs.org wikimediafoundation.org mediawiki.org *.mediawiki.org wss://tools.wmflabs.org; report-uri https://tools.wmflabs.org/csp-report/collect;
2019-07-11 00:47:39.803 9245-9962/fr.free.nrw.commons.beta V/OkHttp: 402
2019-07-11 00:47:39.804 9245-9962/fr.free.nrw.commons.beta V/OkHttp: <-- END HTTP (4-byte body)
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: <-- 200 https://tools.wmflabs.org/urbanecmbot/commonsmisc/feedback.py?labs=commonswiki&user=Neslihan%20Test (597ms)
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: server: nginx/1.13.6
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: date: Wed, 10 Jul 2019 21:47:40 GMT
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: content-length: 31
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: access-control-allow-origin: *
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: strict-transport-security: max-age=86400
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: x-clacks-overhead: GNU Terry Pratchett
2019-07-11 00:47:39.842 9245-10457/fr.free.nrw.commons.beta V/OkHttp: content-security-policy-report-only: default-src 'self' 'unsafe-eval' 'unsafe-inline' blob: data: filesystem: mediastream: wikibooks.org *.wikibooks.org wikidata.org *.wikidata.org wikimedia.org *.wikimedia.org wikinews.org *.wikinews.org wikipedia.org *.wikipedia.org wikiquote.org *.wikiquote.org wikisource.org *.wikisource.org wikiversity.org *.wikiversity.org wikivoyage.org *.wikivoyage.org wiktionary.org *.wiktionary.org *.wmflabs.org wikimediafoundation.org mediawiki.org *.mediawiki.org wss://tools.wmflabs.org; report-uri https://tools.wmflabs.org/csp-report/collect;
2019-07-11 00:47:39.843 9245-10457/fr.free.nrw.commons.beta V/OkHttp: Content-type: application/json
2019-07-11 00:47:39.843 9245-10457/fr.free.nrw.commons.beta V/OkHttp: <-- END HTTP (31-byte body)
2019-07-11 00:47:39.844 9245-10457/fr.free.nrw.commons.beta D/OkHttpJsonApiClient: Response for achievements is Content-type: application/json
2019-07-11 00:47:39.844 9245-9283/fr.free.nrw.commons.beta D/EGL_emulation: eglMakeCurrent: 0xa27055a0: ver 2 0 (tinfo 0xa2703500)
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: <-- 200 https://commons.wikimedia.beta.wmflabs.org/w/api.php?format=json&formatversion=2&errorformat=plaintext&action=query&meta=notifications&notformat=model&notlimit=max&notwikis=wikidatawiki%7Ccommonswiki%7Cenwiki&notfilter=%21read (656ms)
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: date: Wed, 10 Jul 2019 21:47:40 GMT
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: content-type: application/json; charset=utf-8
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: server: deployment-mediawiki-09.deployment-prep.eqiad.wmflabs
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-powered-by: HHVM/3.18.6-dev
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-frame-options: DENY
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-content-type-options: nosniff
2019-07-11 00:47:39.898 9245-10460/fr.free.nrw.commons.beta D/OkHttp: content-disposition: inline; filename=api-result.json
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: cache-control: private, must-revalidate, max-age=0
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: backend-timing: D=447017 t=1562795259908954
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: vary: Accept-Encoding,X-Seven
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-varnish: 223062829, 55652876
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: via: 1.1 varnish (Varnish/5.1), 1.1 varnish (Varnish/5.1)
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: accept-ranges: bytes
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: age: 0
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-cache: deployment-cache-text05 pass, deployment-cache-text05 pass
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-cache-status: pass
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: server-timing: cache;desc="pass"
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-analytics: WMF-Last-Access=10-Jul-2019;WMF-Last-Access-Global=10-Jul-2019;https=1
2019-07-11 00:47:39.899 9245-10460/fr.free.nrw.commons.beta D/OkHttp: x-client-ip: 88.230.48.157
2019-07-11 00:47:39.900 9245-10460/fr.free.nrw.commons.beta D/OkHttp: {"batchcomplete":true,"limits":{"notifications":50},"query":{"notifications":{"list":[{"wiki":"commonswiki","id":5919,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-03-21T11:35:45Z","utcunix":"1553168145","unix":"1553168145","utcmw":"20190321113545","mw":"20190321113545","date":"21 March"},"title":{"full":"File:Deneme 2.jpg","namespace":"File","namespace-key":6,"text":"Deneme 2.jpg"},"agent":{"id":2265,"name":"Silky Priya 44"},"revid":106467,"targetpages":[],"*":{"header":"‪Silky Priya 44‬ thanked you for your creation of <strong>‪File:Deneme 2.jpg‬</strong>.","compactHeader":"‪Silky Priya 44‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Deneme_2.jpg&oldid=prev&diff=106467&markasread=5919&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Silky_Priya_44","label":"‪Silky Priya 44‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Deneme_2.jpg","label":"‪Deneme 2.jpg‬","tooltip":"File:Deneme 2.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":5956,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-03-25T08:20:26Z","utcunix":"1553502026","unix":"1553502026","utcmw":"20190325082026","mw":"20190325082026","date":"25 March"},"title":{"full":"File:1 2.jpg","namespace":"File","namespace-key":6,"text":"1 2.jpg"},"agent":{"id":2218,"name":"Aanshika"},"revid":106442,"targetpages":[],"*":{"header":"‪Aanshika‬ thanked you for your creation of <strong>‪File:1 2.jpg‬</strong>.","compactHeader":"‪Aanshika‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:1_2.jpg&oldid=prev&diff=106442&markasread=5956&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Aanshika","label":"‪Aanshika‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:1_2.jpg","label":"‪1 2.jpg‬","tooltip":"File:1 2.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":5967,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-03-25T08:41:07Z","utcunix":"1553503267","unix":"1553503267","utcmw":"20190325084107","mw":"20190325084107","date":"25 March"},"title":{"full":"File:Test2 3.jpg","namespace":"File","namespace-key":6,"text":"Test2 3.jpg"},"agent":{"id":2218,"name":"Aanshika"},"revid":106464,"targetpages":[],"*":{"header":"‪Aanshika‬ thanked you for your creation of <strong>‪File:Test2 3.jpg‬</strong>.","compactHeader":"‪Aanshika‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Test2_3.jpg&oldid=prev&diff=106464&markasread=5967&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Aanshika","label":"‪Aanshika‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Test2_3.jpg","label":"‪Test2 3.jpg‬","tooltip":"File:Test2 3.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":5974,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-03-25T09:27:46Z","utcunix":"1553506066
2019-07-11 00:47:39.900 9245-10460/fr.free.nrw.commons.beta D/OkHttp: ,"unix":"1553506066","utcmw":"20190325092746","mw":"20190325092746","date":"25 March"},"title":{"full":"File:Tes.jpg","namespace":"File","namespace-key":6,"text":"Tes.jpg"},"agent":{"id":2218,"name":"Aanshika"},"revid":106441,"targetpages":[],"*":{"header":"‪Aanshika‬ thanked you for your creation of <strong>‪File:Tes.jpg‬</strong>.","compactHeader":"‪Aanshika‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Tes.jpg&oldid=prev&diff=106441&markasread=5974&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Aanshika","label":"‪Aanshika‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Tes.jpg","label":"‪Tes.jpg‬","tooltip":"File:Tes.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":6315,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-04-25T12:48:42Z","utcunix":"1556196522","unix":"1556196522","utcmw":"20190425124842","mw":"20190425124842","date":"25 April"},"title":{"full":"File:Test 117.jpg","namespace":"File","namespace-key":6,"text":"Test 117.jpg"},"agent":{"id":2293,"name":"Nahid"},"revid":107410,"targetpages":[],"*":{"header":"‪Nahid‬ thanked you for your creation of <strong>‪File:Test 117.jpg‬</strong>.","compactHeader":"‪Nahid‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Test_117.jpg&oldid=prev&diff=107410&markasread=6315&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Nahid","label":"‪Nahid‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Test_117.jpg","label":"‪Test 117.jpg‬","tooltip":"File:Test 117.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":6320,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-04-25T14:15:44Z","utcunix":"1556201744","unix":"1556201744","utcmw":"20190425141544","mw":"20190425141544","date":"25 April"},"title":{"full":"File:Test XMP on.jpg","namespace":"File","namespace-key":6,"text":"Test XMP on.jpg"},"agent":{"id":2293,"name":"Nahid"},"revid":108055,"targetpages":[],"*":{"header":"‪Nahid‬ thanked you for your creation of <strong>‪File:Test XMP on.jpg‬</strong>.","compactHeader":"‪Nahid‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Test_XMP_on.jpg&oldid=prev&diff=108055&markasread=6320&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Nahid","label":"‪Nahid‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Test_XMP_on.jpg","label":"‪Test XMP on.jpg‬","tooltip":"File:Test XMP on.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":6321,"type":"edit-thank","category":"edit-thank","section":"message","timestamp":{"utciso8601":"2019-04-25T14:19:18Z","utcunix":"1556201958","unix":"1556201958","utcmw":"20190425141918","mw":"20190425141918","date":"25 April"},"title":{"full":"File:Testing default exif values2.jpg","namespace":"File","namespace-key":6,"text":"Testing default exif values2.jpg"},"agent":{"id":2293,"name":"Nahid"},"revid":108050,"targetpage
2019-07-11 00:47:39.900 9245-10460/fr.free.nrw.commons.beta D/OkHttp: ":[],"*":{"header":"‪Nahid‬ thanked you for your creation of <strong>‪File:Testing default exif values2.jpg‬</strong>.","compactHeader":"‪Nahid‬ thanked you.","body":"Uploaded using [[COM:MOA|Commons Mobile App]]","icon":"thanks","links":{"primary":{"url":"https://commons.wikimedia.beta.wmflabs.org/w/index.php?title=File:Testing_default_exif_values2.jpg&oldid=prev&diff=108050&markasread=6321&markasreadwiki=commonswiki","label":"View edit"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/User:Nahid","label":"‪Nahid‬","tooltip":"","description":"","icon":"userAvatar","prioritized":true},{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/File:Testing_default_exif_values2.jpg","label":"‪Testing default...‬","tooltip":"File:Testing default exif values2.jpg","description":null,"icon":"article","prioritized":true}]},"iconUrl":"/w/extensions/Thanks/userTalk-constructive-ltr.svg"}},{"wiki":"commonswiki","id":6797,"type":"login-fail-new","category":"login-fail","section":"alert","timestamp":{"utciso8601":"2019-06-10T09:22:06Z","utcunix":"1560158526","unix":"1560158526","utcmw":"20190610092206","mw":"20190610092206","date":"10 June"},"agent":{"id":1694,"name":"Neslihan Test"},"targetpages":[],"*":{"header":"There has been <b>a failed attempt</b> to log in to your account from a new device. Please make sure your account has a strong password.","compactHeader":"There has been <b>a failed attempt</b> to log in to your account from a new device. Please make sure your account has a strong password.","body":"","icon":"LoginNotify-user-avatar","links":{"primary":{"url":"https://mediawiki.org/wiki/Help:Login_notifications?markasread=6797&markasreadwiki=commonswiki","label":"Help"},"secondary":[{"url":"https://commons.wikimedia.beta.wmflabs.org/wiki/Special:ChangePassword","label":"Change password","description":"","icon":"lock","prioritized":true}]},"iconUrl":"/w/extensions/LoginNotify/UserAvatar.svg"}}],"continue":null}}}
2019-07-11 00:47:39.900 9245-10460/fr.free.nrw.commons.beta D/OkHttp: <-- END HTTP (10103-byte body)
2019-07-11 00:47:40.004 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7139808 , only wrote 7139520
2019-07-11 00:47:40.005 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7139530 , only wrote 7139520
2019-07-11 00:47:40.094 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7143819 , only wrote 7143120
2019-07-11 00:47:40.096 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7143220 , only wrote 7143120
2019-07-11 00:47:40.182 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7147256 , only wrote 7146720
2019-07-11 00:47:40.183 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7146726 , only wrote 7146720
2019-07-11 00:47:40.204 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7147759 , only wrote 7147440
2019-07-11 00:47:40.236 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7148964 , only wrote 7148160
2019-07-11 00:47:40.271 9245-9283/fr.free.nrw.commons.beta D/EGL_emulation: eglMakeCurrent: 0xa27055a0: ver 2 0 (tinfo 0xa2703500)
2019-07-11 00:47:40.286 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7150540 , only wrote 7150320
2019-07-11 00:47:40.299 9245-9245/fr.free.nrw.commons.beta D/MainActivity: Number of notifications is 8
2019-07-11 00:47:40.324 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7152146 , only wrote 7151760
2019-07-11 00:47:40.444 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7157538 , only wrote 7156800
2019-07-11 00:47:40.448 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7156993 , only wrote 7156800
2019-07-11 00:47:40.473 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7158007 , only wrote 7157520
2019-07-11 00:47:40.473 1436-6524/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7157525 , only wrote 7157520
2019-07-11 00:47:40.646 9245-9283/fr.free.nrw.commons.beta D/OpenGLRenderer: endAllActiveAnimators on 0x8b23a400 (NavigationMenuItemView) with handle 0x8e7d2c20
2019-07-11 00:47:41.355 1679-1697/system_process E/BatteryStatsService: modem info is invalid: ModemActivityInfo{ mTimestamp=0 mSleepTimeMs=0 mIdleTimeMs=0 mTxTimeMs[]=[0, 0, 0, 0, 0] mRxTimeMs=0 mEnergyUsed=0}
2019-07-11 00:47:41.932 2231-9231/com.google.android.gms.persistent W/ctxmgr: [AclManager]No 2 for (accnt=account#-517948760#, com.google.android.gms(10013):IndoorOutdoorProducer, vrsn=13280000, 0, 3pPkg = null ,  3pMdlId = null ,  pid = 2231). Was: 3 for 57, account#-517948760#
2019-07-11 00:47:42.258 1436-1637/? W/audio_hw_generic: Not supplying enough data to HAL, expected position 7327981 , only wrote 7243200
2019-07-11 00:47:45.687 9245-9271/fr.free.nrw.commons.beta I/zygote: Explicit concurrent copying GC freed 22305(1550KB) AllocSpace objects, 7(144KB) LOS objects, 49% free, 4MB/9MB, paused 143us total 19.445ms
2019-07-11 00:47:46.932 2231-9231/com.google.android.gms.persistent W/ctxmgr: [AclManager]No 2 for (accnt=account#-517948760#, com.google.android.gms(10013):IndoorOutdoorProducer, vrsn=13280000, 0, 3pPkg = null ,  3pMdlId = null ,  pid = 2231). Was: 3 for 57, account#-517948760#
2019-07-11 00:47:48.294 1450-1450/? I/boot-pipe: done populating /dev/random

• Notifications activiy is correct but the notification bell number is wrong, it does not change even if after I read notificationsw (possibly not related with this PR).

• Achievements worked fine on me:
  

[misaochan]

BTW, I also realized that we cannot rely on users doing a fresh install - most users will just be updating their app automatically, not uninstalling and reinstalling. So we must make sure that our final solution works regardless of fresh install or not.

[misaochan]

Tested with real device running Android 8.1 today.

Successes:

• Contributions and the FAB etc loads now! 👍
• Single upload from Gallery succeeded - https://commons.wikimedia.org/wiki/File:Mushroom_chicken_tea_broth.jpg. Category selection works as expected. Author, license, etc is correct.
• Multiple uploads with Google Photos succeeded. This one seems to have the dates correct - https://commons.wikimedia.org/wiki/File:Newstead_house.jpg . Suffixes appended correctly.
• Logged out of misaochan, logged into misaochan2, and uploaded a single image from Gallery. Upload works and is attributed to the correct account - https://commons.wikimedia.org/wiki/File:Water_rat_.jpg
• Uploading a file that already exists on Commons is detected appropriately
• Uploading a file with existing name is detected appropriately
• Notifications works

Issues found with Contributions:

• This screen is shown for a long time while contributions load, which may make users think that the app is buggy (alongside the 0 bug below)
  
• My "Contributions" tab always says (0) even after contributions have been loaded
• Very interesting bug - as you can see, these image titles do not match the images. This happened after I logged out of misaochan and onto misaochan2. So the titles are the titles of misaochan2's uploads, but the images are the images of misaochan's uploads.
  
• Furthermore, after logging back into misaochan, the images are mine and the titles are mine, but out of sync. So the first image has the title of the most recent image uploaded, but the image shown is actually the one that belongs to an upload that is 4 items down.

Issues found with single uploads (with Android stock gallery):

• My description defaulted to Afrikaans even though "en" was the default desc language - https://commons.wikimedia.org/wiki/File:Mushroom_chicken_tea_broth.jpg
• Dates appear to be using the upload date again instead of the date taken - the picture above was taken on 15 Jul

Issues found with multiple uploads (with Android stock gallery):

• Unable to select more than 1 image in Android stock gallery. It works for Google Photos.

Issues found with multiple uploads (with Google Photos):

• My description defaulted to Afrikaans even though "en" was the default desc language

Achievements:

• Does not work, 0 for everything

Other issues:

[neslihanturan]

For me, similar to @misaochan 's. Achievements is blank screen:

• Contributions loads this time.
• Uploads works.
• Potential upload issue warnings displayed.
• Nearest place notification permission is requested after selecting nearby tab and coming back to contributions tab, it didn't ask on start (possibly irrelevant)
• Contributions is not zero for me, it says 100 in my case

Logs:

2019-07-16 16:12:05.565 7851-7851/fr.free.nrw.commons.beta E/DialogUtil: Could not show dialog.
    java.lang.IllegalStateException: The specified child already has a parent. You must call removeView() on the child's parent first.
        at android.view.ViewGroup.addViewInner(ViewGroup.java:4915)
        at android.view.ViewGroup.addView(ViewGroup.java:4746)
        at android.view.ViewGroup.addView(ViewGroup.java:4718)
        at com.android.internal.app.AlertController.setupCustomContent(AlertController.java:606)
        at com.android.internal.app.AlertController.setupView(AlertController.java:500)
        at com.android.internal.app.AlertController.installContent(AlertController.java:258)
        at android.app.AlertDialog.onCreate(AlertDialog.java:425)
        at android.app.Dialog.dispatchOnCreate(Dialog.java:403)
        at android.app.Dialog.show(Dialog.java:302)
        at fr.free.nrw.commons.utils.DialogUtil.showSafely(DialogUtil.java:33)
        at fr.free.nrw.commons.utils.DialogUtil.showAlertDialog(DialogUtil.java:140)
        at fr.free.nrw.commons.utils.DialogUtil.showAlertDialog(DialogUtil.java:96)
        at fr.free.nrw.commons.contributions.ContributionsFragment.showNearbyCardPermissionRationale(ContributionsFragment.java:508)
        at fr.free.nrw.commons.contributions.ContributionsFragment.checkPermissionsAndShowNearbyCardView(ContributionsFragment.java:489)
        at fr.free.nrw.commons.contributions.ContributionsFragment.onResume(ContributionsFragment.java:469)
        at androidx.fragment.app.Fragment.performResume(Fragment.java:2499)
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:926)
        at androidx.fragment.app.FragmentManagerImpl.moveFragmentToExpectedState(FragmentManagerImpl.java:1229)
        at androidx.fragment.app.FragmentManagerImpl.moveToState(FragmentManagerImpl.java:1295)
        at androidx.fragment.app.FragmentManagerImpl.dispatchStateChange(FragmentManagerImpl.java:2605)
        at androidx.fragment.app.FragmentManagerImpl.dispatchResume(FragmentManagerImpl.java:2577)
        at androidx.fragment.app.FragmentController.dispatchResume(FragmentController.java:267)
        at androidx.fragment.app.FragmentActivity.onResumeFragments(FragmentActivity.java:463)
        at androidx.fragment.app.FragmentActivity.onPostResume(FragmentActivity.java:453)
        at androidx.appcompat.app.AppCompatActivity.onPostResume(AppCompatActivity.java:173)
        at android.app.Activity.performResume(Activity.java:7103)
        at android.app.ActivityThread.performResumeActivity(ActivityThread.java:3620)
        at android.app.ActivityThread.handleResumeActivity(ActivityThread.java:3685)
        at android.app.ActivityThread$H.handleMessage(ActivityThread.java:1643)
        at android.os.Handler.dispatchMessage(Handler.java:105)
        at android.os.Looper.loop(Looper.java:164)
        at android.app.ActivityThread.main(ActivityThread.java:6541)
        at java.lang.reflect.Method.invoke(Native Method)
        at com.android.internal.os.Zygote$MethodAndArgsCaller.run(Zygote.java:240)
        at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:767)
2019-07-16 16:12:05.567 7851-7851/fr.free.nrw.commons.beta E/ContributionsFragment: onFragmentResumed fr.free.nrw.commons.contributions.ContributionsListFragment
2019-07-16 16:12:06.551 7851-7851/fr.free.nrw.commons.beta E/QuizChecker: Fetching upload count failed
    java.lang.IllegalStateException: closed
        at okio.RealBufferedSource.rangeEquals(RealBufferedSource.java:407)
        at okio.RealBufferedSource.rangeEquals(RealBufferedSource.java:401)
        at okhttp3.internal.Util.bomAwareCharset(Util.java:471)
        at okhttp3.ResponseBody.string(ResponseBody.java:175)
        at fr.free.nrw.commons.mwapi.OkHttpJsonApiClient.lambda$getUploadCount$0(OkHttpJsonApiClient.java:102)
        at fr.free.nrw.commons.mwapi.-$$Lambda$OkHttpJsonApiClient$9ZpGX0U5Zngts_jcRCFls7FtEIM.call(Unknown Source:4)
        at io.reactivex.internal.operators.single.SingleFromCallable.subscribeActual(SingleFromCallable.java:44)
        at io.reactivex.Single.subscribe(Single.java:3438)
        at io.reactivex.internal.operators.single.SingleSubscribeOn$SubscribeOnObserver.run(SingleSubscribeOn.java:89)
        at io.reactivex.Scheduler$DisposeTask.run(Scheduler.java:578)
        at io.reactivex.internal.schedulers.ScheduledRunnable.run(ScheduledRunnable.java:66)
        at io.reactivex.internal.schedulers.ScheduledRunnable.call(ScheduledRunnable.java:57)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:301)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1162)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:636)
        at java.lang.Thread.run(Thread.java:764)

[ilgazer]

Can you also update the list?

[ilgazer]

These are both waiting for wikimedia-android-data-client PR's

• isUserBlockedFromCommons -> wikimedia/wikimedia-android-data-client#22
• logEvents -> wikimedia/wikimedia-android-data-client#23

[misaochan]

Can you also update the list?

Which list do you mean?

These are both waiting for wikimedia-android-data-client PR's

Thanks for your help with all this @ilgazer ! :) Let me know if it hasn't been merged after a week or so and I'll try to ping the maintainer.

[ilgazer]

Which list do you mean?

The one above

These are both waiting for wikimedia-android-data-client PR's

They no longer are! Their PR's are waiting for merge, but a completely unrelated test is failing for some reason.

[ashishkumar468]

I think @ilgazer 's PR can be merged, I have tried testing them locally and both seem to work fine on my machine. What do you say @maskaravivek ?

[maskaravivek]

@ashishkumar468 Yes, lets merge @ilgazer PRs.

[ashishkumar468]

Other than the contributions (upload count and possible display inconsistency), and notification progress issue and the achievements bug, is there anything else remaining in this PR ?

[ilgazer]

There is CategoryApi, a class that thtat uses JSON but should be migrated
to retrofit. I'm working on it, along with a rewrite of FileProcessor.

[ashishkumar468]

Thanks @ilgazer :)

[maskaravivek]

Other than the contributions (upload count and possible display inconsistency), and notification progress issue and the achievements bug, is there anything else remaining in this PR ?

Wikidata edits is still not sorted. We might need external help for it.

[misaochan]

Other than the contributions (upload count and possible display inconsistency), and notification progress issue and the achievements bug, is there anything else remaining in this PR ?

Thanks for fixing the Achievements and wrong upload count in #3151 @ashishkumar468 . :) Based on
https://github.com/commons-app/apps-android-commons/issues/3026#issuecomment-511732535 , it seems like the remaining issues are:

• [x] Display issue: Very interesting bug - as you can see, these image titles do not match the images. This happened after I logged out of misaochan and onto misaochan2. So the titles are the titles of misaochan2's uploads, but the images are the images of misaochan's uploads. <- Fixed by #3168

Furthermore, after logging back into misaochan, the images are mine and the titles are mine, but out of sync. So the first image has the title of the most recent image uploaded, but the image shown is actually the one that belongs to an upload that is 4 items down.
Issues found with single uploads (with Android stock gallery):

• [x] Desc language: My description defaulted to Afrikaans even though "en" was the default desc language - https://commons.wikimedia.org/wiki/File:Mushroom_chicken_tea_broth.jpg

• [ ] Wrong dates: Dates appear to be using the upload date again instead of the date taken - the picture above was taken on 15 Jul

• [x] Issues found with multiple uploads (with Android stock gallery): Unable to select more than 1 image in Android stock gallery. It works for Google Photos.

• [ ] Overwrites: when I was testing #3166 I uploaded a file titled "Story Bridge". The app warned me that the file exists, as it should, but it did not add a numbered suffix as it usually does. Instead, it overwrote https://commons.wikimedia.org/wiki/File:Story_Bridge.jpg

[ilgazer]

Wrong dates: Dates appear to be using the upload date again instead of the date taken - the picture above was taken on 15 Jul

IMO we should rewrite FileProcessor and the weird code surrounding it. That will probably also resolve this bug. I've started working on it.

[ashishkumar468]

@misaochan I am not able to reproduce this . "Issues found with multiple uploads (with Android stock gallery): Unable to select more than 1 image in Android stock gallery. It works for Google Photos.". Can you reverify this on the latest commit form this branch, a screenshot for the same would be better, could be a device-specific issue. Just to confirm that is is a bug added by this branch, can you see if the same works for you on master. Has anyone else also faced this issue @maskaravivek @neslihanturan.

[ashishkumar468]

@misaochan About this " Dates appear to be using the upload date again instead of the date taken - the picture above was taken on 15 Jul", I am not able to reproduce this as well, I uploaded one of the images you had sent me (created date 20/05/2016), which seems to show up correctly the web app as well. Attaching the screenshot for the same

[misaochan]

@ashishkumar468 I can't quite figure out the date bug. Upon testing the current version of backend-overhaul, I now get a completely unrelated date.

For instance, I uploaded https://commons.wikimedia.org/wiki/File:Jacaranda_in_bloom.jpg from Google Photos single upload. The template says the photo's date is 7 September, but I definitely took it on 7 October. The EXIF information on Google Photos corresponds to that.

[ashishkumar468]

Hi @misaochan , If this is the case, I am sure this must be there on master as well, if that comes out to be true, can we take that separately?

[misaochan]

Sure, I'm OK with merging backend-overhaul to master with this bug present (might be easier than trying to fix everything before merging), but we still have to fix it before releasing v2.12 with it. :)

[misaochan]

@misaochan I am not able to reproduce this . "Issues found with multiple uploads (with Android stock gallery): Unable to select more than 1 image in Android stock gallery. It works for Google Photos.". Can you reverify this on the latest commit form this branch, a screenshot for the same would be better, could be a device-specific issue.

Yes, the problem still occurs for me on the latest commit. I tested using a Samsung Galaxy s9 running Android 9.0.

This is the stock Android gallery after tapping the FAB "+" button > Gallery. Long-pressing on any image does not initiate multiple selection.

This is the Google Photos gallery after tapping the FAB "+" button > Photos.

I recall encountering a similar problem when we were selecting an image picker library to implement #604 (see also #2286)

[ashishkumar468]

Hi @misaochan, Can you verify if this is happening to you on master as well ?

[misaochan]

Oops, another bug... when I was testing #3166 I uploaded a file titled "Story Bridge". The app warned me that the file exists, as it should, but it did not add a numbered suffix as it usually does. Instead, it overwrote https://commons.wikimedia.org/wiki/File:Story_Bridge.jpg

(Tested master for the multiple uploads bug. Does not work for me on master either. :/)

[ashishkumar468]

@misaochan So if it does not work for you on master as well, we should probably create a separate issue for that. I am sure this is something device-specific (weird though). Thanks for finding the other bug JO, I will look into that, so other than cross-wiki thing, only this is left on backend-ovberhaul, right ?

[misaochan]

Yeah, if it works on your device, I think it's OK to address it separately. In order to be considered complete, the backend overhaul task needs 2FA logins to work in addition to the overwrite bug and cross-wiki calls. But it should be fine to merge the backend-overhaul branch to master after the overwrite bug is fixed, so that alpha testers can try it asap.

[maskaravivek]

All the items in this task have now been completed. Closing this issue now. :)

Thank you, everyone, for your contributions. :)