Apps-android-commons: Don't show hidden categories in suggested categories

[sivaraam]

Summary:
When uploading an image, we ask users to add categories relevant to the image. Currently, the list of categories that we show also includes hidden categories. According to the Commons page about hidden categories:

Many non-topical categories are marked with __HIDDENCAT__ or {{Hiddencat}} on the category page.

As they're non-topical there's not much use in suggesting those categories. Also, I've observed that hidden categories are generally added automatically to the corresponding pages and are not meant to be manually added by users. Examples:

• Category:Pages using authority control without Wikidata link - Wikimedia Commons
• Category:Uses of Wikidata Infobox - Wikimedia Commons

There really seem to be a lot of hidden categories in Commons. Not showing the hidden categories would improve the relevancy of the category search results as it would reduce a lot of irrelevant categories, IMO.

API that could be used

We can identify hidden categories by sending the acprop=hidden to the allcategories API. There seems to be a limitation though. The generator version of that API doesn't seems to be considering the gacprop=hidden parameter sent to it. Something to look into if we consider doing this.

Example API queries (for reference)

• Using list=allcategories: https://commons.wikimedia.org/w/api.php?action=query&format=json&formatversion=2&list=allcategories&acprefix=pages&aclimit=200&acprop=hidden
• Using generator=allcategories: https://commons.wikimedia.org/w/api.php?action=query&format=json&formatversion=2&generator=allcategories&gacprefix=pages&gaclimit=200&gacprop=hidden

[PaulinaQuintero]

Can I work on this? I've been analyzing the improvement, and I think I have a possible solution. It is my first contribution and I want to help.

[neslihanturan]

Hi @PaulinaQuintero , thank you. Please go for it:)

[PaulinaQuintero]

Thank you! I've been looking for where in the code I can implement the solution I already thought, however I didn't find it, could you give me the name of the class or the view where I could see this issue? @neslihanturan

[sivaraam]

As mentioned in the description, you would see the issue when uploading an image. The third step of the upload process would be to add categories. As of now, if you search there, you would see hidden categories such as:

• Category:Pages using authority control without Wikidata link - Wikimedia Commons
• Category:Uses of Wikidata Infobox - Wikimedia Commons

An ideal solution to this issue would result in these hidden categories not being shown in the search results. A possible way to achieve this is also mentioned in the description. Hope this clarifies your confusion.

[macgills]

categoryClient.searchCategoriesForPrefix(term.toLowerCase(), SEARCH_CATS_LIMIT)

This is in CategoriesModel which is invoked from CateogoriesPresenter which is used by the View UploadCategoriesFragment

[PaulinaQuintero]

I appreciate the information! As soon as I can I will be doing a pull request

[PaulinaQuintero]

I could already detect the problem here, what happens is that the API that is used has a bug. I have been testing and apparently the API that it is used to do this which has the generator parameter, is not returning the hidden data correctly. I tried another one that has list as a parameter and apparently this works well.
What could be done is change the whole part of the front to change to this list API, which works well but the drawback is that since they have a very different structure with the generator API it could affect other parts of the code where it is also being used, and the other alternative is ask to change the API generator to meet the specifications, which to me is the best and the easiest solution.

[PaulinaQuintero]

I don't know if you have been checked this, but what I mean is that it is impossible to fix the fact that the hidden categories are shown without fixing the allcategories generator api, part of the backend team. Because replacing the generator API with the list API involves a huge change in the frontend that could affect other parts of the code and I don't know if you agree to do this.

[sivaraam]

@PaulinaQuintero You're observation is right. The generator doesn't consider gacprop=hidden as mentioned in the description. I think the related code also confirms this.

Indeed, requesting the generator API to consider the gacprop=hidden would be the easiest way to solve this. As a first step, search for already open issues related to this on https://phabricator.wikimedia.org/ If you couldn't find any please open a new one with the MediaWiki-API tag.

That would take some time. In the mean time, let's see if someone here could weigh in on whether switching to the list version of the API is a good/bad thing. cc @maskaravivek @misaochan

[misaochan]

Personally I agree that hidden categories should not be shown. However I think the best way to solve this would be to wait for upstream to make the changes, rather than making a switch prematurely (wasted resources if they are going to do it anyway, and potential side effects).

[PaulinaQuintero]

I already did A PR with all the changes, the categories are already hidden. Could you check it? Sorry for all the commits. I'm pretty sure that is working well

[PaulinaQuintero]

The API is working well, it is no necessary @misaochan the problem is that the documentation is missing important information. I could solve it