Apps-android-commons: Make Nearby SPARQL query faster

Created on 27 Jan 2019  路  5Comments  路  Source: commons-app/apps-android-commons

Let's ask the SPARQL optimization gurus 馃檪
We could also first make a simple fast query, then a second one to get second-importance info (labels, is the item ended, item class, grave items, etc)

Wikishootme's query is much faster than ours, but it grabs less data. We could start with a qury like theirs, then get details.

Most helpful comment

I tried removing the emoji attribute from the query and the query became considerably faster ie from ~5200ms to ~2100ms.

New query: http://tinyurl.com/yaafoa5n

Compare it with the original query: http://tinyurl.com/yakxk726

AFAIK, we don't use emoji value.

All 5 comments

For ref this is our current query: http://tinyurl.com/yakxk726

I tried removing the emoji attribute from the query and the query became considerably faster ie from ~5200ms to ~2100ms.

New query: http://tinyurl.com/yaafoa5n

Compare it with the original query: http://tinyurl.com/yakxk726

AFAIK, we don't use emoji value.

Great finding! This will save me a lot of waiting time every day :-)

I have raised a PR removing icon and emoji. Both attributes were not being used by the app. The loading is much faster now. :)

I have a couple of more suggestions:

  • Do not wait for all results before displaying it on the map. As of now the query is fired multiple times until 40 items are fetched. We can make use of Observable from RxJava to emit results as they arrive.
  • SPARQL must have cache control. If you run the same query on the web interface, the first time it takes 2-3 seconds and next time it takes 4-5 ms. That indicates that there should be some caching involved. We can make use of it by setting the cache control headers correctly. Can we check with guys at Wikidata about how to utilize caching? It would be as simple as sending a header in the API request.

(Was asked by @VojtechDostal to make here a suggestion.)
https://www.mediawiki.org/wiki/Wikidata_Query_Service/User_Manual#SPARQL_endpoint states:

It is recommended to use GET for smaller queries and POST for larger queries, as POST queries are not cached.

The simplest way to bypass caching is to add some dummy string to the query, like a comment # and a random number.

Also those SERVICE wikibase:label calls in the query are redundant. They are known to make queries slow even if you make no use of them .

Was this page helpful?
0 / 5 - 0 ratings