Streetcomplete: offline maps

[azrdev]

This is a feature request: As it seems, StreetComplete only uses online map tiles & a local cache. It would be nice if it could use downloaded offline maps, like used by e.g. osmand.

[westnordost]

This is not possible yet with the map rendering library I use. But they are working on it.

[ErinQuinn]

👋 @westnordost, just found your app - so cool. Mapbox has offline downloads. are you working with any of my colleagues?

[westnordost]

Yeah, that's pretty nice, nice job. However, the rendering library I use is tangram-es, they chase behind in terms of features.

[johnwhelan]

I preloaded the quests in the area I wanted to tag, closed the app foolishly. When I reached the area the quest where there but no highways etc so I couldn't add tags.

Later I entered an internet cafe for a coffee and started the app. On leaving I
still had maps. Could some sort of cache be added? Or a note in the instructions to open the app with an Interent connection and leave it open to retain the map?

Thanks John

[westnordost]

There should be a working cache. You can change the cache size in the settings. I have no further control over the cache other than (letting the user) specify the size of it (i.e. when what gets deleted from the cache).

[johnwhelan]

So its a documentation issue. If the cache is set large enough it is possible to work off line. Don't you love documentation things?

Thanks John

[gappleto97]

One thing to look at would be to use the maps present in OsmAnd~. I'm not sure how feasible that is, but it would allow people to save space once this is unblocked.

[tallytalwar]

Just FYI, tangram-es does have functionality to use mbtiles offline tile set. However one will need to create mbtiles from mapzen vector tile data on their own. We at present do not have any service to download mbtiles of a region.

Of course you can use mbtiles from another dataset, but will have to style the scene accordingly.

[westnordost]

Ah interesting, I did not realize that the implementation was finished already. If you permit, I'd ask you a few questions regarding this but I'll just email you, don't want to litter this issue (or a tangram issue) with general questions about mbtiles.

[mycae]

Hi,

The cache feature is OK, but not really useful for example in the use case of a long car journey, using street-complete to pass the time. I do use the cache, but the cache concept is pretty limited.

Osmand's offline files (OBF) files and a vector renderer are really really needed. A pre-rendered tile-based solution will be too large, and wont work in this case.

Its also not easy for an end user to specify the zones required for the cache, even for shorter journeys. Currently I have to zoom in to each area, and then wait for it to download, then zoom to another area. If I have more than one application that requires tiles, my device fills up.

I'm more than happy to point streetcomplete at the required obf files, and to duplicate those, as they are quite small (especially if this can be done from SD).

[gappleto97]

Seconding @mycae on this one. It would especially be useful if you could point StreetComplete to OsmAnd~'s offline maps, so that you don't need to duplicate them.

[HolgerJeromin]

See #243 for thoughts about osmand relationship.

[westnordost]

Can somebody point be to the Osmand offline file format (OBF?) Is this mapsforge format or what exactly?

[johnwhelan]

It's a compressed propitiatory format. Even one version of OSMand cannot handle the .obf file from a different version. Recently they have added height data which caused yet another revision of the .obf format.

They are created by map creator but again you need the right version to match the version of OSMand.

I wouldn't recommend using it.

Cheerio John

[gappleto97]

243 isn't at all what I'm looking for. My suggestion is merely that it should read the same map files, and allow you to specific a custom location. That would allow you to not have duplicates.

[johnwhelan]

osmand uses its own map data files in .obf format and it isn't particularly stable for one version to the next. It's a known issue for people wanting to create their own files. The version of map creator has to align with the version of osmand.

[matkoniecz]

Osmand has very llimited map download count, with fees for additional downloads. If one would decide to rely on data from other app then maps.me seems to be a better idea (though anyway using data from other app and trying to keep up with potential changes seems like a very bad idea).

[krzyk]

@matkoniecz There is an Osmand~ version (available from F-droid) that doesn't have that limitation, only google play version has it.

[Hillside502]

limited map download count, with fees for additional downloads

Are we saying that OsmAnd limits the number of maps installed on a device?

[johnwhelan]

There are two versions a free one and a paid one. The free one limits you to 12 downloads and one of these really needs to the the world overview. If you are updating the local area you really need to update the map from time to time, OSMand on a good day updates once a month. So not an ideal solution.

[azrdev]

the limit applies do downloads, not number of stored obf files, though. And it's only imposed on the app, not downloading manually from http://download.osmand.net/list.php

Also, IIRC the free beta version of osmand has no download limit

[Hillside502]

OsmAnd+ and OSM Live Subscription --- the world's your oyster (updated HOURLY)

Maps & GPS Navigation OsmAnd+ – Android Apps on Google Play
https://play.google.com/store/apps/details?id=net.osmand.plus

[krzyk]

and the version without limits is here: https://f-droid.org/repository/browse/?fdid=net.osmand.plus as advertised on osmand github page

[Hillside502]

@krzyk
There are no download limits in the Google Play Store OsmAnd+ (Plus)

[HolgerJeromin]

There are various reasons why osmand or its data files are no solution for streetcomplete.
The format is not stable and osmand does not provide a rendering library.

So in my opinion we do not have to discuss this app (which I like a lot and bought plus version and live subscription) in this issue further.

[jancborchardt]

Does Maps.me use a format for the offline files which could be reused by StreetComplete? cc @zverik @rokuz @darina from @mapsme. :)

[rokuz]

Unfortunately it does not.

[leschek]

Not sure if it is not off topic, but I use for locus vector maps from this site: http://www.openandromaps.org/en/downloads/countrys-and-regions
It would be great to have offline maps and be able to download database of quests for selected area for later use.

[johnwhelan]

Currently I'm able to download both the quests and the tiles for an area before going there and working disconnected. My device only uses wifi. Sometimes the quests are present and the tiles disappear but I suspect that is a function of how much memory is available.

[mycae]

I've not looked at the code, but If I understand correctly - the data are not cached to disk, but only in ram . Thus if the process is terminated, then the quests & map data are lost. (citation: https://wiki.openstreetmap.org/wiki/StreetComplete )

If this is correct (is it?), an interim solution would be to serialise the in-memory data to disk, and readback from memory start. A "save as" feature might go a long way to mitigating the impact of the lack of offline functionality. Alternatively, perhaps alter the name from "cache", and instead call it a "RAM cache" or "memory cache" or something, to make it clear that the cache is ephemeral?

[johnwhelan]

That would be a major improvement.

[leschek]

johnwhelan wrote:

Currently I'm able to download both the quests and the tiles for an area before going there and working disconnected. My device only uses wifi. Sometimes the quests are present and the tiles disappear but I suspect that is a function of how much memory is available.

Well it is working for smaller areas. For example I'll go to mountains in the north India and it would be great If I can use some kind of offline map (e.g. vector map) and then select area (could be couple of villages) and download quests for that selected area.

Now I'm able to download quests for village, but not much around it (StreetComplete asks me to zoom in) and then there is issue with blank tiles when I zoom in when I'm offline.

[westnordost]

No, it's cached to disk

On 29 June 2017 16:56:29 CEST, mycae [e-mail] wrote:

I've not looked at the code, but If I understand correctly - the data
are not cached to disk, but only in ram . Thus if the process is
terminated, then the quests & map data are lost. (citation:
https://wiki.openstreetmap.org/wiki/StreetComplete )

If this is correct (is it?), an interim solution would be to serialise
the in-memory data to disk, and readback from memory start. A "save as"
feature might go a long way to mitigating the impact of the lack of
offline functionality. Alternatively, perhaps alter the name from
"cache", and instead call it a "RAM cache" or "memory cache" or
something, to make it clear that the cache is ephemeral?

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/westnordost/StreetComplete/issues/122#issuecomment-311992704

--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[Hillside502]

@mycae
Your link above has the text:-
https://wiki.openstreetmap.org/wiki/StreetComplete
but points to:-
https://github.com/westnordost/StreetComplete/issues/url
(a 404 error)

StreetComplete - OpenStreetMap Wiki
https://wiki.openstreetmap.org/wiki/StreetComplete
is how it should be.

[matkoniecz]

Thus if the process is terminated, then the quests & map data are lost. (citation: https://wiki.openstreetmap.org/wiki/StreetComplete )

I fixed wiki.

[mycae]

No, it's cached to disk

Edit: Scratch comment. Unable to fully reproduce issue on demand.

[westnordost]

Weird. I did not look into it myself, I a basically just using a feature of tangram-es. As you have a better insight right now, perhaps you could open an issue at tangram-es.

On 30 June 2017 11:38:10 CEST, mycae [e-mail] wrote:

No, it's cached to disk
In this case, the tile_cache is not working for me. In the application,
if I view an area, close StreetComplete, and re-open it with wifi
disabled, the quests are shown, but the map is not.

Looking in the tile_cache folder, the files are unreasonably small, and
few in number (maybe 20). the largest file is 8k ancd the most common
is around 800 bytes. Each file is named with what appears to be a hash.
The contents of the larger files are in a binary form with a header I
do not recognise. The smaller files however, appear to be a cached http
GET request to tile.mapzen.com - without any payload response.

--
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub:
https://github.com/westnordost/StreetComplete/issues/122#issuecomment-312223247

--
Diese Nachricht wurde von meinem Android-Mobiltelefon mit K-9 Mail gesendet.

[willhelm-mueller]

Hmm cgeo uses mapsforge maps so i already have osnand and mapforge maps on my device ....

[hoschi-it]

Again about maps.me:
You can't access the offline map as a file, but maps.me got a API that can show the map including some points and return to the initial app when a point becomes selected.

[conrad-heimbold]

I can really recommend "Tileless Maps" ( https://github.com/TilelessMap/TilelessMap ). It uses vector instead of raster images. The databases are based on postgis and have the twkb format.
Tileless maps can also be used offline.

[rugk]

Okay, but this would mean switching to a completely different rendering library, as far as I see. So this is not really realistic.
Also vectors can sometimes be bad for performance reasons (OsmAnd uses vectors AFAIK), but for the simple mapstyle of SC, it would likely not be so bad.

[rugk]

New issue for Tileless Maps: https://github.com/westnordost/StreetComplete/issues/971

[GITMichaelK]

Is there any chance to use StreetComplete offline, without Internet-connection abroad. I have osmand installed on my phone and it works fine with all local osm-data. Can I use this ore something similar (e.g. downloading an area first, uploading information later when back home), to work with Street-Complete?

[westnordost]

The quests can be downloaded beforehand and solving them works offline. The map however is online only. What you can do is to (increase the map cache in the settings and) pan the area in different zoom levels where you want to go before you go there. With a little bit of luck, most map tiles will still be in the cache by the time you get there.

Am 18. November 2018 12:53:26 MEZ schrieb GITMichaelK [e-mail]:

Is there any chance to use StreetComplete offline, without
Internet-connection abroad. I have osmand installed on my phone and it
works fine with all local osm-data. Can I use this ore something
similar (e.g. downloading an area first, uploading information later
when back home), to work with Street-Complete?

[westnordost]

This ticket somewhat went down in priority for me after I experienced that mobile Internet is quite available in less developed countries, often faster and cheaper than in Germany. Speaking of South Asia in particular. May I ask where you want to go?

Am 18. November 2018 12:53:26 MEZ schrieb GITMichaelK [e-mail]:

Is there any chance to use StreetComplete offline, without
Internet-connection abroad. I have osmand installed on my phone and it
works fine with all local osm-data. Can I use this ore something
similar (e.g. downloading an area first, uploading information later
when back home), to work with Street-Complete?

[GITMichaelK]

Tobias: Thank you for the program, your effort and response.

I've read something about

downloaded beforehand and solving

without seeing the map; this is a mess for me and not really practical. Same with the method to increase the cache: I could not find an acceptable solution to work with.

May I ask where you want to go?

Sure, rural areas (villages, litlle towns) in Bavaria. Please don't believe, that here is Internet service everywhere. I experienced last year f.e. Russia, Chile, Greece and Spain, almost no issue there. Here is different ...

I'd appreciate, if you could increase the priority for this feature again!

[johnwhelan]

A comment streetcomplete is quite happy off line its the image tiles that
are the problem. Even these if you zoom in they will remain in the cache
so for small areas you can work offline.

What would be nice is the ability to overlay something like osmand behind
streetcomplete.

Cheerio John

On Sun, 18 Nov 2018, 6:53 am GITMichaelK <[email protected] wrote:

Is there any chance to use StreetComplete offline, without
Internet-connection abroad. I have osmand installed on my phone and it
works fine with all local osm-data. Can I use this ore something similar
(e.g. downloading an area first, uploading information later when back
home), to work with Street-Complete?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/westnordost/StreetComplete/issues/122#issuecomment-439687104,
or mute the thread
https://github.com/notifications/unsubscribe-auth/ACUNawxUUhvR9tnac--vaSzEJ4WVRbm-ks5uwUo5gaJpZM4M-GVr
.

[GITMichaelK]

I fully agree

What would be nice is the ability to overlay something like osmand behind streetcomplete.

OSMand is already available (on my phone) and those maps are updated regularly. To use these as tiles for the StreetComplete-tasks would be the preferred way to solve this (at least in my eyes ;) ).

[georg-d]

OSMand is already available (on my phone) and those maps are updated regularly. To use these as tiles for the StreetComplete-tasks would be the preferred way to solve this (at least in my eyes ;) ).

OSMand maps were pointed out above to be no good choice due to frequent format changes, moreover downloads are limited in 1 of the 2 versions.

What about Mapsforge? It's a stable format, so maintenance effort is limited, map files are offered by many distributors like openandromaps, so StreetComplete is not depending on 1 map supplier, and it is used by many apps like Locus and OruxMaps, so map files exist on many phones.

Maybe it's easier to turn the idea upside down, i.e. not making StreetComplete using map files of other apps (weren't filesystem access restrictions changed in Android versions again and again?), but making other apps using StreetComplete? Maybe it's easy to offer different "flavors" of StreetComplete that work as addon within different map apps like Locus or OSMand. These "flavors" do not need to take care about map rendering, GPS position aquisition etc. but just the core feature set, i.e. load & display quests and send the answers. For users, it becomes more convenient, as they do have only one app open for casual mapping instead of steadily changing between 2 apps.

[GITMichaelK]

Even if I don't see any issue with OSMand, I'm really interested to have an offline-usage for StreetComplete. May be, you can give a try to Vespucci by SimonPoole, ref. "Going completely offline with Vespucci".

[GITMichaelK]

Seen the version 11, still without offline-mode. Is there any chance to get it or should I forget abou it?
Thanks and regards, Michael

[matkoniecz]

Is there any chance to get it or should I forget abou it?

This issue is open, it means that it is waiting for someone to implement it. In general, in open source projects you can either implement it by yourself or wait for someone else to do this. There are ways to increase chance for someone working on it, but comments with complaints usually have an opposite effect.

For people unable to significantly help GitHub has nice feature that allows to express interest in the feature - reactions allowing to add +1 without sending spurious notifications.

[matkoniecz]

May I ask where you want to go?

Poland

In Krakow forest within city has big spots without mobile data availability (and at times without even ability to make phone calls). Forest nearby city is completely without mobile data, the same for some rural areas.

For quests this is not a problemb at all, current caching is sufficient. But often one needs some context provided by map to answer question and lack of preloading map tiles makes it impossible.

For example one of my UX testing sessions was initially going well, then most of it was cancelled as we reached forest without mobile data (Ojców near Kraków).

[matkoniecz]

It seems that mbtiles are a good idea - supported in mapzen and fairly standard so in case of switching rendering library not all work will be lost.

But I wonder about how people are going to get mbtiles (or other data). Reusing Osmand/mapsme data is not feasible (would require constant chasing format changes). I am dubious about possibility of providing our own data. And relying on third party providers (some exist) will cause problems once they disappear.

Obviously, it is possible to provide scripts to generate mbtiles (or other format), but how many people will be able to do this?

[westnordost]

MBTiles is just a standard for the format in which a DB table must be in order to be readable (for a library like tangram-es).

So, not people need to get MBTiles, but the app will download an area tile-by-tile into a database table in the right format, downloading from the same source as usual.

The user will have an option in the app to download a certain area for offline use, which downloads the quests (already the case) and the background map. Then, they also need an option to manage their downloaded areas so that they can delete them if they don't need or anymore.
In total, I consider the UX design and UI work the larger part of this feature.

On August 23, 2019 4:38:51 PM GMT+02:00, Mateusz Konieczny [e-mail] wrote:

It seems that mbtiles are a good idea - supported in mapzen and fairly
standard so in case of switching rendering library not all work will be
lost.

But I wonder about how people are going to get mbtiles (or other data).
Reusing Osmand/mapsme data is not feasible (would require constant
chasing format changes). I am dubious about possibility of providing
our own data. And relying on third party providers (some exist) will
cause problems once they disappear.

Obviously, it is possible to provide scripts to generate mbtiles (or
other format), but how many people will be able to do this?