Mapbox-gl-native: MBTiles source

[JesseCrocker]

Im trying to use an MBTiles file as a source for tiles in an iOS application. It looks like there is currently no support for this.

Is this a planned feature?
If i were to implement it would the PR be considered for merging?

[robmaceachern]

Hi Mapbox team @incanus @boundsj @jfirebaugh. Can somebody clarify the current state & future plans for MBTiles in the mobile SDKs?

Earlier comments indicated that MBTile support was likely:
https://github.com/mapbox/mapbox-gl-native/issues/584#issuecomment-81931466
https://github.com/mapbox/mapbox-gl-native/issues/3053#issuecomment-157403202

There seemed to be progress here https://github.com/mapbox/mapbox-gl-native/pull/2939, but it was closed in favour of https://github.com/mapbox/mapbox-gl-native/pull/3715 which doesn't seem to include MBTile support.

The marketing and help page copy currently implies that MBTiles are supported:
https://www.mapbox.com/help/an-open-platform/#storing-tiles

You can also use MBTiles files offline on mobile devices with Mapbox iOS toolkits.

https://www.mapbox.com/help/define-mbtiles/

It’s designed so that you can package the potentially thousands of files that make up a tileset and move them around, eventually uploading to Mapbox or using in a web or mobile application.

My questions are:

1. Is there any officially supported way to use MBTiles with the current versions of the mobile SDKs?
2. If MBTiles can't be used with the current SDKs, what options are available for including map tiles in an application bundle so that no additional downloads are required after app install? Links to sample code would be helpful.
3. Is support for MBTiles something that you'd consider adding, or accepting as a PR?

More background on my client's use case:
The apps currently use MBTiles with a legacy version (Mapbox iOS 1.6) of the Mapbox SDK. The map information they need in their apps is for specific geographic areas and they want to continue including the map tiles in the application bundle so that the app is ready for offline use as soon as it's downloaded by the user.

[incanus]

First, to address the OP:

If i were to implement it would the PR be considered for merging?

We have discussed internally recently, and things along the lines of your (@JesseCrocker) work in https://github.com/mapbox/mapbox-gl-native/pull/6940 are more generically useful given that you can provide a tile as needed from whatever format you like. We are hesitant to broaden the source API options given that we still have yet to port image (https://github.com/mapbox/mapbox-gl-native/issues/1350) and video (https://github.com/mapbox/mapbox-gl-native/issues/601) sources from our own spec across from GL JS.

@robmaceachern To answer your questions, OTOH here is how I would clarify:

1. MBTiles is still a fully-supported format for our API, given that Tippecanoe exports to it and our upload API accepts it. It's still the best way to transport lots of raster or vector tiles.

2. https://github.com/mapbox/mapbox-gl-native/pull/2939 was a bit of a precursor to a broader API such as the above-mentioned https://github.com/mapbox/mapbox-gl-native/pull/6940.

3. In implementing offline support in https://github.com/mapbox/mapbox-gl-native/pull/3715, we decided to not promise to support the MBTiles format for the internal offline storage format, despite both being in SQLite. The reasoning was the additional metadata such as HTTP headers, cache checking and conditional fetching, and other offline-related functionality that didn't make sense for a generic file storage format. While we _may_ merge the two in the future, we decided to keep the offline format API-private to give us the flexibility to change internal implementation in the future. We haven't yet prioritized such a change.

4. https://www.mapbox.com/help/an-open-platform/#storing-tiles is somewhat out of date given mention of the raster mobile tools (in intent) as well as TileMill. We'll revise this. /cc @mapbox/support

So to answer your questions @robmaceachern, I'm afraid there are no good approaches right now. Helping shepherd https://github.com/mapbox/mapbox-gl-native/pull/6940 would be the best way to get flexible local access to tiles into the SDKs (especially Android), at which point you could just serve up SQLite-based tiles easily with some minor linkage code.

I hope this clarifies. This just hasn't been on our radar for a while given many other priorities around internal/GL JS parity, performance and stability, and new work like 3D extrusions.

[namannik]

In case anybody is interested, I've managed to display MBTiles in an MGLMapView by hosting a web server within my app and pointing the map to localhost. I've posted my solution here:

https://gist.github.com/namannik/3b7c8b69c2d0768d0c2b48d2ed5ff71c

[dije]

hai @namannik, can u help me? i use your class, but in let tileImage = UIImage(data: tileData) always return nil ( i use let path = Bundle.main.path(forResource: "countries", ofType: "mbtiles") to get path), thanks before

[namannik]

Hi, @dije. I've made some changes so it won't matter if UIImage(data: tileData) returns nil. However, it probably still won't work for you, as I suspect your data is vector pbf data. (Your MBTiles file is from here, correct?) The MGLMapView+MBTiles.swift gist currently only works with raster (jpg, png) data.

Any other discussion about MGLMapView+MBTiles.swift should take place over on the gist's page to not confuse this issue's discussion.

[dije]

Thanks for the answer @namannik, ill go check latest gist, and try again

[typebrook]

@dije
As @namannik said, if your mbtiles are download from OpenMapTiles, then the data stored in sql is .mvt.

In this case, replace MGLRaterSource with MGLVectorSource should works, if you have other question or feedback, let's discuss in the gist

[1ec5]

Building on @JesseCrocker’s work in #6940, MGLComputedShapeSource will ship with Android map SDK v6.0.0, iOS map SDK v4.0.0, and macOS map SDK v0.7.0. That should in theory make it possible to implement an MBTiles-backed vector source. It’s more likely that this source would be implemented as a plugin than going straight into the SDK proper, so I opened mapbox/mapbox-plugins-ios#19. An MBTiles-backed raster source, meanwhile, is blocked by mapbox/mapbox-gl-native#7471.

[JesseCrocker]

I don't agree that MGLComputedShapeSource eliminates the need for an MBTiles source that works with vector tiles, because unless something has changed recently there are no publicly exposed classes/methods for parsing vector tiles, so the app developer would still have to implement that, and MGLComputedShapeSource has no way to provide data in multiple layers, which is common in vector tiles.

But if support for MBTiles is not going to make it into the core SDK it seems like a great feature for a plugin, because it can be implemented in a self contained fashion by a class that implements MGLOfflineStorageDelegate and transforms mbtiles:// urls to http://localhost urls, and runs a local http server that serves the tiles.

[1ec5]

Thanks, those are good points. I didn’t mean to rule out eventually adding MBTiles support to the SDK in some form – any plugin can potentially be folded into the SDK at a later date. But I figure mapbox/mapbox-plugins-ios#19 is a first step towards making that possible.

[asheemmamoowala]

Even if there was a decoder for vector tiles, #10494 would still be needed before you got blocked by the multiple layers issue.

[typebrook]

@asheemmamoowala
Thanks for your remind! But I can't find CustomGeometrySource and GeometryTileProvider in the latest release (v5.3.1) in android sdk, unless using 6.0.0-SNAPSHOT.

Referenced by Docs, I thought they were already added in sdk after v5.2.1. Is there anything I missed?

[asheemmamoowala]

@typebrook Sorry for the confusion. These classes are not included in the released SDK as yet. The docs should not have those classes visible yet.

[JanC]

Hi,
so is there any plan to include local mbtile support to the SDK or do we need to use @namannik 's custom implementation ?

[typebrook]

@namannik @JanC
I am done with similar Implementation for vector tiles by Kotlin in Android, solution is here

@asheemmamoowala
Also, I found that even running localhost for Vector/Raster source, we still need network connection when doing request. I thought we don't skip request when host is localhost, related code is here:
https://github.com/mapbox/mapbox-gl-native/blob/982fcc9d0a595f8b9c6dcc7b8b9b1f5979abceaa/platform/android/MapboxGLAndroidSDK/src/main/java/com/mapbox/mapboxsdk/http/HTTPRequest.java#L78-L79
Am I thinking wrong?

[JesseCrocker]

closing issue because im guessing this is never going to happen.

[tsemerad]

@typebrook Did you ever come up with a workaround to getting your app able to request MBTiles hosted on localhost while offline?

[halset]

Using a custom NSURLProtocol that handles a custom URL scheme like mbtiles:// and add it to MGLNetworkConfiguration.sharedManager.sessionConfiguration.protocolClasses seem to be a bit simpler than embedding a web server.

[typebrook]

@tsemerad

Did you ever come up with a workaround to getting your app able to request MBTiles hosted on localhost while offline?

Yes. With the help from other user, gist is already updated