Mapbox-gl-js: Allow styles to use multiple sprites and individual images

[mourner]

Make it possible to use multiple sprites (not just one) and reference individual images too.

Multiple sprites can be useful because nature of images can be very different so compression might be much more efficient when breaking down one sprite into several (e.g. one is BW icons, another is fill patterns etc.). They could use different formats too (e.g. color/detail-heavy icons could use JPEG instead of PNG etc.)

Individual images referencing is important to make it easy for people to play with, add custom icons on the map quickly, etc.

[kkaefer]

Currently, we load the image and use the image as is as a texture. Instead, we could use the stylesheet to find the parts of the sprite that are actually used and assemble a sprite texture using the binpack code that we already have for the glyph atlas (and reuse much of the glyph atlas code). That way, we could also enable referencing single individual images, or automatically generated images (e.g. for highway shields).

[ansis]

If we want to do this, should this be in v4?

[mourner]

Not critical I think, could be moved to v5.

[kkaefer]

Sprite Atlas is happening in https://github.com/mapbox/mapbox-gl-js/pull/851

[lbud]

bump :) this could be helpful from an API/editor perspective as well. What do you guys think?

[msbarry]

Hi, @mourner @kkaefer I'm trying to understand what it would take to get individual images for symbols working? It looks like the symbol rendering code is pretty tied in with in with having a single sprite atlas. Would individual images need to go through a completely separate code path?

I'm trying to hack a proof-of-concept together that requires individual images for symbols. Thanks!

[jfirebaugh]

This would require:

• Style spec addition to add URL support for "icon-image" and probably "{fill,line,background}-pattern" as well. To disambiguate from sprite icon names, we'd probably want to use a CSS-like url(...) syntax. This would be part of a new style spec version.
• Implementation in mapbox-gl-js.
• Implementation in mapbox-gl-native.
• Tests in mapbox-gl-test-suite.
• Backend work on the Mapbox style APIs to support the new spec version. Most likely we'd not allow styles uploaded to the API to use this feature, because we want to ensure that styles uploaded to the API can be rendered server-side without requiring loading resources from arbitrary third party domains.

[buchanae]

Have you considered allowing some form of client-defined programmatic evaluation of style spec properties, i.e. allow "icon-image" to be a function that returns an HTML Image.

If that was the case, maybe people would have implemented third-party libraries to handle lots marker/custom-image cases already.

Want to render an SVG elements to canvas, dump that to an Image, and pass that off to Mapbox? No problem! Just return it all via "icon-image": SVGDumper(mySvgEl) (or maybe via sprite?)

[buchanae]

Actually, evaluating a function isn't necessary I guess, since you can use map.setLayoutProperty() anyway. Maybe the core of the idea is considering more complex data types (such as HTML Image) for style spec values.

Or, maybe if you could set the sprite property to an object with some interface, say Spritesheet, people could provide their own implementations, which would allow all sorts of different loading/rendering mechanisms.

[jfirebaugh]

We don't want to introduce language-specific dynamic evaluation to the style specification. See this comment for more.

[jfirebaugh]

Another thing we'll need to think about is how to support high-DPI (retina/@2x and beyond) images.

[mourner]

Another thing we'll need to think about is how to support high-DPI (retina/@2x and beyond) images.

We should consider just using one @2x image for both retina and non-retina. The size difference is negligible and it shouldn't affect performance if we use 2x for 1x case.

[jfirebaugh]

Even if we support only a single pixel ratio per image, we still need a way to indicate what that pixel ratio is, image-by-image. On one hand, not everyone has @2x assets available, and on the other we should build something that can support @3x and up.

[yuvadm]

I'm curious about any workaround that are possible for this use case, until this issue is resolved. It seems that using popups as "enhanced" markers harms performance significantly.

[kkaefer]

We should consider just using one @2x image for both retina and non-retina. The size difference is negligible and it shouldn't affect performance if we use 2x for 1x case.

I don't think this is a good idea, as it degrades the image quality quite a bit. Icons in particular benefit a lot from pixel grid fitting and we've invested a lot to make sure that is the case.

[bhousel]

We should consider just using one @2x image for both retina and non-retina. The size difference is negligible and it shouldn't affect performance if we use 2x for 1x case.

Another point in favor of just using hidpi assets everywhere: we currently don't handle very well the situation of dragging a window from a retina to non-retina screen, and v.v. It seems kind of silly to refetch assets when this happens.

I don't think this is a good idea, as it degrades the image quality quite a bit. Icons in particular benefit a lot from pixel grid fitting and we've invested a lot to make sure that is the case.

Could we just generate a single base+1 mipmap level for each texture, then pass an appropriate level argument to texImage2D depending on whether the viewport is currently hidpi?

[kkaefer]

we currently don't handle very well the situation of dragging a window from a retina to non-retina screen

We used to do this, but dropped support for it, because it was such a rare occurrence.

Could we just generate a single base+1 mipmap level for each texture, then pass an appropriate level argument to texImage2D depending on whether the viewport is currently hidpi?

Yeah, we could do that, but that means we'd have to load both 1x and 2x assets as well, since scaling won't look good, no matter whether it's done by the CPU or GPU.

[lukerollans]

As an end user, I really want to see images from external URL's supported as marker icons in GL. It's core to our application, and without it we need to use mapbox js which just isn't as nice.

Is this planned?

[peterqliu]

@lukerollans ah now I see your use case. We don't have a dedicated solution for that yet, but if you're clever, you can adapt GL popups (which are DOM elements overlaid onto the map) into markers with some CSS tweaking.

[lukerollans]

@peterqliu thanks for the tip, will look in to it!

[gmaclennan]

+1 to the original request here (multiple sprites). I am building an app that overlays markers on a map. We want to allow the user to choose the background - one of the default mapbox styles or their custom vector tiles and/or style. Our markers (in the overlay) need to be sprites, but since there can only be one sprite associated with a map, we can't add those sprites to a map with a layer/style chosen by the user.

[spatialillusions]

It would really be great to create individual images in some way, in some cases the icons are based on so many properties that it is impossible to create an atlas of them. If it would be possible to point to an url, canvas, or base 64 encoding of the image would be nice. My use case is to use military symbology, where the possibilities of the symbols are several millions, and they have to be created according to standard documents. I have built a library for this, https://github.com/spatialillusions/milsymbol, and so far I have been abel to use it in several different mapping libraries, but not in mapbox-gl-js, and it would be very nice if I could.

[lukerollans]

It's honestly surprising to me that you can't just specify an external image as an icon if you wish.. but then again, I'm only just getting in to MapBox and am by far an expert

[knikel]

Hey folks, at @mapillary we're using spritezero-cli to bundle traffic signs from all our traffic sign detection packages and use these sprites in mapbox-gl-js and in our libs . We'll be approaching the limit of 500 signs per sprite soon. Is there an ETA for multiple sprite support? We're updating sprite refs and updating the map-style, but that makes it impossible to display traffic sign detections from multiple sprites on one map.

Using Popup API is not an option, due to performance reasons.

[jfirebaugh]

the limit of 500 signs per sprite

Which limit are you referring to here, specifically?

[knikel]

@jfirebaugh: At that time I was referring to editing style through _Mapbox studio_ limit and assumed it's not supported to have more than 500 images in the sprite. Then I found https://github.com/mapbox/mapbox-gl-js/commit/d6d8ac4cc3199ba82293b168ced403765325abd5, the 500 limit still holds in _Mapbox studio_ (at least it did last week), but we're tweaking style JSON now and use spritezero-cli for the sprites.

[mtoon]

Quick question around this.. We are looking at dynamically adding layers to the scene and _ideally_, each layer could have it's own discreet texture atlas. That way the shared "base" layer has it's texture atlas and the "feature" layers can bring their own atlas with them. Is there any effort underway for something like this?

[lucaswoj]

@mtoon There is no effort underway yet. We recognize this is an important feature request. This issue tracks the feature and will be updated when there's something to report.

[jfirebaugh]

Tracking an API that enables many of the use cases here -- Map#addImage -- in #2059.

[lucaswoj]

Closing in favor of mapbox/mapbox-gl-js#2059 & mapbox/mapbox-gl-style-spec#220