Mapbox-gl-js: Option to keep certain styles and layers on setStyle()

[indus]

Motivation

Injecting custom layers into Mapbox core styles is a common use case and confusing to users, especially when a user wants to swap out the core style and keep the custom layers.

Design Alternatives

3979 current approach

4000 alternative approach (draw-back: need for predefined nested styles)

Design

I would like to suggest an option that would allow the user to copy some layers/sources over to a new style to keep them after setting a new style.

Mock-Up

//layers appended to the end of the new layer array (in order)
setStyle('mapbox://styles/mapbox/' + layerId + '-v9', {
  copySources: ['states'],
  copyLayers: ['states']
})

// layer added before another layer in the new layer array (like addLayer(layer, before))
setStyle('mapbox://styles/mapbox/' + layerId + '-v9', {
  copySources: ['states'],
  copyLayers: [{'states':'waterway-label' }]
})

OR

//layers appended to the end of the new layer array (in order)
setStyle('mapbox://styles/mapbox/' + layerId + '-v9', {
  copy: {
         sources: ['states'],
         layers: ['states']
   }
})

Concepts

It would work like a normal setStyle but after the new style is loaded the copying from the old to the new style would take place before the map gets updated.

Implementation

the copying-code likely could get called at the end of the style-diffing procedure and could look something like that lodash-pseudocode:

var style = _.union( newStyle, _.pick( oldStyle, copyoption ))

[anandthakker]

Thanks for this proposal @indus!

Separate from the copyLayers part and the alternative nested styles proposal in #4000, I think that copySources (or something like it) might fill a current shortcoming of the setStyle API that also wouldn't be addressed by nested styles.

GeoJSONSource in particular may carry a large amount of state if its data is set to a literal GeoJSON value rather than a URL. As @averas describes here, copying this over from the existing style into a new target style when using map.setStyle() may be inconvenient and potentially a performance problem.

While it's true that using a reactive paradigm and immutable objects could mostly the ergonomic and performance issues (respectively), we can't assume that this is necessarily a valid option for all/most users. Thus, I think it makes sense to provide a means for developers to short-circuit the diff for particular sources.

[lucaswoj]

Another design that may address the same problem without requiring the reintroduction of imperative concepts is something akin to React's key attribute -- some developer-provided metadata that hints to GL JS when a source can be copied.

[anandthakker]

As I understand it, the copySources option basically _is_ developer-provided metadata, except perhaps for the word "copy" in its name. It is an option that dictates the interpretation of the main stylesheet parameter being given to setStyle(). @lucaswoj can you elaborate on "reintroduction of imperative concepts"?

[lucaswoj]

A goal of the "smart" setStyle project was to make it possible to use GL JS in a reactive programming paradigm in which each Map#setStyle call need make no assumptions about the current state of the map. copying a source breaks that property, at least conceptually.

Renaming the property to preserveSources would help a little 😄

Having GL JS able to decide whether to preserve or replace a source without the user explicitly having to make this performance decision is even better.

[anandthakker]

Having GL JS able to decide whether to preserve or replace a source without the user explicitly having to make this performance decision is even better.

If we want

{
  sources: {
     myVectorSouce: { ... },
     // there used to be a geojson source here, but now it's gone
  }
}

to sometimes mean "make the style like this -- i.e. if there are any sources that aren't myVectorSource, remove them", and other times to mean "make sure myVectorSource is there, but preserve the existing geojson source", then there will have to be _some_ way for the user to provide explicit information that goes beyond declaring the target style.

Adding something like React's key attribute to the target style is one such means for providing that information--e.g., we could allow an extra hash property on any source, and if the target source's hash matches the existing source with that id, then we skip diffing. It could be argued that this is semantically cleaner than preserveSources, and in the abstract, I'd agree. However, I think that it's _practically_ a bit messy, since it would require GL JS to track a new piece of state (the hash property) on each source.

preserveSources would be less invasive, but (a) is less conceptually clean, and (b) would re-introduce awkwardness for developers using reactive approach.

🤔

[kaitok]

Whats the status now this issue?

[songololo]

Any possible updates on this item? It would be very helpful to have an option to save layer states on style updates within reactive environments.

[andrewharvey]

in case it helps anyone the workaround I'm using at the moment is below. It constructs a new Style object retaining the sources and layers you want copied across to the new style.

import { json } from 'd3-request';
function swapStyle(styleID) {
  var currentStyle = map.getStyle();
  json(`https://api.mapbox.com/styles/v1/mapbox/${styleID}?access_token=${mapboxgl.accessToken}`, (newStyle) => {
    newStyle.sources = Object.assign({}, currentStyle.sources, newStyle.sources); // ensure any sources from the current style are copied across to the new style
    var labelIndex = newStyle.layers.findIndex((el) => { // find the index of where to insert our layers to retain in the new style
      return el.id == 'waterway-label';
    });
    var appLayers = currentStyle.layers.filter((el) => { // app layers are the layers to retain, and these are any layers which have a different source set
      return (el.source && el.source != "mapbox://mapbox.satellite" && el.source != "composite");
    });
    appLayers.reverse(); // reverse to retain the correct layer order
    appLayers.forEach((layer) => {
      newStyle.layers.splice(labelIndex, 0, layer); // inset these layers to retain into the new style
    });
    map.setStyle(newStyle); // now setStyle
  });
}

[indus]

I'm using a factory function to recreate a complete stylesheet everytime the configuration of layers is supposed to change; something like this:

let getStyle = function (map_class) {
    let layers = [];

//...

    if (map_class.basemap)
        layers = [...layers_base]; // from a official stylesheet

    if (map_class.elevation && map_class.hillshade)
        layers.push({
            "id": "elevation_hillshade",
            "source": "elevation_hillshade",
            "type": "raster"
        })
    else {
        map_class.elevation && layers.push({
            "id": "elevation",
            "source": "elevation",
            "type": "raster"
        })
        map_class.hillshade && layers.push({
            "id": "hillshade",
            "source": "hillshade",
            "type": "raster"
        })
    }

// ... 30-50 more if-statements

    let style = {
        version: 8,
        glyphs: "http://localhost:8084/fonts/{fontstack}/{range}.pbf",
        sources: sources, // from a global var
        layers: layers
    };
    return style;
}

this.on("state_changed", function (state) {
    map.setStyle(getStyle(state.map_class))
})

When I compare it to the class based system we had I still think its ugly because the stylesheet is no longer a plain JSON object but a function (and you wouldn't be able to transfer it easily to a native library). It also may be less performant (?), but at least it works for me after the deprecation and until there is some progress towards an incremental/partial setStyle

[anandthakker]

Alternative proposal at #6701

[ggerber]

import { json } from 'd3-request';
function swapStyle(styleID) {
  var currentStyle = map.getStyle();
  json(`https://api.mapbox.com/styles/v1/mapbox/${styleID}?access_token=${mapboxgl.accessToken}`, (newStyle) => {
    newStyle.sources = Object.assign({}, currentStyle.sources, newStyle.sources); // ensure any sources from the current style are copied across to the new style
    var labelIndex = newStyle.layers.findIndex((el) => { // find the index of where to insert our layers to retain in the new style
      return el.id == 'waterway-label';
    });
    var appLayers = currentStyle.layers.filter((el) => { // app layers are the layers to retain, and these are any layers which have a different source set
      return (el.source && el.source != "mapbox://mapbox.satellite" && el.source != "composite");
    });
    appLayers.reverse(); // reverse to retain the correct layer order
    appLayers.forEach((layer) => {
      newStyle.layers.splice(labelIndex, 0, layer); // inset these layers to retain into the new style
    });
    map.setStyle(newStyle); // now setStyle
  });
}

the style object for Mapbox Streets has a 'sprite' property with value "mapbox://sprites/mapbox/streets-v11"
if one changes the baselayer to Mapbox Satellite by doing a http request, the returned style object has a sprite property with value "mapbox://sprites/mapbox/satellite-v9"

My understanding is that these two different sprite values will force setStyle() to rebuild the style from scratch, since it cannot perform a diff as the underlying setSprite() is not implemented.

Should one simply accept that when switching from streets to satellite a complete rebuild of the style is required when using setStyle() or is there a way to reconcile the two different sprite values?

[andrewharvey]

Should one simply accept that when switching from streets to satellite a complete rebuild of the style is required when using setStyle() or is there a way to reconcile the two different sprite values?

@ggerber One approach (out of a few) that I use is in Mapbox Studio I'd download a Streets style, and download a Satellite style, then manually concatenate the layers list and combine the sources from the style JSON, then upload that as a Style to Mapbox Studio and then upload both sets of icons to that style, so you end up with a style containing both streets and satellite but in one style with one spritesheet, then just toggle layers on/off when switching "styles".

[xero88]

Unfortunatly this workaround do not work for me.

Is there an other way ?

[ericbrumfield]

I was able to get a work around from stack overflow working here

In addition, if you've loaded any custom images such as marker/symbol images you'll have to add them back as well. It took me a few, but I figured out that you'd have to do this in the 'style.load' event, making sure you try/catch and gulp the exception for map.removeImage followed by map.addImage. I don't prefer try/catch gulping like that, but there wasn't a method like getImage that I could find to check if it exists or not in the map already.

[stevage]

@andrewharvey
in case it helps anyone the workaround I'm using at the moment is below. It constructs a new Style object retaining the sources and layers you want copied across to the new style.

I had to tweak this slightly to get it to work - looks like the name of the Satellite source has changed to mapbox.

I also tweaked a few other things for readability.

// styleID should be in the form "mapbox/satellite-v9"
async function switchBasemap(map, styleID) {
    const currentStyle = map.getStyle();
    const { data: newStyle } = await axios.get(
        `https://api.mapbox.com/styles/v1/${styleID}?access_token=${mapboxgl.accessToken}`
    );

    // ensure any sources from the current style are copied across to the new style
    newStyle.sources = Object.assign(
        {},
        currentStyle.sources,
        newStyle.sources
    );

    // find the index of where to insert our layers to retain in the new style
    let labelIndex = newStyle.layers.findIndex((el) => {
        return el.id == 'waterway-label';
    });

    // default to on top
    if (labelIndex === -1) {
        labelIndex = newStyle.layers.length;
    }
    const appLayers = currentStyle.layers.filter((el) => {
        // app layers are the layers to retain, and these are any layers which have a different source set
        return (
            el.source &&
            el.source != 'mapbox://mapbox.satellite' &&
            el.source != 'mapbox' &&
            el.source != 'composite'
        );
    });
    newStyle.layers = [
        ...newStyle.layers.slice(0, labelIndex),
        ...appLayers,
        ...newStyle.layers.slice(labelIndex, -1),
    ];
    map.setStyle(newStyle);
}

[sirmspencer]

@stevage Do you know if your method maintains cached layers? I have tried the method were I save the source/layers from the old source and then add them to the map after the style is updated. Im wondering if by keeping the existing soruce/layers in the new style if mapbox is able to keep the layer cache.

[stevage]

It seems to - the preserved layers stay visible on the map, they don't even flicker or disappear.

[sirmspencer]

@andrewharvey
(instead of posting a large quote reply, see two posts up)

Thanks for this code sample. It inspired me to get a closer to what I need where the layers keep their cache when the base map is changed. The piece you are missing is that if the sprite value is modified, mapbox will do a hard refresh on all the layers. In all my layers I dont have custom sprites, I just grab one to set as the default. Then each time I make the fetch call, I set newStyle.sprite = to the default sprite url. Im in clojure and have some helper functions but heres the gist:

(go
    (let [cur-style (js->clj (.getStyle @the-map))
          new-style (js->clj (<! (u/fetch-and-process source {} (fn [res] (.json res)))))
          sources   (->> (get cur-style "sources")
                         (u/filterm (fn [[k _]] (str/starts-with? (name k) "fire"))))
          layers    (->> (get cur-style "layers")
                         (filter (fn [l] (str/starts-with? (get l "id") "fire"))))]
      (-> @the-map (.setStyle (clj->js (-> new-style
                                           (assoc "sprite" "mapbox://sprites/user/id1/id2")
                                           (update "sources" merge sources)
                                           (update "layers" concat layers)))))))

[stevage]

Ah, in my particular case I have hacked the two style sheets to use the same spritesheet, using mbsprite