Gutenberg: Blocks: Versioning and migrations

[aduth]

The behavior of block validation works by comparing the output of what the editor would save as a block against what is actually saved in post content. Therefore, if the generated output of a block ever changes, all previously saved blocks would then be considered invalid.

The options here are:

• Blocks must never change their generated output
• Accept that validation will flag the blocks as invalid. The behavior of "Overwrite" may or may not allow the block to remain intact, depending on the changes and implementation of block attributes (i.e. those sourced from content may be more subject to breakage).
• Implement block versioning, so that previous versions can remain unaffected by the new markup. Presumably each of these implementations would need to be registered, but only the latest version offered for new blocks.
• Implement block migrations: Similar to versioning, but assuming a block implementer understands a change they're making, they could explicitly define a path for migrating attributes of the previous version to the now-updated implementation

[youknowriad]

Thanks for opening this issue, this is something we need to figure out.

• I'm thinking we still need 3 to do 4. Because we need to validate a block before applying a migration (What if a block is invalid and we try to migrate it). Thus, starting with 3 seems like a good approach (while keeping in mind this could be extended with a migration function)

• Also, thinking this mechanism could handle different block names, to allow us handle use-cases like: Renaming a block (aka text 2 paragraph), Merging two blocks (text and cover-text)

[notnownikki]

Migrations! Yay!

Heavy +1 on doing 3, then 4. Happy to help on this one, data migrations are a thing I did lots.

[aduth]

Possible implementation is to have version as an optional property of a block. For all versions except 1, defining also a migrate function to transform attributes from the previous version. Only the latest version needs to define edit, save, or any other properties of a block. Only the latest version is offered for new insertion.

registerBlock( 'myplugin/foo', {
    version: 1,

    attributes: {
        // v1 attributes ...
    },
} );

registerBlock( 'myplugin/foo', {
    version: 2,

    attributes: {
        // v2 attributes ...
    },

    migrate( attributes ) {
        // given v1 attributes, return v2 attributes
    },
} );

registerBlock( 'myplugin/foo', {
    version: 3,

    attributes: {
        // v3 attributes ...
    },

    migrate( attributes ) {
        // given v2 attributes, return v3 attributes
    },

    edit() {
        // current implementation
    },

    save() {
        // current implementation
    },
} );

[aduth]

Another option with single registration:

registerBlock( 'myplugin/foo', {
    versions: [
        // v1
        {
            attributes: {
                // v1 attributes ...
            },
        },

        // v2
        {
            attributes: {
                // v2 attributes ...
            },
            migrate( attributes ) {
                // given v1 attributes, return v2 attributes
            }
        },

        // v3 (latest)
        {
            attributes: {
                // v3 attributes ...
            },
            migrate( attributes ) {
                // given v2 attributes, return v3 attributes
            },
        },
    ],

    edit() {
        // current implementation
    },

    save() {
        // current implementation
    },
} );

[notnownikki]

I find the multiple registration version easier to read. It also allows the migrations to be moved into a 'migrations' module and keep the main file uncluttered.

[BE-Webdesign]

Also worth considering them as distinct blocks so the slug is myplugin/v1/foo and keep the version attribute along with that. Otherwise it will be hard to distinguish between slugs.

[youknowriad]

I'm for the second option, the "old blocks" are not really blocks (any more) and shouldn't be registered as blocks.

[BE-Webdesign]

I'm for the second option, the "old blocks" are not really blocks (any more) and shouldn't be registered as blocks.

Good point. We can put the versions etc. into separate files as well, to address Nicola's point. I'm on board for option two. We still will want to make sure we add versioning into the attributes. That way we know what block is what etc.

[aduth]

A couple considerations are:

• Is it really in the user's best interest to automatically upgrade their block to the latest version, or should we allow them to keep what they already have in content?
  
  
  
  • Conversely, is it in the developer's interest to have to maintain older versions that are still used by users?
  
  
• In the migration case, how would we validate the original content?

Also worth considering them as distinct blocks so the slug is myplugin/v1/foo and keep the version attribute along with that.

Given the above, this is certainly appealing as a simple case. The block implementer, when changing a block, simply creates a block with a different unique slug (maybe just including the version number). We'd then want a way for them to flag their previous versions as disabled (maybe just a disabled flag).

[notnownikki]

I'm for the second option, the "old blocks" are not really blocks (any more) and shouldn't be registered as blocks.

How about having registerMigrations instead? Keep it explicit what they are, and encourage use of a migrations module?

[mcsf]

Is it really in the user's best interest to automatically upgrade their block to the latest version, or should we allow them to keep what they already have in content?

(i.) This exposes three types of version bumps, then:

• Those that are totally innocuous and ultimately translate to zero cosmetic changes for the user's content. Maybe a new attribute has been introduced but its default value results in that attribute not being expressed visually.
• Those that aren't.
• Those who may change things for the user depending on the values currently set for that block's attributes. Maybe an attribute's shape has changed, and _if_ the block is only using v1's default values, then overwriting to v2 results in no change.

Type 3 is obviously harder to confidently deal with and could be assimilated into type 2. However, should the burden of decision be placed on the user for a bump of type 1? It may actually turn out to be confusing or frustrating, as a user, to be presented with the choice to upgrade and then see no resulting change.

(ii.) This brings me, more generally, to the aspect of communication. Could a lower-tech and lower-commitment solution be to provide a mechanism for block authors to explain an upgrade to their users? For instance:

Cover Image now lets you control how much you'd like the background to be dimmed!
[ Sounds good! ] | [ I'll pass. ]

(iii.) If a user chooses not to upgrade — whatever the mechanism chosen, from the most sophisticated of those proposed here to the least — how and when can they change their mind?

[aduth]

It may be simpler to start with the assumption that versioning should always be backwards-compatible, at least in the sense that there should be no content loss or regression in visual appearance, even if the underlying attributes have been remapped.

If we did want to support breaking changes, maybe these ought to be implemented as separate blocks, where transforms are defined to allow a user to explicitly convert the block to the new canonical block type. I assume Undo behavior and revisions should be sufficient here to allow the user to revert back to the original copy should they regret their decision to transform.

[mcsf]

Another thing on my mind: whatever the solution we come up with, it would be nice that the same foundation could be used for a _graceful downgrade_ mechanism for Gutenberg. I had an IRL chat about this — I'm not sure anyone has tracked it — and the gist is:

• Say a plugin provides _Enhanced Galleries++_, which renders image galleries with _much wow_ (perhaps some animated Doge figures float around the images).
• Aside from editing blocks of this type, some assets from the plugin may also be needed for the front-end display of the block.
• Assume the user disables this plugin, or migrates the post, or anything that leads to the plugin no longer being available to Gutenberg.
• By default, the block would no longer be recognized by Gutenberg and that would be the end of the story.
• The _graceful downgrade_ mechanism here would mean that, when the user went back to editing their post, Gutenberg would offer the possibility of keeping the block intact _or_ downgrade it to a regular gallery. It would know how to map the attributes for type _Enhanced Galleries++_ to type _core/gallery_. The enhancements proper to the third-party plugin will have been lost, but the fundamental gallery is preserved.

[mtias]

We have a case here that would need to support sourcing from footer even though the new markup would use cite: https://github.com/WordPress/gutenberg/pull/2859#issuecomment-334185465

I'd be interested in an approach that just handles variations of markup. Maybe:

attributes: {
    citation: {
        type: 'array',
        source: children( 'footer' ) || children( 'cite'),
    },
},

Or...

attributes: {
    citation: {
        type: 'array',
        source: children( 'footer, cite' ),
    },
},

or...

attributes: {
    citation: {
        type: 'array',
        source: children( 'cite' ),
        legacy: children( 'footer' ),
    },
},

[aduth]

I'd be interested in an approach that just handles variations of markup. Maybe:

While the second of these works _as-is_, we would need a mechanism that prevents blocks with the legacy markup from being flagged as invalid, since the behavior for block invalidation is to compare a reserialization of that same block against the saved markup, which is expected to now be different in the new version of the block.

If the parser knows that the attribute value it receives is from a legacy format, we could potentially skip this validation step, though this has a few issues:

• We don't know if the legacy block was in-fact invalid, since we don't know what the original markup shape was
• We would either need to force the block markup to be upgraded to the latest version (potentially destructive), optionally with a prompt, or disallow the user from making further changes (since we don't know how to mix the two versions of the block markup)

The first of these challenges is still an issue for the other proposals here where we merely define attributes of the different versions. The only alternative I could imagine to satisfy this need is to always keep the full serialization (save) implementation for every version of the block. We may as a compromise to keeping this simple decide that we lose this ability to detect legacy invalidations.

If we did decide that this is important, I feel that with a need to define most properties of a block anyways, we may be better off just establishing a convention of creating separate block registrations for new block versions (e.g. myplugin/block-v2') andisPrivate`-flagging old variations to prevent them from being shown as options in the inserter.

[Idealien]

Could a lower-tech and lower-commitment solution be to provide a mechanism for block authors to explain an upgrade to their users?

Is there any part of this which could be delegated to the upgrade process (core/theme/plugin) that generates the block? or CLI ? I can see authors being overwhelmed by prompts if every existing post prompts them to upgrade / convert a few blocks.

If we did want to support breaking changes, maybe these ought to be implemented as separate blocks, where transforms are defined to allow a user to explicitly convert the block to the new canonical block type.

What would the expected impact on themes / headless systems be when a block changes it's type? Some other issues mention wp_query attributes for block type and it is all over https://github.com/WordPress/gutenberg/pull/2649

[aduth]

What would the expected impact on themes / headless systems be when a block changes it's type? Some other issues mention wp_query attributes for block type and it is all over #2649

It's not clear what you have in mind by impact; as I see it, there would be no fundamental difference between how a block is initially represented in post content before and after the change in type, except in the latter case the specific markup may be altered by the modified save implementation (if we assume it to be an upgraded type).

To your point, it's more about deciding on block APIs that allow for attributes to be transferred from one type/version to another, and the UX by which this is communicated to the user (automated, prompted, explicit action).

[youknowriad]

Started looking at this issue, noting that this would add some complexity to the Parsing Flow. I made this diagram to clarify how this would work.

[wpalchemist]

I have a lot of concerns here, some from the user's perspective and some from the developer's perspective.

We are dealing with a couple of related but different scenarios here: (a) a developer pushes a change to a block, and (b) a user deactivates a plugin that builds a block.

1) What will user expect to happen when a block is changed, or when a plugin that creates a block is deactivated? In principle, it's a nice idea to ask users if they want $shiny_new_feature or not, but in practice, that is going to get very messy. First of all, you'll have to rely on developers to put together the code describing the new features and asking about them. But then, what happens if a plugin makes 30 changes in one release? Or a user hasn't updated for several versions, and then gets questions about all of those versions at once? I think that users are going to expect blocks to behave in more or less the same way as shortcodes: when a shortcode is updated, all instances of that shortcode are updated right away. I think that's what users are going to expect when they update a plugin with a block (especially if the update is fixing a bug!). I don't think it makes sense for old content to continue to use the old format while new content uses the new block format - that is going to be confusing for users, and make websites look inconsistent. User expectations around plugin deactivation are more complicated... With shortcodes, you see the shortcode, which is obviously not a good experience for anyone, so we don't want to replicate that. But if I deactivate a plugin, I expect everything that plugin does to go away. I think I would be surprised to deactivate the _Enhanced Galleries ++_ plugin and still see enhanced galleries on my site.

2) I know that I just advocated for automatically updating all blocks when a plugin is updated, but what happens if Gutenberg has to update every post every time a plugin with a block is updated? We've all seen plugins that sometimes have to update the database on upgrade, and although it is necessary sometimes, it can get messy - servers can time out, databases can break, and sometimes it can take a really long time. Users can find it intimidating (especially if there are messages about "backup your database first"), and this could become a barrier to updating. I could see users just deciding to never update instead of having to go through a database update every time.

3) I also just advocated for removing everything that a plugin does when you deactivate it. But if I deactivate and reactivate a plugin, I would expect that plugin's content to still be there, and to look just like it did before I deactivated.

4) From a developer perspective, I don't think it makes sense to maintain old versions. Some developers are going to be too lazy or uninformed to do it. It could make for some really bloated code.

5) Fallbacks (_Enhanced Gallery ++_ downgrading to a plain gallery) seem like a nice idea, but will get very messy in practice. What if a developer creates a block that doesn't have a logical fallback? What if a developer declares a fallback that really doesn't work with the block's attributes? What if the fallback creates something the user really wasn't expecting to happen?

[mtias]

Closing as we have an initial implementation.