Wp-calypso: SPT: Templates using layout-grid block broken when plugin is disabled

[simison]

• Older Atomic sites have layout-grid plugin available but disabled.
• Customers can disable layout-grid plugin on their own
• They still have an SPT feature available at their site.
• Some of the templates rely on the layout grid block.

Steps to reproduce

1. Have an old Atomic site, or new one but just make sure layout-grid plugin is disabled and FSE plugin is enabled.

2. Create a new page (and not a post), or modify old one, and open layout picker:
   

3. Choose this Portfolio template:
   

4. Note how grid block is missing:
   

Reported by @hafizrahman in Slack (p1580880138055100-slack-dotcom-create) and another convo at p1580907695089300-slack-ajax

cc @obenland @iamtakashi

[marekhrabe]

We have already relied on some other non-core blocks before the layout grid - many of them originated or have since been moved to the full-site-editing-plugin so they always ship with Starter Page Templates. Others are in the Jetpack plugin which we assume (but not check) is around everywhere we make use of SPT.

Layout grid lives in a completely different repo (block-experiments) than our other blocks and it seems to be built and deployed as a single-block plugin to sites. Many other blocks also seem to be in wpcom-blocks repo (I'm not sure those are being used in templates).

What we do now

In SPT, we parse blocks from layouts before showing the selector thus we know which blocks are present in each template. I think we can use this knowledge:

1. to hide templates containing blocks that are not supported. Since layout-grid was introduced into many templates, I don't think this would be a great solution.
2. to show a list of missing plugins when you preview a template and ideally offer a one-click solution (is it possible?) to install and/or activate plugins that contain missing blocks. Why did I mention doing this right when previewing and not when selecting? Doing it after selection is too late as you already have a broken preview when blocks are missing
3. anything else you have on your mind we can do with this info?

We can look into shipping required blocks with the FSE plugin. While this solves the problem perfectly, it is a solution that doesn't scale and we will be facing the same dilemma after few weeks when somebody includes a new block in their template.

Can we make the FSE plugin dependant on other WP plugins being installed too? I don't think this is a part of WP plugin system but I'd like to confirm it.

Can we activate the plugin on sites where FSE is active?

Should we limit the amount of templates that we allow template creators to be used? (probably too late for this as layout-grid is now well established)

I think these are all ideas I have in my mind now. Really happy to hear more opinions or getting deeper into any of things I've mentioned.

[simison]

We can look into shipping required blocks with the FSE plugin. While this solves the problem perfectly, it is a solution that doesn't scale and we will be facing the same dilemma after few weeks when somebody includes a new block in their template.

We shouldn't have this problem going forward since everything should just ship in Gutenberg, FSE plugin or Jetpack (p58i-8qD-p2). They could of course be running too old version of Gutenberg but suppose we check for that in FSE somewhere?

[getdave]

hide templates containing blocks that are not supported. Since layout-grid was introduced into many templates, I don't think this would be a great solution.

Personally, despite the downsides Marek notes, I think this is the best longer-term solution as it protects us from showing what are essentially "broken" template offerings.

Whilst we know why the "missing block" notice appears, to users it's just "this is broken" and that's not a good look for us.

We can look into shipping required blocks with the FSE plugin.

This is what needs to happen. Matias has been very clear (p58i-8qD-p2) that there are 3 locations where Blocks can live and we shouldn't have Templates relying on Blocks in other repos.

We need to discover the code owner of the layout-grid block and discuss with them how we can get this merged into FSE. It's likely we'll need to have another implementation where the block lives in another repo but is periodically "synced" to FSE.

While this solves the problem perfectly, it is a solution that doesn't scale and we will be facing the same dilemma after few weeks when somebody includes a new block in their template.

I agree with this. Therefore I suggest we add some functionality to the Master Template Site which:

• Detects which Blocks are provided by the FSE Plugin.
• Detects Blocks used in the current Template and warns the author if any Blocks are not part of FSE.
• Communicate clearly to Template authors about the above so they know the consequences of introducing new Blocks without doing through the require process of having it integrated into FSE.

We shouldn't have this problem going forward since everything should just ship in Gutenberg, FSE plugin or Jetpack (p58i-8qD-p2).

I think @marekhrabe is right that there is always a possibility that a human being won't follow this advice. Therefore I think adding code-based warnings and safeguards on the Master Template site will help.

Pinging @ajlende for his input here.

[ajlende]

@getdave @marekhrabe Right now, the blocks from the block-experiments repo are getting released in the plugin/block directory, including layout-grid. wpcom-blocks is for blocks that we're not sure where to put immediately or for blocks that may be moving elsewhere, but that's where layout-grid started. They're still experiments to us and releasing to the plugin directory has been the easiest way for us to iterate more quickly, gauge interest, and to prove their usefulness.

It seems to me, based on this discussion, that templates is where layout-grid has proven itself, but @mkaz can offer more detail about moving it than I can.

[getdave]

Thanks @ajlende. That's great information to have.

They're still experiments to us

My concern is that building customer-facing Templates using Blocks that are considered "experimental" leaves us open to "broken" experiences for our customers. This is isn't a criticism - purely an observation that we need better guidance for those building Templates.

layout-grid has proven itself

It certainly looks that way and it's a really cool Block. However, this further convinces me that we need an _automated_ way to warn Template authors when they use Blocks that are not supported directly from FSE and thus might cause broken experiences for customers.

Would love further input here.

[obenland]

/cc @ianstewart @iamtakashi ^

[simison]

Detects Blocks used in the current Template and warns the author if any Blocks are not part of FSE.

While that kind of warning might make sense from technical perspective, I'm not sure if it will make sense for customers.

For customer this should "just work".

Does that make sense?

[mkaz]

I'm not quite sure I understand the issue.

Layout grid was developed by Tinker, it is an open source plugin deployed to the block directory (dot org plugins) with source available in block-experiments. It is available on Atomic sites, auto-activated when a user switches over, and installed on WPCOM in gutenberg-blocks directory.

Is the issue when a customer deactivates the plugin it breaks their site templates? What about when using other blocks in templates, won't this be the same issue? I'm not sure bundling everything into a single-plugin is the right solution. A dependency check of some sort would probably be more scalable and allow for a great flexibility for building templates with a rich assortment of blocks.

[getdave]

While that kind of warning might make sense from a technical perspective, I'm not sure if it will make sense for customers.

@simison I would agree except I think you may have the key context that this would _only_ happen on the Master Template Site (see PCYsg-m0L-p2). Given that this wouldn't be customer-facing, does that alter your viewpoint?

---

@mkaz Thanks for offering some considered perspective here. Much appreciated.

Is the issue when a customer deactivates the plugin it breaks their site templates?

The issue is that if a block that is not part of FSE is used in a Template then we cannot guarantee it will be available. It doesn't matter where it comes from or who produces it. If it isn't in FSE then it's a 3rd part dependency and may not be present, thereby leaving users with a ~broken experience~ missing block warning when:

• they preview such a Template in the Page Template Selector UI.
• they insert such a Template into the Editor.
• they return to a Page that was created using one of these Templates (less of an issue as if you remove a Plugin after the fact then you might expect breakages).

Those are my concerns. When we offer Templates to users they should work with no exceptions, or we shouldn't be offering them at all.

A dependency check of some sort would probably be more scalable and allow for a great flexibility for building templates with a rich assortment of blocks.

Thinking on this last night I started leaning towards an approach which simply _removes_ Templates from the Page Template Selector UI (user-facing) if they depend on Blocks that are not currently registered (ie: installed).

With this approach we could (and should) still show a warning on the Master Template Site (context: PCYsg-m0L-p2) to warn template authors when they are using Blocks that are not part of FSE (note this doesn't rule out your suggestions above, but I think it's important to highlight when Blocks are external to FSE bundled Blocks).

We could then either:

1. Create the dependency management solution such as what you and others have suggested above.
2. Incorporate a "synced" version of the layout-grid Block into FSE.

Note that for 2 above, this is "paving the cowpaths" as we already follow a similar pattern for Newspack Blocks.

I'm not sure bundling everything into a single-plugin is the right solution

As noted above this is the current solution that was agreed not only by @Automattic/ajax but by those building out product roadmaps. I see two arguments on this point:

• By requiring everything is bundled we simplify dependency management - ie: if it isn't in FSE then don't make a Template that uses it.

VS

• By requiring everything is bundled we limit the possibilities for Template expression and add a technical overhead to onboarding new blocks (such as the excellent ones you have been building!) into Templates.

I wonder what @obenland feels about this and whether he could request a steer on the preferred direction here?

[marekhrabe]

In case data might help here, I've put together a quick analysis of our templates: https://gist.github.com/marekhrabe/1ca4d5f436040afbced1123444088d7d

It shows the same data from two perspectives - a list of templates and blocks they use and vice versa — a list of blocks and templates that are using it. Line 55 shows that Layout Grid is used by almost all non-homepage layouts, 18 layouts total. I would call it a major dependency at this scale.

If we hide layouts when blocks are not available (which I think might be a good start now), we should figure out how to maybe show something like "There are XYZ more layouts that require plugin ABC to be active. Activate plugin ABC to show them"

[mkaz]

@getdave Thanks for the response, and @marekhrabe thanks for the data.

My point around bundling isn't just around Layout Grid though, as the data shows there are several templates that use blocks from CoBlocks. I don't think we want to also include that into the FSE plugin. My thought is enabling a system that embraces using more blocks from wherever will allow for richer templates and more capabilities for template creators and users.

Could we provide context and information on the plugin page. This is what it shows now:

If someone is using one of the plugins, like CoBlocks or Layout Grid, it can show this plugin is used in X template, disabling will break posts relying on that template, or something like that.

Also, when selecting templates, if plugins are required that not activated, then show a notice saying not all templates are shown due to X plugin is not active.

[davemart-in]

My concern is that building customer-facing Templates using Blocks that are considered "experimental" leaves us open to "broken" experiences for our customers.

I agree.

Two things:

1) The Layout Grid block have been instrumental in the page templates that @iamtakashi and others have been building so far. Let's determine whether the Layout Grid is something that can be included in core. That sounds like the idea scenario as importing it outside of core may lead to dependency issues down the line. @mtias do you have any thoughts on this one?

2) We should proceed with this proposed improvement (hiding templates with missing blocks) regardless. If the Layout Grid block can be merged into core this shouldn't really affect our existing set of templates. If it can't be merged into core, then we'll either need to figure out a way to side load it, or come up with a new pattern of laying out page templates.

cc @ianstewart, @rickybanister

[simison]

leaning towards an approach which simply removes Templates from the Page Template Selector UI (user-facing) if they depend on Blocks that are not currently registered (ie: installed).

That would leave customers with a variety of experiences instead of one solid experience for everyone. That'll eventually cause more issues like this where something isn't available even if it should. I.e. you're just repeating the same problem elsewhere later on. Imagine folks ending up being on e-commerce plan and not seeing e-commerce templates for example.

I think you should go another way around; make sure the blocks are available for the templates that are there.

[iamtakashi]

I'm not sure about the idea of hiding the layouts that require certain plugins. Customers would never know there are more layouts they could use if they install or activate a plugin. It will be much harder for them to know what plugins they'd need to install to have access to certain layouts.

The great thing about Block patterns and Layouts it's intuitive for customers because they are visually oriented, and I think we should show all the visuals and help the customers to have required plugins if they need it.

[davemart-in]

My take:

I agree that we should do everything we can to ensure that the templates we build just work for all WP.com customers. It would be a shame for customers to not have access to all of the great templates that we're building.

That said, I think we need to be mindful of a couple of things:

Core blocks are the best blocks
The absolute best way to ensure consistency in experience is to do our best to stick with core blocks when building page templates. If a foundational block like the Layout Grid block which is used in almost all of our templates to date is not in core, we run a much greater risk of providing an inconsistent experience for folks.

JP blocks are second best
Outside of core blocks, we should probably do everything we can to limit the blocks we use in page templates to those within JP. This way we limit our dependancies down to a single plugin. The more plugins that we require customers to keep installed, the greater the chances are that one will mistakenly or intentionally be disabled or deleted and the greater the chances are that we won't be providing a consistent experience to all customers.

The current way we handle missing blocks is not a great experience
Telling people that the block cannot be loaded, but not giving them an avenue to fix the problem is obviously not ideal:

Page templates could become a trojan horse for spammy block distribution
Granted, this is more of a core concern than a Dotcom one, but if we were to change the way we handle missing blocks within page templates it might open the door for abuse. Say for instance we change the missing block notice so that it links them to install the missing plugin, we'd open up the possibility for any plugin to add a hundred page templates, each intentionally with missing blocks, with the goal of using this solely as an avenue to increase distribution for those blocks.

I think my preference would be to:

• Stick with just core and JP blocks when building templates for use on WP.com.
• Advocate for foundational blocks like Layout Grid to be included in core.
• Putting some thought into improvements for the error message that is shown when blocks are missing from a page template. Perhaps, figure out how to suggest how they can remedy the problem.
• Hiding templates which contain more than one missing block.

[jasmussen]

This is an excellent and crucial discussion. 🎉

The guiding principle for the block editor is to "let a thousand blocks bloom", and the challenge outlined in this issue is important to solve not just for this project, but for the core project itself, notably given a future of block-based themes.

For that reason I would encourage thinking in terms of short term wins and long term considerations. The latter should be what guides principles and high level approaches, and the former should be considered temporary solutions, necessary stepping stones on the path.

Longer term, the block directory will be here, and can benefit this flow in a few ways:

• Instead of "doesn't include support for jetpack/layout-grid", the message could be far friendlier: "The Layout Grid block is missing [Install Block] [Keep as HTML]".
• Installing and activating a block would be a single click with a spinner and instant reload.
• Pasting in content or selecting a template or activating a theme could invoke a dialog: "This template leverages the Layout Grid block. [Install Block] [Keep as HTML]".
• Search for a missing block in the block library and install it then and there.

The block platform is WordPress' "app store", similarly, _there will be a block for that_. Smoothing out that flow will be key to helping blocks drive the WordPress economy. Tackling the very real flow/spam/security/UI challenges involved is difficult, but a challenge worth tackling head-on.

Specifically for templates and block patterns, embracing a diversity of blocks is crucial to enabling designs that are interesting enough for people to click on. If we limit ourselves to just the core blocks, we will be needlessly handcuffing our design language. That's not to say we shouldn't improve the core blocks — we should do that _also_!

In the shorter term, I would encourage creative brainstorming for creating flows that inform the user what to do to enable these plugins and why that is necessary. I've seen flows that are single-click install and activate for any plugin in the plugin directory (which Layout Grid is) — click Tools > Import > RSS:

It seems worth exploring if we can leverage that as a holdover.

[getdave]

Accidental closing. Sorry! I seem to be messing up this thread.

[getdave]

Great comments above. Thank you @simison, @iamtakashi and @jasmussen for considered input.

I completely understand where you are coming from. I acknowledge:

• The need to use more than just Core Blocks (at least for now) to achieve interesting layouts.
• That the Block Directory will make it easier to resolve "missing block" flows.
• That, in the mid/longer term, it might be possible to devise ingenious methods to install missing Block "dependencies" on demand.

That said, we don't have any of the above in place right now and I'm conscious that whilst we are spending time coming up with solutions our customers have a broken experience.

I'm genuinely interested to understand if we all agree that if a user sees the current block missing warning (which is a Core feature and nothing to do with SPT) then they will see the Templates as being broken? If not then we're looking at this from fundamentally different perspectives.

Now, of course, if we had a means of resolving the "missing blocks" in a user-friendly way on-demand today, then I might be inclined to think differently. But we don't.

Therefore, as @jasmussen suggests, in the vein of trying to think of short and long term solutions how do we feel about the following?

• Implement a "fix" to hide all Templates when the current site doesn't have a given Block installed.
• Begin researching and planning around a mid-term solution of mapping key Template blocks to Plugins (those outside of FSE) and providing an "install on demand" prompt and a warning notice. This would then supersede the "hide Templates" fix above.
• Begin planning for a longer-term solution involving the Block Directory and using this to inform the development of this feature in Core.

The benefits of the approach above:

1. We retain the freedom of expression in Templates.
2. 90% (made up figure!) of users (those who will have all the requisite Blocks) will see no change.
3. Some users who don't have the necessary Blocks will miss certain Templates. They won't know they're missing them however and I think that is an acceptable compromise.
4. Users won't have the impression that WP.com offers broken experiences.
5. We will be actively exploring and implementing improved flows safe in the knowledge that users won't have broken experiences.

I appreciate that all things are technically possible, but we do need to acknowledge time constraints vs the impact on users. I'm confident we can devise the optimal solution, but I'm not confident that can be achieved quickly enough given that users are seeing broken Templates right now.

I'd rather deploy a temporary fix to avoid broken experiences and then iterate.

I feel like I may be in a minority here, so I'm open to being told that we're happy to accept broken templates if that's the consensus. I'm really only trying to focus on delivering the best user experience within what is feasible in a given timeframe.

Thanks again for this great discussion.

[marekhrabe]

I'm genuinely interested to understand if we all agree that if a user sees the current block missing warning (which is a Core feature and nothing to do with SPT) then they will see the Templates as being broken?

I'm with you on this!

I'd rather deploy a temporary fix to avoid broken experiences and then iterate.

Same.

90% (made up figure!) of users (those who will have all the requisite Blocks) will see no change.

I think this is important - we are talking here about a fraction of atomic users that have the ability to manage their plugins and be able to disable things like the Layout Grid. On dotcom simple, we always ships all required ones.

Begin researching and planning around a mid-term solution of mapping key Template blocks to Plugins (those outside of FSE) and providing an "install on demand" prompt and a warning notice. This would then supersede the "hide Templates" fix above.

I'd be happy to start explorations on this once we merge the immediate hotfix hiding broken stuff. I think we can start very simply since we only have a small handful of blocks that could be missing (data!). Showing a notice with a number of missing templates, a list of plugins needing activation, and a link to plugins.php screen could be a good enough start.

[jasmussen]

As far as short term solutions go, I would defer to you and your teams to evaluate what's best there. Appreciate, though, that the longer term solutions are also considered.

[retrofox]

In my humble opinion, the question is what causes more impact on customers:

• Hide some templates, reducing the list that we give them in order to speed up the process to create a page.
• Show them templates with broken blocks.

In general, the second one seems to be a matter of plugging settings or something like that.

I'd rather deploy a temporary fix to avoid broken experiences and then iterate.

Making a quick fix hiding the templates with broken blocks sounds reasonable to me, 👍
But also, we should guarantee that we are going to work on a solution with the purpose of showing all the templates that we have. That's the point of SPT: offer as much as possible templates to customers.

Instead of "doesn't include support for jetpack/layout-grid", the message could be far friendlier: "The Layout Grid block is missing [Install Block] [Keep as HTML]".

I like this it's long term solution.

[getdave]

Appreciate, though, that the longer term solutions are also considered.

...we should guarantee that we are going to work on a solution with the purpose of showing all the templates that we have. That's the point of SPT: offer as much as possible templates to customers.

@retrofox @jasmussen I will create a Issue for this longer term solution to ensure it is tracked and (initially at least) assigned to our team for prioritisation.

[simison]

When I opened this issue, I wasn't as much concerned about users disabling plugins and losing features. That's kinda acceptable since it's a destructive action and easy to understand as a user.

What we have here is a missing feature in an environment we fully control, in out of the box experience we've shipped.

Could we just move the grid block to FSE plugin like we've done for other blocks too? That's straightforward and would've avoided this in first place.

90% (made up figure!) of users (those who will have all the requisite Blocks) will see no change.

I think this is important - we are talking here about a fraction of atomic users that have the ability to manage their plugins and be able to disable things like the Layout Grid.

Month ago when I checked these were the numbers of FSE plugin and grid block plugin:

• Layout grid 12,860
• Full site editing 46,004

So for major portion of Atomic sites with templates they're broken. That's maybe smallish absolute number compared to simple sites but these are also our highest paying customers, bringing almost 4MEUR in plan sales.

[jasmussen]

Could we just move the grid block to FSE plugin like we've done for other blocks too? That's straightforward and would've avoided this in first place.

CC: @johngodley @mkaz

[marekhrabe]

We also rely on CoBlocks (dynamic-separator and gallery-masonry), not just a matter of Layout Grid.

Would we merge those blocks too into FSE as they are causing the exact same issue?

[mkaz]

What we have here is a missing feature in an environment we fully control

You obviously do not _fully_ control the environment, or you could just activate the Layout Grid plugin. I think this is the mind set that is the basis of these decisions — it is an environment you want to fully control, but the nature of Atomic is you won't ever be able to.

My thoughts are to build more guides and ways to help users understand what is happening; for example, if a plugin is needed to use a template, then prompt the user to enable that plugin.

Could we just move the grid block to FSE plugin like we've done for other blocks too? That's straightforward and would've avoided this in first place.

The long view is that designers at Automattic should be empowered to use _more_ blocks, that by nature can't all be in the FSE plugin. So while it might mitigate the short-term issue, it doesn't help the long term and punts the issues down the road.

We are about to push a patch to improve the layout grid block,, so if you do end up copying into FSE, after next week's update would be best.

We are considering core itself a potential destination for the layout grid block, but need to figure out how it fits alongside the columns block (cc: @mapk)

To @marekhrabe and my points earlier, copying into FSE only accounts for the single plugin and is limiting in the future on what can be used to create templates — embracing choice and flexibility will provide users a richer experience.

[getdave]

@mkaz @simison @iamtakashi @ianstewart @davemart-in @marekhrabe @jasmussen I've created a followup Issue to ensure this doesn't get forgotten.

It will be for @davemart-in and @obenland to prioritise this for our team.

Feel free to add suggestions to the followup Issue.

[iamtakashi]

@getdave Thanks for creating the followup issue. Much appreciated.