Gutenberg: Render dynamic blocks on the front end in JavaScript instead of PHP

[iandunn]

Is your feature request related to a problem? Please describe.

When creating a dynamic block, developers have to duplicate the rendering logic of the edit JS component in a PHP render_callback, which is time-consuming, error prone, and (in theory) unnecessary.

Describe the solution you'd like

The front end should be rendered by JavaScript, reusing the same components that are used to render the block in the editor. In my mind, that would look like this:

1. PHP could render the initial front end page using the_content() like it does now, and the block's metadata comment would be left in tact -- e.g., <!-- wp:wordcamp/speakers {...} -->
2. A JS function would scan the page content looking for block comments, and asynchronously renders them, injecting the rendered block into the DOM in place of the comment. In this context, the editing controls/toolbar would be removed.

To clarify, that would be the _opposite_ of using ServerSideRender; both the front- and back-ends would be written as components and rendered by JavaScript.

I realize that we can't have the entire front-end page served from JavaScript, because most servers won't have Node.js available, but it seems like we could have a client-side script render the blocks asynchronously after the the PHP page has loaded.

What that rejected for performance reasons? Are there use cases where the editing and viewing experiences are so disparate that they can't reuse the same components? Am I missing some other fundamental flaw with that approach?

I'm guessing this has already been discussed and rejected for one reason or another, but I searched for a long time and couldn't find any previous discussions. If that's the case, can you provide links to those discussions, to make them more discoverable?

[iandunn]

I discussed this a bit with @aduth, and want to summarize that so we can continue the discussion here.

Andrew explained that Gutenberg was designed with a fundamental distinction between the editing and saving modes, which is also reflected in how static blocks have separate edit and save functions.

The ideal editing experience will not always look like a preview of the final content. They often look similar in simple cases, but can diverge greatly in more complex cases, and are fundamentally different concepts. He referred to #104 for some background on the early formation of that distinction, and felt like reusing the edit function for preview would quickly become messy in practice.

After considering that, and playing around with an example, I agree that trying to reuse edit would not be practical. I do still feel like there's a way to vastly improve the developer experience, though.

For static blocks, the separation between edit and save is usually not that painful, because the developer is working in the exact same environment, and can reuse the vast majority of the code they wrote for edit.

For _dynamic_ blocks, though, that's not true, and the developer is forced to re-write everything from scratch, in a different language, using different APIs, taking into account different esoteric security considerations, etc. It's an awful developer experience, and has a huge opportunity cost because the developer has to waste a ton of time reinventing the wheel, manually keeping the two sides in sync when changes are made, and fixing bugs that inevitable arise from the error-prone nature of the separation. Instead of doing all that, developers should be able to spend that time building new things that make an impact for their organizations.

The primary source of that pain is the fact that the rendering for dynamic blocks is done in PHP instead of JavaScript. If we could do it in JavaScript instead, then we could reuse components like we do for static blocks. There'd be some overhead and duplication, similar to save, but it'd be minimal, and a huge improvement from the current situation.

One way to do that would be to add a render.js file inside the block folder to define the rendering markup. The block's save function would render a spinner as a placeholder, and that would appear when the page first loads. The build would generate a separate render.min.js file, which would be enqueued via wp_enqueue_script, and once the DOM is ready, a function in render.min.js would be called, and it'd replace the spinner with the real content.

render.js could import only what it needs from the rest of the block, in order to make itself as lean as possible.

That could eventually be taken a step further, and optional support for Node could be added, so that in professional environments the content could be pre-rendered and bundled into the initial page that gets sent to the visitor. It could even integrate WP Cron, to periodically prime a cache of the markup asynchronously, and save that in post_content (along with the meta-data), so that the pre-rendering doesn't slow down the TTFB.

[felixarntz]

Andrew explained that Gutenberg was designed with a fundamental distinction between the editing and saving modes, which is also reflected in how static blocks have separate edit and save functions.

Agreed, preferably the variant used in edit should provide interactive controls right where the final content would appear, to provide a more intuitive in-content editing experience.
However, for several dynamic blocks, as mentioned this is not viable.

The primary source of that pain is the fact that the rendering for dynamic blocks is done in PHP instead of JavaScript. If we could do it in JavaScript instead, then we could reuse components like we do for static blocks. There'd be some overhead and duplication, similar to save, but it'd be minimal, and a huge improvement from the current situation.

I would like to point out the possible role Web Components could play here. I think they are a viable alternative as they would follow a component-based model close to the one that React uses, but at the same time would be agnostic of the environment so that they could as easily work in the frontend as they would in the editor. Some remarks:

• Web Components can be used inside React, especially on the leaf component level.
• They are browser-native, so would easily work in the frontend (in a few cases still requires a polyfill, but support is improving steadily).
• They can include additional markup, JavaScript and styles so dynamic functionality is covered.
• They could even be printed in PHP if the server wanted to print markup with the same dynamic functionality directly.
• They are context-agnostic and reusable so other projects could benefit from the ones Gutenberg would create and vice-versa.

I explored usage of Web Components in Gutenberg block types in a POC, where I re-created the Latest Posts block type in an alternate variant: https://github.com/felixarntz/web-components-in-gutenberg/blob/master/assets/block-types/latest-posts.js is the block type definition, while https://github.com/felixarntz/web-components-in-gutenberg/blob/master/assets/custom-elements/wordpress-post-list.js is the Web Component definition used by the block type.

[gziolo]

I also want to emphasize that blocks like Latest Posts with edit written in JavaScript and save rewritten in PHP is extremely confusing for new Gutenberg contributors. I had chats with @AmartyaU and @draganescu on this topic in the last two weeks, and it seems like they both were surprised learning that you still need to code the same logic in PHP to make it work.

[felixarntz]

A considerable downside to most likely _any_ approach that solves this in a pure-JS way is that the frontend would need to make API requests to get the data rather and dynamically create the markup which decreases performance and provides a worse user experience than when seeing the content immediately.

The DUX would totally be improved by getting rid of PHP here, but I think we need to carefully weigh the advantages/disadvantages against each other - after all the users come first, and for them the requirement we have for PHP doesn't matter.

[iandunn]

That's a really great point. I wonder if there's a way that the block metadata which gets stored in post_content could include some query arguments, and then the server could include the JSON results of those queries in the initial HTML response?

[iandunn]

Fetching any data that a block needs and bundling it with the initial HTML response would also benefit the editing experience, since it would make the initial render a lot faster by avoiding the HTTP requests that would otherwise be necessary.

[gziolo]

Gutenberg does prefetching for the most important REST API endpoints and stores results in cache of wp.apiFetch middleware. This makes it possible to avoid triggering additional network requests on initial page load.

[coreymckrill]

Gutenberg does prefetching for the most important REST API endpoints

Is it possible to augment the list of endpoints that gets prefetched?

[gziolo]

Is it possible to augment the list of endpoints that gets prefetched?

Yes, it looks like there are ways to do so already:

https://github.com/WordPress/wordpress-develop/blob/17146f7192cd1c63ccde76be93d75465ccd32900/src/wp-admin/edit-form-blocks.php#L42-L64

_It's probably targeting only admin page which renders Gutenberg._

[aduth]

A considerable downside to most likely _any_ approach that solves this in a pure-JS way is that the frontend would need to make API requests to get the data rather and dynamically create the markup which decreases performance and provides a worse user experience than when seeing the content immediately.

Important to note the SEO implications here as well, as even with Google's crawling of JavaScript content having improved over the years, it remains imperfect from my most recent understanding, and is a reason why services like prerender.io exist.

[draganescu]

Because of "circumstances" I have some good experience with server side rendered SPAs. Since we don't enjoy the luxury of running JS on the server, then, in my opinion, I don't think we'll be able to escape using PHP to render content and serve its initial state in the response.

It is both a matter of SEO and one of UX: having content in the page is much better than spinners and flickers or page blocks jumping because they get content later. Of course, everything can be solved client side only, but, in my experience, in a much more complicated way than simply rendering on the server.

In our case, the problem is not that we want to "get rid of PHP", which is a non starter, but to make it such that we don't need to write the same logic twice in two different programming languages, better put: _to be able to skip this step as often as possible_.

And that's what meta data is for: to explain between environments what's happening so that a generic agent can do its thing without all the implementation details.

I think @iandunn 's idea with having some more meta data ("query args") in the post_content for dynamic blocks could be a good direction. However a solution would require access to the template of the content, and since there is no such notion of template anymore, b/c components have the markup in the render function, we need to do more work.

Imagine the system would somehow have used Handlebars templates or similar, then PHP would only need one generic filterable render routine, for which the editor would embed arguments and the template location in the post_content.

We don't have Handlebars or similar, however, most likely a solution would be on the same line: somehow having blocks providing a template which is server render-able and the pointer to data. At least for what can be done this way, _while still supporting server side render callbacks for extremely advanced situations which I assume are rare and worth the effort_.

In other words: we shouldn't aim to "Render dynamic blocks on the front end in JavaScript instead of PHP" but instead aim to "Improve dynamic blocks' PHP rendering to avoid code duplication".

For example, and this is a ballpark solution, the latest posts block could provide as block meta data two things: the loop's template (in a parseable form) and the rest endpoint where the latest posts are. It's trivial to render server side then, without any matching, just by using the keys of whatever REST returns as template variables. The key here is to have a way to extract this template automagically from the save / edit renders :) which is something I have no idea if it can be done.

Of course, this is not a universal solution but it is another step into having less and less need to do the duplication. This kind of approaches incrementally move closer and closer to not needing to do "custom" SSR, but still having that server render callback feature in place as it still is a very good feature, in any environment where GTB is used but there is no JS on the server.

[felixarntz]

For example, and this is a ballpark solution, the latest posts block could provide as block meta data two things: the loop's template (in a parseable form) and the rest endpoint where the latest posts are. It's trivial to render server side then, without any matching, just by using the keys of whatever REST returns as template variables. The key here is to have a way to extract this template automagically from the save / edit renders :) which is something I have no idea if it can be done.

I like this approach a lot, as it's a solid compromise between UX and DUX here. We could define a schema for how dynamic data should be handled in Gutenberg which (as said above) would need to follow three key principles:

• The block type must specify a REST endpoint that returns the data in exactly the right format (so that no further JS/PHP processing is needed).
• The block type must define mappings between block attributes and whether/how to send them to the REST API request.
• The block type must define a template that will be used both server-side and client-side (in a templating language we will decide on that works in both JS and PHP). The REST API response data is directly passed through to that template.

This should cover almost everything - a consideration would be things like date formatting for example: We can theoretically return dates in a formatted way from the REST API, but doing that would really take us away from the structured approach REST APIs should typically have. Although, to be fair, having block type-specific REST endpoints already does that.

[ddluc]

Hi folks,

I'm a long time react developer working with Wordpress and Gutenberg for the first time and find that having to resave my page/post that uses my custom Gutenberg block every time I make an update to be incredibly painful. My "block" is fairly complex and needs to be broken down into multiple sub-components. But, any time the markup for any component in my tree is updated, my block is invalidated.

I've bypassed this pain by setting up a local dev server within my plugin directory, but even that doesn't solve the problem that once I publish the block, if I need to make any changes, every instance of the block on the site needs to be manually updated.

Why not leave it up to the developer to choose whether the block is prerendered? So, instead of two attributes (i.e. save and edit) you could have a third method, say ,render, that allows the block to be rendered on the front end and bypass the block validation?

import './components/MyComponents`. 

registerBlockType( 'example/block', {

  ...

  edit: ( props ) => {
    return (
    <div>Editor Block View</div>
    );
  },

  save: ( props ) => {
    return null
  },

  render: (props) => {
    return (
      <MyComponent />
    )
  }

} );

I don't know enough about what is happening under the hood, but presumably, there is some way to run this render method on the front end and bind it to an html root element for the block:

ReactDOM.render(
 // psuedo code where my block's render method is saved as a reference to be 
 // used by wordpress 
  wp.block.referenceToMyBlock.render(); 
  document.getElementById('')
);

[achatainga]

I have to second what @ddluc is saying. I have been working with React and React Native for a while and trying out Gutenberg for the first time... It turns out my plugin is designed to show a custom React Form and do API calls to login/register/recover_pass a user before displaying the content. Everything works so well until I get to the validation part. WordPress just is not disabling it.

Using a third render as stated above would be the perfect solution for my use case since I could avoid this validation errors and proceed...

The data I saved from the block is encrypted and many more complex things are happening and I am wondering if I have to through away the idea of using wordpress altogether because of this

[mintplugins]

Here's an approach I have taken for my blocks thus far to avoid building in both PHP and JS, and for me it has worked pretty well.

• All of my blocks are dynamic (rendered via PHP)
• But the only things I render are:
  
  
  
  • 
    
    
    1. A div with a unique ID
    
    
    
  
  
  • 
    
    
    1. Within that div, I output the variables for my block settings as JSON, wrapped in a div set to display:none.
    
    
    
  
  
• Upon page render, I use React-Dom to find-and-replace that div with a react component, and I pass that hidden JSON as a prop for the initial state of my react component.
• My react component has an "edit_mode" flag as part of the state, which conditionally outputs editing elements, and passes the values back up to Gutenberg for saving.

[ddluc]

@mintplugins That is not a bad temporary solution until a more permanent solution is in place—I'd love to see how you've configured your plugin and webpack build to make this happen.

Do you have any open-source plugins or examples I could refer to? I think this solution would make a really good blog post. As I mentioned above, I'm more of React developer than a WP developer, but a couple of projects I work on have CMS requirements and this is the first problem I have run into, so I imagine there might be a lot more people out there that are looking for a more passable solution to this other than rendering blocks in PHP.

[dgwyer]

@ddluc Another consideration, if you're rendering blocks on the frontend via React, is SEO. I initially thought it would be a great idea to be able to reuse React components on the frontend for ALL my blocks.

However, depending on the block type this might not be a good idea. For block types like galleries, sliders, and so on where content is mainly visual it's a good fit. But if you're outputting text then I don't know how this affects SEO performance?

The solution @mintplugins posted is an interesting one as the content is rendered as attributes on a hidden element. So it is there on page load in some form, albeit not the final semantic structure. So, again the SEO impact needs taking into account.

[skorasaurus]

For what it's worth, I felt completely in the same boat as @gziolo mentioned... I had been wondering whether was some rule that I didn't find documented anywhere (why where some blocks in JS, part in php?)

@iandunn thank you so much for sharing your chat.