Gutenberg: Block API: Server-side awareness of block types

Created on 20 Sep 2017  ·  31Comments  ·  Source: WordPress/gutenberg

Previously: #2529, #104
Related: #886
Related Slack conversation: https://wordpress.slack.com/archives/C5UNMSU4R/p1505677036000104 (cc @jasonbahl)

When initially developing the API for implementing a block (#104), there were some competing objectives:

  • The editor representation of a block _must_ occur in the client to preserve an ideal user experience
  • A block should _ideally_ be easy to implement for quick prototyping (#27, e.g. defining block in a browser console)
  • General information about a block should be context unaware (title, attributes definition, category, etc need not specifically be defined in a client, server, or an other external static file)

Where we have fallen short is in:

  • The third of these objectives; As a need has arisen for server-side attribute validation (#2529), we have further fragmented the definition of a block (its attributes may be defined on the server _or_ in the client).
  • It is not possible for the server to have reliable awareness of all blocks which would be registered in the editor.

Proposal:

To improve consistency, we should move block attributes (and other general information) to a server block registration. This would likely behave exactly as it does in #2529, including client-side attributes bootstrapping and validation, but would be applied consistently across all blocks. Therefore, every block would have a companion PHP file associated with it, which in addition to current supports defines:

  • Title
  • Icon
  • Category
  • Attributes

The trade-off here is that it is not as simple to implement a block in a single location. We could potentially leave client-defined attributes support as-is, but this could also entice developers to avoid the server registration and reintroduce all the inconsistencies/drawbacks therein.

Some previous discussion had considered the idea of a third JSON "manifest" file, similar to a package.json or composer.json, describing the static general information about a block. While this would be a simple format for what is essentially overview data, is problematic because (a) it is yet another file that a developer could need to become familiar with, (b) it is difficult to bring into the client without a build step understanding JSON imports, and (c) localization of strings would need to implement some amount of "magic" (hard-coding to properties expected to be localized).

It's worth pointing out that a plugin author already needs some amount of server-specific logic for each of their blocks, specifically that of enqueueing the JavaScript files responsible for registering the blocks in the client. By requiring block registration in the server, there may be some added benefit in handling scripts and styles.

  • A block could define its companion scripts and styles as properties of the block type (e.g. script_handle, style_handle).
  • _Or:_ These files could be "discovered" adjacent the block registering file by naming conventions (e.g. find style.css, edit.css, block.js adjacent myblock/index.php). This could encourage developers to define their blocks in a standalone folder/file structure for easier separation overview.

A remaining challenge in moving all block attributes definitions to the server is support for attribute sources. With advent of additional source types (meta, options), we will likely need to expand this concept to cover more than just DOM-based attribute sourcing. Therefore, I would propose creating an alternative structure for representing what currently exists as the assignment of source functions:

_Before:_

{
    url: {
        source: attr( 'img', 'src' )
    }
}

_After:_

[
    'url' => [
        'source' => [
            'type'     => 'attribute',
            'selector' => 'img',
            'name'     => 'src',
        ],
    ],
]

The intent here is not necessarily that the server would become responsible for parsing attributes from the markup of a block (although there is nothing that precludes it from doing so), but rather to be able to represent the structure in a static format, which could then be bootstrapped into the client to recreate the current behavior.

As an example for how this extends into the concept of additional sources:

'author' => [
    'type' => 'string',
    'source' => [
        'type' => 'meta',
        'name' => 'author', // Inferred from attribute name if not specifically defined?
    ],
],
'siteTitle' => [
    'type' => 'string',
    'source' => [
        'type' => 'option',
        'name' => 'name',
    ],
]

__Open questions:__

  • While there is justification for defining block attributions in the server for the purpose of improving consistency, there is still a lack of general understanding of specific use cases for server-specific awareness of blocks, aside from a "nice to have" reasonings. Lack of these use cases is not intended as an argument for a server definition not to exist, but it would help guide specific implementation decisions.

cc @westonruter

Core REST API Task [Feature] Block API [Feature] Extensibility [Type] Tracking Issue

Most helpful comment

@aduth thanks for creating this ticket and CCing me!

there is still a lack of general understanding of specific use cases for server-specific awareness of blocks, aside from a "nice to have" reasonings. Lack of these use cases is not intended as an argument for a server definition not to exist, but it would help guide specific implementation decisions.

I'll share some use cases (some might still be considered nice to have but in my opinion should not be dismissed).

General WordPress Consistency

WordPress thrives on the hook/filter system, and defining many pieces of the block Schema on _just_ the client limits the power of the system that essentially makes WordPress what it is.

I know there's been work on a JS implementation of similar hook/filter functionality, but that's only useful in the context of WordPress itself, and WordPress is used to power much more than just itself (I'll talk more about decoupled apps below).

Take a step back and imagine if the entire admin was rewritten in JS (think Calypso). And we statically defined Post Type and Taxonomy Registration _only_ in JS and _not_ on the server.

Now think about how much of the WP Ecosystem at large would not exist at all because of the lack of a server-side registration. . .a LOT. There would be no way to generate per-post-type feeds or REST endpoints. Calypso wouldn't be able to work with external WordPress sites, because it wouldn't know what post_types and taxonomies exist in the sites, because that info wouldn't be available via REST, because the REST server wouldn't have any knowledge of it. . .

Without comprehensive server-side registration, we drastically reduce the power of WordPress as we know it to interact with the blocks.

Same goes with most extendable parts of WordPress. . .image sizes, post_stati, post_types, taxonomies, admin_menus, etc. . .so many things are statically defined on the server, so _not_ having a solid registration for Blocks server-side seems like it goes agains WP's history. I know there is a minimal schema, but think if register_post_type was just register_post_type('post') and didn't accept any additional args for labels, capabilities, etc. . .

Also consider existing conversations about how troublesome Meta Boxes in Gutenberg are going to be, mostly due to the lack of a true Fields API (server side schema for fields).

Ideally, one should be able to register a block on the server and that becomes _the_ source of truth for the block's capabilities. The REST API could make use of it, other plugins could make use of it, other APIs (XML-RPC, WPGraphQL, etc) could make use of it, and of course the Gutenberg JS itself could make use of it.

Alternative UIs
Let's be honest. Gutenberg is _not_ going to solve everyones's problems. No matter how awesome it is or will become, it simply _will not_ meet _everyone's_ needs. There _will_ still be a market for creating content in different ways. Page builders currently completely dismiss the Post Edit screen, and these alternative post creation solutions will continue to exist post-Gutenberg.

With a solid Server Side schema, hopefully the Page Builders could at least start to adhere to a standard way of interacting with "Blocks". . .that way, if Page Builder A thinks using Modals to edit content makes more sense than the fixed-right sidebar of Gutenberg, they could build their own UI to interact with Gutenblocks, but still ensure the blocks are saved in the same way.

I think having a solid server-side schema for blocks will help standardize the Page Builder market. Page builders could essentially become "Gutenberg Themes" where the editing experience looks and feels different, but at the end of the day the data produced is the same. This probably isn't impossible without a comprehensive server side schema, but it would be much easier.

If there was a comprehensive Block schema on the server, page builders could use that to hydrate their view layer and ensure compatibility with Gutenberg (avoid the lock-in, at least to a degree, that everyone talks about with page builders).

Headless CMS / Decoupled Apps

Headless CMS with WordPress is nothing new. . .one of my first projects with WordPress was populating a Flash site with data from WordPress via XML-RPC back in ~2009. . .but now with the REST API in core (and exciting plugins like WPGraphQL), it's becoming more and more common to see folks interacting with WordPress in non-traditional Theme / WP-Admin contexts.

For any of these decoupled applications to interact with Gutenblocks, they have to have knowledge of the blocks in the same way Gutenberg itself does. . .not just the blocks that have already been created by Gutenberg inside the WP-Admin, but also the capabilities of what _can_ be done to create/modify blocks. That way these decoupled applications can provide an experience comparable in some fashion to Gutenberg. If you edit a post on a Native Mobile App, Calypso or some other decoupled app, the data should be able to be saved back in the same format that Gutenberg saves it. But in order for said decoupled app to provide an experience that allows for data to be saved in the same format as Gutenberg, said decoupled app needs to know what the capabilities of each block are.

If everything is defined in client-side code that lives in the WP-Admin, we're limiting interaction with the blocks to _just_ the WP-Admin.

The way I see this, is that blocks will have a statically defined registry which will act as _the_ source of truth for what blocks are available, what their capabilities are, the attributes they _can_ have, the types (string, int, bool, array, etc) of data their attributes are expecting to store, etc.

With this knowledge, a decoupled application could request this Block registration from the server, hydrate a client and render out a UI based on the Block registration.

The Gutenberg editor could also use this same registration to Hydrate what blocks are available, what UI elements are needed to provide editing of the available attributes per-block, etc.

Having the configuration defined on the server _should_ also help reduce code duplication. If the server is _the_ source of truth, then the client and server can both use that truth for their needs. The client wouldn't need to define block attributes and fields in addition to the server, just know how to read the source of truth from the server and do something with it, which should actually reduce JS file sizes because there's less info in there defining things, and instead just rendering data that gets served from the client.

Alternate Storage Options
Right now the block data is stored as a large string in the post_content (which is genius for backward compatibility), but I see alternative storage being explored by both core and plugin developers.

For example, where I work we will likely at least experiment with offloading Blocks and their associated meta to ElasticSearch, so that we can read/write individual blocks without having to alter the _entire_ post_content to fix a typo in one block, for example. We will also use ElasticSearch to to aggregations, etc to get insight into how blocks are being used across our sites, etc (how we use ES is beyond the point, though). The point is that the server will need to know about the blocks so validation can be properly made when interacting with external systems.

ElasticSearch has a type mapping for the data that gets sent to it, and if there's a server side schema for blocks, plugins like ElasticPress, or even Jetpack's ElasticSearch implementation would be able to provide mapping for blocks to ES to ensure block data gets indexed properly and efficiently. Without a server side schema, the mapping would be created only as data is sent to ES and ES would give it's best guess on things, but we'd all lose out on the potential ES _could_ give us if there were a server side Block registry that could be used to define mapping.

ElasticSearch aside, I imagine the core team or other developers will explore options for storing Block data in WordPress in some fashion. . .whether it's new wp_blocks and wp_blocks_meta tables or something else, I'm sure folks will experiment with storing blocks outside of a massive post_content string and as we've learned with the evolution of post_meta, the lack of a schema makes things "interesting" to say the least.

WPGraphQL

I'm the creator and maintainer of WPGraphQL (https://github.com/wp-graphql/wp-graphql) which brings a GraphQL API to WordPress. . .so I'll pitch my case, which I think is applicable to all the above in some fashion.

WPGraphQL allows for data to be declaratively fetched and for _just_ the requested data to be sent to the client from the server.

An example of a WPGraphQL query right now would be:

{
  post(id:"someId") {
     id
     title
  }
}

And that produces a JSON response like so:

{
  data: {
    post: {
       id: "someId",
       title: "Hello World"
    }
  }
}

As you can see, unlike REST where a request sends back a pretty large payload with the entire post object (unless you've written up some custom feature endpoints 🤢 ), the minimal amount of data needed is transferred from server to client, which is super beneficial, especially on slow mobile networks where data transfer can hold things up.

Anyway, with Gutenberg I envision WPGraphQL being a super powerful tool, especially for decoupled applications. I can picture folks using Gutenblocks with various block meta to indicate whether a block should be viewable on desktop or mobile, (or in our case WordPress even powers our newspaper print content). With WPGraphQL, a native mobile client could ask for _just_ blocks that are meant for use in a native context.

For example, Let's say I've extended all of my Gutenblocks to have a "Context" multi-select which would allow me to apply various contexts for where my blocks should display (Web / Native Mobile / Print, for example).

I could have a post created by Gutenberg that has this HTML:

<!-- wp:core/heading {"align":"center", contexts:[web]} -->
<h3 style="text-align:center" id="some-anchor">Heading to show only on web</h3>
<!-- /wp:core/heading -->

<!-- wp:core/heading {"align":"center", contexts:[mobile-native, web]} -->
<h3 style="text-align:center" id="some-anchor">Heading to show ONLY on mobile</h3>
<!-- /wp:core/heading -->

I could query from a React native (or similar) app, and ask for _just_ the blocks that were tagged with "mobile-native" context. . .something to this tune:

{
   post(id:"someId") {
      id
      title
      contentBlocks( where: { context: MOBILE } ) {
           ...on HeadingBlock {
               rawContent
               __typename
           }
      }
   }
}

I would receive a payload like:

{
  data: {
    post: {
       id: "someId",
       title: "Hello World",
       contentBlocks: [
         {
             rawContent: "<h3 style="text-align:center" id="some-anchor">Heading to show ONLY on mobile</h3>",
              __typename: "HeadingBlock"
          }
       ]
    }
  }
}

My post contains multiple blocks, but my Native client was able to request _just_ the blocks that were intended to be rendered in the Native context. This reduces network bandwidth as the data send to the client is limited to what _should_ be sent to the client, reduces bugs in code as it's a schema with defined type system, reduces transformation on the client as the payload contains what's needed so the client doesn't need to parse/transform the Gutenblocks for _all_ contexts to determine which ones it should use, etc. . .

With the current implementation of gutenberg_parse_blocks my hypothetical scenario is already _partly_ possible with WPGraphQL, but the big missing piece (in this example), would be WPGraphQL knowing what Enum options there are for the Context select. . . WPGraphQL would have no knowledge of what options exist for the Context select field if it's defined in the client, not the server. It wouldn't even know that "context" was a possible attribute of a block/all blocks if the attributes weren't defined on the server.

=====

I'll stop here, as I probably just sound like I'm spouting a bunch of nonsense. . .but happy to elaborate, discuss, clarify anything I've pointed out. . .and who knows, maybe some things I've pointed out already have been addressed since the last time I had to dig through things??

All 31 comments

@aduth thanks for creating this ticket and CCing me!

there is still a lack of general understanding of specific use cases for server-specific awareness of blocks, aside from a "nice to have" reasonings. Lack of these use cases is not intended as an argument for a server definition not to exist, but it would help guide specific implementation decisions.

I'll share some use cases (some might still be considered nice to have but in my opinion should not be dismissed).

General WordPress Consistency

WordPress thrives on the hook/filter system, and defining many pieces of the block Schema on _just_ the client limits the power of the system that essentially makes WordPress what it is.

I know there's been work on a JS implementation of similar hook/filter functionality, but that's only useful in the context of WordPress itself, and WordPress is used to power much more than just itself (I'll talk more about decoupled apps below).

Take a step back and imagine if the entire admin was rewritten in JS (think Calypso). And we statically defined Post Type and Taxonomy Registration _only_ in JS and _not_ on the server.

Now think about how much of the WP Ecosystem at large would not exist at all because of the lack of a server-side registration. . .a LOT. There would be no way to generate per-post-type feeds or REST endpoints. Calypso wouldn't be able to work with external WordPress sites, because it wouldn't know what post_types and taxonomies exist in the sites, because that info wouldn't be available via REST, because the REST server wouldn't have any knowledge of it. . .

Without comprehensive server-side registration, we drastically reduce the power of WordPress as we know it to interact with the blocks.

Same goes with most extendable parts of WordPress. . .image sizes, post_stati, post_types, taxonomies, admin_menus, etc. . .so many things are statically defined on the server, so _not_ having a solid registration for Blocks server-side seems like it goes agains WP's history. I know there is a minimal schema, but think if register_post_type was just register_post_type('post') and didn't accept any additional args for labels, capabilities, etc. . .

Also consider existing conversations about how troublesome Meta Boxes in Gutenberg are going to be, mostly due to the lack of a true Fields API (server side schema for fields).

Ideally, one should be able to register a block on the server and that becomes _the_ source of truth for the block's capabilities. The REST API could make use of it, other plugins could make use of it, other APIs (XML-RPC, WPGraphQL, etc) could make use of it, and of course the Gutenberg JS itself could make use of it.

Alternative UIs
Let's be honest. Gutenberg is _not_ going to solve everyones's problems. No matter how awesome it is or will become, it simply _will not_ meet _everyone's_ needs. There _will_ still be a market for creating content in different ways. Page builders currently completely dismiss the Post Edit screen, and these alternative post creation solutions will continue to exist post-Gutenberg.

With a solid Server Side schema, hopefully the Page Builders could at least start to adhere to a standard way of interacting with "Blocks". . .that way, if Page Builder A thinks using Modals to edit content makes more sense than the fixed-right sidebar of Gutenberg, they could build their own UI to interact with Gutenblocks, but still ensure the blocks are saved in the same way.

I think having a solid server-side schema for blocks will help standardize the Page Builder market. Page builders could essentially become "Gutenberg Themes" where the editing experience looks and feels different, but at the end of the day the data produced is the same. This probably isn't impossible without a comprehensive server side schema, but it would be much easier.

If there was a comprehensive Block schema on the server, page builders could use that to hydrate their view layer and ensure compatibility with Gutenberg (avoid the lock-in, at least to a degree, that everyone talks about with page builders).

Headless CMS / Decoupled Apps

Headless CMS with WordPress is nothing new. . .one of my first projects with WordPress was populating a Flash site with data from WordPress via XML-RPC back in ~2009. . .but now with the REST API in core (and exciting plugins like WPGraphQL), it's becoming more and more common to see folks interacting with WordPress in non-traditional Theme / WP-Admin contexts.

For any of these decoupled applications to interact with Gutenblocks, they have to have knowledge of the blocks in the same way Gutenberg itself does. . .not just the blocks that have already been created by Gutenberg inside the WP-Admin, but also the capabilities of what _can_ be done to create/modify blocks. That way these decoupled applications can provide an experience comparable in some fashion to Gutenberg. If you edit a post on a Native Mobile App, Calypso or some other decoupled app, the data should be able to be saved back in the same format that Gutenberg saves it. But in order for said decoupled app to provide an experience that allows for data to be saved in the same format as Gutenberg, said decoupled app needs to know what the capabilities of each block are.

If everything is defined in client-side code that lives in the WP-Admin, we're limiting interaction with the blocks to _just_ the WP-Admin.

The way I see this, is that blocks will have a statically defined registry which will act as _the_ source of truth for what blocks are available, what their capabilities are, the attributes they _can_ have, the types (string, int, bool, array, etc) of data their attributes are expecting to store, etc.

With this knowledge, a decoupled application could request this Block registration from the server, hydrate a client and render out a UI based on the Block registration.

The Gutenberg editor could also use this same registration to Hydrate what blocks are available, what UI elements are needed to provide editing of the available attributes per-block, etc.

Having the configuration defined on the server _should_ also help reduce code duplication. If the server is _the_ source of truth, then the client and server can both use that truth for their needs. The client wouldn't need to define block attributes and fields in addition to the server, just know how to read the source of truth from the server and do something with it, which should actually reduce JS file sizes because there's less info in there defining things, and instead just rendering data that gets served from the client.

Alternate Storage Options
Right now the block data is stored as a large string in the post_content (which is genius for backward compatibility), but I see alternative storage being explored by both core and plugin developers.

For example, where I work we will likely at least experiment with offloading Blocks and their associated meta to ElasticSearch, so that we can read/write individual blocks without having to alter the _entire_ post_content to fix a typo in one block, for example. We will also use ElasticSearch to to aggregations, etc to get insight into how blocks are being used across our sites, etc (how we use ES is beyond the point, though). The point is that the server will need to know about the blocks so validation can be properly made when interacting with external systems.

ElasticSearch has a type mapping for the data that gets sent to it, and if there's a server side schema for blocks, plugins like ElasticPress, or even Jetpack's ElasticSearch implementation would be able to provide mapping for blocks to ES to ensure block data gets indexed properly and efficiently. Without a server side schema, the mapping would be created only as data is sent to ES and ES would give it's best guess on things, but we'd all lose out on the potential ES _could_ give us if there were a server side Block registry that could be used to define mapping.

ElasticSearch aside, I imagine the core team or other developers will explore options for storing Block data in WordPress in some fashion. . .whether it's new wp_blocks and wp_blocks_meta tables or something else, I'm sure folks will experiment with storing blocks outside of a massive post_content string and as we've learned with the evolution of post_meta, the lack of a schema makes things "interesting" to say the least.

WPGraphQL

I'm the creator and maintainer of WPGraphQL (https://github.com/wp-graphql/wp-graphql) which brings a GraphQL API to WordPress. . .so I'll pitch my case, which I think is applicable to all the above in some fashion.

WPGraphQL allows for data to be declaratively fetched and for _just_ the requested data to be sent to the client from the server.

An example of a WPGraphQL query right now would be:

{
  post(id:"someId") {
     id
     title
  }
}

And that produces a JSON response like so:

{
  data: {
    post: {
       id: "someId",
       title: "Hello World"
    }
  }
}

As you can see, unlike REST where a request sends back a pretty large payload with the entire post object (unless you've written up some custom feature endpoints 🤢 ), the minimal amount of data needed is transferred from server to client, which is super beneficial, especially on slow mobile networks where data transfer can hold things up.

Anyway, with Gutenberg I envision WPGraphQL being a super powerful tool, especially for decoupled applications. I can picture folks using Gutenblocks with various block meta to indicate whether a block should be viewable on desktop or mobile, (or in our case WordPress even powers our newspaper print content). With WPGraphQL, a native mobile client could ask for _just_ blocks that are meant for use in a native context.

For example, Let's say I've extended all of my Gutenblocks to have a "Context" multi-select which would allow me to apply various contexts for where my blocks should display (Web / Native Mobile / Print, for example).

I could have a post created by Gutenberg that has this HTML:

<!-- wp:core/heading {"align":"center", contexts:[web]} -->
<h3 style="text-align:center" id="some-anchor">Heading to show only on web</h3>
<!-- /wp:core/heading -->

<!-- wp:core/heading {"align":"center", contexts:[mobile-native, web]} -->
<h3 style="text-align:center" id="some-anchor">Heading to show ONLY on mobile</h3>
<!-- /wp:core/heading -->

I could query from a React native (or similar) app, and ask for _just_ the blocks that were tagged with "mobile-native" context. . .something to this tune:

{
   post(id:"someId") {
      id
      title
      contentBlocks( where: { context: MOBILE } ) {
           ...on HeadingBlock {
               rawContent
               __typename
           }
      }
   }
}

I would receive a payload like:

{
  data: {
    post: {
       id: "someId",
       title: "Hello World",
       contentBlocks: [
         {
             rawContent: "<h3 style="text-align:center" id="some-anchor">Heading to show ONLY on mobile</h3>",
              __typename: "HeadingBlock"
          }
       ]
    }
  }
}

My post contains multiple blocks, but my Native client was able to request _just_ the blocks that were intended to be rendered in the Native context. This reduces network bandwidth as the data send to the client is limited to what _should_ be sent to the client, reduces bugs in code as it's a schema with defined type system, reduces transformation on the client as the payload contains what's needed so the client doesn't need to parse/transform the Gutenblocks for _all_ contexts to determine which ones it should use, etc. . .

With the current implementation of gutenberg_parse_blocks my hypothetical scenario is already _partly_ possible with WPGraphQL, but the big missing piece (in this example), would be WPGraphQL knowing what Enum options there are for the Context select. . . WPGraphQL would have no knowledge of what options exist for the Context select field if it's defined in the client, not the server. It wouldn't even know that "context" was a possible attribute of a block/all blocks if the attributes weren't defined on the server.

=====

I'll stop here, as I probably just sound like I'm spouting a bunch of nonsense. . .but happy to elaborate, discuss, clarify anything I've pointed out. . .and who knows, maybe some things I've pointed out already have been addressed since the last time I had to dig through things??

@aduth I highly support what you've proposed here. By having a language-agnostic schema defined for what a block's attributes look like, this can only open opportunities for improvements for portability of blocks across systems. If source types can all be defined in a schema then it would open the possibility for a server-side PHP processor to be able to parse and apply transformations to a block, or to parse a block in order to extract data for an indexing service, for example. It could also then be used in mobile apps for native code to do the same. The blocks API written in JS then in Gutenberg would serve as a reference implementation of a standardized/common format.

+1 if we could register blocks via PHP, this would greatly improve the ability for plugins like Pods to register blocks etc. Otherwise we're stuck building our own interface for it that then outputs JS objects that we have another JS file loaded to register those JS objects as blocks in the editor.

After a short discussion, what we want for Pods integration would be PHP registration of blocks, with some way for us to provide JS callbacks for edit/save, so that we can do what we need dynamically. In addition to this, having an ability to hook into a Gutenberg save-specific filter (what data to save) or action (saving) would be very handy for us to do some of the more complicated feature integrations that Pods would need.

If we move toward registering all blocks on the server for a context-unaware definition of a block, what's left to implement in the client editor is to support the user interactions therein: edit is the most obvious. Others like toolbar controls and inspector are included here, though currently as an implementation detail but would otherwise generally belong in the client-specific behaviors as well. Less obvious are details like: icon, category, transforms, and maybe even save. I'm inclined to treat these as client-specific (despite the text of my original comment). save is a can of worms I'm hesitant to explore: The roles it serves in the browser are important to enabling compatibility / reusability toward an edit and save duality, validating blocks, and generating markup for the text editing modes. Defining this on the server would be harmful to all these points, however exposes the possibility of handling block content updates from non-Gutenberg clients.

Seems like this should happen sooner than later, as I'd imagine many are already taking advantage of the JavaScript registration API to add support in plugins etc - it would be a shame to cause duplicative work for any 3rd parties.

Is anyone owning this / have a write up on specifically what we need to do? If the code is written, perhaps it's mainly a documentation effort needed next?

The most recent work here is in #2854, merged in the past week. Effectively we should have all of the pieces to start moving block registration to the server, while still supporting fully client-side block capabilities (there's still some desire to enable a client editor to be fully functional without a server part).

I see this being similar to what Customizer does with registering controls in core. They usually get registered in PHP but this is just a wrapper that passes the parameters that you passed to the client for passing into the corresponding JS API for registration. This gives opportunities for PHP plugins to know about and manipulate the controls, and the same would be true for blocks.

I see this being similar to what Customizer does with registering controls in core. They usually get registered in PHP but this is just a wrapper that passes the parameters that you passed to the client for passing into the corresponding JS API for registration

Effectively this does already exist, currently limited to attributes but easily expandable to any / all block properties we want to support from server registration:

https://github.com/WordPress/gutenberg/blob/7f518a4d0520502b123330926692fb1315e179cb/lib/client-assets.php#L747-L757

@aduth you pointed out that this preloads server-registered blocks:
https://github.com/WordPress/gutenberg/blob/7f518a4d0520502b123330926692fb1315e179cb/lib/client-assets.php#L747-L757

However, it's a completely optional thing. Gutenberg still loads up all 40 (or however many) blocks whether there are any blocks registered server side or not.

With plugin developers being asked to migrate functionality to Gutenblocks, the server-side API needs to be more of a first-class citizen.

Let's take Matias demo from WCUS as an example. He showcased editing a "Book" page that had some content and an image.

Let's say the workflow for an organization is that certain user roles can edit the content, and certain user roles can edit the image.

If all block logic is handled on the client, that means any user role can effectively edit any content, as the restriction to what they can edit is just a client-side prop that could easily be changed via the browser console giving any user power to do whatever they want. . .

The server _needs_ to be the contract between the client and the persistence layer. We can't trust the client 💯 .

I get that the current TinyMCE is basically free-reign, and any content can be placed in there without server-side validation, but that's 1 big text input. Gutenberg is introducing 100's of new text inputs, but _not_ introducing server-side validation/auth checks for any of them.

I feel like for a long time, when learning best practices folks would say: "Look how core does it", but when folks "look how core does it" and see Gutenberg doing things exclusively on the client with no contract on the server determining what's safe and expected, folks will follow suit, and will start writing code that trusts the client explicitly, encouraging bad practices and all sorts of security vulnerabilities down the road.

If Gutenberg is trying to be "the best editor" out there, and "leapfrog medium and squarespace", etc. . .then let's make it "the best editor" by also making sure it adheres to best practices (never trust the client, handle validation on the server, etc)

It would be interesting to see block-level capabilities. In the classic editor, if an admin adds a script tag or iframe and saves the post, but then an editor comes along and updates that same post, the script/iframe will get stripped by Kses.

In Gutenberg, the Custom HTML block could be used by an admin to add a script tag or iframe. If an editor user comes along and edits the post, they should be able to do so without destroying that block's contents. This could be done by sanitizing/validating block-by-block rather than the post_content as a whole. There are a couple server side attributes here: capability, validate_callback, and sanitize_callback. If the user doesn't have the required capability, then the block should be read only and locked. If the validate_callback returns a WP_Error for a given block's attributes, then the post save could be rejected from the REST API. And then the sanitize_callback would be essentially by default just do Kses like core does already.

Some further exploring of expanding the server bootstrapping at #3962

@aduth with this most recent PR in place, what do the docs look like for registering blocks where Server Side Registration will be encouraged as the primary registry?

@jasonbahl The latest documentation for registering a block type via PHP's register_block_type can be viewed here:

https://wordpress.org/gutenberg/handbook/blocks/writing-your-first-block-type/

@aduth ah, ok. I didn't realize that was updated already. Thanks!

Related issues that we need to take into consideration when we work on the final solution. They are also tracked here, but sharing below for better visibility:

Let's use this as the central issue.

So, in digging in and actually developong a site that will be using Gutenberg in production I’m even _more_ convinced of the importance of the Server Side API being the _primary_ way blocks are registered.

The Gutenberg client should be treated like _any other_ client that interacts with WordPress.

The Server Side Block registry should be the source of truth and not only include a list of blocks, but should also be where blocks express what post_types they can be edited on, what users can interact with them, what attributes they can have, what values should be allowed for the attributes, what type of field (text, rich text, etc) should handle the interactions with the attribute, and yes, even how blocks should be saved and _how_ blocks should be edited. . .

If the client determines the save mechanism, then plugin and theme developers who make custom blocks can _never_ have their blocks interacted with on decoupled clients like the iOS/Android apps or Calypso...because it’s custom client code that _only_ the WP Admin would know about. . .bummer.

The Gutenberg Fields Middleware project (https://github.com/rtCamp/gutenberg-fields-middleware) is a great example showcasing that blocks _can_ be registered fully on the server, and with essentially _no_ custom js, I can have heaps of new blocks that _can_ now be interacted with via the Gutenberg client AND are now possible to be exposed and interacted with via REST, WPGraphQL, WP-CLI, Calypso and the native iOS and Android apps.

——

Anyway, several issues I’m having right now with using Gutenberg for a _real_ production project all seem to come back to this ticket:

whitelisting/blacklisting

...Both for the entire editor and for nesting (@noisysocks is making great progress, but still no other client (mobile, etc) will know the rules about what blocks can nest within other blocks, etc if the server doesn’t know and can’t expose the rules to other clients)

filtering fields away from core blocks

pretty much all the inspector fields on the paragraph block need to go for our project...would be trivial to filter out (and/or filter in new fields) had the fields been added via a server side registry like what the middleware provides. . .FWIW I’d be happy with a client filter here for my immediate needs, but a server filter ultimately makes _way_ more sense

hydrating dynamic blocks on page load

With a good Server Side API, when the page loads the data for dynamic blocks can be generated up front and printed to the window, much like __wpBlocks, etc are now, saving the client from having to do many HTTP round trips to hydrate the dynamic blocks on the client. I have working code for hydrating dynamic blocks via GraphQL on the server, and allowing the client to take over, if necessary, and re-query for more data when variables change, etc. . .it should be easily applied to REST too. Blocks should be able to declare their data dependencies on the server and when the Editor loads, it can have _all_ the data loaded up front. Really will help with performance on several levels. . .both perceived performance and real performance.

validation

We still believe it’s important to have the server make the final call on whether something is both valid and save. For the most part Gutenberg still ensures its safe, but doesn’t ensure accuracy in any way. If a block attribute should only be allowed to be Blue or Green, and it gets changed to “red” on the client, the server _should_ be able to catch that, fix it, and notify the client that their input was invalid. Ideally client side validation is good enough that server side validation isn’t needed often, but we still think it’s important in many cases, especially since Gutenberg isn’t the only client, as we’ve established already. . .A robust server side api would be able to validate input from _any_ client to make sure blocks adhere to the shape they’re intended to...this even includes validating allowed blocks, etc. . .this would mean that saving of blocks would move to the server as well...but if course that _needs_ to happen for other clients to interact with blocks anyway.

I’m back 😉

Another issue that comes back to having a good Server Side API: translations

I posted my thoughts here: https://make.wordpress.org/core/2018/05/01/javascript-internationalization-the-missing-pieces/

But it seems like translations can be solved by having the client ask for things like registered blocks and the block inspector fields, (labels, descriptions, etc) and the server respond with these things already translated. 🤔

Another issue that comes back to having a good Server Side API: translations

It would only work for simple use cases. As soon as the number of plugins grows, you will want to have some of them to be lazily loaded when they are really necessary for your application to operate as expected. The same applies to translations. That's why at some point they should be controlled by the JS code and be bundled together with the corresponding logic that uses them.

@gziolo with a proper server side API and the core Gutenberg client JS shaped to be an API itself (like the direction Gutenberg Gields Middleware has gone) there wouldn’t need to be custom JS for _most_ plugins.

If Gutenberg core took what Gutenberg Fields Middleware is doing, and expanded it further, providing all the components needed to build flexible Interfaces, then users could register their blocks and block controls, their data dependencies, and even the edit experience from the server.

The Gutenberg interface is a tree of React components. If you can pass a tree of JSON from a server-side-api to the client, the client can build out whatever UI you tell it, without plugins writing a line of custom JS.

Obviously that’s not the case now, but it’s all possible if Gutenberg re-shaped things.

As it stands though, plenty of things are lazy loaded from the server, like contents of Dynamic blocks, which are already fully translated by the server. Take that a step further where _all_ blocks are registered by the server and _all_ blocks (not just their contents, but their field control labels, etc too) would be already translated when Gutenberg loads them.

If enough components and groups of components are mapped to a server side registry (like Gutenberg Fields Middleware allows) the only JS needed for plugins should be the js already loaded by Gutenberg core. And then custom JS from plugins _could_ still be added for more extreme situations, but the majority of plugins could do what they need to do via server registration of their blocks (and plugins, and other parts of the Gutenberg UI)

I can’t see this happening anytime soon because it would be a major shift in things, but ultimately it will make WordPress more useful to all clients (CLI, iOS, Android, REST, and the client known as Gutenberg).

ACF Flex Fields is conceptually _very_ similar to Gutenberg. Each flex field group is pretty much the same as a block, and when you interact with the flex fields it all happens via a JS client, but you don’t have to write a line of JS to register flex field groups or the fields within them. . .and ACF flex fields _can_ be interacted with via any other client. So plugin developers can register any flex field you could dream of and any client can have the knowledge of it with _just_ the initial core client code to read the API.

Gutenberg _can_ get there.

Even the Customizer is a great example of how server side APIs can create dynamic experiences without writing any custom JS. I can register a widget area and some widgets and I now have “blocks” that I can interact with in Customizer, and external clients can interact with as well if needed without any custom JS.

I think folks _should_ still be able to extend Gutenberg with custom JS, but it should really become the exception, not the rule.

For posterity's sake, and as discussed previous in #5649, #5652, and #5099, server-registered blocks will need to be able to handle common supports types in a generalized fashion (not implemented per-block): customClassName, align, etc.

More conversion about this being a focus in 5.1 in #4116.

Something to consider when we revisit this: https://core.trac.wordpress.org/ticket/45882

For those following this thread, I wanted to cross-link RFC for block registration opened by @youknowriad which is the first step towards making all core (and optionally plugin) blocks discoverable on the server.

I have made a first pass for this API in a core trac ticket here. Can someone review and get back to me?

I have made a first pass for this API in a core trac ticket here. Can someone review and get back to me?

Awesome, I will have a look closer to the end of the week 👍

@aduth @gziolo Decided to move the patch for the block type REST API to a gutenberg PR for easier testing. To that end, I have created #21065 .

Can anyone interested please take a look and see if this REST API has the required fields and data. Remember we can add fields later.

Thank you @spacedmonkey and @TimothyBJacobs for making it happen!

Great to see this one in :)

Was this page helpful?
0 / 5 - 0 ratings