Gutenberg: 1.9.1: Classic "Visual" editor litters Gutenberg markup w/ superfluous `<p>´ tags

[lkraav]

(EDIT still broken in 2.3.0)
(EDIT 2018-07-04 improved in 3.1.1, shortcodes still broken)
(EDIT 2018-07-21 ^^^^ 3.3.0)

Issue Overview

Classic "Visual" editor litters Gutenberg markup w/ superfluous <p> tags.
This is an issue where occasionally the user might have to revert to using the classic editor, due to Gutenberg metabox rendering JS crashing for whatever reason. Even if the user doesn't touch the content block at all, just loading Gutenberg markup in classic Visual editor will cause it to break on Update.

I do not have Classic Editor plugin activated. Unless this is officially going to be a "run Gutenberg" requirement, I probably shouldn't have to activate it, right?

Steps to Reproduce (for bugs)

1. Author a new Gutenberg post with this example markup

<!-- wp:heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:heading -->

<!-- wp:paragraph -->
<p>You can crush me but you can&#x27;t crush my spirit! Is today&#x27;s hectic lifestyle making you tense and impatient? It&#x27;s a T. It goes &quot;tuh&quot;. I&#x27;ll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[column grid="2" span="1"]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don&#x27;t like lilacs! Your &#x27;first&#x27; wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[/column]
<!-- /wp:shortcode -->

<!-- wp:shortcode -->
[column grid="2" span="1"]
<!-- /wp:shortcode -->

<!-- wp:paragraph -->
<p>Hello world.</p>
<!-- /wp:paragraph -->

<!-- wp:shortcode -->
[/column]
<!-- /wp:shortcode -->

1. Open this post in classic editor, switch to Visual mode
2. Switch back to Text mode to review littered markup
   

Expected Behavior

Gutenberg markup stays intact

Current Behavior

Littered markup result - shortcodes are wrapped into <p> tags, all block marker comments empty lines get replaced w/ <p>&nbsp;</p>:

<!-- wp:core/heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:core/heading -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>Hello world.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->

Possible Solution

Not sure which component is the responsible party - core or Gutenberg - could use pointers.

Screenshots / Video

Related Issues and/or PRs

Maybe #3900? Not sure, could use pointers.

[lkraav]

@norewp next step here is to test my two bug scenarios w/ the freshly released 1.9.0.

[lkraav]

Tested on 1.9.0, situation is exactly the same.

[aduth]

Making a note to test this behavior in #4005 if we move in the direction of client-side wpautop . Seems this is where things are likely breaking down. Alternatively the selective wpautop which is currently implemented server-side ought to be trimming whitespace between blocks:

https://github.com/WordPress/gutenberg/blob/6a287a4efa4b0ab8aea05d4abb23717bcfb9c9bd/lib/blocks.php#L193-L232

[azaozz]

Perhaps a better alternative would be to turn wpautop off when editing Gutenberg posts in the classic editor. Leaving the <p> tags intact works quite nicely there.

[aduth]

Perhaps a better alternative would be to turn wpautop off when editing Gutenberg posts in the classic editor. Leaving the <p> tags intact works quite nicely there.

This exists already:

https://github.com/WordPress/gutenberg/blob/91dd36c3162b631c2f8f00fd0da941eba32c0b0d/lib/compat.php#L126-L141

See: #2708

But we still conversely need to turn wpautop on selectively for legacy posts when loading the Gutenberg editor.

[aduth]

The root cause here appears to be in TinyMCE's content serializer during content initialization, which converts the comment demarcation sequence to a new paragraph node:

_(Code reference)_

Off the top of my head, a couple ideas:

• If we could force content to be set as non-raw for the initialization, the serializer would not be applied. We could apply this exclusively to block'd content.
  
  
  
  • Unfortunately I don't see a way to configure this, since it appears that the initial content load hard-codes the html format
  
  
• We could try to find a format which doesn't trigger TinyMCE's default handling of "comment node + newline + comment node = new forced root paragraph"
  
  
  
  • Eliminating the newlines seems to work okay, but this is probably not a format we'd want to save in, and while there's some precedent in server-side-specific per-block formatting, it's a pattern we're seeking to move away from (#4005)
  
  
• Internal changes to TinyMCE's serializer to not apply the forced root node from the serializer on these text patterns

cc @spocke

[azaozz]

Confirmed, also made a fiddle: http://fiddle.tinymce.com/0hgaab/9

Imho both comments and following line breaks should be ignored when forcing root blocks in TinyMCE. There shouldn't be difference between:

<!-- comment -->
<p>a</p>
<!-- comment -->

<!-- comment -->
<p>b</p>
<!-- comment -->

<!-- comment -->
<p>c</p>
<!-- comment -->

and

<!-- comment -->
<p>a</p>
<!-- comment --><!-- comment -->
<p>b</p>
<!-- comment --><!-- comment -->
<p>c</p>
<!-- comment -->

[lkraav]

Tested broken on 2.2.0.

EDIT Tested broken 2.3.0.

[danielbachhuber]

Closing in favor of https://github.com/WordPress/gutenberg/issues/6385#issuecomment-394719216, which has a proposed solution.

[lkraav]

Just to follow up here for at least my own records, 3.1.1 shows improvement.

<p>&nbsp;</p> litter is gone. Unfortunately, shortcodes still become broken (wrapped in <p> tags).

v2.x

<!-- wp:core/heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:core/heading -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->
<p>&nbsp;</p>
<!-- wp:core/paragraph -->
<p>Hello world.</p>
<!-- /wp:core/paragraph -->
<p>&nbsp;</p>
<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->

v3.1.1 (=== 3.3.0, 3.2.0)

<!-- wp:core/heading -->
<h2>Shinier than yours, meatbag.</h2>
<!-- /wp:core/heading -->

<!-- wp:core/paragraph -->
<p>You can crush me but you can't crush my spirit! Is today's hectic lifestyle making you tense and impatient? It's a T. It goes "tuh". I'll tell them you went down prying the wedding ring off his cold, dead finger.</p>
<!-- /wp:core/paragraph -->

<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->

<!-- wp:core/paragraph -->
<p>I daresay that Fry has discovered the smelliest object in the known universe! For the last time, I don't like lilacs! Your 'first' wife was the one who liked lilacs! No argument here. A true inspiration for the children.</p>
<!-- /wp:core/paragraph -->

<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->

<!-- wp:core/shortcode -->
<p>[column grid="2" span="1"]</p>
<!-- /wp:core/shortcode -->

<!-- wp:core/paragraph -->
<p>Hello world.</p>
<!-- /wp:core/paragraph -->

<!-- wp:core/shortcode -->
<p>[/column]</p>
<!-- /wp:core/shortcode -->

[danielbachhuber]

Unfortunately, shortcodes still become broken (wrapped in <p> tags).

Yes, this is tracked with #4456