Html: Compat: Should the parser execute scripts created in another document

[jakearchibald]

The spec says no:

[Scripts] parsed into different Documents than the one the parser was created for do not execute

But Edge, Chrome and Safari all allow it. Reduced test (see the console) - source.

I really like how this silly hack allows you to inject content in a streaming manner in a way that's very close to how a regular page-load behaves, including executing scripts, but some kind of streaming document fragment would be a better way to deliver that.

[zcorpan]

The normative part is

1. If the element is flagged as "parser-inserted", but the element's node document is not the Document of the parser that created the element, then abort these steps.

https://html.spec.whatwg.org/multipage/scripting.html#prepare-a-script

and

1. If the element is flagged as "parser-inserted", but the element's node document is not the Document of the parser that created the element, then abort these steps.

https://html.spec.whatwg.org/multipage/scripting.html#execute-the-script-block

[zcorpan]

Checking git log --grep "execut" I found:

This changed in:
7c52fca3722ede5d22c9de19356c5d5495640b2a

• http://www.w3.org/Bugs/Public/show_bug.cgi?id=11323
• https://bugzilla.mozilla.org/show_bug.cgi?id=592366

Other commits that may be relevant:
90d9255e6948365708bf55f74ff8f998d5a10fa4
2ba7d3a9da55153c9a0cd0d4100d13d0e8d3e29d
41c8f100bab7768fe117783bb0893f540bf93c95

[zcorpan]

cc @hsivonen @bzbarsky

[bzbarsky]

I think https://bugzilla.mozilla.org/show_bug.cgi?id=592366 summarizes the issue well enough, no? It's _possible_ to enable this behavior, but it leads to all sorts of complexity in the interaction of script elements and the parser, which is already quite complex. Absent an actual web compat need, I don't think we should be introducing this complexity into the spec and into implementations. Of course @jakearchibald just tried his hardest to introduce actual web compat need. :(

[jakearchibald]

Meh, I'm proposing it as an alternative to innerHTML = fetchedContent, which doesn't execute script either.

[bzbarsky]

Well, except for the part where you said it executes scripts. ;)

Anyway, for this particular case the real question is whether the non-Gecko browsers are willing to align with the spec. If not, I would like to see a concrete spec proposal that describes what their actual behavior is, assuming they actually have the same behavior.

[bzbarsky]

And unrelatedly, we should in fact just have some sort of streaming-parse-into-an-existing-document API that doesn't involve hacks. Is there an existing issue filed on that?

[jakearchibald]

I know it's something @domenic has had bouncing around his head. I'm hoping this iframe hack shows it could be easier than we initially thought.

I'd love to be able to do:

const div = document.createElement('div');
document.body.appendChild(div);
const response = await fetch(url);
await response.body.pipeTo(div.writable);

…or something.

[domenic]

I think the "concrete proposal" being discussed is to remove the line @zcorpan pointed out. That seems like the right thing to do unless we can get implementer commitment to remove this behavior in other engines. Hmm, @yuki3 maybe?

[bzbarsky]

Is just removing that line enough to deal with the issues Henri raised? As a simple example, has someone carefully stepped through the resulting logic and ensured that the parser being blocked and unblocked ends up being the same parser?

[bzbarsky]

And, importantly, has someone audited every use of "node document" around the <script> processing model to make sure it (1) makes sense and (2) matches the non-Gecko implementations if this restriction is removed?

[zcorpan]

If the script should be executed then this thread is reopened: https://lists.w3.org/Archives/Public/public-whatwg-archive/2010Sep/0030.html

[bzbarsky]

Ah, I had forgotten about that one. It's been a while.

I strongly urge everyone in this discussion to read that entire thread, which describes the incompatibilities in behavior at the time, the crashes this behavior caused in WebKit at the time, and the lack of clarity around interactions with CSP and whatnot....

Then please come back with an actual concrete proposal and tests that show what browsers actually do in the various interesting edge cases.

[jakearchibald]

@bzbarsky I've created https://github.com/whatwg/html/issues/2142 to discuss a less-hacky solution for streaming HTML into an element.

[hsivonen]

I think we should keep the spec as not executing scripts that have moved between docs during the parse. As the earlier-referenced W3C and Mozilla Bugzilla entries show, the spec text was arrived at as a result of implementor feedback resulting from first trying to do things the way that allowed the execution of moved scripts.

[domenic]

Note that Edge allows less than Chrome (and I assume Safari) here; in Jake's reduced test case, it only logs "Inline script", and not "Inline script" + "External script" as Chrome does.

I'm going to write tests against this behavior and hopefully Chrome can conform with the current spec as part of also fixing https://github.com/whatwg/html/pull/2673.

[domenic]

I think we're in a better position to tackle this issue now, as:

• https://github.com/whatwg/html/pull/5154 made explicit concepts for parser document and preparation-time document, which can be compared to each other and to the execution-time document.
• https://github.com/whatwg/html/pull/2673 added one mismatched document check that most everyone agreed on.
• https://github.com/web-platform-tests/wpt/pull/19632 added a framework for comprehensively testing cases of this sort.

In particular there are two pieces of the current spec that are signposted as "under debate":

• Prepare a script step 12 checks the parser document against the preparation-time document
• Execute a script block step 2 checks the parser document against the execution-time document.

@hiroshige-g, would you be able to extend your test generator to see whether browsers implement these checks or not? That seems like the first step.

[domfarolino]

@hiroshige-g, would you be able to extend your test generator to see whether browsers implement these checks or not? That seems like the first step.

Doesn't the test generator already support testing the points scenarios you raised @domenic?

• It looks like here we test that scripts are not executed when moved to a new document mid-parse
  
  
  
  • The results can be seen here, and it seems like Chrome is the only impl (I'm not counting Edge) that actually fails any before-prepare tests
  
  
• I think the "after-prepare" tests also cover the execution-document checks as well, especially since https://github.com/whatwg/html/pull/2673 cited https://github.com/web-platform-tests/wpt/pull/19632 as the relevant WPTs

[domenic]

I think you're right, @domfarolino! Thanks for looking into it.

it seems like Chrome is the only impl (I'm not counting Edge) that actually fails any before-prepare tests

Safari too, right?

[domfarolino]

Safari too, right?

That's correct, not sure why I thought otherwise.

[domfarolino]

Given the fact that Firefox already ships this, and the UseCounter for scripts that were moved between preparation and execution was very low in Chromium, I've gone ahead and shipped this in Chrome. I think we can close this issue now?

[domenic]

Excellent. I'll merge https://github.com/whatwg/html/pull/5575 then.