Jsdom: Implement Blob.stream, Blob.text and Blob.arrayBuffer

[jimmywarting]

• https://bugs.chromium.org/p/chromium/issues/detail?id=945893#c1
• https://chromestatus.com/feature/6206980857266176
• w3c/FileAPI#117
• bitinn/node-fetch#604
• bitinn/node-fetch#392

We over at node-fetch has a response.blob() method that allows you to get a blob like object, it's currently useless cuz you can't read it in any way except from doing:

new Response(blob).arrayBuffer()

We are planing on adding Blob.text() and Blob.arrayBuffer() at least, not sure about blob.stream() yet if it's going to be a node stream or a web stream
we do not currently expose the constructor in any way but we would like to make this compatible with your FileReader. If you changed the FileReader api to read the content of the file using this methods you could read any blob/file like object not only coming from your own jsdom implementation.

fr.readAsBuffer() --> blob.arrayBuffer().then(dispatch)
fr.readAsText() --> blob.text().then(dispatch)
fr.readAsDataURL() --> blob.arrayBuffer().then(buffer2dataURL).then(dispatch)

[domenic]

I'm not sure what the exact ask is here, but I'll note that jsdom, like browsers, reads the contents of things using internal APIs, so that you can't fool it with "non web platform" objects (i.e. non-jsdom objects). This is by design.

[jimmywarting]

There is now new ways to to read blob/files using a promise based api that exist as a method on the Blob class, mainly Blob.prototype.text and Blob.prototype.arrayBuffer when called both returns a promise. this is what i want you to implement

[jimmywarting]

What I'm also asking for is:
That you change the way the FileReader how it reads the content. Using the new "internal public APIs" on the blob itself instead of reading it from blob._buffer

[domenic]

These are not internal APIs; they are public APIs. So this will not generally work, just like what you're asking for won't work in browsers.

[jimmywarting]

ups, meant public apis

[domenic]

Yeah, that would be going against the spec, so it's not an option for us.

[jimmywarting]

Blob.stream, Blob.text and Blob.arrayBuffer are speced, not just implemented anywhere yet

[domenic]

Right, but if you override them to throw an exception, then FileReader will still work (per spec), because FileReader uses internal data access, not the public APIs.

Your request is that if you override them to throw an exception, FileReader will call those public APIs and fail. We are not interested in violating the spec in that way.

[jimmywarting]

okey, so apart from my second request about changing the FileReader to use the public blob api. Would you consider adding the new public blob methods that allows you to get things with a promise?

[domenic]

For sure. Pull request welcome!

[jimmywarting]

Actually, when i read the readAsText spec now it says

1. If fr ’s state is "loading" , throw an InvalidStateError DOMException .
   
   
   
   1. Set fr ’s state to "loading" .
   
   
   2. Set fr ’s result to null .
   
   
   3. Set fr ’s error to null .
   
   
   4. Let stream be the result of calling get stream on blob .
   
   
   5. Let reader be the result of getting a reader from stream .
   
   
   6. ...
   
   
   7. ...
   
   

so it states that the FileReader should read the content of the blob using the Public API's
not some internal things?

[domenic]

get stream links to https://pr-preview.s3.amazonaws.com/w3c/FileAPI/117/795750f...42f8a45.html#blob-get-stream, not to https://pr-preview.s3.amazonaws.com/w3c/FileAPI/117/795750f...42f8a45.html#stream-method-algo

[jimmywarting]

thought https://pr-preview.s3.amazonaws.com/w3c/FileAPI/117/795750f...42f8a45.html#blob-get-stream was the same thing as calling https://pr-preview.s3.amazonaws.com/w3c/FileAPI/117/795750f...42f8a45.html#stream-method-algo

That the "A Blob blob has an associated get stream algorithm" which was the same thing as calling blob.stream()

[domenic]

Nope, #blob-get-stream is private, which two public methods stream() and readAsText() both call. Pretty standard pattern for factoring out "private methods" in spec text.

[jimmywarting]

oh well, still think you should use the public method to read the blob... would help some folks that use node-fetch that calls response.blob() to get a jsdom-like object - not sure why they do it if they are going to read it anyway with with fr.readAsWhatever(), could as well call response.arrayBuffer() directly or something like that

but if you are so strict, so be it.
I'm not using jsdom or any combination of both node-fetch + jsdom. So it's not my problem

[Gozala]

Any chances Blob implementation could be extracted into separate library so both libraries could be made interoperable ? Right now both libraries have a strong opinions and as result they can't interop and combination of two is pretty much unusable.

[Gozala]

@domenic I think @jimmywarting is proposing very specific solution, although actual goal here is to make two libraries interoperable. Maybe we can shift discussion towards how two libraries can interop ?

Specific issue I've being running into in the context of https://github.com/WebMemex/freeze-dry/pull/44 is that project uses node-fetch and jsdom in nodejs where things seems to misbehave because blob instanceof Blob is false when one comes from jsdom and other from node-fetch because each bundles own implementation of Blob.

I believe what @jimmywarting was hoping is for both libraries to migrate from checks like blob instanceof Blob to blob.arrayBuffer(), which would resolve incompatibility issues.

@domenic would you please collaborate with us on a solution to this interop issues which would not require node-fetch to depend on jsdom ?

[Sebmaster]

would you please collaborate with us on a solution to this interop issues which would not require node-fetch to depend on jsdom ?

The problem here is that you include node-fetch at all. If your scripts ran in jsdom, your isomorphic libs should act as if they're in a browser (that is: detect the presence of XmlHttpRequest or similar) and get the "browser's" (jsdom's) native Blob implementation.

[Gozala]

The problem here is that you include node-fetch at all. If your scripts ran in jsdom, your isomorphic libs should act as if they're in a browser (that is: detect the presence of XmlHttpRequest or similar) and get the "browser's" (jsdom's) native Blob implementation.

I am not sure how did you arrive to that conclusion, but I respectfully disagree. Use only JSDOM or no JSDOM at all isn’t a great solution for the community, there are & will continue to be use cases that don’t fit this binary proposition

P.S.: My scripts aren’t run by JSDOM, it is used mostly to represent document in DOM API compatible API.

[Sebmaster]

Use only JSDOM or no JSDOM at all isn’t a great solution for the community

Not what I said.

there are & will continue to be use cases that don’t fit this binary proposition

Please do elaborate with specifics. It usually makes no sense to have a hybrid environment for testing, since that means you're not actually testing in any reasonably expected environment.

P.S.: My scripts aren’t run by JSDOM, it is used mostly to represent document in DOM API compatible API.

jest usually runs tests in jsdom by default unless you explicitly disable it. If you haven't configured that, the problem is in the detection of a "node environment" in the fetch implementation. It should detect that it's running in a browser.

If you _did_ explicitly configure it otherwise, then you're mixing browser and non-browser environments which is something we explicitly recommend against (the pattern doesn't quite apply, but the results are the same) because of exactly these issues.

[domenic]

I agree overall with @Sebmaster that you should be running your scripts inside the jsdom, and that using node-fetch (instead of the whatwg-fetch polyfill) is probably the core of the problem.

Stated another way, jsdom emulates a web browser---and that's a large project, enough to keep us quite busy. It seems like some folks in this issue thread are asking us to expand the project's scope beyond "just" emulating a web browser, to also emulate Electron, i.e. have mechanisms for bridging between Node-level primitives and the "web browser environment" jsdom provides. That is, whatwg-fetch runs in a browser; node-fetch can only run in a browser if that browser also grows all the Node-level capabilities that Electron has.

That's not really feasible for the project as it is currently scoped or envisioned. It may be an interesting project for someone else to pursue on top of jsdom, similar to how Electron is built on top of Chromium.