Fetch: Question — Why can I not read the stream again ?

[tusharmath]

Once I have done fetch('/url').then(x => x.json()) If I do call the json() again, why does it have to throw — TypeError Already Read?

Reference to fetch implementation — https://github.com/github/fetch/blob/4db9f7b4519d85ec19fee540f5467db1569e80f6/fetch.js#L212

[tyoshino]

The Fetch API and Streams API used in it are designed to allow efficient data processing. Calling json(), etc. progressively consumes data from the Response object so that buffering at intermediate objects can be kept small.

Suppose that an app fetches 1MB HTTP response and parse it as a JSON file and then use it, but for nothing else (i.e. once the JSON is generated, the response would be never touched). If the Response is designed to hold the original response body data even after .json() call is made on it, even if we made the response object unreachable from the root so that GC collects it asap, the peak memory foot print for the processing reaches 1MB+1MB (assuming the resulting object is 1MB for simplicity).

If we design the API to allow releasing the parsed data immediately as the resulting object is built, the foot print can be kept around (small buffer)+1MB.

Only those who really want to reuse the Response body should use clone() to manually duplicate the object.

[annevk]

@tusharmath was there something in the Fetch standard that made you believe this should work differently? Can we clarify this somewhere?

[tusharmath]

@annevk Well I can work with this approach as well and I agree with what @tyoshino has explained above. The API wasn't intuitive to me at the same time very difficult for me to explain why. Let me still try to give it a shot —

Intuitively all this time we have been used to thinking that if we have access to the response, we will have access to the body also. And suddenly here I can only access the body once! Which is fine and I might be able to get used to it, but there is a better way I think.

I would have preferred if the body was exposed as a hot observable. I would have been able to reduce the stream to a real JSON, then eventually convert it to a promise (or not).
The advantage of this approach is that I know for a fact that the response body stream will not be available to me for later use as its a Hot Observable.

I can do everything with the current set of APIs also, but this just seems more intuitive to me. Thoughts?

[annevk]

The body will eventually be exposed as a stream everywhere: https://github.com/yutakahirano/fetch-with-streams. The current approach is mostly a shortcut so we could get the API out faster. Closing this as it does not seem this needs changes to the standard.

[jakearchibald]

@tusharmath

Intuitively all this time we have been used to thinking that if we have access to the response, we will have access to the body also. And suddenly here I can only access the body once!

This isn't quite true. If you make an XHR request with responseType "json" you cannot later get the response as text without making another request.

With response.json() you're asking the body to be read as JSON. If you want the JSON multiple times you can store it in a variable/constant…

fetch(url).then(r => r.json()).then(data => {
  console.log(data); // this works
  console.log(data); // and this works
});

…or store the promise in a variable/constant…

const jsonPromise = fetch(url).then(r => r.json());
jsonPromise.then(data => console.log(data)); // this works
jsonPromise.then(data => console.log(data)); // and this works

I would have preferred if the body was exposed as a hot observable.

As @annevk says, this is the plan. You'll still only be able to read that stream once though (unless you clone the response or tee the stream).

Here's an example using streams to search a response https://jsbin.com/tepohi/edit?js,console

[tusharmath]

@jakearchibald This is awesome. Thank you all for being so helpful.

[AdrianZghibarta]

@jakearchibald This doesn't solve the original problem, for exemple :

const responsePromise = fetch(NetworkingUrls.authUrl, requestData);
    responsePromise.then((response) => NetworkingManager.logRequestInfo(response, requestData));
    responsePromise.then((response) => NetworkingManager.checkStatus(response, requestData))
    .then((response) => response.json())
    .then(NetworkingManager.checkJson)
    .then((responseJson) => {
      successHandler(responseJson);
    })
    .catch((error) => {
      failureHandler(error.message);
    })

In that case you will get the same error.

[n4gashi]

Here's a way that you can work around. Basically, you just build off an object containing response and body to next callbacks. This way you just read the Stream once, and you keep both variables in your scope. I do not know much about performance costs of this implementation, but so far, it never failed me 👍

class Api {

  constructor() {
      this.domain = 'http://localhost:5000';
  }

  fetch({url, body, headers = {}, onResponse = () => {}, onSuccess = () => {},
    onError = () => {}, authenticated = false}) {

    if(authenticated === true){
      headers["Authorization"] =
        "some authorization token here";
    }

    return fetch(this.domain.concat(url), { body: body, headers: headers })
    .then((response) => {
      return response.json().then((body) => {
        return { response, body };
      })
      .then((request) => {
        this.checkAuthentication(request, authenticated);
        return request;
      })
      .then((request) => {
        this.handleCallbacks(request, onResponse, onSuccess, onError);
        return request;
      });
    });
  }

  checkAuthentication(request, authenticated){
    if(authenticated === true && request.response.status === 401){
      // STUB
    }
  }

  handleCallbacks(request, onResponse, onSuccess, onError){
      onResponse(request.response, request.body);
      (request.response.status < 200 || request.response.status >= 300) ?
       onError(request.response, body):
       onSuccess(request.response, body);
  }

}

export default Api;

As you can see you can access both response and body from completely external methods. Anonymous functions to pass parameters out to external methods may seem a bit overkill at first, but haven't found a better solution atm.

Hope this helps :)

[Offirmo]

For fellow googlers:

• TL;DR https://developer.mozilla.org/en-US/docs/Web/API/Response/clone
• long version: read @tyoshino 's answer

[pencilcheck]

I used JSON.parse(response.resp.data) instead as clone didn't work with RNFetchBlob in react native.