Gatsby: Calling __refresh while refreshing

[jupl]

Description

Our team is leveraging the Gatsby refresh endpoint (ENABLE_GATSBY_REFRESH_ENDPOINT) for development purposes. The amount of time to build/refresh has increased over time. We are wanting to use a service that will make a POST call to __refresh on demand. The problem that we are encountering is that if you make the POST call in the middle of a refresh then unexpected behavior starts to occur and eventually the dev server crashes. It would be nice to have some way to either offer some kind of throttling or at least have a way to know if we are in the middle of a refresh. (with the latter at least a custom endpoint can be created)

Our understanding was that it did not behave like this before (2.16.5) and if this were to occur Gatsby would wait until the existing refresh would finish.

Steps to reproduce

Make a POST call to __refresh while it's refreshing.

Expected result

Expected behavior.

Actual result

Query failures, crashes, etc.

Environment

(This also occurs in Windows)

EDIT: Updated gatsby to 2.18.21 and it is still an issue.

  System:
    OS: macOS Mojave 10.14.6
    CPU: (8) x64 Intel(R) Core(TM) i7-3615QM CPU @ 2.30GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 12.12.0 - /usr/local/bin/node
    Yarn: 1.19.1 - /usr/local/bin/yarn
    npm: 6.11.3 - /usr/local/bin/npm
  Languages:
    Python: 2.7.16 - /usr/local/bin/python
  Browsers:
    Chrome: 79.0.3945.117
    Firefox: 72.0
    Safari: 13.0.4
  npmPackages:
    gatsby: ^2.18.15 => 2.18.15
    gatsby-background-image: ^0.9.11 => 0.9.11
    gatsby-image: ^2.2.4 => 2.2.36
    gatsby-plugin-lodash: ^3.1.2 => 3.1.18
    gatsby-plugin-manifest: ^2.2.1 => 2.2.33
    gatsby-plugin-polyfill-io: ^1.1.0 => 1.1.0
    gatsby-plugin-prefetch-google-fonts: ^1.4.3 => 1.4.3
    gatsby-plugin-react-helmet: ^3.0.12 => 3.1.18
    gatsby-plugin-remove-serviceworker: ^1.0.0 => 1.0.0
    gatsby-plugin-robots-txt: ^1.5.0 => 1.5.0
    gatsby-plugin-sass: ^2.1.12 => 2.1.26
    gatsby-plugin-sharp: ^2.2.3 => 2.3.7
    gatsby-plugin-sitemap: ^2.2.9 => 2.2.24
    gatsby-plugin-styled-components: ^3.0.7 => 3.1.16
    gatsby-plugin-typescript: ^2.0.15 => 2.1.22
    gatsby-plugin-webpack-bundle-analyser-v2: ^1.1.5 => 1.1.8
    gatsby-source-filesystem: ^2.1.2 => 2.1.42
    gatsby-transformer-sharp: ^2.2.1 => 2.3.9
  npmGlobalPackages:
    gatsby-cli: 2.6.7

[sidharthachatterjee]

This is interesting. I suppose the correct fix here would be to queue refresh calls and only run the next one once the queue is empty?

@pieh Thoughts on this?

[pieh]

Yes! Let's implement queue for that! There also other places I wanted simple queue like that (particularly our redux state persistence which is now just debounced)

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜

[elweab]

@pieh Hey I'm wondering if there has been any progress on this issue?

[pieh]

There wasn't any progress yet on this, but it's up for the taking.

The code for refresh endpoint is in https://github.com/gatsbyjs/gatsby/blob/c3eccce997585d6136bdd0289472c602631681cf/packages/gatsby/src/commands/develop.ts#L199-L230

What we need is to make sure that calling refresh is delayed if there is already one in progress. There might be some trickiness with passing webhookBody - if there are 2 refresh requests with webhookBody we would need to make sure we actually call sourceNodes with both bodies serially

[xavivars]

Even a simple "reject" would be better that the current behavior...

[apaniel90vp]

@pieh hope #24887 fits what you had in mind for this fix