Gatsby-source-wordpress-experimental: Caching plugin breaks pagination: "On first run gatsby develop is fetching Comment nodes indefinitely"

[benknight]

Here's my console output:

▶ GATSBY_CONCURRENT_DOWNLOAD=8 npm start

> vietnamcoracle-static@ start /Users/ben/Code/vietnamcoracle-static
> npm run develop


> vietnamcoracle-static@ develop /Users/ben/Code/vietnamcoracle-static
> gatsby develop

Debugger listening on ws://127.0.0.1:9229/4ff4b552-4816-4e15-94e5-820d9ae11e18
For help, see: https://nodejs.org/en/docs/inspector
success open and validate gatsby-configs - 0.042s
success load plugins - 0.652s
success onPreInit - 0.011s
success initialize cache - 0.017s
success copy gatsby files - 0.094s
success onPreBootstrap - 1.577s
⠀
info  gatsby-source-wordpress

    This is either your first build or the cache was cleared.
    Please wait while your WordPress data is synced to your Gatsby cache.

    Maybe now's a good time to get up and stretch? :D

success  gatsby-source-wordpress  ingest WPGraphQL schema - 1.428s
success createSchemaCustomization - 2.463s
success  gatsby-source-wordpress  PostFormat - 0.848s - fetched 2
success  gatsby-source-wordpress  MenuItem - 0.951s - fetched 73
success  gatsby-source-wordpress  Menu - 0.976s - fetched 1
success  gatsby-source-wordpress  Category - 1.057s - fetched 7
success  gatsby-source-wordpress  ContentType - 1.157s - fetched 3
success  gatsby-source-wordpress  Page - 1.215s - fetched 11
not finished source and transform nodes - 4925.138s
not finished  gatsby-source-wordpress  fetching nodes - 4925.091s - 635297 total
not finished  gatsby-source-wordpress  Comment - 4925.085s - fetched 635200

I let the script run for about 80 minutes before exitting it. It fetched over half a million comment nodes. Yet my database only has around 6,593 records in the wp_comments table.

How should I go about debugging this?

[TylerBarnes]

Hi @benknight , thanks for opening this issue. Usually this is due to some plugins on the WP side that are breaking WPGraphQL pagination, but it may also be due to a bug in the source plugin. Can you share your /graphql endpoint url with me via email so I can take a look? My email is [email protected]

Thanks!

[benknight]

I'm still having issues here. Is there a recommended way to debug network requests? I've tried things like Charles Proxy and the Node debugger with no luck.

[benknight]

I was able to determine that this was being caused by caching being done by the SG Optimizer plugin. Basically the server was returning the same cached page of nodes result regardless of pagination variables.

[TylerBarnes]

Ah, interesting 🤔 Seems like this plugin should protect against that. To repro do I just install SG Optimizer and run a build?

[benknight]

@TylerBarnes yes I believe that's all that necessary to reproduce it.

When I tested, I was using console.log debug the GraphQL request/response and I observed that the same "endCursor" string was being returned for every request, regardless of the GraphQL request variables.

I then went to the SG Optimizer settings and turned off dynamic-caching:

After I did this I observed the correct endCursor being returned for each consecutive node list query.

[TylerBarnes]

This is harder to reproduce than I expected. Looks like SG optimizer will only run on siteground sites. If anyone else runs into this please let me know how to reproduce and I'll reopen this! Other caching plugins don't seem to be causing this.