Gatsby: Infinite Scroll

[thekevinbrown]

Description

Unclear how to implement infinite scroll in Gatsby without loading all data for all pages on initial page load.

Steps to reproduce

The only example I could find was Gatsbygram, but it loads all data, then uses state to show parts of the data. What if we have a large amount of data? How do we load pages and show them?

Expected result

We should be able to implement infinite scrolling on Gatsby sites.

Actual result

I'm not sure we can because the components must be bound up to all of their data at build time.

Environment

• Gatsby version (npm list gatsby): [email protected]
• gatsby-cli version (gatsby --version): 1.1.50
• Node.js version: v8.9.4
• Operating System: OS X 10.13.3

Question

Has anyone had any success doing this without loading all data with the page?

[KyleAMathews]

Something I've thought about in regards to this use case is that you'd be able to mark parts of a query as "deferred" — that you could load on demand. Similar to dynamic imports for JS modules. You'd be given a URL to all the data chunks you specify. Then you could lazy load in additional data as necessary.

[thekevinbrown]

@KyleAMathews that'd be amazing!

[DylanVann]

For users with JS disabled you'd want normal pagination. People have come up with a bunch of ways to do this with Gatsby. So you should have a bunch of pages generated anyways. If there was someway to get the url to the json data for those pages then it would be pretty easy to fetch it and merge it in.

If this could query the path the the output json:

None of the existing info allows finding the final path.

Deferred queries is probably a cleaner solution though.

---

Edit:

I eventually got this to work but had to fork Gatsby to expose getPageQueryData in packages/gatsby/cache-dir/socketio.js to get it to work in dev mode.

// packages/gatsby/cache-dir/socketio.js
window.___getStaticQueryData = getStaticQueryData
window.___getPageQueryData = getPageQueryData

/**
 * This gets data out of Gatsby's cache.
 * It's a hack and it relies on a fork of Gatsby.
 *
 * window.___getPageQueryData is not normally exposed.
 */
export const fetchData = path =>
  new Promise(resolve => {
    const loader = window.___loader
    const getPageQueryData = window.___getPageQueryData
    const emitter = window.___emitter
    loader.getResourcesForPathname(path, data => {
      if (data.json) {
        resolve(data.json)
      }
      const pageData = getPageQueryData()
      if (pageData[path]) resolve(pageData[path])
      const handler = () => {
        const pageData = getPageQueryData()
        if (pageData[path]) {
          emitter.off(handler)
          resolve(pageData[path])
        }
      }
      emitter.on('*', handler)
    })
  })

Using this I can use await fetchData('/2') to get the next pages data (using gatsby-paginate to build the pages. Full source is here.

Hopefully in the future there will be a cleaner solution for both general infinite scroll / deferred queries and for getting the data from a given page.

[gatsbot[bot]]

Old issues will be closed after 30 days of inactivity. This issue has been quiet for 20 days and is being marked as stale. Reply here or add the label "not stale" to keep this issue open!

[gatsbot[bot]]

This issue is being closed due to inactivity. Is this a mistake? Please re-open this issue or create a new issue.

[baobabKoodaa]

@DylanVann @thekevinbrown The solution is to generate your own JSON that corresponds to the pages created by your pagination. I made a starter and a demo: https://github.com/baobabKoodaa/gatsby-starter-infinite-scroll

[ggarcia92]

@baobabKoodaa Your starter is very useful but dosen't solve the problem of @thekevinbrown . Your starter also works with all data and he wants fetch data according to the page, to avoid getting a large amount of data, something like react-infinite-scroll-component but using Gatsby.

[baobabKoodaa]

@ggarcia92 You are mistaken. My starter doesn't load all data for all pages on initial page load. On initial page load it only loads the data corresponding to the initial page. As the user scrolls down, it fetches more data as needed. So it solves exactly the problem described by @thekevinbrown .

[ggarcia92]

@baobabKoodaa In your gatsby-node.js you use a variable images that contains all images to display, and @thekevinbrown wants load server data according to the scrolling to avoid a big response data from the server or I understood his issue in wrong way?

[baobabKoodaa]

@ggarcia92 In my gatsby-node.js there is indeed a variable images which contains all images, but it is never sent to the client. It only exists during build when we create pages on the server. When we create pages, each page is only given "one page worth of images", rather than all of the images. So when a client requests a page, they only fetch the data they need for that page. You can easily confirm this yourself by opening the demo with devtools and looking at the contents of the resources which are sent on initial pageload.