Gatsby: Paginating 11k articles

[nicgutierrez]

Hello everyone!

I'm working on a POC for our Blog and I have a question about how to approach pagination and although I found lots of info about it, I'm still unsure about the best way to handle this.

So, as the title says, I have a blog with over 11k articles. I'm aware on how to create pages like /posts/1, /posts/2 etc, and I'm also aware of the benefits of doing so. However, that's not what I'm after. Our current blog has infinite pagination on every single page where articles are posted: _Category page_, _Tag page_, _Author page_, _Home_ etc.

So, loading _all_ the articles in one go and slicing the array works, however this is quite slow and pages take a bit to load.

Does anyone have any suggestions on how to approach this? I saw somewhere about creating json files in gatsby-node and using those with XHR to load additional posts during window scroll. Is this good? Is there a better approach out there? I will need to do this for every page, so I will end up with _several_ json files.

Thanks!

[nimahkh]

Loading 11K article is not logical. the best way that I will do is :

• creating pagination from backend with offset and limits
• managing count of articles in the backend to handle the first page and last page (if you want to use "load more" you don't need this)

but if you want to load all of them in one request, you have to use visualizers like react-window or react-virtualized components. they are suitable for rendering large lists.

[nicgutierrez]

Hi @nimaa77 , thanks for getting back.

Maybe I forgot to mention a few things, sorry about that. I'm not loading 11k articles in one go. I'm using a paginated REST API and the gatsby-source-wordpress plugin.

I'm also not worried about in page performance, I will use react-window eventually so that's not the problem.

All the articles are available through a page query, but if I don't limit that, the page will take a bit to show because of how many articles I have. So at the moment I'm limiting my graphql to show only 20 articles, because I don't have a normal pagination with different URLs. Now, I'm going to implement infinite scroll and my question is more about what's the best approach to do that.

[nimaa77]

hello @nicgutierrez
I think you mentioned wrong person

you should call @nimahkh .

have nice day
Peace ✌️

[nimahkh]

Hi @nimaa77 , thanks for getting back.

Maybe I forgot to mention a few things, sorry about that. I'm not loading 11k articles in one go. I'm using a paginated REST API and the gatsby-source-wordpress plugin.

I'm also not worried about in page performance, I will use react-window eventually so that's not the problem.

All the articles are available through a page query, but if I don't limit that, the page will take a bit to show because of how many articles I have. So at the moment I'm limiting my graphql to show only 20 articles, because I don't have a normal pagination with different URLs. Now, I'm going to implement infinite scroll and my question is more about what's the best approach to do that.

You mean that what is the best approach to implement infinite scroll?
I think this starter is a good idea: https://www.gatsbyjs.org/starters/baobabKoodaa/gatsby-starter-infinite-scroll/
The main problem is that you will list 11K articles at the end. I know that these articles will show after paginate, but you will have a big size static page. the best approach is to use garbage collector in javascript.

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[github-actions[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜