Gatsby: How to deal with pagination and gatsby-source-graphql?

[marcaaron]

I'd like to use gatsby-source-graphql to pull repository information from the Github GraphQL API for use with gatsby. Only problem is the Github GraphQL API places a limit of 100 on how many list items are returned from list fields e.g. PullRequests, Comments, etc. Is there any existing strategy for queries that are limited by the number of results they can return?

_Example:_

1. get the last 100 pull requests in a repository
2. using the cursor returned by the previous query get the last 100 starting from the cursor
3. and so on until there are no PRs left to fetch
4. use gatsby to query the entire data set without such limit restrictions set

I'm thinking another strategy would be to just set up a node server to run a script that performs a similar process (write to JSON or something) and then pass that into gatsby. But wondering if there's a built in way?

[rrathbone]

Hi marcaaron. Were you able to find a solution to this problem? I just ran into the same issue.

[freiksenet]

Hi! There isn't any existing strategy for that. From the size limited result, you should check if there is next page and then continue until there is no next page. For example with github, assuming you are using gatsby-source-graphql.

const QUERY = `    
  query($after: String) {
      github {
        viewer {
      repositories(first:100, after: $after) {
            nodes {
              id
            }
            pageInfo {
             endCursor
             hasNextPage
           }
        }
      }
    }`

let { data } = await graphql(QUERY)
const repos = data.viewer.repositories.nodes
while (data.viewer.repositories.pageInfo.hasNextPage) {
  data = await graphql(QUERY, {
    after: data.viewer.repositories.pageInfo.endCursor
  })
  repos.push(...data.viewer.repositories.nodes)
}

// create pages

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

Thanks again for being part of the Gatsby community!

[eric-hc]

Does anyone know if there is a more elegant solution to this yet?

[timomayer]

We are also searching for a solution for that