Netlify-cms: Post limits?

[HackGibson]

Is there a limit on how many total posts NetlifyCMS can handle? The content/blogs folder in my Github repo can only handle displaying 1000 files. So does this mean NetlifyCMS can only allow me to access the last 1000 posts?

[erquhart]

We're only limited because we're using GitHub's Contents API instead of their Git Data API for fetching collections. Thanks for opening this, folks are running into it a lot lately.

[wildfalcon]

I ran into this problem too, so +1

I was able to work round it by putting my content into several collections. I'm lucky that this makes sense with my content.

I also have the same issue with the media library, max 1000 images - which I assume is the same root cause

[wing5wong]

+1 on this - just ran into it

good to see an already open issue for it

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[neilkinnish]

Has anyone worked around this issue?

[barthc]

@neilkinnish you can try enabling the graphql option which uses the tree object to list entries, limit should be more than 1k. @erezrokah, please confirm.

[erezrokah]

Good thinking @barthc. We do list the files using the tree object, but we still use the REST API for some other operations (mostly mutations).

Running this query in GitHub GraphQL explorer:

query { 
  repository(owner: "erezrokah", name:"gatsby-netlify-cms-aws-test") { 
    id
    object(expression: "test/many_files:content/posts") {
      id
      ...on Tree {
        entries {
          name
        }
      }
    }
  }
}

Does return 2000+ entries I created for this test.
It's the same query we use to list files when using GraphQL.

@neilkinnish do you mind changing to GraphQL to see if that works?

[barthc]

Nice, according to @erquhart here limit should be 100k.

[erezrokah]

Haven't tested it, but the GraphQL limits seem to go by nodes:
https://developer.github.com/v4/guides/resource-limitations/

So maybe even 500k :)

[neilkinnish]

@barthc this works great! thanks.