Pnpjs: GetAll() doesn't work with filter and with big data list

[enti333]

Category

• [ ] Enhancement
• [ X] Bug
• [ ] Question
• [ ] Documentation gap/issue

Hello. I am working with big list (About 10 000items) and the problem is that it is not working when I want to get some data from this big list...
I have checked this post: https://github.com/SharePoint/PnP-JS-Core/issues/689
But my query doesn't work.

pnp.sp.web.lists.getByTitle("BigList").items.select("Title, Id").filter("substringof('1',Title)").top(50).getAll().then(function (dataF) {

I get this error:

HTTP500: SERVER ERROR - The server encountered an unexpected condition that prevented it from fulfilling the request.
(Fetch)GET - https://neasoft.sharepoint.com/sites/InvoiceArchival/_api/web/lists/getByTitle(BigList)/items?$top=50&$select=Title, Id&$filter=substringof('1',Title)

I have tested many other variations for example

pnp.sp.web.lists.getByTitle("BigList").items.select("Title, Id").filter("substringof('1',Title)").getAll(50).then(function (dataF) {

or for example

pnp.sp.web.lists.getByTitle("BigList").getItemsByCAMLQuery({ ViewXml: "<View><ViewFields><FieldRef Name='ID' /><FieldRef Name='Title' /></ViewFields><Query><OrderBy><FieldRef Name='Title' /></OrderBy>" + "<Where><Contains><FieldRef Name='Title'/><Value Type='Text'>" + param + "</Value></Contains></Where></Query><RowLimit>50</RowLimit></View>" }).then(function (dataF) {

HTTP500: SERVER ERROR - The server encountered an unexpected condition that prevented it from fulfilling the request.
(Fetch)POST - https://neasoft.sharepoint.com/sites/InvoiceArchival/_api/web/lists/getByTitle('BigList)/getitems

Still same error...I have standard list with 10 000 items in the list....

Only this is working

pnp.sp.web.lists.getByTitle("BigList").items.select("Title, Id").top(50).getAll().then(function (dataF) {

this is not filtering my data....

Thank you very much

[koltyakov]

It's by design and can't be fixed in any way. getAll is a helper to organize a sequential recursion with getPaged/getNext, but won't work in situations when API blocks requests due to the threshold limitations.
To use getAll on large lists no filter and orderBy should be provided if they brake large lists requesting.
To filter on a large list, the filtering column should be indexed and after applying the first filtering condition the number of items should be less than throttling limitation.

[enti333]

Hello Andrew. Thank you very much for your answer. So GettAll I can not user for that... But what about the caml query....Hmm...Why this doesn't work> In SSOM this was working I think.... If there was a big list and I use row limit and it was working... But in this scenario not....Maybe I am not right.
Andrew so only chance to get this work is to create Index on my filed and use maybe at least two chars for filtering so the result doesn't return more then 5000 items? Again thank you for your help.

[koltyakov]

With CAML methods all the limitations are the same, always were for client-side APIs.
For indexed columns filtering, if I'm not wrong, you should also be aware that not every comparison works as indexed filtering.

[patrick-rodgers]

Going to close this as answered. As outlined by @koltyakov the method is working as expected. If you need to continue the conversation please _reopen_ the issue. Thanks!