Uwazi: Forcing the query limit to more than the hardcoded 10.000k crashes the request

[txau]

Forcing a query limit form 60 to ie. 60000 leaves the server unresponsive. (crashes the request)

[RafaPolit]

This has been proven not to be the case. It just stalls in answering and gives the 'wrong' maintenance screen. It happens at the exact 10,000 to 10,001 threshold. If you ask for limit:10001, it stalls and presents the maintenance screen. Still, the server (and that instance) remains responsive, and others can do normal queries without delay or the node loop being blocked.

[simonfossom]

Connected to https://github.com/huridocs/uwazi/issues/2708

[kjantin]

I propose we close this issue, and leave this one open: https://github.com/huridocs/uwazi/issues/2708

[txau]

I propose we close this issue, and leave this one open: #2708

This is a bug that crashes the request. Is not the same as #2708.

[txau]

The request wasn't crashing. The server was simply not handling the request until it timed out. Now if users try to access this URL by reloading, instead of a UI error, they will receive a plain HTML error message.

[simonfossom]

I don't see how we solved this 🤔
@daneryl How does the system work now? What did you change?

[txau]

This problem only happened under these conditions:

• A user manually tampers with the limit in the URL setting a limit over 10.000 results, then
• a user forces a reload on the single page application

Now this will trigger a server side rendering error. Previously it would just timeout. It has never been a bug.

[RafaPolit]

I think the name of this issue is somewhat misleading. The fix is that querying for 10.001 no longer crashes when loading the page.

We have not yet fixed the issue of actually returning more than 10.000 documents.

Still, we should not ever do that. We need to implement proper pagination and server replies that report that “there are more records” so you can program several queries to fetch all the records.

Getting more than 10.000 is not the way to go.

Just to clarify what was fixed here.

[simonfossom]

@RafaPolit could we create a ticket (that we don't have to prioritize anytime soon), that would address the underlying problem in the fullest?

[RafaPolit]

Addressing #2708 probably fixes this. I have added a small note there to take that scenario into account.