Kolibri: diff stats are sometimes blank

[indirectlylit]

Observed behavior

note that diff stats are blank:

Expected behavior

diff stats always show

User-facing consequences

no information in UI

Errors and logs

no errors in JS or Python

Steps to reproduce

unknown. observed intermittently - refreshing made the numbers appear:

Context

0.13.0 beta 3

[jonboiser]

This seems like an edge case, that seems to happen when too many identical channeldiffstats requests (i.e. same channel and from same source) are triggered very quickly in a way that the client can't cancel or clean them up quickly enough. You can sort of reproduce this by going back and forth between the Manage Channel Page and this page very quickly and wait until this kind of console error appears:

  File "/Users/jon/.local/share/virtualenvs/kolibri-UH8N_Uf0/lib/python3.7/site-packages/sqlalchemy/engine/default.py", line 509, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) table content_contentnode_related already exists [SQL: '\nCREATE TABLE content_contentnode_related (\n\tid INTEGER NOT NULL, \n\tfrom_contentnode_id CHAR(32) NOT NULL, \n\tto_contentnode_id CHAR(32) NOT NULL, \n\tPRIMARY KEY (id), \n\tFOREIGN KEY(to_contentnode_id) REFERENCES content_contentnode (id), \n\tFOREIGN KEY(from_contentnode_id) REFERENCES content_contentnode (id)\n)\n\n'] (Background on this error at: http://sqlalche.me/e/e3q8)

From the Rest API, the response will be a FAILED task, so I guess we can show an AppError page in that case, which will prompt the user to refresh the page and hopefully get what they need.

[indirectlylit]

too many requests

This may be one way to trigger it, but during bug bash people were definitely seeing it on the "first try", and that's what I experienced also.

[jonboiser]

I wonder if it's because there were many concurrent connections during the bug bush. I.e. me triggering many channeldiffstats task on my computer by going back in forth is the same as multiple users trigger channeldiffstats on the server (esp. if they are checking the same channel).

[indirectlylit]

Here's a video which shows the behavior I sometimes see. From what I can tell, the diffstats API call doesn't fail - it's actually never triggered:

https://drive.google.com/file/d/1n1E0VgFDyB1GrfYGOycL_VD6XwuiZP2s/view?usp=sharing

[jonboiser]

My suspicion now is on this line

https://github.com/learningequality/kolibri/blob/fa38d6ddfb34115d2a7e24c681876a97955af213/kolibri/plugins/device/assets/src/views/ManageContentPage/api.js#L101

The filter for find isn't specific enough since IMPORT and EXPORT tasks also have the same method and channel_id fields. Need to also filter for the type = CHANNELDIFFSTATS

[jonboiser]

The fix in #6300 tries to mitigate one source of this by retrying the CHANNELDIFFSTATS command in case it fails.

There's another fix in #6355 that fixes an oversight when filtering for these tasks to implement some short-circuit logic.

I think between these two, fixes, all the causes I'm aware of for this bug are fixed. 🤞