Android: UploadsStorageManager: Couldn't read row 1872, col 1 from CursorWindow.

[andreapx]

Steps to reproduce

1. Open the app

Actual behaviour

• The app crashes

Environment data

Android version: 9

Device model: OnePlus 6T

Stock or customized system: Stock

Nextcloud app version: 30110090

Nextcloud server version: 18.0.3

Reverse proxy: nginx

*** CAUSE OF ERROR ***

java.lang.IllegalStateException: Couldn't read row 1872, col 1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
at android.database.CursorWindow.nativeGetString(Native Method)
at android.database.CursorWindow.getString(CursorWindow.java:465)
at android.database.AbstractWindowedCursor.getString(AbstractWindowedCursor.java:51)
at android.database.CursorWrapper.getString(CursorWrapper.java:137)
at com.owncloud.android.datamodel.UploadsStorageManager.createOCUploadFromCursor(UploadsStorageManager.java:321)
at com.owncloud.android.datamodel.UploadsStorageManager.getUploads(UploadsStorageManager.java:302)
at com.owncloud.android.datamodel.UploadsStorageManager.getFailedUploads(UploadsStorageManager.java:368)
at com.owncloud.android.files.services.FileUploader.retryFailedUploads(FileUploader.java:1005)
at com.owncloud.android.utils.FilesSyncHelper.lambda$restartJobsIfNeeded$0(FilesSyncHelper.java:222)
at com.owncloud.android.utils.-$$Lambda$FilesSyncHelper$fa2LBpPOxRVQOm4F_Glue5u4hdM.run(Unknown Source:10)
at java.lang.Thread.run(Thread.java:764)

*** APP INFORMATION ***
ID: com.nextcloud.client
Version: 30110090
Build flavor: generic

*** DEVICE INFORMATION ***
Brand: OnePlus
Device: OnePlus6T
Model: ONEPLUS A6013
Id: PKQ1.180716.001
Product: OnePlus6T

*** FIRMWARE ***
SDK: 28
Release: 9
Incremental: 1909112330

[tobiasKaminsky]

When did this happen?

[andreapx]

A couple of days ago when I tried to open it.
Now that I think about it, the last thing that I've done it's been to add the WhatsApp folder to the autoupload. But that seems to be working fine.
The annoying thing is that I can't kill the app because it's not running in the gui, but it keeps showing the crash report every 4/5 seconds. It stopped after about 1 hour

[tobiasKaminsky]

But that seems to be working fine.

So we can close it?

[andreapx]

No, I meant that the upload of the WhatsApp folder is working because I can see the files in NextCloud, but the Android app is still not working.
If I try to open it, it opens (I see all the folders) but I can't do anything, if I try to scroll down to see other folder, nothing happens, if I click on the menu, nothing happens, I just have to kill it.
And sometimes the error reappear doing all the loop.

[tobiasKaminsky]

Can you re-install the app?
Enable auto-upload, but uncheck the option "also upload existing files"

Please report if this then works

[andreapx]

I've just opened the app and...it seems to be workig.
I've disabled the WhatsApp autoupload folder for now, but I'll try to enable it again maibe tomorrow.
Is there something that I can check for you?

[tobiasKaminsky]

Is there something that I can check for you?

No, I will have to have a look if fetching from db is somehow limited…

[tobiasKaminsky]

So, what I tested yesterday was to insert and query 50k uploads.
This worked fine, even on a Android 5 with low mem.

java.lang.IllegalStateException: Couldn't read row 1872, col 1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.

My idea was that this is a faulty stored upload, but I cannot trigger this.
Col1 is local path. But even if I insert a null local path (which is not allowed), I can read it.
It then crashes a bit later, but with error message "java.lang.IllegalArgumentException: Local path must be an absolute path in the local file system".

So it must be something on database level, which I do not really get… :/

[gjedeer]

Again, @tobiasKaminsky do you want me to send you the database file? (continuing from #5180) I'm not going to upload it in public but I'm OK to send it to your email.

BTW, after reinstall:

• the big directory (Camera with 8k files) has been automatically selected
• by the time I found the settings for that folder (~30s) i think it already added entries to database, because:
• when i pressed ok on settings, the app froze for ~5 minutes
• now the errors persist

[gjedeer]

I think I've found something useful: SQLite on Android seems to have a limitation of 2 MB per query? And when it's more it manifests itself as the exception Nextcloud is throwing here.

https://web.archive.org/web/20191219000631/https://medium.com/androiddevelopers/large-database-queries-on-android-cb043ae626e8

Did your 50k rows test use real paths? Because when I sum the lenghts of longest fields in the DB dumped from my phone, it approaches 2MB:

select sum(length(local_path)) + sum(length(remote_path)) + sum(length(account_name)) + sum(length(file_size)) + sum(length(_id)) + sum(length(status)) + sum(length(local_behaviour)) from list_of_uploads;
1919747

I didn't have the patience to include all other fields but it looks like they would easily approach 2MB.

[InfamousUser]

Just updated to 3.11.1 release, got this error upon going to uploads menu, assuming the error is:
Make sure the Cursor is initialized correctly before accessing data from it.
because the numbers are different.

[andreapx]

>

I've just opened the app and...it seems to be workig.
I've disabled the WhatsApp autoupload folder for now, but I'll try to enable it again maibe tomorrow.

I've talked too soon :(
I've opened the app today and it's still frozen. I've also noted that the WhatsApp folder that I've disabled the other day, still gets uploaded.

[gjedeer]

I've implemented a naive pagination (it's not thread safe but it was quick to write) and I think it solves the problem https://github.com/gjedeer/android/commit/36de0dc95fb6a2e2dcbb74bc0b6edd01fb7a031a

[ezaquarii]

I've implemented a naive pagination (it's not thread safe but it was quick to write) and I think it solves the problem gjedeer@36de0dc

Could you please open a PR?

[gjedeer]

@ezaquarii I'd have to implement it the right way (store the last ID in loop and select where ID < last_id, not use the concurrency-unsafe limit/offset thing), the commit is quick and dirty, and I have a work week ahead of me.

[tobiasKaminsky]

Nice.
This can be integrated into #5918 as I reproduced the error there.

@gjedeer if you do not have time, just ping me and I can take this over.

[InfamousUser]

Again on 3.12.0 RC1.

*** CAUSE OF ERROR ***

java.lang.IllegalStateException: Couldn't read row 1496, col 1 from CursorWindow. Make sure the Cursor is initialized correctly before accessing data from it.
at android.database.CursorWindow.nativeGetString(Native Method)
at android.database.CursorWindow.getString(CursorWindow.java:469)
at android.database.AbstractWindowedCursor.getString(AbstractWindowedCursor.java:53)
at android.database.CursorWrapper.getString(CursorWrapper.java:141)
at com.owncloud.android.datamodel.UploadsStorageManager.createOCUploadFromCursor(UploadsStorageManager.java:321)
at com.owncloud.android.datamodel.UploadsStorageManager.getUploads(UploadsStorageManager.java:302)
at com.owncloud.android.datamodel.UploadsStorageManager.getFailedUploads(UploadsStorageManager.java:368)
at com.owncloud.android.files.services.FileUploader.retryFailedUploads(FileUploader.java:1005)
at com.owncloud.android.utils.FilesSyncHelper.lambda$restartJobsIfNeeded$0(FilesSyncHelper.java:222)
at com.owncloud.android.utils.-$$Lambda$FilesSyncHelper$fa2LBpPOxRVQOm4F_Glue5u4hdM.run(Unknown Source:10)
at java.lang.Thread.run(Thread.java:919)

[tobiasKaminsky]

@InfamousUser well…yes…as it is not yet fixed.
If it is fixed, we will link the PR here and close the issue.

[gjedeer]

@tobiasKaminsky the PR is ready for review.
@everyone if you want to test if it solves the issue for you, the APK file is available here

[greve]

This issue seems related to / possible duplicate of https://github.com/nextcloud/android/issues/5995?

Found both when looking for potentially known causes for my issue: https://github.com/nextcloud/android/issues/5995#issuecomment-626318092

So won't file a new issue and wait for the next release.

Meanwhile got the app to not crash again by reinstalling, wiping all data, and hooking it up afresh to my instance.