Joplin: Search on Android not working

[cloo]

Operating system

• Android

Application

• Mobile 1.0.271

Search on android does not work for some notes, even for titles. For some notes works well.
Search on windows desktop works for all notes.

[foxmask]

As you wrote "for some notes works well", did you noticed which notes are not found and those which are not found ?

[cloo]

Yes I have, but I cannot see structural difference between the ones who are found and the ones who arent. All are simple notes, without images/embeds, not todos.

[foxmask]

the ones that are not found were encrypted ? (log seem to say encryption is now off)

[cloo]

yes, encryption now is off.
I had some difficulties with encryption. When switched on, mobile sync got stalled (stopped syncing and didnt advance at all during night), so I switched it off on mobile, but it seemed to stay enabled on windows. That was weird, so I uninstalled mobile app, and then switched encryption off on windows. Maybe that caused some mess.

[foxmask]

When you switch from on to off , as explain in the doc, joplin needs to sync ALL the notes.
Once the first device is synced, you can sync the second one and so on.
Then finally you will get all your notes uncrypted

[cloo]

Hmm, weird,
in the mobile encryption is shown as on (and all notes as decrypted);
in windows encryption is off.
That shouldnt be possible at all.

[laurent22]

Is FTS enabled? (you can check in Config screen) Also before a note can be found in search it needs to be indexes and that can take time.

[laurent22]

From your log it seems the search engine indexing doesn't finish so it's possible there's an error somewhere.

[cloo]

Thanks @laurent22
Does that mean, that I should simply clean all storage/apps everywhere except one location (in my case - windows), and resync from there?

[laurent22]

@cloo, that might indeed fix your issue. But is there any chance you could keep the app on your phone for now? I'll build another version which logs search indexing errors properly and that way we'll know what the issue is. Just so that if it happens again to you or someone else we'll know what to do.

[cloo]

Definitely, for a while. How much time do you need?
(FTS is enabled)

[foxmask]

@cloo when you reinstall the app on android and press "sync" the first time, joplin get the master key you created previously and enable the encryption. So you need to disable it once again on your phone and this will sync again, if i'm not wrong , @laurent22 ? then maybe the search indexing will finish properly ?

[laurent22]

@foxmask, it might help, but there's also a bug in search indexing it seems. Even if everything is encrypted, indexing, should start, skip everything (because it doesn't process encrypted items), and then stop. But from the log it seems it's starting but never prints the finish message "SearchEngine: Updated FTS table in ...", which makes me think an exception happen somewhere in between.

@cloo, adding this is quick, so I'll build a new version later today.

[laurent22]

@cloo, could you give that version a try?

https://github.com/laurent22/joplin-android/releases/download/android-v1.0.277/joplin-v1.0.277.apk

I'm not sure it will help but at least for some errors it should now log them and tell us what the problem is. To trigger search engine indexing, you can modify a note, save it, and wait for at least 30 seconds. After that, if you could send a report again to see if there's any error message that would be great (it is now under Configuration > Export Debug Report)

[cloo]

Did that.
Apparently, newly created notes are not found too.
Btw, UI is better now!

[laurent22]

That's great, thanks @cloo. The good news is that this update now prints proper error message in case of a problem:

06-26T21:18:04,10,"""SearchEngine: Error while processing changes:", "{"message":"Row too big to fit into CursorWindow requiredPos=0, totalRows=100","code":0}""

The bad news is that I have no idea what it means, but it's specific enough that it might be possible to figure out a fix.

In the next version there will also be a button to fix the search index, which might also solve your problem.

[laurent22]

I think this bug happens when requesting a lot for data via SQL. Normally in that case, Android should increase the available memory size, and in fact that's what it does in all my tests (including with simulated low memory conditions), but somehow here it seems it cannot. So for now as a possible fix I've changed the search engine indexing so that it processes less data at the same time so maybe it will fix this issue.

@cloo, by the way what version of Android and what phone are you using?

[cloo]

I'm using Android 9 PKQ1.180904.001 With MIUI Global 10.3.1. stable (I guess MIUI might cause memory issues).

Also I ran into more trouble.. Trying to fix the problem of non-searchable notes, I uninstalled Android app, cleared app data (with built-in cleaner). On windows changed sync location to local, deleted data folder from dropbox. I hoped to create a fresh situation for resyncing with dropbox and android - but it didnt work. Now only new notes are synced Dropbox, both from Windows and Android, even though I removed local sync folder. And Android seems to remember some old test notes from my first installation.. Any idea what is happening?

Here are reports from android and windows.

@laurent22, should I register a new issue on this? Seems to be little different problem.