Qownnotes: current note incorrectly reloaded

[brainchild0]

Expected behaviour

Application detects external change to note only if the timestamp of that file changes.

Actual behaviour

Application incorrectly detect updates to current note while it is being edited. When I am editing a note, the cursor is constantly jumping and I am constantly receiving dialog messages indicating that the note has been changed externally.

As best I can determine, whenever the current note gets saved, within a few minutes the Nextcloud desktop client synchronizes it with the server and writes metadata to the base directory in files beginning with ._sinc. The application detects the change to the directory, which somehow causes it to try to reload the file from disk, reporting the following in the Log panel: [<timestamp>] [status] Loading external changes <filename>.md.

So two events are occurring that ought need to examined:

1. Changes to Nextcloud metadata are considered changes to notes collection, even though the metadata changes every time Nextcloud synchronizes folders, whether or not it downloads any files.
2. Somehow, the changes to the directory are interpreted as changes to the current file.

Steps to reproduce

In my deployment, it is sufficient simply to open a note and to type for no more than a minute, with the Nextcloud desktop client operating on the same directory.

Output from the debug section in the settings dialog

See gist.

Relevant log output in the Log panel

[05:35:50] [status] Stored current note to disk
[05:35:54] [status] Notes directory was modified externally
[05:35:54] [status] Stored 1 note(s) to disk
[05:35:58] [status] Notes directory was modified externally
[05:35:58] [status] Stored 1 note(s) to disk
[05:36:01] [status] Stored 1 note(s) to disk
[05:36:02] [status] Notes directory was modified externally
[05:36:02] [status] Stored 1 note(s) to disk
[05:36:05] [status] Notes directory was modified externally
[05:36:05] [status] Stored 1 note(s) to disk
[05:36:09] [status] Notes directory was modified externally
[05:36:21] [status] Stored 1 note(s) to disk
[05:36:24] [status] Notes directory was modified externally
[05:36:24] [status] Stored 1 note(s) to disk
[05:36:25] [status] Current note was modified externally
[05:36:27] [status] Loading external changes from: foo note.md
[05:36:27] [warning] QTextCursor::setPosition: Position '153' out of range
[05:36:28] [status] Notes directory was modified externally
[05:36:28] [status] Stored 1 note(s) to disk
[05:36:31] [status] Notes directory was modified externally
[05:37:26] [status] Notes directory was modified externally

Suggested resolution?

1. Decouple tests for directory timestamp from reloading file.
2. Avoid checking timestamp of parent directory as a method to test for external updates.

[pbek]

Please don't post the text dumps in files, that make them hard to read. ;) And please post "Output from the debug section in the settings dialog", or it's hard to tell what we are talking about.

This is an issue often discussed here and basically can't be fixed (at least I'm not aware of a way). It depends a lot on operating systems, how many files are open by other applications and so on. You only get inotify notifications for files (that you have to add to the notifier) and directories (where you will get notified by every change in it but don't know which file was changed).

Are there any new insights I don't know of?

[pbek]

this is the Qt implementation: http://doc.qt.io/qt-5/qfilesystemwatcher.html

[brainchild0]

Sorry that I misunderstood the issue template. Also, sorry for posting a duplicate, as I searched beforehand, but the issues I found were all closed.

Regarding the issue, it would surely be disappointing if you correctly say that the issue cannot be fixed, because I am finding that when the open document regresses, and the cursor jumps, the application becomes completely unusable, as I am sure you would understand.

At the same time, I must admit to being confused by your conclusion. We have in front of us, at minimum, two facts that are plain:

1. The application reloads a file into the buffer, reporting that the file has changed since it was last saved by the application, and therefore must have been modified externally.
2. The file appears on the file system with contents and timestamps identical to when the application last wrote the file.

Then, if all else fails, at a bare minimum, the application can test the modification timestamp, and possibly even the file body, and compare against internally saved data, before reloading the file or generating a message for the user. If the application finds that data on the file system matches the internally saved data, then it should be clear, I think, that the application should suppress reloading the file.

Meanwhile, although the above can serve as a fallback, an existence proof, if you like, that the issue can be resolved, I would notice that any of a variety of strategies can be imagined for monitoring directory changes.

Using the existing approach of the QFileSystemWatcher, a wrapper for inotify, I can make two observations:

1. Any application, as I understand, can create multiple watchers, and each of those watchers can be assigned a different set of files or directories to watch, and a different signal handler to process events. Therefore, the application can be designed to respond to the open file through one signal handler and the entire directory through a different signal handler, each handler operating appropriately to the case.
2. Generalizing the earlier comments, the application is not required to process a signal in any particular way once it is triggered. The signal is useful to produce a callback when a particular operation, such as reloading the file, _may be_ appropriate, but the application can surround the call to that operation with any appropriate conditional logic, such as querying the file system and internal application state, to determine whether a particular operation _actually is_ appropriate.

My view, then, is that a variety of possible solutions are available, some that may entail redesign, and surely others that do not.

I would have to review more comments from you to understand what I may be overlooking, that causes you to conclude that the issue cannot possibly be resolved.

[pbek]

My very short answer to your essay (:smile:) is: I'm open to pull requests if anyone can improve on any of the the situations (which I would love to), without introducing regressions. I already tried quite some different things.

In addition (like mentioned in other issues): If anyone is hit by the situation of jumping mouse cursors (macOS seems prone to that), there is an option in the General settings to turn off the cursor restoring. There also is an option to turn off detection of external changes altogether.

[brainchild0]

It's possible that sometime in the future it would be feasible for me become familiar with your codebase and try to make some enhancements that you could consider pulling.

Is the immediate obstacle, for you, to find the time to commit to the work, or to find the way to make the application work better?

[pbek]

Beside finding a system where your problems occur (you still failed to even failed to use the issue helper dialog or the Debug section of the settings to provide information about your setup 😉) and spending the time finding out where they come from it will be hard to find a way around them. I already spent a huge amount of time making it as stable at it is for most of the users. But feel free to play around, this is a good entry-point: https://github.com/pbek/QOwnNotes/blob/develop/src/mainwindow.cpp#L874

[brainchild0]

I understand. I updated the report to include the requested information. (I had misunderstood earlier that you already knew how to reproduce.)

[pbek]

I updated the report to include the requested information.

Please open the QOwnNotes Settings and click on the Debug Item in the settings menu tree. Then you will see what I'm talking about. The issue generator in the Help Main Menu does that automatically.

But feel free to play around

Happy hunting. https://www.qownnotes.org/installation#building will help you

[brainchild0]

Right. I must have pressed "preview", but forgotten to submit. See update.

[pbek]

great (beside still not posting the output inline :laughing:), just tell me if you can find something out or you need help with the code

[pbek]

I will close this until there is more information.