Joplin: [Feature Request] Selective Sync

[dotgreg]

Hello All

First thanks for that amazing piece of software, I really am surprised on how good it is as an evernote replacement.
Something I really liked in the evernote system was the ability of selectively syncing elements that are used, and not the whole database.

That functionality seems important for the following use cases :

• using a device with limited space/limited bandwidth like a phone
• wanting to access to a specific information on a newly added device without needing of synchronizing everything
• wanting to save space on desktop/mobile by not having everything on the local

Those use cases become more and more frequent as the user database grows to several gigabites.
Now I know that the project defines itself as an "offline first" solution, where all the data is available on the device without connection.
I feel that could easen the access to people that want to switch from evernote but accumulated several years of notes they want to migrate.

Now Id love working on it but as it seems a big work, I was wondering :

• Is such an addition technically possible with the current architecture?
• Is it still inside the scope of the project? (if we propose it in the preference like a "selective sync" checkbox, unchecked by default)
• Any advices on how to structure it code-wise and data-wise? (would it needs to store another metadata file for each md note for example?)

Thanks

[laurent22]

It's technically possible but would require quite a lot of work. In order to allow the user to select what to sync on mobile, they'll need to see part of the data first so it means we'd have to support partial sync and that would be tricky since the code everywhere except to get full notes when they are loaded.

One solution which the architecture already supports would be to give an option to download the resources (attachments) only on request. i.e. we display an empty container in the note, and the user needs to click on it to download the resource. Since these files often take a lot of space, it should help a bit with the issue of syncing with mobile.

Another solution is to improve the sync process. Currently when downloading a lot of notes on initial sync, if it fails midway (or if the user switches off the device), the process will need to go back to the beginning and check what's already been downloaded. This is very slow and probably can be improved. If it wasn't doing this, the app could just keep downloading the note over time, even if it takes a long time, and the app would still be usable in the meantime.

[dotgreg]

One solution which the architecture already supports would be to give an option to download the resources (attachments) only on request. i.e. we display an empty container in the note, and the user needs to click on it to download the resource. Since these files often take a lot of space, it should help a bit with the issue of syncing with mobile.

That's a smart and fast way to solve the most of the problem while keeping the base 100% offline first (indeed, lots of implications I imagine, especially on modules like search, which would need to be done backend-wise, which is structurally impossible here I feel).

I am gonna check that out, would love working on that one. So if a PR arrives with that functionality, it seems for you merge-able design and scope-wise?

[laurent22]

Yes it would definitely be within the scope and merge-able. The way I thought about doing it is add an option:

Download attachments: "Always" or "On demand"

If it's "always", everything works as now. If it's "on demand", resource synchronisation is skipped in lib/synchronizer. Then I guess whenever a user clicks on a resource, we can add the ID to a list (saved using the Setting class), then any resource in that list will be synced in lib/synchronizer, and anything not in there will still be skipped.

I can't think of any big drawback to this approach, because while the code doesn't handle partial items, it handles the item simply not being there, so everything should work like before even if we don't download some resources.

If you look at this, you might find a better way to get this working though, so feel free to share your findings. Or if you need some info about the architecture let me know.

[dotgreg]

thanks for the details, gonna take a look on that point

[laurent22]

Some WIP there: https://github.com/laurent22/joplin/pull/1527

[dotgreg]

ah wow, nice!