Orgzly-android: Implement auto synchronization

[rynffoll]

Please, add auto synchronization for remote repositories.
Manual synchronization isn't comfy. also sometimes it could be cause of synchronization conflicts.

[bridgeofstraws]

If you have tasker and a rooted phone, try running this in tasker via a shell command (it'll force a sync):
su-c 'am startservice com.orgzly/com.orgzly.android.sync.SyncService'
To stop the service (may need to do this before starting again):
su-c 'am stopservice com.orgzly/com.orgzly.android.sync.SyncService'

[nevenz]

If you have tasker and a rooted phone, try running this in tasker via a shell command (it'll force a sync):
su-c 'am startservice com.orgzly/com.orgzly.android.sync.SyncService'

Service can be made "exported", so it doesn't require root to start, if that would help, short(-ish)-term.

[bridgeofstraws]

It would be lovely to have this exported (even with auto-sync). Thanks.

[anderspapitto]

I'd like to provide my usecase and desired UX as an example

I run orgzly against a folder in local storage, which (without any orgzly integration) I have synced to my computer with Syncthing.
I'm careful to not make conflicting updates - I have one org file which I update from my phone. The rest I update only on my computer, and orgzly is read-only for those files. So, I never get conflicts, but I do have to press the sync button every so often.

Ideally (for me) orgzly would just attempt a sync every so often (say hourly). If it works, great. If it would produce a conflict, skip the sync entirely (i.e. don't make the multiple conflicted versions), and just give a notification that says "automatic sync failed. push the sync button whenever you want and resolve your conflicts manually"

[bridgeofstraws]

Anderspapitto, I am currently using tasker to do a periodic sync as you suggest and foldersync to handle the remote sync and file conflict resolution. So in at least 1 form, what you are asking for can be done already. :)

[nevenz]

I've also added actions for better control, so what happens doesn't depend on current status:

am startservice -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService
am startservice -a com.orgzly.intent.action.SYNC_STOP com.orgzly/com.orgzly.android.sync.SyncService

If sync is already running, sending SYNC_START does nothing. If not, sending SYNC_STOP does nothing.

Toggling remains the default if no action is specified or recognized.

[bridgeofstraws]

This is great! It is very useful to do this without needing root. Anyone having tasker that wants autosync should be very happy. Do you know when this may be released? Not pushing, just curious.

[nevenz]

Do you know when this may be released?

Week or two from now probably.

[alphapapa]

Would it be possible to implement a kind of on-demand auto-sync, so that when a file is loaded or when Orgzly regains focus, it would quickly check to see if the remote copy has been modified and sync that file if necessary?

As it is now, I have several Org files synced to Dropbox, and Orgzly takes a long time to sync all of them.
If I have to wait 1-2 minutes for it to sync every file before I can load the one file I need, it means that there are lots of situations in which Orgzly simply isn't useful. But if it could just sync the one file I'm loading, it would be much faster.

Thanks.

[ghost]

I think the golden standard for autosync (for me anyway) is Simplenote. I never have to do anything, and everything stays in sync.

In fact, "getting out of people's way" is something Simplenote does really well. Notice the difference in workflows:

In Orgzly:

• Click Notebook
• Click Note
• Click "Edit Content" *
• Type
• Click "Close" (w/Save -- the checkmark) *
• Click Sync *

In SimpleNote:

• Click Note
• Type

Big difference!

* Why do I have to do this?

I wish Simplenote and Orgzly would have a baby. It would be the most beautiful child in the world to me, and it's uncle Emacs would definitely support it!

[nevenz]

Click "Edit Content" *
* Why do I have to do this?

For the same reason we have "Write" and "Preview" here (GitHub comments editor). 😄

Currently, only links are supported ([[link][description]]), but in the future, we'll have *bold*, /italic/, lists, etc.

"Write" mode is default when you're creating new note, "Preview" when editing the existing one, as a attempt to speedup things.

Suggestions for improving UX welcome. Perhaps the button can be renamed for start?

[ghost]

If we are not using a rooted phone, how can we run these commands?

am startservice -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService am startservice -a com.orgzly.intent.action.SYNC_STOP com.orgzly/com.orgzly.android.sync.SyncService

[licaon-kter]

Use an automation tool like Tasker, you don't need root for these commands.

Be aware that it might complain about the user this runs on, so make it say: am startservice --user 0 -a ... and the same for stop.

[ghost]

Oh, thanks, @licaon-kter ! The user error was the issue I was having! (I was running this from termux.)

[licaon-kter]

@nevenz After testing that in Termux (start and then stop), I've look at the app to see that it synced, it did, but after exiting the app I got 2 crashes: https://gist.github.com/licaon-kter/346c8f7012dc61b7db30f99401808e4e

I'll try to look at the bigger log soon, maybe see what happened before the actual crash.

[nevenz]

@licaon-kter Thanks, fixed.

[timoc]

I would prefer to offer a different strategy - Git branching. Whenever you make changes, it is committed to an orgzly branch, and visa versa. This implementation allows for more fun things, like having a limited subest of a larger git repo being under orgzly. Whenever there is a conflict, it can automatically create a new branch. Its then up to the repo-owner to tidy everything up.

[mkaito]

If you were to use git as a backend, which is an idea I definitely like, it can probably solve a lot of conflicts itself.

[ghost]

I like the idea of git for merge conflict resolution.

I use termux. I have it running orgzly sync via cron (stop _then_ start). I also have setup inotifyd to watch my repository directory, and when notes change, it runs a script that commits to git, which I also setup in that directory. I am using syncthing to sync the whole directory between my machines. Finally, I am using rclone via cron to 'backup' (end-to-end encrypted) the directory (actually the parent with lots of other non-note files in it) to gdrive, and, using a sneaky trick on my always-on desktop of sharing directories, duplicating that on yandesk.disk and mega.

Sorry. TMI, I guess.

Anyway, I want concurrency, privacy, security, redundancy, integrity, and as much automation as possible in my setup. So, automating sync would still help.

[angrybacon]

This discussion is becoming a little long, I'm not sure someone has mentioned Drive Sync already.

https://play.google.com/store/apps/details?id=com.ttxapps.drivesync

I like the way each folder can have its own strategy. More details here https://metactrl.com/userguide/?app=drivesync#folder-pair (all the options are detailed at the end).

[alphapapa]

This discussion is becoming a little long, I'm not sure someone has mentioned Drive Sync already.

https://play.google.com/store/apps/details?id=com.ttxapps.drivesync

Um, well...:

Contains ads · Offers in-app purchases

D:

[angrybacon]

@alphapapa you can just read the second link, no need to install. :-)

[alphapapa]

Oh, I thought you were suggesting to use it. :)

[timoc]

Back to git - Why?
The thing about it is that it can solve the merge problem without requiring that you to change your sync method. It is the git repository that is the important part, because it is designed from the ground up for distributed content management. With stash, branch and other git state storage, it has autosync etc handled, and has had for some time. it should be easy to help users maintain multiple syncable/mergable versions of the same org-document base with some hand holding and documentation.
I would also suggest that you do not need to write the code yourself. You can leverage existing tools (via extents?) or use code from open android ports like https://github.com/maks/MGit or https://github.com/rtyley/agit.

[alphapapa]

@timoc Git can sometimes merge automatically, but in the case of conflicts, I can't imagine resolving them on a smartphone UI being practical.

it should be easy to help users maintain multiple syncable/mergable versions of the same org-document base with some hand holding and documentation.

I don't follow you here. I've become familiar with git via the command line and Magit, and I know how to merge branches, resolve conflicts, diff branches, etc--but it was not "easy" or quick to learn, and that was with the power of the CLI and Magit and copious documentation available to me on a desktop PC. Trying to do all that on a tiny touchscreen will surely be frustrating and error-prone, and it will surely by no means be easy to teach people who aren't familiar with git how to do that, especially on a little smartphone.

Have you actually maintained a set of large Org files in a git repo and tried to merge changes made across different systems? If you have, you know that it's easy to lose data by choosing the wrong hunk when resolving conflicts. You have to be slow and deliberate and careful, and you need to be able to see more than a smartphone can display on its small screen.

I just don't see how merging conflicts with git could be feasible here, other than a simple ours/theirs merge strategy choice--but that would not allow users to truly resolve conflicts and would surely result in data loss at times.

[ghost]

@alphapapa

"Have you actually maintained a set of large Org files in a git repo and tried to merge changes made across different systems?"

I realize this was not a question to me, but for me, Yes.

However, if this feature were to be implemented, I don't think it would be mandatory. If you don't like using git or dealing with merge conflicts in it, I'm sure that could be accommodated by either not using git as your repo type, or by disabling the use of git for automatically handling conflicts. Making that an optional preference would be a good thing.

Git's conflict resolution stuff is basically line-by-line, whereas orgzly's is file-by-file. It is nice to have something handle the easy merges for you, so I like the idea of using git for that especially.

I had an Orgzly conflict last week, and it was in different parts of my org file, but orgzly just refused to deal with it (write/read or mine/theirs were the only choices I was given). However, the external version was already in git (as I am using git and a script to do auto-commits for me), so I just forced the write from orgzly, then went to a shell and diffed the versions and manually applied the proper changes in a different editor. Luckily both versions were at that point already in git, so this was not too difficult. However, if I didn't have this ability (because I wasn't using git) how would I have been able to resolve this issue at all?

Ideally, if I were just adding this feature for myself, I wouldn't want to have to deal with merge conflicts in a separate editor, or another program, or by commandline. I wouldn't want to use mgit at all. Git would be built into Orgzly (by integrating the mgit library), and if enabled for a filesystem repository, would automatically "just work" -- no learning curve. However, the real gem for me would be if the git conflicts were left in the file itself, and the file is committed with them marked. The notebook would also be flagged as conflicted in Orgzly so that I would know I need to fix it. So, both versions are merged together in both Orgzly and git, and all I have to do is edit it in Orgzly to "fix" it and let autosync commit it again.

For example, say there is autosync enabled, and say that there is a git library enabled for my git repo. During a sync there is a conflict when git tries to merge it. If orgzly automatically just "committed" the conflict, as marked by git and mark the notebook as conflicted, I could open the notebook and would see it as (ignoring the fact that this is plain text):

...
* Another heading
<<<<<<< HEAD:something.org
 This is my new line
 =======
 This is a new line from the other me
 >>>>>>> 4e2b407f501b68f8588aa645acafffa0224b9b78:something.org
* My last heading
...

I could then edit the file in orgzly to fix it and the next autosync would commit it again.

I don't see why this would be so hard or painful?

(I realize this doesn't deal with push/pull, but in my case, I don't need a remote repo. At any rate, config for and usage of push/pull during autosync could be by preference, too.)

[alphapapa]

I realize this was not a question to me, but for me, Yes.

Thanks, I appreciate your insightful comments.

I don't see why this would be so hard or painful?

You're right that committing the unresolved conflicts would make it possible to resolve them later. But as far as resolving them in Orgzly, it just seems to me that that would be cumbersome. On the other hand, maybe Orgzly could support the syntax and making resolving conflicts as simple as choosing from A or B, rather than having to manually edit all the lines of text.

[mkaito]

Git aside, most of my conflicts would be resolved by a 'sync' button in the incoming note dialog. You know, the one you get when 'sharing' something to orgzly.

Or (better), an option to sync before and after storing a new note.

Right now, when I want to store a note, I have to manually switch to orgzly, sync, switch back to wherever I was (and hope Android did not decide to clear the app's memory in the interim), store the note via 'share', switch back to orgzly again, and sync again.

I guess I've gotten used to fancy cloud apps where you don't even have to think about sync. Things just somehow stay in sync.

[alphapapa]

Right now, when I want to store a note, I have to manually switch to orgzly, sync, switch back to wherever I was (and hope Android did not decide to clear the app's memory in the interim), store the note via 'share', switch back to orgzly again, and sync again.

Yes, this is the crux of the issue.

[mkaito]

Quoting alphapapa (2017-05-11 19:14:09)

Right now, when I want to store a note, I have to manually switch to orgzly, sync, switch back to wherever I was (and hope Android did not decide to clear the app's memory in the interim), store the note via 'share', switch back to orgzly again, and sync again.

Yes, this is the crux of the issue.

Well, as far as I'm concerned, anything that makes the above less painful would
be very, very welcome.

If we could reach sync nirvana, and just forget sync even exists, while
everything magically appears on all our devices, hey, that'd be effin great.
But that's a pretty complicated proposition without a strict protocol and data
schema. Org-mode doesn't really fit very well with strict anything.

[psamim]

For me it would be great and enough if Orgzly had such an option as "Automatically sync after a new note is created".

[mkaito]

It is actually often more problematic when you don't sync before adding a note.

El 13 de mayo de 2017 10:29:29 GMT+01:00, Samim Pezeshki [e-mail] escribió:

For me it would be great and enough if Orgzly had such an option as
"Automatically sync after a new note is created".

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

[psamim]

As I said it works only for me because of the way I organize my files. I
have one file which I only add to from my phone.

As Emacs and Org-mode are very customizable and configurable people use it
in many different ways. So I think we should keep options open in the app
too. For me having an option to sync after a note is created works, we can
have this and other similar options in the settings.

So in my opinion we should first implement these easy scenarios (forced one
way syncs triggered by real events) as an option and think about more
complex scenarios, such as interval sync, merging, diffing and git-style
conflicts resolution later.

On Sat, May 13, 2017, 4:53 PM Christian Höppner [e-mail]
wrote:

It is actually often more problematic when you don't sync before adding a
note.

El 13 de mayo de 2017 10:29:29 GMT+01:00, Samim Pezeshki <
[email protected]> escribió:

For me it would be great and enough if Orgzly had such an option as
"Automatically sync after a new note is created".

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/orgzly/orgzly-android/issues/8#issuecomment-301244903,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AByDh0d60Hg7xQ1Z6aAGn8zD0EPV_VtAks5r5aCugaJpZM4MEUH6
.

[timoc]

Have you actually maintained a set of large Org files in a git repo and tried to merge changes made across different systems?

@alphapapa yes i have maintained fleets of org files spanning multiple git repositories, and small coteries of Big Fragging Org files of everything, also under git.

Git can sometimes merge automatically, but in the case of conflicts, I can't imagine resolving them on a smartphone UI being practical.

@alphapapa I agree with you here. I find that:

• Org files are hard to diff and merge, much like Yaml or XML - a special tool needs to be created
• Phone and Table UI's are impractical for this work, even with hand-holding and documentation. Luckily i can recommend another tool to help with the text editing part :), where documentation may be more effective.

The ;TLDR; is, that i stand by the proposal for using git to manage the orgzly org-files. Its a no-brainer IMHO, because it is a robust, cross platform, out of the box document sync tool, with global support and training for those who need it. Git supports syncing by design. It supports branching and merging for managing conflicts and sharing with others.

The longer explanation is:

Git does not enforce sync mechanism, rather Git enforces consistency and integrity, the golden standard of syncing documents. Git can do its own syncing, but you can use Syncthing, rsync, filesync or any other tool to copy your repo around the place. Used properly, git will maintain the integrity of your org document set across multiple devices.

Using a git as a document repo for org-files is not without some workflow difficulties, and possibly a learning curve, but i think the benefits outweigh the problems. The problems outlined in earlier comments are all workflow related, with the presumption that people's workflow includes maintaining their org-documents on an android device. I think this is an edge-case at best, (and if not, then using git is even more important)

Let me explain my workflow, I'm on the GTD bandwagon, so my general workflow is:

• _Capture_ -> _Process_ -> _Organize_ -> _Clarify/Review_ -> _Do_.

The only tool that GTD recommends is some form of list manager. Org is not a list manager.

Org agenda is a list manager. it flattens out information based on queries of your org document-set/knowledge-base/repository into lists/reports based on context, tags etc. These agenda lists are what i use to _Do_ and make proper choices in the moment, and the equivalent of the GTD managed list. These lists are the equivalent of tasks in a todo.txt or similar.

I used to use todo.txt type tools to share my task lists with other applications, so they were useful when i didn't have Emacs available. Especially for the _Do_ phase, where Android apps like that can use todo.txt like clockwork tomato, are fantastic aids to getting stuff done.

Orgzly is for me, a tool at the beginning of the GTD process (_Capture_) and at the the end (_Do_). is not a tool for the middle of the process (i.e. _Process_, _Organise_, _Clarify/Review_). That core part is what i use Emacs and org for.

In orgzly i can capture to an org file, and while _Do_ing i have access to the source org-document as support material for doing. It It would be nice to have an automatic feedback loop, but because i know Emacs+git, i am happy with the manual merge/update as it fits into the _Process_ step in my general approach. Automation would only mean that instead of that task needing re-_Process_, it would come up in _Review_.

[claiire]

Could we, for the sake of thorough documentation, have a brief guide on how to use the Tasker service for people who might not have used Tasker before?

EDIT:
I figured it out. That wasn't so bad actually. Just a bit of a learning curve.
For future googlenauts stumbling accross this thread, if you want sync to run every X minutes/hours,

1. Open up tasker and go to the Profiles tab.
2. Click the + at the bottom-center of the screen and select Time
3. Leave From alone. Uncheck To, check Repeat and put in the update interval you'd like.
4. Now that you've set up your Context for updating, go back to the Profile screen and there should be an Enter task box hovering over your new profile. Hit New Task, give it a name like "Sync Orgzly" or something, and hit the checkmark. This should open the Task Edit screen.
5. Add a new Action to the Task with the plus button at the bottom of the screen. Select Code > Run Shell on the popup.
6. In the Command field, write
   am startservice --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService
   Tasker should now sync orgzly every x minutes/hours/whatever you used. Probably. I'm bad at this stuff and new to it.

[psamim]

Calling sync service also can be done using Automate, which is an app similar to Tasker.

[carllacan]

Can we just get an option where Orgzly reads and writes the notes to a specific folder in the phone? (Rather than loading/exporting them to a repository, as it does now). That way we could just use Dropbox to sync.

[psamim]

@carllacan
You can choose a directory as a repository, then it reads and writes to a normal directory on syncs.

[carllacan]

@psamim Sure, but you still have to manually tell Orgzly to sync to/from that repository. What I am suggesting is that Orgzly directly stores notes in that repository, which is automatically synced by Dropbox (or whatever).

[psamim]

@carllacan What you want is exactly what we are talking about here. Automatic sync means notes are automatically synced with a repo, want it be a simple directory repo or a Dropbox repo. Maybe you are suggesting that if the repo is a simple directory repo, the app should sync on each read and write.

[carllacan]

@psamim Yeah, I'm basically suggesting that there is a sync after each read and write, but since there's a discussion here as to how exactly conflicts should be handled I was suggesting making it so that Orgzly simply reads and overwrites the files in a certain folder and let the user deal with conflicts however they want.

[mkaito]

Quoting Carles (2017-06-13 12:53:57)

Can we just get an option where Orgzly reads and writes the notes to a specific folder in the phone? (Rather than loading/exporting them to a repository, as it does now). That way we could just use Dropbox to sync.

You can sync to a "folder" repository, but sync still has to be manually
triggered. That's the whole problem here. I don't want to have to remember to
hit the sync button.

[khinsen]

As a new user, having discovered Orgzly recently, I almost immediately ran into the synchronization issue, which is for the most important obstacle to adopting Orgzly as part of my daily workflow.

My point of view is that the idea of "synchronization with a local directory" makes no sense. Why would anyone want to have two copies of the same information on the same device and sync between them?

There are two models of integrating a mobile device into a multi-device data synchronization scheme:

1. A mobile app manages data internally and provides synchronization with cloud services (WebDav, git, Dropbox, ...).

2. One mobile app stores data in local files and leaves synchronization with the external world to other mobile apps (FolderSync, MGit, ...).

The first approach is easiest for users if their preferred cloud storage is supported. Orgzly implements it only for Dropbox, which I suppose Dropbox users are happy with (I am not one of them).

The second approach is the most flexible one, but shifts the burden of synchronization to the user. Power users are probably happiest with it, and in particular those using "exotic" cloud storage systems (e.g. git). In the org-mode world, this user profile is probably much more present that among general Android users. I suspect most people contributing to this thread would like to have this approach implemented (I certainly do), and see Orgzly's "synchronization with local files" as a poor implementation (which it is). Making this on-device synchronization automatic is just a complicated way of getting the same result: data storage in the file system.

[tulth]

I am considering adding short-term quality of life sync improvements.

My use case:

• file-based syncing
• using syncthing
• single user, editing my files on different platforms, (phone, tablet, desktop) but never simultaneous editing.
• notes files are relatively small

Approach:

• for all features, add checkbox defaulted to not enabled in preferences, option to enable
• hook into MainActivity::OnResume() to sync all notes on resume
• on starting to edit a note, force a sync of that note
• big floating sync button in the bottom left similar to the floating plus for adding a new note

[tulth]

i have not tried to do the note specific sync. It might be worth an architecture discussion. I am thinking each note would be more autonomous... They could maintain file mapping and have an async task to sync. Sync all would iterate over notes and tell each to sync. On starting or finishing a note update a specific note could be synced.

Typed on phone...

[IvanMalison]

What is the status of this / #24 ? Has anyone made an attempt to get this working?

[tulth]

See pull request #159 for the experimental auto sync

[dncrash]

If I am understanding this correctly, using sync or running those sync commands only saves the changes you made in Orgzly, to the repo. Is there any such command that updates Orgzly if the files were modified outside of it ? I sync my orgfiles using git but when changes are made to the files directly I need to force load each file in order to see changes in Orgzly.

[tulth]

it can read in the changes whenever the app is started (or comes forward) if you enable the resume sync.

It does not yes have file observers to catch changes that happen while the app is open

[IvanMalison]

It does not yes have file observers to catch changes that happen while the app is open

Why not add some type of reasonable polling mechanism?

[psamim]

It seems that this feature is experimentally implemented in Orgzly 1.6 and it has been working fine for me untill now. I use a local file-based repository.

[ghost]

@psamim Using a local file-based repository, how do you autosync to web? I use Dropbox normally, but I'm open to other options.

[karlicoss]

@apnewberry I'm using Dropbox + dropsync (for Android)

[psamim]

I don't use web. I sync local files with files on my laptop using Syncthing.

On Sun, Apr 15, 2018, 1:18 AM apnewberry [e-mail] wrote:

@psamim https://github.com/psamim Using a local file-based repository,
how do you autosync to web?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/orgzly/orgzly-android/issues/8#issuecomment-381359545,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AByDhwSrkCLFJFt9zhkjtH8tp-qYAGfnks5tomCPgaJpZM4MEUH6
.

[kajmaj87]

@claiire wrote:

1. In the _Command_ field, write
   am startservice --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService
   Tasker should now sync orgzly every x minutes/hours/whatever you used. Probably. I'm bad at this stuff and new to it.

This does not work anymore on Android 9 (and probably 8) because of added restrictions to background services. It's quite easy to get it working though, it is enough to start the sync as a foreground service. If you have problems in Android 8/9 change the above line to:

am start-foreground-service --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService

[codygman]

It looks like for my most common case where I leave orgzly open on client A, update syncthing based folder from client B, that the updates from tasker using the command @kajmaj87 specified happen... but it's only every 10-15m that I'll actually get a notification from tasks added by client B.

I guess after the activity is disposed of?

Does orgzly not automatically get updates from the command:

am start-foreground-service --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService

Well, I just got a batch of notifications over different times so maybe it does work.

[memeplex]

@nevenz why is the auto-sync with local folder still experimental? What else is missing?

If what's missing is auto-sync with remote folders, is that really relevant?

Couldn't orgzly settle down on what @khinsen suggested a long time ago in https://github.com/orgzly/orgzly-android/issues/8#issuecomment-311001467?

I mean, you're already giving an easy although manual way to get dropbox sync working. You're algo giving "auto-revert from" plus "auto-save to" the local storage, which allows to use every sync tool/remote storage out there. You even provided hooks to call in-built sync functions from automation apps (there is an open request to implement a plugin for tasker or something, I believe energy would be best employed in adding a howto section to the user guide explaining how to configure tasker and other tools to get it working).

IMO the only thing it's missing is a clear vision of what is the scope of orgzly regarding synchronization. Maybe this depends on a proper characterization of orgzly user profiles and I think @khinsen hit the spot there. I believe effort is being wasted trying to provide all kinds of built-in sync options while a basic "save to disk" is still tagged experimental. Maybe git integration could be ok, because git tools for android aren't that good, but in any case -again IMO- it's far more relevant to get the basic approach established for good.

If you asked to me I would answer that, at the sync front, orgzly already does what it needs to be done:

1. An easy but basic way to use dropbox (which anyone can install for free... the "but I use XXXCloud" argument is quite weak given that you also provide (2) below).

2. An easy (but experimental) way to locally auto-sync.

Remove experimental and you have a coherent approach to synchronization and more time to work on other greater ROI features.

[memeplex]

Also, I wouldn't even talk about "local auto-sync", I would enable it by default and talk about "store in local folder" vs "sync with cloud storage".

[memeplex]

Has anyone experimented with syncing at screen on/off event (using tasker)? I have three devices (a phone, a tablet and a laptop). It's not common for two of them to be in active use at the same time. If I'm not using an android device its screen is off with 99% probability. So it seems reasonably to trigger sync from screen on/off events, plus a safeguard sync every 1h or so.

[codygman]

I use tasker to auto-sync and it works, but I don't use the on/off events
and instead sync every 45s. The trial ran out though and I can't modify or
turn it off. Been meaning to replace it with a termux alternative that just
calls the activity to sync.

On Mon, Feb 18, 2019 at 4:45 PM memeplex [e-mail] wrote:

Has anyone experimented with syncing at screen on/off event (using
tasker)? I have three devices (a phone, a tablet and a laptop). It's not
common for two of them to be in active use at the same time. If I'm not
using an android device its screen is off with 99% probability. So it seems
reasonably to trigger sync from screen on/off events, plus a safeguard sync
every 1h or so.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/orgzly/orgzly-android/issues/8#issuecomment-464909759,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANyNwJPO_By3UKfpFO89GN3nmidYfFoks5vOyz2gaJpZM4MEUH6
.

[memeplex]

I have implemented both automagic and tasker workflows that do the following:

1. When the display is turned on and orgzly is the current app, sync and set a global flag to 1.

2. When orgzly is put into the foreground and the flag is 0, sync and set the flag to 1.

3. When the display (or the system) is turned off sync and set the flag to 0.

This kinda defines a "session" as anything between display on and display/system off. Then syncs the first time you use orgzly in the session and once you leave a session that has used orgzly. This way syncs are reduced to a minimum but the mechanism is still very robust.

It's working quite well for me. If you're interested I can add an entry to the docs explaining how to configure it and even share automagic/tasker exported workflows ready to be imported.

[novoid]

I use tasker to auto-sync and it works

Can you please tell me how you do this?
I tried several Tasker commands and it doesn't work on my OnePlus 5 with the latest Android 9.

[memeplex]

@novoid create a Send Intent action with:

• action = com.orgzly.intent.action.SYNC_START
• package = com.orgzly
• class = com.orgzly.android.ActionReceiver
• target = Broadcast Receiver

Works for me in Android 7.1.1.

[kajmaj87]

@novoid on Android 9 be sure to follow what I wrote in https://github.com/orgzly/orgzly-android/issues/8#issuecomment-436028424

This should do the trick for you.

[novoid]

am start-foreground-service --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService

... did the trick for me, yes. Thanks!

[codygman]

I'm was just now now seeing your question, so glad you found a
working @novoid!

On Sat, Feb 23, 2019 at 11:44 AM Karl Voit [e-mail] wrote:

am start-foreground-service --user 0 -a com.orgzly.intent.action.SYNC_START com.orgzly/com.orgzly.android.sync.SyncService

... did the trick for me, yes. Thanks!

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/orgzly/orgzly-android/issues/8#issuecomment-466672631,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AANyN4mAF6saklx6AmZN66-Ff7Kx8cgTks5vQX3sgaJpZM4MEUH6
.

[nevenz]

Auto-sync as it is now shouldn't be experimental anymore. It was marked as such while it was working with Dropbox repo type too, mainly to warn about how easy was to transfer full notebooks back and forth very often over the net.

Orgzly keeps data in its database, so syncing org file with any location is a two-part process -- generating or parsing the org file and transferring the file.

The first part is the heavy one -- when sync needs to happen, entire org file needs to be generated or parsed. That's why syncing with local directory is being treated the same way as syncing with Dropbox.

One idea is to try to update the org file after every user action, or at least prepare the data for faster building of the org file. This is one of the reasons for new use cases in v1.7 for example.

Syncing in other direction though will still need to parse entire file, or somehow compare it with the "current" one and then only update notebook's data in database.

I think the only thing left for this issue is to remove experimental flag and implement auto-syncing with any repository type (guarding from frequent data transfer).

We can open a new issue for improving the efficiency of syncing.

Let me know if I missed to address some points (very likely).

[memeplex]

Syncing in other direction though will still need to parse entire file

Just out of curiosity, how is this implemented? By periodically polling or by doing a full two-way sync upon each local change?

[nevenz]

Just out of curiosity, how is this implemented? By periodically polling or by doing a full two-way sync upon each local change?

The latter. Periodic (and/or triggered by some events like phone waking up) checking of remote repo is planned. (Last disabled preference in Auto-sync settings.)

[pellenilsson]

My take on the sync-outside-orgzly strategy, here driving it from the PC instead of from the mobile:

https://pantarei.xyz/posts/sync-org-mode-with-mobile/

[harshitgarg22]

+1, are there any updates on this issue?

[NiceFeather]

What I want to say is that after the loss of the network, the changes made to notebook, the automatic synchronization problem after re-linking to WIFI, is a special case of this issue.

I just did some tests on the support of LAN WebDAV server after re-linking to WIFI automatic synchronization function. First of all, in the case of interruption with WIFI, I made some changes to the notebook, after the phone is reconnected to the WIFI, the phone is not automatically synchronized with the local area network WebDAV server. It is suggested that after the device automatically obtains the WIFI signal, the synchronization attempt should be made and the message of the synchronization result should be given.

[nevenz]

after the phone is reconnected to the WIFI, the phone is not automatically synchronized

Yeah, this is another good case to handle.

[nevenz]

Summary of things to do to improve auto-sync:

• [ ] Periodically check for changes in repositories
• [ ] Enable auto-sync for all repository types
  
  
  
  • [ ] Queue sync requests to avoid running sync too often (for Dropbox API)
  
  
  • [X] Consider adding per-repo option (except Dropbox) or just enable it for WebDAV
  
  
• [ ] Check for remote changes before local modification (#434)
• [ ] Send sync request when connection is re-established

[Nebucatnetzer]

Sounds very reasonable to me.

[MartinX3]

In my opinion the the merge conflict handling needs a good strategy. :)

1. You accidentally could do changes on 2 devices without syncing first.
2. Or one of the devices could have a wrong system time by accident or any other reason.

[Nebucatnetzer]

A solid system would be nice for sure. I think however that it is better to have something basic quickly would be better. If it syncs every time the app opens would already be a huge improvement since you wouldn't be editing old files on your phone.

[MartinX3]

Yeah, a sync on every save / opening since it's not intended as multi user application.

[nevenz]

In my opinion the the merge conflict handling needs a good strategy. :)

Conflicts can always happen, no matter what services are used, auto-syncing just minimises the chance of them happening. I don't think there's an issue opened for handling conflict resolution.

If it syncs every time the app opens would already be a huge improvement since you wouldn't be editing old files on your phone.

This is already implemented, but only enabled for Directory repository type. Perhaps it should have been enabled for WebDAV too, as the restriction was added to limit hitting Dropbox API. Something will be done for v1.8.1.

[doak]

That's quite a hot and long requested feature ;)

TL;DR
This is about a workaround using inotifyd to trigger Orgzly's sync.

Some days back in 2017 from another user:

I use termux. I have it running orgzly sync via cron (stop then start). I also have setup inotifyd to watch my repository directory, and when notes change, it runs a script that commits to git, which I also setup in that directory.

I just stumbled over this after I scripted something similar and wanted to leave a note here ;)

My script does the following:

• Listen on file changes in specified directories using inotifyd.
  (This does not work recursively, hence in case of _Orgzly_, you need to specify every directory you sync with.)
• Trigger synchronisation of _Orgzly_ on "every" relevant change. (It debounces a little bit).

For now it seems to work quite well.
Orgzly itself is configured to automatically sync in case of own changes, but it is disabled on _App started or resumed_. This is not necessary anymore and disturbs after all in this case.
The syncing to the device is done using _Syncthing_ which does a quite good job here. The notes get updated within 20s latest usually. This also works for the Widget btw ;)
It uses _Termux_ and uses Termux:Boot to start-up automatically. Battery usage seems okayish.

I've also added safety checks in case the changes respectively the sync trigger goes crazy (and the debouncing would not work) which stops the sync for some time (and opens a notification to inform about that). Better handling of the script, like manually stop or start is not available yet.

If anybody is interested, the code is available here:
https://gitlab.com/doak/orgzly-watcher

[lytex]

That's quite a hot and long requested feature ;)

I bet it is!

I've also been using for some months inotifywait inside termux, which does support recursive scans (see the exact command I use, which also excludes the .git folder). I use this specific script to commit automatically when there is a file change.

The recommended intent is a broadcast instead of SyncService directly. I have experienced some inconsistencies using SyncService on Android 10 (it's not always being triggered ok).

I also have battery issues (faster sync means more battery drain), but I don't mind this high battery usage very much.