Foam: Automatic git sync

[chrisands]

Hi, just today started to use foam. Since there is no discussion about automatic git sync...

I think it is possible to use vscode "editor.codeActionsOnSave" to git commit and git push (& pull?) every time file is saved. I'm not familiar with this option, but, briefly looking at it, I think foam needs to be markdown provider to able use it.

What you think about it? I could do this, but I don't think I'll have time in next two or three weeks.

[scott-joe]

I have this in my personal list of desired features. I had some ideas on how to cut down on the clutter as well. Perhaps some sort of automated time based gitflow? I'm interested in finding a balance between the granularity of backups and overloading the commit log. So, some sort of local branching each day with a squash commit to the week/month/etc and only daily squash commits to remote? The usefulness the idea is down to the timing of these events in relation to the needs of the user. If you write a lot often, you may want something better than that? But...perhaps at that point, you'd be better off with something other than Git for providing change logs to your files...

[riccardoferretti]

What's the "problem" you are trying to solve with this approach @scott-joe ?
Asking because I have been thinking about it myself, and I wonder whether a EOD commit is best (which would give me timeframes), or editing session commit (which would give some sort of clustering of ideas/notes), or even on save (per @chrisands 's comment) just to ensure things are backed up..

[SanketDG]

I think given the large spectrum of audience that Foam could be useful too, among all other possible options, manually comitting should be _default_, because otherwise, you would be defining for the user.

One thing I'd like to see it commit message summarization based on the changes.

[scott-joe]

@riccardoferretti I'd like to emulate some of the features I rely on in Bear/Notion, such as syncing. To that end, I'd like my workspace to automatically commit or at least stage changes at some regular interval and push those changes to my remote repos.

To keep those repos tidy, I'd like to have some sort of housekeeping done there, but that's just a bonus at this point.

The simplest in-workflow solution would be to wire up a Command Palette entry to something like git commit -am "{timestamp}" && git push origin main so I could easily push my local changes to a remote repo before closing my work machine for the day and do the opposite when opening up Foam on my personal machine later on.

[chrisands]

So there should be two separate things. First is when and how foam commits changes. And the second thing is how often we sync remote repo with local.

About commits: I think the best would be on save run hook which will check how and when we commit. As options: on save or timer. Also foam can check for last sync with remote repo, compare with last commit time and amend commit, so repo isn't cluttered.
e.g.

// settings.json
{
  "git.postCommitCommand": "none",
  "foam.commit": {
    "type": "manual" || "onSave" || "timer", // by default manual
    "timer": "5m", // not sure about timer format.
    "batch": true || false // on call option "amend" instead of batch
  },
  "foam.sync": {
    "type": "manual" || "onSave" || "timer", // by default manual
    "timer": "1h",
  }
}

About sync: foam persists last sync time and after each commit run a hook which says when we should sync or even it can run with save hook.

Added reference for on save hook implementation

[riccardoferretti]

@scott-joe is git integration what you are talking about? That should cover at least the staging of changes.
Possibly in combination with a vscode macro extension it could also push.

I was also wondering whether this could be as easily achieved with a task (https://code.visualstudio.com/docs/editor/tasks).

Might be worth starting off by writing a recipe for how to approach various scenarios before building it right into foam. Feels like there are a lot of different cases, the workarounds are fairly solid (e.g. tasks can live with the repo and therefore be shared across devices), and in the meantime we can learn about how people like to sync things.

(more info at https://github.com/foambubble/foam/blob/master/docs/recipes.md#version-control)

[SanketDG]

I have been trying to look into git-hooks with VSCode to see if I can get basic summarization, but unfortunately the support is still WIP at https://github.com/microsoft/vscode/pull/95266, but it can work at the terminal

.git/hooks/prepare-commit-msg:

#!/bin/bash

updates="Update $(git status --short | grep "^M" | awk '{print $NF}' | grep '.md$' | sed -E 's/-/ /g' | sed -E 's/.md//g' | sed -E "s/\b(.)/\u\1/g" | sed -E ':a;N;$!ba;s/\n/, /g')"
add="Add $(git status --short | grep "^A" | awk '{print $NF}' | grep '.md$' | sed -E 's/-/ /g' | sed -E 's/.md//g' | sed -E "s/\b(.)/\u\1/g" | sed -E ':a;N;$!ba;s/\n/, /g')"
echo "$add, $updates" > "$1"

So if you have a status like this:

On branch master
Your branch is up to date with 'origin/master'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        modified:   foam-tips.md
        new file:   technical-blog-ideas.md

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   100-things-product.md
        modified:   ideas.md

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        2020-10-04-22-15-21.png

It would summarize:

Add Technical Blog Ideas, Update Foam Tips

I am trying to figure out on how I can extend this, where I can even scan diffs for block level summarization (this would really help with the planned markdown ASTs), and also the UX of how it should work (right now the user has to git add and then git commit populates the message)

Thoughts?

[chrisands]

I'm in progress of implementation auto commit function https://github.com/chrisands/foam/commit/bde65d834677e7fc862ceacf5367d6c06d14429a.

[ingalless]

@chrisands I prefer this approach to the terminal approach from @SanketDG (although those commands are seriously cool!) simply because not _everyone_ will be on a system that is using bash or similar (and perhaps greedily because I'm on windows and would enjoy having this functionality too!)

[SanketDG]

Yes, I would definitely love something that's integrated in the extension (so it just works in VSCode irrespective of everything else) with CLI as secondary (just like janitor)

[leejoramo-d51]

I use the SaveAndRun extension to automate things like this. Naturally, this only makes sense to use in something like personal notes that you want automatically committed and pushed Here is my settings.json configuration to commit and push on every save.

    "saveAndRun": {
      "commands": [
        {
          "cmd": "git add --all && git commit -m \"saved notes \"(date +%F-%H%I%S) && git push",
          "match": "\\.md$",
          "silent": false,
          "useShortcut": false
        }
      ]
    },

The commit message will be in the form saved notes 2020-12-10-101050. I use the Fish Shell, If your shell is more bash-flavored the command would be

git add --all && git commit -m \"saved notes `date +%F-%H%I%S`\" && git push

While this works fine for me, I can see this kind of functionality maybe better to be included in Foam. Perhaps the commit notes could be more informative, or it could be set to commit and push commands set to run on a periodic schedule, etc.

[riccardoferretti]

@leejoramo-d51 have you seen https://github.com/foambubble/foam/blob/master/docs/recipes/automatic-git-syncing.md ?
I believe the GitDoc extension is quite remarkable, and I saw they have the ability of filter fiiles:
https://marketplace.visualstudio.com/items?itemName=vsls-contrib.gitdoc#auto-commiting-specific-files

[riccardoferretti]

I am going to close this issue because I believe it has now been fully addressed in https://github.com/foambubble/foam/blob/master/docs/recipes/automatic-git-syncing.md

Feel free to comment/re-open if you feel something is missing