Joplin: [Feature request] Git as a storage provider

[harmancasvi]

Operating system

• Windows
• macOS
• Linux
• Android
• iOS

Application

• Desktop
• Mobile
• Terminal

First of all, I thank all the good people who make this app possible. I'd code to make it better but you'd get more trouble than benefit from my coding skills hahaha...

Heres's my suggestionion: please implement git as storage provider. Like GitHub or GitLab work as the storage provider for pass, that could even give us versioned notes 8-)
Im thinking about using a single git group/sub-group/repo, whichever picks the user, no matter if the repo is public or private, we'd restrict Joplin to a repo and down there our notepads as directories, with sub directories and notes.

This app uses gists for git notes on the CLI, you could check it out if it helps.

Thank you!

[mschaaf]

I use a remote git repository already for sharing between devices:

1. create a repo in github, gitlab or anywhere else
2. checkout on desktop via git
3. checkout on android via mgit
4. use ssh keys to acces the repo
5. sync to filesystem on your desktop and android device
6. on the desktop i have a script that automates the synchronisation and commit and push process
7. on android it is a manual step of commit and push via mgit

[harmancasvi]

@mschaaf thank you for the suggestion, I know I can sync via git, not as an app supported feature tough.

The benefit of having it integrated into the app is that it would run on Android AND iOS without extra effort. It's also good to use the app's internal storage [on mobiles] and not having to git checkout, git commit and git push manually while relying in some folder in the public storage.

The reason I provided pass as an example is that you setup the git related info and they take care of the sync process before/after every change, just like using any other provider (but better!).

Don't get me wrong @mschaaf , I do appreciate your guide (it might be very useful for git newcomers), it's just that I'd like to see the app integrate such an important feature without needing my own tricks to make it work on external folders.

Regards.

[stv0g]

I think adding Git support might be possible by extending the existing directory backend.
On the desktop clients, we could simply rely on the system Git installation.
However, this becomes more difficult on the mobile clients. Bundling a Git version seems a bit inelegant.

I am wondering how the Android pass app is handling it?

[virtadpt]

@stv0g

https://github.com/zeapo/Android-Password-Store/tree/master/app/src/main/java/com/zeapo/pwdstore/git

[stv0g]

@virtadpt Thanks. Seems like they use jGit

[harmancasvi]

I use that for password management, I hadn't thought of using a specific folder for notes, will try it that way. Thanks!

[cbop-dev]

There's also isomorphic-git, an open-source javascript git library which works for node, browsers, and android, and seems to be actively developed.

[goodevilgenius]

I made this work myself by syncing to filesystem and using a local git repo as my target. And then I set up a very simple auto-syncing mechanism for that repo to regularly push/pull from remote. I even managed to get it to work on Android with the addition of Termux and Tasker.

Here's my code to make it work: https://gitlab.com/danjones000/simple-git-sync

[berot3]

from #1619:

That's not technically possible at the moment.

But maybe we can still leave this one open so it won't be forgotten to think of in the future! 😃

[virtadpt]

Please. I don't have a use case for it right now, but I can think of quite a few of them (least of all, Keybase's Git server functionality).

[stale[bot]]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[harmancasvi]

Any news on this? Is it really just me who wants tis feature so badly?

I can't code properly that's why I haven't done it myself, but I don't think it would be that difficult for a proper programmer.

Passwordstore looks like good place to look IMO. Has any kind developer had the opportunity take a look at @stv0g or @cbop-dev 's suggestions?

[virtadpt]

It's not just you. I'd love to have it. I was able to work around it with WebDAV, which is why I'm not complaining too hard about it.
If it wasn't an Electron app, I'd be working on it, too.

[stv0g]

I am also still interested in this feature

[tessus]

I think a git storage provider is not going to happen. Please be aware that Joplin is a react-native app. Git would require us to add key support to Joplin, which could be tricky. (Tricky, because of react-native.)
Besides that, how would one handle merge conflicts? Joplin is not a git client, nor does it have a diff engine included.

I understand that git sounds great as a storage provider, but the architectural design to get it right is out of the scope of this project. But maybe I'm wrong. @laurent22 makes the decisions, so let's see what he has to say.

[virtadpt]

Maybe this would be a good place to start?
https://github.com/isomorphic-git/isomorphic-git

[Khorsan]

Besides that, how would one handle merge conflicts? Joplin is not a git client, nor does it have a diff engine included.

@tessus In my point of view, git would be just used as a storage provider addon (the storage would be local). Joplin doesn't need to do diffs, i think. Diff is a git feature and you could do that on any cloud host (Github, GitLab, etc).

[tessus]

Ok, then, please explain the process to me, because I really don't know how this could work in any other way than starting a new project.

git follows a distributed approach, which means you always require a local copy, so if you really wanted to use git, you'd have to use a local copy and a remote. The remote requires authentication, which I've already explained is hard to implemenent due to the react-native code base.
So who fixes the pull/push conflicts? Are you going to do that manually? Because Joplin can't do that for you. Joplin is not a git client either, so we'd also need a git component in Joplin.

Another discussion point:

Are you talking about the data at rest or the sync target? Don't forget that we use a sqlite db to store local notes. The entire code dpends on it. So rewriting that part of the code would be excessive. And then you still had the problem of rsolving conflicts between the data at rest and the sync target.

[Khorsan]

If you perform a commit each time you save, I can't see when would be a
conflict. Maybe if one doesn't have Network connection in both devices.. I
have to give this a bit more thought

A quinta, 5/03/2020, 21:01, Helmut K. C. Tessarek [e-mail]
escreveu:

Ok, then, please explain the process to me, because I really don't know
how this could work in any other way than starting a new project.

git follows a distributed approach, which means you always require a local
copy, so if you really wanted to use git, you'd have to use a local copy
and a remote. The remote requires authentication, which I've already
explained is hard to implemenent due to the react-native code base.
So who fixes the pull/push conflicts? Are you going to do that manually?
Because Joplin can't do that for you. Joplin is not a git client either, so
we'd also need a git component in Joplin.

Another discussion point:

Are you talking about the data at rest or the sync target? Don't forget
that we use a sqlite db to store local notes. The entire code dpends on it.
So rewriting that part of the code would be excessive. And then you still
had the problem of rsolving conflicts between the data at rest and the sync
target.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/laurent22/joplin/issues/753?email_source=notifications&email_token=ABQVF3TZLIWOI6IM4VB6ROTRGAHLHA5CNFSM4FS3J4LKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEN64IKQ#issuecomment-595444778,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABQVF3XFEGXNWTSOXWCDQR3RGAHLHANCNFSM4FS3J4LA
.

[Drallas]

I also would like to have GitHub / GitLb Sync support. I'm syncing to a local folder that is a local Git Repo. But native support would be nicer!

[GETandSELECT]

@mschaaf

on the desktop i have a script that automates the synchronisation and commit and push process

please share the script in a gist

[Drallas]

@mschaaf

on the desktop i have a script that automates the synchronisation and commit and push process

please share the script in a gist

I use something like this plain and simple script :
https://gist.github.com/Drallas/8c57dc87c05648d5cbac4f0a743d6ce5

1. My Joplin Sync Folder has a .git repo connected to upstream Git Repo.
2. My Joplin Syncs with File System to this Joplin Sync Folder.

And an alias in my .~/.bash_aliases file.
alias jopsync="/path/to/joplin/sync/folder/joplinsynctogitrepo.sh"

[mschaaf]

@GETandSELECT the script can be found here https://gist.github.com/mschaaf/0fd33587d3d6220abdff4a391a68cf83
It should be as system independent as possible so it should be only necessary to change the path to the repository.

[goodevilgenius]

@mschaaf

on the desktop i have a script that automates the synchronisation and commit and push process

please share the script in a gist

I mentioned mine earlier in the thread, but here it is again: https://gitlab.com/danjones000/simple-git-sync

I use this for a number of things, and it works really well.

[stale[bot]]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[captainwasabi]

bad bot

Still interested

[stale[bot]]

Hey there, it looks like there has been no activity on this issue recently. Has the issue been fixed, or does it still require the community's attention? This issue may be closed if no further activity occurs. You may comment on the issue and I will leave it open. Thank you for your contributions.

[stale[bot]]

Closing this issue after a prolonged period of inactivity. If this issue is still present in the latest release, please feel free to create a new issue with up-to-date information.

[FilBot3]

Even though this is closed, some how GitJournal, which is an Android application can create a repository on GitHub, create a deployment key, and then sync your plain markdown files between your android device and github. Then I can use whatever markdown editor on my desktop that I want.

https://github.com/GitJournal/GitJournal

I was just having issues with it and my BT keyboard where it would crash whenever the tablet disconnected the BT.

[thwaller]

Knowing this is closed, I do have comments that directly apply to this topic.....

I was looking for this ability yesterday and long story short, I wonder why I would want Joplin vs using something like Atom, or even VSCode, as the note taking software? The tree structure of notes is basically the same as directory structure, and the notes could simply be a series of markdown files.

Curious as to what others think on that topic...

[harmancasvi]

Curious as to what others think on that topic...

@thwaller the thing is that I can't use VSCode on my phone. Not having the ability to use the app without having to manage the notes in a public folder, outside of the app's sandbox with a 3rd party app, that's what doesn't sound right to me and it's one of the reasons why I don't use Joplin anymore (that and the fact that Joplin doesn't encrypt it's own DB).

For the people looking for a good alternative at least on android, I moved to GitJournal, it might not be a complete replacement in some cases, but as I said in another comment, the mobile experience is one of the reasons holding me back from using Joplin, because of the lack of built-in git support.
One thing about GitJournal for you people, I use it only on android, on my desktop I use VSCode or something else to view/edit my notes, although most of the time, I write and read my Git notes on my phone, when walking to a server in the data center or something like that, so my phone is the device I use the most to view/edit notes.

I feel like GitJournal has added a lot of features in very little time, it wouldn't surprise me if they implement everything that Joplin has very soon, with the fast work they are putting out there it really wouldn't surprise me. The multi platform app is one of the main drawbacks with that alternative.

[harmancasvi]

A comment to @laurent22 on the features of the app and the responses to the people in the issues:

The good thing with Joplin is that it's simple (for the end user, they don't care about code quality), multi-platform, OSS, etc. But the bad thing is that with the limited work force, most of the useful features get declined or dismissed with the reasoning of it being _"not necessary"_, or because _"you could use this or that hack to accomplish the same"_.

I know @laurent22 has already a lot of work, and I don't want to sound like a d!ck for saying this, but I kinda feel like the responses fall into the category of _"I won't do it because I don't need it, and if I don't need it, you shouldn't need it either"_. It's ok, I know it's a lot of work and not so many people contribute to alleviate that burden, but I don't think that's the right way of replying to people who are willing to use your software, and have valid concerns or suggestions. In the end the software is yours and is up to you to decide whether or not to include a feature, I just say, if you reply in a certain way to people, they will get demotivated and stop them from using your software or contributing to your project (myself included due to the lack of some key features to me).

I don't want to sound like I have some importance or you owe me anything, of course not, I'm just pointing that less people = less popularity for your app, and less popularity also means less contributors so, it's something to keep in mind.

Also, @laurent22 your approach to new features -as I said- comes of like a disguised rejection rather than _"I'll actually consider that"_. Not saying it is, it's just the vive that your replies project at least to me.
Take #289 as an example: people provided you with valid use cases for partial or full encryption, and you just kept recommending them Veracrypt, without even considering the multiple use cases they exposed. That's like Chrome or Firefox saying to people, that they will not encrypt the password database in your local profile because the OS should be already secure enough, or that they should use an encrypted container to prevent people from revealing the DB; I mean, come on!
A lot of people are hesitant to use Joplin due to the lack of full encryption of stored notes (myself included), and what's more, other note taking apps have had that feature for a long time: guess what, people love it, sure not everyone, but the ones who care, DO like that a lot, and they go as far as migrating to a new -or even their old- software if they don't receive that feature in your app.

The whole point is: you don't have to implement that feature on your app, you're not obliged to, and that's fine, but don't underestimate people's needs or use cases, that's the people who use your app, and if they are giving you the examples of why they need that option from the competing apps, -I think- you should at least listen to them, not necessarily to implement the feature, but to understand why people use this kind of software and what's important to them.

Again, thank you for your hard work and the awesome app you have built with much effort, I hope you don't get upset by my appreciation of the situation, and I hope it's not a bad thing to post this here, I don't know where else to post it and make it relevant the topic.
I really hope one day your app meets my needs and I can use it again, I like the concept. Wish you the best of luck with the project, you've done amazing until now, and I hope you keep growing.

[FilBot3]

I LOVE the idea of Joplin, and its near true cross-platform, FOSS, software that comes in an AppImage. As @harmancasvi mentioned, I love the idea of the app. I just have my own preferences and a lot match up, but others don't.

1. Cross-Platform (Windows, Linux, MacOS, iOS, Android, wherever NodeJS runs)
2. FOSS
3. Source Controlled files
4. Import/Export to many formats (using pandoc or whatever)

I hope to see more from this application in the future. I've turned many people to this app recently who were annoyed with Evernote or OneNote and wanted something to sync markdown in.

[Khorsan]

That's what I was looking for. But I also wanted to take notes on my phone
and have it automatically sync to git

On Sat, Sep 26, 2020 at 4:40 PM thwaller [e-mail] wrote:

Knowing this is closed, I do have comments that directly apply to this
topic.....

I was looking for this ability yesterday and long story short, I wonder
why I would want Joplin vs using something like Atom, or even VSCode, as
the note taking software? The tree structure of notes is basically the same
as directory structure, and the notes could simply be a series of markdown
files.

Curious as to what others think on that topic...

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/laurent22/joplin/issues/753#issuecomment-699511621,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/ABQVF3QYABEJJG4WSL7GBFDSHYDOZANCNFSM4FS3J4LA
.

--
Francisco Castel-Branco