Client: git remote helper and lfs support

[apeyroux]

I want to use Git LFS with keybase remote helper but :

Remote "keybase" does not support the LFS locking API. Consider disabling it with:
  $ git config lfs.keybase://private/apeyroux/docs.git/info/lfs.locksverify false

Do you plan to add git lfs support?

Regards

[strib]

Thanks for the suggestion. We looked into it already, and found that the LFS protocol can only run over HTTPS. Unfortunately for us, that would require running a server on a port on localhost, which is a non-starter on some platforms (like Windows) due to local firewall restrictions. Maybe we'll revisit this someday or work with the LFS team on tweaking the protocol, but for now I don't see how it can work in a cross-platform way.

[AngryAnt]

@strib What sort of priority does this revisit have? I realise that your team probably doesn't rely on a lot of large binary files versioned, but that obviously doesn't necessarily hold true for your community.

[strib]

Again, it's not about how our team uses git. It's about the fact that to support LFS, we have to run an HTTPS server. However, since I wrote the above, we've actually made some progress on that, not for git but for other stuff.

cc: @jzila @songgao I wonder if we can use the existing file server for LFS somehow.

[AngryAnt]

Apologies. I did not mean to imply that the git usage patterns of your team informs technical issues - just that it could very naturally affect awareness of the problem.

Like many others, I would be very curious about this progress you mention and, if applicable, any advice allowing us to connect dots in order to arrive at a functional LFS/-like setup for teams.

[jzila]

@strib @songgao I'll make an internal ticket to investigate.

[brenthuisman]

Has there been progress on this issue?

[jzila]

We've investigated and have it on our list. But it's relatively low priority right now.

[kav]

Worth noting you don't need an http server if you define a custom transfer type https://github.com/git-lfs/git-lfs/blob/master/docs/custom-transfers.md you get json via stdin. Might make this process significantly more straightforward. Also +1 to the issue in general

[hiradyazdan]

Is there any update on this yet?

[dukejones]

It looks like there is no hard requirement for a HTTP server with the standalonetransferagent config option. Here is a simple example in go:

https://github.com/sinbad/lfs-folderstore

My go is really bad but I may take a crack at it. Something like:

1. keybase lfs install : adds config to local .git/config to use the local binary for keybase:// urls, and no API server. [Think about concurrency option.]
2. keybase lfs : program listening on stdin for tus.io json commands.
3. Parse the init message href, keybase://team// or keybase://private// and upload files to /keybase/team//git-lfs/
4. Do the uploads, returning progress messages
5. Complete.

Similar with downloads.

.git/config something like:

[lfs "customtransfer.keybase"]
    path = keybase-lfs
[lfs]
    standalonetransferagent = keybase

[strib]

Thanks for putting in the legwork here! This looks pretty promising; I didn't realize this option existed. We'll look into it soon!

[AngryAnt]

[Everyone liked that]

[dukejones]

Thank for the encouragement! I would love to have a decentralized storage workflow for managing asset versions for my project using LFS & keybase.

I have verified that my go is not at the level for me to make a substantive contribution here. For the record here are the references I was using to develop:

https://github.com/git-lfs/git-lfs/blob/master/docs/custom-transfers.md

https://github.com/git-lfs/git-lfs/wiki/Troubleshooting

Many thanks! I am continually impressed by keybase. Such amazing work.

[strib]

Good news everyone! Using @dukejones's suggestion, the current master version of Keybase now supports LFS. If you use the Linux nightly build, you can try it out. See some info on how to set it up at https://keybase.io/docs/git/index#support-for-git-lfs-large-file-storage- .

This has only been lightly tested, so let me know if you run into any issues.

If you're not running the Linux nightly build, you'll be able to try it out as of our next release, 5.1.0, out in a couple weeks.

[dideler]

It seems the LFS documentation has been removed from https://keybase.io/docs/git/index#support-for-git-lfs-large-file-storage-. Any idea where it's been moved to?

[nandalopes]

Run keybase git lfs-config from command line. This will add the following settings on your git/config file:

[lfs]
    standalonetransferagent = keybase-lfs
[lfs "customtransfer.keybase-lfs"]
    path = git-remote-keybase
    args = lfs origin <keybase remote url>