Isomorphic-git: Implement Partial Clone

[willstott101]

We'd love to see https://git-scm.com/docs/partial-clone support in isomorphic git. Which could be very useful for e.g. browsing a plain git server or just generally downloading parts of a repo on-demand.

This seems especially relevant in a web context where we could significantly reduce time to commit, if a full clone is not necessary (even with a shallow depth that can be expensive)!

[willstott101]

Related to #711 & blocked by #585

[willstott101]

Looks like blob filters are supported by GitLab out the box now. I'll give it a try.

[willstott101]

Both GitLab and GitHub seem to support --filter=blob:none!

$ git clone --bare --single-branch --depth=1 --filter=blob:none https://gitlab.com/gitlab-org/gitlab
Cloning into bare repository 'gitlab.git'...
warning: redirecting to https://gitlab.com/gitlab-org/gitlab.git/
remote: Enumerating objects: 7151, done.
remote: Counting objects: 100% (7151/7151), done.
remote: Compressing objects: 100% (5316/5316), done.
Receiving objects: 100% (7151/7151), 1.51 MiB | 7.48 MiB/s, done.
remote: Total 7151 (delta 5), reused 5666 (delta 3), pack-reused 0
Resolving deltas: 100% (5/5), done.
$ git clone --bare --single-branch --depth=1 https://github.com/gitlabhq/gitlabhq gitlabhq-all
Cloning into bare repository 'gitlabhq-all'...
remote: Enumerating objects: 31411, done.
remote: Counting objects: 100% (31411/31411), done.
remote: Compressing objects: 100% (28655/28655), done.
remote: Total 31411 (delta 2389), reused 16690 (delta 1482), pack-reused 0
Receiving objects: 100% (31411/31411), 87.89 MiB | 14.83 MiB/s, done.
Resolving deltas: 100% (2389/2389), done.
$ git clone --bare --single-branch --depth=1 --filter=blob:none https://github.com/gitlabhq/gitlabhq gitlabhq-noblob
Cloning into bare repository 'gitlabhq-noblob'...
remote: Enumerating objects: 4657, done.
remote: Counting objects: 100% (4657/4657), done.
remote: Compressing objects: 100% (3517/3517), done.
remote: Total 4657 (delta 5), reused 3268 (delta 4), pack-reused 0
Receiving objects: 100% (4657/4657), 1.08 MiB | 2.66 MiB/s, done.
Resolving deltas: 100% (5/5), done.
$ du -h -d 1
89M ./gitlabhq-all
1.4M    ./gitlabhq-noblob
1.9M    ./gitlab.git

The clone from gitlab is bigger probably cause the mirror on github differs somehow :shrug:

[wmhilton]

Yes indeed! I think partial clone would be very cool, and I did some experiments a while back and reached a similar conclusion, that --single-branch --depth=1 --filter=blob:none results in _much_ faster initial loads... potentially fast enough if you were building an online code editor that you could do it on page load to populate a file tree and then dynamically retrieve blobs only as a user clicked to open & edit the file. I hypothesize (but didn't prove out) that you could also build new commits without having to download all the blobs. It would be amazing!

I was hoping I'd get to implement this as part of my work on Stoplight Studio, but it doesn't look like that's going to happen anytime soon. 😞 I did _start_ on Wire Protocol 2 though, and support getting the list of remote branches via Wire Protocol 2 (which is nice bc of server-side filtering so you don't get a huge response with every PR branch that's ever been opened). Oh no! The docs website doesn't have it? It's called listServerRefs bah now I have to go see why it's not showing up on the docs website.

Edit: I fixed the website. https://isomorphic-git.org/docs/en/listServerRefs