Julia: Pkg, and git clone depth.

[Carreau]

Hi all,

It look like when Pkg.init() or Pkg.add(...) install some packages, git does a full clone.

Would it make sens to clone only with a limited depth as most people will not push/clone from theses repo ?

[StefanKarpinski]

The whole idea of using git to install packages is that when someone installs a package, they already have all of its version history and are ready to make changes and contributions to it easily. Making shallow clones would defeat the purpose of using git in the first place. I don't see this being a huge issue since git packs are quite efficient – often the entire packed history of a repository is smaller than the compressed tarball of just the current code [citation needed].

[Carreau]

It used to be true that you couldn't push or clone from a shallow clone, (so contribute). But I don't think this is the case anymore. I still think that when doing a shallow clone, it should be consequent enough for you to get some versions, and completely understand your point.

Anyway, I mostly pointed this out as my installed was screwed up, and it feel like especially Homebrew package was quite long to download (But I guess that could be due to the package itself). And for new users, I know that the time to install packages can be quite important for the first judgement you get for a language.

Feel free to close if you wish.

[ihnorton]

@Carreau see https://github.com/JuliaLang/julia/issues/4545 and https://github.com/staticfloat/Homebrew.jl/issues/13

In the long run it would be nice to optimize this if we can, but at this point the most active Pkg repos only have a few hundred commits AFAIK. It seems that the problem with pushing from a shallow clone only arises when the repo histories have diverged such that a common ancestor cannot be found. But a) this should probably be less of a problem with a managed system like Pkg and b) it is pretty easy to re-up your repo if needed by pulling in more history.

[quinnj]

I think I remember discussion a while back about the future when static compilation will be achieved and packages can be compiled down smaller/more efficient. Maybe at that point a user could Pkg.add for just the compiled source, and Pkg.adddev to get the full repo........not sure if that makes sense or not though. I'll try to look back over to find the discussion.

[Carreau]

I know that julia is still an early dev language, and really focused to dev that are not afraid to bugfix, and get their hand dirty. I feel more and more that requiring git for a project is not such a big deal, I just want to point at that in the long term it will make (some) user run away. I'm reminded of that every time I have a hard time getting git protocol go through my work proxy, and everytime I got home and do not have a 100Mb symmetric connexion.

Sadly, I see for example colleague with MacBook air trying to reduce their disk usage at maximum (who can live with 64Gb today in science...) and they start removing hevyweight software, and they use that as argument against versioning ("Cause it keeps all version, and they need only the more recent one") .

In the meantime I agree that requiring full repo download and git is a good way to keep people with not enough technical knowledge to use julia at bay, and I'm fine with it. I was just wondering if you had thought about that
and my question has been answered.

Thanks.

[jingpengw]

To deploy an application, people only need one version, and would like the docker image to be small for faster deployment and saving some cloud storage space. It would be nice at least have an option to use one version without history.

[KristofferC]

This will be possible in Pkg3.