Pkg.jl: Make `activate` instantiate by default

[davidanthoff]

Would it make sense that activating an env automatically instantiates that env, unless one explicitly opts out?

We are starting to prep a fair number of replication repositories for stuff we publish. All of them have a Project.toml/Manifest.toml in their root, but now we need a fair bit of explanation in our READMEs to walk folks through activating and instantiating of these environments. That is a lot of concepts that our average users won't have heard before...

I guess my other question is: isn't it way, way more common that I activate an env and then actually want it to be instantiated, rather than the other way around? In fact, what is the scenario where I want an env activated but not instantiated? Those seem really corner cases?

If it wasn't fully automatic, maybe there could be at least a prompt that offers this? Something like "You activated an env that still needs to download stuff. Do you want to download these things now? y/n".

[StefanKarpinski]

We could add an opt out option like activate --instantiate=no and have it default to yes.

[davidanthoff]

Yes, I think that would be great. Probably needs a command line flag as well? So that one can also opt out of julia --project foo.jl instantiating?

[StefanKarpinski]

Yeah, that's the complication. It seems a bit much to automatically instantiate just because you started Julia in a directory with the --project flag. Maybe we can prompt for instantiation the first time you try to load something that isn't installed.

[KristofferC]

but now we need a fair bit of explanation in our READMEs to walk folks through activating and instantiating of these environments.

Do you have an example? Since it is literally just running Pkg.instantiate() after activating the project for the first time, I wonder how much explanation is needed. The error message you get when you try to load a package before it also tells you to run this exact command.

I really like that activate is instant and doesn't try to download and build a bunch of stuff. You can get a nice view of what the project contains with st, you might want to remove some packages before instantiating etc.

[davidanthoff]

Maybe the default should be a prompt with an option to save your answer? Say:

Some of the packages in the active environment are not installed.
Do you want to install them now? yes / no / always / never

That would require some way to save the answer as a config setting, but I think that might the option that accommodates the widest range of users?

Do you have an example?

It literally is just an academic paper repository that has replication code (not public yet). But the audience we have their typically has never heard of julia before, and I'd much prefer instructions a la "Install Julia, then run julia --project src/main.jl". Yes, it is only a few commands more today, but it introduces a lot of concepts that these users won't be familiar with, which I think is not great. They might literally not know what a package in the julia sense is, what activating means, what instantiating means etc.

I really like that activate is instant and doesn't try to download and build a bunch of stuff. You can get a nice view of what the project contains with st, you might want to remove some packages before instantiating etc.

Yeah, I completely see that this is the preferred option for some folks that are familiar with julia, but I think there is another user group out there for which this is entirely irrelevant and where this becomes a distraction.

[KristofferC]

They figured out how to download and install Julia, clone the paper repository, cd to it etc, but saying

The first thing to do is to run `julia --project -e 'import Pkg; Pkg.instantiate()' which installs everything that is needed for the code to run

is where the limit is? Has this really been a problem in practice?

[Datseris]

Hi, I would just like to chime in and say that this would be a big and bad hit to DrWatson. As @KristofferC said, activating should be instant, and there are a lot of benefits to it being instant, as for example knowing for where to load packages, but most importantly using DrWatson instantly.

As a scientist, I switch several Julia projects during my work day, each corresponding to a different scientific project. If every time I switch I would have to wait for the registry to update, I would be a sad man.

If you add this, please consider not making it the default behavior, although I fully agree with Kristoffer, I see no reason that this should happen.

[davidanthoff]

I don't think this would require hitting the registry every time you switch, presumably it would only hit the registry if you switch _and_ there is a package in your new env that isn't instantiated. I completely agree, this feature would have to be implemented in such a way that switching between env that are fully instantiated is still instantaneous.

[davidanthoff]

Has this really been a problem in practice?

In general, very much yes. My lab is about 10 folks that use julia pretty much daily for their scientific work, but are not package contributors (roughly). My not super systematic sense is that maybe half of them are still not using environments in any systematic sense, and when I chat with them about it, it is pretty clear that they find the concepts confusing and don't fully understand the story. The other half is using environments, but it took me quite a while to convince them, and explain things.

But our work is actually targeting folks that use julia _much_ less than daily. We want our replication code to be easily usable by folks that might not have used Julia ever. I'm worried that a) if the folks that use this daily (but have no interest in CS topics, i.e. they are really users that don't show up on github, the forum etc.) already have quite a jump to make to grasp these concepts, then I think for even less involved users this will be even more tricky, and b) I'm worried that if the first experience of Julia these folks see is a complicated packaging story to even just run a quick replication, we are losing an opportunity to impress new users how easy things are.

[StefanKarpinski]

The alternative is to prompt for install when someone runs code and the required package isn't installed. That seems just as effective since the user just needs to hit y to agree but safer since they still have to approve to installing the packages that they're about to run.

[davidanthoff]

I guess that would also work, but on the other hand it would be nice to not have multiple prompts in a row, i.e. one prompt for package A, then a few seconds a prompt for package B etc. Would be nicer if there was just one prompt a la "your env is not instantiated, do you want to instantiate it?". But I guess the trigger for that could be the first using statement that fails.

[StefanKarpinski]

It could list all the packages that aren't installed in one prompt.