Pants: Allow for reloading of python code changes in a repl or development session

[rmangi]

One of the frequent complaints we get from developers in our pants environment is how long it takes to reload a repl or restart a service they're working on due to the chroot creation overhead. In a non-pants world we could use imp or ipython autoreload in a repl and the restart time for an api-server outside of pants is usually negligible. The feedback loop time is challenging.

Would it be possible to sync local code changes or even symlink to it from the chroot to enable something like this?

If this is actually something that's possible today and we're not aware of it that would be great too :)

Thanks!

[Eric-Arellano]

Thanks for opening this, @rmangi. We're actively working on speeding up the performance of ./pants repl and ./pants run, particularly when using the V2 implementation.

Right now, the biggest issue is that a change to your source file will mean that we re-resolve all your 3rd party requirements, even though that's not necessary. This is our first priority to fix.

As mentioned on Slack, it's possible to have this auto-reload mechanism via the option --loop. Once we make these improvements to the repl goal, we expect auto-reloading to work peformantly.

[Eric-Arellano]

Hi @rmangi, update that we made the first major performance enhancement in 1.27.0rc0 for the V2 implementation of repl. We no longer re-resolve requirements unless those have changed.

We don't have hot reloading working quite yet. If you make a change, you'll still need to close out of the REPL and restart it with ./pants repl, unfortunately. But, that should be much less painful than it was before, albeit still not ideal.

We do plan to put some time into getting hot reloading working properly. It's not an immediate priority, but it's something we're thinking about.

--

Let us know either on this issue or on Slack if you run into any issues with this! I don't recall if you were using the V2 implementation of REPL before? If not, we'd love to help you get that set up.

[rmangi]

Geraet, we're using the v2 repl. We'll upgrade and see how it goes.

[rmangi]

Hey Eric, we're actually not using the v2 repl yet, we'll get that going. I'm not on the pants slack but the rest of my team is and they say I should be too so I'll get on that. Thanks!

[Eric-Arellano]

We'd love to have you join Slack! See https://pants.readme.io/docs/getting-help-1 for instructions on how to join.

[stuhood]

This also applies to Django runserver, and possibly also to jupyter. cc @benjyw

--loop (Pants' existing "watch" mode) does not immediately work for this usecase because it won't kill an InteractiveProcess or @_uncacheable rule (including @goal_rules) if a file has changed under it. In some cases it should... it's possible that goals/processes/nodes should be marked interruptible/restartable (which would split that property out of the @_uncachable decorator).

It's also possible that this is a user-by-user decision, rather than being goal/tool specific, such that we would have --loop and --loop-restart (or --watch/--watch-restart), with an override, but the default continuing to be that we don't restart a process until it has completed.

[butlern]

👍 Just ran into this testing out pantsv2.

Our use case is as follows:

Developers want to run a local django server locally with pants run command which ultimatley calls django's runserver command. They want to make code changes and have the server restart automatically on detected code changes.

[benjyw]

That is likely an extremely common use-case, and one we should definitely be able to support.