Flux-core: python 2 end-of-life

[SteVwonder]

@trws's comment in #2390:

Since python 2 EOL is coming imminently, it might be worth checking with our customers to see how long we need to hang on to python 2 support. Alternately, we may require python3 and optionally additionally build python 2 bindings so we don’t have to deal with this stuff in our own commands.

The latter suggestion seems like a great way to go in the short to medium term. I'm afraid that python 2 will live on at the lab for quite a few more years 😭 I'm guessing up until the point that Red Hat stops supporting it (any maybe even a little while longer after that). So complete removal may be a ways off.

[trws]

Given the RHEL/TOSS cycle, I'm predicting ~6 years since RHEL 8 still has python 2 available. That said, if we do a bit of rework to be able to build bindings for both the 3 by default option should be pretty easy, and let us clean up some nastiness as well (not as much as we might like, but at least for our commands and modules and whatnot). Would give us a way to include async/await support for our stuff too come to think of it.

[SteVwonder]

Based on a recent coffee discussion (and discussions elsewhere on GH: https://github.com/flux-framework/flux-core/pull/2482, https://github.com/flux-framework/flux-core/issues/2480): one plan going forward would be "fork" the python bindings into python3 and python2, with the python3 staying in-tree and the python2 being spun out into a separate repo. To ensure that any breaking changes to the python2 bindings are caught quickly, we can add a flux-core travis builder that pulls in the python2 bindings and run their tests. The in-tree bindings would then be free to drop py2 support and thus the six module. Additionally, we can push forward on the async support and start using things like f-strings

[jameshcorbett]

@SteVwonder I seem to recall (and #2480 suggests) that Flux was ready to drop support for Python 2 if it wasn't for the UQP. Is that still true? If so, I could pretty easily include a switch in our code so that we use Flux's CLI for Python 2, and either the CLI or the Python bindings for Python 3. We're almost certainly going to keep an implementation that uses the CLI anyway--because that's how we interact with LSF and Slurm--and right now our code doesn't even use Flux's python bindings.

Sorry, I should have thought of this sooner. Let me know if it's something you'd like me to do.

I still have some hope that the UQP can switch to Python 3 before long...

[SteVwonder]

I could pretty easily include a switch in our code so that we use Flux's CLI for Python 2, and either the CLI or the Python bindings for Python 3.

That sounds like a reasonable solution. You could also use the python2 bindings fork as well, but they will quickly become a second-class citizen before being deprecated and finally killed off. So your solution would keep you on the well-maintained path for both python versions.

Maybe @trws has thoughts as well?

[trws]

I could pretty easily include a switch in our code so that we use Flux's CLI for Python 2, and either the CLI or the Python bindings for Python 3.

That sounds like a reasonable solution. You could also use the python2 bindings fork as well, but they will quickly become a second-class citizen before being deprecated and finally killed off. So your solution would keep you on the well-maintained path for both python versions.

Maybe @trws has thoughts as well?

I agree with all of this. If it's something you can do, or especially
will do anyway, it's even worth considering whether we'll maintain the
python2 bindings fork in the first place. All new work, except for
critical bugfixes etc, is planned to be on the python3 version either
way.

[jameshcorbett]

I went ahead and implemented the switch, so as far as the UQP is concerned, you can drop all Python 2 support in the bindings.

[SteVwonder]

Closed by #2805