Ranger: [Discussion] Abandon python2 and hug async

[kevinhwang91]

From https://en.wikipedia.org/wiki/Python_(programming_language)

The Python 2 language was officially discontinued in 2020 (first planned for 2015), and "Python 2.7.18 is the last Python 2.7 release and therefore the last Python 2 release."[30] No more security patches or other improvements will be released for it.[31][32] With Python 2's end-of-life, only Python 3.5.x[33] and later are supported.

Why should ranger abandon python2?

1. No one is willing to do work to be compatible with a dead language, especially non-commercial projects.
2. Using python3 with async will improve code quality and performance.

[zyklotomic]

I have asked this exact question on IRC before, quite a while back before python2 was discontinued, and the biggest concern for the project was maintaining legacy support. A lot of sysadmins have uses for ranger in SSH-ed boxes. However, now that python2 is officially discontinued, I see no reason to not slowly start working towards using async.

However, it would be a large structural change that probably needs to be phased in incrementally. I suggest creating an experimental async branch that we could experiment on.

For starters, I have tried tested using asyncio within the main loop itself. This can be found under ranger/core/fm.py under loop() if you want to look into it more. In my testing, I have realized so many functions have to be retrofitted to become async. In short, this is no easy undertaking.

[toonn]

That's exactly the problem. This is a huge change. Plenty of people complain about python 2 compatibility being hard but that's not true ime. It requires checking two versions of the stdlib docs and that's about it.

There's plenty of drive-by contributors who want to contribute some async-based improvement but there's no one so far who wants to put in the effort of simply discussing the design with us. If we accept such a contribution we're left with an additional maintenance burden (async code is harder to understand), we lose compatibility with many versions of python, probably all the way up to 3.7 and all of this for a minor improvement to some niche usecase, disregarding interactions with all the synchronous code.

Then there's the fact that alternative frameworks exist that would not require dropping backwards compatibility (I mean threading specifically). Yes, python's async code is nicer to read, write and reason about but this trade-off needs to be properly evaluated considering the amount of effort this will entail.

[zyklotomic]

Yes, when I said phased in slowly, I was thinking on the magnitude of at least multiple years for sure, since asyncio was only recently an official part of python. Thorough testing from volunteers would be necessary too, if we were to pull this off.

I would be interested in starting such a design conversation. What kind of design considerations would you like to hear from contributors? Even if this is a dead end for now, I think it is good to have a serious discussion on this topic, especially since "wants-async" is so commonly brought up that it is now an issue label.

[kevinhwang91]

FYI, https://github.com/prompt-toolkit/python-prompt-toolkit has abandoned python2 since v3.0.

Please notice that this branch is the prompt_toolkit 3.0 branch. For most users, it should be compatible with prompt_toolkit 2.0, but it requires at least Python 3.6. On the plus side, prompt_toolkit 3.0 is completely type annotated and uses asyncio natively.

ranger can borrow some ideas from it.

[kevinhwang91]

I have asked this exact question on IRC before, quite a while back before python2 was discontinued, and the biggest concern for the project was maintaining legacy support. A lot of sysadmins have uses for ranger in SSH-ed boxes. However, now that python2 is officially discontinued, I see no reason to not slowly start working towards using async.

However, it would be a large structural change that probably needs to be phased in incrementally. I suggest creating an experimental async branch that we could experiment on.

For starters, I have tried tested using asyncio within the main loop itself. This can be found under ranger/core/fm.py under loop() if you want to look into it more. In my testing, I have realized so many functions have to be retrofitted to become async. In short, this is no easy undertaking.

I don't think to refactor from the main loop in step one is good idea, because the ncurse is only synced. We should solve the performance first, to refactor the loader[ranger/core/loader.py] use asyncio and lock instead of mainloop + extra loader. And then refactor the code about preview into asyncio.

If the above steps are well-off, it' will be time to refactor mainloop and abandon python2.