Pelican: Remove Python 2.7 support

[justinmayer]

Official support for Python 2.7 ends in two months, after which no bug fixes or security updates will be released. In preparation for that, it is time to remove code from Pelican related to Python 2.7, as well as removing Python 2.7 from the CI test matrix.

The following checklist tracks the progress of this endeavor:

• [x] sign the pledge to drop Python 2
• [x] remove Python 2 support in .travis.yml
• [x] remove Python 2 classifiers in setup.py, pyproject.toml, and poetry.lock
• [x] remove six from the code
• [x] remove utils.python_2_unicode_compatible
• [x] update CONTRIBUTING.rst anddocs/ accordingly
• [x] remove comments related to Python 2 (e.g., in settings.py & tools/pelican_import.py)

[Lucas-C]

Great plan !
Maybe a checklist of things to do in order to achieve this would be helpful ?
A few items from the top of my head, completing the ones you already mentioned:
_[moved to issue description]_

[justinmayer]

Agreed, @Lucas-C. I've moved that checklist to the issue description. Many thanks for writing that up! 👏

Unless there are any objections, I will submit a pull request to the repo for https://python3statement.org/ in order to pledge our solidarity with the other projects that have committed to dropping support for Python 2 by the end of the year. (cc: @getpelican/reviewers)

I'm working on making the appropriate changes to the CONTRIBUTING and docs/contributing.rst files. Would anyone like to submit pull requests for the other tasks on the checklist above?

[pauloxnet]

I tried to contribute to the Python 2.7 removal with a small PR #2647

[avaris]

I was planning to work on this this weekend (at least CI and code cleaning bits) as part of #2644, mainly because I don't like submitting PRs without a green check mark :).

[iKevinY]

I haven't had the time to contribute to Pelican in a _long time_ and I've found a couple of hours tonight, so I think I'll take a stab at tearing out the Python 2 compatibility (I hope you don't mind @avaris!)

Edit: Work being tracked in #2649.

[avaris]

@iKevinY not at all :). I'll go through the PR.

[t-mart]

To be more specific, which minor versions of Python 3 should the project support with this change?

Do we have any data the breaks down which Python versions people run Pelican in? Maybe from PyPI or something. That might help influence which minor version is targeted.

On the other hand, the answer could also be "whichever version supports the features we want to use". This is a breaking change, so the sky's the limit.

The setup.py implies Python >= 3.5 because of the classifiers, but these should also be hard-enforced with a python_requires argument. I'd recommend this be added to the checklist too. (Actually, this argument should have _already_ been added.)

Since this is a breaking change. it might also be time to do more than just remove Python 2/compatibility things, but also add Python 3(.5 or greater) things.

For example, asyncio could be incorporated:

• I've seen an issue claiming that pelican is slow #2042, and blindly, I'd imagine that post generation could be sped up significantly with this since there are no interdependencies between posts. Jinja also supports async renders if it's Python >= 3.6. (It might be possible to instead just do async calls against the Jinja environment externally...haven't thoroughly tested if the environment is stateful during renders, but it seems like it might not be.)
• Asset copies, a strictly IO task, would also benefit in performance.
• It would be trivial to run the watchers and the webserver together instead of threading them. There are file watchers like watchgod (not super polished) and servers like aiohttp (very polished!) that are async-compatible. This would at least resolve #2400 and make the code a lot simpler if you agree that concurrency is hard.

\

There was another Python 3 thing I was thinking about while I was writing this, but I forgot, and I'm sure there are many more. This is the time to add/remove anything we want, whether that be a Python or Pelican feature.

[t-mart]

Moving to Python 3 probably also has consequences for a lot of the downstream projects, such as the plugins.

[justinmayer]

To be more specific, which minor versions of Python 3 should the project support with this change?

The versions of Python that are maintained: https://devguide.python.org/#status-of-python-branches

Do we have any data the breaks down which Python versions people run Pelican in? Maybe from PyPI or something.

I don't have time to analyze it at the moment, but that data is available. https://packaging.python.org/guides/analyzing-pypi-package-downloads/

Moving to Python 3 probably also has consequences for a lot of the downstream projects, such as the plugins.

We are already in the process of moving plugins to their own individual repositories under the new organization created for that purpose. Given that this is a shared community endeavor (much like Jazzband), it will be incumbent on folks who help out with that migration to ensure Python 3.5+ compatibility as the plugins are migrated. Removing Python 2 support from plugins, on the other hand, is IMHO outside the scope of this issue.

[Lucas-C]

I agree with the suggestion to add python_requires to the setup.py (and hence to this issue checklist)

[avaris]

Even Jinja docs state that async support comes with overhead, so I'm not really sure how useful it is for speeding things up. It actually sounds more like the opposite.

[pauloxnet]

For example, asyncio could be incorporated

I think it's an in interesting proposal but is not related to this issue. After the Python 2 removal we can think about Python 3.x only features (e.g. f-string, async, ...)

[t-mart]

is not related to this issue

Yep, you're right. I apologize for stuffing up this issue.

Would it instead be helpful to create another issue about other breaking changes we could incorporate? I just don't want to miss the opportunity to discuss these kinds of ideas before we harden what the next major version of Pelican looks like.

[justinmayer]

@t-mart: No apologies necessary. You brought up some interesting points. By all means, please feel free to create a separate issue to track other breaking changes we might consider.

In other news, I just submitted a PR (see above) that removes Python 2.x references from the documentation and adds some new development-related tooling.

[pauloxnet]

@justinmayer I saw that you've added this change in one of the latest commit in the file contribute.rst:

Please note that Python 3.6+ is required for Pelican development.

So we're going to support only Python 3.6/3.7/3.8 ?

[justinmayer]

So we're going to support only Python 3.6/3.7/3.8 ?

No. The operative word in the above sentence is _development_. I added that sentence because certain tools, such as Black, require Python 3.6+. For Pelican _end users_, the supported versions will continue to be 3.5+ for now.

[justinmayer]

Many thanks to everyone who contributed to removing Python 2.7 support in Pelican. If anyone later discovers something we missed, please add a comment to this issue.