Obspy: Dropping Python 2

[megies]

This is just intended as a place to summarize any things to keep in mind once we drop Python 2.7.
Just add bullet points for any things that should be done after dropping Py2K, e.g. making use of Python 3 only features in code.

High Priority

• [ ] bump major version to 2
• [ ] drop basemap, full switch to cartopy
• [x] remove usages of the future package (see #1752 and #1613)
• [ ] make sure docs discourage using pip < 9.0 or setuptools < 24.3
• [x] make sure docs discourage using the setup.py directly (favor using pip)
• [x] raise an informative exception if the user tries to import obspy > 2.0.0 on python 2 as recommended here
• [x] raise an informative exception if setup.py is run by python 2 as recommended here
• [ ] replace usages of urllib with requests (see #1254)
• [ ] add obspy to the python 3 statement timeline (see #1764)
• [x] bumpy dependecy versions, e.g. numpy to 1.15, see NEP 29

• [ ] ...

  Low Priority

• [x] remove usages of the __future__ imports (perhaps only until pep 563 is applicable)

• [ ] add type hints to public functions/methods (or use stub files?)
• [x] maybe can revert part of #2167? Mostly affects Py2 I think
• [ ] use pathlib
• [ ] ...

Handle some deprecations on 3.7

• [x] DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working

[d-chambers]

Would it make sense for a major version bump after dropping 2.7?

[krischer]

Would it make sense for a major version bump after dropping 2.7?

Yes, I think we should do this. Seems to be how all the other big projects handle it as well.

In terms of actually doing it we could follow some form of this strategy: https://blog.jupyter.org/release-of-ipython-6-0-2068cfb2c2a

Probably on in light-version as might not have the manpower to do it at full scale.

[QuLogic]

I applied the changes that IPython did into Matplotlib; it is fairly straightforward and should be easy to do for ObsPy.

[d-chambers]

@QuLogic are you referring to the changes highlighted here: http://python3statement.org/practicalities/ ? I will add a few of these to @megies list.

[QuLogic]

Yep, basically those things.

[hugovk]

A handy tool called pyupgrade can help you upgrade syntax and remove some dead wood once you're ready:

pyupgrade `find . -name "*.py" -type f` --py3-plus

https://github.com/asottile/pyupgrade

[heavelock]

How about dropping at the same time support for 3.4? Some of big projects such as Matplotlib drop its support along with 2.7.

[megies]

How about dropping at the same time support for 3.4? Some of big projects such as Matplotlib drop its support along with 2.7.

Debian Jessie has LTS until June 2020 and has Python 3.4 so I would rather only drop Python 3.4 if there's a good reason.

[krischer]

Dropping 3.4 and jumping straight to 3.5 would enable us to use the typing module which would indeed be quite nice and give us a chance to clean up the documentation a lot. Also, the generalized unpacking in 3.5 is quite handy to have around.

But similar arguments can be made for some things in 3.6 so we have to make a cut somewhere.

My vote would go to jumping straight to 3.5 and possibly even to 3.6. Dropping Python 2 support will require a major version jump in any case and we can still bugfix the old version which will also keep running on all the LTS versions. I feel like most people these days install conda in any case so they are less dependent on system packages.

[megies]

Well, in any case, with the current major release frequency or well.. period, rather.. of 1-2 years the discussion about py3.4 might be completely obsolete anyway, since it's gone for sure in 3 years.

[heavelock]

Another dependency that supports currently 3.5+ is pyproj https://github.com/pyproj4/pyproj/releases/tag/v2.3.0rel

[trichter]

I think it's time to warm up the discussion about the supported python versions for obspy 2.0.

According to the table in the Python Developer Guide, Python 3.5 will be end-of-life in 09/2020. Can we directly jump to python 3.6?

[hugovk]

NEP 29 suggests Scientific Python projects support Python 3.6+ as a minimum:

This NEP recommends that all projects across the Scientific Python ecosystem adopt a common “time window-based” policy for support of Python and NumPy versions. Standardizing a recommendation for project support of minimum Python and NumPy versions will improve downstream project planning.

https://numpy.org/neps/nep-0029-deprecation_policy.html

[megies]

This question is mostly about what package based distributions we aim to support. Obviously you can just use a local Anaconda anywhere, but having ObsPy in the system Python can be pretty neat.

From the De/buntu perspective, we have Python 3.5 in "stretch 9" (EOL 2020, EOL LTS 2022) and "xenial 16.04LTS" (EOL 04/2021, EOL LTS 04/2024).

I know from personal experience that these LTS windows usually get used till pretty much the last day of security support on a lot of server machines where it can be useful to have ObsPy in the system Python.
So how big of an advantage would dropping 3.5 have? f-strings mostly? Is it worth not having system packages on some distributions?
But I'm open for discussion.

[trichter]

From the De/buntu perspective, we have Python 3.5 in "stretch 9" (EOL 2020, EOL LTS 2022) and "xenial 16.04LTS" (EOL 04/2021, EOL LTS 04/2024).
So how big of an advantage would dropping 3.5 have? f-strings mostly? Is it worth not having system packages on some distributions?

For me it does not matter much. Python 3.5 feels modern enough :)

[heavelock]

@megies How about 3.4? Are you still for keeping support for it?
In my opinion 3.5 is a minimum since it has typing.

[barsch]

3.4 is a pain on Windows as it needs extra compiler - so I opt for Python >= 3.5

[megies]

3.4 has been out of security fixes for a full year, and Debian/Ubuntu have two supported LTS each with >=3.5, so :+1: for >=3.5 from me

[megies]

See #2577 removed py3.4

[krischer]

I'd personally vote to go as high as possible/reasonable. The main rational behind this is that it will likely be at least another year before the next feature release where this becomes important and a new language version always means a bit more flexibility. Also venv is available for every Python 3 installation so system packages are IMHO no longer that important.

But I completely understand the desire to stay compatible with the debian cycle so this is just my opinion.

Main reasons for 3.6 for me would be:

• f-strings - just nice to have and convenient
• type hints for variables. These are sometimes just necessary for mypy to understand the code but they can also be added as comments so they are not absolutely crucial.
• Dictionaries preserve order - in 3.6 this is only an implementation detail but it can be relied on.

Reasons for 3.7:

• Dictionaries are officially ordered. But there is no practical difference to CPython 3.6 so this would not matter much.
• dataclasses. These would actually be useful for ObsPy I think.

Reasons for 3.8:

• No crucial reason really. Assignment expressions are kind of neat but might actually also confuse people.

[trichter]

Well, according to the mentioned NEP 29 it would be enough to support python 3.7 with a release in half a year let alone next year. And I would also vote to go as high as possible. But I can completely understand @megies point and that is why I am also happy with 3.5.

[d-chambers]

I would vote for 3.6+. The ability to use fstrings could potentially clean up quite a lot of code. Supporting system packaging is a non-issue for me.

[lijunzh]

The ability to use fstrings could potentially clean up quite a lot of code.

Agree. Once swtiched to f-string, I never looked back.

[megies]

Ah well, so I guess we could drop 3.5 as well. I guess it might really not be worth that much to comply with non-latest distro Python versions. The heck, I guess I'm down to go straight to 3.7

Vote for lowest supported version
:tada: >= 3.5
:rocket: >= 3.6
:heart: >= 3.7

[QuLogic]

All I want is >2 :wink:

[megies]

Bumped to Python 3.7 in #2577

[trichter]

In this case, what will be minimal supported Numpy and Scipy versions? Lowest versions supporting python 3.7 are numpy 1.14 an scipy 1.0, but may we can choose newer versions?

[hugovk]

NEP 29 suggests NumPy 1.15 as a minimum:

• https://numpy.org/neps/nep-0029-deprecation_policy.html

[megies]

Yeah, let's just go with that NEP and bump to numpy 1.15

[heavelock]

Going further down the dependencies list, how about Matplotlib? 3.0 is first py>=3.5, the current release 3.2 is py>=3.6 and numpy>=1.11 so we would not be in conflict here if we even went straight to the 3.2.

I would vote for bumping straight to 3.2. It does not make that much of a difference dependency wise but probably will save quite a few test image comparison errors.

https://matplotlib.org/devel/min_dep_policy.html#python-and-numpy

[steve8x8]

WIth the Debian Bullseye (11) final freeze ahead (scheduled for Jul 17 now), what about packaging for Python3 only? https://deb.obspy.org/pool/main/p/python-obspy/ hasn't been updated for quite a while...

[steve8x8]

Any news?

[trichter]

I think we can close this issue. We have separate issues for cartopy and requests. pathlib is already used.