Dvc: Drop support for Python 2.7

[ghost]

Python will drop support for 2.7 in January 1st, 2020, I think DVC should too, it will ease the maintenance (spending less time dealing with str/unicode issues, default support for type checking syntax, access to a more robust standard library, etc.).

_Countdown for Python 2.7 EOL: https://pythonclock.org/_

I don't know how many users are currently running on Python 2.7, but we can get a clue by analyzing pypi stats with BigQuery or using pypinfo.

There are some maintenance tasks to migrate to a Python 3.x only codebase:

• [x] Remove 2.7 and 3.4 from our conda package
• [x] Remove Python 2.7 from TravisCI (EDIT(30 Dec) removed from travis.yaml already, but need to cleanup scripts/ci/* from py2 code)
• [x] Upgrade AppVeyor's Python version
• [x] Modify python_requires and explicit version classifiers (kudos to @hugovk !)
• [x] [Update the package](https://packaging.python.org/guides/dropping-older-python-versions/)
• [x] Remove all calls/dependencies to future
• [x] Remove all code linked to utils.compat
• [x] Refactor code with comments about python 2 backwards compatibility
• [x] Update super() to new syntax
• [x] Refactor DvcExceptions to use default traceback instead of custom one
  
  
  
  • [x] Refactor logger.py. Remove logging formatter parse_exc logic and use formatException and formatStack instead
  
  
• [x] Refactor system.py module to use the default os methods for linking, as they include Windows support
• [ ] ~Make keyword params kwonly in dvc.api. Consider the same in other places.~ (#3108)

Feel free to edit this comment, modifying the list of tasks that should happen to make a clean transition.

_NOTE: This issue was created to keep track of what should be done. I'm not implying that this will happen in a near future, neither expressing the ideas of the rest of the maintainers.
It would be great if you can write down your opinion so we reach a consensus between the maintainers and the community._

[hugovk]

I'd recommend adding python_requires and explicit version classifiers, especially useful when 2.7 is dropped, and also useful now. Please see PR https://github.com/iterative/dvc/pull/1845.

---

I don't know how many users are currently running on Python 2.7, but we can get a clue by analyzing pypi stats with BigQuery or using pypinfo.

Here's the pip installs for dvc from PyPI for March 2019:

| category | percent | downloads |
|----------|--------:|----------:|
| 3 | 73.99% | 6,320 |
| 2 | 17.53% | 1,497 |
| null | 8.49% | 725 |
| Total | | 8,542 |

Source: pypistats python_major dvc --last-month # pip install pypistats

| category | percent | downloads |
|----------|--------:|----------:|
| 3.6 | 42.39% | 3,621 |
| 3.5 | 20.21% | 1,726 |
| 2.7 | 17.51% | 1,496 |
| 3.7 | 11.33% | 968 |
| null | 8.49% | 725 |
| 3.4 | 0.06% | 5 |
| 2.6 | 0.01% | 1 |
| Total | | 8,542 |

Source: pypistats python_minor dvc --last-month # pip install pypistats

That's a very low proportion of Python 2, see https://dev.to/hugovk/python-version-share-over-time-4-26hm for some other projects.

[ghost]

Thank you so much for the detailed stats and for contributing with #1845 and #1847 , @hugovk ! :fire:

[ghost]

Time flies when you're having fun)
EOL is around the corner, I'm eager to close this issue))

[efiop]

For the record: trying to drop support for 2.7 (and 3.4, as it has reached EOL already) in https://github.com/conda-forge/dvc-feedstock/pull/54 , as it is starting to cause problems during the build.

[ghost]

what do you mean, @efiop ? is it happening? are we finally dropping 2.7?
I can happily spend my weekend on the migration :sweat_smile:

[efiop]

@mroutis No, just dropping it where it causes serious issues. The main feast will come later, you won't miss it 😄

[ghost]

I'm starting to feel the christmas spirit, we all joyful writing python 3.x code :slightly_smiling_face:

[efiop]

@mroutis The joy probably won't fully come until py35 is dead, which is 1 more year 🙂

[ghost]

https://github.com/asottile/pyupgrade

[casperdcl]

Eek python_requires is the most important version-related keyword. Really bad that it wasn't included all this time. Definitely need at least one release explicitly allowing py>=3,<=3.4 and py<3 otherwise things could break in future. Looks like that's happened already though.

[efiop]

@casperdcl Last release was 0.78.1 and it has py2 support, so not broken. We'll limit to >py3.4 in the next one. Plus would be probably nice to put some kind of sanity check for that in CLI and when importing dvc

[ghost]

It is happening!

[ghost]

yay or nay for _relative imports_?

(cc: @pared , @Suor)

[ghost]

what about using ABCs?

[casperdcl]

Are there py3-specific ABC features you refer to?

[ghost]

No, @casperdcl; I thought that ABCs weren't implemented in Python 2.7, don't know why we are not using them for Remotes and Outputs, then :thinking:

[efiop]

@mroutis Let's not do that within this sprint, but rather create a separate ticket for that for later. We'll keep in mind them next time we are going to create appropriate classes. Relative imports could be moved to a separate ticket as well, as it is a bit unrelated and we currently mismatch relative and abs imports all over the codebase :slightly_smiling_face:

[casperdcl]

I agree with @efiop - both features were supported in py2 (may have needed things like __future__.absolute_import) but not related to deprecation. I'm fine with removal of __future__ (which has already happened) but as @efiop said would leave actually using relative imports and ABCs everywhere for another issue.

[ghost]

Great, @efiop , @casperdcl ; should check better next time, I thought they were not supported in Python 2.7, this is why I was asking :sweat_smile:

[ghost]

The last thing to address is using kwargs only on api.py module.

This is how it looks like:

>>> def hello(*, name='world'): print('hello', name)
>>> hello()
"hello world"
>>> hello("people")
TypeError: hello() takes 0 positional arguments but 1 was given

It makes the interface more explicit.
To make it work you only need to put an asterisk before the keyword arguments.

I can do it for api.py but not sure if make sense for other modules. I would leave it as a separate issue. (#3108)

[ghost]

Should we close this one or anyone has any other suggestion? @iterative/engineering

[ghost]

Let's not leave this one open, we can handle any further suggestions on separate issues :)