Pillow: DeprecationWarning in Python 3.6 and 3.7

[uri-rodberg]

What did you do?

Upgrade to Pillow==5.4.0 (with pip install --upgrade -r requirements-pip-upgrade.txt on Speedy Net - https://github.com/speedy-net/speedy-net)

What did you expect to happen?

Nothing.

What actually happened?

Tests with Python 3.6 failed - https://travis-ci.org/speedy-net/speedy-net/jobs/475706499
(we run tests with deprecation warnings - python -W error::DeprecationWarning manage.py test)

DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

What are your OS, Python and Pillow versions?

• OS: Ubuntu 14.04.5 LTS (same problem with Ubuntu 16.04.5 LTS)
• Python: 3.6.3 (same problem with Python 3.6.7 and Python 3.7.1)
• Pillow: 5.4.0

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

https://github.com/speedy-net/speedy-net/tree/uri_merge_with_master_2018-12-31_a

Same code except Pillow version - Pillow==5.3.0:

https://github.com/speedy-net/speedy-net/tree/uri_try_pillow_5.3.0_2019-01-05_a

Tests with Python 3.6 don't fail - https://travis-ci.org/speedy-net/speedy-net/jobs/475719810

Same problem with Python 3.7.1:

https://travis-ci.org/speedy-net/speedy-net/jobs/475722137

[hugovk]

Pillow has supported Python 3.6 since 4.x and Python 3.7 since 5.2.0.
https://pillow.readthedocs.io/en/stable/installation.html#notes

I can't see anywhere where Pillow uses imp directly. grep -r "import imp" . and grep -r "from imp" . return nothing.

Here's your error from the logs:

0.20s$ python -W error::DeprecationWarning manage.py test
Traceback (most recent call last):
  File "manage.py", line 12, in <module>
    from django.core.management import execute_from_command_line
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/core/management/__init__.py", line 13, in <module>
    from django.core.management.base import (
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/core/management/base.py", line 17, in <module>
    from django.db.migrations.exceptions import MigrationSchemaMissing
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/migrations/__init__.py", line 2, in <module>
    from .operations import *  # NOQA
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/migrations/operations/__init__.py", line 1, in <module>
    from .fields import AddField, AlterField, RemoveField, RenameField
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/migrations/operations/fields.py", line 4, in <module>
    from django.db.models.fields import NOT_PROVIDED
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/models/__init__.py", line 3, in <module>
    from django.db.models.aggregates import *  # NOQA
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/models/aggregates.py", line 5, in <module>
    from django.db.models.expressions import Func, Star
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/models/expressions.py", line 6, in <module>
    from django.db.models import fields
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/db/models/fields/__init__.py", line 14, in <module>
    from django import forms
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/forms/__init__.py", line 7, in <module>
    from django.forms.fields import *  # NOQA
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/forms/fields.py", line 17, in <module>
    from django.core import validators
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/core/validators.py", line 507, in <module>
    allowed_extensions=get_available_image_extensions(),
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/django/core/validators.py", line 502, in get_available_image_extensions
    Image.init()
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/PIL/Image.py", line 428, in init
    __import__("PIL.%s" % plugin, globals(), locals(), [])
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/PIL/JpegImagePlugin.py", line 41, in <module>
    from . import Image, ImageFile, TiffImagePlugin
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/site-packages/PIL/TiffImagePlugin.py", line 57, in <module>
    import distutils.version
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/distutils/__init__.py", line 4, in <module>
    import imp
  File "/home/travis/virtualenv/python3.6.3/lib/python3.6/imp.py", line 33, in <module>
    DeprecationWarning, stacklevel=2)
DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
The command "python -W error::DeprecationWarning manage.py test" exited with 1.

https://travis-ci.org/speedy-net/speedy-net/jobs/475706499#L587

That show's Pillow's TiffImagePlugin.py imports distutils.version, and that the DeprecationWarning comes from distutils.

It looks similar to https://github.com/django-ldapdb/django-ldapdb/issues/138, which says "the problem is the module virtualenv which tox uses" (https://github.com/tox-dev/tox/issues/436). You could also add a workaround like https://github.com/django-ldapdb/django-ldapdb/commit/2a7d1363cb419657e77248f80bdfc30a65676635.

[uri-rodberg]

But it works with Pillow==5.3.0:

https://travis-ci.org/speedy-net/speedy-net/jobs/475719810
https://travis-ci.org/speedy-net/speedy-net/jobs/475726531

I didn't see this problem before upgrading to Pillow==5.4.0.

You can also see:
https://travis-ci.org/speedy-net/speedy-net/builds/475775321
https://travis-ci.org/speedy-net/speedy-net/jobs/475775324
https://travis-ci.org/speedy-net/speedy-net/jobs/475775325

[hugovk]

Ah yes, that's because import distutils.version was only added in 5.4.0 (in https://github.com/python-pillow/Pillow/pull/3350).

[uri-rodberg]

It seems to me that distutils itself is also deprecated or about to be deprecated.

https://docs.python.org/3/library/distutils.html
https://docs.python.org/3/distutils/index.html
https://docs.python.org/3/distributing/index.html
https://docs.python.org/3/install/index.html
https://docs.python.org/3/installing/index.html

[hugovk]

Is there a replacement for distutils.version.StrictVersion in the stdlib?

[radarhere]

I've created PR #3552, replacing distutils.version with simple parsing.

[radarhere]

After investigation, I believe that the deprecation warning is only occurring in virtualenv, and that regular distutils is fine. If so, then it's possible that this will be fixed for you via https://github.com/pypa/virtualenv/pull/1238 once virtualenv 16.1 propagates.

[jdufresne]

To me, this looks like a issue with distutils and not Pillow. I think the efforts to fix this deprecation warning should be spent there. While distutils is not recommended for use by packages, AFICT, it is not deprecated and continues to serve as the foundation for setuptools.

[hugovk]

Agreed. There's a workaround to ignore DeprecationWarnings from distutils (eg. https://github.com/django-ldapdb/django-ldapdb/commit/2a7d1363cb419657e77248f80bdfc30a65676635).

[jdufresne]

As best I can tell, this was already fixed in CPython:

https://github.com/python/cpython/commit/61c3556c14d52e553ca5eecc4e178124fac75167

[radarhere]

Just a note that https://github.com/pypa/virtualenv/pull/1238 does not fix this issue after all, as the Python 3.7 job has virtualenv 16.1.0.

https://github.com/pypa/virtualenv/issues/955 is an issue seeking a proper fix for this.

[hugovk]

https://github.com/pypa/virtualenv/issues/955 has been fixed by https://github.com/pypa/virtualenv/pull/1289, and released in virtualenv 16.3.0.

[hugovk]

Thanks for the report, but closing because the deprecation warning is external to Pillow, Pillow is using non-deprecated distutils in an acceptable way, and there's a workaround to silence the warning specifically from distutils.

[uri-rodberg]

Tests also fail with virtualenv==16.4.3. What is the problem?

(with Pillow==5.4.1):
https://travis-ci.org/speedy-net/speedy-net/builds/514284524

(with Pillow==6.0.0):
https://travis-ci.org/speedy-net/speedy-net/builds/514595887

DeprecationWarning in Python 3.6 and 3.7 (with Pillow) - on Stack Overflow

[uri-rodberg]

I created a new issue for setuptools: https://github.com/pypa/setuptools/issues/1933

[radarhere]

distutils has now been removed in #4305