Pillow: Release Pillow 6.1.0 on July 1, 2019

Created on 8 May 2019 · 36Comments · Source: python-pillow/Pillow

Release notes needed

Release checklist

[x] Open a release ticket e.g. https://github.com/python-pillow/Pillow/issues/3154
[x] Develop and prepare release in master branch.
[x] Check Travis CI and AppVeyor CI to confirm passing tests in master branch.
[x] Check that all of the wheel builds Pillow Wheel Builder pass the tests in Travis CI.
[x] In compliance with PEP 440, update version identifier in src/PIL/_version.py
[x] Update CHANGES.rst.
[x] Run pre-release check via make release-test in a freshly cloned repo.
[x] Create branch and tag for release e.g.:
bash git branch 5.2.x git tag 5.2.0 git push --all git push --tags
[x] Create source distributions e.g.:
bash make sdist
[x] Create binary distributions

Windows

[x] Contact @cgohlke for Windows binaries via release ticket e.g. https://github.com/python-pillow/Pillow/issues/1174.
[x] Download and extract tarball from @cgohlke and twine upload *.

Mac and Linux

[x] Use the Pillow Wheel Builder:
bash git clone https://github.com/python-pillow/pillow-wheels cd pillow-wheels ./update-pillow-tag.sh [[release tag]]
[x] Download distributions from the Pillow Wheel Builder container.
```bash
wget -m -A 'Pillow-*' \
http://a365fff413fe338398b6-1c8a9b3114517dc5fe17b7c3f8c63a43.r19.cf2.rackcdn.com

```
[x] Upload all binaries and source distributions e.g. twine upload dist/Pillow-5.2.0*
[x] Create a new release on GitHub
[x] In compliance with PEP 440, increment and append .dev0 to version identifier in src/PIL/_version.py

Publicize Release

[x] Announce release availability via Twitter e.g. https://twitter.com/PythonPillow/status/1013789184354603010

Documentation

[x] Make sure the default version for Read the Docs is up-to-date with the release changes

Release

Source

radarhere

👍1

Most helpful comment

🚀Pillow 6.1.0 is released! Thanks everyone for the help.

Edit: https://twitter.com/PythonPillow/status/1146339393046814721

hugovk on 3 Jul 2019

🎉3

All 36 comments

Less than a week to release!

Let's review some more contributor PRs.

Is there anything still needing release notes?

hugovk on 25 Jun 2019

👍1

@hugovk @radarhere Can we start a new thing where whoever is planning to do or lead the release self-assigns themself to these release issues? We now have 4, count them 4 (if you include me) folks capable of doing a release but I've lost track of what our schedule is (if we even have one). With @wiredfool and I lurking, it's typically either @hugovk or @radarhere so let's just figure out what the plan is for say, one year in advance. E.g. every other between @hugovk and @radarhere ? All four of us so as not to overwhelm one developer? I don't have any strong preference just want to firm it up … thanks.

aclark4life on 26 Jun 2019

Sure, @radarhere did the last one, I'm good to do this one.

@radarhere and I have been alternating, which I think's fine, and if there's one we can't do then we can just pipe up in advance and one of the other three can do it. For example, I can't guarantee to be available every single 1st January, because I might be away on holiday and without a computer.

We've been updating the release checklist as needed each release.

hugovk on 26 Jun 2019

👍1

Will start the release soon. Any last minute additions needed?

To merge:

Release notes for #3608, #3861 and #3886 in PR https://github.com/python-pillow/Pillow/pull/3930
Docs for https://github.com/python-pillow/Pillow/pull/3577 deprecation PR in https://github.com/python-pillow/Pillow/pull/3929

hugovk on 1 Jul 2019

All merged in for this release. Now waiting for master to turn green on the CIs (AppVeyor is sooo sloww, last 3 were 53/63/47 mins. We need to start shifting it to AZP).

hugovk on 1 Jul 2019

(I accidentally pushed 6.2.x to the 5.2.x branch. I've reset this back to the 5.2.0 tag.)

@cgohlke Please could we have Windows binaries for 6.1.0?

hugovk on 1 Jul 2019

Some new failures on pillow-wheels, TestImageFont.test_unicode_extended failed:

Mac MB_PYTHON_VERSION=2.7
Linux MB_PYTHON_VERSION=2.7 UNICODE_WIDTH=16
Linux MB_PYTHON_VERSION=2.7 PLAT=i686 UNICODE_WIDTH=16

With:

AssertionError: average pixel value difference 14.7190 > epsilon 6.2000
AssertionError: average pixel value difference 14.7190 > epsilon 6.2000
AssertionError: average pixel value difference 14.7280 > epsilon 6.2000

https://travis-ci.org/python-pillow/pillow-wheels/builds/552752918

hugovk on 1 Jul 2019

I suggest:

1) we fix the pillow-wheels build
2) because the 6.1.0 tag is already pushed, create a new 6.1.1 release with the fix
3) we don't bother with binaries for 6.1.0 but do 6.1.1 instead

Thoughts?

For 1) see https://github.com/python-pillow/Pillow/pull/3931 to loosen the epsilon for FreeType 2.10.

hugovk on 1 Jul 2019

@hugovk How about delete the tag and just release 6.1.0? Any downside to that?

aclark4life on 1 Jul 2019

That is possible, a downside is the tag is already "out there" and we don't know if anyone has done anything with it already.

hugovk on 1 Jul 2019

Could just delete it quickly now and hope for the best?

hugovk on 1 Jul 2019

@hugovk I'm comfortable taking responsibility for any damage done by updating the tag rev. I don't feel like that's as offensive as releasing different versions of 6.1.0 to PyPI (which we'd obviously never do). In other words, the requirement for a point/bug fix release should be that there is an actual release, not just that we've tagged a release. In this case, we just tagged it (right?) and we caught a bug so I think it's OK to delete and re-tag.

aclark4life on 1 Jul 2019

👍1

OK, 6.1.0 tag deleted and will re-tag the new one. Thanks!

hugovk on 1 Jul 2019

👍1

The new tag is ready and pillow-wheels is green.

Take 2: @cgohlke Please could we have Windows binaries for 6.1.0?

hugovk on 1 Jul 2019

I'm running into two issues:

pypy3 builds are failing

_imagingft.obj : error LNK2019: unresolved external symbol _PyUnicode_AsUCS4Copy referenced in function _text_layout_raqm

reproducible segfault with heap verification (Déjà vu)

================================================= test session starts =================================================
platform win32 -- Python 3.7.3, pytest-4.6.3, py-1.8.0, pluggy-0.12.0 -- X:\Python37\python.exe
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('D:\\Build\\Pillow\\Pillow-6.1.0\\.hypothesis\\examples')
rootdir: D:\Build\Pillow\Pillow-6.1.0
plugins: hypothesis-4.24.6, palladium-1.2.1.1, cov-2.7.1, faulthandler-1.6.0, forked-1.0.2, localserver-0.5.0, pep8-1.0.6, xdist-1.29.0
collected 1355 items

Tests/test_000_sanity.py::TestSanity::test_sanity PASSED                                                         [  0%]
Tests/test_binary.py::TestBinary::test_big_endian PASSED                                                         [  0%]
<snip>
Tests/test_image_convert.py::TestImageConvert::test_p_la PASSED                                                  [ 54%]
Tests/test_image_convert.py::TestImageConvert::test_rgba_p PASSED                                                [ 54%]
Tests/test_image_convert.py::TestImageConvert::test_sanity Windows fatal exception: access violation

Current thread 0x00001abc (most recent call first):
  File "X:\Python37\lib\site-packages\PIL\Image.py", line 1056 in convert
  File "D:\Build\Pillow\Pillow-6.1.0\Tests\test_image_convert.py", line 9 in convert
  File "D:\Build\Pillow\Pillow-6.1.0\Tests\test_image_convert.py", line 31 in test_sanity
  File "X:\Python37\lib\unittest\case.py", line 615 in run
  File "D:\Build\Pillow\Pillow-6.1.0\Tests\helper.py", line 61 in run
  File "X:\Python37\lib\unittest\case.py", line 663 in __call__
  File "X:\Python37\lib\site-packages\_pytest\unittest.py", line 224 in runtest
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 123 in pytest_runtest_call
  File "X:\Python37\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 81 in <lambda>
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 87 in _hookexec
  File "X:\Python37\lib\site-packages\pluggy\hooks.py", line 289 in __call__
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 198 in <lambda>
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 226 in from_call
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 198 in call_runtest_hook
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 173 in call_and_report
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 93 in runtestprotocol
  File "X:\Python37\lib\site-packages\_pytest\runner.py", line 78 in pytest_runtest_protocol
  File "X:\Python37\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 81 in <lambda>
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 87 in _hookexec
  File "X:\Python37\lib\site-packages\pluggy\hooks.py", line 289 in __call__
  File "X:\Python37\lib\site-packages\_pytest\main.py", line 271 in pytest_runtestloop
  File "X:\Python37\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 81 in <lambda>
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 87 in _hookexec
  File "X:\Python37\lib\site-packages\pluggy\hooks.py", line 289 in __call__
  File "X:\Python37\lib\site-packages\_pytest\main.py", line 250 in _main
  File "X:\Python37\lib\site-packages\_pytest\main.py", line 206 in wrap_session
  File "X:\Python37\lib\site-packages\_pytest\main.py", line 243 in pytest_cmdline_main
  File "X:\Python37\lib\site-packages\pluggy\callers.py", line 187 in _multicall
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 81 in <lambda>
  File "X:\Python37\lib\site-packages\pluggy\manager.py", line 87 in _hookexec
  File "X:\Python37\lib\site-packages\pluggy\hooks.py", line 289 in __call__
  File "X:\Python37\lib\site-packages\_pytest\config\__init__.py", line 82 in main
  File "X:\Python37\lib\site-packages\pytest.py", line 102 in <module>
  File "X:\Python37\lib\runpy.py", line 85 in _run_code
  File "X:\Python37\lib\runpy.py", line 193 in _run_module_as_main

cgohlke on 1 Jul 2019

pypy3 builds are failing

_imagingft.obj : error LNK2019: unresolved external symbol _PyUnicode_AsUCS4Copy referenced in function _text_layout_raqm

Likely from https://github.com/python-pillow/Pillow/pull/3780 (ping @nulano):

PyUnicode_AS_UNICODE is now deprecated, replace it with PyUnicode_AsUCS4Copy or PyUnicode_READ_CHAR.

What's the full version of the failing pypy3?

hugovk on 1 Jul 2019

Segfault at https://github.com/python-pillow/Pillow/blob/9300ced1e1812569091fcf330dd6aef909376436/src/libImaging/Convert.c#L1045

>   _imaging.cp37-win_amd64.pyd!p2i(unsigned char * out_=0x000001ef25cc5d58, const unsigned char * in=0x000001ef2202ffac, int xsize=128, const unsigned char * palette=0x000001ef142fabe7) Line 1045    C
    _imaging.cp37-win_amd64.pyd!frompalette(ImagingMemoryInstance * imOut=0x000001ef13dcdfa0, ImagingMemoryInstance * imIn=0x000001ef142e6fa0, const char * mode=0x000001ef19dd9f40) Line 1204  C
    _imaging.cp37-win_amd64.pyd!convert(ImagingMemoryInstance * imOut=0x0000000000000000, ImagingMemoryInstance * imIn=0x000001ef142e6fa0, const char * mode=0x000001ef19dd9f40, ImagingPaletteInstance * palette=0x0000000000000000, int dither=3) Line 1492   C
    _imaging.cp37-win_amd64.pyd!ImagingConvert(ImagingMemoryInstance * imIn=0x000001ef142e6fa0, const char * mode=0x000001ef19dd9f40, ImagingPaletteInstance * palette=0x0000000000000000, int dither=3) Line 1542  C
    _imaging.cp37-win_amd64.pyd!_convert(ImagingObject * self=0x000001ef2ddaafb0, _object * args=0x000001ef2dea26c8) Line 933   C
    python37.dll!_PyMethodDef_RawFastCallKeywords(PyMethodDef * method=0x0000000000000000, _object * self=0x000001ef2ddaafb0, _object * const * args=0x000001ef2a1f8fb0, __int64 nargs=2, _object * kwnames) Line 694   C
    [Inline Frame] python37.dll!_PyMethodDescr_FastCallKeywords(_object *) Line 288 C
    python37.dll!call_function(_object * * * pp_stack=0x00000028847e91f8, __int64 oparg, _object * kwnames=0x0000000000000000) Line 4593    C

  | Name | Value | Type
-- | -- | -- | --
◢ | in | 0x000001ef2202ffac "\v.\v.\v\x10" | const unsigned char *
  |   | 11 '\v' | const unsigned char
◢ | out_ | 0x000001ef25cc5d58 "ÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀÀ... | unsigned char *
  |   | 192 'À' | unsigned char
◢ | palette | 0x000001ef142fabe7 "" | const unsigned char *
  |   | 0 '\0' | const unsigned char
  | v | 158 | int
  | x | 86 | int
  | xsize | 128 | int

cgohlke on 1 Jul 2019

What's the full version of the failing pypy3?

Python 3.6.1 (784b254d6699, Apr 16 2019, 12:10:48)
[PyPy 7.1.1-beta0 with MSC v.1910 32 bit] on win32

and

Python 3.5.3 (928a4f70d3de, Feb 08 2019, 12:56:35)
[PyPy 7.0.0 with MSC v.1910 32 bit] on win32

cgohlke on 1 Jul 2019

The segfault is likely from https://github.com/python-pillow/Pillow/pull/3225, @DerDakon please could you check it?

hugovk on 1 Jul 2019

How could the tests ever pass? Maybe I'm missing something, but shouldn't this be:

diff --git a/src/libImaging/Convert.c b/src/libImaging/Convert.c
index 038a83fe..5df48fb2 100644
--- a/src/libImaging/Convert.c
+++ b/src/libImaging/Convert.c
@@ -1041,7 +1041,7 @@ static void
 p2i(UINT8* out_, const UINT8* in, int xsize, const UINT8* palette)
 {
     int x;
-    for (x = 0; x < xsize; x++, in += 2, out_ += 4) {
+    for (x = 0; x < xsize; x++, out_ += 4) {
         INT32 v = L(&palette[in[x]*4]) / 1000;
         memcpy(out_, &v, sizeof(v));
     }
@@ -1060,7 +1060,7 @@ static void
 p2f(UINT8* out_, const UINT8* in, int xsize, const UINT8* palette)
 {
     int x;
-    for (x = 0; x < xsize; x++, in += 2, out_ += 4) {
+    for (x = 0; x < xsize; x++, out_ += 4) {
         FLOAT32 v = L(&palette[in[x]*4]) / 1000.0F;
         memcpy(out_, &v, sizeof(v));
     }

cgohlke on 1 Jul 2019

The very same change I came up with, see #3932.

DerDakon on 1 Jul 2019

👍2

The very same change I came up with, see #3932.

Good. All tests are passing here. No segfaults.

cgohlke on 1 Jul 2019

👍2

I've untagged 6.1.0.

PR #3932 is merged to fix the segfault.

That leaves the PyPy3 failure. I guess that's because PyUnicode_AS_UNICODE may well be "Deprecated since version 3.3, will be removed in version 4.0" in CPython, that doesn't necessarily mean it's deprecated in PyPy.

Our tests didn't pick it up because we don't test PyPy3 on Windows; and whilst we do test PyPy3 on Travis CI, there is a "implicit declaration of function ‘PyUnicode_AsUCS4Copy’" warning we didn't notice as all the test_imagefont tests are skipped because there's no FreeType (or RAQM) on PyPy3. We should improve these testing gaps.

I've opened a ticket with PyPy asking if they also plan to deprecate/replace/remove PyUnicode_AS_UNICODE.

And here's a PR to revert that change, in case we don't get a better solution in the short term: https://github.com/python-pillow/Pillow/pull/3933.

I'll need to continue tomorrow unless someone in another timezone would like to pick up the reins.

Thanks everyone for the help so far!

hugovk on 1 Jul 2019

@hugovk Probably fine to wait until tomorrow, thanks for all the hard work! Much appreciated.

aclark4life on 1 Jul 2019

👍1

From the PyPy FAQ:

The external C-API has been reimplemented in PyPy as an internal cpyext module. We support most of the documented C-API [...]

I guess that means no modern PyUnicode functions. There might be a way to fix the original issue (#3777) using PyPy-specific functions, but probably not in time for this release.

It should be possible to revert the change for PyPy only by adding a preprocessor if block; looking through the PyPy include files, the macro PYPY_VERSION_NUM seems like a good candidate. I can try to write that tomorrow.

nulano on 1 Jul 2019

👍2

Thanks @nulano

aclark4life on 1 Jul 2019

3935 should fix the PyPy3 build failure introduced in 8d4bb339a6ec55ff5ddfc61aafc2dc50bb4cbf6c:

This disables the fix for #3777 for PyPy3, as PyPy's cpyext module hasn't been updated to support the new PyUnicode functions yet.

Our tests didn't pick it up because ... whilst we do test PyPy3 on Travis CI, there is a "implicit declaration of function ‘PyUnicode_AsUCS4Copy’" warning we didn't notice as all the test_imagefont tests are skipped because there's no FreeType (or RAQM) on PyPy3. We should improve these testing gaps.

The FreeType library was probably disabled due to a dynamic link error, not because it's not installed. FreeType works again with this PR: https://travis-ci.org/nulano/Pillow/jobs/553153042#L2976

nulano on 2 Jul 2019

Some new failures on pillow-wheels, TestImageFont.test_unicode_extended failed:

Mac MB_PYTHON_VERSION=2.7

Linux MB_PYTHON_VERSION=2.7 UNICODE_WIDTH=16

Linux MB_PYTHON_VERSION=2.7 PLAT=i686 UNICODE_WIDTH=16

With:

AssertionError: average pixel value difference 14.7190 > epsilon 6.2000

AssertionError: average pixel value difference 14.7190 > epsilon 6.2000

AssertionError: average pixel value difference 14.7280 > epsilon 6.2000

https://travis-ci.org/python-pillow/pillow-wheels/builds/552752918

For 1) see #3931 to loosen the epsilon for FreeType 2.10.

I suspect this is actually a real failure. When I tried the test on Windows without the #3780 fix, I got the same values:
E AssertionError: 2.5 not greater than or equal to 14.728 : average pixel value difference 14.7280 > epsilon 2.5000
https://ci.appveyor.com/project/nulano/pillow/builds/23676028/job/jjd897m8b68sx96a#L6647.

I suggest reverting #3931 and changing the test's skip conditions to skip on all platforms when running Python 2.

nulano on 2 Jul 2019

I suggest reverting #3931 and changing the test's skip conditions to skip on all platforms when running Python 2.

Done in #3936.

nulano on 2 Jul 2019

👍1

Thanks @nulano! Let's merge #3936 after the release to minimise changes, as it only changes a single test file.

@cgohlke Please could you test https://github.com/python-pillow/Pillow/pull/3935? If it's fine, I'll merge it and continue the release.

hugovk on 2 Jul 2019

👍1

Regarding #3935, I just realized PyPy does support the new PyUnicode_READ_CHAR function (used with basic layout), but not PyUnicode_AsUCS4Copy (used with raqm layout). In the PR I removed them both for PyPy.

I could add this back, but I think it's best to wait for the coming Python 2 end of support. At that point I would like to simplify and improve both layout functions. What do you think @hugovk?

nulano on 2 Jul 2019

Yeah, let's wait. It's only 6 months until Python 2 EOL (and 3 months until PRs can be merged). And the changes are in part to replace deprecated functions in CPython, which will be removed in CPython 4.0, due maybe 2021 or 2022 or who knows; no rush.

hugovk on 2 Jul 2019

👍1

could you test #3935?

3935 builds OK with pypy3 on Windows.

cgohlke on 2 Jul 2019

🚀1

3935 merged and 6.1.0 retagged.

pillow-wheels is queued up, although Travis is running slow at the moment so we'll need to be patient.

Take 3! @cgohlke Please could we have Windows binaries for 6.1.0?

hugovk on 2 Jul 2019

👍1

could we have Windows binaries for 6.1.0?

Here you go.

cgohlke on 2 Jul 2019

🎉1

🚀Pillow 6.1.0 is released! Thanks everyone for the help.

Edit: https://twitter.com/PythonPillow/status/1146339393046814721

hugovk on 3 Jul 2019

🎉3

Was this page helpful?

0 / 5 - 0 ratings

Related issues

How to open gif file in python 3, anybody knows a perfect method, please do a favour on me

amithnikhade · 4Comments

Python.h not found when installing on Ubuntu / virtualenv / pypy

steph-ben · 4Comments

Quantization tables (qtables) being altered

damianmoore · 4Comments

PIL.PngImagePlugin.PngImageFile images can't be saved as GIFs in 7.1.1

maxhumber · 3Comments

adding multiple images to PDF

thinrhino · 3Comments

Pillow: Release Pillow 6.1.0 on July 1, 2019

Release notes needed

Release checklist

Windows

Mac and Linux

```

Publicize Release

Documentation

Most helpful comment

All 36 comments

3935 should fix the PyPy3 build failure introduced in 8d4bb339a6ec55ff5ddfc61aafc2dc50bb4cbf6c:

3935 builds OK with pypy3 on Windows.

3935 merged and 6.1.0 retagged.

Related issues