Pillow: Provide wheels for pypy3

[mattip]

What help is needed to begin creating and uploading binary wheels for PyPy on windows32 bit, manylinux2010, and macOS?

[hugovk]

Hello! Our current process for binary wheels is documented here:

• https://github.com/python-pillow/Pillow/blob/master/RELEASING.md#binary-distributions

Mac and Linux

Made here using multibuild. Perhaps you know what would be needed and could send a PR?

• https://github.com/python-pillow/pillow-wheels

Windows

Currently built and provided by Christoph Gohlke. Is Pillow-7.1.2-pp373-pypy36_pp73-win32.whl 32 bit? Or did you mean 64 bit?

• https://pypi.org/project/Pillow/#files

[hugovk]

And can multibuild do armv7/arm64 wheels too? Re: https://github.com/python-pillow/Pillow/issues/4656.

[mattip]

multibuild can do arm64.

[mattip]

xref python-pillow/pillow-wheels#147 which adds wheels for arm64 and PyPy

[radarhere]

In my attempt to setup macOS, I've created PR https://github.com/matthew-brett/multibuild/pull/334

[radarhere]

https://github.com/python-pillow/pillow-wheels/pull/153 added Linux PyPy jobs.
https://github.com/python-pillow/pillow-wheels/pull/154 adds macOS PyPy jobs.

[radarhere]

The new PyPy wheels will be a part of the next Pillow release, due out on July 1.

[mattip]

Thanks @radarhere !

[hugovk]

And thank you too, @mattip!

[mattip]

Thanks for all the work here. I see the 7.2 release is up and it seems the only PyPy wheel that made it in is the Pillow-7.2.0-pp36-pypy36_pp73-win32.whl one (which is itself fantastic, thanks). Am I missing something?

[radarhere]

When putting together the release, I found that wheelhouse-uploader doesn't seem to support PyPy - https://github.com/ogrisel/wheelhouse-uploader/blob/01ee128247a2284813c84ab9d8cd48105f2d9e06/wheelhouse_uploader/utils.py#L120-L131

@hugovk created https://github.com/ogrisel/wheelhouse-uploader/pull/37 to try and help this situation.

[hugovk]

We should stop uploading to Rackspace anyway (https://github.com/python-pillow/pillow-wheels/issues/146), and whatever the alternative is may mean we don't need to use wheelhouse-uploader.

But let's re-open this, as we're building but not quite actually providing the PyPy3 wheels.

[hugovk]

PR to use GitHub Releases as an intermediary before going to PyPI:

• https://github.com/python-pillow/pillow-wheels/pull/156

[hugovk]

https://github.com/python-pillow/pillow-wheels/pull/156 has been merged!

---

I suggest we do a test build of the last release (7.2.0) for two reasons:

• Check it works with a first full build before the next release day:

  • confirm it creates and uploads wheels to GH Releases, which can be downloaded

• The PyPy wheels for 7.2.0 didn't get uploaded to PyPI due to a bug with wheelhouse-uploader (https://github.com/python-pillow/Pillow/issues/4655#issuecomment-653846923)

  • wheelhouse-uploader is since fixed, but we're not using it anymore with GHA
  • we can then "complete" the release by uploading the 7.2.0 PyPy wheels to PyPI

---

Process should be something like this:

git clone https://github.com/python-pillow/pillow-wheels
cd pillow-wheels
./update-pillow-tag.sh 7.2.0

Then wait for it to complete,
then download the PyPy wheels,
then upload to PyPI:

twine upload dist/Pillow-7.2.0*

[radarhere]

@hugovk sounds good to me

[hugovk]

./update-pillow-tag.sh 7.2.0 stopped at the commit, because there was nothin to commit this time, because that was done during the last release. So ran the last two commands manually (see below), and it's now building: https://travis-ci.org/github/python-pillow/pillow-wheels/builds/717224081

⌂171% [hugo:/private/tmp/pillow-wheels] master ± cat update-pillow-tag.sh
#!/usr/bin/env bash

set -e # exit on error

if [ $# -eq 0 ]; then
    echo "Usage: update-pillow-tag.sh [[release tag]]"
    exit
fi

git checkout master
git submodule init
git submodule update Pillow
cd Pillow
git fetch --all
git checkout $1
cd ..
git commit -m "Pillow -> $1" Pillow
git tag $1
git push origin $1

⌂171% [hugo:/private/tmp/pillow-wheels] master ± ./update-pillow-tag.sh 7.2.0
Already on 'master'
Your branch is up to date with 'origin/master'.
Submodule 'Pillow' (git://github.com/python-Pillow/Pillow.git) registered for path 'Pillow'
Submodule 'multibuild' (https://github.com/matthew-brett/multibuild.git) registered for path 'multibuild'
Cloning into '/private/tmp/pillow-wheels/Pillow'...
Submodule path 'Pillow': checked out '2bd74943fb9f320def6c066e732b701d1c15f677'
Fetching origin
HEAD is now at 2bd74943 Updated CHANGES.rst [ci skip]
On branch master
Your branch is up to date with 'origin/master'.

nothing to commit, working tree clean



md5-da54d018f61456a5602f467ba8ca97be



⌂170% [hugo:/private/tmp/pillow-wheels] master 16s 1 ± git tag 7.2.0



md5-da54d018f61456a5602f467ba8ca97be



⌂162% [hugo:/private/tmp/pillow-wheels] master ± git push origin 7.2.0
Total 0 (delta 0), reused 0 (delta 0), pack-reused 0
To https://github.com/python-pillow/pillow-wheels
 * [new tag]         7.2.0 -> 7.2.0

[hugovk]

All built, except for 3.6 macOS PyPy due to a temporary network failure:

...
==> Installing openblas
==> Downloading https://homebrew.bintray.com/bottles/openblas-0.2.20_1.high_sierra.bottle.tar.gz
curl: (6) Could not resolve host: homebrew.bintray.com
Error: Failed to download resource "openblas"
Download failed: https://homebrew.bintray.com/bottles/openblas-0.2.20_1.high_sierra.bottle.tar.gz
Warning: Bottle installation failed: building from source.
==> Downloading https://github.com/xianyi/OpenBLAS/archive/v0.2.20.tar.gz
curl: (6) Could not resolve host: github.com
Error: Failed to download resource "openblas"
Download failed: https://github.com/xianyi/OpenBLAS/archive/v0.2.20.tar.gz

https://travis-ci.org/github/python-pillow/pillow-wheels/jobs/717224091

I restarted it.

[hugovk]

All built!

https://travis-ci.org/github/python-pillow/pillow-wheels/builds/717224081

And deployed!

https://github.com/python-pillow/pillow-wheels/releases/tag/7.2.0

And downloaded!

$ brew install fetch
...
$ mkdir /tmp/assets
$ fetch --repo https://github.com/python-pillow/pillow-wheels --release-asset="\.whl" --progress --tag 7.2.0 /tmp/assets
Downloading release asset Pillow-7.2.0-cp36-cp36m-manylinux2014_aarch64.whl to /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux2014_aarch64.whl
Downloading release asset Pillow-7.2.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl to /tmp/assets/Pillow-7.2.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl
Downloading release asset Pillow-7.2.0-cp36-cp36m-manylinux1_i686.whl to /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux1_i686.whl
Downloading release asset Pillow-7.2.0-cp37-cp37m-manylinux2014_aarch64.whl to /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux2014_aarch64.whl
Downloading release asset Pillow-7.2.0-cp38-cp38-manylinux1_i686.whl to /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux1_i686.whl
Downloading release asset Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl
Downloading release asset Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl
Downloading release asset Pillow-7.2.0-cp37-cp37m-manylinux1_i686.whl to /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux1_i686.whl
Downloading release asset Pillow-7.2.0-cp36-cp36m-manylinux1_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux1_x86_64.whl
Downloading release asset Pillow-7.2.0-cp37-cp37m-macosx_10_10_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp37-cp37m-macosx_10_10_x86_64.whl
Downloading release asset Pillow-7.2.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl to /tmp/assets/Pillow-7.2.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl
Downloading release asset Pillow-7.2.0-cp36-cp36m-macosx_10_10_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp36-cp36m-macosx_10_10_x86_64.whl
Downloading release asset Pillow-7.2.0-cp38-cp38-manylinux1_x86_64.whl to /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux1_x86_64.whl
Downloading release asset Pillow-7.2.0-cp38-cp38-manylinux2014_aarch64.whl to /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux2014_aarch64.whl
Downloading... 1.3 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp37-cp37m-macosx_10_10_x86_64.whl
Downloading... 844 kB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl
Downloading... 1.3 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux2014_aarch64.whl
Downloading... 748 kB / 2.1 MBDownloaded /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux2014_aarch64.whl
                                   Downloaded /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux1_x86_64.whl
Downloading... 766 kB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux1_x86_64.whl
Downloading... 1.8 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux1_i686.whl
Downloading... 574 kB / 2.1 MBDownloaded /tmp/assets/Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl
Downloading... 2.2 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux2014_aarch64.whl
Downloading... 2.1 MB / 2.1 MBDownloaded /tmp/assets/Pillow-7.2.0-cp38-cp38-manylinux1_i686.whl
Downloading... 2.2 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl
Downloading... 2.1 MB / 2.1 MBDownloaded /tmp/assets/Pillow-7.2.0-cp36-cp36m-manylinux1_i686.whl
Downloading... 671 kB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp36-cp36m-macosx_10_10_x86_64.whl
Downloading... 2.2 MB / 2.2 MBDownloaded /tmp/assets/Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl
Download of release assets complete
$ l /tmp/assets
total 71704
drwxr-xr-x  16 hugo  wheel   512B 12 Aug 15:36 .
drwxrwxrwt  33 root  wheel   1.0K 12 Aug 15:45 ..
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp36-cp36m-macosx_10_10_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.0M 12 Aug 15:36 Pillow-7.2.0-cp36-cp36m-manylinux1_i686.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp36-cp36m-manylinux1_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp36-cp36m-manylinux2014_aarch64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp37-cp37m-macosx_10_10_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.0M 12 Aug 15:36 Pillow-7.2.0-cp37-cp37m-manylinux1_i686.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp37-cp37m-manylinux1_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp37-cp37m-manylinux2014_aarch64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp38-cp38-macosx_10_10_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.0M 12 Aug 15:36 Pillow-7.2.0-cp38-cp38-manylinux1_i686.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp38-cp38-manylinux1_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-cp38-cp38-manylinux2014_aarch64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl
-rw-r--r--   1 hugo  wheel   2.1M 12 Aug 15:36 Pillow-7.2.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl

And uploaded!

$ twine upload /tmp/assets/Pillow-7.2.0-*pypy*
Uploading distributions to https://upload.pypi.org/legacy/
Uploading Pillow-7.2.0-pp36-pypy36_pp73-macosx_10_10_x86_64.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████| 2.07M/2.07M [00:12<00:00, 171kB/s]
Uploading Pillow-7.2.0-pp36-pypy36_pp73-manylinux2010_x86_64.whl
100%|█████████████████████████████████████████████████████████████████████████████████████████████| 2.10M/2.10M [00:05<00:00, 404kB/s]

View at:
https://pypi.org/project/Pillow/7.2.0/

https://pypi.org/project/Pillow/7.2.0/#files

[mattip]

Thanks!