Pillow: setup.py build doesn't copy headers -> wheels don't contain headers

[misuzu]

What did you do?

Trying to build Minecraft-Overviewer from source

What did you expect to happen?

Minecraft-Overviewer build is successful

What actually happened?

overviewer_core/src/overviewer.h:40:10: fatal error: Imaging.h: No such file or directory

It seems that setup.py build doesn't copy header files into its build directory.
This prevents from distributing applications that use libImaging via PyPI because it's impossible to build them without hacks. I have checked macOS and Linux wheels and they do not contain headers. For example numpy does contain necessary header files and there is no build issues when using wheels for building apps with C extensions.

Build page of Minecraft-Overviewer contain this:

If you choose to install pillow through pip instead of your distribution’s package manager, you won’t get the pillow headers which Overviewer requires to build its C extension. In that case, you should manually download the header files specific to the version of pillow you installed, and point at them with the PIL_INCLUDE_DIR environment variable. A version mismatch between the installed pillow library and the headers can lead to segfaults while running Overviewer due to an ABI mismatch.

So you can either install Pillow using distribution package manager (Debian package contains hack to copy header files) or copy headers manually.

I'm using Pillow included in nixpkgs which builds it using setup.py build and doesn't have hack to copy header files (like Debian). I can probably send pull-request to nixpkgs but maybe this can be fixed upstream and make everyone happy. Just take a look at this horror when you need to build Minecraft-Overviewer for macOS.

What are your OS, Python and Pillow versions?

• OS: NixOS
• Python: 3.7
• Pillow: 6.2.1

[radarhere]

Hi. Thanks for your input. Feel free to disagree with my thoughts.

Looking at Minecraft Overviewer, it has its own version of Draw.c.

Once you've copied that much code from Pillow, for the problem you're describing, I'd actually be more inclined to suggest to Minecraft Overviewer that they copy the header code as well. I would think that is safer, in terms of avoiding compatibility issues with whatever version of Pillow might be used.

In fact, one of the numpy header files thinks along these lines as well -

• If you want to use this for your own projects, it's recommended to make a
• copy of it. Although the stuff below is unlikely to change, we don't provide
• strong backwards compatibility guarantees at the moment.

Looking at the description on https://github.com/overviewer/Minecraft-Overviewer/commit/03a86978665548d1ddbdaf30a9fc57778893cb14, it seems that there has already been a conflict between Pillow and Minecraft Overviewer in the past.

If you feel like this would be useful generally, I'd be interested if you know of another project that uses Pillow C code.

[hugovk]

This does sound like an issue in Minecraft Overviewer.

Pillow's public API is in Python. The C code is a private, internal API, not something we make available for other C programs.

If Minecraft Overviewer is failing to build (vendored or otherwise) Pillow C code, I'd suggest reporting it directly to Minecraft Overviewer.

[misuzu]

Thanks for your input!

[wiredfool]

I’m with @hugovk on this, the C interface can be changed arbitrarily, and has no guarantees at all, while the python interface is the public portion of the api.

[radarhere]

It turns out that Minecraft Overviewer isn't interested in fixing this from their end.

Sorry @misuzu. Sounds like there isn't anything to be done here.

[radarhere]

You responded 'confused', so let me try and clarify. The comments from other Pillow members above indicate that we don't actually intend for Pillow to be used at a C level externally. Pillow is a Python library, not a C library. So Pillow won't make a change for the sake of a scenario it doesn't support.

The Minecraft Overviewer member thinks otherwise - maybe they think that we are obligated to support this scenario, or maybe they just didn't read the comments on this issue.

[misuzu]

Note that most distribution do contain pillow's headers in their packages.
Anyway closing this. Thanks for your time.