Pillow: Support for HEIF

[pythonicrubyist]

Do you have any plans to support HEIF image format, recently introduced by apple?

[wiredfool]

I would welcome support, but I'm not aware of any appropriately licensed HEIF or HEVC libraries to do encoding/decoding.

[CounterPillow]

The reference HEVC encoder/decoder is licensed as BSD 3-clause (but obviously extremely slow and tuned for PSNR), the openHEVC decoder is licensed as LGPL2.1.

Patent licenses are the bigger issue; the current patent licensing situation with 3 competing pools means that it'd be practically impossible for any FOSS project to produce binaries including a HEVC decoder which don't violate some patents.

So even if someone were to do the work to implement it, you wouldn't find it in most Pillow builds, because nobody wants to get harassed by the patent trolls.

[wiredfool]

The HEIF container decoder is also not a free license: https://github.com/nokiatech/heif/blob/master/LICENSE.TXT .

[CounterPillow]

There's an initial HEIF support patch for FFmpeg floating around on their mailing list, but as the post itself says, it's early work and the whole format is batshit insane. The container is based off of MP4 for a start.

I don't think it'll be possible to support HEIF in Pillow in a sane manner based on the container and codec complexity alone, and even if someone were to make a demuxer+decoder in one library with a license Pillow could use, you wouldn't be able to build a product on it for around 30 more years, assuming the patents remain unlicenseable for anyone without big money lawyers.

[wiredfool]

I'm definitely -1 on batshit insane.

I think that the only way it could work is if we shipped support that was dynamically linked and based off of OS libraries, e.g. CoreImage on OSX. But it's not clear that would work, and it would only be the one platform at this point.

I'm going to close this, if the world comes around to the point that it's reasonable, we can look at it.

[rspeed]

What about libheif? It takes care of the packaging while using libde265 for reading and x265 for writing. All three libraries are LGPL.

My understanding is that there would be no patent issues whatsoever for Pillow. The HEIF container itself has a blanket patent waiver, and the HEVC licensing authority doesn't require a license for software that isn't bundled with devices.

Under the software initiative, HEVC Advance will not seek a license or royalties on HEVC functionality implemented in application layer software downloaded to mobile devices or personal computers after the initial sale of the device, where the HEVC encoding or decoding is fully executed in software on a general purpose CPU. Examples of the types of software within the policy include browsers, media players and various software applications.

As far as I can tell, there's nothing blocking this other than the current instability of libheif's API.

Edit: Found a better link for the royalty-free license announcement and added a summary.

[farindk]

libheif encoder API is stable since version 1.1.0. It will appear in Debian testing in a few days:
https://packages.qa.debian.org/libh/libheif.html

I have not much experience with python, but if you need any assistance, feel free to contact me (I'm the libheif author).

[CounterPillow]

@rspeed HEVC Advance is only one of three competing HEVC licensing authorities.

[rspeed]

@CounterPillow So we'd need similar guarantees from the others?

[CounterPillow]

Yes, or just hope none of them feel like trying to enforce their patents against you. Though I don't think patent issues should prevent HEIF support from being added to Pillow. It's up to the companies or individuals building commercials products using Pillow to ensure their product doesn't infringe on any patents; I was simply pointing out that Linux distros like RedHat or OpenSuSE likely will want to ship Pillow without HEIF support.

[ncalligaro]

It seems Nokia released an open source HEIF Javascript reader, and also C libraries to read and write HEIF: https://github.com/nokiatech/heif/wiki . Wouldn't it be possible to use this to add support to the library?

[hugovk]

@ncalligaro Unfortunately no, see above: https://github.com/python-pillow/Pillow/issues/2806#issuecomment-338200112.

[oebilgen]

Sorry, why are you not using libheif?

[radarhere]

@oebilgen See https://github.com/python-pillow/Pillow/issues/2806#issuecomment-338200112

[Cykooz]

May be I wrong, but this license only applies to Nokia's software, not the HEIF container format.

Nokia Technologies Ltd (“Nokia”) hereby grants to you a ... license,
under its copyrights and Licensed Patents only to, use, run, modify (in a way that still complies
with the Specification), and copy the Software within the Licensed Field.

libheif is not based on Nokia`s code and don't uses Nokia's library. This is an independent implementation of decoder/encoder of HEIF images under GPLv3 license. This library are included in the official repositories of Debian, Ubuntu and other Linux distros.

[radarhere]

@Cykooz I feel like this has already been mentioned - https://github.com/python-pillow/Pillow/issues/2806#issuecomment-370985911

[rspeed]

@radarhere I'm not sure what you're referring to. Nothing I said in that comment disagrees with the point @Cykooz was making.

[r0bdiabl0]

I think it's time to implement libheif if not as a decoder only. I'm noticing a lot of products out there support the conversion of heic files to jpg but not the other way around. I think this is how they are getting away with not worrying about patent issues, etc... But as @rspeed noted in #2806 , there are no issues if you are using the format in the application layer as long as the device already has a license in it's initial software:

Under the software initiative, HEVC Advance will not seek a license or royalties on HEVC functionality implemented in application layer software downloaded to mobile devices or personal computers after the initial sale of the device, where the HEVC encoding or decoding is fully executed in software on a general purpose CPU. Examples of the types of software within the policy include browsers, media players and various software applications.

ImageMagick is using it so why not Pillow.

[instantlinux]

This issue, I think, very definitely needs revisiting in 2020. A large percentage of all image/video files in the world are now generated, managed and stored natively in heic/heif format. The python community needs a tool for coping with this reality.

[simonw]

If anyone needs a workaround for this, I managed to open a HEIC file using the pyheif library and convert it to a Pillow Image like this:

from PIL import Image
import pyheif

heif_file = pyheif.read_heif("IMG_7424.HEIC")
image = Image.frombytes(mode=heif_file.mode, size=heif_file.size, data=heif_file.data)
image.save("IMG_7424.jpg", "JPEG")

[bagipriyank]

is this something that PIL might support in the future?

[qarthandgi]

I'm here to reiterate my support for HEIF/HEIC files in mid-2020. We've seen a wave of usage with this file format, and the de-facto Python image processing library not being able to handle it is becoming an issue.

[crawsome]

Thank you for all your work so far, PIL team. Bump for the PIL to support HEIC/HEIF formats. I was trying to automate conversion the other day and was a little shocked to see it wasn't supported.

Supporting reasons HEIC should be a priority:

• I'm running Windows Education version 10 build 2004 and I see there is still has no native support yet.
• For many end users, the format is not trival to work with
• It is becoming more prominent as Apple takes more market share
• Users need relief from low-quality software. They are currently driven by search engines to potentially-dangerous closed-source programs for working with HEIC/HEIF.

Thank you :)

[hugovk]

I can't really comment on licensing, but Pillow uses a plugin for each image format, and you can use a third-party plugin with Pillow, without needing changes to Pillow itself. Here's the guide:

• https://pillow.readthedocs.io/en/stable/handbook/writing-your-own-file-decoder.html

Someone could already write a plugin. And then if the licensing was subsequently deemed not a blocker, it would be fairly easy to integrate it into Pillow.