Pillow: Unknown pseudo-tag 65538

[v-python]

What did you do?

Put program and file below in same directory. Ran program.

What did you expect to happen?

Get two copies of the file, each a little bigger.

What actually happened?

Well, it worked, but it produced the following error:

TIFFSetField: tempfile.tif: Unknown pseudo-tag 65538.

Since the original image is .png, it would seem that Pillow introduced the unknown pseudo-tag, and then complained about it.

The tag seems to be a deprecated tag for some form of JPEG compression inside TIFF, but I specified, explicitly, ZIP compression ('tiff-deflate'), so the unknown/deprecated tag is a surprise.

What are your OS, Python and Pillow versions?

• OS: windows 10
• Python: 3.6
• Pillow: 5.3

Please include code that reproduces the issue and whenever possible, an image that demonstrates the issue. Please upload images to GitHub, not to third-party file hosting sites. If necessary, add the image to a zip or tar archive.

The best reproductions are self-contained scripts with minimal dependencies. If you are using a framework such as plone, Django, or buildout, try to replicate the issue just using Pillow.

from PIL import Image, ImageOps
def foo():

    im = Image.open('testovauto.png')
    im = ImageOps.expand( im, 44, 0xffffff )
    im.save('testout.tif', dpi=(300.0, 300.0), compression='tiff_deflate')
    print('hi there')
    im = Image.open('testout.tif')
    im = ImageOps.expand( im, 44, 0xffffff )
    im.save('testout2.tif', dpi=(300.0, 300.0), compression='tiff_deflate')
    print('hi there')
    im = Image.open('testout2.tif')
    im = ImageOps.expand( im, 44, 0xffffff )
    im.save('testout3.tif', dpi=(300.0, 300.0), compression='tiff_deflate')


foo()

testovauto.png: https://user-images.githubusercontent.com/3078240/47123380-e447b280-d22f-11e8-97a4-6c18ed22045d.png

[hugovk]

TIFFSetField: tempfile.tif: Unknown pseudo-tag 65538 suggests the TIFF decoder cannot read all the tags that were written by the TIFF library. It's not a failing error, but reporting this info.

tiff_deflate compression is 32946:

https://github.com/python-pillow/Pillow/blob/2fa54408d46411be9978e3b7ec6a5c70f44e09ab/src/PIL/TiffImagePlugin.py#L119-L135

Libtiff's TIFF 6.0 Specification Coverage says:

In addition, the library supports some optional compression algorithms that are, in some cases, of dubious value.

Compression tag value | Compression algorithm
-- | --
32766 | NeXT 2-bit encoding
32809 | ThunderScan 4-bit encoding
32909 | Pixar companded 11-bit ZIP encoding
32946 | PKZIP-style Deflate encoding (experimental)
34676 | SGI 32-bit Log Luminance encoding (experimental)
34677 | SGI 24-bit Log Luminance encoding (experimental)

---

And further:

_The deflate algorithm is experimental. Do not expect to exchange files using this compression scheme; it is included only because the similar, and more common, LZW algorithm is claimed to be governed by licensing restrictions._

Therefore I'd recommend using a non-experimental compression, or just ignoring the warning.

[v-python]

I realize it isn't a failure, as the program continues. The tiff_deflate 32946 isn't what is being complained, about, though, but rather the archaic JPEG 65538. Switching to tiff_lzw produces the same error.

I found no references to 65538 in Pillow source. The only reference to the hex equivalent 10002 that I found is in JpegImagePlugin.py ... which shouldn't be involved in writing a TIFF file, although there are references to this being a TIFF-like structure. 10002 is defined as "Large Thumbnail". But this is probably a false match, regarding TIFF files.

You say that tag is being written by the TIFF library: but it seems likely that Pillow is telling it to write it... or that there would be a way Pillow could tell the TIFF library not to write it? But, looking in tifflib, the message is produced in tif_dir.c in the function OkToChangeTag... which is called from all over... setting a breakpoint there would help, but I'm not set up to compile/debug tifflib or Pillow at present.

Comments in tifflib indicate that these pseudo tags are not written to or read from the TIFF files!?

tif_getimage.c has a reference to that tag (TIFFTAG_JPEGCOLORMODE) but only in a switch for COMPRESSION_JPEG which is not in use in my test, at least not intentionally. The error message talks about "tempfile.tif", which I don't mention either!

There is a file tif_jpeg.c that has a lot of references to TIFFTAG_JPEGCOLORMODE, but unless this mysterious "tempfile.tif" is using tiff_jpeg or jpeg compression, I'm not sure why it would be involved in my test. I don't find a tempfile.tif in my directory. I don't find a reference to tempfile.tif in tifflib, but I do see one in TiffDecode.c in Pillow.

Hmm. That seems to be enlightening: TiffDecode.c has a line 237 (master branch) that says:
TIFFSetField(tiff, TIFFTAG_JPEGCOLORMODE, JPEGCOLORMODE_RGB);

I don't have a clue what this tempfile.tif is being used for, or why it would have a line such as above using an archaic (now reported as unknown) TIFFTAG value on a file that I don't mention (and which doesn't get created in my TEMP directory or current directory but maybe some other temp directory?), but since it is part of Pillow, maybe you can figure it out.

[kkopachev]

That's actually thrown on tiff read stage. I found the cause, will submit PR soon.

[radarhere]

@kkopachev has now created PR #3417

[bschollnick]

I just ran into this issue while trying to convert a batch of TIFs into PDFs...
Do we have a time table for a release, or is this available as pre-release?

[radarhere]

This will be available as part of 5.4.0, due for release on the 1st of Jan. If you would like it sooner, you can download and install from the master branch.