Pillow: Tiff not being fully read(?)

[elunty]

I have a few tiffs that produce some strangeness when i try to read them. The lower and right edges of the image turns green when you try to .show or .save the file. Its in YCbCr so the green is just because its reading [0,0,0] for whatever reason, but i'm curious if anyone has seen something like this before. The image is using compression: "jpeg" and i look at the .info values.

I'm using pillow 4.0.0.. If i try to use newer versions i get a different error (ValueError: tile cannot extend outside image) when i try to call .show

[wiredfool]

Can you post the actual file that you're having issues with?

[elunty]

Sure. Too large to upload here, but here is a link

https://drive.google.com/file/d/0B4T9Q7D8qCmJWERvU1kxRXhRT1E/view?usp=sharing

[elunty]

have you had a chance to look at this?

[wiredfool]

Briefly.

I think what's happening is that the tiles of the image extend outside the bounds of the image. I'm guessing that there's not actually data there, so the image is getting extended to the boundary of the tiles with 0,0,0 data, in this case, green. (In this image, there are a bunch of 256x256 tiles, extending to pixel 16128,16128, vs an image size of 16000x16000. )

There was a change a while back that disallowed changes to the extents of the images based on the tile extents, partially because we were getting some cases where the tile extents were clearly wrong and acting as a memory exhaustion attack.

I'm not sure what the spec says about tiles extending outside the image bounds, but I would tend to think that they're invalid.

[elunty]

Oh.. so it probably gets up to 15872, sees that the next tile (15873-16128) extends past the boundary of 16000, and rather than grabbing just the valid pixels lower than the boundary (15873-16000) ... it throws away the whole thing.

[homm]

Faced with the same here:

https://github.com/jcupitt/vips-bench/issues/6#issuecomment-334449717

I'm not sure what the spec says about tiles extending outside the image bounds, but I would tend to think that they're invalid.

While de jure docs can say they're invalid, de facto they are valid because most libs can read them.

[aclark4life]

@elunty @wiredfool @homm I'm going to lean towards invalid unless we can come up with something better than "Most libs can read them" … maybe they shouldn't 😄

[homm]

@aclark4life Here is the explanation from TIFF specification: https://github.com/python-pillow/Pillow/issues/3044#issuecomment-376399226

maybe they shouldn't

I believe this is the wrong approach. Each time when Pillow doesn't open a file which others open, we punish the end user, who may even don't know about image formats and graphics libraries.

[aclark4life]

@homm Fair enough!

[wiredfool]

In another thread, it's been noted that Tiffs can have tiles that extend beyond the extents of the file, and generally decoders are supposed to throw away te extra pixels.

[aclark4life]

@wiredfool Decisions decisions decisions …

[radarhere]

Good news: This should now be fixed by #3227
Bad news: A DecompressionBombError is now generated for this image by #2583