Pillow: Quantization tables (qtables) being altered

[damianmoore]

I am working on a project where I am creating small JPEG thumbnails that will all share the same quantization table. I want to get the best quality for my file size so am prepared to spend time manipulating and comparing my sample set to get the best qtable for the use case.

It is my understanding that using the quality attribute is essentially just selecting a predefined qtable within the JPEG library and that providing my own qtable should make the quality attribute unnecessary. However, it seems that there is always some other quantization happening when I want my own tables to be used.

In my test I have re-sampled images to 48x48 and saved them as 7% quality JPEGs.

If I open the image using Pillow and inspect the quantisation I get this.

In [13]: im.quantization
Out[13]: {0: array('b', [114, 79, 86, 100, 86, 71, 114, 100, 93, 100, -127, 121, 114, -120, -85, -1, -70, -85, -99, -99, -85, -1, -6, -1, -49, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]), 1: array('b', [121, -127, -127, -85, -106, -85, -1, -70, -70, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1])}

If I then re-sample the same source image again to the same dimensions and use the qtable from above, save it (at 100% quality as I have to set something) then load and inspect the quantization attribute I get the following.

In [15]: im.quantization
Out[15]: {0: array('b', [114, 79, 86, 100, 86, 71, 114, 100, 93, 100, -1, 121, 114, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1]), 1: array('b', [121, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1, -1])}

You can see that the tables do not match and there is a lot less detail in both the Luma and Chroma tables as there are more values set as -1. The file size is also slightly less.

I would expect that a user supplied qtable would take precedence over the one that would be from the quality if it is set. Unfortunately I can't see that there is any way not to provide a quality attribute. Instead I think what is happening is that both qtables are being applied to the image which degrades it more than it should.

In the test image below you'll see detail is lost and noticeable chroma variations are introduced when trying to re-use the same qtable. Curiously, when the quality parameter is lowered the file size goes up which I assume it is because the two qtables are fighting one another. This is very noticeable in the output at 10% quality.

[aclark4life]

Is this a bug?

[thebostik]

(randomly came across this). There was an overflow previously due to using signed chars, you can see this was the issue in the report (e.g., the first occurrence of -127 turned into -1).

This should be fixed as of #1814 / #1921 (with test cases to do round-trips on custom qtables!) so this can probably be closed.

[wiredfool]

Ok, Closing. Feel free to a new issue if it's not working in the current release.

[damianmoore]

Yes, I am happy to confirm this is now fixed after I upgraded to the current Pillow version. I'm able to provide a qtable without having to provide a quality and this has allowed me to continue with my work on https://github.com/damianmoore/microjpeg
Thanks!