Dali: Lanczos3 interpolation vs PIL

[melgor]

Hi,
I'm using DALI for training and works well. But I would like to try Lanczos interpolation. However, while training with dali, in production use-case I use PIL. And after some analysis, look like DALI Lanczos works much different compared to PIL (and also ImageMagic).
I provided minimal example to reproduce the difference in JupyterNotebook.
To just show the results (difference are 5x amplified to see the differences better)
DALI vs PIL

PIL vs ImageMagic

Look like that Lanczos filter is not consistent in alll library.
Also, I added 2 images in images folder. First one works better, but in second one the difference are more visible.
minimal_example.zip

[JanuszL]

@mzient can you look at it?

[mzient]

@melgor

1. The archive with example is empty.
2. Regarding the difference - it will only appear when downscaling. libImaging, which is used by PIL, keeps the filter scale from becoming less than 1 (which would happen when downscaling). This causes the filters to stop working as advertised - e.g. linear filter becomes in fact a triangular filter, Lanczos filter becomes something different yet (in Lanczos filter, the zeros should be spaced one by sample, forcing the filter scale to 1 breaks that). This produces some reasonably well looking image without much mathematical foundation.
   To get the best results, I recommend the following:
   mag_filter=types.INTERP_LANCZOS3, min_filter=types.INTERP_GAUSSIAN
   which will switch from Lanczos when upscaling to Gaussian when downscaling in given axis.

[mzient]

...conttinued:
what PIL does causes severe overshoot when downscaling - this image was scaled anisotropically (vertical downscaling from 960 to 256 pixels) - see ringing artifacts around horizontal edges:

...and lack thereof in DALI when using Gaussian for dowscaled axis:

[melgor]

minimal_example.zip

1. Sorry for uploading empty zip, now everything should work.
2. Unfortunately, Gaussian does not match the results for PIL.

Thanks for detailed explanation!

Look like I would not be able to use Lanczos as I have fashion images and look like that when there are striped shirt or checkered shirt, resized artefact influence the model. I need to look closer why such small changes in resize operation influence the model accuracy (>2%)

[JanuszL]

@melgor

Unfortunately, Gaussian does not match the results for PIL.

Do you mean DALI Gaussian or PIL Gaussian?

[JanuszL]

And do you test inference? If so you need to retrain the model using the same interpolation as you are using during the inference as model recognizes resize artifacts as the object features.

[melgor]

1. About matching the results, I mean PIL Lanczos vs DALI Gaussian. It is of course obvious :)
   But this stop me from using Lanczos interpolation from PIL in production.

2. I only test inference. This issue was about using Lanczos interpolation during training and inference stages. But as PIL have difference Lanczos interpolation than DALI, it is not possible currently. The only thing I have in my mind is to resize all images before training to have size 224x224 and just omit resizing in DALI.

[JanuszL]

PIL Lanczos vs DALI Gaussian

It won't match. The point from @mzient was to use DALI Gaussian as it provides better results when down-sampling but it won't match PIL Lanczos.

But as PIL have difference Lanczos interpolation than DALI, it is not possible currently. The only thing I have in my mind is to resize all images before training to have size 224x224 and just omit resizing in DALI.

If you use DALI for training and inference with the same resize interpolation you should get better results than mixing DALI and PIL (or any two libraries together like OpenCV and PIL).

[melgor]

I get your answers, but currently there is not option for using DALI for inference.

[JanuszL]

Sure

[mzient]

@melgor Since what we had was not really usable for downscaling anyway, we've decided to follow PIL way of dealing with Lacznos3 - it's already merged as #2041 and should be available in release 0.24 (and tomorrow's nightly build if you need it sooner).