Sharp: JPEG artifacts when resizing via libjpeg-turbo/mozjpeg but no artifacts resizing via official libjpeg

[quintonparker]

Using sharp 0.18.2 to resize a jpeg on OSX 10.12.5 with a reasonable pipeline

const img = mod_sharp(input);

img.resize(415, 556);

img.jpeg({
    quality: 85,
    progressive: true,
    chromaSubsampling: '4:2:0'
});

Noticeable artifacts in the following scenarios:

system vips with mozjpeg

1.brew install vips --with-mozjpeg --with-webp
2.npm install sharp
3.run reasonable sharp resize pipeline
4.Artifacts! https://www.dropbox.com/s/hxbsfziz70fct17/brew%20install%20vips%20--with-mozjpeg%20--with-webp.jpg?dl=0

system vips with jpeg turbo

1.brew install vips --with-webp --with-jpeg-turbo
2.npm install sharp
3.run reasonable sharp resize pipeline
4.Artifacts! https://www.dropbox.com/s/hl878dmsdm0f4w1/brew%20install%20vips%20--with-webp%20--with-jpeg-turbo.jpg?dl=0

Artifact-free in these scenarios:

systems vips (without jpeg turbo and without mozjpeg)

1.brew install vips ie. (no extra options. presumably bundled libjpeg is used?)
2.npm install sharp
3.run reasonable sharp resize pipeline
4.No artifacts! https://www.dropbox.com/s/hs6t0164ucsaqrj/brew%20install%20vips%20%28no%20options%29.jpg?dl=0

bundled vips

1.brew remove vips
2.brew remove libjpeg-turbo
3.brew remove mozjpeg
4.npm install sharp (on mac os)
5.No artifacts! https://www.dropbox.com/s/bmb4stf6vn78jcf/vips%20bundled%20with%20sharp%20%28macos%29.jpg?dl=0

vipsthumbnail (with jpeg turbo)

1.brew install vips --with-jpeg-turbo --with-mozjpeg
2.vipsthumbnail original.jpg -s 415x556 new.jpg[Q=85,no-subsample,optimize-coding,strip]
3.No artifacts! https://www.dropbox.com/s/shh8ykq2mvy5p54/vipsthumbnail%20%5BQ%3D85%2Cno-subsample%2Coptimize-coding%2Cstrip%5D.jpg?dl=0

Summary

Artifacts arise when resizing jpegs via sharp when vips is compiled with jpeg-turbo (or mozjpeg). No artifacts when vips compiled against libjpeg.

Worth noting all of these scenarios are also reproducible on Ubuntu Trusty

Dropbox share to all test files + original
https://www.dropbox.com/sh/aqtukbjdv0l8lom/AAAAtKp37f52b7SLunPwtBn4a?dl=0

[quintonparker]

Help. Please? What am I doing wrong?

[lovell]

Hello, we need to discover if the moire pattern is being introduced during input or output. What happens if you output to PNG format in all these cases?

If the problem is on the input side, this might be related to JPEG shrink-on-load. What happens if you add .gamma(1) to your pipeline (which disables shrink-on-load)?

[quintonparker]

Spent so much effort worrying about getting the details right on issue create I forgot my manners!

Hi @lovell and thanks for the reply 👍

re: gamma

Tried the .gamma(1) trick as advised in #848. When .gamma(1) did nothing to remedy the situation I figured this might be a new/different issue

re: png output

const img = sharp(input);
img.resize(415, 556);
img.png();

systems vips (without jpeg turbo and without mozjpeg)

No artifacts. moiré is gone!

https://www.dropbox.com/s/em1luhb853wqaiw/brew%20install%20vips%20%28no%20options%29%20%2B%20png%20output.png?dl=0

system vips (with jpeg turbo, mozjpeg, and webp)

Artifacts! Moiré is back!

https://www.dropbox.com/s/tf3qq35lj3sfwh1/brew%20install%20vips%20%28--with-jpeg-turbo%20--with-mozjpeg%20--with-webp%29%20%2B%20png%20output.png?dl=0

Hope this helps and thanks for sharing sharp

[lovell]

From your detailed reports, thank you, this problem only occurs when using a brew-installed vips that has been compiled from source on your machine (as this is what happens when optional --with-* flags are specified).

Which version of xcode/clang are you using? Are there any environment variables such as CFLAGS or CXXFLAGS set via brew --env?

[quintonparker]

> xcodebuild -version
Xcode 8.3.2
Build version 8E2002
~/workspace/imgop SSA-531* quinton.parker@02CORM-N1UHG8WN
> clang -v
Apple LLVM version 8.1.0 (clang-802.0.42)
Target: x86_64-apple-darwin16.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

> brew --env
HOMEBREW_CC: clang
HOMEBREW_CXX: clang++
SDKROOT: /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk
MAKEFLAGS: -j8
CMAKE_PREFIX_PATH: /usr/local
CMAKE_INCLUDE_PATH: /usr/include/libxml2:/System/Library/Frameworks/OpenGL.framework/Versions/Current/Headers
CMAKE_LIBRARY_PATH: /System/Library/Frameworks/OpenGL.framework/Versions/Current/Libraries
MACOSX_DEPLOYMENT_TARGET: 10.12
PKG_CONFIG_LIBDIR: /usr/lib/pkgconfig:/usr/local/Homebrew/Library/Homebrew/os/mac/pkgconfig/10.12
ACLOCAL_PATH: /usr/local/share/aclocal
PATH: /usr/local/Homebrew/Library/Homebrew/shims/super:/usr/bin:/bin:/usr/sbin:/sbin

Happy to upgrade or try something else!

[quintonparker]

Just to clarify, the problem also presents itself on Ubuntu Linux 14.04.5 LTS

[lovell]

Thank you. When you say you can reproduce this on Ubuntu 14.04, is that only when using a globally-installed libvips and/or with the pre-compiled libvips binaries? If only when self-compiling, what CFLAGS/CXXFLAGS if any were used?

[quintonparker]

HI @lovell

I can reproduce moire pattern on ubuntu using globally installed libvips (8.5.6 compiled from source with no configure flags) and I can also reproduce with pre-compiled libvips binaries

In both these cases libvips is compiled/linked against jpeg-turbo AFAIK. Apparently ubuntu packages jpeg-turbo under the package name of libjpeg* (https://packages.ubuntu.com/trusty/libjpeg-dev)

I can avoid the moire pattern if I deliberately compile libjpeg from src. And then compile global libvips from src but this time link it to my custom compiled libjpeg as follows

./configure --with-jpeg-includes=/usr/local/jpeg/include/ --with-jpeg-libraries=/usr/local/jpeg/lib/ --prefix=/usr/local/vips/

I can only conclude that the moire pattern is introduced when libvips is compiled with jpeg-turbo

I might have a strong confirmation bias 😛

Happy to explore other avenues. Let me know how can help further?

[lovell]

Thanks for all the additional info, this is an excellent bug report and a great test image. I'll need to make some time to investigate further.

[lovell]

Using the djpeg command line tool provided by libjpeg-turbo-progs on Ubuntu 16.04:

$ djpeg -version
libjpeg-turbo version 1.4.2 (build 20160222)
$ djpeg -scale 1/2 -bmp original.jpg >original-half.bmp
$ djpeg -scale 1/4 -bmp original.jpg >original-quarter.bmp
$ djpeg -scale 3/8 -bmp original.jpg >original-three-eighths.bmp

1/2 and 1/4 produce the moire pattern, 3/8 does not. The former both use a SIMD path, the latter does not, so my best guess would be rounding/clipping in the libjpeg-turbo SIMD scaling code.

[quintonparker]

great. sounds like progress. thanks for your time on this

[lovell]

Our best bet is probably to add a new, optional shrinkOnLoad boolean option to the resize() operation that exposes control over the use of shrink-on-load, defaulting to the current true behaviour.

[quintonparker]

Ok. Sounds like a pragmatic solution

Don't rush a fix or compromise on my account. I've reworked my project to use vipsthumbnail. very happy with those results

Maybe a related issue. When thumbnailing jpeg to webp via Sharp the quality is noticeably worse than thumbnailing jpeg to jpeg

With vipsthumbnail the output quality of webp is amazing. Indistinguishable vs jpeg (at least to the human eye)

[rn4391]

Added https://github.com/lovell/sharp/pull/956 to expose shrinkOnLoad in resize()

[lovell]

v0.19.0 will expose a new fastShrinkOnLoad option to resize(), defaulting to the current behaviour of true.

https://github.com/lovell/sharp/blob/suit/docs/api-resize.md#resize

[quintonparker]

@lovell sounds great

for my understanding please when enabled what does fastShrinkOnLoad offer? I guess the name is self-explanatory but I would like to know is what is the tradeoffs?

[lovell]

@quintonparker fastShrinkOnLoad: true, the default, will use the current behaviour. fastShrinkOnLoad: false will halve the shrink-on-load factor, so will be slower but can reduce the aliasing/moiré effects.

[quintonparker]

@lovell thanks for snappy response. i am enlightened!

[lovell]

sharp v0.19.0 now available.