Sharp: How do I tell if Sharp is using libvips with imagequant?

[kingjerod]

I'm trying to get Sharp to use globally installed libvips that has imagequant linked. I believe I have it installed correctly but I can't get Sharp to change the quality of a png.

This is my docker file:

FROM ubuntu:eoan
ARG VIPS_VERSION=8.9.1
ARG VIPS_URL=https://github.com/libvips/libvips/releases/download

# basic build tools
RUN apt-get update && apt-get install -y build-essential autoconf automake libtool nasm unzip wget git pkg-config curl

RUN apt-get install -y libglib2.0-dev libexpat1-dev libpng-dev libexif-dev liblcms2-dev libimagequant-dev

RUN cd /usr/local/src  && wget ${VIPS_URL}/v${VIPS_VERSION}/vips-${VIPS_VERSION}.tar.gz \
    && tar xzf vips-${VIPS_VERSION}.tar.gz

RUN cd /usr/local/src/vips-${VIPS_VERSION} && ./configure && make && make install && ldconfig

// install node, etc

When I check if Sharp is using the libraries:

ldd node_modules/sharp/build/Release/sharp.node
        linux-vdso.so.1 (0x00007ffd12100000)
        libvips-cpp.so.42 => /usr/local/lib/libvips-cpp.so.42 (0x00007fcede737000)
        libvips.so.42 => /usr/local/lib/libvips.so.42 (0x00007fcede272000)
        libgobject-2.0.so.0 => /lib/x86_64-linux-gnu/libgobject-2.0.so.0 (0x00007fcede210000)
        libglib-2.0.so.0 => /lib/x86_64-linux-gnu/libglib-2.0.so.0 (0x00007fcede0e8000)
        libstdc++.so.6 => /lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fceddefa000)
        libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcedddab000)
        libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fceddd8f000)
        libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fceddb9e000)
        libpng16.so.16 => /lib/x86_64-linux-gnu/libpng16.so.16 (0x00007fceddb66000)
        libimagequant.so.0 => /lib/x86_64-linux-gnu/libimagequant.so.0 (0x00007fceddb56000)
        libgmodule-2.0.so.0 => /lib/x86_64-linux-gnu/libgmodule-2.0.so.0 (0x00007fceddb50000)
        libexpat.so.1 => /lib/x86_64-linux-gnu/libexpat.so.1 (0x00007fceddb22000)
        liblcms2.so.2 => /lib/x86_64-linux-gnu/liblcms2.so.2 (0x00007fceddac5000)
        libexif.so.12 => /lib/x86_64-linux-gnu/libexif.so.12 (0x00007fcedda7f000)
        libffi.so.6 => /lib/x86_64-linux-gnu/libffi.so.6 (0x00007fcedda75000)
        libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcedda52000)
        libpcre.so.3 => /lib/x86_64-linux-gnu/libpcre.so.3 (0x00007fcedd9de000)
        /lib64/ld-linux-x86-64.so.2 (0x00007fcede7bb000)
        libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fcedd9c0000)
        libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcedd9ba000)

I'm not sure how to tell if libvips is actually using imagequant. When I try to set the quality on a png, it doesn't change anything. I can't tell if Sharp isn't using imagequant or vips isn't linked correctly.

Did I miss something in the ./configure for vips? From what I understand it uses pkg-config to figure out what other packages are available to use.

This is my JS code for testing:

  await sharp("test.png").png().toFile("test-sharp-100.png");
  await sharp("test.png").png({quality: 80}).toFile("test-sharp-80.png");
  await sharp("test.png").png({quality: 60}).toFile("test-sharp-60.png");
  await sharp("test.png").png({quality: 40}).toFile("test-sharp-40.png");

[lovell]

Hi, it looks like you've installed and configured libimagequant+libvips+sharp correctly. To use its quantisation feature you'll want to set the palette: true option (quality is an advanced option that you may not need).

[kingjerod]

I'm trying to reduce the quality of a png file until it goes under a certain file size. Right now I can do this with Imagemin, but I'm trying to reduce the number of dependencies I have. So I can't do that with Sharp?

[lovell]

The quality setting will only have an effect when palette: true is also set.

https://github.com/lovell/sharp/blob/1a98c390fce9ad333b1c7db2c432f8c9491ad5d1/lib/output.js#L290-L291

Perhaps an Error should be thrown when palette is false but an attempt is made to set quality? Happy to accept a PR with such a change if you're able, to help prevent others running into this problem.

[kingjerod]

Quality doesn't seem like it does anything. I have this:

for (let i = 100; i > 50; i -= 5) {
  await sharp("test.png").png({palette: true, quality: i}).toFile(`test-sharp-${i}.png`);
}

And all the files have the exact same size. Quality 100 looks the same as 50. The starting png file is 2000 kb, and quality 100 is 481 kb.

[Emibg]

I am having trouble reducing the file size of PNG images as well. Sharp works perfectly well with other file formats though (e.g. JPEG & TIFF)

[kingjerod]

@lovell What can I do to help here? Is making a Docker image with a full test helpful for you?

[lovell]

@kingjerod Yes please.

[kingjerod]

@lovell Hopefully this helps https://github.com/kingjerod/sharp-test

In my testing the PNG goes from 2.08 MB to 2.48 MB after going through Sharp (at every quality setting).

[lovell]

Thanks, it looks like there's a pngPalette -> palette typo in https://github.com/kingjerod/sharp-test/blob/master/test.js#L8

-    await sharp("test.png").png({pngPalette: true, quality: i}).toFile(`test-${i}.png`);
+    await sharp("test.png").png({palette: true, quality: i}).toFile(`test-${i}.png`);

[kingjerod]

Ahh not sure how that slipped through. My initial code had it correct. Well seems to be working now!

In a side note, I tested Imagemin vs Sharp for optimizing, Sharp can be over twice as fast! This is a test where PNG is already in a buffer, just reduced in quality and saved to disk. Times are in ms.

Creating PNG with quality 90
Sharp time: 769
Imagemin time: 1883

Creating PNG with quality 85
Sharp time: 774
Imagemin time: 1892

Creating PNG with quality 65
Sharp time: 873
Imagemin time: 1299

Creating PNG with quality 60
Sharp time: 869
Imagemin time: 1212

Thanks @lovell

[Emibg]

@kingjerod Hey, did you manage to solve your issue? Are your png files now compressing correctly?

[kingjerod]

Yes. If you're using Docker you can check out the repo I posted above, it has the very basics of an install with Sharp that will work to reduce png file size. Keep in mind it doesn't build all the same libraries as Sharp does by default so some image formats might not work right (webp, svg etc).

[Emibg]

Cool, thanks. But I'm having some issues and I was wondering if you could help me: I cloned your repo code but my png files are not compressing correctly. I am not using Docker though and when I try to verify that Sharp is using the installed vips system libraries with:

ldd node_modules/sharp/build/Release/sharp.node

I get this message back:

ldd: node_modules/sharp/build/Release/sharp.node: Exec format error

Any idea as to what might be happening?

[kingjerod]

If you're not using Docker or Ubuntu, then my repo is not helpful. The only reason it works is because all the useful things happen in the Dockerfile. If you are running Ubuntu 19.10 (Eoan) then you could copy the commands from the Dockerfile to install the libvips and other libraries and it might work. Older versions of Ubuntu might not work. Most likely any other OS will not work with those commands.

I don't know what that error is but it looks like Sharp did not install correctly.

[Emibg]

Ok, so I get that the Dockerfile is what sets up the vips libraries correctly and hence makes it work for that specific environment (Eoan Ubuntu). Would it possible for me to rewrite your build script so that it works on another platform (e.g. Windows)?

[kingjerod]

I'm not sure if you would need to build it on Windows. From what it looks like, they use Docker to build the exe for Windows anyways. They say to use the Windows builds here:

https://github.com/libvips/libvips/releases

According to that page:

The libvips Windows binary vips-dev-w64-all-x.y.z.zip contains ALL the file format loaders that libvips supports, including some very minor ones

Looks like it doesn't have Imagequant, but it does have Imagemagick in it which will do png optimization. You would need to take that zip file, unzip it somewhere and make sure your Windows path env variable points to it. Then install Sharp. I don't know if Sharp will automatically pick it up from the path variable.

[Emibg]

@kingjerod Thanks a lot!

[D0xzen]

I'm sorry to reopen but i have the same problem, my code:

sharpData.png({ palette: true, quality: 5, compressionLevel: 8, }) .toFile(filepath)

It doesn't change if change quality or compression level