Pwa-studio: [bug]: Upward crashes on Image Optimization

[sven-codeculture]

Describe the bug
When turning on the PWA Image optimization, the Upward Server crashes as soon as someone tries to load any image.

Error log shows:

events.js:174
      throw er; // Unhandled 'error' event
      ^
Error: Input buffer contains unsupported image format
Emitted 'error' event at:
    at sharp.pipeline (/usr/src/app/node_modules/sharp/lib/output.js:707:18)
error Command failed with exit code 1.

Expected behavior
Image is shown to the customer and Optimized for the PWA.

Please complete the following device information:

• PWA Studio Version: 6.0.1
• NPM version npm -v: v6.13.4
• Node Version node -v: v10.18.1

Please let us know what packages this bug is in regards to:

• [ ] venia-concept
• [ ] venia-ui
• [ ] pwa-buildpack
• [ ] peregrine
• [ ] pwa-devdocs
• [x] upward-js
• [ ] upward-spec
• [ ] create-pwa

[m2-assistant[bot]]

Hi @sven-codeculture. Thank you for your report.
To help us process this issue please make sure that you provided sufficient information.

Please, add a comment to assign the issue: @magento I am working on this

---

• Join Magento Community Engineering Slack and ask your questions in #github channel.

[Jordaneisenburger]

Hi @sven-codeculture ,

I'm experiencing the same issue, did this happen after you've updated to 6.x ?

I tried connecting the PWA to our staging domain instead of live and that seems to resolve the issue. Making me wonder if the new version of pwa studio's sharp lib can't properly handle nginx redirects.

[sven-codeculture]

@Jordaneisenburger i think it was after the 6.x update yes. I did not do the update myself, but one of my co-workers did.
I hope this issue gets solved ASAP since turning of the optimization is also not a solution but only a workaround.

[Jordaneisenburger]

Yeah it's been mind boggling for us, looks like hastly is throwing it's error to sharp. Still investigating .

Strange thing is that we don't have the issue if we talk to our staging env but only get it when talking to live. Seems to be server related.

Our best bet is waiting for @zetlen to chime in on this.

[awilcoxa]

Created PWA-666

[Jordaneisenburger]

"666" the issue from hell

[Jordaneisenburger]

@sven-codeculture did you find a solution, clue or workaround for this issue?

[sven-codeculture]

@Jordaneisenburger unfortunatly no, since we don't have the time to look into this we just disabled the Image optimization for now

[Jordaneisenburger]

@sven-codeculture Thanks for the response, unfortunately that's not a solution for us. Will keep you posted on any progress

[Jordaneisenburger]

Update:

So we have 1 customers running in Live modus hosted on a hypernode server from Byte and our own internal project running on the same version.

When I'm developing locally and connect to the live MAGENTO_BACKEND_URLS it crashes both for the internal project and the live customer :(

But I'm developing locally and I connect the MAGENTO_BACKEND_URLS to our stagings (hosted by ourselves) it all works fine :)

BUT, we also have our internal project running in live mode on that hypernode server and that is working fine. So that way the storefront is basically talking to itself instead of my local machine to an external server.

I think it might have something to do with the veniaProxy in conjunction with not running the storefront not on the same server.

[zetlen]

It only took 16 days for me to chime in! Sorry for the wait, folks.

The repro for this is tough. My first task will be to try and float errors from sharp better in hastily.

In the meantime, an independent repro would be really nice to have.

[zetlen]

Aha, just caught @Jordaneisenburger mentioning nginx redirects. That could be it! I don't think that's a use case I tested for in hastily.

The main change in hastily is that we now process the image _as the request is streaming_, by patching the streaming methods of the HTTP response. In prior versions which used express-sharp, the middleware was fetching the entire image in a separate HTTP call, then processing it and returning it. In the latter case, I believe the fetch followed redirects implicitly. In the former case...maybe not! I'll have a look.

[Jordaneisenburger]

No worries, thanks for taking the time to look at this. If it helps I can give you a Magento backend url that fails or even the whole repo for that matter. DM me if needed.

[Jordaneisenburger]

@sven-codeculture James figured out that it has something to do with Brotli-compression so if you are able to turn it of server side (NGINX), you should.

[zetlen]

@sven-codeculture Jordan is right! Your issue is that your nginx is configured out of the box to gzip images, which is a bad idea!

The bug was in hastily; it crashed instead of bypassing when it encountered a gzipped image. I've released a new version which logs an appropriate warning message when your backend isn't configured right, and forwards the image through without optimizing. This is the same behavior hastily shows when the image is an unrecognized type, or when it's already been compressed: https://github.com/zetlen/hastily#request-filtering

Opening a PR to update Buildpack shortly!

[Jordaneisenburger]

@zetlen While developing I found another issue not sure if this is just on dev or on live as well I should check PM2 logs

` throw er; // Unhandled 'error' event
^

Error: VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 0
VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 0
VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 0

Emitted 'error' event at:
at sharp.pipeline (/Users/jordan/experius/domains/CUSTOMER_NAME/node_modules/sharp/lib/output.js:707:18)`

[Jordaneisenburger]

Mhh I assume we might have some sort of broken image, just crashed again in development. So for completeness:

Error: VipsJpeg: Premature end of JPEG file
VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 0
VipsJpeg: out of order read at line 0
VipsJpeg: Premature end of JPEG file
VipsJpeg: out of order read at line 0

Emitted 'error' event at:
    at sharp.pipeline (/Users/jordan/experius/domains/bascom-m2-frontend/node_modules/sharp/lib/output.js:707:18)

(sharp:5932): GLib-CRITICAL **: 13:24:32.627: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_insert_internal: assertion 'hash_table != NULL' failed

(sharp:5932): GLib-CRITICAL **: 13:24:32.628: g_hash_table_lookup: assertion 'hash_table != NULL' failed
error Command failed with signal "SIGSEGV".

[zetlen]

Uh oh. Can you reproduce that with any other image?

[zetlen]

@Jordaneisenburger From our conversation at the meeting this morning, I took your suggestion to try an invalid PNG instead of a JPG, and I reproduced your error.

I just wasn't handling errors in the sharp mapper or sharp stream properly. It's a quick fix! I just released hastily v0.4.7 and the PR #2535 is now up to date with that version. Let me know if it fixes your issue (we can continue in the #2535 comments)

[Jordaneisenburger]

Hi @zetlen, It was hard to consistently reproduce this error so I will update my packages and keep monitoring it. I'll close this issue