Gatsby: updating gatsby-plugin-sharp from 2.8.0 to 2.9.0 breaks build
Description
error pngload: non-recoverable state
pngload: non-recoverable state
vips2png: unable to write to target target
Error: pngload: non-recoverable state
pngload: non-recoverable state
vips2png: unable to write to target target
not finished run page queries - 1.474s
not finished Generating image thumbnails - 1.299s
error Command failed with exit code 1.
I can up- and downgrade repeatedly. v2.8.0 always works, v2.9.0 always errors.
Steps to reproduce
This is a complex app. I have no idea if this happens generally.
- clone https://github.com/barbalex/apf2
- check that gatsby-plugin-sharp is set to v2.8.0
yarn && yarn dev> should succeed- set gatsby-plugin-sharp to v2.9.0 > should error
Expected result
Build should succeed
Actual result
see above
Environment
Run gatsby info --clipboard in your project directory and paste the output here.
$ yarn gatsby info --clipboard
yarn run v1.22.5
$ C:\Users\alexa\apf2\node_modules\.bin\gatsby info --clipboard
System:
OS: Windows 10 10.0.19042
CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
Binaries:
Node: 15.0.1 - ~\AppData\Local\Temp\yarn--1605869962190-0.9665755114141126\node.CMD
Yarn: 1.22.5 - ~\AppData\Local\Temp\yarn--1605869962190-0.9665755114141126\yarn.CMD
npm: 7.0.3 - C:\Program Files\nodejs\npm.CMD
Languages:
Python: 2.7.17
Browsers:
Chrome: 86.0.4240.198
Edge: Spartan (44.19041.423.0), Chromium (86.0.622.69)
npmPackages:
gatsby: 2.27.0 => 2.27.0
gatsby-cli: 2.14.0 => 2.14.0
gatsby-image: 2.6.0 => 2.6.0
gatsby-plugin-create-client-paths: 2.5.0 => 2.5.0
gatsby-plugin-eslint: 2.0.8 => 2.0.8
gatsby-plugin-manifest: 2.7.0 => 2.7.0
gatsby-plugin-offline: 3.5.0 => 3.5.0
gatsby-plugin-react-helmet: 3.5.0 => 3.5.0
gatsby-plugin-sharp: 2.8.0 => 2.8.0
gatsby-plugin-styled-components: 3.5.0 => 3.5.0
gatsby-plugin-typography: 2.7.0 => 2.7.0
gatsby-remark-autolink-headers: 2.6.0 => 2.6.0
gatsby-remark-copy-linked-files: 2.5.0 => 2.5.0
gatsby-remark-embed-video: 3.0.10 => 3.0.10
gatsby-remark-emojis: 0.4.3 => 0.4.3
gatsby-remark-external-links: 0.0.4 => 0.0.4
gatsby-remark-images: 3.6.0 => 3.6.0
gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
gatsby-remark-prismjs: 3.8.0 => 3.8.0
gatsby-source-filesystem: 2.6.0 => 2.6.0
gatsby-transformer-remark: 2.11.0 => 2.11.0
gatsby-transformer-sharp: 2.7.0 => 2.7.0
Done in 1.99s.
barbalex
All 14 comments
Thanks for the report, I did run it locally and confirmed I see same result locally.
Only thing to keep in mind that repo as it as right now have mix of incompatible sharp versions:
➜ i28203 git:(master) yarn why sharp
yarn why v1.22.10
[1/4] 🤔 Why do we have the module "sharp"...?
[2/4] 🚚 Initialising dependency graph...
[3/4] 🔍 Finding dependency...
[4/4] 🚡 Calculating file sizes...
=> Found "[email protected]"
info Has been hoisted to "sharp"
info Reasons this module exists
- Hoisted from "gatsby-plugin-manifest#sharp"
- Hoisted from "gatsby-transformer-sharp#sharp"
info Disk size without dependencies: "25.43MB"
info Disk size with unique dependencies: "26.81MB"
info Disk size with transitive dependencies: "29.48MB"
info Number of shared dependencies: 54
=> Found "gatsby-plugin-sharp#[email protected]"
info This module exists because "gatsby-plugin-sharp" depends on it.
info Disk size without dependencies: "29.48MB"
info Disk size with unique dependencies: "31.02MB"
info Disk size with transitive dependencies: "33.94MB"
info Number of shared dependencies: 58
Which result in
success open and validate gatsby-configs - 0.036s
⠁ dyld: lazy symbol binding failed: Symbol not found: _g_once_impl
Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib
dyld: Symbol not found: _g_once_impl
Referenced from: /Users/misiek/test/i28203/node_modules/sharp/build/Release/sharp.node
Expected in: /Users/misiek/test/i28203/node_modules/gatsby-plugin-sharp/node_modules/sharp/build/Release/../../vendor/lib/libvips.42.dylib
/bin/sh: line 1: 45613 Killed: 9 gatsby build
error Command failed with exit code 137.
Crash almost immediately (at least on MacOS) - using yarn resolutions in package.json to set sharp version to either 0.25.4 (it works) or 0.26.3 (it crashes) was my way around this.
It feels like issue is in sharp package - given that toggling between sharp version and not even touching gatsby-plugin-sharp version cause the failure / successful builds.
Some initial details - here is example image data and arguments we pass to sharp where it fails (I'm not saying it's faulty image - it's just a way to be able to do minimal reproduction we could report in sharp repository):
ERROR {
file: {
id: '3e170652-32f4-59b6-ad85-4d7a4e836841',
children: [ '1c940b52-b428-5f8c-92b8-b237dd0ccd5e' ],
parent: null,
internal: {
contentDigest: 'db413ad7ad34df66d973d41a2829073b',
type: 'File',
mediaType: 'image/png',
description: 'File "src/benutzerDoku/ekPlanen/ekplanen_global.png"',
counter: 147,
owner: 'gatsby-source-filesystem'
},
sourceInstanceName: 'benutzerDoku-pages',
relativePath: 'ekPlanen/ekplanen_global.png',
extension: 'png',
prettySize: '13.7 kB',
modifiedTime: '2020-11-20T11:54:51.702Z',
accessTime: '2020-11-20T12:16:31.549Z',
changeTime: '2020-11-20T11:54:51.702Z',
birthTime: '2020-11-20T11:54:51.702Z',
root: '/',
dir: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen',
base: 'ekplanen_global.png',
ext: '.png',
name: 'ekplanen_global',
absolutePath: '/Users/misiek/test/i28203/src/benutzerDoku/ekPlanen/ekplanen_global.png',
relativeDirectory: 'ekPlanen',
dev: 16777221,
mode: 33188,
nlink: 1,
uid: 501,
rdev: 0,
blksize: 4096,
ino: 205212343,
size: 13714,
blocks: 32,
atimeMs: 1605874591548.6438,
mtimeMs: 1605873291702.1418,
ctimeMs: 1605873291702.1418,
birthtimeMs: 1605873291702.044,
atime: '2020-11-20T12:16:31.549Z',
mtime: '2020-11-20T11:54:51.702Z',
ctime: '2020-11-20T11:54:51.702Z',
birthtime: '2020-11-20T11:54:51.702Z'
},
args: {
plugins: [],
maxWidth: 1500,
wrapperStyle: 'margin-left: 0;',
linkImagesToOriginal: false,
showCaptions: false,
markdownCaptions: false,
sizeByPixelDensity: false,
backgroundColor: 'white',
quality: 50,
withWebp: false,
tracedSVG: false,
loading: 'lazy',
disableBgImageOnAlpha: false,
disableBgImage: false,
pathPrefix: ''
}
}
Those are arguments we pass to fluid function in https://github.com/gatsbyjs/gatsby/blob/fe18a18812276c19c2bef654f08039f1037113f7/packages/gatsby-remark-images/src/index.js#L151-L156
pieh
on 20 Nov 2020
@barbalex It appears to be caused by corrupt images, though sharp should clearly not be crashing. The images in question are the "ekplanen" pngs. I have reported the issue to sharp, and we will be reverting our version until they have a fix. Thanks for the report.
ascorbic
on 20 Nov 2020
Just 2 cents about crashing part - it's not crashing the process, but it is throwing (we can catch it). We do fail the build with it tho (which we should of course when something fails). It didn't use to throw before (in [email protected] line) - it's not clear to me if it was silent no-op or not - I didn't get that far in my investigation
pieh
on 20 Nov 2020
Please see https://github.com/lovell/sharp/issues/2448#issuecomment-731279993
lovell
on 20 Nov 2020
I wonder what "corrupt" exactly means. The image can be viewed locally and is shown in the app just fine.
barbalex
on 20 Nov 2020
I wonder what "corrupt" exactly means.
This image contains an ICC (colour) profile with an invalid checksum.
$ pngcheck ekplanen_ap.png
ekplanen_ap.png CRC error in chunk iCCP (computed 66723ff5, expected 62c14516)
The venerable libpng, as used by Chrome, ignores this error.
As of v0.26.0, sharp has started to provide the more modern libspng, hence the need to now explicitly confirm you are willing to accept/trust this image via failOnError: false.
lovell
on 20 Nov 2020
Also, gatsby-plugin-sharp supports failOnError option:
https://www.gatsbyjs.com/plugins/gatsby-plugin-sharp/#allow-build-to-continue-on-image-processing-error
Was introduced in #27345
vladar
on 20 Nov 2020
This will be addressed in the next release of libspng, the default configuration is supposed to match the reference implementation: https://github.com/randy408/libspng/issues/14#issuecomment-731580332
randy408
on 21 Nov 2020
Thanks @randy408
ascorbic
on 21 Nov 2020
I've created an internal ticket to catch these errors and log a more friendly message along the lines of "Could not process the image 'foo.png' because it is corrupt"
ascorbic
on 23 Nov 2020
@vladar
In gatsby-config.js I changed 'gatsby-plugin-sharp' to:
{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }
But I still get the same error when using "gatsby-plugin-sharp": "2.10.0".
barbalex
on 7 Dec 2020
I tried this using "gatsby-plugin-sharp": "2.11.1". Result: still get the exact same error. But it does not block the build any more. So I feel that this has been solved. Thanks guys!
barbalex
on 16 Dec 2020
Uups. Suddenly the following error breaks the build again:
alexa@PCA C:\Users\alexa\apf2
$ yarn build
yarn run v1.22.5
$ gatsby clean && gatsby build
info Deleting .cache, public, C:\Users\alexa\apf2\node_modules\.cache\babel-loader, C:\Users\alexa\apf2\node_modules\.cache\terser-webpack-plugin
info Successfully deleted directories
success open and validate gatsby-configs - 0.104s
success load plugins - 1.035s
success onPreInit - 0.038s
success delete html and css files from previous builds - 0.003s
success initialize cache - 0.005s
success copy gatsby files - 0.425s
success onPreBootstrap - 0.021s
success createSchemaCustomization - 0.046s
success Checking for changed pages - 0.002s
success source and transform nodes - 0.259s
success building schema - 0.363s
info Total nodes: 262, SitePage nodes: 38 (use --verbose for breakdown)
success createPages - 0.054s
success Checking for changed pages - 0.001s
success createPagesStatefully - 0.076s
success update schema - 0.017s
success onPreExtractQueries - 0.001s
success extract queries from components - 2.593s
success write out redirect data - 0.002s
success Build manifest and related icons - 0.754s
success onPostBootstrap - 0.762s
info bootstrap finished - 8.410s
success run static queries - 0.024s - 1/1 42.07/s
warn code block or inline code language not specified in markdown. applying generic code block
ERROR #gatsby-transformer-remark_gatsby-plugin-sharp-20000
Failed to process image C:/Users/alexa/apf2/src/benutzerDoku/ekPlanen/ekplanen_global.png.
It is probably corrupt, so please try replacing it. If it still fails, please open an issue with the image attached.
Error: pngload: non-recoverable state
vips2png: unable to write to target target
not finished run page queries - 1.246s
not finished Generating image thumbnails - 1.137s
error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.
Here the result from gatsby info --clipboard:
$ yarn gatsby info --clipboard
yarn run v1.22.5
$ C:\Users\alexa\apf2\node_modules\.bin\gatsby info --clipboard
System:
OS: Windows 10 10.0.19042
CPU: (16) x64 Intel(R) Core(TM) i9-9900K CPU @ 3.60GHz
Binaries:
Node: 15.0.1 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\node.CMD
Yarn: 1.22.5 - ~\AppData\Local\Temp\yarn--1608657460259-0.46723258413605295\yarn.CMD
npm: 7.0.3 - C:\Program Files\nodejs\npm.CMD
Languages:
Python: 2.7.17
Browsers:
Chrome: 87.0.4280.88
Edge: Spartan (44.19041.423.0), Chromium (87.0.664.66)
npmPackages:
gatsby: 2.29.1 => 2.29.1
gatsby-cli: 2.16.1 => 2.16.1
gatsby-image: 2.8.0 => 2.8.0
gatsby-plugin-create-client-paths: 2.7.0 => 2.7.0
gatsby-plugin-eslint: 2.0.8 => 2.0.8
gatsby-plugin-manifest: 2.9.0 => 2.9.0
gatsby-plugin-offline: 3.7.0 => 3.7.0
gatsby-plugin-react-helmet: 3.7.0 => 3.7.0
gatsby-plugin-sharp: 2.11.1 => 2.11.1
gatsby-plugin-styled-components: 3.7.0 => 3.7.0
gatsby-plugin-typography: 2.9.0 => 2.9.0
gatsby-remark-autolink-headers: 2.8.0 => 2.8.0
gatsby-remark-copy-linked-files: 2.7.0 => 2.7.0
gatsby-remark-embed-video: 3.0.10 => 3.0.10
gatsby-remark-emojis: 0.4.3 => 0.4.3
gatsby-remark-external-links: 0.0.4 => 0.0.4
gatsby-remark-images: 3.8.0 => 3.8.0
gatsby-remark-images-medium-zoom: 1.7.0 => 1.7.0
gatsby-remark-numbered-footnotes: 1.0.1 => 1.0.1
gatsby-remark-prismjs: 3.10.0 => 3.10.0
gatsby-source-filesystem: 2.8.0 => 2.8.0
gatsby-transformer-remark: 2.13.0 => 2.13.0
gatsby-transformer-sharp: 2.9.0 => 2.9.0
and here an excerp from my gatsby-config:
{ resolve: 'gatsby-plugin-sharp', options: { failOnError: false } }
So that is just the first image that breaks the build. I would probably have to replace _lots_ of images to get my build to work again 😢
barbalex
on 22 Dec 2020
Related issues
magicly
·
3Comments
totsteps
·
3Comments
3CordGuy
·
3Comments
jimfilippou
·
3Comments
dustinhorton
·
3Comments
Most helpful comment
I've created an internal ticket to catch these errors and log a more friendly message along the lines of "Could not process the image 'foo.png' because it is corrupt"