Gatsby: [bug]: potrace fails randomly with "cannot read property bitmap of undefined"

[baobabKoodaa]

Running gatsby develop works most of the time, but I randomly get this error. It seems that I'm _more likely_ to get the error after running gatsby clean. Today I ran gatsby develop 3 times in a row with no changes to anything. The first 2 times I got this error and the 3rd time it worked.

Generating image thumbnails [==============================] 5/5 0.1 secs 100%
error UNHANDLED EXCEPTION


  TypeError: Cannot read property 'bitmap' of undefined

  - Potrace.js:1000 Potrace._processLoadedImage
    [blog]/[potrace]/lib/Potrace.js:1000:35

  - Potrace.js:1046 Jimp.<anonymous>
    [blog]/[potrace]/lib/Potrace.js:1046:14

  - index.js:85 Jimp.throwError
    [blog]/[jimp]/index.js:85:44

  - index.js:298 
    [blog]/[jimp]/index.js:298:44

  - png.js:81 exports.PNG.<anonymous>
    [blog]/[pngjs]/lib/png.js:81:7

  - parser-async.js:34 module.exports.ParserAsync._handleError
    [blog]/[pngjs]/lib/parser-async.js:34:8

  - parser.js:50 module.exports.Parser._parseSignature
    [blog]/[pngjs]/lib/parser.js:50:12

  - chunkstream.js:174 module.exports.ChunkStream._processRead
    [blog]/[pngjs]/lib/chunkstream.js:174:13

  - chunkstream.js:193 module.exports.ChunkStream._process
    [blog]/[pngjs]/lib/chunkstream.js:193:14

  - chunkstream.js:61 module.exports.ChunkStream.write
    [blog]/[pngjs]/lib/chunkstream.js:61:8

  - chunkstream.js:74 module.exports.ChunkStream.end
    [blog]/[pngjs]/lib/chunkstream.js:74:10

  - png.js:98 exports.PNG.PNG.end
    [blog]/[pngjs]/lib/png.js:98:16

  - png.js:88 exports.PNG.PNG.parse
    [blog]/[pngjs]/lib/png.js:88:8

  - index.js:297 Jimp.parseBitmap
    [blog]/[jimp]/index.js:297:17

  - index.js:202 
    [blog]/[jimp]/index.js:202:29

  - read_file_context.js:53 FSReqWrap.readFileAfterClose [as oncomplete]
    internal/fs/read_file_context.js:53:3

[baobabKoodaa]

Additional information: today I saw this error on a Netlify build. After hitting "retry build" it worked. So this issue affects not only gatsby develop but also gatsby build.

[jonniebigodes]

@baobabKoodaa that's not a gatsby error per se. The problem is Potrace package throwing a "fit", i've been observing the behaviour while answering some issues that popped up around here ranging from people with markdown issues to simple image import issues. While i was coming up with a code reproduction for those people, I've seen that package throw that exact error with .png files, to .jpg files above 1 MB for instance. And if i'm not mistaken this issue is one that's been appearing on and off since version 1.x.x

[baobabKoodaa]

Wow, that's surprising. I thought it must be a concurrency issue with multiple Gatsby plugins doing things to images concurrently.

[VinSpee]

I just started experiencing it this week. This has been a killer of my productivity. I'd love any possible workaround, even temporary

[VinSpee]

^ My bad, sorry!

[pieh]

Can any of you share site you are seeing this problem with? I was able to kind of reproduce it (but I needed to disable memoization that we use, which should exactly prevent this problem).

[pieh]

Few notes for anyone who could be interested in tracking this down - this is changes I made in gatsby-plugin-sharp to disable memoization:

https://github.com/gatsbyjs/gatsby/commit/f235e9ebdd89225f6243150090f1900c449ef9ba

And this is custom code to excisively call not memoized traceSVG function - can be placed in gatsby-node.js

/**
 * Implement Gatsby's Node APIs in this file.
 *
 * See: https://www.gatsbyjs.org/docs/node-apis/
 */
const { traceSVG } = require(`gatsby-plugin-sharp`)

const fill = file => {
  return new Array(99).fill(0).map(async () => {
    return await traceSVG({
      file,
      args: {},
      fileArgs: {},
    })
  })
}

// You can delete this file if you're not using it
exports.onPreBootstrap = () => {
  let i = 1
  return new Promise(async () => {
    while (true) {
      const result = await Promise.all(
        ...fill({
          extension: "png",
          name: "a",
          absolutePath: `/Users/misiek/dev/gatsby-starter-default/src/images/gatsby-astronaut.png`,
          internal: {
            contentDigest: "5",
          },
        }),
        ...fill({
          extension: "png",
          name: "b",
          absolutePath: `/Users/misiek/dev/gatsby-starter-default/src/images/gatsby-icon.png`,
          internal: {
            contentDigest: "6",
          },
        }),
        ...fill({
          extension: "jpg",
          name: "c",
          absolutePath: `/Users/misiek/dev/gatsby-starter-default/src/images/download.jpg`,
          internal: {
            contentDigest: "7",
          },
        }),
        ...fill({
          extension: "jpeg",
          name: "d",
          absolutePath: `/Users/misiek/dev/gatsby-starter-default/src/images/download.jpeg`,
          internal: {
            contentDigest: "8",
          },
        })
      )

      console.log(`iteration complete #${i}`)
      i++
    }
  })
}

This crashes after some random time with exact same error.

Adding some logging to jimp, potrace and pngjs - this crashes here https://github.com/lukeapage/pngjs/blob/master/lib/parser.js#L53
and editing that to see signature and data with:

      console.log({
        dataF: [...data].slice(0, signature.length),
        signature,
      })
      this.error(new Error("Invalid file signature"))

produce (example) output:

{
  dataF: [ 255, 216, 255, 219, 0, 67, 0, 16 ],
  signature: [ 137, 80, 78, 71, 13, 10, 26, 10 ]
}

which does seem like problem with potentially gatsby writing to same temporary file multiple times concurrently

This should not happen unless memoization is failing, so samples of sites that are known to produce this problem would be extremely helpful as then we could verify memoization keys vs tmp file paths.

[baobabKoodaa]

Can any of you share site you are seeing this problem with? I was able to kind of reproduce it (but I needed to disable memoization that we use, which should exactly prevent this problem).

I can't reliably reproduce it, but this is the site that I'm working on which occasionally throws this error: https://github.com/baobabKoodaa/blog

[arnorhs]

just a theory, but could it be that there are conflicting job IDs in the reducer for each task in createJob?
https://github.com/gatsbyjs/gatsby/blob/master/packages/gatsby-plugin-sharp/src/scheduler.js#L91

you'd think that maybe the id should contain the inputFileKey instead of the inputPath.. just a random idea, since it's kind of peculiar that there'd be a different key chosen for the set in toProcess vs the job set that the CREATE_JOB reducer uses.

anyways, we're seeing the same issue pop up now and then but haven't been able to repro consistantly

[chenrico]

I had this issue in a project querying Contentful images.
I simple change GatsbyImageSharpFluid_tracedSVG to GatsbyImageSharpFluidand the issue seems to be solved.

[DSchau]

This looks very similar to #8301, so let's continue the discussion there.

Thanks everyone!

[pieh]

I opened https://github.com/gatsbyjs/gatsby/pull/12927 PR that hopefully fixes that. Because of random nature of this errors I can't be 100% sure on this

[maciekgrzybek]

Hey guys, not sure if that will help, but I'm having same issue, when I'm querying images from Contentful. I've managed to narrow it down to this situation:

1. Initial version of query when I was getting the error

query {
   ...
   cardImage {
        fluid: fluid(maxWidth: 800, maxHeight: 600) {
              base64
              tracedSVG
              aspectRatio
              src
              srcSet
              srcWebp
              srcSetWebp
              sizes
          }
     }
}

1. I've removed the 'fluid(maxWidth: 800, maxHeight: 600)' part, and it's started working.
2. Next day I got the same error again.
3. I've put back the 'fluid(maxWidth: 800, maxHeight: 600)' part, and it started working again.

Looks pretty random. If you need more info let me know. Hope that helps.

[svengau]

in my case, JIMP (used by Potrace) throws an error because it doens't support webp. And Potrace doesn't handle this error. I've created a bug report in Potrace project: https://github.com/tooolbox/node-potrace/issues/8

[dbvisel]

Still happening to me with latest versions; I got my builds to work by changing "GatsbyImageSharpFluid. tracedSVG" to "GatsbyImageSharpFluid". I think #8301 was closed prematurely?

[briangonzalezmia]

I can confirm this still happens with the following:

"gatsby": "^2.22.3",
"gatsby-image": "^2.4.5",
"gatsby-plugin-sharp": "^2.6.8",
"gatsby-transformer-sharp": "^2.5.3",

I was able to build using @dbvisel workaround. Was also using "GatsbyImageSharpFluid_tracedSVG" fragment.

[AlmogRnD]

I just got this issue when trying GatsbyImageSharpFluid_tracedSVG anyone solves this?

[stemry83]

Solution: remove all .webp and .svg (or any other new formats I guess) from your build content. I made copies from .webp and .svg into .png, and kept the original, then the error is gone and the build is finally successful. I believe the reason is this, as mentioned by @svengau :

JIMP (used by Potrace) throws an error because it doens't support webp. And Potrace doesn't handle this error.

If you still encountered the same error, try moving the original .webp and .svg to another location.