Gatsby: [gatsby-remark-images-contentful] intermittent build errors with/to base64

[floriangaechter]

Description

We're having an issue during the build phase of our project: sometimes the run graphql queries... step finishes without any issues and the site builds fine. But since recently the build process gets stuck at the ⡀ run graphql queries — 123/125 46.15 queries/second step and keeps either spinning forever or, if it finishes, takes a very long time (it doesn't always stop at 123. Sometimes it stops at 7/125 or other numbers).

Steps to reproduce

1. delete .cache folder
2. run gatsby build

Expected result

The build process should finish without issues.

Actual result

The build process gets stuck at the "run graphql queries..." phase.

Environment

  System:
    OS: macOS 10.14.3
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Shell: 5.3 - /bin/zsh
  Binaries:
    Node: 11.10.0 - ~/.nvm/versions/node/v11.10.0/bin/node
    npm: 6.8.0 - ~/.nvm/versions/node/v11.10.0/bin/npm
  Languages:
    Python: 2.7.15 - /usr/local/bin/python
  Browsers:
    Chrome: 72.0.3626.109
    Firefox: 65.0.1
    Safari: 12.0.3
  npmPackages:
    gatsby: ^2.1.4 => 2.1.4 
    gatsby-image: ^2.0.29 => 2.0.29 
    gatsby-plugin-manifest: ^2.0.17 => 2.0.17 
    gatsby-plugin-offline: ^2.0.23 => 2.0.23 
    gatsby-plugin-postcss: ^2.0.5 => 2.0.5 
    gatsby-plugin-react-helmet: ^3.0.6 => 3.0.6 
    gatsby-plugin-remove-trailing-slashes: ^2.0.7 => 2.0.7 
    gatsby-plugin-robots-txt: ^1.4.0 => 1.4.0 
    gatsby-plugin-sharp: ^2.0.20 => 2.0.20 
    gatsby-plugin-sitemap: ^2.0.5 => 2.0.5 
    gatsby-plugin-svgr: ^2.0.1 => 2.0.1 
    gatsby-remark-embed-video: ^1.7.0 => 1.7.0 
    gatsby-remark-images-contentful: ^2.0.8 => 2.0.8 
    gatsby-remark-responsive-iframe: ^2.0.9 => 2.0.9 
    gatsby-source-contentful: ^2.0.29 => 2.0.29 
    gatsby-source-filesystem: ^2.0.20 => 2.0.20 
    gatsby-source-hire-with-google: ^1.3.0 => 1.3.0 
    gatsby-transformer-remark: ^2.2.5 => 2.2.5 
    gatsby-transformer-sharp: ^2.1.13 => 2.1.13 
  npmGlobalPackages:
    gatsby-cli: 2.4.10

[wardpeet]

@floriangaechter it seems to me that this could be a case like https://github.com/gatsbyjs/gatsby/issues/11747. I'll be closing it in favour of that one please tell us as much as you can about this issue (maybe share your repo).

If it does not, feel free to re-open 😄

[floriangaechter]

@wardpeet thanks for your response. I’m not sure if those two are related as in our case, deleting the cache folder actually makes the issue worse instead of better. I’ll leave the issue closed for now and follow the other topic but might reopen it later if it turns out that the issues are not related.

[floriangaechter]

Turns out the issue is the _tracedSVG part of the images we're using. If we just query for "normal" images it works just fine.

[wardpeet]

lets re-open. Could you maybe try to create a small reproduction? ^^

[floriangaechter]

Sure, check out https://github.com/floriangaechter/tracedsvgtest

I hard-coded the api keys for contentful, since it seems to only happen when the images are stored there.

If you have a look at the query in https://github.com/floriangaechter/tracedsvgtest/blob/master/src/pages/index.js#L37 and remove the _tracedSVG part, the build process runs without any issues. But with the _tracedSVG part, it gets stuck during the run graphql queries step.

[floriangaechter]

Some new information – sometimes I'm getting the following error:

error UNHANDLED REJECTION


  Error: read ETIMEDOUT

  - stream_base_commons.js:162 TLSWrap.onStreamRead
    internal/stream_base_commons.js:162:27

Here's the stack trace:

{ Error: read ETIMEDOUT
      at TLSWrap.onStreamRead (internal/stream_base_commons.js:162:27)
    errno: 'ETIMEDOUT',
    code: 'ETIMEDOUT',
    syscall: 'read',
    config:
     { adapter: [Function: httpAdapter],
       transformRequest: [Object],
       transformResponse: [Object],
       timeout: 0,
       xsrfCookieName: 'XSRF-TOKEN',
       xsrfHeaderName: 'X-XSRF-TOKEN',
       maxContentLength: -1,
       validateStatus: [Function: validateStatus],
       headers: [Object],
       method: 'get',
       responseType: 'arraybuffer',
       url:
        'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
       data: undefined },
    request:
     Writable {
       _writableState: [WritableState],
       writable: true,
       _events: [Object],
       _eventsCount: 2,
       _maxListeners: undefined,
       _options: [Object],
       _redirectCount: 0,
       _redirects: [],
       _requestBodyLength: 0,
       _requestBodyBuffers: [],
       _onNativeResponse: [Function],
       _currentRequest: [ClientRequest],
       _currentUrl:
        'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40' },
    response: undefined } }

and:

{ Error: read ETIMEDOUT
    at TLSWrap.onStreamRead (internal/stream_base_commons.js:162:27)
  errno: 'ETIMEDOUT',
  code: 'ETIMEDOUT',
  syscall: 'read',
  config:
   { adapter: [Function: httpAdapter],
     transformRequest: { '0': [Function: transformRequest] },
     transformResponse: { '0': [Function: transformResponse] },
     timeout: 0,
     xsrfCookieName: 'XSRF-TOKEN',
     xsrfHeaderName: 'X-XSRF-TOKEN',
     maxContentLength: -1,
     validateStatus: [Function: validateStatus],
     headers:
      { Accept: 'application/json, text/plain, */*',
        'User-Agent': 'axios/0.18.0' },
     method: 'get',
     responseType: 'arraybuffer',
     url:
      'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
     data: undefined },
  request:
   Writable {
     _writableState:
      WritableState {
        objectMode: false,
        highWaterMark: 16384,
        finalCalled: false,
        needDrain: false,
        ending: false,
        ended: false,
        finished: false,
        destroyed: false,
        decodeStrings: true,
        defaultEncoding: 'utf8',
        length: 0,
        writing: false,
        corked: 0,
        sync: true,
        bufferProcessing: false,
        onwrite: [Function: bound onwrite],
        writecb: null,
        writelen: 0,
        bufferedRequest: null,
        lastBufferedRequest: null,
        pendingcb: 0,
        prefinished: false,
        errorEmitted: false,
        emitClose: true,
        autoDestroy: false,
        bufferedRequestCount: 0,
        corkedRequestsFree: [Object] },
     writable: true,
     _events:
      [Object: null prototype] {
        response: [Function: handleResponse],
        error: [Function: handleRequestError] },
     _eventsCount: 2,
     _maxListeners: undefined,
     _options:
      { protocol: 'https:',
        maxRedirects: 21,
        maxBodyLength: 10485760,
        path:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
        method: 'get',
        headers: [Object],
        agent: undefined,
        auth: undefined,
        hostname: 'images.ctfassets.net',
        port: null,
        nativeProtocols: [Object],
        pathname:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif',
        search: '?w=40' },
     _redirectCount: 0,
     _redirects: [],
     _requestBodyLength: 0,
     _requestBodyBuffers: [],
     _onNativeResponse: [Function],
     _currentRequest:
      ClientRequest {
        _events: [Object],
        _eventsCount: 6,
        _maxListeners: undefined,
        outputData: [],
        outputSize: 0,
        writable: true,
        _last: true,
        chunkedEncoding: false,
        shouldKeepAlive: false,
        useChunkedEncodingByDefault: false,
        sendDate: false,
        _removedConnection: false,
        _removedContLen: false,
        _removedTE: false,
        _contentLength: 0,
        _hasBody: true,
        _trailer: '',
        finished: true,
        _headerSent: true,
        socket: [TLSSocket],
        connection: [TLSSocket],
        _header:
         'GET /i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40 HTTP/1.1\r\nAccept: application/json, text/plain, */*\r\nUser-Agent: axios/0.18.0\r\nHost: images.ctfassets.net\r\nConnection: close\r\n\r\n',
        _onPendingData: [Function: noopPendingOutput],
        agent: [Agent],
        socketPath: undefined,
        timeout: undefined,
        method: 'GET',
        path:
         '/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40',
        _ended: false,
        res: null,
        aborted: false,
        timeoutCb: null,
        upgradeOrConnect: false,
        parser: null,
        maxHeadersCount: null,
        _redirectable: [Circular],
        [Symbol(isCorked)]: false,
        [Symbol(outHeadersKey)]: [Object] },
     _currentUrl:
      'https://images.ctfassets.net/i8q4h01orwwf/3LL7xtOMlcXjYKi2S5Xpto/573dc94143c45c10636562f753e76df0/beloved_features_fonts.gif?w=40' },
  response: undefined }

I think it has to do with the way the images are pulled from Contentful…

[ColeTownsend]

@floriangaechter We've been running into similar issues using Contentful with @robinpowered

[dirtyredz]

I am having a similar issue, which was confusing to me since my latest build on Netlify was working perfectly fine.
I tried multiple troubleshooting steps but one in particular worked. I dont remember why I thought to try disabling my antivirus but after I did everything worked again without issue.
Digging deeper it was my HIPS (Host Intrusion Protection System, Whenever this protection is activated I get this error:

> gatsby develop

error UNHANDLED EXCEPTION


  Error: read EINVAL

  - stream_base_commons.js:162 TTY.onStreamRead
    internal/stream_base_commons.js:162:27

But with HIPS disabled I am able to perform gatsby develep without any issues. I made sure to delete both the public and .cache dir after each test.

I dont know exactly what this means for Gatsby or why, but I hope this helps.

Edit:
It seems this was being blocked:

[claudiobmgrtnr]

We have the same problem. A fix would be appreciated.

[floriangaechter]

It really seems to be an issue with the images coming from Contentful. If I add the _noBase64 suffix, the build seems to succeed every time.

Maybe @Khaledgarbaya has an idea what the issue could be?

[Khaledgarbaya]

Hey Folks,
I think when the plugin tries to load the image and generate a base64 out of it times out could be this part

But I am not sure where exactly this is failing. Is it happening only with GIF images?

[floriangaechter]

Yes, I think it has something to do with the generation of either the base64 or tracedSVG variant of the image. Because if I remove both of them and tell Gatsby to generate neither base64 or tracedSVG the build succeeds each time. And it happens to all images, not just GIF files.

[coreyward]

We're running into this issue as well with the Figma marketing site when building on Ubuntu. We had a similar issue on some machines on macOS, but running yarn cache clean resolved the issue on those machines.

To resolve in production we've replaced all Gatsby Image gql queries w/ the _noBase64 variants.

Any chance someone on the @gatsbyjs/core team can take a look?

  System:
    OS: Linux 4.4 Ubuntu 16.04.5 LTS (Xenial Xerus)
    CPU: x64 Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
    Shell: 4.3.48 - /bin/bash
  Binaries:
    Node: 10.14.1 - /usr/local/bin/node
    Yarn: 1.15.2 - /usr/local/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm
  Languages:
    Python: 2.7.12 - /usr/bin/python
  npmPackages:
    gatsby: ^2.1 => 2.1.23 
    gatsby-image: ^2.0.19 => 2.0.19 
    gatsby-plugin-drift: ^1.0.0 => 1.0.0 
    gatsby-plugin-emotion: ^4.0.1 => 4.0.1 
    gatsby-plugin-google-tagmanager: ^2.0.0 => 2.0.5 
    gatsby-plugin-postcss: ^2.0.0 => 2.0.0 
    gatsby-plugin-react-css-modules: ^2.0.0 => 2.0.1 
    gatsby-plugin-react-helmet: ^3.0.0 => 3.0.0 
    gatsby-plugin-resolve-src: ^2.0.0-beta.1 => 2.0.0-beta.1 
    gatsby-plugin-segment-analytics: ^0.1.1 => 0.1.1 
    gatsby-plugin-sharp: ^2.0.20 => 2.0.20 
    gatsby-plugin-svgr: https://github.com/coreyward/gatsby-plugin-svgr#904c71904d10e1c9856cb6ee4346c1767563ccef => 1.0.0 
    gatsby-plugin-web-font-loader: ^1.0.4 => 1.0.4 
    gatsby-source-contentful: ^2.0.1 => 2.0.1 
    gatsby-source-filesystem: ^2.0.1 => 2.0.1 
    gatsby-source-lever: ^2.0.0 => 2.0.0 
    gatsby-transformer-json: ^2.1.1 => 2.1.2 
    gatsby-transformer-sharp: ^2.1.13 => 2.1.13 

[wardpeet]

@floriangaechter thanks for the repro i'll try to look at this today or else tomorrow.

[claudiobmgrtnr]

Hey @wardpeet
Thx for looking into the problem! 🙌
Do you know if there will be a fix for this in the near future?

Cheers

[ddsr17]

Any updated guys, its still occuring?

[floriangaechter]

@ddsr17 yes, as soon as we pull in data from Contentful the build task crashes, stalls, or, if we're very lucky, succeeds 😄 but to be serious, yes, we still have the issue.

Currently we're doing without any image preprocessing, just serving the images without traced SVGs or blurred versions of the image.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[benday280412]

Hi all,

I'm also seeing this behaviour, but it's originating from gatsby-remark-images-contentful.

The issue seems intermittent for me also, sometimes building fine, other times hanging on the queries stage.

I ran the build via Charles Proxy and can see the image requests going out. When the build hangs, I can see one or more images that have started transferring but don't complete, axios then hangs indefinitely. I've only seen this effect the original images, not the w=40 variants.

I tried adding a timeout to the axios call, but this doesn't have any effect. I believe this is due to the fact that the connection has started, rather than failed to connect at all.

With my very limited knowledge in this area, it does look like the code always expects the connection to be successful and timely. I wonder if we could bolster the connection handling to allow it to time out and then gracefully re-try the failed resources.

Interestingly when I requested the same resources via a bash script, concurrently requested, it succeeded every time. The transfers appeared to slow down towards the end, but no dropouts.

System:
    OS: macOS 10.14.5
    CPU: (8) x64 Intel(R) Core(TM) i7-4980HQ CPU @ 2.80GHz
    Shell: 3.2.57 - /bin/bash
  Binaries:
    Node: 10.15.0 - /usr/local/bin/node
    Yarn: 1.13.0 - /usr/local/bin/yarn
    npm: 6.4.1 - /usr/local/bin/npm
  Languages:
    Python: 2.7.15 - /usr/local/bin/python
  Browsers:
    Chrome: 74.0.3729.169
    Firefox: 66.0.5
    Safari: 12.1.1
  npmPackages:
    gatsby: ^2.8.5 => 2.8.5
    gatsby-image: ^2.1.2 => 2.1.2
    gatsby-link: ^2.1.1 => 2.1.1
    gatsby-plugin-catch-links: ^2.0.15 => 2.0.15
    gatsby-plugin-facebook-pixel: ^1.0.3 => 1.0.3
    gatsby-plugin-feed: ^2.2.2 => 2.2.2
    gatsby-plugin-google-analytics: ^2.0.20 => 2.0.20
    gatsby-plugin-google-tagmanager: ^2.0.15 => 2.0.15
    gatsby-plugin-manifest: ^2.1.1 => 2.1.1
    gatsby-plugin-nprogress: ^2.0.10 => 2.0.10
    gatsby-plugin-offline: ^2.1.1 => 2.1.1
    gatsby-plugin-preact: ^3.0.0 => 3.0.0
    gatsby-plugin-react-helmet: ^3.0.12 => 3.0.12
    gatsby-plugin-s3: ^0.2.5 => 0.2.5
    gatsby-plugin-sass: ^2.0.11 => 2.0.11
    gatsby-plugin-sharp: ^2.1.3 => 2.1.3
    gatsby-plugin-sitemap: ^2.1.0 => 2.1.0
    gatsby-plugin-twitter: ^2.0.13 => 2.0.13
    gatsby-plugin-web-font-loader: ^1.0.4 => 1.0.4
    gatsby-remark-component: ^1.1.3 => 1.1.3
    gatsby-remark-copy-linked-files: 2.0.12 => 2.0.12
    gatsby-remark-external-links: ^0.0.4 => 0.0.4
    gatsby-remark-images: ^2.0.6 => 2.0.6
    gatsby-remark-images-contentful: ^2.0.12 => 2.0.13
    gatsby-remark-responsive-iframe: ^2.1.1 => 2.1.1
    gatsby-remark-smartypants: ^2.0.9 => 2.0.9
    gatsby-source-contentful: ^2.0.67 => 2.0.67
    gatsby-source-filesystem: ^2.0.38 => 2.0.38
    gatsby-source-wordpress: ^3.0.64 => 3.0.64
    gatsby-transformer-json: ^2.1.11 => 2.1.11
    gatsby-transformer-remark: ^2.3.12 => 2.3.12
    gatsby-transformer-sharp: ^2.1.21 => 2.1.21

[itsmylife]

Hi all,

I am also facing this problem and this makes development process really tedious. I tried all possible solutions to make it faster but did not work any of them so far. I hope you guys would introduce something soon.

[azamatsmith]

I can confirm that this is still happening intermittently for me as well.

[thetrevorharmon]

I just started experiencing this issue and found this to be the root cause. No problems building without gatsby-remark-images-contentful, but with the plugin enabled, it hangs at building pages. Version:

gatsby-remark-images-contentful: 2.1.4,
gatsby: 2.13.28

Edit: I decided to update to version 2.1.24 on the images plugin, still experiencing the same issue.

[thetrevorharmon]

For those still facing this issue: I decided to update my version of gatsby-remark-images-contentful to 2.1.27 and it seems to be resolved (at least for now). I have noticed that it seems to get stuck at ~98% when running the page queries for a little longer than normal, but it doesn't timeout anymore.

[azamatsmith]

@thetrevorharmon, I ended up having to remove the plugin for now. gatsby-remark-images-contentful doesn't seem to ignore .gif files like gatsby-remark-images does which was causing a whole slew of errors for me. Once I removed the plugin, all of my build errors were resolved.