Gatsby: createRemoteFileNode crashes when url doesn't have an extension

Created on 2 Mar 2020 · 16Comments · Source: gatsbyjs/gatsby

Description

calling createRemoteFileNode with an url (https://fakeimg.pl/300x300/?text=2) without an extension crashes.

Unexpected error value: "failed to process https://fakeimg.pl/300x300/?text=2\nError: ENOENT: no such file or directory, stat '.../.cache/gatsby-source-filesystem/7a1221ea4c61551f928908dd873a3738/300x300'"

I think it has something to do with how the caching is handled.

Steps to reproduce

use createRemoteFileNode to create remote file nodes from an url without an ext
use the remote file node in eg. sharp-image
the first build goes through
all builds afterwards crash

Environment

  System:
    OS: Linux 5.3 Ubuntu 18.04.3 LTS (Bionic Beaver)
    CPU: (8) x64 AMD Ryzen 5 2500U with Radeon Vega Mobile Gfx
    Shell: 4.4.20 - /bin/bash
  Binaries:
    Node: 13.9.0 - /usr/bin/node
    Yarn: 1.22.0 - ~/bin/yarn
    npm: 6.13.7 - /usr/bin/npm
  Languages:
    Python: 2.7.15+ - /usr/bin/python
  Browsers:
    Firefox: 73.0.1
  npmPackages:
    gatsby: ^2.19.21 => 2.19.24
    gatsby-cli: ^2.9.0 => 2.9.0
    gatsby-image: ^2.2.41 => 2.2.42
    gatsby-plugin-graphql-codegen: ^2.2.2 => 2.2.3
    gatsby-plugin-react-helmet: ^3.1.21 => 3.1.22
    gatsby-plugin-react-svg: ^3.0.0 => 3.0.0
    gatsby-plugin-sass: ^2.1.27 => 2.1.29
    gatsby-plugin-sharp: ^2.4.5 => 2.4.5
    gatsby-source-filesystem: ^2.1.48 => 2.1.48
    gatsby-transformer-json: ^2.2.26 => 2.2.26
    gatsby-transformer-sharp: ^2.3.14 => 2.3.16

Repro

https://github.com/chpio/gatsby-remote-wo-ext commit https://github.com/chpio/gatsby-remote-wo-ext/commit/8d898f6915221495e092798e5bfcbacdd3faaf30

help wanted filesystem bug

Source

chpio

👍1

Most helpful comment

@vladar Seems like FileType.fromFile() should do the job?

ynnoj on 16 Oct 2020

🎉1 👍1

All 16 comments

See this documentation about using createRemoteFileNode without extension

freiksenet on 2 Mar 2020

Ok, what are you saying?

If the url does not contain an extension, we use the file-type package to infer the file type.

That works for the 1. build, but not for following builds. The cache file gets created with an extension, but the following builds try to read the file from cache without an extension and crash due to a read on a non-existing file.

chpio on 2 Mar 2020

👍2

@chpio You should pass extension of file explicitly, does it work?

  ext: ".jpg",
  name: "image"

freiksenet on 3 Mar 2020

Why should i do that? The doc even says, that it can be passed in:

Finally, the name and the extension can be explicitly passed
to pass it in i need to know it, but i do not. My first workaround was to just pass in ".jpeg", but then i needed the actual extension (to generate the download argument on an <a> tag)
a crash is clearly a bug

chpio on 3 Mar 2020

👍2

Can you create a reproducie test case?

muescha on 11 Mar 2020

@muescha https://github.com/chpio/gatsby-remote-wo-ext

chpio on 11 Mar 2020

Following... I'm having this issue using the same technique in the provided test case.

darcy on 11 Mar 2020

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

github-actions[bot] on 1 Apr 2020

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.
Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community! 💪💜

github-actions[bot] on 11 Apr 2020

We're having the same issue.

Gatsby CLI version: 2.7.47
Gatsby version: 2.20.22

THE108 on 16 Apr 2020

When a new user logs in to my Gatsby website with Google, I get their avatar from their Google account. Google's URLs for avatars look like this:
https://lh3.googleusercontent.com/a-/AAuE7mBhk7bsX8uoSh9mRLwM2An0u1tyPcu_bv1TQI6Y.

In gatsby-node.js, I use createRemoteFileNode with this url.

The first build, as @chpio mentioned, finishes successfully.

If I don't clear Gatsby cache, upon consecutive builds I get:
ERROR Fetching Author Avatar: failed to process https://lh3.googleusercontent.com/a-/AAuE7mBhk7bsX8uoSh9mRLwM2An0u1tyPcu_bv1TQI6Y Error: ENOENT: no such file or directory, stat '/usr/src/app/www/.cache/caches/default-site-plugin/766d6ed63d9b327f1affaea984564e91/AAuE7mBhk7bsX8uoSh9mRLwM2An0u1tyPcu_bv1TQI6Y'

dimileeh on 25 Apr 2020

The issue is probably somewhere along those lines:

https://github.com/gatsbyjs/gatsby/blob/03803999bd35018a599a8644f16424b75e6c89a3/packages/gatsby-source-filesystem/src/create-remote-file-node.js#L249-L250

fileTypes.minimumBytes were removed here: https://github.com/sindresorhus/file-type/pull/319

So we probably need to update gatsby-source-filesystem to accommodate this change. PR is welcome!

vladar on 16 Oct 2020