Gatsby: ENAMETOOLONG: name too long for long headers / slugs

[lightstrike]

Description

I'm experimenting with Gatsby & Wordpress (really cool stuff), when running gatsby build I hit this error:

error Generating JavaScript bundles failed

Error: ENAMETOOLONG: name too long, open '/home/user/applications/wpgatsby/public/path---crazy-insane-long-slug-why-on-earth-did-anyone-ever-choose-to-create-a-header-this-long-its-completely-ridiculous-please-make-it-stop-48ae67005d81988049b2.js'

What drives the file naming here?

Environment

Gatsby version: 1.1.40
Node.js version: 8.9.4
Operating System: Linux Mint 18.3

File contents (if changed):

gatsby-config.js:

module.exports = {
  siteMetadata: {
    title: 'Gatsby Default Starter',
  },
  plugins: [
    'gatsby-plugin-react-helmet',
    {
      resolve: `gatsby-source-wordpress`,
      options: {
        /*
        * The base URL of the Wordpress site without the trailingslash and the protocol. This is required.
        * Example : 'gatsbyjswpexample.wordpress.com' or 'www.example-site.com'
        */
        baseUrl: `sitewithlongnames.com`,
        // The protocol. This can be http or https.
        protocol: `http`,
        // Indicates whether the site is hosted on wordpress.com.
        // If false, then the asumption is made that the site is self hosted.
        // If true, then the plugin will source its content on wordpress.com using the JSON REST API V2.
        // If your site is hosted on wordpress.org, then set this to false.
        hostingWPCOM: false,
        // If useACF is true, then the source plugin will try to import the Wordpress ACF Plugin contents.
        // This feature is untested for sites hosted on Wordpress.com
        useACF: false,
        verboseOutput: true,
      },
    }, 
    'gatsby-plugin-netlify', // make sure to put last in the array
  ],
};

package.json:

...
"dependencies": {
    "gatsby": "^1.9.188",
    "gatsby-image": "^1.0.35",
    "gatsby-link": "^1.6.34",
    "gatsby-plugin-netlify": "^1.0.18",
    "gatsby-plugin-react-helmet": "^2.0.3",
    "gatsby-source-wordpress": "^2.0.54",
    "react-helmet": "^5.2.0"
  },
...

gatsby-node.js:

const _ = require(`lodash`)
const path = require(`path`)
const Promise = require(`bluebird`)
const slash = require(`slash`)

// Implement the Gatsby API “createPages”. This is
// called after the Gatsby bootstrap is finished so you have
// access to any information necessary to programmatically
// create pages.
// Will create pages for Wordpress pages (route : /{slug})
// Will create pages for Wordpress posts (route : /post/{slug})
exports.createPages = ({ graphql, boundActionCreators }) => {
  const { createPage } = boundActionCreators
  return new Promise((resolve, reject) => {
    // The “graphql” function allows us to run arbitrary
    // queries against the local Wordpress graphql schema. Think of
    // it like the site has a built-in database constructed
    // from the fetched data that you can run queries against.

    // ==== PAGES (WORDPRESS NATIVE) ====
    graphql(
      `
        {
          allWordpressPage {
            edges {
              node {
                id
                slug
                status
                template
              }
            }
          }
        }
      `
    )
      .then(result => {
        if (result.errors) {
          console.log(result.errors)
          reject(result.errors)
        }
        // Create Page pages.
        const pageTemplate = path.resolve(`./src/templates/page.js`)
        // We want to create a detailed page for each
        // page node. We'll just use the Wordpress Slug for the slug.
        // The Page ID is prefixed with 'PAGE_'
        _.each(result.data.allWordpressPage.edges, edge => {
          // Gatsby uses Redux to manage its internal state.
          // Plugins and sites can use functions like "createPage"
          // to interact with Gatsby.
          createPage({
            // Each page is required to have a `path` as well
            // as a template component. The `context` is
            // optional but is often necessary so the template
            // can query data specific to each page.
            path: `/${edge.node.slug}/`,
            component: slash(pageTemplate),
            context: {
              id: edge.node.id,
            },
          })
        })
      })
      // ==== END PAGES ====

      // ==== POSTS (WORDPRESS NATIVE AND ACF) ====
      .then(() => {
        graphql(
          `
            {
              allWordpressPost {
                edges {
                  node {
                    id
                    slug
                    status
                    template
                    format
                  }
                }
              }
            }
          `
        ).then(result => {
          if (result.errors) {
            console.log(result.errors)
            reject(result.errors)
          }
          const postTemplate = path.resolve(`./src/templates/post.js`)
          // We want to create a detailed page for each
          // post node. We'll just use the Wordpress Slug for the slug.
          // The Post ID is prefixed with 'POST_'
          _.each(result.data.allWordpressPost.edges, edge => {
            createPage({
              path: edge.node.slug,
              component: slash(postTemplate),
              context: {
                id: edge.node.id,
              },
            })
          })
          resolve()
        })
      })
    // ==== END POSTS ====
  })
}

[m-allanson]

I think those paths will be generated based on the content of your WordPress install. Could you try the using-wordpress example and see if you get the same error?

If you can get a successful gatsby build, have a look in the ./public folder and you'll see all the path----slug-slug-slug.js files that are generated.

[m-allanson]

Ergh, I just tried the WordPress example site and it gives an error The server response was "530 Site is frozen" No routes to fetch. Ending.

[m-allanson]

@sebastienfi or @KyleAMathews are you able to unfreeze the WordPress example site at https://dashboard.pantheon.io?

[KyleAMathews]

Doesn't look I do have access to him.

[lightstrike]

Thanks for taking a look @m-allanson. It seems like the header title (as opposed to the slug) is driving the file names - when I changed the post title, this error was not raised. Where does the file naming occur in the gatsby source? While I can control header lengths in my dev environment, I don't believe I can do so in a production environment, so adding some logic that trims the slugified header to the max allowed would be helpful to prevent broken builds.

After resolving that issue, I am hitting the following stack size limit error as described in #3341. Changing my build command to node --max_old_space_size=20000 ./node_modules/.bin/gatsby build --prefix-paths, then running npm run build, this still happens:

=END PLUGIN=====================================: 191370.411ms
success source and transform nodes — 719.879 s
success building schema — 31.160 s
success createLayouts — 0.035 s
success createPages — 23.849 s
success createPagesStatefully — 0.104 s
success onPreExtractQueries — 0.000 s
success update schema — 17.400 s
success extract queries from components — 0.565 s
success run graphql queries — 75.960 s
success write out page data — 0.093 s
success write out redirect data — 0.000 s
success onPostBootstrap — 0.000 s

info bootstrap finished - 870.841 s

success Building CSS — 6.955 s
⠐ Building production JavaScript bundles

error Generating JavaScript bundles failed


  Error: app-55a9936c7d0185aab11b.js from UglifyJs
  RangeError: Maximum call stack size exceeded
      at RegExp.test (<anonymous>)
      at is_identifier_string (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:1527:36)
      at eval (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:8133:73)
      at AST_Sub.eval [as optimize] (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:5387:23)
      at Compressor.before (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:5370:21)
      at AST_Sub.eval [as transform] (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2985:35)
      at eval (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:3120:31)
      at Compressor.before (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:5369:9)
      at AST_Assign.eval [as transform] (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2985:35)
      at eval (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:3102:29)
      at Compressor.before (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:5369:9)
      at AST_Seq.eval [as transform] (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2985:35)
      at eval (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:3103:29)
      at Compressor.before (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:5369:9)
      at AST_Seq.eval [as transform] (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:2985:35)
      at eval (eval at <anonymous> (/home/.../wpgatsby/node_modules/webpack/node_modules/uglify-js/tools/node.js:28:1), <anonymous>:3103:29)

npm ERR! code ELIFECYCLE
npm ERR! errno 1
npm ERR! [email protected] build: `node --max_old_space_size=20000 ./node_modules/.bin/gatsby build --prefix-paths`
npm ERR! Exit status 1
npm ERR! 
npm ERR! Failed at the [email protected] build script.
npm ERR! This is probably not a problem with npm. There is likely additional logging output above.

npm ERR! A complete log of this run can be found in:
npm ERR!     /home/.../.npm/_logs/2018-02-20T16_57_02_408Z-debug.log

The log file:

0 info it worked if it ends with ok
1 verbose cli [ '/home/.../.nvm/versions/node/v8.9.4/bin/node',
1 verbose cli   '/home/.../.nvm/versions/node/v8.9.4/bin/npm',
1 verbose cli   'run',
1 verbose cli   'build' ]
2 info using [email protected]
3 info using [email protected]
4 verbose run-script [ 'prebuild', 'build', 'postbuild' ]
5 info lifecycle [email protected]~prebuild: [email protected]
6 info lifecycle [email protected]~build: [email protected]
7 verbose lifecycle [email protected]~build: unsafe-perm in lifecycle true
8 verbose lifecycle [email protected]~build: PATH: /home/.../.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/npm-lifecycle/node-gyp-bin:/home/.../wpgatsby/node_modules/.bin:/home/.../.nvm/versions/node/v8.9.4/bin:/home/.../.pyenv/plugins/pyenv-virtualenvwrapper/shims:/home/.../.pyenv/libexec:/home/.../.pyenv/plugins/python-build/bin:/home/.../.pyenv/plugins/pyenv-virtualenvwrapper/bin:/home/.../.pyenv/shims:/home/.../.pyenv/bin:/home/.../bin:/home/.../.local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
9 verbose lifecycle [email protected]~build: CWD: /home/.../wpgatsby
10 silly lifecycle [email protected]~build: Args: [ '-c',
10 silly lifecycle   'node --max_old_space_size=20000 ./node_modules/.bin/gatsby build --prefix-paths' ]
11 silly lifecycle [email protected]~build: Returned: code: 1  signal: null
12 info lifecycle [email protected]~build: Failed to exec build script
13 verbose stack Error: [email protected] build: `node --max_old_space_size=20000 ./node_modules/.bin/gatsby build --prefix-paths`
13 verbose stack Exit status 1
13 verbose stack     at EventEmitter.<anonymous> (/home/.../.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/npm-lifecycle/index.js:285:16)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at EventEmitter.emit (events.js:214:7)
13 verbose stack     at ChildProcess.<anonymous> (/home/.../.nvm/versions/node/v8.9.4/lib/node_modules/npm/node_modules/npm-lifecycle/lib/spawn.js:55:14)
13 verbose stack     at emitTwo (events.js:126:13)
13 verbose stack     at ChildProcess.emit (events.js:214:7)
13 verbose stack     at maybeClose (internal/child_process.js:925:16)
13 verbose stack     at Process.ChildProcess._handle.onexit (internal/child_process.js:209:5)
14 verbose pkgid [email protected]
15 verbose cwd /home/.../wpgatsby
16 verbose Linux 4.10.0-38-generic
17 verbose argv "/home/.../.nvm/versions/node/v8.9.4/bin/node" "/home/.../.nvm/versions/node/v8.9.4/bin/npm" "run" "build"
18 verbose node v8.9.4
19 verbose npm  v5.6.0
20 error code ELIFECYCLE
21 error errno 1
22 error [email protected] build: `node --max_old_space_size=20000 ./node_modules/.bin/gatsby build --prefix-paths`
22 error Exit status 1
23 error Failed at the [email protected] build script.
23 error This is probably not a problem with npm. There is likely additional logging output above.
24 verbose exit [ 1, true ]

Any recommendations here? The sites I'm looking at porting have 10,000 - 30,000 items between posts, pages, media and comments.

[lightstrike]

When I updated to "node --max_old_space_size=1200000 --stack-size=65000 ./node_modules/.bin/gatsby build --prefix-paths", I got a passing build! :rocket: But these parameters were a total stab in the dark...is there any general guidance you recommend @KyleAMathews regarding larger site builds? Happy to dig in and try to write some docs if it would be helpful.

[KyleAMathews]

--max_old_space_size is generally what I've had to fiddle with for large builds in the past. But --stack-size might be needed too. A docs page with some research and suggestions on this would be great!

[scissorsneedfoodtoo]

Hi all, I wasn't sure if I should open a new issue or not, but I'm having a similar problem with the slug length being too long.

Recently I was trying to clone and build the new freeCodeCamp learn project and ran into an error when running yarn develop, which in turn runs yarn build:frame-runner && gatsby develop:

ERROR  Failed to compile with 3 errors                               16:22:30

These dependencies were not found:

* /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-object-oriented-programming-use-closure-to-protect-properties-within-an-object-from-being-modified-externally.json in ./.cache/sync-requires.js
* /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-functional-programming-use-the-every-method-to-check-that-every-element-in-an-array-meets-a-criteria.json in ./.cache/sync-requires.js
* /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-functional-programming-use-the-some-method-to-check-that-any-elements-in-an-array-meet-a-criteria.json in ./.cache/sync-requires.js

To install them, you can run: npm install --save /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-object-oriented-programming-use-closure-to-protect-properties-within-an-object-from-being-modified-externally.json /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-functional-programming-use-the-every-method-to-check-that-every-element-in-an-array-meets-a-criteria.json /home/kris/Documents/GitHub/forks/learn/.cache/json/javascript-algorithms-and-data-structures-functional-programming-use-the-some-method-to-check-that-any-elements-in-an-array-meet-a-criteria.json

When I try to run npm install --save ... from the error message, I get an error from npm saying that the filename is too long. I believe it's because Linux Mint 18.3 home directory is encrypted, so filenames can only be ~143 characters.

Is there an easy way to work around this limitation? Maybe change the way that the pages are saved in .cache/json to create a file structure that mimics that of the source data? For example, saving a page as javascript-algorithms-and-data-structures/object-oriented-programming/verify-an-objects-constructor-with-instanceof.json rather than javascript-algorithms-and-data-structures-object-oriented-programming-verify-an-objects-constructor-with-instanceof.json.

[KyleAMathews]

I think the only full-proof solution the long filenames is to just ensure that filenames never go beyond (some arbitrary number that all file systems support).

A possible algorithm could be:

1) Naively create the original filename
2) Create a hash the original file name
3) remove common words e.g. and, an, or, etc. I'm sure there's an NPM package for this,
4) Append the hash to the shortened file name
5) if the file name is too long still, split the name at dashes and remove words until the name + hash is < MAX_FILE_NAME_LENGTH

[KyleAMathews]

Anyone interested in implementing this or an equivalent algorithm? We'd need to do some research into what's a safe max length. Also this should be in a util module as there's a few places we generate filenames.

[abumalick]

Hello,
Is there a quick fix for this ?

I am refactoring our app to use gatsby and we have very long paths.

Can you give me indications to where I should start to hack with your indications @KyleAMathews ?

[KyleAMathews]

Here's one place. Probably a few more https://github.com/gatsbyjs/gatsby/blob/4ec878d7be89321f4f61035e3282eb1c96dd2b90/packages/gatsby/src/redux/actions.js#L232

[KyleAMathews]

Due to the high volume of issues, we're closing out older ones without recent activity. Please open a new issue if you need help!

[jshearer]

Anyone interested in implementing this or an equivalent algorithm? We'd need to do some research into what's a safe max length. Also this should be in a util module as there's a few places we generate filenames.

Hi there! I'm relatively new to Gatsby but I just encountered this same issue. My understanding of @lightstrike's comment is that he just changed his titles/slugs to be short enough to work, but that unfortunately won't work for me. I'm building a Gatsby project that needs to support arbitrarily deep paths, and am encountering issues like the following when creating pages with path lengths over about ~250:

success extract queries from components — 0.158 s
success run static queries — 0.057 s — 4/4 72.10 queries/second
error UNHANDLED REJECTION


  Error: ENAMETOOLONG: name too long, open '/Users/js/project/public/static/d/382/path---dita-spec-architectural-specification-base-architecture-configuration-specialization-and-constraints  -specialization-specialization-module-coding-requirements-xsd-schema-specialization-module-coding-requirements-attribute-domain-coding-requirements-b-88-575-W7qcN9ELUNRgMyb3BGcgBQfJPfY.json'

I did some digging, and it turns out that this filename is generated by this line:

https://github.com/gatsbyjs/gatsby/blob/d4b60f2b900a59b70dd2d97cf14125c77c1de4df/packages/gatsby/src/redux/actions.js#L277

Like you said, @KyleAMathews

The length restriction of approx. 250 characters makes sense, since at least on OSX, it looks like filenames have a max length of 255 characters.

I was able to (very) temporarily solve this problem by patching the generated code for the line mentioned above to the following:

// At the top of the file
const createHash = require("crypto").createHash;

// At line 258
jsonName = `${createHash("md5").update(page.path).digest(`hex`)}`;

Would you (@KyleAMathews) offer some guidance as to the best way to make a PR for this? Should I create and publish a separate package that just generates this md5 hash and then make a PR in Gatsby that imports/uses it? That _seems_ a bit excessive to me, but what do you say?

---

Also: Is it a requirement that these generated JSON files have vaguely human-readable names? Because if so, it's possible that the logic for generating the name could be something like:

jsonName = `${page.path.slice(-60)-${createHash("md5").update(page.path).digest(`hex`)}`;

So that we would at least have the last ~60 characters of the path. I imagine the rightmost characters are more valuable than the leftmost, but if you think some of the path root is important too, it could be a bit more complex and take at most ~50 characters from the beginning and at most ~50 characters from the end. Thoughts?

[KyleAMathews]

Someone else ran into this, we should fix this.

[KyleAMathews]

@jshearer thanks for looking into this! Did you see my algorithm? https://github.com/gatsbyjs/gatsby/issues/4125#issuecomment-384741839

It'd be best to try to not change the name as much as possible. We could try each step and see if the name is short enough and as the last resort, hash the original name like you've done. The name doesn't matter per say but it's helpful for debugging what's going on to know what pages the file name is for.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contributefor more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[leonfs]

We've been experiencing this problem too.