Gatsby: [gatsby-plugin-mdx] cannot find module

[muuvmuuv]

Description

I'm trying to add a custom plugin to gatsby-plugin-mdx. I've made sure:

• It is located in /plugins/gatsby-remark-normalize-url
• It has a package.json file and index.js
• It is added into gatsby-config.js

{
  "name": "gatsby-remark-normalize-url",
  "description": "Normalize URL's with sindresorhus popular normalize-url lib",
  "version": "1.0.0",
  "dependencies": {
    "normalize-url": "^5.0.0",
    "unist-util-visit": "^2.0.2"
  }
}

const gatsbyPluginMdx = {
  resolve: `gatsby-plugin-mdx`,
  options: {
    gatsbyRemarkPlugins: [
      `gatsby-remark-normalize-url`,
    ],
  }
}

Steps to reproduce

My project: https://github.com/muuvmuuv/portfolio (changes not included but can be added)

1. add a plugin described above into plugins (it shouldn't do anything)
2. run gatsby develop

Expected result

What should happen?

Actual result

ERROR #11321  PLUGIN

"gatsby-plugin-mdx" threw an error while running the onCreateNode lifecycle:

Cannot find module 'gatsby-remark-normalize-url'
Require stack:
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-plugin-mdx/1.0.75_48af87626be11f1a2f9cb7c62b6c17dc/node_modules/gatsby-plugin-mdx/utils/get-source-plugins-as-remark-plugins.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-plugin-mdx/1.0.75_48af87626be11f1a2f9cb7c62b6c17dc/node_modules/gatsby-plugin-mdx/utils/gen-mdx.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-plugin-mdx/1.0.75_48af87626be11f1a2f9cb7c62b6c17dc/node_modules/gatsby-plugin-mdx/gatsby/source-nodes.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-plugin-mdx/1.0.75_48af87626be11f1a2f9cb7c62b6c17dc/node_modules/gatsby-plugin-mdx/gatsby-node.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bootstrap/resolve-module-exports.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bootstrap/load-plugins/validate.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bootstrap/load-plugins/load.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bootstrap/load-plugins/index.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bootstrap/index.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/commands/develop.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-cli/2.10.0/node_modules/gatsby-cli/lib/create-cli.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby-cli/2.10.0/node_modules/gatsby-cli/lib/index.js
- /Users/marvinheilemann/Development/Portfolio/node_modules/.pnpm/registry.npmjs.org/gatsby/2.19.28_1b55037e18e5731ef1b673b5516aa9fb/node_modules/gatsby/dist/bin/gatsby.js

Environment

  System:
    OS: macOS 10.15.3
    CPU: (8) x64 Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
    Shell: 5.7.1 - /bin/zsh
  Binaries:
    Node: 12.16.1 - ~/.nvm/versions/node/v12.16.1/bin/node
    Yarn: 1.22.0 - /usr/local/bin/yarn
    npm: 6.13.4 - ~/.nvm/versions/node/v12.16.1/bin/npm
  Languages:
    Python: 2.7.16 - /usr/bin/python
  Browsers:
    Firefox: 73.0.1
    Safari: 13.0.5
  npmPackages:
    gatsby: ^2.19.28 => 2.19.28 
    gatsby-image: ^2.2.42 => 2.2.42 
    gatsby-plugin-breadcrumb: ^9.0.1 => 9.0.1 
    gatsby-plugin-canonical-urls: ^2.1.20 => 2.1.20 
    gatsby-plugin-catch-links: ^2.1.26 => 2.1.26 
    gatsby-plugin-humans-txt: ^1.1.4 => 1.1.4 
    gatsby-plugin-layout: ^1.1.22 => 1.1.22 
    gatsby-plugin-manifest: ^2.2.42 => 2.2.42 
    gatsby-plugin-mdx: ^1.0.75 => 1.0.75 
    gatsby-plugin-offline: ^3.0.35 => 3.0.35 
    gatsby-plugin-pnpm: ^1.2.3 => 1.2.3 
    gatsby-plugin-postcss: ^2.1.20 => 2.1.20 
    gatsby-plugin-preact: ^3.1.26 => 3.1.26 
    gatsby-plugin-react-axe: ^0.4.0 => 0.4.0 
    gatsby-plugin-react-helmet-async: ^1.0.15 => 1.0.15 
    gatsby-plugin-react-svg: ^3.0.0 => 3.0.0 
    gatsby-plugin-remove-generator: ^1.0.4 => 1.0.4 
    gatsby-plugin-remove-trailing-slashes: ^2.1.21 => 2.1.21 
    gatsby-plugin-robots-txt: ^1.5.0 => 1.5.0 
    gatsby-plugin-sass: ^2.1.29 => 2.1.29 
    gatsby-plugin-sharp: ^2.4.5 => 2.4.5 
    gatsby-plugin-simple-analytics: ^1.0.3 => 1.0.3 
    gatsby-plugin-sitemap: ^2.2.27 => 2.2.27 
    gatsby-plugin-webpack-bundle-analyzer: ^1.0.5 => 1.0.5 
    gatsby-remark-autolink-headers: ^2.1.24 => 2.1.24 
    gatsby-remark-check-links: ^2.1.0 => 2.1.0 
    gatsby-remark-copy-linked-files: ^2.1.37 => 2.1.37 
    gatsby-remark-embedder: ^1.12.0 => 1.12.0 
    gatsby-remark-images: ^3.1.44 => 3.1.44 
    gatsby-source-filesystem: ^2.1.48 => 2.1.48 
    gatsby-source-graphql: 2.1.33 => 2.1.33 
    gatsby-transformer-json: ^2.2.26 => 2.2.26 
    gatsby-transformer-leasot: ^1.2.2 => 1.2.2 
    gatsby-transformer-sharp: ^2.3.16 => 2.3.16 

[muuvmuuv]

This however works:

const gatsbyRemarkPlugins = [
   {
      resolve: require.resolve('./plugins/gatsby-remark-normalize-url'),
   }
]

[Js-Brecht]

Check out how gatsby-plugin-mdx resolves those plugins here. It doesn't use Gatsby's method for resolving plugins... it just does a require().

It probably wouldn't be hard to make it to use Gatsby's plugin resolver.

[ksjogo]

I assume it is pnpm related which I am also running into. Would be great to have the standard gatsby resolve doing the work. Though I have some slightly different issue.
I have some base package sharing some code and dependencies, but at the moment I have to add

"@mdx-js/mdx": "~1.5.8",
"@mdx-js/react": "~1.5.8",
"gatsby-plugin-mdx": "^1.1.9",

to every child package to make the mdx rendering work, even though these packages should be taken from the baselib.

[Js-Brecht]

I believe this issue is about resolving plugins that are in the local plugins directory.

I think my last comment was a little vague:
gatsby-plugin-mdx uses require() to get the plugin. That means it is using node's module resolution, and node's module resolution doesn't look in the plugins folder. That's why providing the full path to the plugin works, but using just the name of the plugin doesn't.

---

I'm not sure what the issue you're facing is, @ksjogo. It sounds like you're having more trouble with dependencies as opposed to using the local plugins folder. Can you elaborate a little more?

[ksjogo]

Sure.
My repo is structured like this:

root/
->packages/
-->@ksjogo/
--->gatsby (depends on gatsby, gatsby-plugin-mdx, etc.) 
->sites/
-->somesite (depends on @ksjogo/gatsby)

This works fine for having all plugins etc just in @ksjogo/gatsby.
But wanting to render mdx posts/pages want work, as

"@mdx-js/mdx": "~1.5.8",
"@mdx-js/react": "~1.5.8",
"gatsby-plugin-mdx": "^1.1.9",

are not present top-level within sites/somesite/node_modules as everything is installed with pnpm.
Gatsby afaict is basically tree deep resolving everything with gatsby starting in the local node_modules, then going to @ksjogo/gatsby and then all plugins/other things there.
The mdx resolver doesn't seem to use this functionality.
The workaround of linking the above deps isn't too bad, but it would be nicer if it wasn't needed. :D

[Js-Brecht]

It sounds like you’re having trouble using MDXRender, or something similar, during the rendering stage, which is not really related to this issue.

I think the issue you’re having is the “phantom dependencies” issue, or you could say you’re trying to use “sibling dependencies.” That’s not supported by any package manager, though npm and yarn can trick you into thinking it’s supported.

Try using gatsby-plugin-pnpm in @ksjogo/gatsby. Use the include option, and include the names of those dependencies. If that doesn’t work, you could open an issue in the gatsby-plugin-pnpm repo and I’ll help you out there, or you could maybe open a new issue here.

[ksjogo]

Yeah, it is definitely a children dependency thing.
The weird thing is that gatsby is kinda doing a lot of resolving these for plugins. I added some more investigation results at #23265

[johno]

Thanks for opening up the issue and the investigation! We have an open issue that we're going to track adding the API needed in order to achieve this feature (#21592), so I'm going to close this and we can move any needed conversation over there.