Gatsby: [gatsby-plugin-mdx] `html` field doesn't render custom components when those components are provided in a layout file via`MDXProvider shortcodes`

[jjdevenny]

Description

I have created some custom components and I use them in my blogpost markdown files. These custom components render fine in the browser when they're supplied to MDXProvider via shortcodes in my layout file. Unfortunately, these custom components are not rendered properly in the html field that is used by gatsby-plugin-feed when generating the RSS feed.

I've done some digging, and it seems that the custom component _will_ be rendered correctly if the component has been imported directly into the .mdx file, but the component _will not_ be rendered correctly if it has been supplied via shortcodes to MDXProvider.

Here's an example of one component, Caption:

import React from "react"

const Caption = ({children}) => {

    return(
        <div style={{
            fontSize:"14px",
            textAlign:"center",
            color:"grey",
            marginTop:"-1rem",
            marginBottom:"2rem"}}>
            {children}
        </div>
    )
}
export default Caption

It gets supplied to the blogPost.js layout file via a shortcode given to MDXProvider:

import React from 'react'
import { graphql } from 'gatsby'
import Caption from '../components/Caption'
import { MDXProvider } from "@mdx-js/react"
import { MDXRenderer } from "gatsby-plugin-mdx"

const shortcodes = { Caption }

export default function PageTemplate( { data, pageContext } ) {

    const { mdx } = data
    const title = mdx.frontmatter.title
    const timeToRead = mdx.timeToRead
    const relativeDate = mdx.frontmatter.date

    return (

        <div>
            <div id='middle-column'>
                <div>
                    <h1 style={{fontFamily:"Georgia", marginBottom:"8px"}}>{title}</h1>
                </div>
                <div className='blogpost'>
                    <MDXProvider components={shortcodes}>
                        <MDXRenderer>{mdx.body}</MDXRenderer>
                    </MDXProvider>
                </div>
            </div>        

        </div> 
    )
}

export const pageQuery = graphql`
    query($pathSlug:String!){
        mdx(fields: { slug: { eq: $pathSlug} }){
            id
            body
            frontmatter {
                title
                date (fromNow: true)
            }
            timeToRead
        }
    }
`

Here's my plugin config in gatsby-config.js:

plugins: [
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `pages`,
        path: `${__dirname}/src/pages`
      }
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        name: `content`,
        path: `${__dirname}/content`
      }
    },
    {
      resolve: `gatsby-plugin-mdx`,
    },
    {
      resolve:`gatsby-plugin-feed`,
      options: {
        query: `
          {
            site {
              siteMetadata {
                title
                description
                siteUrl
                site_url: siteUrl
              }
            }
          }
        `,
        feeds: [
          {
            serialize: ({ query: { site, allMdx } }) => {
              return allMdx.edges.map(edge => {

                const siteUrl = site.siteMetadata.siteUrl
                const slug = siteUrl + edge.node.fields.slug
                const frontmatter = edge.node.frontmatter

                return Object.assign({}, edge.node.frontmatter, {
                  description: edge.node.excerpt,
                  date: frontmatter.date,
                  url: slug,
                  guid: slug,
                  custom_elements: [
                    { "content:encoded": edge.node.html }
                  ],
                })
              })
            },
            query: `
              {
                allMdx(
                  sort: { order: DESC, fields: [frontmatter___date] },
                ) {
                  edges {
                    node {
                      excerpt
                      html
                      fields { slug }
                      frontmatter {
                        title
                        date
                      }
                    }
                  }
                }
              }
            `,
            output: "/rss.xml",
            title: "RSS Feed",
          },
        ],
      },
    },
  ]
}

When the Caption component is imported directly into the Markdown file, i.e:

---
date: "2020-01-11"
title: "Blogpost test - title"
excerpt: "Excerpty text"
---
import Caption from "../../components/Caption"

## Something about this awesome image

Imagine an image goes here...
<Caption>A custom caption for my image!</Caption>

# And so it begins

Something or other about my blogpost.

the following is rendered in the allMdx.edges.node.html field, and thus my RSS feed (as expected):

<div style="font-size: 14px; text-align: center; color: grey; margin-top: -1rem; margin-bottom: 2rem;">The moguls below the ski lifts</div>

When the Caption component is supplied to MDXProvider via shortcodes (i.e. not imported directly), it is rendered as a div with none of the styling of the custom Caption component:

<div>The moguls below the ski lifts</div>

Additionally, I also get this error at gatsby build time:

warn Component Caption was not imported, exported, or provided by MDXProvider as global scope

It could be that I just don't know how to properly supply shortcodes in a way that the html field knows about them. I've read the docs, but can't find anyway of doing this.

Otherwise, it seems like a potential feature request to me.

Steps to reproduce

1. Create a new gatsby project
2. Install gatsby-plugin-mdx and gatsby-plugin-feed
3. Create a custom layout file, and supply a custom to MDXProvider using shortcodes.
4. Use createPages function in gatsby-node.js to generate pages using the above layout
5. gatsby build -- produces an warning in the console (see above), and the custom component styling isn't applied to the html field in allMdx.edges.node.mdx

Repo with two blogs is here: https://github.com/jjdevenny/gatsby-rss-shortcodes-bug
One blog imports the Caption directly and works (./src/pages/direct-import-test.mdx), the other imports the caption via shortcodes and doesn't work (./content/shortcodes-test.mdx).

Expected result

The html field supplied to gatsby-plugin-feed should respect the custom components I've supplied to MDXProvider via shortcodes in my layout file. My RSS feed should include the custom component styling + layout.

Actual result

The html field supplied to gatsby-plugin-feed doesn't include any of the custom components I have supplied via MDXProvider in my layout file.

I also get an error in the console on gatsby build:

warn Component Caption was not imported, exported, or provided by MDXProvider as global scope

Environment

System:
OS: macOS 10.15.2
CPU: (12) x64 Intel(R) Core(TM) i9-8950HK CPU @ 2.90GHz
Shell: 5.7.1 - /bin/zsh
Binaries:
Node: 13.6.0 - /usr/local/bin/node
Yarn: 1.19.2 - /usr/local/bin/yarn
npm: 6.13.4 - /usr/local/bin/npm
Languages:
Python: 2.7.16 - /usr/bin/python
Browsers:
Chrome: 79.0.3945.117
Safari: 13.0.4
npmPackages:
gatsby: ^2.18.21 => 2.18.21
gatsby-cli: ^2.8.26 => 2.8.26
gatsby-plugin-feed: ^2.3.26 => 2.3.26
gatsby-plugin-mdx: ^1.0.67 => 1.0.67
gatsby-source-filesystem: ^2.1.46 => 2.1.46
npmGlobalPackages:
gatsby-cli: 2.8.26
gatsby: 2.18.7

[daneden]

I’m experiencing this issue on my site too!

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[jjdevenny]

Can't seem to add a "not stale" label. This issue still exists and it'd be awesome to have someone look at it. Thanks.

[daneden]

+1! This has actually been resulting in a pretty bad experience for RSS readers subscribed to my site; for instance, I have some components passed via MDXProvider that render to links, but in RSS they get reduced to regular old div tags.

[github-actions[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.
If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!
As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[Js-Brecht]

Hi @jjdevenny. I just had a closer look at the html resolver, and it looks like you just need to put the <MDXProvider> in wrapRootElement.

I have this .mdx page:

// ./content/blog/index.mdx
---
title: Test
date: '2020-04-08T04:37:20.569Z'
---

<FooComponent>This is a test!</FooComponent>

And this setup:

// ./src/components/FooComponent.js
import React from 'react';

export const FooComponent = ({ children }) => (
    <div style={{
        fontWeight: 900,
        color: 'green',
        fontSize: 24,
    }}>
        <p>RENDERED CHILDREN:</p>
        <div>
            {children}
        </div>
    </div>
)

// ./src/wrap-root-element.js
import React from 'react'
import { MDXProvider } from '@mdx-js/react'
import { FooComponent } from './components/FooComponent';

const components = {
  FooComponent,
}
export const wrapRootElement = ({ element }) => (
  <MDXProvider components={components}>{element}</MDXProvider>
)

// ./gatsby-ssr.js
export { wrapRootElement } from './src/wrap-root-element'

All of this results in this HTML (I've converted its encoding XML->JSON):

"encoded": {
    "__prefix": "content",
    "__text": "<div style=\"font-weight:900;color:green;font-size:24px\"><p>RENDERED CHILDREN:</p><div>This is a test!</div></div>"
}

That's basically what you're trying to do, yeah?

[Js-Brecht]

gatsby-plugin-mdx could potentially be altered to accept a provider option parameter, which could just be a path to a module (or maybe a function :man_shrugging:)

What gatsby-plugin-mdx does, when the html field is requested, is wrap the mdx body with wrapRootElement from all of your plugins. So they essentially layer themselves over the top of body; pretty much how it happens during the normal rendering process. What a new provider option could do is wrap the final result with the defined provider module.

That might be more work than it's worth, though. Should work fine by just putting a provider inside of wrapRootElement. I can't think of any use case that might require a custom provider layer that couldn't be handled by wrapRootElement, but let me know if you can and what they would be.

[PaulBunker]

On my site I only use MDX for one template so having the provider available site wide would mean I have lots of shortcode components redundantly imported on other pages therefore increasing bundle size.

A provider option would be useful to me so I could have a module called shortcodes in one place, import that file in my 'docs' layout and also gatsby-config.

If you could give me a kick in the right direction (the file for the html resolver) I'm happy to have a stab at creating a PR.

[jjdevenny]

@Js-Brecht - thank you very much for the above suggestion. That has worked, and I can see the custom styles being applied to my RSS feed now.

I wasn't aware that the majority of RSS readers throw away inline styles, so the styles aren't rendering anyway, however they're there in rss.xml, and I no longer get 100+ warnings when I run gatsby build - which is awesome.

Thanks again. Closing this issue as your fix above is valid.

[daneden]

+1, @Js-Brecht your solution worked for me too! Unlike @jjdevenny this was less about inline styles for me and more about correct semantic output; I had custom Image and Link/a components, for example, that were getting completely ignored/transformed into a div in RSS because of this issue.

[Js-Brecht]

@PaulBunker I can see how that would be desirable, and I will show you how I can make that happen already; however, in this case it still might make sense to implement that feature just for resolution of the html field. I'll explain:

The code that you put in gatsby-ssr doesn't automatically make it into your runtime bundle... it is used to create an SSR bundle, which is then used to create the runtime bundle. The stuff that goes into gatsby-browser, as well as the rest of your app, is what makes it into the runtime bundle.

Lets say I make a module like ./src/components/Provider.js; all it does is create my <MDXProvider> and feed it the desired components:

// ./src/components/Provider.js
import React from 'react'
import { MDXProvider as Provider } from '@mdx-js/react'
import { FooComponent } from './foo-component'

const components = {
    FooComponent,
}

export const MDXProvider = ({ children }) => <Provider components={components}>{children}</Provider>

I can import that into something like ./src/templates/blog-post.js template, and it will be bundled up into the runtime component chunk for that template. If I also import and use that same module in gatsby-ssr (but NOT gatsby-browser), then I can use gatsby-plugin-feed as expected, and the final production bundle still only contains that module in the chunk for the blog-post.js template.

What this also means is that during SSR, you'll have two layers of <MDXProvider>... in short, something like this:

<MDXProvider>
   <MDXProvider>
      <MDXRenderer>
         {mdx.body}
      </MDXRenderer>
   </MDXProvider>
</MDXProvider>

This shouldn't cause any problems, because the innermost provider will take precedence, and the runtime tree won't have the duplication. However, this is the reason I think that it might still make sense to implement the provider option; to eliminate that duplication, and any confusion it might cause to see <MDXProvider> used in gatsby-ssr, but ignored in gatsby-browser.

---

If you're still interested in implementing the provider as a plugin feature, you'll find the necessary processing for the html generation following this chain:

create loader -> create webpack instance -> create React tree

Let me know if you need more direction.

[PaulBunker]

That took me a couple of read throughs 😅

I'll have some time over the weekend to have a stab at the plugin feature for sure!

Thanks again again