Gatsby: Rich Data or Structured Data not being rendered to "view page source"--- thus not being picked up by search engines

[pfftdammitchris]

Description

Metadata markup from components under the tree (anything in the Helmet components rendered after html.js) are not being rendered statically (thus not being picked up by Google Search Console's data structured tool tester). I used the Google structured data testing tool to test my metadata and it showed 0 results:

But this is not correct because I do have structured data and everything set up correctly with no errors, as I am able to inspect the page element and copy outerHTML of the head tag on google chrome on the page, and then use that now on the google structured data tool and see correct results:

This originally was found out when fetching the page by URL from google's remote tester which, as before, rendered 0 results, which led me to these discoveries.

I am using plain head tags on html.js, however for the rest of the components im using react-helmet like every other project out there.

Expected result

Should be rendered with the static pages and be viewed when viewing by "View page source" on google chrome.
Should be picked up and validated by google's testing tools

Actual result

Only the content inside the head tags from html.js are being rendered to the page source (for example on google chrome, right clicking on the page and viewing the source). However I can clearly see rich data and structured data when inspecting the page.

Environment

• Gatsby version (npm list gatsby): 1.9.247
• gatsby-cli version (gatsby --version): 1.1.50
• Node.js version: v9.9.0
• Operating System: Windows 10

File contents (if changed):

gatsby-config.js:

require('dotenv').config({
  path: `.env.${process.env.NODE_ENV}`,
})
const siteConfig = require('./siteConfig')

module.exports = {
  siteMetadata: {
    siteUrl: siteConfig.siteUrl,
    title: siteConfig.title,
    description: siteConfig.description,
    keywords: siteConfig.keywords,
    siteLogo: siteConfig.siteLogo,
  },
  plugins: [
    'gatsby-plugin-react-next',
    'gatsby-plugin-react-helmet',
    'gatsby-transformer-sharp',
    'gatsby-plugin-sharp',
    'gatsby-transformer-remark',
    {
      resolve: 'gatsby-plugin-google-fonts',
      options: {
        fonts: ['Roboto:300,400,500'],
      },
    },
    {
      resolve: 'gatsby-plugin-webpack-bundle-analyzer',
      options: {
        analyzerPort: 3000,
        production: false,
        disable: true,
      },
    },
    {
      resolve: 'gatsby-source-filesystem',
      options: {
        name: 'pages',
        path: `${__dirname}/src/pages/`,
      },
    },
    {
      resolve: 'gatsby-plugin-sitemap',
      options: {
        output: '/sitemap.xml',
        exclude: ['/contact', '/about', '/privacy-policy'],
        query: `
          {
            site {
              siteMetadata {
                siteUrl
              }
            }
            allSitePage {
              edges {
                node {
                  path
                }
              }
            }
          }
        `,
      },
    },
    {
      resolve: 'gatsby-plugin-google-analytics',
      options: {
        trackingId: '<censored>',
        // head: true,
        // exclude: []
      },
    },
  ],
}

package.json:
gatsby-node.js:

const path = require('path')
const { createFilePath } = require('gatsby-source-filesystem')
const kebabCase = require('lodash').kebabCase // eslint-disable-line
const uniq = require('lodash').uniq // eslint-disable-line
const random = require('lodash').random // eslint-disable-line

exports.onCreateNode = ({ node, getNode, boundActionCreators }) => {
  const { createNodeField } = boundActionCreators
  if (node.internal.type === 'MarkdownRemark') {
    const slug = createFilePath({ node, getNode, basePath: 'pages' })
    createNodeField({ node, name: 'slug', value: slug })
  }
}

exports.onCreatePage = ({ page, boundActionCreators }) => {
  const { createPage } = boundActionCreators
  return new Promise((resolve) => {
    // Remove sidebar for certain routes
    const disableSidebarForRoutes = ['/contact/']
    if (disableSidebarForRoutes.length) {
      disableSidebarForRoutes.forEach((route) => {
        const reg = new RegExp(route)
        if (reg.test(page.path)) {
          page.layout = 'indexNoSidebar'
          createPage(page)
        }
      })
    }
    resolve()
  })
}

exports.createPages = ({ graphql, boundActionCreators, store }) => {
  const { createPage } = boundActionCreators

  const staticTemplate = path.resolve('src/templates/static.js')
  const postTemplate = path.resolve('src/templates/post.js')
  const tagTemplate = path.resolve('src/templates/tags.js')
  const categoryTemplate = path.resolve('src/templates/categories.js')
  const adTemplate = path.resolve('src/templates/ad.js')

  return graphql(`
    {
      site {
        siteMetadata {
          siteUrl
          title
          description
          keywords
        }
      }
      allMarkdownRemark(
        sort: { order: DESC, fields: [frontmatter___date] }
        limit: 2000
      ) {
        edges {
          node {
            fields {
              slug
            }
            frontmatter {
              tags
              categories
              pageType
            }
          }
        }
      }
    }
  `).then(({ data, errors }) => {
    if (errors) {
      return Promise.reject(errors)
    } else {
      const allTags = []
      const uniqueTags = []
      const allCategories = []
      const uniqueCategories = []
      const posts = []
      const marksdowns = data.allMarkdownRemark.edges
      const { site: { siteMetadata } } = data

      store.dispatch({ type: 'LOAD_SIDEBAR_ITEMS', data })

      marksdowns.forEach(({ node }) => {
        if (node.frontmatter.pageType === 'static') {
          return createPage({
            path: node.fields.slug,
            component: staticTemplate,
            context: { siteMetadata },
          })
        }
        if (node.frontmatter.pageType === 'post') {
          if (Array.isArray(node.frontmatter.tags)) {
            allTags.push(...node.frontmatter.tags)
          }
          if (Array.isArray(node.frontmatter.categories)) {
            allCategories.push(...node.frontmatter.categories)
          }
          posts.push(node)
        }
        if (node.frontmatter.pageType === 'ad') {
          return createPage({
            path: node.fields.slug,
            component: adTemplate,
            layout: 'indexNoSidebar',
            context: { siteMetadata },
          })
        }
      })

      // Create blog post pages
      posts.forEach((node, i) => {
        // remember that posts were grabbed "descending"
        // so latest posts are first
        const previousPage =
          i !== posts.length - 1 ? posts[i + 1].fields.slug : null
        const nextPage = i !== 0 ? posts[i - 1].fields.slug : null
        return createPage({
          path: node.fields.slug,
          component: postTemplate,
          layout: 'indexNoSidebar',
          context: { previousPage, nextPage },
        })
      })

      // Create tag pages
      if (allTags.length) {
        uniqueTags.push(...uniq(allTags))
        uniqueTags.forEach((tag) => {
          return createPage({
            path: `/tags/${kebabCase(tag)}/`,
            component: tagTemplate,
            context: { tag },
          })
        })
      }

      // Create categories pages
      if (allCategories.length) {
        uniqueCategories.push(...uniq(allCategories))
        uniqueCategories.forEach((category) => {
          return createPage({
            path: `/${kebabCase(category)}/`,
            component: categoryTemplate,
            context: { category },
            layout: 'categories',
          })
        })
      }
    }
  })
}

gatsby-browser.js:

import React from 'react'
import { Router } from 'react-router-dom'
import { Provider } from 'react-redux'
import createStore from 'config/store'

exports.replaceRouterComponent = ({ history }) => {
  const store = createStore()

  const ConnectedRouterWrapper = ({ children }) => (
    <Provider store={store}>
      <Router history={history}>{children}</Router>
    </Provider>
  )

  return ConnectedRouterWrapper
}

gatsby-ssr.js:

import React from 'react'
import { Provider } from 'react-redux'
import { renderToString } from 'react-dom/server'
import { JssProvider } from 'react-jss'
import getPageContext from './src/getPageContext'
import createStore from 'config/store'

exports.replaceRenderer = ({
  bodyComponent,
  replaceBodyHTMLString,
  setHeadComponents,
}) => {
  // Get the context of the page to collected side effects.
  const pageContext = getPageContext()
  const store = createStore()

  replaceBodyHTMLString(
    renderToString(
      <Provider store={store}>
        <JssProvider
          registry={pageContext.sheetsRegistry}
          generateClassName={pageContext.generateClassName}
        >
          {React.cloneElement(bodyComponent, {
            pageContext,
          })}
        </JssProvider>
      </Provider>
    )
  )

  setHeadComponents([
    <style
      type="text/css"
      id="server-side-jss"
      key="server-side-jss"
      dangerouslySetInnerHTML={{
        __html: pageContext.sheetsRegistry.toString(),
      }}
    />,
  ])
}

[KyleAMathews]

Are you looking at the development or production version of the site?

[pfftdammitchris]

It happens both on the development and production version

[KyleAMathews]

Not sure but almost certainly it's a bug in your SSR setup. Hopefully someone else with experience with your setup can give you feedback!

[pfftdammitchris]

@KyleAMathews just for the future so we can both help out future questions about this same issue, I found out that it was because I had stuffed the layout/pages with attempted performance optimizations (layouts rendering null until componentDidMount fires, pages with the same optimizations, etc) which blocked the static build from rendering the inner components (the pages had react-helmet integrated but was not being rendered in time for the static builds)

[rayriffy]

Hi. I just pass along and found this issue. I can say that there's a real wired behavior for sure with crawler. Take a look https://productforums.google.com/d/msg/webmasters/CzznGbqVFho/cEdkSbz_BAAJ