Gatsby: How to keep new posts appear in index page with gatsby-plugin-offline

[rayriffy]

Summary

New blog/posts will not appear in front page immediately, I must navigate to other page or keep refreshing until it appear

Relevant information

This happened to me since the begining of using Gatsby V2.

When I published my blog to production with Netlify, new blog will not appear to the frontpage. I need to navigate to somewhere else or wait for a while before refreshing (also force reload with clearing cache CTRL+F5 doesn't work, too) in order to make new blog appear.

I think here must be some issue with gatsby-offline-plugin. I want to know that am I the only one who experienced this? Or if anyone can help me out with this problem.

Here's repo of my blog https://github.com/rayriffy/rayriffy-blog. It's migrated from gatsby-starter-blog

Environment (if relevant)

  System:
    OS: Windows 10
    CPU: x64 Intel(R) Core(TM) i7-5500U CPU @ 2.40GHz
  Binaries:
    Yarn: 1.13.0 - C:\Program Files (x86)\Yarn\bin\yarn.CMD
    npm: 6.5.0 - C:\Program Files\nodejs\npm.CMD
  Browsers:
    Edge: 44.17763.1.0
  npmPackages:
    gatsby: ^2.0.106 => 2.0.106
    gatsby-image: ^2.0.29 => 2.0.29
    gatsby-paginate: ^1.0.17 => 1.0.17
    gatsby-plugin-feed: ^2.0.12 => 2.0.12
    gatsby-plugin-google-analytics: ^2.0.12 => 2.0.12
    gatsby-plugin-google-fonts: ^0.0.4 => 0.0.4
    gatsby-plugin-manifest: ^2.0.15 => 2.0.15
    gatsby-plugin-netlify: ^2.0.7 => 2.0.7
    gatsby-plugin-netlify-cache: ^1.0.0 => 1.0.0
    gatsby-plugin-offline: ^2.0.22 => 2.0.22
    gatsby-plugin-react-helmet: ^3.0.5 => 3.0.5
    gatsby-plugin-robots-txt: ^1.4.0 => 1.4.0
    gatsby-plugin-sharp: ^2.0.19 => 2.0.19
    gatsby-plugin-sitemap: ^2.0.4 => 2.0.4
    gatsby-plugin-typography: ^2.2.6 => 2.2.6
    gatsby-remark-copy-linked-files: ^2.0.8 => 2.0.8
    gatsby-remark-embed-gist: ^1.1.5 => 1.1.5
    gatsby-remark-embed-spotify: ^2.0.2 => 2.0.2
    gatsby-remark-images: ^3.0.1 => 3.0.1
    gatsby-remark-prismjs: ^3.2.3 => 3.2.3
    gatsby-remark-responsive-iframe: ^2.0.8 => 2.0.8
    gatsby-remark-smartypants: ^2.0.5 => 2.0.7
    gatsby-source-filesystem: ^2.0.18 => 2.0.18
    gatsby-transformer-json: ^2.1.7 => 2.1.7
    gatsby-transformer-remark: ^2.2.2 => 2.2.2
    gatsby-transformer-sharp: ^2.1.13 => 2.1.13

error The system cannot find the path specified.

File contents (if changed)

gatsby-config.js:

Code

const {GATSBY_ENV = 'development'} = process.env

var hostname

if (GATSBY_ENV === 'production') {
  hostname = 'https://blog.rayriffy.com'
} else if (GATSBY_ENV === 'staging') {
  hostname = 'https://blog-staging.rayriffy.com'
} else if (GATSBY_ENV === 'development') {
  hostname = 'https://localhost:8000'
}

module.exports = {
  siteMetadata: {
    title: 'Riffy Blog',
    author: 'Phumrapee Limpianchop',
    description: 'The Nerdy Blogger',
    siteUrl: `${hostname}`,
  },
  pathPrefix: '/',
  plugins: [
    {
      resolve: `gatsby-plugin-google-fonts`,
      options: {
        fonts: [`kanit`],
      },
    },
    `gatsby-transformer-json`,
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `./src/assets/database`,
      },
    },
    {
      resolve: 'gatsby-plugin-robots-txt',
      options: {
        resolveEnv: () => GATSBY_ENV,
        env: {
          production: {
            policy: [
              {
                userAgent: '*',
                disallow: ['/pages', '/category', '/author'],
              },
            ],
          },
          staging: {
            policy: [
              {
                userAgent: '*',
                disallow: ['/'],
              },
            ],
          },
          development: {
            policy: [
              {
                userAgent: '*',
                disallow: ['/'],
              },
            ],
          },
        },
      },
    },
    {
      resolve: `gatsby-plugin-sitemap`,
      options: {
        output: `/sitemap.xml`,
        exclude: [
          '/pages/*',
          '/category',
          '/category/*',
          '/author',
          '/author/*',
          '/category/lifestyle/pages/*',
          '/category/misc/pages/*',
          '/category/music/pages/*',
          '/category/programming/pages/*',
          '/category/review/pages/*',
          '/category/tutorial/pages/*',
          '/author/rayriffy/pages/*',
          '/author/SiriuSStarS/pages/*',
        ],
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/src/pages`,
        name: 'pages',
        ignore: [`**/.*`],
      },
    },
    {
      resolve: `gatsby-source-filesystem`,
      options: {
        path: `${__dirname}/src/assets`,
        name: 'assets',
        ignore: [`**/.*`],
      },
    },
    {
      resolve: `gatsby-transformer-remark`,
      options: {
        plugins: [
          'gatsby-remark-embed-spotify',
          'riffy-gjs-embeded-video',
          {
            resolve: 'gatsby-remark-embed-gist',
            options: {
              username: 'rayriffy',
              includeDefaultCss: true,
            },
          },
          {
            resolve: `gatsby-remark-images`,
            options: {
              maxWidth: 1000,
              linkImagesToOriginal: false,
              sizeByPixelDensity: true,
              withWebp: true,
              quality: 80,
            },
          },
          'gatsby-remark-responsive-iframe',
          'gatsby-remark-prismjs',
          'gatsby-remark-copy-linked-files',
          'gatsby-remark-smartypants',
        ],
      },
    },
    `gatsby-transformer-sharp`,
    `gatsby-plugin-sharp`,
    {
      resolve: `gatsby-plugin-google-analytics`,
      options: {
        trackingId: `${
          GATSBY_ENV === 'production'
            ? 'UA-85367836-2'
            : GATSBY_ENV === 'staging'
            ? 'UA-85367836-3'
            : ''
        }`,
      },
    },
    `gatsby-plugin-feed`,
    {
      resolve: `gatsby-plugin-manifest`,
      options: {
        name: `Riffy Blog`,
        short_name: `Riffy Blog`,
        start_url: `/`,
        background_color: `#f5f5f5`,
        theme_color: `#1e88e5`,
        display: `minimal-ui`,
        icon: `src/assets/logo.png`,
      },
    },
    `gatsby-plugin-offline`,
    `gatsby-plugin-react-helmet`,
    {
      resolve: 'gatsby-plugin-typography',
      options: {
        pathToConfigModule: 'src/utils/typography',
        omitGoogleFont: true,
      },
    },
    {
      resolve: `gatsby-plugin-netlify`,
      options: {
        headers: {
          '/feed.json': ['Access-Control-Allow-Origin: *'],
        },
      },
    },
    `gatsby-plugin-netlify-cache`,
  ],
}

package.json:

Code

{
  "name": "rayriffy-blog",
  "description": "The Nerdy Blogger",
  "version": "1.1.0",
  "author": "Phumrapee Limpianchop <[email protected]>",
  "bugs": {
    "url": "https://github.com/rayriffy/rayriffy-blog/issues"
  },
  "dependencies": {
    "cross-env": "^5.2.0",
    "gatsby": "^2.0.106",
    "gatsby-image": "^2.0.29",
    "gatsby-paginate": "^1.0.17",
    "gatsby-plugin-feed": "^2.0.12",
    "gatsby-plugin-google-analytics": "^2.0.12",
    "gatsby-plugin-google-fonts": "^0.0.4",
    "gatsby-plugin-manifest": "^2.0.15",
    "gatsby-plugin-netlify": "^2.0.7",
    "gatsby-plugin-netlify-cache": "^1.0.0",
    "gatsby-plugin-offline": "^2.0.22",
    "gatsby-plugin-react-helmet": "^3.0.5",
    "gatsby-plugin-robots-txt": "^1.4.0",
    "gatsby-plugin-sharp": "^2.0.19",
    "gatsby-plugin-sitemap": "^2.0.4",
    "gatsby-plugin-typography": "^2.2.6",
    "gatsby-remark-copy-linked-files": "^2.0.8",
    "gatsby-remark-embed-gist": "^1.1.5",
    "gatsby-remark-embed-spotify": "^2.0.2",
    "gatsby-remark-images": "^3.0.1",
    "gatsby-remark-prismjs": "^3.2.3",
    "gatsby-remark-responsive-iframe": "^2.0.8",
    "gatsby-remark-smartypants": "^2.0.5",
    "gatsby-source-filesystem": "^2.0.18",
    "gatsby-transformer-json": "^2.1.7",
    "gatsby-transformer-remark": "^2.2.2",
    "gatsby-transformer-sharp": "^2.1.13",
    "lodash": "^4.17.11",
    "prismjs": "^1.15.0",
    "prop-types": "^15.6.2",
    "react": "^16.7.0",
    "react-adsense": "^0.0.6",
    "react-dom": "^16.7.0",
    "react-helmet": "^5.2.0",
    "react-icons": "^3.3.0",
    "react-typography": "^0.16.18",
    "riffy-gjs-embeded-video": "^1.3.1",
    "typeface-merriweather": "0.0.54",
    "typeface-montserrat": "0.0.43",
    "typography": "^0.16.17"
  },
  "devDependencies": {
    "babel-eslint": "^10.0.1",
    "eslint": "^5.12.1",
    "eslint-config-prettier": "^4.0.0",
    "eslint-config-standard": "^12.0.0",
    "eslint-plugin-import": "^2.16.0",
    "eslint-plugin-node": "^8.0.1",
    "eslint-plugin-prettier": "^3.0.1",
    "eslint-plugin-promise": "^4.0.1",
    "eslint-plugin-react": "^7.12.4",
    "eslint-plugin-standard": "^4.0.0",
    "prettier": "^1.16.2"
  },
  "homepage": "https://blog.rayriffy.com",
  "keywords": [
    "gatsby",
    "gatsbyjs",
    "react",
    "reactjs",
    "es6",
    "blog"
  ],
  "license": "MIT",
  "main": "n/a",
  "repository": {
    "type": "git",
    "url": "git+https://github.com/rayriffy/rayriffy-blog.git"
  },
  "scripts": {
    "dev": "cross-env GATSBY_ENV=development gatsby develop --https",
    "dev-staging": "cross-env GATSBY_ENV=staging gatsby develop --https",
    "dev-prod": "cross-env GATSBY_ENV=production gatsby develop --https",
    "lint": "./node_modules/.bin/eslint --ext .js,.jsx --ignore-pattern public .",
    "test": "echo \"Error: no test specified\" && exit 1",
    "format": "prettier --trailing-comma es6 --no-semi --single-quote --write 'src/**/*.js' 'src/**/*.md'",
    "develop": "cross-env GATSBY_ENV=development gatsby develop --https",
    "start": "npm run develop",
    "build": "cross-env GATSBY_ENV=production gatsby build",
    "build-staging": "cross-env GATSBY_ENV=staging gatsby build",
    "deploy": "gatsby build --prefix-paths",
    "fix-semi": "eslint --quiet --ignore-pattern node_modules --ignore-pattern public --parser babel-eslint --no-eslintrc --rule '{\"semi\": [2, \"never\"], \"no-extra-semi\": [2]}' --fix gatsby-node.js"
  }
}

gatsby-node.js:

Code

const _ = require('lodash')
const Promise = require('bluebird')
const fs = require('fs')
const path = require('path')
const {createFilePath} = require('gatsby-source-filesystem')

const {GATSBY_ENV = 'development'} = process.env

exports.createPages = ({graphql, actions}) => {
  const {createPage} = actions

  var siteUrl

  return new Promise((resolve, reject) => {
    resolve(
      graphql(
        `
          {
            site {
              siteMetadata {
                siteUrl
              }
            }
            allMarkdownRemark(
              sort: {fields: [frontmatter___date], order: DESC}
            ) {
              edges {
                node {
                  fields {
                    slug
                  }
                  frontmatter {
                    title
                    subtitle
                    status
                    author
                  }
                }
              }
            }
            allCategoriesJson {
              edges {
                node {
                  key
                  name
                  desc
                }
              }
            }
            allAuthorsJson {
              edges {
                node {
                  user
                }
              }
            }
            lifestyle: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/lifestyle/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            misc: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/misc/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            music: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/music/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            programming: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/programming/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            review: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/review/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            tutorial: allMarkdownRemark(
              filter: {frontmatter: {category: {regex: "/tutorial/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            rayriffy: allMarkdownRemark(
              filter: {frontmatter: {author: {regex: "/rayriffy/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
            SiriuSStarS: allMarkdownRemark(
              filter: {frontmatter: {author: {regex: "/SiriuSStarS/"}}}
            ) {
              edges {
                node {
                  frontmatter {
                    status
                  }
                }
              }
            }
          }
        `,
      )
        .then(result => {
          siteUrl = result.data.site.siteMetadata.siteUrl
          var filteredResult
          if (GATSBY_ENV === 'production' || GATSBY_ENV === 'staging') {
            filteredResult = {
              data: {
                allMarkdownRemark: {edges: null},
                allCategoriesJson: {edges: null},
                allAuthorsJson: {edges: null},
                lifestyle: {edges: null},
                misc: {edges: null},
                music: {edges: null},
                programming: {edges: null},
                review: {edges: null},
                tutorial: {edges: null},
                rayriffy: {edges: null},
                SiriuSStarS: {edges: null},
              },
            }
            const filterNode = [
              'allMarkdownRemark',
              'lifestyle',
              'misc',
              'music',
              'programming',
              'review',
              'tutorial',
              'rayriffy',
              'SiriuSStarS',
            ]
            _.each(filterNode, node => {
              filteredResult.data[node].edges = _.filter(
                result.data[node].edges,
                {node: {frontmatter: {status: 'published'}}},
              )
            })
            filteredResult.data.allCategoriesJson.edges =
              result.data.allCategoriesJson.edges
            filteredResult.data.allAuthorsJson.edges =
              result.data.allAuthorsJson.edges
          } else if (GATSBY_ENV === 'development') {
            filteredResult = result
          }
          return filteredResult
        })
        .then(result => {
          if (result.errors) {
            console.error(result.errors)
            reject(result.errors)
          }

          const posts = result.data.allMarkdownRemark.edges
          const catrgories = result.data.allCategoriesJson.edges
          const authors = result.data.allAuthorsJson.edges

          var filter
          const postsPerPage = 5
          if (GATSBY_ENV === 'production' || GATSBY_ENV === 'staging') {
            filter = 'draft'
          } else if (GATSBY_ENV === 'development') {
            filter = ''
          }

          // Create blog lists pages.
          const numPages = Math.ceil(posts.length / postsPerPage)

          _.times(numPages, i => {
            createPage({
              path: i === 0 ? `/` : `/pages/${i + 1}`,
              component: path.resolve('./src/templates/blog-list.js'),
              context: {
                limit: postsPerPage,
                skip: i * postsPerPage,
                status: filter,
                numPages,
                currentPage: i + 1,
              },
            })
          })

          // Create blog posts pages.
          var count = 0
          var jsonFeed = []
          _.each(posts, (post, index) => {
            const previous =
              index === posts.length - 1 ? null : posts[index + 1].node
            const next = index === 0 ? null : posts[index - 1].node

            if (count < 5) {
              jsonFeed.push({
                name: post.node.frontmatter.title,
                desc: post.node.frontmatter.subtitle,
                slug: siteUrl + post.node.fields.slug,
              })
            }

            createPage({
              path: post.node.fields.slug,
              component: path.resolve('./src/templates/blog-post.js'),
              context: {
                author: post.node.frontmatter.author,
                slug: post.node.fields.slug,
                previous,
                next,
              },
            })
            count++
          })

          fs.writeFile('public/feed.json', JSON.stringify(jsonFeed), function(
            err,
          ) {
            if (err) {
              console.error(err)
              reject(err)
            }
          })

          // Create category pages
          var categoryPathPrefix = 'category/'
          _.each(catrgories, category => {
            var totalCount = result.data[category.node.key].edges.length
            var numCategoryPages = Math.ceil(totalCount / postsPerPage)
            var pathPrefix = categoryPathPrefix + category.node.key
            _.times(numCategoryPages, i => {
              createPage({
                path: i === 0 ? pathPrefix : pathPrefix + `/pages/${i + 1}`,
                component: path.resolve('./src/templates/category.js'),
                context: {
                  category: category.node.key,
                  currentPage: i + 1,
                  limit: postsPerPage,
                  numPages: numCategoryPages,
                  pathPrefix,
                  regex: '/' + category.node.key + '/',
                  skip: i * postsPerPage,
                  status: filter,
                },
              })
            })
          })

          // Create author pages
          var authorPathPrefix = 'author/'
          _.each(authors, author => {
            var totalCount = result.data[author.node.user].edges.length
            var numAuthorPages = Math.ceil(totalCount / postsPerPage)
            var pathPrefix = authorPathPrefix + author.node.user
            _.times(numAuthorPages, i => {
              createPage({
                path: i === 0 ? pathPrefix : pathPrefix + `/pages/${i + 1}`,
                component: path.resolve('./src/templates/author.js'),
                context: {
                  author: author.node.user,
                  currentPage: i + 1,
                  limit: postsPerPage,
                  numPages: numAuthorPages,
                  pathPrefix,
                  regex: '/' + author.node.user + '/',
                  skip: i * postsPerPage,
                  status: filter,
                },
              })
            })
          })
        }),
    )
  })
}

exports.onCreateNode = ({node, actions, getNode}) => {
  const {createNodeField} = actions

  if (node.internal.type === `MarkdownRemark`) {
    const value = createFilePath({node, getNode})
    createNodeField({
      name: `slug`,
      node,
      value,
    })
  }
}

gatsby-browser.js: N/A
gatsby-ssr.js: N/A

[krismorf]

Hi @rayriffy , you can use the onServiceWorkerUpdateReady API for that.

https://www.gatsbyjs.org/docs/add-offline-support-with-a-service-worker/#displaying-a-message-when-a-service-worker-updates

[rwasiak]

@rayriffy I observed the same problem with my blog. My config setup is very similar to yours. I also use Netlify. Blog is built on gatsby-starter-blog.

Recently my blog "disappeared" after deploy. I got following error in console:
https://ibb.co/vhzCWy7
I had to unregister service worker and refresh page - blog came back to life.

@krismorf
Asking users for reloading page each time the blog post is published is not good solution in my opinion. Blog's content should be up-to-date for every user visit without reloading the page :)

[krismorf]

@Remigiusz-Wasiak You don't have to ask the user to reload or click a button. You only need window.location.reload(). The rest is an example if the user wants to add a prompt or notification.

[rayriffy]

So, the solution will be something like

gatsby-browser.js:

Code

exports.onServiceWorkerUpdateReady = () => {
  window.location.reload()
}

I will try this and I will give resolution within week

[rayriffy]

I just published new blog on my site. My expectation is when new ServiceWorker is updated with new blog it's going to automatically reload and show a new one. But it didn't the experience still the same.

[rayriffy]

EDIT: It's working but it need to reload one more time to get gatsby-browser in action. Closing issue now.