Gatsby: [www]: fix prism language warnings

[DSchau]

Description

We heavily use Markdown and, in particular, Markdown code fencing for our code snippets. For example:

````markdown
Hello, a code snippet is below

var a = 'waddup'
var b = 'its dat boi'

````

We _also_ use some custom features, e.g. titles. When building out gatsbyjs.org some of these code languages and features (titles) are not correct, and are therefore throwing a bunch of warnings. Let's fix 'em.

Steps to reproduce

1. Clone this repo
2. cd www
3. yarn && yarn build

warn code block or inline code language not specified in markdown. applying generic code block
warn unable to find prism language 'jsx:jsx:title=src/pages/index.js' for highlighting. applying generic code block
warn unable to find prism language 'es6' for highlighting. applying generic code block
warn unable to find prism language 'title:gatsby-node.js' for highlighting. applying generic code block
warn unable to find prism language 'javascript=gatsby-config.js' for highlighting. applying generic code block
warn unable to find prism language 'mdx' for highlighting. applying generic code block
warn unable to find prism language 'sh' for highlighting. applying generic code block
warn unable to find prism language 'js:gatsby-node.js' for highlighting. applying generic code block
warn unable to find prism language 'js:src/templates/post-list.js' for highlighting. applying generic code block
warn unable to find prism language 'js:src/components/layout.js' for highlighting. applying generic code block
warn unable to find prism language 'javascript=gatsby-node.js' for highlighting. applying generic code block
warn unable to find prism language 'title=.env.example' for highlighting. applying generic code block
warn unable to find prism language 'title=.env' for highlighting. applying generic code block
warn unable to find prism language 'javascript=' for highlighting. applying generic code block
warn unable to find prism language '[' for highlighting. applying generic code block
warn unable to find prism language 'terminal' for highlighting. applying generic code block
warn unable to find prism language 'title=public/' for highlighting. applying generic code block
warn unable to find prism language 'title=gatsby-config.js' for highlighting. applying generic code block
warn unable to find prism language 'console' for highlighting. applying generic code block
warn unable to find prism language 'env' for highlighting. applying generic code block
warn unable to find prism language 'hjson' for highlighting. applying generic code block
warn unable to find prism language 'htaccess' for highlighting. applying generic code block
warn unable to find prism language 'styl' for highlighting. applying generic code block
warn unable to find prism language 'gql' for highlighting. applying generic code block
warn unable to find prism language 'ml' for highlighting. applying generic code block
warn unable to find prism language '.json' for highlighting. applying generic code block
warn unable to find prism language '.gitignore' for highlighting. applying generic code block
warn unable to find prism language 'bash:' for highlighting. applying generic code block
warn unable to find prism language 'dosini' for highlighting. applying generic code block
warn unable to find prism language 'flowchart' for highlighting. applying generic code block

Expected result

0 or at least minimal warnings.

Actual result

A lot of warnings

Description

Some of the above warnings are fairly easy to fix. For example, several that have a title do not have a language, e.g.

title=gatsby-config.js -> js:title=gatsby-config.js # title is not stripped unless it has a language
javascript= -> javascript # remove the = sign
.json -> json # remove the dot
jsx:jsx:title=src/pages/index.js -> jsx:title=src/pages/index.js # remove the duplicate language

Others still are not using the correct or available alias, e.g.

gql -> graphql
env -> shell
mdx -> md # mdx is not a supported extension yet (hopefully soon!)

This issue can be marked done when there are zero warnings. A great approach would be to use an editor like VSCode, search for the string pattern in the docs/ folder, and fix the errors.

To validate, run yarn build and you should see _zero_ warnings.

[tusharf5]

Hey, @DSchau I would like to work on this. In fact, I've already started working on it and modified a lot of files.

The following warnings are still left though. I searched the full repo but I couldn't find the patterns (htaccess,styl,gql,ml, etc) mentioned in the following warnings. Do you have an idea where these could be? I don't think these are in the repo or specifically in the docs/ directory.

warn unable to find prism language 'console' for highlighting. applying generic code block
warn unable to find prism language 'es6' for highlighting. applying generic code block
warn unable to find prism language 'env' for highlighting. applying generic code block
warn unable to find prism language 'hjson' for highlighting. applying generic code block
warn unable to find prism language 'htaccess' for highlighting. applying generic code block
warn unable to find prism language 'styl' for highlighting. applying generic code block
warn unable to find prism language 'gql' for highlighting. applying generic code block
warn unable to find prism language 'ml' for highlighting. applying generic code block
warn unable to find prism language '.json' for highlighting. applying generic code block
warn unable to find prism language '.gitignore' for highlighting. applying generic code block
warn unable to find prism language 'dosini' for highlighting. applying generic code block
warn unable to find prism language 'mdx' for highlighting. applying generic code block
warn unable to find prism language 'flowchart' for highlighting. applying generic code block

[DSchau]

@tusharf5 I believe that those are third party packages/plugins, which is frustrating!

If you're feeling really fiery, you could identify which packages (their README.md) and then PR changes.

Here's a helpful query to detect (open up localhost:8000/___graphql after running gatsby develop)

{
  allNpmPackage {
    nodes {
      name
      readme {
        childMarkdownRemark {
          rawMarkdownBody
        }
      }
    }
  }
}

For example, the .json lang is coming from gatsby-plugin-intl-url, flowchart is from gatsby-remark-flowchart, etc.

[tusharf5]

@DSchau Did you mean to say that these files that exist in other packages, I should identify these packages and then submit a PR to their respective projects? Also, can I send a PR of the changes done in markdown files in this repo now or should I do it after I've identified all of them (including those from 3rd party packages)?

[tusharf5]

We can also put a config like this in gatsby-config.js to make it work for 3rd party markdowns.

    {
            resolve: `gatsby-remark-prismjs`,
            options: {
              aliases: {
                sh: "shell",
                es6: "javascript",
                env: "bash",
                mdx: "md",
                ".json": "json",
                ...other aliases
              },
            },
          },

[kml1326]

We can also put a config like this in gatsby-config.js to make it work for 3rd party markdowns.

    {
            resolve: `gatsby-remark-prismjs`,
            options: {
              aliases: {
                sh: "shell",
                es6: "javascript",
                env: "bash",
                mdx: "md",
                ".json": "json",
                ...other aliases
              },
            },
          },

I also puted this things and it works.

[gatsbot[bot]]

Hiya!

This issue has gone quiet. Spooky quiet. 👻

We get a lot of issues, so we currently close issues after 30 days of inactivity. It’s been at least 20 days since the last update here.

If we missed this issue or if you want to keep it open, please reply here. You can also add the label "not stale" to keep this issue open!

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks for being a part of the Gatsby community! 💪💜

[gatsbot[bot]]

Hey again!

It’s been 30 days since anything happened on this issue, so our friendly neighborhood robot (that’s me!) is going to close it.

Please keep in mind that I’m only a robot, so if I’ve closed this issue in error, I’m HUMAN_EMOTION_SORRY. Please feel free to reopen this issue or create a new one if you need anything else.

As a friendly reminder: the best way to see this issue, or any other, fixed is to open a Pull Request. Check out gatsby.dev/contribute for more information about opening PRs, triaging issues, and contributing!

Thanks again for being part of the Gatsby community!

[lannonbr]

Reopening this as it is still a issue at hand

[rmorabia]

I'll tackle this --

I'm going to work on fixing the bugs in error message in core Gatsby.

I'm a bit confused about how to implement the alias fix, though. Should this be default behavior that's added?

[sidharthachatterjee]

@rmorabia Thanks for picking this up!

The aliases could be added in gatsby-config.js for the www site so that's at https://github.com/gatsbyjs/gatsby/tree/master/www/gatsby-config.js

[rmorabia]

I think I have to abandon this -- running gatsby develop on the www folder seems to be too heavy for both my main computer & my AWS server. It seems to need to store gigabytes in memory and always fails on running.

If anyone has any advice, let me know! Otherwise, my computer just isn't powerful enough for this.

[kimbaudi]

For anybody who plans to work on this task, please keep in mind that this task can become cumbersome due to the sheer number of code blocks and inline codes you need to specify the correct prismjs language (or alias).

To get rid of the unable to find prism language warnings, you either need to specify the correct prismjs language or set an alias.

Specifying correct prismjs language on fenced code blocks

change from:
sh
to:
bash

set an alias to get around setting correct language on fenced code blocks

      {
        resolve: `gatsby-remark-prismjs`,
        options: {
          aliases: { sh: "bash" },
        },
      },

To get rid of the code block or inline code language not specified in markdown. applying generic code block warning, you either need to set language for inline code or set noInlineHighlight: true in gatsb-remark-prismjs plugin so that inline code would not be highlighted (not sure if this is desirable).

P.S.: The reason I am writing this is because I tried to remove prismjs language warnings on my personal site built with GatsbyJS by adding the correct prismjs language on all my fenced code blocks. I was able to get rid of all warn unable to find prism language warnings, but still kept getting the code block or inline code language not specified in markdown. applying generic code block warning. I later figured out that the warning was due to inline code blocks missing language. I never specified language on inline code (only block code) so I just set noInlineHighlight: true option in gatsby-remark-prismjs plugin and now all prism language warnings are gone.

[alexfornuto]

...you either need to set language for inline code...

I haven't found any documentation on how to do this globally, can you point us to it?

[kimbaudi]

I haven't found any documentation on how to do this globally

There is no way to set the language for inline code globally.
The most we can do is disable syntax highlighting for inline code globally.
This would be the same thing as setting language to none globally for all inline code.
(Again, not sure if this is desirable).
This will get rid of the code block or inline code language not specified in markdown warning.

I think for this task, it would be better to try tackling the warn unable to find prism language warnings first... one warning at a time. We should worry about the inline code warning last.

[kimbaudi]

@DSchau @tusharf5 - I resolved most unable to find prism language warnings by adding aliases in gatsby-config.js for known languages:

resolve: `gatsby-remark-prismjs`,
options: {
    aliases: {
    console: `bash`,
    dosini: `ini`,
    env: `bash`,
    es6: `js`,
    flowchart: `none`,
    gitignore: `none`,
    gql: `graphql`,
    htaccess: `apacheconf`,
    mdx: `markdown`,
    ml: `fsharp`,
    sh: `bash`,
    styl: `stylus`,
    terminal: `bash`,
    },
},

I also created PRs in a gatsby plugin (https://github.com/vtellier/gatsby-plugin-intl-url/pull/3/files) and a gatsby theme (https://github.com/danielhusar/gatsby-theme-spring/pull/6/files) to address the following warnings:

unable to find prism language '.json' for highlighting. applying generic code block
unable to find prism language 'js!gatsby-config.js' for highlighting. applying generic code block

I still can't figure out how to resolve the remaining 2 warnings:

unable to find prism language '.gitignore' for highlighting. applying generic code block
unable to find prism language 'bash:' for highlighting. applying generic code block

Searching for data-language=\".gitignore\" in the gatsby repo after running yarn build seems to point to /public/page-data/packages/gatsby-plugin-git/page-data.json so I tried to take a look at gatsby-plugin-git repo, but all I could find is ```text:title=.gitignore in the README.md file. I'm not sure if that is what is causing the warning and how to resolve it.

Also, searching for data-language=\"bash:\" in the gatsby repo after running yarn build seems to point to /public/page-data/packages/gatsby-source-plone/page-data.json so I tried to take a look at gatsby-source-plone repo, but couldn't find bash:.

[muescha]

I see there is a problem to find the code blocks throwing the errors.

Are there any debug options for the plugin to print out also the filename and an excerpt of the code to better find the places?

[kimbaudi]

I was actually thinking about this today because it was becoming really difficult to track where these warnings are coming from.

I'm not 100% sure if what you are asking is possible, but maybe it is. Take a look at the source. There is a try/catch block where the warning message is being created. The catch block's e argument might contain additional info about the error such as filename, line-number, excerpt.

    try {
      loadPrismLanguage(language)
    } catch (e) {
      // Language wasn't loaded so let's bail.
      let message = null
      switch (language) {
        case `none`:
          return code // Don't escape if set to none.
        case `text`:
          message = noInlineHighlight
            ? `code block language not specified in markdown.`
            : `code block or inline code language not specified in markdown.`
          break
        default:
          message = `unable to find prism language '${language}' for highlighting.`
      }

Actually, looking at loadPrismLanguage method, the error thrown doesn't have such info (filename, line-number, excerpt).

It might still be possible, but not with e argument from catch block. Looking a bit further, I see this line of code here:

highlightCode(languageName, node.value, highlightLines, noInlineHighlight)

So I'm hoping node object contains filename, line-number. I think node.value should contain the code excerpt, which is still helpful.

Update:

• node.value - returns the code excerpt
• node.position - returns the start/end positions (line, column, offset).

Unfortunately, I think it's impossible to get the filename/filepath where the warning occurs in third-party packages/plugins, which is what I was hoping for 😞. Trust me, I've tried.

[kimbaudi]

@DSchau @tusharf5 - Unfortunately, I'm unable to get rid of the remaining warnings because those warnings occur in third-party packages/themes:

unable to find prism language '.gitignore'
unable to find prism language 'bash:'
unable to find prism language '.json'
unable to find prism language 'js!gatsby-config.js'

We _could_ create aliases for these invalid languages as suggested by @tusharf5 (".json": "json",), but we really shouldn't do that just to get rid of these warnings.

---

Anyways, I was able to find the line number and code excerpt for the remaining warnings by using node.position and node.value and manually checking each third-party packages/plugins.

Note: gatsby-source-plone strangely shows the warning for bash:, but the fenced code block looks fine to me (i.e. ```bash instead of ```bash:)

unable to find prism language '.gitignore'

• https://raw.githubusercontent.com/etnbrd/gatsby-plugin-git/master/README.md

    ```text:title=.gitignore
    /content
    ```

unable to find prism language 'bash:'

• https://raw.githubusercontent.com/collective/gatsby-source-plone/master/README.md

    ```bash
    git clone [email protected]:collective/gatsby-source-plone.git
    ```

unable to find prism language '.json'

• https://raw.githubusercontent.com/vtellier/gatsby-plugin-intl-url/master/README.md

    ```.json
    {
        resolve: `gatsby-plugin-intl-url`,
        options: {
            defaultLanguage: 'fr',
            translationFile: 'src/locale.json'
        }
    }
    ```

    ```.json
    {
        "fr": {
            "slugs": {
                "boats": "bateaux",
                "mono-hull": "mono-coque",
                "multi-hull": "multi-coque",
                "sailboat": "voilier",
                "motorboat": "bateau-a-moteur",
            }
        }
    }
    ```

unable to find prism language 'js!gatsby-config.js'

• https://raw.githubusercontent.com/danielhusar/gatsby-theme-spring/master/demo/src/blog/installation.mdx

    ```js!gatsby-config.js
    module.exports = {
    siteMetadata: {
        ...
    },
    plugins: [
        {
        resolve: '@daniel.husar/gatsby-theme-spring',
        options: {
            paginationOffset: 5,
            author: 'Daniel Husar',
        },
        },
    ]
    }
    ```

• https://raw.githubusercontent.com/danielhusar/gatsby-theme-spring/master/theme/README.md

    ```js!gatsby-config.js
    module.exports = {
    siteMetadata: {
        ...
    },
    plugins: [
        {
        resolve: '@daniel.husar/gatsby-theme-spring',
        options: {
            paginationOffset: 5,
            author: 'Daniel Husar',
        },
        },
    ]
    }
    ```

---

So I suggest we only add aliases for valid languages and let the warnings show for invalid languages coming from third-party packages/themes.

[sidharthachatterjee]

@kimbaudi Incredible detective work on this! I agree that we should go ahead and add the aliases for the valid cases. Let me go ahead and merge your PR in!

In the mean time, anyone want to PR the third party repos to clean up the invalid cases? 😛

[alexfornuto]

I know this is a closed thread, but hopefully it will ping someone who can help. I've set noInlineHighlight: true and went through every single code block in my project to make sure it has a language specified. But I still get warn code block language not specified in markdown. applying generic code block.

Much like the Each child in a list should have a unique "key" prop., I wish this would specify where the offending code is.

[muescha]

You can put a breakpoint in the code where the error message is printed to find the doc where the warning comes from

[alexfornuto]

You can put a breakpoint in the code where the error message is printed to find the doc where the warning comes from

     ->->->->->inserting breakpoints into submodules of Gatsby ->->->->

     O <- My head
    /|\
    / \

[alexfornuto]

So I did try to set a breakpoint, and I found this in gatsby-remark-prismjs/highlight-code.js:

        case `text`:
          message = noInlineHighlight ? `code block language not specified in markdown.` : `code block or inline code language not specified in markdown.`;
          break;

Am I reading this correctly, that you get the warning regargless of the value of noInlineHighlight? Should that be the case?