Linguist: Incorrect highlighting of React (JSX) fragments

[Mottie]

Sorry for adding more work on your plate 🙏

This appears to be a problem in the JSX grammar for https://github.com/github-linguist/babel-sublime, which doesn't allow me to open an issue.

Preliminary Steps

Please confirm you have...

• [x] reviewed How Linguist Works,
• [x] reviewed the Troubleshooting docs,
• [x] considered implementing an override,
• [x] verified an issue has not already been logged for your issue (linguist issues).

Problem Description

Example:

import React from 'react';

export default const test = () => (
  <>
    <p>Testing</p>
    <p>Fragments</p>
  </>
);

When React fragments (<> and </>) are used, the highlighting of these opening & closing (empty) tags are not consistent

Opening

<span class="pl-k">&lt;&gt;</span>

Closing

<span class="pl-k">&lt;</span>
<span class="pl-sr">
  <span class="pl-pds">/</span>&gt;</span>
</span>

And in one case, a diff is showing the opening block as an error

<span class="blob-code-inner blob-code-marker" data-code-marker="-">
  <span class="pl-ii">&lt;</span>&gt;
</span>

URL of the affected repository:

I shared a link to the diff above

Last modified on:

N/A? This isn't a problem with the language stats.

Expected language:

JSX

Detected language:

JSX

[lildude]

Sorry for the delay. Yes, this is an unfortunate side effect we have due to a breaking upstream change and no alternative grammar being found. See the comments in the long list of closed issues related to JSX highlighting. If you know of a maintained grammar that we can switch to, please let us know and we'll switch in a shot. Once again, this is something we've tried in the past but didn't succeed due to deficiencies in the alternate grammars.

[stale[bot]]

This issue has been automatically marked as stale because it has not had activity in a long time. If this issue is still relevant and should remain open, please reply with a short explanation (e.g. "I have checked the code and this issue is still relevant because ___."). Thank you for your contributions.

[stale[bot]]

This issue has been automatically closed because it has not had activity in a long time. Please feel free to reopen it or create a new issue.

[rajadain]

This is still a problem. VSCode's syntax highlighting works correctly. Is it possible to use one of their grammars? https://github.com/microsoft/vscode/tree/master/extensions/javascript/syntaxes

VSCode:

GitHub:

Notice how in GitHub the syntax highlighting stops after </> whereas it does not in VSCode.

[lildude]

VSCode's syntax highlighting works correctly. Is it possible to use one of their grammars?

Unfortunately not whilst it's only available in the main VSCode repo and we really don't want to have to clone that massive repo (as a submodule) every time someone clones this repo.

That said, I've done some quick testing and using those grammars instead of the current Javascript/JSX grammars looks like they might cover both uses (which I'd expect given the "name" in the grammar files).

I think I might have an idea of how we can take advantage of those whilst keeping the maintenance burden to a minimum.

Watch this space.

[Alhadis]

I think I might have an idea of how we can take advantage of those whilst keeping the maintenance burden to a minimum.

If the idea is _"shallow-clone submodules instead of checking out the whole damn history"_, then maybe hold off until I've finished this PR to achieve exactly that. 😝

I literally started working on this arvo, so...

Watch this space.

[lildude]

If the idea is _"shallow-clone submodules instead of checking out the whole damn history"_, then maybe hold off until I've finished this PR to achieve exactly that. 😝

Nope. My idea is to borrow the code from the VSCode repo, create a new repo and do exactly what the VSCode repo says automatically via GitHub Actions to build and update the grammars 😁... like this https://github.com/lildude/linguist-javascript 🎉

I still need to test this more (later after my 🏃) but the initial PoC seems to work and do the trick.

[Alhadis]

Ah, righto.

BTW, while I'm reminded, I should point out that forking a submodule as a way of version-locking is unnecessary — adding the following field to .gitmodules exempts a submodule from being updated via git submodule update:

~~~diff
[submodule "vendor/grammars/NSIS"]
path = vendor/grammars/NSIS

• update = none
  url = https://github.com/github-linguist/NSIS
  ~~~

Ergo, some of the forks at @github-linguist may be unnecessary.

[lildude]

🤦 I've just remembered, Linguist is no longer in control of Javascript highlighting so replacing the grammar will have no effect.

You can see this clearly by comparing how GitHub renders this sample:

• https://github.com/github/linguist/blob/master/samples/JavaScript/classes.js

... vs how Lightshow does which uses the grammar included in Linguist:

• https://github-lightshow.herokuapp.com/?utf8=%E2%9C%93&scope=source.js&grammar_format=auto&grammar_url=&grammar_text=&code_source=from-url&code_url=https%3A%2F%2Fgithub.com%2Fgithub%2Flinguist%2Fblob%2Fmaster%2Fsamples%2FJavaScript%2Fclasses.js&code=

GitHub now uses the treesitter grammar at https://github.com/tree-sitter/tree-sitter-javascript to highlight JavaScript so any problems with syntax highlighting need to be reported to that repo, not Linguist.

[lildude]

@rajadain Linguist is still however responsible for the grammar used if you've used the .jsx extension or set an override to force the language to be identified as JSX. The grammar used for this (mentioned in the OP) is very old and known to not work very well.

[lildude]

I know how we can solve this... move the .jsx extension into JavaScript and completely remove this section from Linguist:

https://github.com/github/linguist/blob/e8de1c0c9ae29984651661e9e242aca3d753dc68/lib/linguist/languages.yml#L2545-L2554

This will then use the treesitter grammar which already has React support and as can be seen from sample.js appears to correctly render the JSX code which exibits the problem in the sample.jsx file in the same gist at https://gist.github.com/lildude/a67721aeaef649bbd3f8a38101d99798