Linguist: GitHub does not support .sublime-syntax grammars

[pchaigno]

3079 added support to script/convert-grammars for '.YAML-tmlanguage' and '.sublime-syntax' grammar file extensions. @aroben confirmed that YAML wouldn't be an issue for GitHub's back-end as it uses the JSON files produced by convert-grammars.

Unfortunately, while .YAML-tmLanguages and .tmLanguages have the same format (same keys and structure), .sublime-syntax uses a different format (for instance, see AWK.sublime-syntax). I failed to see the extent of the differences before. But, since NSIS and awk syntax highlighting do not work on github.com anymore (see .nsi files and .awk files), my guess is that GitHub's back-end does not support .sublime-syntax's format. Is anyone able to confirm this?

On the short term, #3208 should fix this. On the longer term, I see several solutions:

• remove support for .sublime-syntax in convert-grammars and find new grammars for these languages
• find a way to convert between the .tmLanguage format and the .sublime-syntax format
• add support for the .sublime-syntax format in GitHub's back-end

/cc @arfon @aroben

[Alhadis]

Writing a small program to convert that to JSON format shouldn't be too hard. I'm guessing that's out of the question...?

[pchaigno]

I don't think it's out of question but it would have a maintenance cost: each format will evolve and we need backward compatibility.

[Alhadis]

Heh, don't worry. I write clean code. ;) Don't judge me for being a tab-user. :p

Is this the source I should be treating as a canonical and up-to-date reference?

[pchaigno]

It's not a matter of clean code @Alhadis. It's a matter of having several developers around who can maintain the parser. And I haven't even checked properly if such a script doesn't already exist.

[Alhadis]

To be honest, this isn't the first time this has crossed my mind. Somebody suggested that Atom support Sublime's format too. If there were a JavaScript tool to convert a Sublime-style grammar into other formats, it'd benefit both teams. =)

Having said that, if you _do_ find a tool that's already doing that, feel free to let the Atom fellas know.

[aroben]

@pchaigno It looks like Puppet highlighting is broken as well. https://github.com/russCloak/SublimePuppet/tree/a90d5a4e084a3bb17809e3b59033fbf813749ced/Syntaxes contains both a .sublime-syntax and a .tmLanguage file; I guess we're choosing the .sublime-syntax one?

[pchaigno]

3214 should stop the bleeding. Sorry for this :bow:

[lildude]

@vmg the submodule update I did for v6.0.0 has resulted in Vue and Elm doing a TextMate-to-Sublime switcheroo and this slipped past me without me noticing.

How hard would it be to implement Sublime syntax conversion as part of the new grammar compiler?

[vmg]

I'll be looking at this today ✨

[vmg]

The Vue and Elm error seems to be fixed in https://github.com/github/linguist/pull/4012, although this is not related to Sublime vs Textmate.

Closing this issue because we'll never support Sublime grammars in GitHub until the authors of Sublime release a reference OSS implementation -- which seems very unlikely to happen.

[trevordevore]

@vmg As I'm interested in seeing .sublime-syntax support on Github I asked about a reference implementation on the ST forums. One forum member pointed to the following project which is a Rust library for syntax highlighting using Sublime Text syntax definitions.

https://github.com/trishume/syntect/

Here is the post for reference:

https://forum.sublimetext.com/t/releasing-a-reference-oss-implementation-for-sublime-syntax/34904/4

I don't know if that helps or not but thought I would post it in case it does.