Theia: Why are TextMate files used for syntax highlighting?

[jeanlucburot]

Is there a special reason for the use of TextMate files (*.tmlanguage.json) in Theia? i.e. the Java langauge server has a JavaTextmateContribution implementing LanguageGrammarDefinitionContribution that points to two files: javadoc.tmlanguage.json and java.tmlanguage.json. So both of those JSON files consist of defintions specific to the Java language in this case. I assume those are all only for the visual part, hence for syntax highlighting.

Also, there are TypeScript libraries focusing on TmLanguage that end up in VS Code. There is also the same for the Eclipse IDE.

Now, TextMate by itself is a GUI editor that was initially created for use under macOS. So, are those TmLanguage files just a little part of the entire editor? Or are those in a separate project? Why is TextMate in use within Theia? Is there a special reason or has this just been adopted because of the Monaco Editor? Well, I assume there must be a connection between the Monaco Editor's Monarch library for syntax highlighting and the TextMate files that contentwise do the exact same, right?

[akosyakov]

Why is TextMate in use within Theia?

TextMate grammars (not editor) is a well-known format to define syntax highlighting. There are many ready to use grammars which are supported by the community. It used as well by other editors, like VS Code, Atom, Sublime and so on.

Is there a special reason or has this just been adopted because of the Monaco Editor? Well, I assume there must be a connection between the Monaco Editor's Monarch library for syntax highlighting and the TextMate files that contentwise do the exact same, right?

Right, Monaco has its own way to define tokenization with Monarch. It was initially built to support languages in VS Code. Eventually, they decided to switch for TextMate as well because of reasons listed above: https://github.com/Microsoft/vscode/issues/174#issuecomment-158083640

Theia could not use TextMate from the start since it was not possible to run the tokenization library in the browser (see "Why doesn't the editor support TextMate grammars?" here) Now, thanks to Wasm, it is possible and there are no reasons for Theia to stick with less supported format.