Monaco-editor: Colorization Clarification

[JKillian]

Had a quick question on colorization / syntax highlighting. I see Monaco supports Monarch-style languages files as collected here.

In this comment, @aeschli indicates that VSCode will mainly be using .tmLanguage files since they're a standard format and well-defined in the community.

The comment also states that Monaco will continue to use the JSON-based Monarch format because of implementation details. This is somewhat a shame - it'd be great if the .tmLanguage format was supported by Monaco! Are there any plans for that in the future, or is it fairly set that only the Monarch format will be supported. Is there an easy way to convert from .tmLanguage to Monarch format? (I'm not closely familiar with either)

[JKillian]

For context: I'm looking for Groovy syntax-highlighting (as requested in #129) and trying to figure out how much time/work it'll take to support it.

[alexdima]

@JKillian The underlying reason for not being able to support .tmLanguage grammars is very simple:

• all the regular expressions in TM grammars are based on oniguruma, a regular expression library written in C.
• the only way to interpret the grammars and get anywhere near original fidelity is to use the exact same regular expression library (with its custom syntax constructs)
• in VSCode, our runtime is node.js and we can use a node native module that exposes the library to JavaScript
• in Monaco, we are constrained to a browser environment where we cannot do anything similar
• we have experimented with Emscriptem to compile the C library to asm.js, but performance was very poor even in Firefox (10x slower) and extremely poor in Chrome (100x slower).
• we can revisit this once WebAssembly gets traction in the major browsers, but we will still need to consider the browser matrix we support. i.e. if we support IE11 and only Edge will add WebAssembly support, what will the experience be in IE11, etc.

[JKillian]

Thanks for the explanation - your reasoning makes sense to me.

we have experimented with Emscriptem to compile the C library to asm.js, but performance was very poor even in Firefox (10x slower) and extremely poor in Chrome (100x slower).

Neat idea! Too bad it didn't work out.

[oldrich-s]

@alexandrudima Any news on that? Could you please advise on how to implement the tmLanguage feature? There already exists a webassembly version of oniguruma:

https://github.com/NeekSandhu/onigasm

and the webassembly also seems to gain traction:

https://caniuse.com/#feat=wasm

[NeekSandhu]

@czb Say no more!!

monaco-textmate

[brijeshb42]

https://github.com/brijeshb42/monaco-ace-tokenizer

[gkemp94]

Would it be easier to convert a visual studio language extension to a monaco recognized format? Or use the above suggested workarounds to use the theme in monaco? If it's easier to convert the language extension, what would be the best way to achieve that conversion?

I'm trying to convert the following vscode language support.

https://github.com/lwshang/vscode-q

[NeekSandhu]

If your goal is to support just the modern browsers which are webassembly ready, it's recommended to go the easy way.

Rewriting the language syntax vs. plugging some wires.

https://github.com/NeekSandhu/monaco-editor-textmate