Vscode: VSCode do not support .sublime-syntax which is more powerful than .tmLanguage

[sandogeek]

• VSCode Version:1.0.0
• OS Version:Windows 7
  would it be possible to support it in the future?I think .sublime-syntax is more easy to write and readable.

[kurtextrem]

Similar issue, which may be of some value here once it's fixed: https://github.com/atom/first-mate/issues/50

[borela]

I wrote a syntax that combines reactjs with flow which would be impossible without the extra control push/set of the context that the sublime-syntax allows; the yaml files are cleaner and you can break the syntax into multiple files.

[rattrayalex-stripe]

There are syntax highlighting bugs in ES7/Flow/JSX that a Microsoft employee claims are unsolvable with the current syntax highlighting setup.

[kumarharsh]

+1

This would be greatly appeciated. There are bugs like these: https://github.com/joshpeng/Sublime-Babel-VSCode/issues/24, and a few flowtype cases which I don't see being fixed by the tmlanguage approach for now. Is there some work being done on the highlighter internally @mjbvz?

Just discovered a new parser which seems to be better: https://github.com/tree-sitter/tree-sitter

[bobvanderlinden]

Tree-sitter sounds very promising. Here is a talk from the recent fosdem: https://youtu.be/0CGzC_iss-8

It is currently used in Atom as an experimental feature.

[kumarharsh]

Thanks for the video link - that was very informative!

[hickford]

For an explanation of .sublime-syntax files, see https://www.sublimetext.com/docs/3/syntax.html

Sublime Text can use both .sublime-syntax and .tmLanguage files for syntax highlighting. Sublime Syntax files are YAML files with a small header, followed by a list of contexts. Each context has a list of patterns that describe how to highlight text in that context, and how to change the current

[RaphaelDDL]

👍

One of the things that make me "yuck" now that I'm trying to move from Sublime to VSCode is exactly how highlighters are weak, e.g.: Sublime's package "Sass" which comes with scss and sass sublime-syntax is a lot superior to vscode's "scss".

When trying to discover how convert a sublime-syntax into tmLanguage, which ends not being possible as syntax is a superset of tm.

So I can't add enough how supporting something already established is the way to go instead of going to implement a new tree-sitter (#50140 && #585) which doesn't look like it's happening as per the time already open. But then again, this one also is old, seems MS doesn't care :(

[Yankie]

+1

[gtaifu]

+1

[michaelblyons]

@jeff-hykin, @DrSensor; bringing this over here from the off-topic at #50140:

The conversion is straightforward, but the Sublime engine still has the vast majority of the same problems.

Non-deterministic parsing kind of changes that. Thom had interactions with the Tree Sitter creator about capabilities that became a little cranky on one side. Syntect does not appear to be finished with the branch_point support, but there's a BNF-to-sublime-syntax converter now.

Jeff, I'd be interested in reading your article when you're done.

[jeff-hykin]

Thank you for calling me out on this! I was unaware of meta_backtracking_key, meta_backtracking_else, andfail as last time I dove deep into sublime parsing was before 2018. The I should've done better research. The branch point support also looks very interesting.

I'm glad you're interested in the article! I'll try to remember to @ you somewhere once I post it.

I'm especially excited by these sublime automaton features because the they will be trivial to add to the library that @matter123 and I have been working on. Not only that, but since we implemented it in a real language (Ruby) abstraction tools can be created to make the API much more practical and user friendly.

The branch point will be much more difficult to add, however the library already uses a very BNF-looking style of writing so its possible that once branch points are implemented, it could be easy (maybe even automatic) to have old syntaxes utilize branch points. Here's an example pattern for matching #include <iostream> using the library:

grammar[:include] = Pattern.new(
    # std_space allows for spaces or inline comments 
    start_of_line.then(std_space).then(
        Pattern.new(
            match: /#/,
            tag_as: "punctuation.definition.directive"
        ).maybe(spaces).then(
            match: /include/.or(/include_next/),
        ).then(word_boundary)
    ).maybe(spaces).then(
        # include_partial is a pattern for matching <>, "", and some other common possibilities
        include_partial
    ), 
)

My first criticism was that, yes that's impressive that it is a deterministic pushdown automaton, but technically BrainF**k is Turing complete haha, although I wouldn't want to write a parser using it either. But again, this is much alleviated by a library that can start doing abstractions on top of the raw features.