Vscode-ruby: Some, but not all, syntax highlighting has stopped working.

[jknichel]

Your environment

• vscode-ruby version: 0.16.0
• Ruby version: 2.4.3
• VS Code version: 1.19.3
• Operating System: macOS Sierra (10.12.6)
• Hardware (optional): MacBook Pro (15-inch, 2016)

Other Ruby VSCode Plugins:

• ruby-rubocop
• rspec-snippets

Expected behavior

Syntax highlighting should work. Method names should be highlighted in yellow, symbols in blue, etc.

Actual behavior

Method name highlighting, variable name highlighting, and symbol highlighting have stopped working. There may be more, but these are the most obvious culprits right now. I'll keep exploring to see if there's more.

Highlighting for things like require, module/class declaration and names, def/end, if/else/end, and a lot of other things are still working normally.

Here's a screenshot showing the state of some functionality:

Steps to reproduce the problem

This happened without any intervention on my part at some point today. I'm assuming the 0.16.0 update did this because it was the first update in 6 months and was released today.

[jknichel]

UPDATE: I manually reverted to 0.15.0 and all syntax highlighting is back. If you're experiencing this and want to do the same to stay productive, follow the VSCode docs for manually installing extensions.

You'll have to do the following:

1. Download the extension by following the steps in the VSCode docs, linked above.
2. Change the file extension to: .VSIX instead of .VSIXPackage
3. Uninstall the Ruby extension
4. Reload
5. Open the Command Palette and search for VSIX, select the option that reads "Install from VSIX..."
6. Select the file you downloaded/renamed in steps 1 and 2, then click reload at the top of the screen

VSCode is going to update to the latest version of this extension every time you reboot it, so you'll have to perform steps 3-6 each time you open it.

Note: you can add "extensions.autoUpdate": false to the user settings file to avoid having to repeat this every time you start up VSCode, but if you do this I'd recommend checking back regularly to see if this issue has been fixed, so you can keep up with the latest.

[rafasoares]

I'm facing a similar issue, with the following differences:

• Ruby version: 2.5.0
• Operating system: macOS High Sierra version 10.13.3

Downgrading brings it back.

[meddario]

I can confirm this in VSCode 1.19.2, on Ubuntu 17.10 and macOS Sierra 10.12.6.

[nmoadev]

Confirmed with:
vscode-ruby version: 0.16.0
Ruby version: 2.4.2p198 (2017-09-14 revision 59899) [x64-mingw32]
VS Code version: 1.19.3
Operating System: Windows 10 Enterprise, (10.0.15063)

[bbtdev]

Same problem, confirm fix by jknichel
Here is the url
https://rebornix.gallery.vsassets.io/_apis/public/gallery/publisher/rebornix/extension/ruby/0.15.0/assetbyname/Microsoft.VisualStudio.Services.VSIXPackage

[TeeSeal]

In fact, all syntax highlighting from vscode-ruby is gone. Try disabling the package and see that the highlighting doesn't change at all. Something dun broke.

[wingrunr21]

I just looked into this. Syntax highlighting is not gone or broken but it has changed. This looks to be due to the change I introduced when I pointed this project back at the upstream grammars instead of using the grammar that was vendored.

TL;DR: Reverting #242 should put things back

My reasoning for repointing at an upstream grammar was that the TextMate grammar that was being used here was originally tweaked in 2013 (see here). The tweak was what the author called an "optimistic grammar" in that it tried to highlight as much as it could instead of highlighting the language grammar itself. As this grammar and the original TextMate grammar are relatively unmaintained, it made sense to point it at Atom's grammar as it is highly maintained due to GitHub's involvement there. If you check that same Ruby file within Atom, you'll see syntax highlighting is also "broken" there.

My reasoning was further compounded by the lack of maintenance this project was getting. Not relying on an upstream grammar means the grammar vendored here must be maintained on its own.

IMO the best way moving forward would be to implement something like tree-sitter-ruby so that a proper syntax tree can be constructed and highlighted

[meddario]

@wingrunr21 for some parts I agree with you that the previous grammar was far from perfect and sometimes inconsistent, but what about trying to keep it simple and just re-add highlighting for symbols and method names? (don't know about the required effort, I never tried to do this in VSCode)
Regarding atom, personally I don't think that it has great syntax highlighting for ruby, as far as I know Sublime Text and RubyMine are the "reference" editors/IDEs for the ruby world, maybe it would be better to aim at their syntax highlighting than atom (IMHO, obviously).

p.s. thanks for all the work you have done (and that you are doing) for this extension, it's great to know that it's maintained again!

[wingrunr21]

Well, we could take a look at the changes made in that "optimistic" grammar and see how to apply them against Atom's upstream grammar. It'd probably take some work as Atom's grammar has deviated quite a bit from the original TextMate grammar.

I think Sublime's syntax was originally based on TextMate as well. The BetterRuby syntax improved things from the TextMate grammar, but we'd have to convert it from the sublime-syntax format.

I don't think RubyMine is a fair comparison to a simple grammar. That's a full IDE. It has the ability to build a syntax tree in order to do highlighting. My proposal to use something like tree-sitter-ruby should in theory give this extension the same ability.

I'm just going through and triaging issues as I see lots of the same issues every day too. I'm still not a maintainer on the project so just trying to get a handle on things so as to get as far as possible.

[rebornix]

@wingrunr21 I just added you as Collaborator, thanks for your help on issue triaging.

[avandecreme]

@wingrunr21 Do you think we could add a configuration to set the grammar to use?

It looks like a lot of people prefer the optimistic one so I think it would make sense if it is technically doable.

[wingrunr21]

@avandecreme the grammar is defined at the package level. I don't know if VSCode even supports dynamically loading a grammar. I'd have to dig through the docs to find out.

[wingrunr21]

ok, started digging into this more. Here's what I've got so far:

• Symbol highlighting: It appears a lot of themes (including core VS Code themes) don't implement the theme scope necessary for it. Based on the docs, the Sublime/Textmate scope for Ruby symbols should be constant.other.symbol. In this repository, da78acb renamed that scope to be constant.language.symbol, probably to work around this. We could provide an automatic rename of this scope, but that would mean that themes/users (such as Monokai) who do provide a correct scope color would not be themed correctly.
• Variable highlighting: the grammar is missing the variable.other.ruby scope. This can be added.
• Method name highlighting: the grammar is missing the meta.function-call.ruby scope. This can be added

I'm working to figure out how we can support both scopes for the symbol highlighting and then I can push a fix.

[wingrunr21]

Hey all,

Sorry, I have not forgotten about this. My company is in the middle of several large client launches that are monopolizing my time. I thought I'd have time to look this past weekend but ended up having to deal with other items. It's the first thing on my list right now.

Thanks!

[TeeSeal]

Thanks a lot for maintaining this @wingrunr21! I really appreciate your dedication as VSCode is my editor of choice and I want it to have support for ruby that rivals the other editors. I will also try to contribute to things other than auto indentation 😅 when I have time.

As for this issue, maybe we could have a flag/option that would trigger between the upstream grammar and the vanilla one, and if we do make an option, make it an opt-out until we get the upstream grammar sorted out. Opt-out as in: use vanilla grammar by default and have the option explicitly declare we want the upstream one. I'm not quite sure about implementation as I haven't worked with vscode packages a lot, but this is just an idea.

It kinda makes me feel bad having to use 0.15.0 and disabling package auto updates just to have proper highlighting.

[wingrunr21]

The problem is the grammar is shipped as a vendored file and then referenced by VSCode in the extensions package.json file. As far as I've been able to determine so far, there's no way to dynamically select a grammar to apply to a given file based on a user space config setting. The only way to configure it is via the package.json file.

I'm 100% sure I can restore the previous highlighting functionality while also using the upstream grammar. The holdup right now is the symbol highlighting. I think what I'm going to end up doing is putting in support for constant.language.symbol and then do a follow-up PR once I figure out how to support both highlighting cases.

[wingrunr21]

279 should fix this