Vscode: Wrong token colors display in editor

[awerlogus]

Issue Type: Bug

description:

While "Initializing JS/TS language features" message is shown, everything works right.

But, after initialization, some tokens (type and function names) somehow become colored as plain text.

Meanwhile token colors are still inferred right.

Steps to reproduce:

1) Install Onyx theme
2) Open any .js/.ts file
3) Write any function
4) Reload window

VS Code version: Code - Insiders 1.42.0-insider (c4d53a19df8799f5d49a4f49a6900c499e6fa27c, 2020-01-23T05:39:57.148Z)
OS version: Windows_NT x64 10.0.18363

System Info

|Item|Value|
|---|---|
|CPUs|Intel(R) Core(TM) i7-6500U CPU @ 2.50GHz (4 x 2592)|
|GPU Status|2d_canvas: enabled
flash_3d: enabled
flash_stage3d: enabled
flash_stage3d_baseline: enabled
gpu_compositing: enabled
multiple_raster_threads: enabled_on
oop_rasterization: disabled_off
protected_video_decode: unavailable_off
rasterization: enabled
skia_renderer: disabled_off_ok
video_decode: enabled
viz_display_compositor: enabled_on
viz_hit_test_surface_layer: disabled_off_ok
webgl: enabled
webgl2: enabled|
|Load (avg)|undefined|
|Memory (System)|7.87GB (2.35GB free)|
|Process Argv|.|
|Screen Reader|no|
|VM|0%|

[NovaLogicDev]

Been having the same issue, doesn't seemed to be tied to anything but the initialization of JS/TS language features. Probably related to #77140 as I'm not seeing the issue in non insider builds. This also has nothing to do with the integrated terminal.

[NovaLogicDev]

I spent a moment and realized that this issue is not just effecting colors within the document but also the appearance of those tokens in the code minimap. notice the missing pieces of code on in the screenshots below.

[awerlogus]

Version: 1.42.0-insider (system setup)
Commit: f3dbcea32a6b52ce819dc863e778fb8517357f61
Date: 2020-01-24T05:38:14.538Z
Electron: 7.1.7
Chrome: 78.0.3904.130
Node.js: 12.8.1
V8: 7.8.279.23-electron.0
OS: Windows_NT x64 10.0.18363

In the latest release function and type names are colored right, but this time variables became blue (because of wrong color inference only?)

While "Initializing JS/TS language features" message is shown, everything works right.

After initialization variables become colored blue

Color of variables is inferred wrong

Even if we force the color of token, it doesn't help

[aeschli]

Thus is caused bugs in by semantic highlighting that was just added. Some fixes have already gone in:

• I improved the Inspect Hover to show the rule that was matched in the theme. Because the token got a semantic token of type variable, we look for a theming rule matching scope variable.
• There's a bug with colors that have alpha components. that's fixed in tomorrows build.

Sorry for the trouble and I hope we get all the issue fixed soon. Please let me know if you still see issues with tomorrows I-Build.

If you want to turn off semantic highlighting, you can use "editor.semanticHighlighting.enabled": false

[awerlogus]

Sorry for semi-offtop, but can you make a feature with semantic detection of functions assigned to constants? I want all functions to be colored the same, but in some cases (when I return lambda expression from another function, for example) function selector is not being added to the token. (screenshots are made without of semantic highlight)

[awerlogus]

Also I realized just now that information about colors of tokens doesn't depend on "editor.semanticHighlighting.enabled" setting

---

[aeschli]

Also I realized just now that information about colors of tokens doesn't depend on "editor.semanticHighlighting.enabled" setting

Oh, yes, that's a bug that I wanted to fix

can you make a feature with semantic detection of functions assigned to constants

That's https://github.com/microsoft/vscode/issues/89337

[NovaLogicDev]

Looking at todays build with semantic highlighting enabled and I'm seeing the same issue, no apparent improvement.

[NovaLogicDev]

Bug seems to be resolved in todays build

[SPGoding]

VSCode Info

Version: 1.42.0-insider (user setup)

Commit: be0aca7188ec6a76e7c2379758c0fbc1e9c21f7b

Date: 2020-01-31T13:53:58.701Z

Electron: 6.1.6

Chrome: 76.0.3809.146

Node.js: 12.4.0

V8: 7.6.303.31-electron.0

OS: Windows_NT x64 10.0.18363

Though I don't have this problem with TS/JS, I encountered this with my own language server:

The Inspect Hover told me that these tokens in the second line were applied correctly, but they did not have the specific color. This happened after I deleted some tokens in the end of the last line.

I was using the SemanticTokensBuilder provided by vscode-languageserver package to build edits.

[aeschli]

I think all the mentioned issues got fixed.

• issue with colors and alpha
• don't render semantic tokens white if there's no color rule for it
• inspect hover improvents

Please file a new issue if you still see problems with semantic colors in TypeScript.

@SPGoding Please file a separate issue with the info of what tokens are created.