Aspnetcore: Update TextMate grammar to properly represent Razor content.

[NTaylorMullen]

Right now we're re-using the built-in Razor TextMate grammar but it has its own flaws. One of which is the following end </p> tag is not colored correctly: <p>@DateTime.Now</p>

---

Updates after initial release

Have done a lot of work to get the Razor grammar into a usable state for most scenarios. That being said there are several known things I still haven't done:

• [x] Markin in functions
  
• [x] Razor templates
  
• [x] await foreach
  
• [x] Email addresses
  

• [x] Nested HTML in non-transitioned C# constructs
  

• (Optional) Known TagHelpers. This can technically be done at the semantic colorization level as well
  

• (Optional) TagHelper C# attribute content.This can technically be done at the semantic colorization level as well
  

---

Areas for improvement

• Mark embedded languages in the grammar and extension configuration. This will enable things such as contextual comments (C# comment = // or /* ... */, or HTML = <!-- ... -->) and specific embedded language brace matching/indentation.

• Customize default VSCode themes to make the Razor colorizations more familiar. For instance, in VS the @ transitions are yellow and the C# bits that follow are distinct from HTML.

[NTaylorMullen]

This new grammar should be in its own repo a lot like https://github.com/Microsoft/TypeScript-TmLanguage

That way VSCode itself will ship our grammar.

[NTaylorMullen]

https://github.com/aspnet/Razor.VSCode/issues/383#issue-473664717 has some very specific examples on when this falters. @danroth27 this is by far our number 1 area for improvement in Razor VSCode, issues piling up as dups 😄. I also haven't linked every dup issue so there's more than just mentioned here, i'll get better at that!.

[peterblazejewicz]

@NTaylorMullen

• [ ] @code blocks (this specific screen comes from Mac OS, but I consider this xplat):

Where should someone look when considering contr., here or Omnisharp/VSCode?

[NTaylorMullen]

@peterblazejewicz this is the correct repo 😄. I've gone ahead and added a lot of colorization support to the next release. Once O# releases (sometime this week) you should see improvements.

[NTaylorMullen]

The only required pieces of the grammar that are yet to be completed are intermingling HTML into deep C# structures. Blocking issues for this are tracked by microsoft/vscode-textmate#122, microsoft/vscode#87496

Now for why, when we hand off the grammar to the C# grammar it doesn't always give us the opportunity to enable HTML constructs. For top level constructs like @if (...) { we understand everything in the declaration portion, the keyword, condition and opening brace. This means we can allow for HTML inside of the body (the opening brace) because we declare what the body allows; however, in the case of:

@code {
    public void RenerHTML()
    {
        // Content
    }
}

we don't understand method declarations (Razor hasn't had a need to understand them). This means when we let the C# grammar parse the body of @code it will understand method declarations in their entirety and never unweave to give us the opportunity to "allow" HTML.

The two issues I have filed on VSCode take advantage of a TextMate grammar concept called injection grammars. Basically I had the idea that we could tie into specific C# constructs with an injection grammar to give us an opportunity to allow HTML. Turns out they don't currently support injections into child grammars, aka when a top-level grammar (aspnetcorerazor) calls into a child grammar (source.cs).

If VSCode isn't willing to fix the issue we'll need to copy the C# grammar wholesale and edit each piece individually. Or if VS' TextMate grammar support doesn't support injection grammars either we'll have to do this as well.

Leaving this issue as blocked until we hear back from the VSCode and VS folks (i've started those conversations).

[NTaylorMullen]

Hmm, VSCode's PHP grammar does similar things and they've been able to get this to work. When I have time i'll have to look into this more.

[NTaylorMullen]

Heard back from VSCode folks and if we change how we inject we can get this to work. Unblocking

[NTaylorMullen]

Syntactic colorization work complete!