Highlight.js: Propose: More nuance for match classes (class title? function title? etc.)

[joshgoebel]

Table of some specific suggestions:

|Name|Description|
|---|---|
|title.class| A class name |
|title.function| A function name |
|title.namespace| A namespace name |
|comment.block| A block comment |
|meta.shebang| A shebang line for a script file |

---

Is your request related to a specific problem you're having?

Yes, I'd like to have a way to label/highlight names of a given type outside of their original context.

For example:

class Cat extends Pet { ... 

Turns into <class><title>Cat</title><keyword>extends</keyword><title>Pet</title>. And if we wanted to style it a certain way we could target .hljs-class .hljs-title. Although Pet is semantically different here and that's not indicated at all.

The same is true for functions:

function cleanMyRoom() {}

Turns into <function><title>cleanMyRoom</title... and hence could be targeted by a theme with .hljs-function .hljs-title... but now we have the usage of these things outside of their definition:

var kitty = new Cat();
Cat.prototype.pet = function() { ... }

In both cases [re: Cat] we can easily figure out this is a "class" object and we could apply title to it... but is it a class? a namespace? a function name? We've lost all context.

And for functions:

let launchTheRockets = () => { ...

Technically launchTheRockets is a name/title here (and we could easily detect that and highlight)... _but it's not the function itself_ (which is () => {...}... so again we have a name without context... and we're forced to fall back to the generic title.

And this problem get worse for languages that also have namespaces that could also be syntactically inferred... all we have is title and perhaps context at the definition site - although we have no namespace class...

The solution you'd prefer / feature you'd like to see added...

I think we may need to consider some new classes, or expanding existing ones with sub classes:

• Alternative 1: New classes class_name, function_name, namespace_name
• Alternative 2: Expand existing classes with subclass: title.function, title.class, title.namespace...

The way alternative 2 would unfolding during rendering CSS might look something like:

<span class="hljs-title hljs-title-function">launchTheRockets</span>

IE, the existing "broader" class is preserved. This would require no changes to any existing styles, while allowing new themes (or upgrades) to target items with more specificity. I'm a little more attached to system 2 as it seems more easily "backwards compatible".

Any alternative solutions you considered...

Nope, I outlined my two thoughts above.

Additional context...

I spent some time reviewing how Prism handles this and it has a lot more named options for tracking what "kind of thing" a given identifier might be:

• constant
• class-name
• function-name (though most grammars use function)
• namespace

This allows it to make some really nice distinctions when highlighting (like treating classes differently than function name, etc).

[joshgoebel]

Option 2 also leaves a lot of runway for expanding other classes:

• variable
• variable.declaration
• title.constant
• title.function
• function.invocation
• meta.doctype
• meta.shebang
• comment.block
• number.hex
• literal.boolean

This allows grammars to start slowly becoming more expressive while also still thinking in terms of our initial list of classes to avoid all our existing themes from suddenly becoming invalid overnight. _Some of these would even come for "free" to existing grammars simply by our upgrading the mode library to be more explicit, such as "comment.block"..._

[joshgoebel]

I should note that technically it's possible to do this today in a grammar:

      { // ES6 class
        className: 'class', beginKeywords: 'class', ...,
        contains: [
          { skipBefore:/extends\s+/, begin: IDENT_RE, className: "title title-class title-class-super" },
          hljs.inherit(hljs.UNDERSCORE_TITLE_MODE, {className: "title title-class"})
        ]
      },

Ok, scratch that ALMOST possible (or possible if you want to hack the CSS a bit)... we only add the hljs prefix at the beginning of the string so you end up with:

hljs-title title-class

If someone called this a bug though I'm not sure I'd disagree. But I'd like to formalize this and come up with naming guidance so that over time we can see themes start to taken advantage of these things.

[joshgoebel]

I get tripped up on things like function.call/function.invoke vs title.function-call. I think one theme might decide it's nice to highlight function invocations while another might decide that's too noisy.

So filing them all under title.function is technically correct in both cases, yet semantically not enough information to be truly useful.

[egor-rogov]

Nice idea.
I've had a look into the classes reference and saw names like meta-keywords and meta-string, that indicates a desire for some hierarchical system.

[joshgoebel]

Yes, meta is the only thing we have broken up like that. It's a very good idea. TextMate goes a little overboard with it... going 3-5 levels deep - you can get VERY nuanced. :-)

And if we did it the way I'm suggesting above we could technically switch those to meta.keyword and meta.string and they'd output would still be exactly the same as it's always been.

[joshgoebel]

Oh Actually we have builtin-name also, which pairs very poorly with built_in (dash vs underscore, separation vs one word)... so our naming now isn't the most consistent. :-)

[egor-rogov]

I'm not sure you really need that level of details in highlighting, especially given that you can't 100% correctly parse source code with regexps... But consistency in classes naming (and usage) certainly won't hurt us (:

[joshgoebel]

Oh I wasn't suggesting we go that crazy, but it is easily possible for us to do things like contextually detect differences (in many cases) from a class name and say a function name, etc. So those are the easy wins.

[TheElectronWill]

Being more precise is a good idea!
Is there a reason why Pet in class Cat extends Pet is labeled as a "title"? It does seem wrong to me: what we are declaring here is Cat, this is the title, this is what is important here. Pet should be a type IMHO.

[joshgoebel]

Just how it's always been done? OTTOMH I think your logic sounds right. It's also still a class though, just not the one being defined... that might make it different than say string, not sure.

[joshgoebel]

The title below would also become name.function as grammars were updated.

return options.noHighlightRe.test(languageName);
//             \             ^-- name.function
//              `-- variable.property

Spending some time thinking about this... and it seems "variable.property" and "name.function" would add a lot of fidelity (often requested) and also be fairly easy to pattern recognize for at least most C-like languages. So instead of adding these and no themes having support, and then perhaps waiting forever (or living with only a very few themes that are "enhanced")... I think we could try our best to reuse the existing styling (even if it's not perfect)... so then immediately existing styles pick up the new highlighting and if someone wants to tweak them... I think this might cause some slight pain in the immediate future but have the best long-term result. So here is what I'm currently thinking:

• title renamed to "name" => name, name.class, name.function, etc.
• variable adds nuance => variable, variable.property, variable.constant, etc.
• normal (unhighlighted) used to auto-generate => punctuation (~85% opacity)
• string or number used to auto-generate => operator (tweaked opacity and small hue shift)

_All existing themes would receive the new styles_ via programatic updates during the build process, ie these new styles will simply be added when missing. If a theme wants to opt-out (or override the defaults) they need only add the missing selector to the style to prevent the auto-upgrade from adding a selector.

The "hierarchy" here would be handled by the build process... for example a CSS rule for .name would be auto-expanded to cover .name-function (name.function), .name-class (name.class), etc... I like the . notation, so I'd probably support that in the grammars, but we wouldn't necessarily have to - could just use the older - syntax there. So the generated HTML would include only a single class:

class <span class="name-class">Person</span> 

The alternative (for hierarchy) is generating a bunch of extra HTML which doesn't seem like a good idea to me. Rather a few additional selectors than a bunch of new HTML.

Thoughts? @allejo @egor-rogov

[joshgoebel]

At this point I think we could also perhaps consider allowing enhanced grammars to define their own semantic subgroups... Ie we dictate the top-level domain space "name" but then might allow name.whatever if 100% of programmers in that language would agree whatever is the appropriate contextual naming. This could also be solved with language aliases though (and is probably how we should do it)... So the fictional "Bicycle" language would just define an alias from "name.bicycle" to say "name.class"... then they could use the more descriptive name.bicycle when writing their grammar... this is a trick Prism uses to great effect.

[joshgoebel]

One final note about what I mean when I say:

I think this might cause some slight pain in the immediate future

I worry that some (not all) of our themes may have skewed towards being optimized for "scarcity"... ie perhaps "title" pops more than it should because this class was handed out more rarely in the past. That some themes may not handle the sudden flood of color as well as others... but again I think this could be dealt with on a case by case basis - even using the auto-generation logic to purposely "tune" or "exclude" some themes that got flagged as doing very poorly. This isn't an entirely new problem though, I noticed this a bit with the recent change to start highlighting class functions, etc.