Packages: [Python] decorators highlighting is broken

[malexer]

It seems that detection of decorators in Python sources was broken somewhere between versions 3114 and 3124 of Sublime Text.

OS: Ubuntu 16.04

Steps to reproduce:

1. Download 3114
2. $ rm -r ~/.config/sublime-text-3
3. Open Sublime Text
4. Create new file with Python syntax
5. Paste code:

@decorator
def foo():
    pass

ER: decorator is highlighted in some way
AR: it is highlighted in green color

Do the same steps for 3124:
ER: decorator is highlighted in some way
AR: it is in white color - seems to be not highlighted at all

Also I have checked the "Python.sublime-syntax" file in both versions and it seems that decorator section was reduced a lot.

[FichteFoll]

Decorator highlighting was changed to make it more sane in most situations by not abusing the entity.name.function scope name (for entire function calls!).

Now it just highlights the @ as a keyword and assigns a meta.statement.decorator.python scope the the entire thing, which you may opt to colorize in your color scheme.

I'm interested in your suggestions however.

[FichteFoll]

The Java syntax, for example, assigns storage.type to decorators (and nothing to the @). We could do that too, but as it is now this would be the same scope as is used for def and class (see #582 for more).

Furthermore, unline in Java, decorators in Python can also be function calls, which means that any arbitrary expression may occur as an argument within the decorator statement. Looking at the code above, I don't think I would like everything in there to be highlighted in orange (before that change, the entire call had a purple background for me), so ... I guess we could limit the scope to the function beeing called.

How does that sound?

[wbond]

The issue of annotation scoping is one that we could use to sort of standardize on and add to the Scope Naming docs. Syntaxes that I think use something like this include: C#, Python, Java, Scala, Rust (attributes), Swift (attributes)

I think the approach of Java (applying a single scope to the entire construct) breaks as soon as multiple tokens are used. I believe this is true for all of the syntaxes listed above, including Java.

It may even be possible to argue some uses of __declspec() in C++ may be enough to use similar scopes to what we are discussing here. However, those are limited by the compiler, so they probably aren't really equivalent to what we are discussing here.

My general opinion of the issue (which I am open to having my mind changed about) is that:

1. There should be a consistent meta scope applied. Perhaps meta.annotation?
2. Any leading punctuation should get a scope
3. Individual syntax tokens should get their normal scopes (strings, integer, etc)
4. The annotation/attribute name should get a special scope something like variable.annotation (based on variable.function). Perhaps this is an opportunity to use the proposed identifier.* scope @FichteFoll?

[malexer]

I agree that having everything highlighted is not a very nice option here.

It will be more comfortable to have only @ and decorator name (function name) highlighted, in your example it is @pytest.mark.parametrize.

[malexer]

I have double checked and it seems that I can update the issue.

Decorator is highlighted as expected (the same way as before) for Python in case when it has parameters:

When we provide decorator without param it's name is not highlighted and it is colored in white:

Note: it is a clean install of sublime.

[FichteFoll]

FYI, this will most likely be addressed after #709 has concluded.

[wbond]

Just a note, I've made a decision at https://github.com/sublimehq/Packages/issues/709#issuecomment-266835130 about decorator/annotation scope names.

[wbond]

This should be addressed in the next dev build by a combination of #743 and tweaks to Monokai to highlight the new standard of variable.annotation.