Packages: [Meta] A new workflow for scope naming guidelines

[FichteFoll]

Current Situation

The scope naming guidelines on the official docs can't catch up with how frequently and detailed scoping is being discussed at this repo currently. Because the website source isn't public, the only people to work on them are Will and Jon, and they arguably have more important things to do.

We currently have a couple labeled issues on this repo, most of which are still open and without having reached a conclusion. The direction of the discussions is sometimes clear, but not always as it's fairly loose (which is fine by me), but the lack of a properly formulated conclusion that can be applied to actual syntax definitions is frustrating.

Thus, I propose a more formal workflow for scope naming guidelines that will hopefully produce more, faster and better results.

Goals

1. Have a referable collection of scope naming conventions and guidelines for specific syntax constructs that we have discussed previously. The guidelines should be clear and cover all usual and the most common edge cases for the topic they address.
2. That collection should have a certain authority over scope naming so that it can be adapted for the default packages as well as third-party syntax definitions.
3. Reduce delay between discussions of guidelines to the actual formulation of these.

For example, there would be a document about string scoping that covers current practices of the general string scopes but also how string interpolation is handled and what that actually is.

Proposal

I propose to adapt a workflow similar to PEPs, where a document is prepared by a member of the community and other community members can discuss and refine it until it is eventually added to the guideline collection or rejected.

To concretize:

• There would be a repository on the SublimeText Github organization.
• That repo will be used to store files with markup text explaining the individual guidelines, as well as outline the workflow.
• New proposals are suggested through pull requests and then discussed and adjusted until it is "accepted" (see later).
• Guidelines can be edited later (again though a PR and review process)
• Before a guideline is formulated, discussion may take place in issues on the repo.

Details:

• The potentially multiple authors of a guideline become the primary negotiators for any further changes to that guideline.
• Guidelines will need a number associated with them so they can be renamed without losing referability, sort of. I think a zero-padded three-digit number will suffice. Filenames would then follow the schema <number>-<title>.<ext>.
• The guidelines need to be versioned. A header section with metadata, including the document's version, last change date and authors would be useful. Semver seems overkill, but makes thinking about our own version schema unnecessary. The first accepted version should then always be 1.0.0.

Questions

1. How should our repo and the proposals be named?

   • "ScopeNaming" and "Scope Naming Proposal", short SNP
   • "ScopeNamingGuidelines" and "ScopeNamingGuideline", short SNG
   • any other suggestions?

2. When will a guideline be "accepted"? We could have a majority vote of a selected circle of editors or a single authority in the form of @wbond, which will most likely have to consent to guidelines we want to apply to the default packages anyway. Until that, it's just discussing the guideline and summarizing it for @wbond, so he doesn't have to spend as much time deciding and reading through most comments as currently.

   A good alternative sounds like a mix of the two. Once a majority of editors agreed to the guideline, it could be presented to @wbond to have him do a review and make a decision (accept, reject, refine).

3. Will this workflow actually reduce the time needed from @wbond to decide on guidelines? He previously expressed that he isn't opposed to such a change, but worries about whether it would increase the time he spends on these not so immediate tasks compared to core tasks, in which case it becomes counter-productive in a sense.

4. (future) Would this guideline repository be useful for other text editors that use TextMate-like syntax definitions with hierarchical scopes assigned to tokens? If yes, would they be interested in joining the committee or would they be fine with following Sublime Text's dictatorship regarding this? I do not know how they handle this currently or whether they have their own thing like this, to be honest.

   I do believe that we can focus on ourselves for now. Especially considering how our syntax definition format is more powerful than TextMate's.

5. What markup format will we use? It should be something that can be rendered by Github directly.

   I think Markdown (or Github Flavoured Markdown) is pretty sound, as it is simple but covers all the functionality we would need.

Next Steps

Once we figured out answers to the question, I would create a repo and either myself or someone else would begin working on a guideline based on current existing standards that we then use to go through the review and acceptance process as an example. We will use that to determine a guideline template and refine our workflow depending on difficulties we face.

Afterwards, the repo could be considered "open for everyone" and guidelines may be proposed in new pull requests or discussed in issues freely. The initial focus should be on standardizing the "status quo" of most top level scopes, such as string, variable, or storage, and to summarize the RFC issues here.

---

Other Ideas: Wiki

Another idea was to use the Packages repository's Github wiki, but that has several disadvantages in the review process because you cannot comment on commits, cannot create pull requests and cannot even easily view individual commits. Furthermore, it is either writable by everyone or only by collaborators. Being writable by everyone would hurt the collection's authority, and I don't think I need to explain why the latter is bad.

---

Please discuss and ask questions below.

Pinging @keith-hall, @Thom1729, @jfcherng, @r-stein, @gwenzek, @deathaxe, @rwols, @djspiewak because of their previous involvement with RFC issues.

[djspiewak]

I don't have much time to work on syntax things right now, but this proposal does sound very good to me. My main concern would be that it a) not create more work for @wbond, and b) ideally allow de novo scoping problems (such as language constructs with poor precedent) to be decided a little more quickly. At any rate, it seems like an improvement over the current process, which is somewhat ad hoc and buried amongst PR reviews.

No strong preferences or thoughts on naming or versioning. I would be amenable to any and all of your proposed alternatives.

As an aside on the editor standardization point, that's one of those which seems like a good idea at the onset, but I doubt is going to gain much traction. The way in which languages are scoped is a relatively core thing to the identity of an editor. While most people might not consciously think of it in that way, the reality is that it is, as much as anything else, part of the user experience design. Editors aren't going to want to give up that control, which pushes them to always have their own notion of scoping – however adjacent or nearly compatible it may be to Sublime's.

[r-stein]

What do you think about creating a github repo, which is the wiki of an other repo? (e.g. the packages repo) Hence everybody could create PRs, discuss, and contribute; and the changes could be pushed to the only collaborateurs wiki with every "release".

[gwenzek]

I also think we need a place to discuss syntax conventions.

But in the other hand, if nobody have done before it is mostly because nobody really care about the precise scopes.
Today the scopes are mostly used for syntax highlighting.
Persons that creates color schemes usually don't really have the time to read a convention file, and they mostly use just enough scope so it look good in their language. Also a lot of users would be pissed off if we change all the scopes and that it break their scheme.

I think if we want to make the guidelines useful and used we should also provide a few reasons to use them.
I'm thinking of:

• 1 or 2 templates for creating a color scheme that would look good on most language. Posting a color scheme to PC would require following this template.
• a solid set of default themes using the above templates.
• a list of popular themes that we don't want to break
• a convention for scoping build/test output so that we can have generic plugin to eg jump to the next error/aggregate the number of failed tests ...
• I have other ideas like scope based indentation, but that's more ambitious and probably requires integration in the main app to be efficient

For the process I think having the guidelines in a .md file edited through PR is enough.
This Repo should have a CONTRIBUTING.md linking to the Guidelines.
A SNP (Scope Naming Proposal) would be in the form of a PR (with a specific label) over this file.
It is a bit what I started doing in my Sublime Syntax Manifesto repo.

Regarding the time spend by wbond@ I think what we need is probably a metric eg. "number of occurences of a deprecated scope". Higher than some threshold we would need his approval, otherwise, we could just start translating it into PRs on the syntaxes.

[tajmone]

I've actually seen some attempts at creating repositories with documentation and examples for syntaxes and scopes. The problem is that they end up being isolated and detached from the ST community — ie, you'd really need to know about its existence and link to reach it.

If [sublimehq] were to dedicate either a Wiki on their project (see https://github.com/sublimehq/Packages/issues/1522), or a create a repository for that purpose (ie: having folders with documentation and examples, as well as a Wiki and maybe even a GitHub Pages website via Jekyll), then probably most people who created individual contributions could move them over there.

The idea of some semi-official (or at least, supported by ST dev team) central place for information, examples, and tests, would improve the current situation where useful info is spread all over the place (from ST forums to other online communities, or private repos, specific discussions on threads, and so on); but most of all it would allow to have info which is up to date.

When I find a forum thread that seems to answer my question, I often fear that it might be too old and behind the current Dev Build of ST. In a centralized documentation, open to collaboration, one could at least add a notice to any obsolete info (no need to wipe it away, but just add some note of why it no longer concenrs newer versions, etc.).

Just having the chance to contribute links to useful articles, organized by topics, would be a great resource in itself — and GitHub Wikis are a great tools for this.

[FichteFoll]

Just a quick note on this. The SublimeText organization is quite well-known (or at least a known figure) and has about 50 or so members, all of which would get push access to this proposed repo. They probably won't actively engage in the repo because they aren't particularly interested in it, but there are enough active people to manage a repo like the proposed one. For context, I've been managing things over there for about 5 years now.

[FichteFoll]

I created a ScopeNamingGuidelines repository at the SublimeText org and added a basic readme explaining its purpose. I also added a few issues for general questions regarding handling of the process and, most importantly, I made a pull request with some very general guidelins to experiment with the process. Please check it out at https://github.com/SublimeText/ScopeNamingGuidelines/pull/3.

Please also watch the repository if you want to stay updated.