Gatsby: [RFC] Implement an improved labeling system for Gatsby issues

Created on 1 Mar 2018 · 10Comments · Source: gatsbyjs/gatsby

With the rapid growth of Gatsby and the large number of issues in the repo, a more structured labeling system seems worth exploring to help contributors figure out, at a glance, a few key pieces of information about an issue:

The type or kind of an issue (e.g. bug, feature request, question)
The stage or status of an issue (e.g. blocked, in progress, needs clarification)

This is closely related to #4295 and will play a critical role in its success. If we get a sane labeling system in place, we should be able to make it easy to filter for

@m-allanson pointed out the serverless and stylelint label definitions, which are both great examples.

Borrowing liberally from their setup, I'd like to propose the following label structure for the Gatsby repo:

Type labels:
- type: bug — describes a bug in Gatsby
- type: question — asks a question about Gatsby
- type: documentation — improves or adds documentation
- type: tests — adds new tests or improves existing tests
- type: refactor — improves code quality or stability without affecting Gatby’s functionality
- type: feature — adds a new feature or capability to Gatsby
- type: rfc — proposes larger ideas that need discussion and will be broken down into subtasks, if accepted
  - @KyleAMathews are RFCs going to be split into a separate repo?
- type: design — creates or modifies the design of Gatsby assets (e.g. t-shirts, website)
Status labels:
- status: in development — issue is assigned to a contributor and under active development
- status: blocked — issue cannot move forward until another issue is resolved
- status: needs clarification — issue is missing key information (e.g. a reproducible test case)
- status: discussion — issue does not contain work items (e.g. an RFC)
- status: ready — issue is ready for development but has not been assigned or started
Target labels:
- target: core — the resulting code will be merged into the Gatsby core
- target: cli — the resulting code will be merged into the Gatsby CLI
- target: plugin — the resulting code will be merged into a Gatsby plugin
Roadmap labels:
- roadmap: v1 — will be merged into Gatsby v1
- roadmap: v2 — will be released with Gatsby v2
- roadmap: future — will be released in the future but is not currently part of a release
Miscellaneous labels:
- good first issue — an issue that is ideal for first-time contributors due to lower effort and complexity
- help wanted — synonymous with status: ready
  - should we drop one or the other of these labels?
- hacktoberfest — used during October to signify good issues for Hacktoberfest participants

Another idea, just for fun, could be to use emoji instead of the initial qualifier label (e.g. type:):

🏷 — type (e.g. 🏷 refactor)
📍 — status (e.g. 📍 ready
🎯 — target (e.g. 🎯 core)
🗺 — roadmap (e.g. 🗺 v1)

We'd leave emojis out of the help wanted and good first issue labels since GitHub looks for those as-is.

What does everyone think? Does this seem like a manageable number of labels and a reasonable organizational structure for them? Did I miss anything?

question or discussion

Source

jlengstorf

👍7

Most helpful comment

RFC number 1 is up at https://github.com/gatsbyjs/rfcs/pull/1

m-allanson on 21 Mar 2018

👍2

All 10 comments

Great stuff @jlengstorf!

I think key to making this work will be balancing simplicity against the amount of data we want to add via labels. It needs to be really quick and simple to decide which labels apply to an issue. But if the system is too simple then we may not be able to extract as much automation goodness out for e.g. issue #4295.

Maybe we can work out what info the roadmap should contain, and then work backwards to a minimum set of labels that gets us there? Alternatively, your list above looks good. I'd be tempted to drop the _Target_ category and maaaaybe the _Roadmap_ category...

And yeah let's try the emojis 🤖 Hopefully they'll make it easier to visually scan the issues list.

m-allanson on 1 Mar 2018

Good point about working backward from what we'd want to display on the roadmap.

If we use Serverless’s Scope instance as a starting point, I'd love to see the following columns:

Discussion | Backlog | In Progress | On Hold | Needs Review | Done
--- | --- | --- | --- | --- | ---
tracks open discussions & RFCs | all issues that are unblocked and unassigned | assigned, unblocked issues | issues that are blocked or awaiting clarification | open PRs | closed issues & PRs
Tags:
🏷 rfc, 🏷 question, 📍 discussion | Tags:
🏷 feature, 🏷 bug, 🏷 refactor, 🏷 documentation, 🏷 tests | Tags:
🏷 feature, 🏷 bug, 🏷 refactor, 🏷 documentation, 🏷 tests | Tags:
📍 blocked, 📍 needs clarification | – | –
_Not_ tagged with:
📍 blocked, 📍 needs clarification | _Not_ tagged with:
📍 blocked, 📍 needs clarification, 📍 discussion | _Not_ tagged with:
📍 blocked, 📍 needs clarification, 📍 discussion | – | – | –
– | Only unassigned | Only Assigned | – | – | –

Additionally, it would be great to highlight:

🏷 bug
help wanted
good first issue
hacktoberfest (only when Hacktoberfest is active, but I guess we'd just remove this tag unless the event is on)

I'm hesitant to drop the roadmap tags because I think it's really helpful when people ask things like, "When will Gatsby upgrade to [package]@[version]?" or "When will [feature] be available?" If we can open issues to track those targets and tag them with the appropriate roadmap release, it helps reduce noise.

Also, it would help contributors prioritize what to work on because they can filter down for a release. If something is tagged 🗺 future, there's less urgency and maybe a 🗺 v1 issue should take precedence.

I'm happy to drop the target group of labels — I added those because core/plugin labels are part of the repo's current label structure.

Filtering isn't supported in Scope (though it looks like it might be soon), but it'll still be really useful in a GitHub Project or for filtering the Issues board.

jlengstorf on 1 Mar 2018

Looking at this, I may have overcomplicated the tagging in that table above. We could probably control that whole table with _only_ status tags. 😕

jlengstorf on 1 Mar 2018

Maybe it's just me but I never use labels or look at labels here or on any other project on Github 🤷‍♂️

I can see "good issue for new contributors" type labels being useful but beyond that, they just seem like busy work. Hope that doesn't sound offensive :-P

Perhaps we could start with some user stories? What's the problems we're trying to solve.

There's other tools/techniques that could be considered.

KyleAMathews on 2 Mar 2018

@KyleAMathews sure, user stories seems like a good way to approach it.

To put a few out there:

As a developer who's new to Gatsby, but wants to get involved, I need an easy way to understand the current status of the project and identify issues that are ready for development but unclaimed.
As a development manager who's considering Gatsby for a production project, I need an at-a-glance way to determine the general health and direction of the project, e.g. what Gatsby is planning, how many bugs (vs. discussions, questions, etc.) there are.
As a junior developer who wants to get into open source, I need a way to quickly identify issues that are less complex and/or have a mentor available (this is a good label, actually: mentor available).
As a maintainer, I need a quick way to triage new issues and know whether or not other maintainers have already addressed the issue.
As a developer, I need a way to filter issues down to just the things I'm interested in (e.g. "What's being added to v1?" "What kind of bugs have been reported?" "Which issues have been reviewed and approved by the maintainers and are ready for development?")

Without reading through all the issues on a repo, it's really hard to get a sense of scope or context around _what's happening now_ without some kind of organizational system, in my opinion. Issue filters are one of my most used tools on GitHub.

I'd also make the argument that labeling — while I agree it takes extra time — is a strong signal that a repo is healthy, well-maintained, that the maintainers are on the same page. Otherwise, maintainers have to generate a lot of comment noise to acknowledge issues and communicate opinions about status. It may not have a lot of benefit for those of us who spend the majority of our working days in the issues, but for people who lack that context, I think it can make the difference between getting involved and getting overwhelmed.

(I'm not married to the idea of labeling, of course; if we can find another way of solving these problems that's faster for maintainers to manage, I'm happy to be convinced otherwise.)

Am I making sense?

jlengstorf on 2 Mar 2018

👍2

Thanks for the full response :-)

Here's a suggestion — I definitely see that labels (and organization in general) have value. I'm just very wary of premature optimizations/abstractions.

Perhaps we do this incrementally? Decide what weaknesses in the community development process are most pressing or what low-hanging fruit opportunities we have and then once we fleshed out the problem/opportunity, decide on the best solution for that.

Sounds like a great workshop topic to introduce at next week's maintainer summit :-D

KyleAMathews on 3 Mar 2018

👍2

Following up from the maintainer summit a week ago, there was some great discussion on this. Here's what I took away from the session:

The aim is to make it easier for new and existing contributors to work on Gatsby without adding too much admin overhead. The broad areas discussed were:

Type: labels like bug, discussion, feature request
Status: lables like needs repro, ready
Misc: labels like good first issue
Target (area of responsibility): Use GitHub projects
Roadmap: Use GitHub Milestones

At a minimum every new issue should have a _Type_ label applied to it, everything else is optional. Labelling can be fluid - a lot of issues might start with the label 'type: discussion', before being changed to a more specific _type_.

Taking that information along with the existing repo structure gives a setup something like this:

Type label

Every issue should have exactly one _Type_ label.

labels: bug, discussion, feature, maintenance, documentation

Status label

Optional

labels: needs repro, help wanted

Misc label

Optional

So far these are labels indicating that an issue is a well-defined piece of work or labels used by bots.

label: good first issue, hacktober, stale?, review

Projects

Optional

Assign an issue to one of the existing projects: docs, v2, UX (add projects as required)

Question: Do we need to create any additional projects?

Milestones

We talked about using GitHub's Milestones as a way of defining a roadmap however, we already have a v2 Project which contains more info than a Milestone could. In the interest of keeping this plan as simple as possible, I'm going to drop Milestones for now (unless anyone has strong objections). This could be looked at again once the labelling and projects is in place.

Existing labels

There are 20 existing labels. While we're adding new labels, it makes sense to revisit the existing ones. Having a small, clearly defined set of labels makes it much more likely that the labelling system will be used. Here's what I propose should happen to each of them:

| Label | Destiny | Notes |
|---|---|---|
| API/Plugins | ❌delete | too specific (currently applied to 66 issues) |
| bug | ✅keep | rename to type: bug or similar |
| Documentation | ❌delete | redundant - add all issues with this label to the 'Documentation' project before deleting the label |
| DX | ❌delete | too specific (currently applied to 13 issues) |
| Feature Request | ✅keep | rename to type: feature or similar |
| good first issue | ✅keep | exact name is used by GitHub, don't rename |
| GraphQL | ❌delete | too specific (currently applied to 13 issues) |
| Hacktoberfest | ✅keep | exact name is used by Hacktoberfest, don't rename |
| Help Wanted for Plugins | ❌delete | too specific (currently applied to 4 issues) |
| Help Wanted in Core | ✅keep | rename to help wanted or similar |
| needs-repro | ✅keep | rename to needs repro or similar |
| performance | ❌delete | too specific (currently applied to 4 issues) |
| question | ✅keep | rename to type: discussion or similar |
| review | ✅keep | Used by wafflebot? Investigate this |
| stale? | ✅keep | Used by stalebot to close stale issues. Maybe rename to bot: stale? |
| upstream-issue | ❌delete | I'm 50/50 on this one. There's only two issues using it so have opted to delete it |
| UX Design | ❌delete | redundant - add all issues with this label to the 'UX' project before deleting the label |
| v0 | ❌delete | it's never been used |
| v1 | ❌delete | We _could_ keep this but I don't think it adds much value (currently applied to 8 issues) |
| v2 | ❌delete | redundant - add all issues with this label to the 'v2' project before deleting the label (currently applied to 7 issues) |