Https-everywhere: Release 2017.10.24 forthcoming

[Hainish]

Type: other

cc @jsha @pde @J0WI @diracdeltas @jeremyn @gloomy-ghost

To be made Tuesday.

[Hainish]

This will be the last one before the full WebExtensions version.

[ghost]

@Hainish We have too many unreviewed PRs. They should be reviewed, but both you and @J0WI are constantly offline.

[Hainish]

@koops76 this is unspecific and unhelpful. Perhaps wait for review before opening more PRs. Our internal prioritization is of primary concern - not all the HTTPS Everywhere work happens on GitHub, and this means PRs sometimes will have to wait.

[ghost]

@Hainish Can you tell @I-have-50-open-PRs-to-HTTPS-EVERYWHERE to slow down a bit? The backlog is growing.

[Hainish]

@I-have-50-open-PRs-to-HTTPS-EVERYWHERE are you using an automated tool to generate these rulesets?

[ghost]

@koops76 @Hainish No, all my rulesets are manually crafted. Most of my last ones were re-activations from #12677 (which I will no longer maintain since @cschanaj made an automatic list simple-default-off-list.txt) who are easier to re-activate since they usually have only one problematic domain.

Also sorry if this was a problem for you guys I have no problem of holding off and keeping changes locally and submit them only when there's enough overhead.

[Bisaloo]

@Hainish, I ran hsts-prune for you (since you said it was time-consuming). Would you be okay with merging it before release?

https://github.com/EFForg/https-everywhere/pull/13319

[jeremyn]

I don't think we should tell volunteer contributors to hold off submitting PRs, especially if they are writing them by hand.

[J0WI]

11962 should be addressed before the full WebExtensions release.

[Hainish]

@jeremyn I also don't particularly have a problem with a PR backlog. It is inevitable to some degree or another in a project that requires maintaining a large dataset. Methods for managing that backlog are important, and we should continue to improve this process. But pointing fingers at volunteers and maintainers for taking time off is decidedly uncool.

[jeremyn]

@Hainish That's not exactly what I meant. A big PR backlog is bad for at least two reasons. One is that it wastes volunteer work that was submitted in good faith and was invited by the EFF (via CONTRIBUTING.md and the EFF website). As websites change and the rest of HTTPS Everywhere changes, PRs might have to be updated or thrown out entirely. The other reason is that a large backlog can discourage future contributors, including contributors who could eventually turn into maintainers and reduce the backlog.

What I disagree with is telling a contributor that they are welcome to create PRs but should keep them private until some arbitrary low number of open PRs in the main repository is reached. Instead we should be honest with them: tell them they are welcome to create PRs but there's a chance they won't ever get reviewed and their time will be wasted.

[Hainish]

@jeremyn there's obviously going to be some time between submission and review. When I mention managing the backlog, I don't mean simply sitting on PRs until they get stale. What I mean is creating a manageable workflow so that PRs are addressed, given the reviewers we have available at a given time. This management entails a combination of tools and effective use of volunteer time. I've tried to make this as painless as and streamlined a process as possible for ruleset maintainers.

Insisting that PRs are addressed as they are made isn't always possible, and we've never made any promises that they would be. If a contributor is insisting that all their PRs be addressed, they should be aware that this is only possible if they allow reviewers the time to look at existing PRs. Demanding their time by opening lots of PRs and demanding they are all addressed while pointing fingers when time is taken off is not conducive to project growth.

[Hainish]

These release issues were started as a nicety, allowing contributors to request specific issues or flag problems that may arise if they are not addressed pre-release. This shouldn't be a forum for venting on general workflow or placing unfair blame. Again, let's try to be respectful of each others time and effort in making this project what it is.

[ghost]

@Hainish I won't make any new PRs until the PR count will fall under 500 to not make things worse.

[ghost]

What I mean is creating a manageable workflow so that PRs are addressed, given the reviewers we have available at a given time.

Sorry to intrude on the conversation, but has anyone considered creating a separate repository for rulesets or am I the first?

[Bisaloo]

Sorry to intrude on the conversation, but has anyone considered creating a separate repository for rulesets or am I the first?

https://github.com/EFForg/https-everywhere/issues/2697

[bardiharborow]

Speaking from personal OSS experience (primarily over at twbs/bootstrap, where we had a delayed major version rewrite which took _1500+ days_, much to the community's distress), there's no good solution to lack of human capacity.

I doubt @koops76 meant their comment aggressively, but I understand that it's desirable to keep any criticism well away from those (volunteers!) who are doing the work, as every minute they spend thinking about this, 10 minutes of productive work is lost. I know what it's like to have 2k+ GitHub notifications, and I feel the pain. <3

I personally have been trying to start doing things like bulking many reactivations which are very similar into bulk PRs, to minimise context switch overhead. If there's further optimisation or structure that I can use, I'd appreciate ideas.

I still make frequent mistakes, and I'm glad that in such a security-critical project we have access to such highly experienced reviewers. Fixing capacity issues is not as simple as adding a ton of people to the organisation, because reviewers presumably need to be competent, and earn the project's trust first. If anyone reading this wants PRs merged faster, then they are going to need to be that person who puts in the work and becomes the next ruleset maintainer.

[ghost]

May I ask @J0WI and @gloomy-ghost to be a bit more active, at least until we will get the PR count down to reasonable numbers? As I said to not make things worse I won't create any new PRs.

[jeremyn]

@Hainish The EFF fully owns this project, since it makes all major decisions and not always publicly. So I would prefer to see both complaints and explanations framed in terms of EFF decision-making and prioritization, instead of in terms of what particular people are or aren't doing. Your earlier comment saying "internal prioritization is of primary concern" was harsh but appropriate. Your later comments (https://github.com/EFForg/https-everywhere/issues/13292#issuecomment-338790524, https://github.com/EFForg/https-everywhere/issues/13292#issuecomment-338840304) indirectly associating delays with volunteers "taking time off" was, I feel, less appropriate.

There have been two issues lately (https://github.com/EFForg/https-everywhere/issues/12448, https://github.com/EFForg/https-everywhere/issues/13022) requesting that the EFF "hire" more maintainers. I closed both as being answered by CONTRIBUTING.md, but in https://github.com/EFForg/https-everywhere/issues/13022#issuecomment-335309838 I wrote in part:

However a more interesting question is how the EFF sees this project, that is, what priority they attach to it. If they wanted more people involved, they could make that happen somehow. Since more people aren't getting involved, HTTPS Everywhere must not be a big priority for them. If someone wants to create an issue about that, that wouldn't be a duplicate.

I haven't made that issue, but I would be curious to see that discussion.

Also, regarding https://github.com/EFForg/https-everywhere/issues/13292#issuecomment-338841213 about release issues, I agree this is a derail. However if @koops76 had made a new issue from the start, I think this would be a valid discussion to have.

[jeremyn]

@koops76 That is not okay to ask. If you want to ask anything, ask the EFF to find more maintainers and/or improve the ruleset creation/review process.

[ghost]

13331 is related to cleaning up the backlog.

[Hainish]

@jeremyn

Your later comments (#13292 (comment), #13292 (comment)) indirectly associating delays with volunteers "taking time off" was, I feel, less appropriate.

I was actually referring to my own time. I don't think anyone should feel pressured to work on this project as volunteers. The bottom line is that your time is your own, and any volunteer hours put into this project are certainly appreciated, but there is no minimum time requirement. Nor should there be - volunteers are here of their own accord because they want to help the project. Asking for more of their time seems to me, to use your words, inappropriate. Everyone needs time off to maintain their own health, and no one should have judgement for that. That's why when I see finger pointing for taking time off, I'm going to call it out - _whether it's true or not_ - because every moment volunteers give is actually a gift to the project.

I agree with your assessment that we need more cycles from EFF, and we're in the process of building capacity here. But this is not an immediate thing, and onboarding takes time. So please bear with me and take my words at face value.

[Hainish]

Release 2017.10.24 has been made.

[jeremyn]

@Hainish I don't entirely understand the first paragraph of your response, because you start by saying you were responding to accusations against yourself but then spend the rest of it talking about volunteers. And one of your "taking time off" comments (https://github.com/EFForg/https-everywhere/issues/13292#issuecomment-3387905240: "But pointing fingers at volunteers and maintainers for taking time off is decidedly uncool.") was unambiguously about volunteers, not (just) you.

Anyway I'm okay with leaving aside this semantic argument, but let me ask this before we leave this issue, if that's all right: if you're a contributor with over a hundred open PRs and are frustrated by lack of response, is there any effective way to complain about that? Or is the only answer to just not make that many PRs?