Go-ipfs: Improving how we work - reflection

[eingenito]

Context

We don't currently have a routine place where go-ipfs contributors can reflect on the challenges of working on the project and iterate on improvements. This team doesn't get much time together synchronously, so I want to find a constructive async way to do so. This team is really good at commenting on issues and PRs, thus....

Proposal

Let's use this issue to enumerate our challenges and suggest (and agree to try out) solutions.

• For a bounded period (weekish) let's reflect on how we can improve the go-ipfs project. It can be challenges (assuming any exist) with or without proposed solutions, or things we currently do and want to do even more of. Consider starting comments with "I wish..." or "I like...."
• Then we'll vote for the challenges we think are most important to address first - maybe because they're the most acute, or maybe because they're the ones that don't seem intractable. Up to the voter. We may group similar ideas to avoid vote splitting. You get three votes.
• Then I'll create new issue/PR summarizing the most popular challenges and solutions. The team will have an opportunity to refine the solution part, I'll restate the solution and after a final call for dissenters we'll agree to try our solution out. Keep in mind these might be as un-momentous as say 'Make a PR template for go-ipfs with these fields ...'. We don't have to solve everything at once.
• Then we'll do it again, and this time we'll also reflect on how our changes have gone.

Guidelines

• We agree not to create another issue discussing the process defined by this one. This is the top issue.
• We don't have to be super ambitious; I always like simple changes to repeated behaviors. This project isn't going away, so we can feel free to defer tough questions if we don't have good answers.
• This is a public forum, and people with zero context can read this issue, so just frame things constructively. We don't want to scare away passers by.
• One 'I wish/like' per comment so we can easily vote on them afterwards.

Notes

Obviously David already interviewed the team and produced a list of challenges and possible solutions in the Golang Core Dev Team Update, August 2018. Those are all fair game. I'll add them if they're not added by someone else - but they're all pretty big, and these don't have to be that big.

[michaelavila]

I like that we are introducing a process where the team drives process changes!

[Stebalien]

I wish there were some way to flag ourselves as busy/available. Going to IRC and just shouting into the ether hoping someone's listening can be a bit isolating.

I believe other OSS teams solve this by having an intimate chat where people sign on/off (with good mornings, be back in 10, stuff like that). However, that can be very distracting (and works best on tiny teams in private channels).

IRC has an /away command but many of us have switched to matrix so that won't work so well.

[eingenito]

I overlap @Stebalien's wish. I wish there were a more chatty place where just go-ipfs development was discussed, both the social side and more day to day development discussion and coordination. The ipfs-dev channel is positively lousy with javascript, and it's all business. I don't think it requires a private channel, but something more conversational, and with a smaller set of interested parties.

[hannahhoward]

I like that everyone on the team is so friendly and committed to the project

[hannahhoward]

I like that people leave detailed PR comments that encourage better technical solutions

[hannahhoward]

I like it when we use a meta-issue to track a series of tasks together

[hannahhoward]

I wish we could schedule a face-to-face over video to resolve some disputes that happen in PRs.

I would like it if we had a culture of being able to say "hey, can we jump on a call and work this out together real quickly?"

[hannahhoward]

I wish we had clear standard for when a PR is ready to merge. Usually, I use 2 LGTMs + build passing.

The main project is different because only certain people can merge. But the merge gating is mostly to keep the merging in order and consistent. I would like it if we could apply the same standard there, with the additional step of Steven needing to review and merge it.

Instead of just "someone reviews before Steven" I'd like to get to "Steven just double checks and merges, and only on the main project"

[hannahhoward]

I wish it were clear if we should post issues for a sub package in the main repository or in the sub package.

[kevina]

I wish we could schedule a face-to-face over video to resolve some disputes that happen in PRs.

I would like it if we had a culture of being able to say "hey, can we jump on a call and work this out together real quickly?"

Although some of this is helpful. There are downsides (1) Synchronize time is expensive (2) by doing things over a P.R. or even IRC we have a written record of the conversation, if we get in the habit of resolving too many disputes via face-to-face we run the risk of tribal knowledge that makes it hard for outsiders to jump in and contribute. We already have a bit too much of that in my option.

[kevina]

I wish we had clear standard for when a PR is ready to merge. Usually, I use 2 LGTMs + build passing.

Who the LGTM is from makes a difference. I actually think we should have one or two experts on each part of system and we need to get a LGTM from them before merging. Of course part of that contract is that they review P.R. quickly.

[kevina]

I wish it were clear if we should post issues for a sub package in the main repository or in the sub package.

Yeah. I am not so happy about us splitting go-ipfs in so many tiny peaces. In addition to not knowing where to post issues it makes keeping up with all that is going on rather difficult.

I don't have an easy answer to that question. In some ways posting to go-ipfs is better as it will get more attention as more people follow go-ipfs then the sub-packages, but then again it does notify more people and posting to the sub-package may get the attention of the people most knowledgeable of the issue.

[warpfork]

I wish we used github issues less for discussion. Issues that have no clear road to being closed create a lot of distraction for me, and since issues that don't close will accumulate in number over time, it makes issues overall drastically less manageable and less usable as time passes.

Issues cannot easily be refactored, merged or split, etc; and especially, they cannot easily accumulate into summary -- in other words, as the number of comments in an issue goes up, it becomes linearly harder to make a constructive addition to it, because you have to read and re-process every existing comment first. If discussions go on long enough that any of these concepts come up, it means the discussion is moving beyond what issues are appropriate to track and contain.

I don't have a particular action in mind for what to do about this, but I think we should find one. Making it cultural norm to eventually roll up long issues into "experience reports" or "design docs", which gives us a way to turn the information into files we can keep in a repo, is one possibility that might help.

---

This issue is a fantastic example of doing it right, incidentally!

For a bounded period (weekish) let's reflect

:heart: ! A bounded period of discussion is great; issues do work for this.

[hannahhoward]

Three reflections on the issue of accessiblity:

I wish every repo has a really solid readme or a docs folder that provided a general guide to the code for getting up and running.

I wish that we did regular reviews of code for readability and comprehensibility -- sometimes what we're working on is fundamentally complex, but other times we just need to take the time to refactor, use smaller functions, longer variable names, etc.

I wish we marked issues as "good first time issue" for first time contributors

[eingenito]

I wish we used github issues less for discussion. Issues that have no clear road to being closed create a lot of distraction for me, and since issues that don't close will accumulate in number over time, it makes issues overall drastically less manageable and less usable as time passes.

@warpfork I agree with everything you said. This issue is only an issue because I couldn't think of a better place to do it that people would actually go to. We pretty much default to github for everything and there's a barrier to going outside it.

I would love to hear if people have suggestions about a better tool for technical/process discussions.

[warpfork]

@eingenito I might've ninja-edit'd the end of my comment there in such that you didn't see it, but fwiw, I actually think this issue is AOK! Declaring it time-boxed means there's a clear path to closure, and that's :+1: :+1: :+1:

[djdv]

@Stebalien

I wish there were some way to flag ourselves as busy/available. Going to IRC and just shouting into the ether hoping someone's listening can be a bit isolating.
I believe other OSS teams solve this by having an intimate chat where people sign on/off (with good mornings, be back in 10, stuff like that).

I've been contemplating this as well.
In my own experience, I've used systems like Teamspeak and Mumble for remote collaboration, and feel as though there's some patterns innate to them that resulted in better cooperation compared to other platforms/systems.
I'm a fan of passive rooms with a "check-in/check-out" style over transient sessions (à la Zoom & Slack).

While I won't delve deeper on this here, I do think how we communicate on a regular basis, is an important thing to focus on, and would like to discuss this at some point somewhere.
To me subtle changes in our communication patterns and tools could have a big impact.

@kevina

if we get in the habit of resolving too many disputes via face-to-face we run the risk of tribal knowledge that makes it hard for outsiders to jump in and contribute. We already have a bit too much of that in my option.

Agreed. However, I feel like this is more of a result from the lack of note taking.

If a meeting is worth having, it's likely that a conclusion summery, video recording, graphical presentation, problem-solution list, PR, or whatever, should probably come out of said meeting, instead of restricting knowledge to the minds and personal notes of the attendees.
Maybe it's worth discussing this in more depth. (and producing something as a result ;^)

[magik6k]

I've been contemplating this as well.
In my own experience, I've used systems like Teamspeak and Mumble for remote collaboration, and feel as though there's some patterns innate to them that resulted in better cooperation compared to other platforms/systems.
I'm a fan of passive rooms with a "check-in/check-out" style over transient sessions (à la Zoom & Slack).

:+1:, IMO this is definitely worth trying at some point. Some things to consider:

• We should probably make this open to everyone
  
  
  
  • I don't know about Mumble, but Teamspeak allows for private / hidden / encrypted channels with very granular permission management
  
  
• TeamSpeak is closed-source, some people may not be fully ok with this, it also has a long history of serious vulns, including client side RCEs
  
  
  
  • We can make this accessible to closer contributors only (not open)
  
  
  • Not sure how it security looks on the Mumble, but it's at least open soure
  
  
• This is almost guaranteed to make the tribal knowledge problem worse unless we are vary careful to note everything that is not directly related to managing workflow
• We should probably move this to a separate issue

[eingenito]

I really like Steven's making time for people to chat with him directly via open hours or Calendly. We should take him up on it.

[eingenito]

I wish we had a place where we could easily track progress on our highest priority initiatives. I guess the meta issues are that.

[eingenito]

I wish the barrier to contributing use-case specific features weren't so high. I don't know what to say to people who have a specific need. It seems unlikely to me that these features would ever get merged because they can easily complicate the existing code.

[eingenito]

I wish I understood which issues are the ones that should be worked on. I don't feel qualified to apply priority labels to most incoming issues.

[eingenito]

I (selfishly) wish that we had a more collaborative development process. I'm not used to being on a team that defaults to communication in github so I don't have a lot of insight into how to improve efficiency (apart from being more collaborative which is like a tautology). I miss synchronous story writing and backlog grooming meetings which are important parts of of how I'm used to learning.

[eingenito]

I wish that there were an established process for discussing work to be done prior to PRs. It seems that, particularly for relatively new team members, it would be helpful to have a few discussions along the way with the most experienced members of the team. Steven's office hours and availability on Calendly could be the solution here (or maybe not just Steven, but the same practice from the team's most senior devs).

[eingenito]

I wish I knew what to do with all the old issues in go-ipfs. They're basically ancient, but many of them are still really good issues. They really slow down waffleboards I think.

[eingenito]

I wish that I knew what to tell people who had issues that are unlikely to ever get worked on. I guess I should just say they're going to have to go after this one themselves.

[kevina]

I (selfishly) wish that we had a more collaborative development process. I'm not used to being on a team that defaults to communication in github so I don't have a lot of insight into how to improve efficiency (apart from being more collaborative which is like a tautology). I miss synchronous story writing and backlog grooming meetings which are important parts of of how I'm used to learning.

I am almost the opposite. I don't really like in-person meetings via zoom but will reluctantly agree to join them when required. I prefer IRC first when more imminent feedback is helpful and then when that fails a video call.

I am also not sure of the value of having a video call all the time and would sometimes prefer the option of a simple phone call or audio only zoom call.

And again, doing things over IRC provides an automatic transcript that we can copy and paste into an issue, of which I often do.

[kevina]

I don't particularly like Slack and am generally not logged on most of the time, however every now and then I get pinged via slack instead of IRC so it seams like there is an implicit requirement now that you stay on slack now so people can reach you.

[eingenito]

I am almost the opposite. I don't really like in-person meetings via zoom but will reluctantly agree to join them when required. I prefer IRC first when more imminent feedback is helpful and then when that fails a video call.

I am also not sure of the value of having a video call all the time and would sometimes prefer the option of a simple phone call or audio only zoom call.

And again, doing things over IRC provides an automatic transcript that we can copy and paste into an issue, of which I often do.

Yeah - I realize that many (all?) people on this team don't want to work in the way that I'm used to. It's important for me to hear ways that we can improve things without me just automatically going to More Collaboration because this is a new way of working for me.

[DonaldTsang]

In my own experience, I've used systems like Teamspeak and Mumble for remote collaboration, and feel as though there's some patterns innate to them that resulted in better cooperation compared to other platforms/systems.
I'm a fan of passive rooms with a "check-in/check-out" style over transient sessions (à la Zoom & Slack).

I would say that the best vehicle has to be Mumble if you want things to Open Source, or maybe a public Matrix room with Jitsi.

• TeamSpeak is closed-source, some people may not be fully ok with this, it also has a long history of serious vulns, including client side RCEs

  • We can make this accessible to closer contributors only (not open)
  • Not sure how it security looks on the Mumble, but it's at least open soure
• This is almost guaranteed to make the tribal knowledge problem worse unless we are vary careful to note everything that is not directly related to managing workflow

If you want a wider coverage, and gain traction to bigger, younger, more general crowds, Discord would be one of the best solutions if non-FOSS services are allowed

I wish the barrier to contributing use-case specific features weren't so high. I don't know what to say to people who have a specific need. It seems unlikely to me that these features would ever get merged because they can easily complicate the existing code.

In regards to fixing issues, the best way to approach it is the Kanban or post-it note method, which is the "Project Board" in Github, people can rally and organize in an organic fashion such that nature can take its course. Different related issues can be organized by category.

[DonaldTsang]

Here are a few suggestions:

• Break IPFS down into its basic components, e.g. API, multihash, file management, DHT, and delegate, elect, or volunteer at least 2 people to manage each component
• Establish the baselines of each component and make the current to-do or issue list very explicit either by Kanban or by other means, and sort through duplicates
• Promote each component by encouraging code sprints, collaborative events, or if allowed, have voluntary non-profit driven "bug bounties", to speed up the coding process
• Have a basic standard in user manuals and design documentations, and in how people write issues in regards to editing the former two docs i.e. issue formatting
• If possible, allow a code freeze or code audit every winter and summer holiday (or any suitable holiday that lasts two weeks), and have periodic checks to make sure nothing goes off the rails
• If possible, bring in people from the awesome-ipfs repos that are proactive (those who receives weekly updates) to hightlight important issues for IPFS to prioritize

[eingenito]

Thanks everyone: summarized for voting here: https://github.com/ipfs/go-ipfs/issues/5819