Go: proposal: dep: move out of github.com/golang

[theckman]

Hi there,

Over the past few months the signals we've gotten is that The Core Go team has no interest in continuing the work of dep, or to even contribute code to it to help make the modules transition easier. After the tweet from @rsc apologizing for how the dependency management story wasn't handled well[1], I'm not aware of active work from the core team to help improve the current situation which isn't a confidence booster.

As a result, I'd like to ask that the dep source and licenses be transferred out from Google/Golang to the Gofrs GitHub organization so that, at a minimum, we can try to make dep work with projects that have modules enabled / use the semantic import versioning. We'd like to retain the current issue and PR history, while having full administrative control over the project and repository.

We may also want to pick up additional work to continuing to support dep long-term, as there is a deep philosophical divide between those who want a dep-like experience and those who are happy with modules. I don't believe these disagreements can be resolved in a single solution or implementation, and we should instead work together to ensure we can interoperate.

While I'd love to be able to have modules work for me the way I want, I don't think that's a reasonable goal. I now think it's more realistic to support dep in the interest in strengthening the Go community and the tools that are used to support it. I also think competition in dependency management tooling is as important as having competing businesses to help drive progress, and so I believe adopting dep is one step we can take towards that.

Who would be the best person to talk to get the ball rolling?

Cheers!
-Tim

[1] https://twitter.com/_rsc/status/1022588240501661696

Edit: I had someone reach out to me privately to clarify what I meant by the following line:

...I'm not aware of active work from the core team to help improve the current situation which isn't a confidence booster.

The current situation I was citing here was the gap in interoperability between modules and dep, which can result in people being in a less-than-ideal situation if their dependencies have adopted modules before they are comfortable moving.

[theckman]

For some additional context, I tweeted about this today and Sam Boyer replied:

it would certainly make sense to move dep out, given that we wrote the core of it elsewhere and then relicensed it to the Go team in good faith

• https://twitter.com/sdboyer/status/1081887613341130753

[theckman]

@adg @bradfitz @rsc I see that you were all tagged in earlier efforts to move code in here; so being the opposite I figure you may have some insight on this process.

Who would be the right people to talk to about having this project donated to the Go Community?

[ianlancetaylor]

CC @spf13

[bradfitz]

Assuming @sdboyer et al want this (please confirm), we can discuss at our next proposal review meeting. We're a bit behind due to the holiday but we'll catch up in the next couple weeks.

[carolynvs]

:shipit:

[rsc]

As a result, I'd like to ask that the dep source and licenses be transferred out from Google/Golang to the Gofrs GitHub organization

I don't understand what you mean by this.

The source code copyright is held by "The Go Authors", which is the group of relevant contributors. Google does not hold the copyright and therefore cannot transfer it to anyone else.

The license is BSD. It already grants permission for anyone to make a copy, make changes, and distribute those changes, with the usual notice requirements. Google is not going to change the license on this code any more than we are going to change the license on the Go project itself.

If you want to start a fork of dep, the "source and licenses" are not stopping you.

We'd like to retain the current issue and PR history, while having full administrative control over the project and repository.

This is different. Here you are asking that the GitHub project itself be transferred. That's something we could do (while leaving the copyright and licenses alone), but I am not even sure that this is appropriate. Dep both helped us learn what package management in the go command might look like and also still serves as a useful tool until modules are completely ready. I don't see why that wouldn't stay in github.com/golang. And once modules are ready, dep should go away. If others want to fork it and maintain a fork then that's fine. But the official dep that the Go project recommends should probably be the one in our GitHub org. If more people need commit access to github.com/golang/dep we can clearly do that.

I hadn't seen the gofrs org until today. I am skeptical of that org because it purports to be (according to the headline at github.com/gofrs) "The Go Community". The only crisp definition of "The Go Community" that I've ever seen is "all Go programmers". Clearly the gofrs github org is a small, self-selected set of developers, not the entire Go community. A small group of 17 people literally claiming to be (or even to represent) the entire Go community is for me a red flag. Why should we trust gofrs to take over dep?

[rsc]

Sorry, I clicked the wrong "send" button; meant to leave this open for further discussion.

[theckman]

@ianlancetaylor @carolynvs @bradfitz Thank you so much for your :eyes: on this, and I appreciate you being willing to discuss it when getting back to things after the holidays.

[theckman]

@rsc Thank you for taking time to look in to this and reply. There are quite a few points you've raised, so hopefully I can address them individually. I'm sorry that this ended up becoming a bit more long-winded than I had originally intended, but I think sharing context transparently is the best I can do to help alleviate any concerns that you or others have.

I hadn't seen the gofrs org until today. I am skeptical of that org because it purports to be (according to the headline at github.com/gofrs) "The Go Community". The only crisp definition of "The Go Community" that I've ever seen is "all Go programmers".

It's probably worth starting with some context sharing on the Gofrs, since I think that might be the more important piece here. It's unfortunate you weren't able to make it to GopherCon this year; I'd have tried to get you to come to my lightning talk[1] where I broke ground on the idea. That said, not knowing me from Adam it might have been a hard sell to get you to attend. 😄

The Gofrs initiative originally came out of the general channel on the Slack Workspace (where we now have over 35.3k registered users). While it started there, it's an effort we're hoping to keep open to the larger Go Community no matter how you interact with it. I do volunteer on Slack as an admin, so I'm generally keeping an eye on things throughout the day for things to improve. Whether in how the community is running, or for feature requests to Go itself. In doing so I noticed two patterns emerging that made me feel like there was a gap to be filled:

• Because our ecosystem is a bit newer than other languages out there, there are often a few options for solving the same problem without a clear choice of which to use. This was especially challenging to those newer to Go or software development who don't have as much experience measuring the quality of a project
• Individuals who invested their time in building libraries that had become widely used in the Go ecosystem had no explicit support from the community, and there were a few cases where those authors vanished with outstanding critical bugs.

One of the biggest things that continually comes up and sticks with me is the number of people who comment that our community is one of the most supportive and helpful they've ever interacted with, especially when coming in with a Go problem and looking for some insight. It brings me lots of pride and joy to know that we were able to build that community together, and that everyone (even non-admins) is biased towards making sure it remains welcoming.

Knowing that made me wonder if there was more we could do as a community to support those who are investing their time in making our ecosystem attractive to so many developers, and could we also build a more clear set of community-maintained "trusted" packages that we can suggest to newcomers? That is basically the foundational origin of The Gofrs. If there's more I can share I'd be happy to do it, but I'm not sure this issue would be the best medium for that.

Clearly the gofrs github org is a small, self-selected set of developers, not the entire Go community. Why should we trust gofrs to take over dep?

You're totally right that we're self-selected, and we see it as being similar to how these community spaces (Slack, the forum, etc.) and GopherCon itself were started. We feel they've ended up being effective because of the passion of those who are involved, and we hope to use that same kind of passion in The Gofrs.

A small group of 17 people literally claiming to be (or even to represent) the entire Go community is for me a red flag.

Specifically around the number of users in the Gofrs organization, most have chosen not to publicize their membership so we're actually above 40 members. I see this only growing as we put in more time in supporting the community, and our needs become more diverse. That said I don't think our size should be the judge of how effective we can be, but rather the passion we have for the Go programming language and the community that's formed around it. We really care about making Go, and those who use it and build libraries in it, successful.

Why should we trust gofrs to take over dep?

This is a great question.

Trust is something that needs to be built, and we feel that our work on the gofrs/uuid package has so far started to build trust in the Go community. We can only continue to build trust by delivering on our goals to support the community, and we feel alleviating some of the pains today around dep is important. We also value how important it is not to destroy trust, because it's harder to bounce back from that than it was to build the trust in the first place.

Not only that, longer-term we're hoping to use the projects we manage as an opportunity for more junior developers to contribute and pair with more senior Gophers and to learn from that experience. While we're not set up for this yet, it's honestly something I wish I could spend more of my community time doing (pair programming).

This is different. Here you are asking that the GitHub project itself be transferred. That's something we could do (while leaving the copyright and licenses alone), but I am not even sure that this is appropriate.

I think it could be an agreeable solution to simply have the project moved, the license and copyrights for current code retained, and future contributions kept under the same licenses albeit copyrighted to a different organization. Retaining the history is absolutely the most important part for us.

I believe I crossed the streams in my head with the Google CLA and another, which is why I mentioned the copyright change. Apologies on that one.

Dep both helped us learn what package management in the go command might look like and also still serves as a useful tool until modules are completely ready.

In its current form the latter is becoming less true, and is what prompted this issue being raised. If you are a dep user, and one of your dependencies moves to modules and changes their import statements to use SIV, dep is not going to appreciate that since it cannot find the /vN/ folder in the repo.

Things like this are starting to become more common as adoption grows, and so we're hoping to address these sharp edges sooner rather than later.

I don't see why that wouldn't stay in github.com/golang. And once modules are ready, dep should go away. If others want to fork it and maintain a fork then that's fine. But the official dep that the Go project recommends should probably be the one in our GitHub org. If more people need commit access to github.com/golang/dep we can clearly do that.

Our goal here is to provide the best support/experience to the Go community and we're worried that having it alongside the official Go repositories could cause unnecessary confusion, especially if we intend to continue development of dep after modules are deemed "ready". When we hear things like "dep should go away", it makes it unclear what would happen to our work at that moment of "ready". It's the uncertainty that makes us uncomfortable, and we feel being in control of the project would help alleviate those concerns.

This isn't an ask to take our cookies and run away, but rather giving us the freedom and responsibility to support the community and to provide a solution to those who have needs outside of the scope of modules. Maybe by the time modules are ready we won't see a need, but it feels like a disservice to let it languish until then.

Sorry, I clicked the wrong "send" button; meant to leave this open for further discussion.

Heh. I do this all the time, I really wish they'd make it harder to do.

[1] https://www.youtube.com/watch?v=BB09FyaQN5I

Edit: I am traveling in Japan to see family; so with timezones and what-not I may be a bit delayed on responding. I'll definitely do my best to get back to any questions/concerns as soon as I can.

[therealplato]

I have no current interest in modules, ~they do not solve any problem that I currently encounter~ I have not encountered a problem that requires them. Dep works great and I have no reason to believe it will stop working. I would appreciate dep being placed in custodianship of those who care about continuing to use it.

[rsc]

We transferred this to the main golang/go repo to be with the other proposals (and to see how well "Transfer Issue" worked).

[rsc]

We talked about this at the proposal meeting today. We have concerns about the history of all the issues and repo completely disappearing from github.com/golang. Those events happened in that org, and we would regret the history just being gone. On the other hand we do support moving the responsibility for the code to people who want to maintain it well. What would be ideal is if GitHub would let us _both_ (1) mark this project archived, so that we retain a golang copy of all the issues and other project data; and (2) send a copy of all the issues etc over to gofrs/dep. I'm also concerned about old "go get" (which is still how people get dep) getting confused by the automatic redirect implied by a transfer. It has never handled those well.

I don't believe GitHub can do exactly what we want. So probably the best thing would be for you to fork it and for us to post a prominent note here that development has moved to gofrs/dep. We can even mark the project archived. Then people can see that and move over to the new place. The issue history won't transfer, but otherwise you'll be off and running. Does that work for you, @theckman?

If we can get an explicit "Yes, let's do that." from @theckman, @sdboyer, and @carolynvs, then we'll do it.

Thanks.

[FiloSottile]

I think it could be an agreeable solution to simply have the project moved, the license and copyrights for current code retained, and future contributions kept under the same licenses albeit copyrighted to a different organization.

(Note that contributions to the Go project are not copyrighted by a single organization, the CLA is not a copyright assignment, and each contributor retains copyright over their contributions. You are free to use a different arrangement in a fork of course, but I wanted to make this clear.)

[adamdecaf]

So probably the best thing would be for you to fork it and for us to post a prominent note here that development has moved to gofrs/dep. We can even mark the project archived. Then people can see that and move over to the new place.

That won't help all the blog posts, projects and scripts people use to download "dep". All of those would need to be updated if golang/dep doesn't redirect to gofrs/dep.

See:

• https://www.google.com/search?q=%22go+get+github.com%2Fgolang%2Fdep%22
• https://github.com/search?q=%22go+get+github.com%2Fgolang%2Fdep%22&type=Code

[eleniums]

That won't help all the blog posts, projects and scripts people use to download "dep". All of those would need to be updated if golang/dep doesn't redirect to gofrs/dep.

I'm fine with the forking solution if golang/dep is updated with a prominent notice that it has been deprecated in favor of the forked gofrs/dep. I don't mind updating my scripts to point to gofrs/dep.

[neetle]

If we're open to suggestions, adding some logging letting the user know that the project is now over at gofrs could help in the transitionary period.

[mvdan]

That won't help all the blog posts, projects and scripts people use to download "dep". All of those would need to be updated if golang/dep doesn't redirect to gofrs/dep.

I'd argue the opposite. If someone does go get github.com/golang/dep, they very likely want to get the official dep tool that the Go project maintains or maintained. That breaks if the URL redirects to a git repository owned by a small set of Go developers. Now that we have modules, these blog posts would need to be updated with context, anyway.

I share Russ's reservation about the Gofrs organisation. There have been other orgs in the past like pkg, but none tried to represent the Go community, which already has its own organisation on GitHub.

Confusingly enough, googling "golang community github" and similar keywords consistently shows the gofrs org before golang on my system, presumably because the gofrs one uses the appropriate keywords to be more visible.

[carolynvs]

My preference is to do whatever it takes to keep dep alive and maintained for those who want to use it. So if @theckman wants to maintain it over at Gofrs, then that's great!

I disagree that keeping the repo and issues archived under golang/dep helps people who still want to use dep going forward. Having it under golang has always confused people (even its maintainers) about if it was an official Go project, worked on by the Go team anyway. 😀

I am not familiar with the known problems of continuing to use go get with the old package name after the repository has been transferred. From my own experience, it has worked fine.

If y'all ultimately decide that transferring won't work, @kris-nova helped the dep project out during the gps move by using a bot to copy the issues between repos. So we could use the same process to preserve the old issues. It backlinked to the original issues, did a few other helpful things.

[rsc]

FWIW I am still concerned about the fact that github.com/gofrs claims to be "The Go Community", as I noted above. Now that I think more about it, I would like to see that overbroad claim removed before we transfer dep.

[peterbourgon]

Are there concrete, technical reasons that keeping dep where it is would make ongoing maintenance more difficult? If not, I am against moving it for principally optical or philosophical reasons.

[DamareYoh]

Does the fact nobody has fixed deps incompatibility with modules even though someone has been trying to work on a PR count as a technical reason?

[Xe]

I think that is a chicken and egg problem. If you give the gofrs time to try, they might end up with something that is compatible with modules. But you need to give them time to try.

[peterbourgon]

As far as I can tell, there is nothing stopping the people in gofrs from contributing to dep as it stands today.

[theckman]

One open technical question would be around how we add new users to contribute to the repo. It's been awhile since I looked at GitHub groups and permissions, but would we have the power to add and remove people with write access? Would we have the level of access needed to make Settings changes to the repo?

There are other organizational questions about what level of control the language team would try to have on a repo under the golang organization. Would there be a commitment to ensuring we will always have commit access to the repository we are maintaining? When modules are "done" are we going to be asked to stop working in the main repo and then incur a tax on any user who wishes to continue using dep? How about any confusion resulting from development continuing after modules are "done"?

[theckman]

@rsc We can likely commit to a 3 character name change. We could maybe keep it here, especially if we can get clarification and commitments on the above points.

Edit: To clarify, I haven't spoken to everyone about it yet.

[niaow]

After some discussion and brainstoriming, the name The Go Commune was chosen to replace The Go Community as the display name of gofrs.

In addition, a corresponding gofrs-side issue has been created to track gofrs-side setup of dep, should we proceed: https://github.com/gofrs/help-requests/issues/33

[myitcv]

We are the Gofrs: a community-driven effort to provide maintainers for valuable projects

Small nit: isn't the phrase "community-driven" superfluous? Because by definition the Go community is anyone who uses Go.

, as well as for building projects that benefit the larger Go community

Another small nit, but isn't this the definition of open-source?

The only reason for picking up on these two points is that some folks often use the term "community" to mean "everyone who uses Go outside of Google" or "not the core team".

For me, everyone who uses Go is part of that community.

Alternatively:

gofrs is an effort to provide maintainers for valuable projects, as well as for building new projects. We very much welcome newcomers to the language looking for projects to work on

[Merovius]

While I am also uncomfortable with some of the messaging and self-perception of the gofrs organization, I personally think that's an independent issue from this. I don't have any personal stakes here and probably am making some wrong and biased assumptions, so take this with a grain of salt. But it seems counterproductive to try to dictate the self-presentation of a group who clearly views themselves as independent and takes special pride in that. In particular, when the resolution of a technical problem is coupled to it¹. IMHO this should be resolved by either transferring or copying the repo (don't have any stakes in that, so don't have an opinion on which) and having the discussion around their self-identification in a gofrs-specific forum like their issue tracker, the slack, some ML or wherever else they usually discuss things like these.

Just… TBH, the dep vs. modules history is tense enough as it is. I'd just like it to settle down and if transferring control over the repository is helping with that, I'd say go for it :)

[1] an inflammatory way to phrase it would be "dep is being held hostage". Not saying this because I think it's correct, but to illustrate the communication-pitfalls I'm seeing here.

[flibustenet]

It can be very confusing to have two dep, i believe it's more simple (and we like that !) to simply fork and rename the project. It'll also give you more freedom.
There was enough confusion with the history of dep, please don't add one more and turn the page !

[myitcv]

to try to dictate the self-presentation of a group

Just to clarify, this is not what I'm seeking to do. Rather, if the official experiment that is dep is to be officially handed over to a group in some way, shape or form, then it seems prudent to determine whether the proposed new location is, for want of a better word, appropriate.

All that said, it's entirely within the gift/power of the gofrs team to either work on dep in its current location (https://github.com/golang/go/issues/29639#issuecomment-453169100), or fork dep and do as they wish. I really don't see there being a technical issue preventing either of these.

[theckman]

@myitcv I want to make sure I am clearly understanding your message, since I don't see an explicit mention of the technical concern I raised. Are you implying that you don't believe the concern, about working on in it's current location, is an issue?

Edit: To be more specific, I am fairly certain to manage additions and removals we would need the power to add (and remove) people from the golang organization. While I'd commit to not overstep any lines by having that power, I'm not sure I'd want it and I would assume the Go Team would also not like me to have it.

Even then, I become a gate keeper in a way that concerns me with regards to the health of projects we manage.

[Merovius]

@myitcv I think the technical issue really is that a project is made up of more than just the code. As Russ acknowledged, github doesn't really provide all the tools to smoothly migrate "the rest". I do hope, though, that the compromise proposed is sufficient. But that's just the technical side (i.e. clicking the fork-button and having or not-having a redirect and all that stuff) - what I'm referring to is that the technological compromise is then made conditional on changing the self-presentation. And yes - even without giving in to that condition and without any cooperation from the admins of the golang-organization admins there will still be the possibility of a regular fork. I'd just personally hope that this doesn't end in such a stalemate :)

[myitcv]

@theckman but you don't need to be able to add/remove people from the golang organisation to make progress on managing the dep repo itself, issues, PRs, correct? You could just need certain permissions within the dep repo.

Concretely, what is stopping someone from adding modules compatibility to dep today?

[myitcv]

(sorry that sent too quicky).

If a handful of folks need to be granted permissions/made collaborators on the dep repo, in the short term, I'd think that could be arranged and would be appropriate?

[theckman]

Finally having the time get in front of my laptop to look in to it further, it seems like GitHub may have the granularity. If I'm an admin on the repo I _should_ be able to add other admins by hand/a script. I wouldn't be able to use a direct team-based approach like we're doing in the Gofrs, so we'd need to write tooling to do the continual cross-organization sync to give those in the commune write access.

The other non-technical concerns raised in https://github.com/golang/go/issues/29639#issuecomment-453181447 would be the only final thing that provides direction. I believe the squishy bits are harder and more important than the technical ones here.

[as]

I support the idea of dep not being in the official Go repo. With the negative experience it gave my team, I would not recommend it for large repositories containing many dependent components (see "panic" in issue tracker). Dep works until you hit one of these non-squishy bits, and then you're out of luck because it gets stuck in a loop or panics.

It is easy to conflate dep with something endorsed by the Go team, and having large visibility in the Go org creates an attractive advertisement for people. The original maintainers of dep have not been fixing these issues and they are still in the open state. A recipe for frustration while the transition to modules is finalized.

[thepudds]

A brief comment focusing on just one of the several questions raised here.

The question has been asked what is stopping someone today from adding modules compatibility to dep in the current location of the dep repo.

Just focusing on the the technical side of that question for a moment, I think (?) the primary answer on the technical side is that in order to add modules compatibility to dep, there is a need for some threshold of expertise in the mechanics of how Go modules work. That also needs to be coupled with some threshold of expertise in the mechanics and internals of dep.

That could be expertise or knowledge that already exists, or knowledge that is learned. It could be knowledge that all resides within one human brain, or knowledge spread across > 1 human brain.

One concrete attempt to improve interoperability between dep and Go modules is this PR golang/dep#1963:

"[WIP] dep minimal module awareness"

I can't speak to whether or not the individual who initiated that PR has the sufficient level of expertise in the multiple areas that might be needed in order to bring that PR to a final/mergeable state.

edit: this was an implicit part of my comment, but just to make it explicit: in terms of technical reasons, I am not aware of any _other_ technical restrictions preventing someone from adding modules compatibility to dep today. In other words, I think someone could send a PR today to improve compatibility, and in fact, someone did. As I said, I think the _primary_ technical aspect holding back a "ready to merge" PR is sufficient expertise to get the PR into a mergable state, and of course, someone to approve the PR.

[anjmao]

The question has been asked what is stopping someone today from adding modules compatibility to dep in the current location of the dep repo.

I'm confused here. Why would you want to use dep tool and expect it to support go modules. It's already possible to migrate from dep to go modules with a few cmd commands. I would rather focus on solving tooling issues related to go modules support rather that dep which should become obsolete at some point.

[bcmills]

@anjmao, see https://github.com/golang/dep/issues/1962.

[theckman]

@anjmao Let me try to address you points, and reach back to earlier comments where necessary.

I'm confused here. Why would you want to use dep tool and expect it to support go modules. It's already possible to migrate from dep to go modules with a few cmd commands.

Modules are experimental. There are people who are not yet comfortable moving, and so they are continuing to rely on tools like dep. When someone hits a dependency that uses a SIV import, dep is unsure of how to resolve that. This is one gap that could be filled.

I would rather focus on solving tooling issues related to go modules support rather that dep which should become obsolete at some point.

To bring up a quote from the opening comment:

We may also want to pick up additional work to continuing to support dep long-term, as there is a deep philosophical divide between those who want a dep-like experience and those who are happy with modules. I don't believe these disagreements can be resolved in a single solution or implementation, and we should instead work together to ensure we can interoperate.

Over the past few days of different discussions it's cemented the idea that a few prominent features of modules are fundamentally different in their foundation, making it hard to see a path in which modules handles both. So I believe there is a possibility that dep does not become obsolete for some, and we could maybe make it a fully modules compatible tool while allowing the alternative workflow. There's also this later comment I made https://github.com/golang/go/issues/29639#issuecomment-452181753:

This isn't an ask to take our cookies and run away, but rather giving us the freedom and responsibility to support the community and to provide a solution to those who have needs outside of the scope of modules. Maybe by the time modules are ready we won't see a need, but it feels like a disservice to let it languish until then.

I hope this helps.

[bitfield]

I see the arguments for dep _as is_ remaining under the Go org. It has been the 'official experiment' for quite some time, and people will still be using it for some years, I should think. And I see the arguments for keeping the official version more or less as it is now.

But I also sympathise with the people who worked on dep, and want to keep working on it. While forking software is sometimes a bad idea, this seems like an ideal candidate for an 'amicable fork'. An alternative dependency manager for Go, which supports both modules and some 'dep++'-type functionality, would be no bad thing in my view. I probably wouldn't use it myself, but plenty of people would, I dare say. Maybe it should have a new name, to distinguish it from 'dep classic'.

dep started out as an independent project, and was moved under the official Go umbrella. You can't unscramble that egg, to mix a metaphor, so maybe the future of dep is as an independent project—just not _this_ project.

[natefinch]

If a handful of folks need to be granted permissions/made collaborators on the dep repo, in the short term, I'd think that could be arranged and would be appropriate?

I think the concern is that the members of the gofrs don't want to have to "ask permission" to add someone to the list of people who can collaborate on dep, or ask permission to get PRs landed etc. It sounds like that may be doable if the go team is willing to grant one or more of them admin access to the dep repo.

[aarzilli]

One open technical question would be around how we add new users to contribute to the repo

dep has 170 contributors total, which in the 2.5 years it existed averages to approximately one new contributor every 5 days. If you want to give the commit flag to, say, 20% of the contributors that's one new committer every month. It doesn't seem like it would be a big deal.

[mattouille]

I feel like the question we're potentially not asking, that should be addressed, is how to make it more possible for outside contributors to join Go projects. I'm sure there's a reason it's not being asked, so forgive my naivety if that's the case.

A point of reference for this is how Kubernetes is managed versus Go. It seems a lot of the bumpy road around community involvement and especially the need for an organization like Gofrs to jump in and help would be smoothed over by such a framework.

Kubernetes seems to function off of special interest groups (SIG's) and working groups (WG's) in order to satisfy collaboration from multiple corporations and contributors.

[theckman]

@mattouille Let me reply to your points individually:

I feel like the question we're potentially not asking, that should be addressed, is how to make it more possible for outside contributors to join Go projects. I'm sure there's a reason it's not being asked, so forgive my naivety if that's the case.

What do you mean when you say "Go projects"? Are you referring to the golang/go repo, things under the golang org, or just Go software in general? The talking points are way different for each as the scope is much different.

For _this specific issue_ we wanted to focus solely on dep and how we can improve the modules transition in relation to it. Other issues should probably be addressed using other channels (a different GH issue, 1:1 discussions, etc.). But to your contribution question, by and large, I think the answer today is: open an issue, file a PR/CL, poke for a follow-up on any reviews to those PRs/CLs.

Now if the question is how do people _not_ on the core team get permission to merge changes, that's a discussion well outside the scope of this issue.

A point of reference for this is how Kubernetes is managed versus Go. It seems a lot of the bumpy road around community involvement and especially the need for an organization like Gofrs to jump in and help would be smoothed over by such a framework.

If someone in the wider k8s ecosystem builds a tool that interacts with the k8s API, and a lot of people use it but the author stops working on it, does the Kubernetes project have a procedure for their maintainers to adopt that project? I am not aware of such an effort, and I really don't think it should be on those _maintaining_ k8s to adopt the maintenance of what would effectively be a k8s user.

In the same way, I don't think what we (The Gofrs) are hoping to help with should be on those directly maintaining the language. That said if someone from the Core/Language Team is passionate about supporting the ecosystem's packages/tools/etc, and has the spare cycles to join us and help out, we'd love it. I don't think we expect that.

Kubernetes seems to function off of special interest groups (SIG's) and working groups (WG's) in order to satisfy collaboration from multiple corporations and contributors.

I think there is some value in this point, albeit discussed through different channels (not this issue). That might be an issue on the golang/go repo, or it might be starting conversations at GopherCon 2019.

My focus on this issue is dep and reducing toil in our ecosystem that's impacting our fellow Gophers. How that goes is likely a datapoint to consider in relation to a few of your points above.

[leighmcculloch]

the best thing would be for you to fork it and for us to post a prominent note here that development has moved to gofrs/dep.

The offer ☝️ that @rsc posted is a generous response I think because when we consider the wider Go community, and not the gofrs Go community, encouraging dep to live on in an official fashion linked to or redirected to by golang/dep won't be in the best interest of the wider Go community. The wider community includes a lot of developers who need to build things and not dwell on which dependency manager to use, in the same way we don't dwell on how to format Go code because of gofmt.

We may also want to...support dep long-term, as there is a deep philosophical divide between those who want a dep-like experience and those who are happy with modules.

Having multiple dependency managers in the Go ecosystem will do more harm than good. We have other ecosystems to look at to see how that turns out. e.g. Java with Ant/Ivy, Maven, Gradle, Bazel. When I started writing Java the build ecosystem was overwhelming, and integrations with each option were inconsistent and fraught with issues. I expect having to navigate multiple dependency managers will become a huge hurdle to newcomers to Go, and a distraction to everyone else.

And once modules are ready, dep should go away.

Can we have one dependency manager? 🙏

[theckman]

@leighmcculloch I think comparing the Java ecosystem's tools to ours is a bit unfair, because the tools you cite often do way more than just manage dependencies. There's also the issue, as you say, with them being inconsistent / not interoperating well. That's not the path we'd ever pursue here.

This is a bit of a heavy question, but I'd love to hear your answer: If there are cases where the MVS methodology does not make the user comfortable with Go, or it just doesn't _work_ for them, would you just tell them to pick another language? Would you tell them to write their own dependency management toolkit and then hope it works well with Modules?

The goal I have in mind with dep would be to make it interop with Modules, with it being a superset. If someone wanted to use modules (the default) they could. If someone wanted to use dep with MVS they could, or if they wanted a resolution model that's more like other languages (MxVS [Maximum Version Selection]) they could.

If it's something people can opt-in to if Modules doesn't fit their needs, and it will play well with Modules (including rendering the appropriate metadata files) I don't see an issue.

At the worst case scenario I'd much rather we take the approach that provides the option of alternative resolution while working well with modules, over ending up with a myriad of tools out there that don't work well with Modules. The latter would feel more like where we are today, and it's not good.

Edit: I'd also feel a bit bad about telling someone to find another language that has a dependency manager that fits their needs.

[theckman]

@leighmcculloch I forgot another thing I wanted to ask. I'd love to hear your ideas on solutions to the differences in philosophy, and how to resolve them without two separate tools (where one resolves itself to pay the tax of supporting the other).

[Merovius]

@theckman FWIW, I don't think MVS vs. SAT is any different than GC vs. ARC, or nominal vs. structural subtyping, or the decentralized import-path convention vs. monolithic package servers. Yes, I'd totally tell people that Go isn't for them if they are not comfortable with any of these - that just seems the decent thing to do. If they are unhappy with a decision Go made and feel like they can't use it, but we think the decision was right and don't intent to change it, then telling them that is absolutely the right thing to do. That is not the same, however, as preventing them from working on an alternative dependency manager (or whatever) while sticking with the language itself, if they so choose.

We don't even have to talk in hypotheticals here: The Go project decided to move ahead with modules and you and others don't like that decision. The Go project is still sticking with it and you are free (and using that freedom) to invest your time into an alternative package manager.

IMO the discussion of whether or not dep should continue exist really isn't getting us anywhere. I don't believe anyone seriously doubts that some people will continue to invest in it. Or that the Go project will continue to move ahead with modules and will eventually recommend against the usage of dep.

This should really focus on what the Go project, the dep authors and the gofrs group agree on is the best way to make that future happen. In that spirit, to summarize what I see as the state of that conversation (apologies if I'm misunderstanding someone, don't want to put words into people's mouth):

• @rsc has made a suggestion on what to do - namely, archive golang/dep with a notice redirecting to gofrs/dep, foregoing having issue history in gofrs/dep as a result. He also added a condition, which AIUI has been fulfilled.
• I interpret @carolynvs as giving her OK.
• @theckman has brought up some questions, but AFAICT Russ' suggestion answers them (as the primary dep-repository would be in the gofrs org). Unless I'm overlooking something, there hasn't been a clear Yes or a clear No from him in regards to Russ' suggestion.
• @sdboyer has not participated so far (understandably). I understand Russ' comment as requesting him (as sort of the dep-point-person, I assume) to also explicitly OK it, so maybe he could give a clear yes/no too :)

This discussion might be redundant, given that there's a clear suggestion on the table - and while it's probably not ideal, I would've thought it's a compromise everyone can live with :)

[peterbourgon]

I just want to observe that, while a lot of words have been written in this thread, there has still been no concrete technical reason given for making this change. As far as I can tell, nobody who wants to contribute to golang/dep is blocked from doing so, contributors with commit/merge rights are still active, and it's possible to expand that group of people if useful/necessary. Moving a repo for a widely-used tool is disruptive to the ecosystem, we shouldn't do it without a very good rationale.

[thepudds]

@peterbourgon as far as I can tell, I think you are right -- there are no technical reasons preventing anyone from contributing to golang/dep in its current location (aside from perhaps knowledge/expertise, which perhaps rightly could be considered "not a technical issue").

But the discussion here is not just about technical matters? Perhaps you could say a few words about either (a) your viewpoint on the non-technical matters, or perhaps instead (b) a few words about why you view this as purely a technical discussion (which you might or might not actually be thinking)? Or a brief comment hitting something along those lines?

In the spirit of the attempt by @Merovius to help summarize (and at the risk of me making a hash of it), I think there are some open/unanswered non-technical questions if the repo was to stay at golang/dep, including but not limited to some of the questions asked by @theckman in https://github.com/golang/go/issues/29639#issuecomment-453181447, such as:

When modules are "done" are we going to be asked to stop working in the main repo and then incur a tax on any user who wishes to continue using dep? How about any confusion resulting from development continuing after modules are "done"?

[as]

I think the issue is visibility and potential confusion. If someone is having issues with dep, it is more confusing to say "oh, you have to use this other dep in this other repo". It is possible to fork a project, but not possible to replace it when the original is more popular. I imagine having two deps out of phase with each other is worse than having one broken one.

That said, I am more trying to clarify from my point of view why there would be a non technical facet to this than advocating or endorsing dep (as my support for modules over dep is clear from my previous post above in this discussion).

[andybons]

Thanks, everyone, for your thoughtful points. We appreciate the passion and insight shown in this discussion.

We will not be transferring the dep repo. It is an important part of Go’s development history and still a useful tool for many. It will remain in the golang organization indefinitely.

If you wish to contribute, the original owners still have all their rights and are active as mentioned above.

Once modules is moved out of experimental (we are targeting Go 1.13), we will likely archive the dep repo shortly after to avoid confusion on which dependency management solution the Go project officially supports. We will do our best to communicate when exactly the archive will happen to avoid surprise, but it should be planned for.

The gofrs organization is, of course, free to fork golang/dep. Any messaging in the README or elsewhere is up to the current stewards of the golang/dep project (as I understand it, this is @carolynvs and @sdboyer).

[theckman]

@andybons Well, since there are issues outstanding with dep and an issue with availability of the current maintainers, how can we work to get fixed merged in? How can I become a maintainer?

[theckman]

cc @carolynvs & @sdboyer on the above. Trying to understand how we can improve things here.

[andybons]

Both @peterbourgon and @carolynvs, who have been active on this thread, are admins of the repo. I will leave it up to them how they would like to proceed.

[theckman]

@andybons thank you.

[kevinburke]

Here's one technical reason - the primary maintainers of the project are not Googlers and the Google CLA bot actively impedes getting PR's merged even when all maintainers have signed the CLA. I suppose with administrative privileges we could merge things without the CLA.

https://github.com/golang/dep/pull/2117#issuecomment-469345824

[peterbourgon]

That's very annoying. @andybons could you be a cla: yes sponsor for this repo, or suggest someone who could fill that role?

[andybons]

@peterbourgon sure. Anyone on the @golang/osp-team should be able to help in these situations. If it becomes too onerous then we’ll bring it up with the googlebot folks.

[FiloSottile]

@andybons I don't actually have write access to that repo. You might have to fix the permissions to give access to osp-team.

[andybons]

@FiloSottile done. Thanks for the heads up.