Protobuf: google.golang.org/protobuf versioning is confusing

[sylr]

Hi,

This morning I stumble upon a tweet linking to the golang blog's entry announcing the release of APIv2 of this project.

On a strict naming and versioning level I'd like to express my personal thought which tells me it would have been very hard to do things more wrong.

First of all you have chosen to release a new NON backward compatible version of your project, that you don't hesitate to name APIv2, with the tag v1.20.0 although the last tag of the previous line was v1.3.4. How could you not see how this is highly confusing ?

I can't understand why you chose to do that instead of tagging your new API v2.0.0.

Changing the import path from github.com/golang/protobuf to google.golang.org/protobuf does not make it right. It should have been a v2.0.0 tag.

And what about that change of import path ? Why ? How is that a good idea to have two different import path for the same project for two different branches ? How does one tell which is the new one from the old one from those import paths ?

Why was it wrong to have github.com/golang/protobuf and github.com/golang/protobuf/v2, which, by the way, is the convention enforced by go mod ...

And if the change of import path was mandatory (I don't want to assume why), then it should have been google.golang.org/protobuf/v2.

Also what about https://github.com/protocolbuffers/protobuf-go vs https://github.com/golang/protobuf ? Which one is supposed to be the good one ? At this time you cannot post issues to the first one but you can open PRs on both ...

So, to summarize, you decided to ignore of both SEMVER and go mod conventions resulting with a high level of confusion.

I really hope that no one will take example on what you've done here for their own projects.

Regards.

[dsnet]

I commented about this on Reddit.

Most of your questions are answered above. Some selected responses below:

How could you not see how this is highly confusing ?

We're aware. No matter how we slice the cake, confusion is going to be there. We decided that it was better to have more up-front confusion to pursue a future with less long-term confusion.

It should have been a v2.0.0 tag.

A v2.0.0 tag requires that the module path have a v2 suffix. We considered it, but decided that it was not worth the overhead and other forms of confusion it brings (e.g., confusion between that and the concept of proto2).

Why was it wrong to have github.com/golang/protobuf and github.com/golang/protobuf/v2, which, by the way, is the convention enforced by go mod ...

To be clear, it's a convention enforced by Go modules for breaking changes to the _same base module path_. The google.golang.org/protobuf module is an _entirely new module_. This is a critical distinction.

So, to summarize, you decided to ignore of both SEMVER and go mod conventions resulting with a high level of confusion.

Since an entirely new Go module was created, neither semantic versioning nor Go modules were violated.

[sylr]

Since an entirely new Go module was created

Then you should not mention APIv1 vs APIv2 everywhere.

Even if the so called v1 and v2 branch diverged a long time ago they share the same git repo and thus history and tags, so no, it's not a brand new module, it's a branch.

[dsnet]

Then you should not mention APIv1 vs APIv2 everywhere.

Thanks for the feedback. Other than the blog post, mentions of "v1" or "v2" will gradually all Go away. Currently, our issues were tagged "APIv2" for development purposes to distinguish what was what, but they will probably be renamed in the future.

The even if the so called v1 and v2 branch diverged a long time ago they share the same git repo and thus history and tags, so no, it's not a brand new module, it's a branch.

If you go back in history on the "api-v2" branch, it actually has no shared history with "master" or "api-v1". It really is an entirely new project. In fact, the api-v2 "branch" is just a mirror of the go.googlesource.com/protobuf repo. Not only does it share no history, it really does live in an entirely separate repository.

[sylr]

All right, then it's even more confusing than I believe it was.

I would recommend that github.com/golang/protobuf be stripped of any "APIv2" bits (branch, tags, issues if any) and that github.com/protocolbuffers/protobuf-go be stripped of any "APIv1" bits the same way.

[kk44]

@dsnet

...v2... (e.g., confusion between that and the concept of proto2).

google.golang.org/protobuf's next major version, is there a /v2 or /v3 suffix?

[bcmills]

@kk44, why would there be a next major version?

(And, even if there is, why worry about its potential import path now?)

[neild]

We went through several rounds of discussion on the import path to use for the new protobuf implementation. (If you look at the git commit history for the go.mod, you can see the import path shift several times--from google.golang.org/proto, to github.com/golang/protobuf/v2, to google.golang.org/protobuf.) Ultimately, we had to pick something, and given that we needed to change import paths anyway we decided to take the opportunity to switch to one not tied to a specific hosting provider.

(The package currently known as github.com/golang/protobuf/proto began its life as goprotobuf.googlecode.com/hg/proto, changed to code.google.com/p/goprotobuf/proto, and then changed again to its current location. This is a good lesson in not tying the import path to a specific VCS or host, and we took it to heart.)

Given that the new implementation shares no revision history with the old one and has an entirely new import path, we then debated whether to tag it v1 or v2. Either is acceptable under the rules of semantic import versioning, since both would produce entirely new import paths that adhere to the import compatibility rule: "If an old package and a new package have the same import path, the new package must be backwards compatible with the old package."

In favor of the v2 version, it would make the version in use visually distinct in import paths and aligned with the APIv2 terminology we were using in conversations.

In favor of the v1 version was the fact that the new module has no earlier version, that in 5-10 years the new module would hopefully be the only one anyone cared about so any confusion would be short-lived, and that it might avoid some confusion with the proto2/proto3 versions of the protocol buffer language.

We went back and forth for a while, but again we ultimately had to pick something and we settled on v1.

Maybe that was the wrong decision. If it was the worst decision we made in the design of google.golang.org/protobuf, I will be absolutely delighted.

If the choice we made was confusing, attempting to change it at this point would be even more so. I don't see any way in which that would be a good idea.

Going forward, we will not use the APIv1 or APIv2 terminology. The github.com/golang/protobuf module is the legacy implementation, and while we will continue to maintain it as necessary indefinitely the focus of new development will be on google.golang.org/protobuf.

[sylr]

@neild & @dsnet all right, thank you both for the explanation.

I'm still wondering if the legacy branch (v1) in google.golang.org/protobuf and the new implementation branch (api-v2) in github.com/golang/protobuf could be removed in order to not induce confusion for people browsing the repos ?

[dsnet]

The v1 branches and api-v2 branches are populated by automated systems. Manually deleting them will be ineffective as they will magically appear again. One of those automated mirrors is managed by me, but I'm out of the office until next week.

[dsnet]

The api-v2 branch on this repository has been deleted.

[dsnet]

FAQ updated to avoid any reference to APIv1 and APIv2. What remains is getting rid of the v1 branch on github.com/protocolbuffers/protobuf-go.

[dsnet]

The v1 branch on github.com/protocolbuffers/protobuf-go have been deleted. I think this is sufficient for confusion reduction. I'm going to close this issue.