Chi: What's up with go module support?

[MaikuMori]

Go 1.13 is out, GOPATH is gone. Most of the packages have go module support.

What's holding Chi back? How can I help?

[VojtechVitek]

From https://github.com/golang/go/wiki/Modules#releasing-modules-v2-or-higher

Note that if you are adopting modules for the first time for a pre-existing repository or set of packages that have already been tagged v2.0.0 or higher before adopting modules, then the recommended best practice is to increment the major version when first adopting modules.

As per the above official recommendation, chi's not going to adopt Go modules until we plan to release v5.0.0.

Otherwise, we'd have to introduce a breaking API change, as we'd have to change the import paths to "github.com/go-chi/chi/v4". See https://github.com/go-chi/chi/pull/379/files how the changes would look like. We opted out for now.

The +incompatible flag works OK for now. You can install latest version of chi with GO111MODULE=on go get -u github.com/go-chi/[email protected].

[pkieltyka]

I've been thinking about this for a while and I believe I have the approach that will make me the happiest. Chi needs to get proper go.mod compatibility, and with our current versioning from v1.0.0 to v4.0.0, it requires as @VojtechVitek suggests to add the suffix /v4 at the end of all import paths. If I could go back in time, I would have just released chi versions v1.1, v1.2, v1.3, and v1.4 -- instead of v2.0.0, v3.0.0, v4.0.0. In keeping with v1.x versioning, we can keep the import path as it is with github.com/go-chi/chi, and we get module compatibility.

Therefore, here's my idea. We can still do that! and re-version v2.0.0 as v1.2, v3.0.0 as v1.3, and v4.0.0 as v1.4.

[VojtechVitek]

That is an interesting idea!

But we'd have to make it clear that v1.4 is not really backward compatible with all v1.x versions, as it should be per semver specs. Example of breaking changes:

• v1.0 had pre-context signatures (func(ctx context.Context, w http.ResponseWriter, r *http.Request))
• v2.0 had old URL parameter syntax (/:id instead of /{id})
• probably more stuff, I'm not sure

But thinking about it.. when did we switch from https://github.com/pressly/chi to https://github.com/go-chi/chi? Maybe go-chi/chi v1.0.0 wasn't actually referenced by anyone and we're good to go? (and maybe we could even go for v1.0.0 ?! :D)

👍

[pkieltyka]

@VojtechVitek yes thats true. v1.0.0 was used by github.com/pressly/chi which has been deprecated for many years. We could peg our next release as v1.0.0 -- but what would we do to retag previous versions?

[VojtechVitek]

@pkieltyka I propose:

1. Forking github.com/go-chi/chi back to github.com/pressly/chi so we can leave v1.0.0, v2.0.0 and v2.1.0 tags working forever with the old "github.com/pressly/chi" import path.
2. On this project, we can force delete the old (pre-v3.0.0) tags, since they were never released with the new "github.com/go-chi/chi" import path. And start from scratch, assuming we won't release v3/v4 modules any time soon.

[zikaeroh]

With the module proxy, sumdb, and existing projects with versions set, I can't see force-dropping a bunch of tags and re-versioning going too well. I'd personally be much happier to see a /v5 suffix with the understanding that the major version is really only there to indicate API breaks. (That, and losing all of the releases and release notes on github would be a shame.)

[mvrhov]

There is one another thing.. if we are braking BC.. can we also get support for "named" routes, so we can reverse build URLs

[pkieltyka]

hey all, I'm going to approach this one cautiously. I don't want to break backwards-compat and hurt people's experience, but I also can't move forward with our versioning and go.mod's functionality as a lot of editors (specifically vscode) have a poor experience with modules (since they're still kinda new). I have an idea of what to do but I'll need to test it in isolation in a separate mock environment. I'll report back once I've had a chance to do that. If someone else has other ideas/insights, lmk too please. Thanks

[zikaeroh]

@thepudds previously helped out with the module situation in github.com/ugorji/go, perhaps they have a suggestion.

Otherwise (if they're not too busy getting the 1.14 beta out the door 😄 ), I'm sure @bcmills or @jayconrod have good suggestions.

My personal suggestion is to try to not be fancy with the solution. I'm not sure what backwards compatible scenarios you're trying to enable, but if it's solely trying to handle non-module users, I'm not sure how valuable that'd be. The version directory method could help (but it's not my personal favorite).

For what it's worth, my experience in VS Code with gopls has not been bad at all; I've been using it for months with some minor issues (which I've reported and have mostly been fixed), but maybe your experience has been different.

[bcmills]

The “major subdirectory” layout is compatible with both module mode and GOPATH mode.

If you don't want to maintain two copies of the code, one simple option is to leave forwarding packages at the unversioned paths that use aliases and forwarding functions to redirect to the versioned ones.

Also note that there is an upcoming change (in 1.14) to ignore incompatible major versions when resolving the “latest” version if a go.mod file is present in the latest v1 release. If you decide to go back to the v1 line, it will be a bit awkward for users still on Go 1.13 or earlier, but should be reasonably smooth for users on Go 1.14 or later. (See https://tip.golang.org/doc/go1.14#incompatible-versions.)

[pkieltyka]

thanks @bcmills for the feedback and nice to know about the 1.14 updates too

[MaikuMori]

I believe the best would be to just release a new v5 with new import path and leave the past in the past as it is.

[Mentioum]

Please can I get some clarity on what we're supposed to do right now to get the latest version.

Is it just to put /v4 suffix on the import path?

[VojtechVitek]

@Mentioum Get the latest chi (as of today) with GO111MODULE=on go get -u github.com/go-chi/[email protected]. The import path stays on "github.com/go-chi/chi".

[Mentioum]

Thanks @vojtechvitek

[icholy]

Fighting against the /vN suffix is the same as fighting against gofmt. The suffix may seem a little aesthetically off-putting at first, but that's because it's new and unfamiliar. Just like fully qualified import paths, SIV will become "normal" looking.

edit: I'm asking for this project to embrace SIV and add the /v4 or /v5 to the module import path.

[coolaj86]

At the very least, let's update the README.md to be very clear about the expected behavior.

[zikaeroh]

Color me a bit confused, but I see now that there's a new tag v1.5.0 after v4.1.2. This does get picked as the "latest" in a new project without chi, but in an existing project that's at v4, go get with @latest won't do anything as I believe it thinks it's a downgrade. Once at v1.5.0, it at least doesn't attempt to go back to the +incompatible series, which is nice, at least.

If I do go get -d github.com/go-chi/chi@master, then I get v4.1.3-0.20201129224101-81b0a6f7ed20+incompatible, which appears to use the v4 tags. The README also says to get at @v4 which with the current tag setup is before v1.5.0.

Is there no chance of just bumping this up to /v4 or /v5 to make it more clear what's going on? I think this only ends up adding more confusion as to what actually is the latest (as compared to either ignoring modules altogether [not my favorite] or switching to SIV)... It'd be nice to get some consistent guidance as to what we should be depending on if we want to continue to get updates.

[pkieltyka]

Hey all, chi now has go.mod support! Please see https://github.com/go-chi/chi/pull/560 -- thanks for your patience. Please read the PR description for more information on the versioning update.

[zikaeroh]

Unfortunately, this does not play nicely when other dependencies I have depend on "newer" versions of chi by version.

Somewhere in my dependency tree, a dependency requests at least v3.3.2+incompatible, which prevents me from adding v1.5.0 or v1.5.1 without a replacement clause, as the go tool sees v3.3.2+incompatible as a higher compatible version when tidying things and therefore upgrades me. The only reason @master worked before was that the Go tool was picking the next compatible version for its number, which was a v4. Now it's tagged.

It's now essentially impossible for me to depend on chi at anything but v4+, because attempting to use anything else will effectively "downgrade" me to a higher version number.

I understand that you aren't a fan of the aesthetics of having a version number in your import path, but this is really not going to work well when existing projects with modules already have versions above v1 in their dependency graph.

If you don't want to add /v4 or /v5 (which I still personally find to be totally acceptable), maybe you can use Go 1.16's version retraction support to effectively "delete" all older versions. But I know that I'm going to be stuck at v4 for a while. :slightly_frowning_face:

[zikaeroh]

Go at tip has better messages when moving between versions. Here's what's happening to me:

$ gotip list -u -m github.com/go-chi/chi
github.com/go-chi/chi v4.1.2+incompatible

$ gotip get -d github.com/go-chi/[email protected]
go get: downgraded github.com/99designs/gqlgen v0.13.0 => v0.7.2
go get: downgraded github.com/go-chi/chi v4.1.2+incompatible => v1.5.1

$ gotip list -u -m github.com/go-chi/chi
github.com/go-chi/chi v1.5.1

$ gotip mod tidy                        
go: finding module for package github.com/vektah/gqlparser/gqlerror
go: finding module for package github.com/99designs/gqlgen/graphql/handler
go: finding module for package github.com/99designs/gqlgen/graphql/playground
go: finding module for package github.com/vektah/gqlparser/ast
go: found github.com/99designs/gqlgen/graphql/handler in github.com/99designs/gqlgen v0.13.0
go: found github.com/99designs/gqlgen/graphql/playground in github.com/99designs/gqlgen v0.13.0
go: found github.com/vektah/gqlparser/ast in github.com/vektah/gqlparser v1.3.1
go: found github.com/vektah/gqlparser/gqlerror in github.com/vektah/gqlparser v1.3.1

$ gotip list -u -m github.com/go-chi/chi
github.com/go-chi/chi v3.3.2+incompatible [v4.1.2+incompatible]

[pkieltyka]

Sorry to hear you're having trouble with the upgrade. I believe part of the issue is go module cache. With Go 1.16 and retractable versions this will definitely get smoother, but you can try to remove from go.mod and go.sum the incompatible versions of chi and then add @v1.5.1 -- lmk how it goes

[zikaeroh]

That's the first thing I tried, yes. I cleared my entire mod cache and even used GOPROXY=direct to bypass the proxy.

I'm pretty certain v1.5.1 is treated as before v3.3.2+incompatible when it comes to Go's sorting of versions.

For experimentation, I edited my go.mod to exclude every version from 3.3.2 and above to simulate what'd happen if those versions were retracted:

exclude (
    github.com/go-chi/chi v3.3.2+incompatible
    github.com/go-chi/chi v3.3.3+incompatible
    github.com/go-chi/chi v3.3.4+incompatible
    github.com/go-chi/chi v4.0.0+incompatible
    github.com/go-chi/chi v4.0.0-rc2+incompatible
    github.com/go-chi/chi v4.0.1+incompatible
    github.com/go-chi/chi v4.0.2+incompatible
    github.com/go-chi/chi v4.0.3+incompatible
    github.com/go-chi/chi v4.0.4+incompatible
    github.com/go-chi/chi v4.1.0+incompatible
    github.com/go-chi/chi v4.1.1+incompatible
    github.com/go-chi/chi v4.1.2+incompatible
)

But the compiler says:

go: github.com/99designs/[email protected]: github.com/99designs/gqlgen(v0.13.0) depends on excluded github.com/go-chi/chi(v4.1.2+incompatible) with no newer version available

As v1.5.1 does not satisfy the minimum version required by my dependencies. So though I was hopeful retraction would help, I don't think it will.

[zikaeroh]

Clarification, Go 1.16 with that list of excludes does actually compile after a go mod tidy. But I'm not really counting on it working without also having the exclude lines to drop things, as if v1.5.1 and v3.3.2+incompatible are in the requirements graph, v3.3.2+incompatible will be chosen as it is still semantically newer than v1.5.1, as v0, v1, and +incompatible are all considered to be "compatible" versions and compared directly.

[pkieltyka]

thanks for helping to debug. Once github.com/99designs/gqlgen upgrades to [email protected], you'll be able to upgrade comfortably. The issue is when other dependencies, depend on chi@v4+incompatible, the go mod tool will force the chi version to the dependency's version of chi -- I believe this is happening because they're the same import path.

The silver lining here is that chi doesn't change much between versions and apis are the same -- meaning upgrading to the head/master/v1.5.1 will be trivial for any dependency, such as 99designs/gqlgen

[zikaeroh]

Yes, exactly, because v1.5.1 is still semantically lower than any of the +incompatible versions. That's why I'm saying this is going to turn out to not be so fun to handle, if the ecosystem needs to collectively forget about all of the versions released in the past few years when the tooling is fighting against it.

I'm going to try and submit a PR to gqlgen to modify the version to try and make it so I can move to v1.5.1, but I don't think this is how module versioning was intended to be used.

I hope I'm wrong, though, and the retraction system in Go 1.16 is able to smooth this over. Chi is still my favorite router and I want to use it!

[lwc]

@zikaeroh @pkieltyka I've updated chi on gqlgen master, let me know how it goes, if it's all good i'll cut a proper release for gqlgen.

[zikaeroh]

Thanks for that merge. I've already bumped it up in my project that uses gqlgen without issue. Hopefully I don't import anything else still using the old chi code. :slightly_smiling_face:

[pkieltyka]

Hi all, I've just tried gqlgen@master and seems to work with [email protected]

Here is my general upgrade path for a project:

1. go mod graph | grep "go-chi/chi" -- prints the graph and shows packages on the left which depend on chi
2. Assuming all sub-deps are upgraded to [email protected], you can just run go get -u, but, for more clear upgrade path then, for each package which depends on chi, first ensure that package is upgraded to [email protected], then run go get <the-dep-which-deps-on-chi>@latest or instead of @latest use vX of the version which is the newest for that dep (in case of cache..). @master also works fine
3. go get github.com/go-chi/[email protected] -- or @latest also works but sometimes cache will get in the way
4. go mod tidy -- always good to run

---

for gqlgen, I simply ran:

1. go get github.com/99designs/gqlgen@master
2. go get github.com/go-chi/[email protected]
3. go mod tidy

[bcmills]

@zikaeroh, lazy loading will also help with pruning out spurious transitive dependencies, but unfortunately I didn't get that finished in time for 1.16. 😞