Go: cmd/go: default GONOSUMDB to GONOPROXY if not specified

[rogpeppe]

I suspect that GONOPROXY will be used mainly for private modules. If you're using private modules, you probably don't want to be using the sum database for those either, so it seems like it would be useful if GONOSUMDB would use GONOPROXY if not set.

That would have saved me some time, because if you just set GONOPROXY for a private repo but not GONOSUMDB, you'll get errors - it's not immediately clear that you need to set GONOSUMDB=$GONOPROXY to get it to work.

[rogpeppe]

@bcmills @rsc

[bcmills]

CC @FiloSottile @jayconrod

[FiloSottile]

On one hand, it is not good to default to a less secure state silently. On the other hand, this would help greatly with not training users to blindly set GONOSUMBD when something goes wrong with the checksum database. (See #32291.)

Given there's a lower risk of MITM anyway when going direct (although we do want to protect against malicious origins) and since I can't really think of many cases where you want to add a path to GONOPROXY and not GONOSUMDB, I'm in favor of this.

I would make an exception for GONOPROXY=*, but I guess most people will just do GOPROXY=off instead, which does not disable the sumdb, so maybe there's no need.

[bcmills]

@FiloSottile, s/GOPROXY=off/GOPROXY=direct/?

[FiloSottile]

@bcmills yes, thanks.

[rsc]

It is definitely true that there is something missing here. GONOSUMDB and GONOPROXY are both low-level things and you shouldn't have to configure both. I think we should do something for Go 1.13.

Separately, @ianthehat asked me in the context of editor integration if there was some way to tell for a given package "is this an import that would make sense to hyperlink to godoc.org?".

I am starting to think we should add a higher-level control knob GOPRIVATE and encourage that instead of the lower-level knobs. GOPRIVATE would set the default for both GONOSUMDB and GONOPROXY but other programs could also look at it to decide the more general question of whether this is a package known to public infrastructure or not.

Thoughts?

[rogpeppe]

If you have GOPRIVATE, is there still a significant use case for GONOPROXY and/or GONOSUMDB at all, in fact?

[rsc]

@rogpeppe, you might still have modules that are private but served from your local proxy. For those you want to get them from a proxy but not do the checksum database thing. So GOPRIVATE by itself is not enough: you really do need different knobs for proxy vs sumdb in the general case. But the general case is not anywhere near the common case, which is why GOPRIVATE might make sense for the common case.

@bcmills and I both feel like GOPRIVATE is still not perfect but substantially better than making people set both GONOSUMDB and GONOPROXY, and probably we should do this soon (before Go 1.13 beta 1). I'm curious what @ianthehat, @jayconrod, and @myitcv think.

/cc @andybons for beta 1 timeline FYI

[myitcv]

But the general case is not anywhere near the common case, which is why GOPRIVATE might make sense for the common case.

This would be my first/best guess based on a "finger in the air" analysis of our _current_ setup and my understanding of others' _current_ non-public setups (notice emphasis). Because the landscape might well/will change.

That said, right now (i.e. for 1.13) it feels like https://github.com/golang/go/issues/32184#issuecomment-498756353 is about right.

But to be perfectly honest I'm not as close to the detail on all of this as others, so will happily defer 😄

[jayconrod]

I'm ambivalent on introducing GOPRIVATE vs. defaulting GONOSUMDB to GONOPROXY. I'd lean a little bit toward GOPRIVATE, but either way is better than requiring both GONOSUMDB and GONOPROXY to be set.

GONOSUMDB means "don't ask sumdb about these modules". As I understand, it only makes sense to put modules in this list if the sumdb doesn't have access to them and shouldn't know about them. It should be rare for GONOSUMDB and GOPRIVATE should be different. Maybe if there's another private sumdb implementation?

GONOPROXY means "don't ask proxies about these modules". The correct set of modules depends on the proxy configuration, so the correct value of this seems less clear. Without GOPRIVATE, I think it makes more sense to default GONOPROXY to GONOSUMDB instead of the reverse.

GOPRIVATE isn't strictly necessary, but it's a nicer interface because it lets users communicate intent rather than specifics. My only concern is that, as Paul said, the landscape may change in the future, and we might want GOPRIVATE to do something else later on.

[bcmills]

Without GOPRIVATE, I think it makes more sense to default GONOPROXY to GONOSUMDB instead of the reverse.

I was thinking about that yesterday and came to pretty much the same conclusion. GONOSUMDB essentially declares “these packages are not available to the general public", and GONOPROXY declares “these packages are not available via my selected proxy”. Those conditions converge when the proxy indicated by GOPROXY is one that serves the general public.

[rsc]

@jayconrod

My only concern is that, as Paul said, the landscape may change in the future, and we might want GOPRIVATE to do something else later on.

You sound like the change is an argument against GOPRIVATE, but isn't it an argument in favor of GOPRIVATE? If GOPRIVATE means "these are non-public packages, act accordingly" then that can adapt to what actions need doing later, in ways that the more low-level GONOPROXY and GONOSUMDB do not.

[jayconrod]

@rsc If GOPRIVATE is allowed to adapt to the changing landscape, then I'm in favor of it. I was assuming it would be frozen as a default value for GONOPROXY and GONOSUMDB, but it doesn't sound like that's the case.

[gopherbot]

Change https://golang.org/cl/181719 mentions this issue: cmd/go: add GOPRIVATE environment variable