Go: cmd/go: support sending credentials to an insecure GOPROXY

[elioengcomp]

What version of Go are you using (go version)?

→ go version
go version go1.13.4 darwin/amd64

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

→ go env

GO111MODULE="on"

GOARCH="amd64"

GOBIN=""

GOCACHE="/Users/eliom/Library/Caches/go-build"

GOENV="/Users/eliom/Library/Application Support/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="darwin"

GONOPROXY=""

GONOSUMDB=""

GOOS="darwin"

GOPATH="/Users/eliom/workspace/jfrog-cli-go/gopath"

GOPRIVATE=""

GOPROXY="https://gocenter.io"

GOROOT="/Users/eliom/workspace/go-1.13.4"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/Users/eliom/workspace/go-1.13.4/pkg/tool/darwin_amd64"

GCCGO="gccgo"

AR="ar"

CC="clang"

CXX="clang++"

CGO_ENABLED="1"

GOMOD="/Users/eliom/workspace/go.mod"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/td/hj3dbxrx6mqf61t14259l2q8_9k8fn/T/go-build358474542=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

• Set GOPROXY to point to an insecure proxy using basic authentication
• Try to fetch a module using go mod download

What did you expect to see?

I expect the module to be download successfully. I understand that, for security reasons, additional steps should be performed to allow the client to communicate with the insecure proxy, but it seems those additional steps are only available for the go get command through the -insecure flag.

We need to fully support insecure proxies for all the go mod commands. We cannot assume that all proxies are production ready specially when working on proxy development. Making all development environments secure is not feasible and we still need to be able to send authenticated requests against it to validate proxy features like user permissions.

What did you see instead?

The command fails since the client refuses to send the credentials to the insecure proxy.

{
    "Path": "github.com/kr/pretty",
    "Version": "v0.1.0",
    "Error": "github.com/kr/[email protected]: refusing to pass credentials to insecure URL: http://admin:%5Bredacted%5D@localhost:8082/github.com/kr/pretty/@v/v0.1.0.info"
}

[bradfitz]

It's very unlikely we'll make a change here.

Have you seen https://blog.filippo.io/mkcert-valid-https-certificates-for-localhost/ ? That is likely the easier path forward for you.

[bradfitz]

/cc @FiloSottile @bcmills

[bcmills]

We _do_ support insecure proxies as GOPROXY targets. You can use a plain http:// URL for a GOPROXY and it should work fine.

We intentionally _do not_ support sending _credentials_ to insecure proxies. Accidentally dropping the s in the https:// URL should not result in credentials being sent over the network unencrypted.

[bcmills]

Your use-case here is for testing a GOPROXY implementation. Tests should ideally be as realistic as possible, and there are important details in the way the go command handles secure URLs beyond just attaching credentials. (For example, we also disallow redirects from https to http URLs so that misconfigured servers won't silently downgrade what the user intended to be a secure connection.)

Ideally, you should test your GOPROXY implementation as an actual HTTPS server, possibly using the mkcert tool that Brad linked above.

That said, if you intentionally want to test the server with _unrealistic_ requests — for example, in order to write tests that users can run without generating or installing certificates — it should be fairly trivial to inject a second proxy between the go command and your GOPROXY implementation. That proxy could accept requests (without credentials) over a plain HTTP connection, and forward those requests along with the testing credentials to the server under development.

[bcmills]

(CC @jayconrod)

[bcmills]

Please try the second-proxy approach and let us know if that works out.

[elioengcomp]

@bcmills I would not expect it to work by default to prevent the dropped s issue you mentioned, but instead via some additional configuration where we explicitly instruct the client to trust the proxy (like we do with Docker using the insecure-registries property on the daemon.json file).

The use case I have in mind for this requirement is development environment exclusively. I understand there are better solution for test/stage/production environments.

I'll give the workarounds you pointed out a try and let you know about the results.

[elioengcomp]

@bcmills I tried the mkcert approach and it works great. @bradfitz Thank you for sharing the link. I'll make it part of our development environment bootstrap automation and it will solve the problem for us.

[bcmills]

Great! Closing the issue, since you have an acceptable workaround and we don't have any other use-cases for this at the moment.

[elioengcomp]

Hi Mayer,

Basically, you can use mkcert to generate private and public keys to any
domain and install them in a local CA in your machine.

I've just followed this tutorial to get it working:
https://medium.com/@shriramsharma/mkcert-you-changed-my-life-b157466880bf

Best,
Elio

On Wed, Nov 18, 2020 at 8:22 PM Mayer Ma [e-mail] wrote:

@elioengcomp https://github.com/elioengcomp I face the same issue with
you, i need to use a internal proxy like http://user:[email protected]:8080
to download the package via go mod.
But I'm not sure how to use mkcert to let things work. Could you please
give me a clearly guide?
Thank you!

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/golang/go/issues/35975#issuecomment-730120054, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AFQYW3J55DIUE4YQTI57WS3SQSMQPANCNFSM4JVQWYDQ
.