Go: cmd/go: "go get" should create a replcae line automatically if the get path and moudle path don't match

[go101]

What version of Go are you using (go version)?

go version go1.12 linux/amd64

Does this issue reproduce with the latest release?

yes

What did you do?

$ go get github.com/go101/[email protected]

The module path set in go.mod is go101.org/tinyrouter, not github.com/go101/tinyrouter.

What did you expect to see?

no errors, and automatically create a replace line in go.mod file of the app project after parsing the go-got go.mod file:

replace go101.org/tinyrouter => github.com/go101/tinyrouter v1.0.1

What did you see instead?

go: github.com/go101/[email protected]: parsing go.mod: unexpected module path "go101.org/tinyrouter"
go: error loading module requirements

[dmitshur]

What's the motivation for it to automatically create a replace line in go.mod file?

At first glance, that kind of behavior would be unexpected and surprising to me, given how go get is documented at https://golang.org/cmd/go/#hdr-Module_aware_go_get. It would be helpful to better understand the motivation behind this.

[go101]

I think the motivation is straightforward. The intention of why I run "go get" is just to add a dependency to the current module-aware user package. For the library package is hosted on github.com, certainly I can't run "go get go101.org/[email protected]", for the library package isn't hosted on go101.org. The go101.org can even be a non-existing website.

[go101]

And I found that, even if I put the replace line manually to the go.mod file, then go get still reports errors when I run either go get go101.org/[email protected] or go get github.com/go101/[email protected].

$ go get go101.org/[email protected]
go get go101.org/[email protected]: unrecognized import path "go101.org/tinyrouter" (parse https://go101.org/tinyrouter?go-get=1: no go-import meta tags ())

$ go get github.com/go101/[email protected]
go: github.com/go101/[email protected]: parsing go.mod: unexpected module path "go101.org/tinyrouter"
go: error loading module requirements

I really don't know why go get still would connect to go101.org even if I have set a replace line in go.mod. What is the intention of replace lines?

[go101]

go build has no problems.

[dmitshur]

What is the intended import path for Go packages in tinyrouter?

If you want the import path to be github.com/go101/tinyrouter, then you should modify your go.mod file to specify the module path as github.com/go101/tinyrouter, not go101.org/tinyrouter. Then you can go get github.com/go101/[email protected] (or, a newer version).

If you want the import path to be go101.org/tinyrouter, then the https://go101.org/tinyrouter?go-get=1 page should serve a go-import meta tag, as described at https://golang.org/cmd/go/#hdr-Remote_import_paths. Right now, it's serving 404. After that, you will be able to go get go101.org/[email protected] (or later version).

[go101]

The intention is to avoid my library users being locked down to github.com.
I would like they use go101.org/tinyrouter as import paths in their source code.
When later, I move the package to another place, such as gitlab.com,
users can just modify the line in go.mod, no needs to modify all the import paths in source code.

Here, the go101.org could be anything other than github.com, for example, website.noexist/tinyrouter. I never prepare to create a http service to response the https://go101.org/tinyrouter?go-get=1 request. Is this not ok?

[go101]

If go build thinks it is ok, why not go get?

[jayconrod]

Module paths are canonical: if a module sets the path go101.org/tinyrouter in its go.mod, users must import it by that name in module mode (and as @dmitshur said, there must be an HTTPS server there pointing to wherever the repository is located). This constraint makes a lot of things simpler. Packages can only be imported by one name, and there's much less risk the same package being built multiple times.

replace directives don't seem like the right solution for this. There are a few limited use cases for them (e.g., a short-term fork of an upstream module), but it's better to avoid them if at all possible. Keep in mind that replace directives are only effective for builds in the main module, so adding them is very likely to break downstream users.

Please see When should I use the replace directive? on the module wiki for more information on replace.

See Remove import paths for more information on custom imports. These are popularly (but not officially) called "vanity imports", and there are some articles on setting that up. You definitely need to serve https://go101.org/tinyrouter?go-get=1 though.

[dmitshur]

Here, the go101.org could be anything other than github.com, for example, website.noexist/tinyrouter. I never prepare to create a http service to response the https://go101.org/tinyrouter?go-get=1 request. Is this not ok?

It's not okay if you want users to be able to install the dependency from the internet. To do that, they would use go get on the canonical import path of your package. If they did go get website.noexist/tinyrouter, it would fail because https://website.noexist/tinyrouter?go-get=1 would not return the expected go-import meta tag.

That's why the import path should be a URL that you have control over (such as go101.org/tinyrouter), so that you can serve the expected go-import meta tag from it.

[go101]

@jayconrod

Keep in mind that replace directives are only effective for builds in the main module, so adding them is very likely to break downstream users.

I did run the go get command under the path of main module. My intention of running this command is I hope this command will add a replace line in the go.mod file of the main module.

Do you mean I should not place a replace line in the go.mod file of tinyrouter package? This is not my intention.

@dmitshur

If they did go get website.noexist/tinyrouter, it would fail because https://website.noexist/tinyrouter?go-get=1 would not return the expected go-import meta tag.

I just think it is totally unnecessary to let go get makes a request to https://website.noexist/tinyrouter?go-get=1. When it gets the responses from by calling github.com/go101/[email protected], it has succeeded to get the go.mod file of the package. The go.mod file clearly specifies that the canonical import path of the package is website.noexist/tinyrouter. The information is ~efficient~sufficient, why will the command still make a request to https://website.noexist/tinyrouter?go-get=1?

[go101]

sorry, I use a wrong word in the last comment, "efficient" should be "sufficient".
I mean what go-import meta tag does the go get command expect to get?

[go101]

I just hope

go get github.com/go101/[email protected]

is a shortcut for

go mod edit -replace=go101.org/tinyrouter=github.com/go101/[email protected]

I think this a reasonable hope.

[jayconrod]

@go101 When a user imports go101.org/tinyrouter, the Go command needs to find the location of that repository, so it sends a request to https://go101.org/tinyrouter?go-get=1. That allows people to "register" unique module paths by serving that URL, using existing DNS and HTTPS infrastructure.

If it didn't work this way, anyone would be able to write module go101.org/tinyrouter in their module. When someone imports go101.org/tinyrouter, we'd have no way to know which one was intended. Replace directives are not intended for this purpose.

[go101]

Then why doesn't go build also send a request to https://go101.org/tinyrouter?go-get=1?

[go101]

Ok, I think I got it.

The difference is, one is "initiative", and the other is "passively".