Go: cmd/go: go mod init fails to determine module path in subdirectory

[andig]

Please answer these questions before submitting your issue. Thanks!

What version of Go are you using (go version)?

1.11

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

❯ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/andig/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/andig/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/Cellar/go/1.11/libexec"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.11/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/andig/htdocs/goelster/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/73/89ycv7qn51j4kbm04jsz9b840000gn/T/go-build453007721=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Trying to init go modules, I'm seing it fail from one folder but working from another. Both folders only contain two simple go files:

~/htdocs/canprogs/go master*
❯ go mod init
go: cannot determine module path for source directory /Users/andig/htdocs/canprogs/go (outside GOPATH, no import comments)

~/htdocs/goelster master
❯ go mod init
go: creating new go.mod: module github.com/andig/goelster

❯ ls
elster.go main.go

[bcmills]

Please attach the output of go env (as requested by the issue template).

[bcmills]

Which directory (if any) is at the root of your VCS checkout, and what is your GOPATH?

[andig]

Please attach the output of go env

Sorry, only noticed the "operating system" question, added go env above. ~/htdocs/canprogs is a git root, so is ~/htdocs/goelster. Neither is part of the gopath.

[andig]

@bcmills I see this has been scheduled- seems I've hit an actual issue here?

[xswordsx]

Have you tried go mod init <module_name>? _(e.g. go mod init goelster)_

[andig]

That creates a go.mod:

❯ go mod init goelster
go: creating new go.mod: module goelster

❯ cat go.mod
module goelster

is this the expected behavior?

[myitcv]

go mod init with no arguments does it's best to determine a module path if the current directory belongs to a VCS repository that has a remote:

$ cd $(mktemp -d)
$ git init
Initialized empty Git repository in /tmp/tmp.T8elQyoqel/.git/
$ git remote add origin https://github.com/myitcv/gobin
$ go mod init
go: creating new go.mod: module github.com/myitcv/gobin

You can alternatively provide an explicit argument, as you outlined in https://github.com/golang/go/issues/27951#issuecomment-429533293.

The bug you correctly identified is that go mod init with no arguments does not do the correct thing in a subdirectory of the VCS repo root:

$ mkdir inner
$ cd inner
$ go mod init
go: cannot determine module path for source directory /tmp/tmp.T8elQyoqel/inner (outside GOPATH, no import comments)

[rfay]

It seems to me that go mod init is completely dependent on the existence of a git remote named "origin". I got the OP's output here because I only had the origins "upstream" and "rfay". Once I added an "origin" (equivalent to upstream), then go mod init worked. This doesn't seem like the behavior I'd expect.

[myitcv]

This doesn't seem like the behavior I'd expect.

@rfay - what behaviour would you expect if there are multiple remotes?

Having go mod init specifically look for a remote called origin seems like a reasonable starting point on the basis:

• this is the default remote name used by GitHub and Gitlab
• it allows things to work where there are multiple remotes (else, which remote would it choose?)

[rfay]

The remote name "origin" is absolutely arbitrary (although ever-so-widely used), and should not be a fundamental assumption. Just a --remote= arg would solve the problem and make it explicit that go mod init is dependent on the remote (and on a specific named remote). It would be fine for "origin" to be the default.

[myitcv]

@rfay

should not be a fundamental assumption
...
Just a --remote= arg would solve the problem and make it explicit that go mod init is dependent on the remote

go mod init is documented as follows:

usage: go mod init [module]

Init initializes and writes a new go.mod to the current directory,
in effect creating a new module rooted at the current directory.
The file go.mod must not already exist.
If possible, init will guess the module path from import comments
(see 'go help importpath') or from version control configuration.
To override this guess, supply the module path as an argument.

Therefore, I don't think it's accurate to describe go mod init as being dependent on the remote configuration. Nor do I think we can describe the searching for a git remote called "origin" as a fundamental assumption. With no module path provided, go mod init's behaviour is, at best, a guess. And if the guess is bad you can supply the module path.

go mod init with zero arguments is not, I suspect, going to be a command that people run every day. Indeed I suspect the most common use of go mod init with no args will be converting from another dependency management solution to Go Modules.

Hence to my mind the current behaviour of go mod init is good enough. More flags, config options etc run the risk of bloating what is a very basic command.

But I'll leave this for @rsc and @bcmills to opine.

[rfay]

It's good enough for people who already know the idiosyncrasy, not good enough for n00bs like me coming from dep. There will be lots of n00bs.

[ptolemy428]

Being new to go mod, I run into the exact same issue today. It did take me a while googling, with the right combination of key words to finally land in here. The documentation makes sense to me and I'm OK with the implicit dependency of go mod and git. However, I'd suggest (at least) make the error message more meaningful or explicit. It will save quite a bit of time for n00bs. Cheers,

[joshuamkite]

This has bitten me for the second time today. Needs a meaningful error message at the very least

[quite]

The original issue here seems to rely on the "last resort" guessing the module import path, by detecting a git repo that has a remote named origin pointing to github.com (see: modload/init.go#L562).

What makes it work in one folder and not the other, is because the detection looks in the current directory for .git/config, not recursively up the directory hierarchy (as the git tool does).

I agree that instead extending and complicating (bloating, as @myitcv says) the guesswork that go mod init does, the way to go is to provide more expressive and helpful message that can point the novice user in the right direction. Otherwise, should it be tough how to recurse for .git/config, and then further to handle other the VCS (hosting) systems that Go supports elsewhere?

I find this helpful: https://github.com/golang/go/wiki/Modules#quick-start
But a message should probably not link that?

Should go help mod init be more verbose? I don't find it very helpful currently. Then the message could point to that.

[jayconrod]

I changed the wording of the message in CL 166319 to fix #30678 (which I think is a duplicate of this). It now says "cannot determine module path for source directory %s (outside GOPATH, module path not specified)". This suggests that a module path should be specified explicitly without being too wordy.

Is this enough info, or should go help mod init be expanded as well? It's pretty brief, but I don't think anything important is missing.

It would be feasible to walk up parent directories, looking at .git/config, but in my opinion, it's too magical, especially for modules that aren't at the repo root, which require a great deal of care and consideration.

I don't think other VCS support can be added here; the .git/config special case is just for GitHub, not even git in general.

[mxork]

maybe a hint towards the correct command?

cannot guess module path for source directory ...
try specifying module path explicitly: 

    go mod init <PATH>

I'd :+1: for expanding go help mod init as an alternative.

[iWdGo]

go get is supporting several VCS including other git platforms. It is unclear from the online documentation about modules and its FAQ answer that support seems currently limited to github as the regex and its use above implies. Mentioning the current limit would improve the understanding of the message.

[bcmills]

@iWdGo, to be honest, I'm not sure that inferring github.com paths is really the right behavior either. Some of those are served through other import paths (via redirects and/or <meta> tags), so go mod init can very easily pick the wrong path.

The right solution might be to remove the github.com special-case and always require an explicit path when outside of GOPATH.

[iWdGo]

I agree that best effort to locate path is confusing and removal seems most appropriate.

A default value set to the folder name where the module resides could be considered but might be the subject of a separate issue. The behavior would become similar to the $GOPATH related code which sets a value only when the module resides in $GOPATH/src/<module-path>

The discussed message would be irrelevant and could be replaced with "default module path set to ".

[rsc]

Killing the github.com special case seems fine. I would like to leave all the other, better, higher-priority auto-deriving of module paths, since those are much more accurate and helpful.

[gopherbot]

Change https://golang.org/cl/172019 mentions this issue: cmd/go: remove auto-deriving module path for github.com

[joshuamkite]

I have been able to resolve this with code in a repo subdir with

go mod init `pwd`

Surely this can be automated?!

[SunSparc]

I have been able to resolve this with code in a repo subdir with

go mod init `pwd` 

This solved it for me! Thank you @joshuamkite!!

[qhkm]

I have been able to resolve this with code in a repo subdir with

go mod init `pwd` 

Surely this can be automated?!

This solved for me too. Thanks!!

[MrTrans]

why is go mod init pwd solved? why?

[bcmills]

go mod init `pwd`​ is highly unlikely to be correct: the argument to go mod init is a module path, but pwd is generally an absolute filesystem path. (See #28389.)

If the go command cannot determine an appropriate module path, the general solution is no pass one explicitly. See https://golang.org/cmd/go/#hdr-Defining_a_module and https://blog.golang.org/using-go-modules.