Go: how to add local module to vendor?

[yanyandenuonuo]

What version of Go are you using (go version)?

$ go version
go version go1.11.1 linux/amd64

Does this issue reproduce with the latest release?

yes, and i used the latest version

What operating system and processor architecture are you using (go env)?

go env Output

$ go env


GOARCH="amd64"

GOBIN="/usr/local/go/bin"

GOCACHE="~/.cache/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOOS="linux"

GOPATH="$Workspace/go"

GOPROXY=""

GORACE=""

GOROOT="/usr/local/go"

GOTMPDIR=""

GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"

GCCGO="gccgo"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build913779639=/tmp/go-build -gno-record-gcc-switches"

What did you do?

1. create new project
2. execute go mod init
3. modify go.mod

exclude privaterepo.com/bb/bb

1. copy my local module to vendor because the local module is on a private repo which not support https.
2. now the vendor just like:

|-github.com/aa/aa
|-privaterepo.com/bb/bb

1. import "privaterepo.com/bb/bb"
2. execute go build -mod vendor
3. than i got error "cannot find module for path privaterepo.com/bb/bb"
4. i always try with replace, but it also not work

replace privaterepo.com/bb/bb v0.0.0 => ./vendor/privaterepo.com/bb/bb

What did you expect to see?

I just want go mod support i add local module and build success.

What did you see instead?

Before i used go dep which support ignored can solve the problem. Now go mod exclude not work for this.

[yanyandenuonuo]

i had read some issue, but none of them solve the problem.

[agnivade]

@bradfitz - Are we not closing questions now ? From the latest comment on https://github.com/golang/go/issues/23745, it seemed that we were waiting on the questions repo to be created.

[bcmills]

The vendor directory is not intended for adding arbitrary modules. Use a replace directive instead.

See also #28835.

[bcmills]

i always try with replace, but it also not work

What about it doesn't work?

[yanyandenuonuo]

@bcmills I try add replace privaterepo.com/bb/bb v0.0.0 => ./vendor/privaterepo.com/bb/bb, but when i execute go mod vendor, it will remove the module.
If i don't do this and direct execute go build -mod vendor main.go, it failed and say this is cannot find module for path privaterepo.com/bb/bb.
Why not support ignored for go mod just like go dep?

[yanyandenuonuo]

@bcmills Most of my friends write go in commercial project, some of depence common modules is place in private vcs just like privaterepo.com/xxx/xx, and cause some reason the vcs not support https or need auth when they use https. So, maybe go dep should support ignored and go get support ssh protocol for most of gopher who with privaterepo depence.

[bcmills]

I try add replace privaterepo.com/bb/bb v0.0.0 => ./vendor/privaterepo.com/bb/bb, but when i execute go mod vendor, it will remove the module.

Did you also add require privaterepo.com/bb/bb v0.0.0? go mod vendor only vendors in modules that are actually in the requirement graph. (See #26241.)

[bcmills]

and cause some reason the vcs not support https or need auth when they use https

Configuration of private repose is independent of vendor. See #26134 and related issues.

[yanyandenuonuo]

@bcmills You are right, i try to add

require privaterepo.com/bb/bb v0.0.0
replace privaterepo.com/bb/bb v0.0.0 => ./vendor/privaterepo.com/bb/bb

than i can build with go build -mod vendor main.go.

But when i require new module in go.mod and i need to execute go mod vendor for copy the module to vendor folder, i got new error go: parsing vendor/privaterepo.com/bb/bb/go.mod: open ~/ProjectPath/vendor/privaterepo.com/bb/bb/go.mod: no such file or directory. I only can modify my project and has no permission to modify the privaterepo.com/bb/bb. As I known, the project privaterepo.com/bb/bb use go dep up to now.

So i think there has three way to fix this case:

1. go mod vendor won't check the package if there exist reaplace and the uri is local file just like ./vendor/xxx or ./xxx.
2. go mod exclude support ignored which means the package manager shouldn't check the module in exclude.
3. go get support ssh protocol, so the private package can import with ssh.

[yanyandenuonuo]

@bcmills I try to add go.mod in vendor/privaterepo.com/bb/bb/go.mod with:

module bb

than i execute go mod vendor, the vendor/privaterepo.com/bb/bb gone.

[bcmills]

I would not expect go mod vendor to work if the replace directive itself points into the vendor directory. The replace directive should point to a copy of the repository somewhere else in the filesystem (in which case you can add a go.mod file there).

[yanyandenuonuo]

@bcmills So ho to vendor a local module?
There has more than one developer in one project, i don't think there exist some way to fix this case.
As your words, all of developer should put the private module in same path at each computer, or put the private module as a common lib such as ./bb other than a package in ./vendor/privaterepo.com/bb/bb.

[yanyandenuonuo]

For now, i turn back to use go dep and use ignored to keep the private module in vendor.
I think go mod should support such of case to manage private module in vendor.
network, security, business and so on, there is too much limit in real work situation, i think most of gopher had go through some of them.
As a official tool, i think go mod should support this case for the gopher just like me.

[bcmills]

So [how] to vendor a local module?

Vendoring and local modules are two separate things.

You can put a local module in a directory within the same repository, or you can put it somewhere in the local filesystem, or really any place that isn't in the vendor directory. If you run go mod vendor within your main module, it will also be copied to the main module's vendor directory.

[yanyandenuonuo]

1. In fact , vendor is used as a go package manager for most of gopher.
2. local module is a special package which has private permission for some reason.
3. "You can put a local module in a directory within the same repository", i don't think it's a reasonable and graceful to handle the special package.
4. I don't known the real num who had meet similar situation, but i really heared so much gopher complain similar case.

If the official don't want go dep support such case, so you can close the issue.

[bcmills]

Precision is important.

• vendor is not a “package manager” in the usual sense: it does not understand versions, let alone resolve them.
• A “local module” (to be precise, a “replacement module”) is a _module_, not a _package_. A module is a collection of packages.
• In general I would expect private modules to be served via private VCS servers, private Go proxies, or some combination of the two. replace is not a long-term solution for that, nor is vendor.
• “reasonable” and “graceful” are subjective judgments, not technical ones. We try to focus on addressing concrete problems.
• At this point, dep is an independent project from Go modules. go dep is not a command that exists today, nor do I expect that it will in the foreseeable future.

[echopairs]

@bcmills Most of my friends write go in commercial project, some of depence common modules is place in private vcs just like privaterepo.com/xxx/xx, and cause some reason the vcs not support https or need auth when they use https. So, maybe go dep should support ignored and go get support ssh protocol for most of gopher who with privaterepo depence.

1. we can make commom private golang module as our project sub repo with (git module/git subtree)
2. edit our project go.mod by replace.

[yanyandenuonuo]

@echopairs

1. we can make commom private golang module as our project sub repo with (git module/git subtree)

Most developer who use depence common modules may not have the permission to make the common private golang modules for some reason:

• Code sync
• Security
• Others like historical, company rule ...

I had give up to use go mod in the project and keep on use go dep. May be i will use go mod in new project which doesn't need import private modules.