go get downloads all directories except the directory specified on bitbucket

[dwschulze]

What version of Go are you using (go version)?

$ go version
1.14

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

What did you do?

In my git repo (bitbucket.org) I have

reponame/
    .git/
    integration1/
    integration2/
    integration-common/
    └── util
        ├── go.mod
        └── readascii.go

The module directory is a subdirectory of the git repository. The repository is tagged with v0.0.1.

Here's the go.mod

    module bitbucket.org/orgname/reponame/integration-common/util
    go 1.14

When I do go get for this module I get

go get bitbucket.org/orgname/reponame/integration-common/[email protected]
go: downloading bitbucket.org/orgname/reponame v0.0.1
go get bitbucket.org/orgname/reponame/integration-common/[email protected]: module bitbucket.org/orgname/[email protected] found, but does not contain package bitbucket.org/orgname/reponame/integration-common/util

I can see the integration-common/util/ directory in bitbucket if I look with the web browser. The go.mod has the contents shown above.

What is even more odd is that go get does manage to download the other directories in the repository (integration1 and integration2). For some reason it thinks the v0.0.1 tag contains all the other directories except for the integration-common/util directory, even though bitbucket shows it there for that tag.

I download with ssh, not https, because https reqires 2fa on our bitbucket repo. I have this in my ~/.gitconfig:

`[url "[email protected]:"]insteadOf = https://bitbucket.org/`

I know bitbucket is weird when it comes to using go get. Is there something I missed, or is the a bug with go get?

What did you expect to see?

I wanted to see one directory from a repo pulled down.

What did you see instead?

go get pulled all the other directories instead of the one specified

[dwschulze]

Following the third paragraph of the docs I created the tag

git tag integration-common/util/v0.0.1

and verified that it got pushed to bitbucket. When I go get the tag it shows

go get bitbucket.org/orgname/reponame@integration-common/util/v0.0.1
go: bitbucket.org/orgname/reponame/integration-common/util/v0.0.1 => v0.0.1

but nothing gets downloaded. A silent failure.

I also tried this

go get bitbucket.org/liqid/northbound/common-liqid-integration/lctlutil/@v0.0.1

but it returns immediately and does nothing.

[mvdan]

Have you tried asking this as a question first? https://golang.org/wiki/Questions

It's likely this has something to do with the way you set up the module(s). You could also try using the latest version of Go.

[bcmills]

@dwschulze, the argument to go get is in general a module path and version, not a repo and tag.

The command you are probably looking for here is

go get bitbucket.org/orgname/reponame/integration-common/[email protected]

with the desired commit tagged integration-common/util/v0.0.1 (per https://golang.org/ref/mod#vcs-version).

[bcmills]

That said, in general we recommend only one module per repo. (Is there a particular reason you need to have the util package in a separate module?)

[dwschulze]

I've tried the command

go get bitbucket.org/orgname/reponame/integration-common/[email protected]

with the tag integration-common/util/v0.0.1 and it doesn't download anything.

One module per repo could lead to proliferation of repos, which we could probably handle. But what do groups that put everything into one repo do with modules? (I've heard Google has one huge repo for all of their code which I find hard to believe.)

[bcmills]

In general it is fine to have all of your packages in one big module in one big repo.

(IMO the only really compelling use-case for multi-module repos is to be able to tag independent releases of packages within that repo. But that seems like a rare use-case, and often isn't worth the overhead of maintaining multiple modules.)

[dwschulze]

@mvdan - I've asked this question on both the golang-nuts forum and stackoverflow. Thanks for asking. I got one response that didn't fix anything.

As far as upgrading to a newer version of Go, was there a bug fix relevant to this issue? If not that sounds like the mindless advice you get by calling an 800 number for tech. support for a childrens computer game.

This is a serious problem and I need to either get a bug fixed or find out if it is a documentation problem.

[bcmills]

As far as upgrading to a newer version of Go, was there a bug fix relevant to this issue?

There have been fixes in the module system in every release since Go 1.11. Many of the recent improvements are specifically to provide clearer diagnostics.

Also note that the issue template specifically asks “Does this issue reproduce with the latest release?”, and the latest supported releases are go 1.15.4 and go 1.14.11 respectively. An issue posted for go 1.14 (without a minor release) does suggest a reasonable likelihood that _something_ about the issue may have changed in the ~year since the development freeze began for the initial 1.14 release.

Finally, note that our Gopher values include to “be patient” and “avoid … snarking”. We have the common goal of making Go development simple and productive. I understand that you're feeling frustrated, but please keep comments constructive.

[mvdan]

Thanks for having asked the question elsewhere. A significant portion of modules issues raised here are essentially questions phrased as bug reports, so that's why I tend to ask.

And, like Bryan says, trying the latest stable release is indeed part of the bug reporting process. Go 1.14.x is still supported, but you should still check if your problem has been solved in 1.15.x (or even the upcoming 1.16, once beta1 is out in a few weeks).

[dwschulze]

I added the -v to go get but it gives no output and downloads the same thing (all the subdirectories except the one I'm asking for)

$ go get -v bitbucket.org/orgname/reponame/integration-common/util/@v0.0.1

Something unusual is that that the downloaded dirs/files have permission 500.

The directory I'm trying to get was created on a branch as was the tag v0.0.1. That directory doesn't exist on any other branch. I can see the directory under the tag on the bitbucket UI. My understanding is that tags are independent of branches.

[bcmills]

@dwschulze, what is the output of go list -versions -m bitbucket.org/orgname/reponame for your repo? The symptom you are describing is consistent with the tag not existing.

You could also try using a totally different version for the nested module to remove any ambiguity. (What happens if you apply the tag integration-common/util/v0.0.2 and then try the same steps with version v0.0.2 instead of v0.0.1?)

[bcmills]

I could also imagine that you might be running into issue #37438, which is fixed at head but not in any prior release.

(You could confirm that one way or the other by trying the same steps using golang.org/dl/gotip or a go command built from source at head.)

[dwschulze]

I upgraded to go v1.15.4. Same results.

$ go list -versions -m bitbucket.org/orgname/reponame
bitbucket.org/orgname/reponame

I added the tag integration-common/util/v0.0.2 and pushed. I can see it in bitbucket with my directories and files in it. When I do

go get -v bitbucket.org/orgname/reponame/integration-common/util/@v0.0.2

it downloads [email protected] however with all the directories except the one I'm asking for.

I deleted the v0.0.1 tag

git push --delete origin integration-common/util/v0.0.1

and ran the same go get with @v0.0.2 and it still downloads [email protected].

Something is really haywire.

[bcmills]

When I do

go get -v bitbucket.org/orgname/reponame/integration-common/util/@v0.0.2

it downloads [email protected] however with all the directories except the one I'm asking for.

Ok, that really sounds like #37438. 😞

Try gotip or a go command built from head? I'm hoping it's fixed (I literally just spent the past ~month or more on it), but if not I'd rather know sooner than later.

[bcmills]

$ go list -versions -m bitbucket.org/orgname/reponame
bitbucket.org/orgname/reponame

That's possibly as expected? What about go list -versions -m bitbucket.org/orgname/reponame/integration-common/util? (If the nested path has its own go.mod file, it is its own module, distinct from bitbucket.org/orgname/reponame.)

[dwschulze]

$ go list -versions -m bitbucket.org/orgname/reponame/integration-common/util
bitbucket.org/orgname/reponame/integration-common/util v0.0.2

Makes sense since the go.mod is in the util/ directory. But v0.0.1 is gone so why should go get download it? Is it pulling the v0.0.1 off of the proxy even though I have bitbucket.org/orgname/* set in GOPRIVATE?

[bcmills]

0.0.1 is gone so why should go get download it?

Are you sure it is downloading, and not just pulling from the local cache? (That's in either $GOMODCACHE or $GOPATH/pkg/mod.)

If the repo is private, the public proxy can't possibly have the module cached. (The public proxy does not receive credentials or module content from the go command.)

[dwschulze]

I deleted the cache.
I installed gotip as you suggested and ran gotip get with v0.0.2 and got the same results. It downloads v0.0.1 which doesn't even exist in bitbucket.org and downloads all the directories except for the one I asked for.

Later today I'm going to setup a public bitbucket repo and try to reproduce this. I'll send you the repo once I get it set up and if it reproduces.

[dwschulze]

I took out all the require and replace statements in the go.mod and then go get started working. Whatever it was in go.mod that was causing the problem it makes no sense at all that go get would download every directory in the repo except for the one I was asking for.

I don't know if this is a bug or just an FAQ.

[bcmills]

@dwschulze, you didn't mention any require or replace statements in the original report!

That could certainly explain it, but we'd need more detail to figure out how that maps to the behavior you observed.

[gopherbot]

Timed out in state WaitingForInfo. Closing.

(I am just a bot, though. Please speak up if this is a mistake or you have the requested information.)