Go: x/tools/gopls: support module-local references

[anjmao]

What version of Go are you using (go version)?

[Info  - 3:24:12 PM] Build info
----------
golang.org/x/tools/cmd/gopls v0.1.1-cmd.gopls
    golang.org/x/tools@(devel)
    golang.org/x/[email protected] h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=

Go info
-------
go version go1.12.6 darwin/amd64

GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/anjmao/Library/Caches/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOOS="darwin"
GOPATH="/Users/anjmao/go"
GOPROXY=""
GORACE=""
GOROOT="/usr/local/go"
GOTMPDIR=""
GOTOOLDIR="/usr/local/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/anjmao/s/hugo/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/jc/q8b4yjwd5dvgc42xpg6yvm8c0000gn/T/go-build904784213=/tmp/go-build -gno-record-gcc-switches -fno-common"

Does this issue reproduce with the latest release?

Yes

What did you do?

In Vscode I used Find All References.

What did you expect to see?

See all usages.

What did you see instead?

I see usages only in the file in which objects are defined.

[suzmue]

Could you provide more information about this issue? Does this happen with all identifiers or only certain ones?

[suzmue]

Find all references only supports in package references right now, so if the uses are in a different package, it will not be able to find them.

[anjmao]

@suzmue Yes it works only in a single package, thanks. Maybe it make sense to link this issue to gopls wiki Known Issues as this behaviour is a bit confusing.

[arthurkiller]

same as me

[jadekler]

Also seeing this (see: https://github.com/microsoft/vscode-go/issues/2665)

[henvic]

I just learned that [outside gopls], if you want to quickly find all references of something on a given path, you can use:

go list -f '{{$pkg := .}}{{range .Deps}}{{if eq . "encoding/json"}}{{$pkg.ImportPath}}{{end}}{{end}}' ./...

[ochaloup]

I struggle with the same (the 'Find all references' or peek them with SHIFT+F12) shows only the method itself or the usage in the local file.
On another issue, I found that switching off the golang language server may help, see https://github.com/Microsoft/vscode-go/issues/1473#issuecomment-358990901
By disabling this in the golang extension the references are searched over all files in my workspace.
Still, I'm not sure what other features I'm loosing by disabling the language server.

[zhangpy]

I struggle with the same (the 'Find all references' or peek them with SHIFT+F12) shows only the method itself or the usage in the local file.
On another issue, I found that switching off the golang language server may help, see microsoft/vscode-go#1473 (comment)
By disabling this in the golang extension the references are searched over all files in my workspace.
Still, I'm not sure what other features I'm loosing by disabling the language server.

How to disable this feature (it seems I have to enable gopls.)?
(My vscode looks like a mess since enable module. - gopls is not ready for this change, unbelievable!)

[stamblerre]

We are currently in the process of adding support for this feature. The language server is designed to be more performant for modules, so turning it off will be fine if you are in GOPATH mode, but it may result in degraded performance if you are using modules.

@zhangpy: If you can file an issue with a repro case and the information requested here, I would be happy to investigate it.

[zhangpy]

We are currently in the process of adding support for this feature. The language server is designed to be more performant for modules, so turning it off will be fine if you are in GOPATH mode, but it may result in degraded performance if you are using modules.

@zhangpy: If you can file an issue with a repro case and the information requested here, I would be happy to investigate it.

Thanks for the reply. I could wait until gopls is ready, thanks.

[stamblerre]

This has been complete as of a few weeks ago. Closing.

[arthurkiller]

good job

[roopakv]

@stamblerre is there a timeline on when this will make its way to the gopls installed from vscode?

[chengyayu]

@stamblerre I have used the gopls at master, it works. thanks. hope it will take effect in vscode soon.

[magnetikonline]

From what I see testing right now, it "just works".

Restarted VSCode - was even told a gopls update was available (handy!) - hit Update - gopls binary was re-compiled and we're good! Can now "Find all references" across packages in a (large) test project. And wow - it's _very_ fast vs. the previous guru reference crawl method. 👍

If you don't get the Update popup - select Go: Install/Update Tools from the commandpalettee to recomplile that way your full VSCode Golang util toolchain.

Edit: actually, scrap that - it _seems_ better (picks up more references) - but isn't catching them all..... ~Certainly _does_ seem better though to last time I tested this a few months back.~ ... actually, scrap all that - it's still just finding references A) in the current package B) the source function/method signature. 😄 As @stamblerre says, maybe waiting a few weeks for a tagged release of gopls to hit which has the bugs ironed out!

Edit: for those that are interested, this is how I'm using the language server right now _except_ for "Find all references" (e.g. uses guru) - which is slower, but reliable in results (note: "findReferences": false) . Added to settings.json:

"go.languageServerExperimentalFeatures":{
    "autoComplete": true,
    "diagnostics": true,
    "documentLink": true,
    "documentSymbols": true,
    "findReferences": false,
    "format": true,
    "goToDefinition": true,
    "goToImplementation": true,
    "goToTypeDefinition": true,
    "hover": true,
    "rename": true,
    "signatureHelp": true,
    "workspaceSymbols": true,
},

[stamblerre]

The next gopls release will likely not be for another few weeks, which is when VS Code will pick it up. In the meantime, you can certainly try it out at master, but do be warned that master is not very stable and may be broken at times.

[roopakv]

ok n00b question : what would be a good way to install gopls from master?

Update: found it https://github.com/golang/tools/blob/master/gopls/doc/user.md

[stamblerre]

To install gopls at master, you will have to clone the golang.org/x/tools repo, cd into the tools/gopls directory, and then run go install.

[magnetikonline]

Gave this a _very quick_ go ☝️ - still only finds all references from within the current package - not workspace/project wide with gopls compiled from master. Maybe waiting for the next VSCode Golang extension release is the best course of action 👍

[stamblerre]

@magnetikonline: This should work at master. If it's not working, please file a new issue with a repro case, and we can help investigate. It would also be helpful if you attach the output of the gopls references command.

[magnetikonline]

Will do @stamblerre - it's more than likely my user error - I did try moving my (largish) non-gomodules test project in/out of GOPATH thinking that might have been the cause.

Outside of GOPATH, it didn't work at all - as expected - bit thought it might have been the trigger. Might also try to mock up a really simple project with a few packages to confirm as well.

[stamblerre]

gopls looks at your workspace root to set the scope for the references, so it will only find things within your workspace root. In GOPATH mode, you will need to open $GOPATH/src if you want to find a variable in multiple projects. With modules, it should work to find a variable within the scope of your module (as long as you open your workspace to the root of your module).

[magnetikonline]

Yeah I'm pretty sure I'm doing all that. I'll give it a crack over the weekend and raise an issue if no luck. Thanks!

[problame]

I installed 6e064ea0cf2dcfaca8da2da9f42da458bab38756(contains commit caa0b0f7d508201474af7e7db3ebee9b0fc6bd37 which I think is the relevant one for this issue) and when cd internal/lsp/testdata/references && code ., and right-clicking 'find all references' on refs.go:15 var Q string //@mark(declExpQ, "Q"), refs("Q", declExpQ, assignExpQ, bobExpQ), only that very line shows in the results (i.e. only one result). Despite references.Q being used in other/other.go.

[stamblerre]

@problame: Can you try reproducing in a real Go project? Our testdata directory is not a good place for repro examples, as the testing framework does extra work to set up the directory structure. You will notice that gopls shows an error in internal/lsp/testdata/references/other/other.go, since the import in that file is not actually valid.

[magnetikonline]

Hello @stamblerre I played with this some more just now!!!:

• Compiled gopls against commit https://github.com/golang/tools/commit/6e064ea0cf2dcfaca8da2da9f42da458bab38756.
• Not sure what I need to pass as "position" to ~/go/bin/gopls references.
• Created a test go module where:
  
  
  
  • Root package references a function in sub package A.
  
  
  • Root package references a function in sub package B.
  
  
  • Sub package B references a function in sub package A.
  
  

Sorry if this next bit is confusing!

This is what I see with "default" gopls and "Find all references":

• Find sub package A function reference from root package returns A) root package (itself) and B) sub package A function signature.
• Find sub package A function references from _within_ sub package A returns only _itself_ - the function signature.
• Same results from sub package B - for the related B package function.

Then switching to gopls 6e064ea0 is better, but not perfect....:

• Find sub package A function reference from root package returns A) root package (itself) and B) sub package A function signature. No change from default/release build.
• Find sub package A function references from _within_ sub package A returns _everything_ 🎉 A) root package B) sub package A reference and C) sub package B reference!
• Find sub package A function references from _within_ sub package B returns _everything_ 🎉 A) root package B) sub package A reference and C) sub package B reference!
• Find sub package B function references from _within_ sub package B returns A) root package B) sub package B reference.

So... right now if I do a "find all references" from within the sub/child packages it finds _everything_ - but from the root package, no change.

Does that make sense? Sorry for adding more noise to this thread - if you like I can port this over to a new issue. Certainly gopls is getting better in this regard - but still not quite _100%_.

[problame]

@stamblerre

@problame: Can you try reproducing in a real Go project? Our testdata directory is not a good place for repro examples, as the testing framework does extra work to set up the directory structure. You will notice that gopls shows an error in internal/lsp/testdata/references/other/other.go, since the import in that file is not actually valid.

It's reproducible in: https://github.com/zrepl/zrepl/blob/problame/resumable-and-encrypted-send-recv/zfs/holds.go#L78

ZFSReleaseAllOlderAndIncludingGUID is used in other packages within the same module (=repo in this case), but those use-sites are not found by Find All References.

[stamblerre]

I filed https://github.com/golang/go/issues/35936 to continue this discussion.

@magnetikonline: The position arguments to gopls references should be of the format filename:line:column. Thanks for the detailed report. If you could provide a precise repro case that might be most helpful, but for now, we can start by taking a look at the one that @problame provided.

@problame: Thanks for that report - we'll take a look (I was able to reproduce).

[magnetikonline]

Not a problem @stamblerre - if you do have a need for that, let me know. But it seems there is a nice workable case with what @problame has shown, already in a repo.

[magnetikonline]

Hello @stamblerre I see version 0.2.2has landed, but I'm still seeing the "Find references" isn't finding all of them in my test repo and results - behavior has changed though, somewhat for the worst.

I'll outline with some code this time - hope this helps, and sorry for all the fruit package names!

// main.go

package main

import (
    "fmt"

    "magnetik.test/hello/apple"
    "magnetik.test/hello/orange"
)

func main() {
    fmt.Println("testing")
    fmt.Println(apple.WhatColorApple())
    fmt.Println(orange.WhatColorOrange())
}

// apple/main.go

package apple

func WhatColorApple() string {
    return "it's a red apple"
}

// orange/main.go

package orange

import (
    "magnetik.test/hello/apple"
)

func WhatColorOrange() string {
    return "it's an orange orange, but also have... " + apple.WhatColorApple()
}

• Find apple.WhatColorApple() from main.go finds itself and apple/main.go (the definition), but not orange/main.go reference.
• Find orange.WhatColorOrange() from main.go finds itself and orange/main.go (the definition) - which is complete.
• Find WhatColorApple() from apple/main.go only finds itself (the definition), not main.go - this is a regression.
• Find WhatColorOrange() from orange/main.go only finds itself (the definition), not main.go - this is a regression.
• Last one is interesting/change - find apple.WhatColorApple() from orange/main.go finds from apple/main.go (the definition) and orange/main.go (itself) - not main.go.

$ gopls version
golang.org/x/tools/gopls 0.2.2
    golang.org/x/tools/[email protected] h1:ujGisyytgY1VGcmd66wIJ9+wVAfmodXj6daHM43HRXk=

Hope that helps!

[stamblerre]

@magnetikonline: v0.2.2 is a patch release, so it doesn't contain the changes made on master. It only had a few minor bug fixes, so I would not expect any difference of behavior from the previous tagged version. The next version with the changes for find references will be tagged v0.3.0. Still, thank you for the detailed report!

[magnetikonline]

@stamblerre ugh, my bad! 🤦‍♂ - thanks for clarifying that! I'll check in once v0.3.0 drops.

[krpn]

@stamblerre, why does not v0.3.0 drop? :)

[stamblerre]

It'll be out as soon as we work through the issues here: https://github.com/golang/go/milestone/129. I will post on https://github.com/golang/go/issues/33030 when it is released.

[yoesmat]

"go.languageServerExperimentalFeatures":{
    ...
    "findReferences": false,
    ...
},

This didn't work for me, it still used the language server.

[magnetikonline]

@yoesmat that's because the options offered with languageServerExperimentalFeatures has been cut right back with the last VSCode Golang extension release at the very start of the year, so you can't do this anymore.

Only options are to disable language server entirely, or wait for 0.3.0 of gopls

[yoesmat]

Thanks @magnetikonline

[magnetikonline]

Hello @stamblerre - I've eagerly installed v0.3.0 when I saw the release announcement yesterday - and can say that the issues I identified in the comment above have now been full resolved!! 🎉

$ gopls version
golang.org/x/tools/gopls 0.2.2
    golang.org/x/tools/[email protected] h1:ujGisyytgY1VGcmd66wIJ9+wVAfmodXj6daHM43HRXk=
$ GO111MODULE="on" go get golang.org/x/tools/[email protected]
$ gopls version
golang.org/x/tools/gopls v0.3.0
    golang.org/x/tools/[email protected] h1:l9KKK1/n6CIbfgaUvHBWAvCfOxcl1N+KSOK79OlPIao=

Then from VS Code restart/restart language server. All those "Find all references" actions work exactly as expected.

Thanks for being so active on this topic and great work on the v0.3.0 release. Golang support with VS Code has just improved dramatically for it.

[stamblerre]

Glad to hear the update's working well for you, and thanks for following up!