I'm using go version go1.12.5 darwin/amd64
and the latest gopls release and the latest go extension in VSCode.
So I'm working a a project with a rather weird project structure for a Go project. It's a mono-repo of sorts. Here is the basic structure:
<config_files>
services/
- go_service1
- go_service2
README.MD
I usually open up the entire workspace in VSCode when I work in this project. With that I've noticed a rather weird (or maybe not) behaviour from gopls
.
You see it's looking at code in the wrong mod versions. I've discovered that it's because it's reading a go.mod
file that were in the root of the workspace - which has since been removed. That go.mod
file contained an old version of a module. Even though I were actually working in go_service1
it preferred that one. I realized it's most likely because VSCode is invoking gopls
from the workspace level, since it works perfectly if i open up either of the service-folders in a separate window.
I didn't find any flags that I could set in gopls
. Is it possible to configure how gopls works with modules? To properly support working in monorepo's like this?
This issue were spawned from this discussion: https://github.com/microsoft/vscode-go/issues/2490
Right now, the best way to work with gopls
in VSCode is to use Workspace Folders per module. For every go.mod
file that you have, you should do "File" > "Add Folder to Workspace" and then add the module root.
Sure π, but are there any plans to implement a feature like this? I'm betting most editors have this issue, not just VSCode.
I am having a similar issue, and am currently using bingo
right now since it does support this.
Our monorepo has a similar setup as the issue reporters, where we have several services with their own go.mod
s, and most of them import another local library that contains common code for setting up logs and such.
root/
deployables/
service1/
go.mod
service2/
/go.mod
lib/
common/
go.mod
My experience has been that gopls
doesn't load the full repository, and then is unable to figure out where things are.
How can I explicitly specify the module root for a given project?
This depends on your editor - if you are using VSCode, this can be done via File -> Add Folder to Workspace.
@stamblerre thank you for quick reply. I understand and can work with the limitation of having one module per-hierarchy, but it does not appear to matter _unless_ the root folder is the one that has .mod
. My projects pull supporting files one step up and have go.mod
in {workspaceFolder}/src
subfolder. Usually, for things like running debug, I can set the root:
{
"version": "0.2.0",
"configurations": [
{
"name": "Launch",
"type": "go",
"request": "launch",
"mode": "auto",
"program": "${workspaceFolder}/src", // <- specify where Go code is located
"args": []
}
]
}
Is there a similar workaround for gopls
?
Yes, for gopls
to work correctly the root folder has to be the one that contains your go.mod
file. I do not believe there is any such workaround for gopls
, but we will ultimately try to resolve this bug to make this work more seamlessly.
If there is a code path you can point to Iβd love to take a stab at restoring my normality.
We still have to make some decisions about if this is the responsibility of gopls
or of the editor. More likely than not, we would start off by making this change in VSCode-Go rather than in gopls
.
fwiw @stamblerre the scenario I mentioned above is handled by https://github.com/saibing/bingo.
I guess I did mention that in my original comment.
@sint: The way that bingo does this is technically a "hack" (see https://github.com/saibing/bingo/commit/169f54c97a8b1dd98d84daead9d4515a411286d4). The module root is not necessarily the directory of the current file. A better solution would be to search for all directories containing mod files and notify gopls
about them, but we still haven't made a final decision on how we want to implement this.
I'm under vim+ale+gopls
and facing the same problem.
And I can not go to references
in the standard library, I'm not sure if this is the same problem.
We still have to make some decisions about if this is the responsibility of
gopls
or of the editor. More likely than not, we would start off by making this change in VSCode-Go rather than ingopls
.
@stamblerre I think it might be helpful to look at how other LSPs/editor combos handle this problem.
I've just run into this issue using coc.nvim + gopls. In case you're not familiar with coc it is a bridge between vim and a multitude of language servers, so for example, there's also coc-python.
With the python setup if you open a python file without cd'ing into the project root folder first the LSP gets confused about unresolved imports (https://github.com/neoclide/coc-python/issues/26) very similarly to this issue.
There is a somewhat hacky workaround to use "workspace folders" which requires users to place an .env
file into the project root and configure coc to look for it. Coc in turn is able to call the language server with the pwd and path set correctly as a result.
This is not a complete solution but seems directionally correct as vim unfortunately doesn't have a concept of a project folder.
A better solution would be to search for all directories containing mod files and notify gopls about them, but we still haven't made a final decision on how we want to implement this.
This seems very much the way to go. What do you think of a .gopls
?
I am assuming it's possible the same logic that Go uses to figure out how to build modules with submodules expressly defined (handling submodules with go.mod
files, rather than inferred) to create a set of subtreet contexts. Figure out which go.mod
subtree the _currently edited_ file belongs to when it is edited and change the context for go toolchain calls.
Another issue is program-scope linting. If I change inner-most modules I want to see which code paths that change turned red
. Assuming one can relate go.mod
contexts to each other, we could subsequently run the toolchain on the _root_ context or even recursively run it on the ancestors until reaching the _root_ for the currently open file to see the change impact up the tree.
Adding folders to the workspace instead of just honoring whats defined in go.mod
is extremely alienating
It just turns the VSCode workflow into a mess
Right now, the best way to work with
gopls
in VSCode is to use Workspace Folders per module. For everygo.mod
file that you have, you should do "File" > "Add Folder to Workspace" and then add the module root.
This sorted my "issue" out, which I guess wasn't really so much of an issue with the extension, just how I expected it to work
I also had to add a go.mod
file to my root project and my undeclared name
errors disappear. In my case it is with the test file, tested function is undeclared
(but test works well, the issue comes from VsCode).
Apart from the workaround in vscode described above, is there any solution we're looking at for the possible future?
We are working on better support for managing multiple module roots, but this probably won't be released soon. It is something that we are planning on and prioritizing, however.
I wonder if it'd be possible to at least provide a way for gopls to not modify the go.mod
file? I'm assuming it's gopls that's doing that, for example if I open a file in a sub-module, it'll change the root go.mod
Yep - this is possible if you use the newly-released Go 1.14 with a new gopls
configuration: "tempModfile": true
. There are more details in the release notes, but the relevant portion is:
Go 1.14 will support running the go command without modifying the userβs go.mod file. This new feature is used in gopls to show diagnostics and suggest edits in a userβs go.mod file rather than change the file without the user noticing. Currently, gopls will suggest removing unused dependencies and warn the user if there is a parse error in the go.mod file. Enable this behavior by using the go1.14 beta and setting "tempModfile": true in your gopls settings. (@ridersofrohan)
That's great, thank you!
thx alot
Is there a cheap workaround in the VSCode workspace settings like:
"gopls": {
"modPath": "./src/project1",
},
For projects that are structured like:
βββ src
βββ project1
βββ go.mod
βββ project2
βββ go.mod
If not, just that would be a huge quality-of-life improvement since switching to mods.
Multiple folder workspace workaround just broke:
Your workspace is misconfigured:
go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./]: exit status 1: go: cannot find main module, but found .git/config in /Users/ermik/Projects/[PATH] to create a module > there, run: go mod init .
Please see https://github.com/golang/tools/blob/master/gopls/doc/troubleshooting.md for more information or file an issue (https://github.com/golang/go/issues/new) if you believe this is a mistake.
(I have a workspace with a non-module root and 3 modules in root's subfolders added as workspace siblings.)
@tylfin: VS Code has per-workspace settings and I think that you can achieve that using them: https://code.visualstudio.com/docs/editor/multi-root-workspaces. If not, please follow up here in case I am misunderstanding you.
@ermik: Would you be able to share the complete gopls
logs for this case?
@stamblerre The multi-root-workspaces doesn't work because I want to open the root project (to get quick access to docs, configs, etc.)
So it ends up being:
βββ src
βββ project1
βββ go.mod
βββ project2
βββ go.mod
βββ docs
βββ configs
βββ project1
βββ go.mod
Which appears to fix the gopls
issue, but introduces linter problems and is just a poor dev experience compared to setting src/project1
to the workspace GOPATH...
What linter errors are you seeing? I'm sorry that the experience isn't ideal, but for now it is the best option. We are working on fixing this issue for gopls/v0.5.0
, so you can follow along with our progress in that milestone.
@stamblerre yes, thanks. Try me on twitter for quicker response time.
gopls logs
These show what happens on workspace open.
[Info - 6:46:53 PM] 2020/03/23 18:46:53 Build info
----------
golang.org/x/tools/gopls v0.3.4
golang.org/x/tools/[email protected] h1:4GC7q/pXQ/tsxHBGVdsMdlB4gCxVC06m/7rIXg1Px4E=
github.com/BurntSushi/[email protected] h1:WXkYYl6Yr3qBf1K79EBnL4mak0OimBfB0XUf9Vl28OQ=
github.com/sergi/[email protected] h1:Kpca3qRNrduNnOQeazBd0ysaKrUJiIuISHxogkT9RPQ=
golang.org/x/[email protected] h1:WG0RUwxtNT4qqaXX3DPA8zHFNm/D9xaBpxzHt1WcA/E=
golang.org/x/[email protected] h1:8gQV6CLnAEikrhgkHFbMAEhagSSnXWGV915qUMm9mrU=
golang.org/x/[email protected] h1:hKrQy/q8/Xivoqgw6nGiz1jqpn1WGBLDcWLZwW0983E=
golang.org/x/[email protected] h1:/atklqdjdhuosWIl6AIbOeHJjicWYPqR9bpxqxYG2pA=
honnef.co/go/[email protected] h1:sXmLre5bzIR6ypkjXCDI3jHPssRhc8KD/Ome589sc3U=
mvdan.cc/xurls/[email protected] h1:KaMb5GLhlcSX+e+qhbRJODnUUBvlw01jt4yrjFIHAuA=
Go info
-------
go version go1.13.4 darwin/amd64
GO111MODULE="on"
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/ermik/Library/Caches/go-build"
GOENV="/Users/ermik/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/ermik/Projects/go"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.13.4/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.13.4/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/dev/null"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/wp/bpblv6xj33n17_8pr7h8k1t40000gn/T/go-build342151811=/tmp/go-build -gno-record-gcc-switches -fno-common"
2020/03/23 18:46:53 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./ builtin]: exit status 1: go: cannot find main module, but found .git/config in /[REDACTED PATH TO ROOT WORKSPACE FOLDER]/
to create a module there, run:
go mod init
2020/03/23 18:46:53 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./]: exit status 1: go: cannot find main module, but found .git/config in /[REDACTED PATH TO ROOT WORKSPACE FOLDER]/
to create a module there, run:
go mod init
snapshot = 0
directory = 0x16de8b0
[Error - 6:46:53 PM] 2020/03/23 18:46:53 initial workspace load failed: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./ builtin]: exit status 1: go: cannot find main module, but found .git/config in /[REDACTED PATH TO ROOT WORKSPACE FOLDER]/
to create a module there, run:
go mod init
[Info - 6:46:53 PM] 2020/03/23 18:46:53 go/packages.Load
snapshot = 0
query = [./ builtin]
packages = 0
[Info - 6:46:53 PM] 2020/03/23 18:46:53 go/packages.Load
snapshot = 0
query = [./]
packages = 0
[Error - 6:46:53 PM] 2020/03/23 18:46:53 diagnose: no workspace packages: go [-e -json -compiled=true -test=true -export=false -deps=true -find=false -- ./]: exit status 1: go: cannot find main module, but found .git/config in /[REDACTED PATH TO ROOT WORKSPACE FOLDER]/
to create a module there, run:
go mod init
snapshot = 0
directory = 0x16de8b0
[Info - 6:46:54 PM] 2020/03/23 18:46:54 go/packages.Load
snapshot = 0
query = [./... builtin]
packages = 14
Thanks for the logs. It looks like gopls
is not finding the module roots for most of your workspaces (except for in the last case, where it finds 14 packages). Are there go.mod
files in all of the redacted directories? Does gopls
work if you open them individually? I've been discussing multi-project workspace related issues on https://github.com/golang/go/issues/37978, so you may find some of the discussion there helpful. I have a few outstanding changes that will improve the quality of the debug logs that you can try once they are merged.
All of the redacted directories are the same path, the root of the project where .code-workspace
is located:
{
"folders": [
{
"name": "/",
"path": "."
},
{
"path": "src"
}
],
"settings": {
"debug.openDebug": "openOnDebugBreak"
}
}
The root folder is reference to _self_, while src
is the sole location of go.mod
files. As you are aware, gopls
is unable to initialize, unless workspace refers to a directory with a go.mod
file in it, which is why this kind of workspace configuration is used here.
I think 14 packages may refer to the hierarchy of directories within the scope of src
module. Certainly, we have other projects where .code-workspace
has more entries referring to its root's subdirectories with distinct go.mod
files (multiple module roots), but this one is not it.
Change https://golang.org/cl/225237 mentions this issue: internal/lsp: stop showing workspace misconfiguration message
I see, thanks for clarifying. Is there any issue with the functionality of gopls
, or is the issue the error message that pops up? If the only error is the message, I just mailed a CL to remove it: https://golang.org/cl/225237.
The message did not seem to affect the functionality. I can't determine that for sure.
I also work with a monolithic repository. When adding my src
via Codes' "Add Folder to Workspace" gopls works, otherwise it doesn't. Would be great if it worked automatically tho.
One solution that worked for me is
I can still work out off the root folder this way and i don't need to add a go.mod to root
@stamblerre does this support not having to use tagged dependencies across submodules?
@jakesyl: I'm sorry, I'm not sure I understand what you're referring to. Do you mind giving an example?
@stamblerre absolutely. so our use case for go modules is as follows:
We have a "standard library" w/ common utils we use across our org: e.g. redis/elasticsearch/etc. client's, common logging formats, etc.
The annoying part about this is if we use say, our logger, in a small microservice, we need to install _all_ the dependencies of "standard library" which means our CI has to download all of these, etc.
What we'd like to do is have submodules that we can require individually, but that can depend on each other without tagging a version so:
standard library:
etc.
It sounds like this isn't a use case go modules were designed to support, but if this is something we can do I'm not sure how to implement it
Ah I see, thanks for clarifying. This issue is specific to gopls, the Go language server, so it's not covering the different patterns for using modules. I'd suggest filing a new issue or asking this question on the #modules channel of the Gophers Slack.
@stamblerre According to https://blog.golang.org/v2-go-modules the recommended way of moving to a v2 of a module is to copy the go.mod and go files of the original project into a v2 directory. This creates the multiple module root problem. In my case gopls is adding an indirect dependency on the v2 code into the v1 go.mod. This occurs even if I use the workaround to open v2 as a new workspace folder.
This means that the problem occurs in the standard Go workflow as opposed to the "weird project structure" mentioned by the original poster. Would this make the issue a higher priority?
Has this issue been prioritised yet and, if so, is there any idea when it will be fixed?
In my case gopls is adding an indirect dependency on the v2 code into the v1 go.mod. This occurs even if I use the workaround to open v2 as a new workspace folder.
This sounds like https://github.com/golang/go/issues/32499, which should have been fixed but does need more work. We'll look into it there.
This means that the problem occurs in the standard Go workflow as opposed to the "weird project structure" mentioned by the original poster. Would this make the issue a higher priority?
Has this issue been prioritised yet and, if so, is there any idea when it will be fixed?
This issue is high priority - most of the issues in the gopls/v0.6.0 milestone relate to it. We will be working on implementing the design described in this design doc.
This issue still exists. It is more than a year.
Right now, the best way to work with
gopls
in VSCode is to use Workspace Folders per module. For everygo.mod
file that you have, you should do "File" > "Add Folder to Workspace" and then add the module root.
This should be pinned. This fixes issues with VSCode working locally with two modules using replace
.
Right now, the best way to work with
gopls
in VSCode is to use Workspace Folders per module. For everygo.mod
file that you have, you should do "File" > "Add Folder to Workspace" and then add the module root.This should be pinned. This fixes issues with VSCode working locally with two modules using
replace
.
It doesn't completely fix the issue btw. Find All References is inconsistent depending on which workspace folder you trigger it from (as in either from where it's defined or from where it's used in some other workspace folder). In no case however does it return the complete list of all references.
Yep, that's correct - this design doc explains our plan for supporting features across multiple workspace folders for gopls/v1.0.0
.
Change https://golang.org/cl/244117 mentions this issue: internal/lsp: treat the module root as the workspace root, if available
Change https://golang.org/cl/247819 mentions this issue: internal/lsp/cache: find all of the modules under the workspace root
In #37720 I raised a concern around the use of the hidden super module. Per our offline discussion, the go
commands invoked by the editors and their underlying tools (dlv, ...) may need to work with the same super modules. How exactly the super module will be reflected to the codebase and how to make gopls
and the go command invoked outside gopls
share the same view will will need to be sorted out.
More questions: according to the proposal,
The go command does require that its working directory contain a go.mod file, but we want to run commands from the super-module without exposing super-module's go.mod file to the user. To handle this, we will create a temporary directory containing the super-module's go.mod file, to act as the module root for any go/packages queries.
Some editors (e.g. vscode) often invoke go commands (go test
, ...) and other tools (dlv
) from the package directory, not from the workspace or module roots. Should that be avoided - in case the package is in a nested module.
ps. Since the #37720 was for the discussion around the particular design proposal but not that's closed, we need to update the link from the proposal.
The workspace module will be used for providing language features and diagnostics, but I don't think it should be used with go
commands that need to be run from a specific module. For things like go mod tidy
diagnostics, we will still be running them from within the correct modules, so I would expect go test
to work similarly. A possible conflict I imagine is if go build
reports different errors than the gopls
diagnostics because of some issue in constructing the workspace module.
I'm not familiar enough with debugging to know if that would need to interact with the workspace module in some way.
I want to second @stamblerre wrt modifying tools to run commands from some super module root. In vim-go's case, that would be a massive and risky undertaking and probably wouldn't work as expected without changes to the go
tool and all the community supported tools, too. I strongly prefer to avoid a design that would require such sweeping changes to the tool ecosystem.
@hyangah
In #37720 I raised a concern around the use of the hidden super module
I share these concerns (and indeed I think I mentioned them on one of the tools calls). I wonder whether you could clarify the following sentence in your comment above:
How exactly the super module will be reflected to the codebase and how to make gopls and the go command invoked outside gopls share the same view will be sorted out.
You say this "will be sorted out": as in, there is a plan that describes how this will be resolved? Or is it instead "this will need to be sorted out": as in, there is currently no plan?
If there is a plan as to how this will be addressed/fixed, can that be shared?
If there is not a plan, is this not a significant problem as far as the proposal is concerned?
Thanks
@stamblerre For debugging, editors will just use go build
(either directly, or by delegating dlv
to run the go command). So, as long as there is a way to teach the go command to construct the same view as gopls
, I think it's fine.
Code editing and go build
should be consistent as much as possible.
Let's assume a workspace with multiple modules including app1
and app2
, and app2
depends on a package under app1
and that is app1/pkg
.
module gopls-workspace
require (
example.com/app1 v0.0.0-00010101000000-000000000000
example.com/app2 v0.0.0-00010101000000-000000000000
)
replace (
example.com/app1 => /abs/path/to/app1
example.com/app2 => /abs/path/to/app2
// Further replace directives included from app1 and app2
)
User didn't realize app2 depends on app1 yet thanks to the workspace module magic. Using Go to Definition
gopls provides with the workspace module, the user finds app1/pkg.F
source file and makes some edits in app1/pkg.F
. Now the user wants to check if the app1/pkg.F
change works by building app2
and possibly launching a debugger. In order to pick up the change in app1/pkg.F
that was made with the assist from gopls, the go build
has to run with a modified go.mod
that has the modified replace
statement. The proposal doesn't discuss this part yet.
@bhcleek if it's not clear, yes, that's why I am raising this issue - I also want to avoid such churn.
@myitcv I updated my comment :-)
I don't think there is a concrete plan to address this aspect yet.
I don't think if this is a significant problem from the gopls's point of view :-)
But for editors, they need to find a way to guide users between different views when gopls starts to operate with its own view on the world (now one more go.mod).
@hyangah: Yeah, that's a tough situation - gopls
would behave as though app1
was replaced but Delve would not. But we can't really intuit what the user wants - maybe we could add this into the set of gopls
diagnostics, like "do you want to add a replace directive for this module?"
@stamblerre Yeah, that kind of diagnostics and code actions maybe very helpful. We should still be careful with the replace directive change though. Many replace statements are not appropriate to be checked in to the source repo. :-/
Change https://golang.org/cl/254368 mentions this issue: go/packages: support overlays for replaced modules
Change https://golang.org/cl/254317 mentions this issue: internal/lsp: enable multi-module workspace mode by default
Change https://golang.org/cl/254940 mentions this issue: internal/lsp: handle modifications to the workspace module
Change https://golang.org/cl/256042 mentions this issue: internal/lsp: add a command to generate the gopls.mod file
Change https://golang.org/cl/256364 mentions this issue: internal/lsp/mod: add a code lens to regenerate gopls.mod
Change https://golang.org/cl/256582 mentions this issue: internal/lsp/cache: use gopls.mod for the workspace module if it exists
Change https://golang.org/cl/254753 mentions this issue: internal/lsp: handle initial workspace load failure per module
Change https://golang.org/cl/256941 mentions this issue: internal/lsp: allow multiple go.mod files in a view
Change https://golang.org/cl/257417 mentions this issue: internal/lsp: remove the view's modURI field
For reference, gopls does not work with this project structure (at the moment):
.
βββ cmd
βΒ Β βββ go.mod
βΒ Β βββ logger
βΒ Β βΒ Β βββ main.go
βΒ Β βββ printer
βΒ Β βββ main.go
βββ lib
βββ count
βΒ Β βββ words.go
βΒ Β βββ z_words_test.go
βββ go.mod
βββ list
βββ list.go
βββ z_list_test.go
We are working on implementing support for this by default in gopls
, but in the meantime, you can work with such a project by adding the cmd
and lib
folders separately as workspace folders (File -> Add Folder to Workspace in VS Code).
Partial support for multi-module workspaces is now available in gopls/v0.5.1
, with the experimentalWorkspaceModule
setting.
gopls.mod
workspace module file (https://github.com/golang/go/issues/41837)This list will be updated as work is completed and new items are added.
How do we enable the experimentalWorkspaceModule
setting for the VS Code extension? The release notes say to add this to our settings:
"gopls": {
"experimentalWorkspaceModule": true,
}
Which settings is this referring to?
If you open the Command Palette (Ctrl+Shift+P) and select "Preferences: Open Settings (JSON)", you can paste that directly into the file.
@stamblerre VS Code says it's an unrecognized setting when I paste it into settings.json -- don't VS Code settings keys have to be prefixed with the extension name? (I'd expect it to be something like go.experimentalWorkspaceModule: true
)
The setting will actually still work--https://github.com/golang/vscode-go/issues/197 is tracking declaring the gopls settings in the Go extension.
Awesome, it's working wonderfully so far, thanks Rebecca!
Most helpful comment
Partial support for multi-module workspaces is now available in
gopls/v0.5.1
, with theexperimentalWorkspaceModule
setting.Outstanding work
gopls.mod
workspace module file (https://github.com/golang/go/issues/41837)This list will be updated as work is completed and new items are added.