Wails: Cross compilation using xgo does not copy local (unpublished) packages

[DanielKrolopp]

Description

I've been trying to cross compile for windows from my Linux machine and ran into an issue. I have a go module in a different directory than the one in which my wails project is located---I import this module in a go.mod file in my wails project directory using the replace keyword. When I try to compile, I get the following message:

Compiling for windows-4.0/amd64...
go: github.com/username/projectname/[email protected]: parsing /home/dk/go/src/github.com/username/projectname/client/go.mod: open /home/dk/go/src/github.com/username/projectname/client/go.mod: no such file or directory

I suspect that the output is from the Docker container, and that the necessary files are not being copied in.

To Reproduce
Steps to reproduce the behaviour:

1. Import a local go module outside the immediate project directory using go modules' replace keyword from within the go.mod of your wails project directory.
2. Run wails build -x windows/amd64 --verbose. The error described above should appear.
3. Wails will claim that the project built successfully, but the build folder will be empty.

Expected behaviour
The cross-compilation builds a functioning windows executable.

System Details

| Name | Value |
| ----- | ----- |
| Wails Version | v1.7.1 |
| Go Version | go1.14.2 |
| Platform | linux |
| Arch | amd64 |
| GO111MODULE | (Not Set) |
| GCC | 7.5.0 |
| Npm | 6.14.4 |
| Node | v12.17.0 |

[leaanthony]

The project directory is mounted for the build so if it's not in there then yes, it won't get picked up. What do you think would be a good solution to this?

[tmclane]

This is a tricky one. If you make the go module accessible over the internet go itself will copy it into the build container.
If it is private it is a bit harder.

One thought i had would be to:

• optionally mount the GOPATH directory into the build container at /go.

[leaanthony]

That's not a bad idea @tmclane! We could then add it to $GOPATH. Trying to think how this works with modules.

[tmclane]

Well as long as you place the module in the GOPATH it will work fine.

Perhaps we could allow additional mount mappings to help place them appropriately inside the container.

[leaanthony]

I guess the problem is that go.mod can reference any arbitrary location on disk. What about doing go mod vendor first to pull it into the project directory? I know wails itself has an issue with that so potentially remove that directory before compiling

[tmclane]

That might work. I haven't used that feature before.

[leaanthony]

It should work fine, just need to vendor, remove wails then start docker. Wails specifically doesn't work in vendor because of the funky stuff we do with module paths.

[DanielKrolopp]

It looks like the two of you have some pretty good ideas on how to accomplish this already, but I'll add my two cent's worth. As @leaanthony mentioned, since vendoring doesn't work, for now I'm just writing a script to copy the needed modules to the build directory before compiling and using a different path with the replace directive.

[leaanthony]

We can get around the fact that wails won't vendor correctly, but your issue is something we need to solve so I think we'll try that approach and see where we get.

[DanielKrolopp]

Yeah, I agree. Thanks for your prompt response!

[leaanthony]

Another user reported this on slack: using replace in go.mod will also fail cross-compile:

// go.mod
require (
  ...
  ...
  github.com/Me/myproject v0.1.0
)
replace github.com/Me/myproject => ../myproject

[tmclane]

Yes of course that replace will fail. That path isn't mapped into the container properly.

[tmclane]

I'll eventually have time to work on this again.

[leaanthony]

I'm not really sure how to fix the replace modules. If mod vendor pulls that in then it'll work. Otherwise....?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[leaanthony]

@DanielKrolopp - Did you find a work around for this? I think this will be too hard for us to fix for cross-compilation due to the complexity of the issue.

[tmclane]

I apologize for having been too busy to check into this . I need to fix the build image for another ticket and I'll look into this as well. I think we will have to prescribe placement restrictions to make it work but I think that should be fine.

[DanielKrolopp]

The only workaround I have is to basically mimic go vendor by writing a script to copy the local packages into a "shadow" directory within the scope of the wails project and modifying all references to the missing packages with a

replace github.com/DanielKrolopp/project/package => ./shadow/package

I include the copy as a step in my Makefile as well.

[leaanthony]

I think I have a solution for this. Please bear with us :-)

[tmclane]

@DanielKrolopp Would you mind checking out the feature branch feature/goroot I just pushed up that allows you to specify the -goroot flag and provide the location on your drive of the GOROOT environment value?

Passing that in causes the location on your host to be mounted at /go inside the cross-compiler container. This should enable a replace github.com/wailsapp/wails => github.com/DanielKrolopp/wails style of substitution.

| |     / /___ _(_) /____
| | /| / / __ `/ / / ___/
| |/ |/ / /_/ / / (__  )  v1.8.0
|__/|__/\__,_/_/_/____/   https://wails.app
The lightweight framework for web-like apps

wails build - Builds your Wails project

This command will check to ensure all pre-requistes are installed prior to building. If not, it will attempt to install them. Building comprises of a number of steps: install frontend dependencies, build frontend, pack frontend, compile main application.

Flags:

  -d    Build in Debug mode
  -f    Force rebuild of application components
  -goroot string
        Specify your goroot location. Mounted to /go during cross-compilation.
  -help
        Get help on the 'wails build' command.
  -ldflags string
        Extra options for -ldflags
  -p    Package application on successful build
  -t string
        Generate Typescript definitions to given file (at runtime)
  -verbose
        Verbose output
  -x string
        Cross-compile application to specified platform via xgo
         - darwin/amd64
         - linux/amd64
         - linux/arm-7
         - windows/amd64

Example:

wails build -x windows/amd64 -goroot ~/go

[DanielKrolopp]

Yeah, I tried it and it works with my local replace keywords. Thanks!

Edit: is the -goroot option meant to be -gopath?

[tmclane]

Yes I guess I should rename it.

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[leaanthony]

Was this solved? Do we need to merge in the fixes on the branch?

[tmclane]

Was this solved? Do we need to merge in the fixes on the branch?

It was merged to develop. I don't know if we have cut a release yet containing it.