Packer: Possible issues with building packer on the windows platform due to number of files in oracle/oci-go-sdk/core package and limitation of golang's compile on windows.

[arizvisa]

This issue is just for tracking issue golang/go#18468 from go and is similar to hashicorp/terraform#16917.

The issue is that normally, golang's build takes a list of .go files from a package, expands the package name by prefixing it with the absolute path, and then hands off the .go files to compile in order to pass onto the next stage. On windows, the environment has a size limitation (whee) and internally uses the UNICODE_STRING type for the commandline which has a size limitation of 32k.

Since oracle/oci-go-sdk/core includes 261 files, go-build will prefix all of these with the package's absolute path and in some cases overshoot that 32k limit.

Workaround 1

A simple work-around would be to use an ntfs junction (windows symbolic link sorta) to place the oracle/oci-go-sdk package in a directory that results in the concatenation of the package path and the files having a size smaller than 32k.

Taking the sum of all the chars of the files in oracle/oci-go-sdk/core results in 8977 chars. Divide that by 261 files results in a max prefix path of ~91 chars. So putting the package in a directory where the absolute path is <91 chars will then allow you to use go build on it.

Workaround 2

oracle/oci-go-sdk was introduced by commit 31973d9f8b45f17c3988580e8b37da4d15e23851 on 4/11/2018. So reverting this merge (and castrating builder/oracle) will also allow you to build on windows.

[prydie]

Thanks for the report @arizvisa. I've raised an issue with our SDK team (https://github.com/oracle/oci-go-sdk/issues/108) and we'll do our best to get this resolved ASAP.

[arizvisa]

@prydie, if you prefer. The correct fix is actually to fix golang itself because it's an oversight on their part wrt the windows platform.

If you look at the golang/go#18468 issue, bradfitz proposed a solution which got turned down in favor of the idea of response files, and then I offered an implementation of response files that I don't have time to maintain.

The one issue with my implementation, however, is the way I implemented it by copying buildargv from GPL licensed software will "infect" golang's license (whee gpl...). If one of your sdk devers works on that patch by re-implementing a bsd-licensed buildargv and works on getting that merged upstream, you guys will save a ton of dev effort and cost and fix golang for everyone...

[eginez]

Agreed the right fix is not in the sdk, this should get fixed in the compiler itself. We can do hacks to make it build on windows but this will happen again as soon as we add more files, and those packages will grow.

[tb3088]

overrunning command-line char limits shows up in some of the Makefile steps as well (Windows/Cygwin). The solution there is xargs. Why not have Go chunkify it's processing in a similar manner? For example arbitrarily process 100 files in one iteration.

[arizvisa]

@tb3088, In order to link all the object files into the final executable, each object file emitted by the compiler needs to be known by the linker. So you may be able to chunk out compilation, but linking will be where it stops. just fyi.

[arizvisa]

Looks like golang/go#18468 was merged upstream. During the next release of go, this issue can be closed.

[eginez]

@SwampDragons Hey all, has the fix been applied to packer builds? I am still seeing the following fail: https://ci.appveyor.com/project/hashicorp/packer/build/5609

[arizvisa]

@eginez, that particular error that you linked is not with packer but with appveyor using a version of go that does not use that PR that I mentioned. The fix has apparently been backported to go 1.10, so if appveyor fixes their build then things should be good to go. Also, once golang hits a major version, then things should also be fixed.

Best thing to do is to re-install/re-build go, and then run the tests using a golang version that's patched.

[harveylowndes]

Looks like there is a fix in go1.11beta1, awaiting appveyor to use a version with the fix.

[arizvisa]

I think the fix is included in go1.11 which was released last friday. Unless anybody else has issues or complaints, I'm going to close this bug in like maybe a week.

[SwampDragons]

I think now's fine -- good catch @arizvisa

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.