Go: cmd/go: "go get: unzip: malformed file path: double dot"

[sirkon]

• go version

  go version go1.11 linux/amd64
  

  downloaded yesterday (27 august)

• $ go env
  GOARCH="amd64"
  GOBIN=""
  GOCACHE="/home/emacs/.cache/go-build"
  GOEXE=""
  GOFLAGS=""
  GOHOSTARCH="amd64"
  GOHOSTOS="linux"
  GOOS="linux"
  GOPATH="/home/emacs/go"
  GOPROXY=""
  GORACE=""
  GOROOT="/usr/local/go"
  GOTMPDIR=""
  GOTOOLDIR="/usr/local/go/pkg/tool/linux_amd64"
  GCCGO="gccgo"
  CC="gcc"
  CXX="g++"
  CGO_ENABLED="1"
  GOMOD="/home/emacs/Desktop/main/go.mod"
  CGO_CFLAGS="-g -O2"
  CGO_CPPFLAGS=""
  CGO_CXXFLAGS="-g -O2"
  CGO_FFLAGS="-g -O2"
  CGO_LDFLAGS="-g -O2"
  PKG_CONFIG="pkg-config"
  GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build252846370=/tmp/go-build -gno-record-gcc-switches"
  

• I am trying to use ANTLR4 runtime support via go modules, steps to reproduce:

  1. Choose some directory
  2. Create new Go package/module:
     mkdir newpack cd newpack go mod init main
  3. Try to get ANTLR4 runtime support module
     go get github.com/antlr/antlr4/runtime/Go/antlr
  4. Obviously, successful download was expected, got an error message instead:
     go: finding github.com/antlr/antlr4/runtime/Go/antlr latest go: finding github.com/antlr/antlr4/runtime/Go latest go: finding github.com/antlr/antlr4/runtime latest go: finding github.com/antlr/antlr4 latest go: extracting github.com/antlr/antlr4 v0.0.0-20180728001836-7d0787e29ca8 -> unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot go get github.com/antlr/antlr4/runtime/Go/antlr: unzip /home/emacs/go/pkg/mod/cache/download/github.com/antlr/antlr4/@v/v0.0.0-20180728001836-7d0787e29ca8.zip: malformed file path "runtime/Cpp/demo/Windows/antlr4-cpp-demo/antlr4-cpp-demo-vs2015.vcxproj..filters": double dot

  I guess double dots in some unrelated file is hardly a reason to diagnose a error

[sjbodzo]

I am also facing this issue. Since DynamoDB uses ANTLR to parse expression grammar, this effectively means we cannot use the aws-sdk in Golang with go modules until there is a fix.

[bcmills]

Consolidating into #28001.

[odeke-em]

Hey @bcmills @rsc, I realize that perhaps we plainly reject all double-dot-containing paths because we are trying to avoid trafficking/arbitrary/malicious filesystem access e.g. "foo/../secrets" where ".." contains the keys to the kingdom.
However, this seems broadly sweeping and I think we can make an exception to this rule like other filesystems to, by checking that the base of the path doesn't just consist of
dots(after checking if the entire path has "..") so basically something like this

        if strings.Contains(path, "..") {
                // Rejecting paths with ".." in them is a security consideration meant to
                // prevent arbitrary filesystem accesses. However, the exception to
                // this rule is that if the path's base consists of more
                // than just "."+ which would disambiguate between ".", ".." and valid
                // paths, then accept it.
                base := filepath.Base(path)
                if strings.Count(base, ".") == len(base) {
                        // Consists entirely of "."*
                        return fmt.Errorf("double dot")
                }
        }

which would match and allow say "foo..ps..mp4" as well as the bug report from the ANTLR parser generator.

I realize that we are almost releasing Go1.12 and perhaps could punt this to Go1.13 for safety but if this issue is deemed high priority perhaps let's discuss this -- also note that the ANTLR project replace the double dots that triggered this bug report, in August 2018(right after this issue was filed) as per https://github.com/antlr/antlr4/pull/2341 but there might be other projects like this in the wild.

I'll also kindly page @mikesamuel for an evaluation of the security considerations of how I've proposed accepting ".." in the base.

[DanielJonesEB]

Is there a workaround for this issue? Hitting it trying to depend on https://github.com/SpectoLabs/hoverfly

go: extracting github.com/SpectoLabs/hoverfly v0.17.7
-> unzip /Users/deejay/go/pkg/mod/cache/download/github.com/!specto!labs/hoverfly/@v/v0.17.7.zip: malformed file path "core/handlers/v2/hoverfly_upstream_proxy_handler..go": double dot

[sirkon]

@DanielJonesEB good workaround is to create an issue in the project

[DanielJonesEB]

good workaround is to create an issue in the project

Sadly I'm depending on an old version of the library, so we can't go back in time to fix it.

[keitwb]

Found this same problem in the latest version of the Telegraf repo. The use of double dots appears to be perfectly legitimate since they are testing that the config loader doesn't descend into directories that start with double dots. I guess the test could be changed to dynamically create the double dot directory to avoid it being committed to the repo, but would be nice not to have to and still work with modules.

[zouyee]

go get -u golang.org/x/tools/... found the same issue.

[DavidGamba]

Used the workaround provided here: https://github.com/golang/go/issues/26672#issuecomment-409685692 to bypass the error, basically add a go.mod file in my test file tree so it is considered as a separate module and it doesn't get unzipped with the main one.

[gopherbot]

Change https://golang.org/cl/197720 mentions this issue: cmd/go: do not reject internal double-dots in path elements

[liesauer]

i compiled the latest master source and still get this error, am i missing something?

go version devel +3322f3e0ce Wed Oct 9 23:03:55 2019 +0000 darwin/amd64

GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/imac/Library/Caches/go-build"
GOENV="/Users/imac/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/imac/Projects/go/bin"
GOPRIVATE=""
GOPROXY="https://goproxy.cn"
GOROOT="/Users/imac/Projects/go"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/imac/Projects/go/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/imac/Projects/go/src/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/xb/5ys3kh657kdd45gb9kmlqjq80000gn/T/go-build166768544=/tmp/go-build -gno-record-gcc-switches -fno-common"

[liesauer]

alright! vscode use the absolute path /usr/local/bin/go which is still a soft link of /usr/local/Cellar/go/1.12.7/bin/go, relink to /Users/imac/Projects/go/bin/go will get fix