Golangci-lint: goimport: check wrong path

[daixiang0]

WARN [runner] Can't run linter goanalysis_metalinter: goimports: open /go/src/github.com/grafana/loki/pkg/logql/pkg/logql/expr.y: no such file or directory 

The expr.y exists under /go/src/github.com/grafana/loki/pkg/logql indeed.

Reproduce step:

1. git clone https://github.com/daixiang0/loki.git
2. check out update-golint
3. run make lint

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version 1.21.0 built from 645e794 on 2019-10-15T18:16:56Z

Config file

refer to link

Go environment

$ go version && go env
go version go1.12.1 linux/amd64
GOARCH="amd64"
GOBIN=""
GOCACHE="/root/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/deploy/go"
GOPROXY="http://192.168.20.27:8000"
GORACE=""
GOROOT="/usr/lib/golang"
GOTMPDIR=""
GOTOOLDIR="/usr/lib/golang/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build972039009=/tmp/go-build -gno-record-gcc-switches"

Verbose output of running

$ golangci-lint run -v
# make lint
GO111MODULE=on GOGC=10 ./bin/golangci-lint run -v
INFO [config_reader] Config search paths: [./ /go/src/github.com/grafana/loki /go/src/github.com/grafana /go/src/github.com /go/src /go  /] 
INFO [config_reader] Used config file .golangci.yml 
INFO [lintersdb] Active 18 linters: [deadcode errcheck goconst gofmt goimports golint gosimple govet ineffassign interfacer misspell staticcheck structcheck typecheck unconvert unparam unused varcheck] 
INFO [loader] Go packages loading at mode 575 (compiled_files|exports_file|name|types_sizes|deps|files|imports) took 2.286228456s 
INFO [runner/filename_unadjuster] Pre-built 1 adjustments in 140.588798ms 
INFO [runner/goanalysis_metalinter/goanalysis] analyzers took 10m37.67786349s with top 10 stages: buildssa: 7m33.633120004s, ctrlflow: 24.053607327s, fact_purity: 21.935412659s, fact_deprecated: 21.49902466s, printf: 20.845789535s, inspect: 20.572082345s, goimports: 10.618553436s, gofmt: 7.597685167s, interfacer: 5.816696433s, unparam: 4.599163088s 
WARN [runner] Can't run linter goanalysis_metalinter: goimports: open /go/src/github.com/grafana/loki/pkg/logql/pkg/logql/expr.y: no such file or directory 
INFO [runner/unused/goanalysis] analyzers took 14.862776971s with top 10 stages: buildssa: 13.553725907s, U1000: 1.309051064s 
INFO [runner] Issues before processing: 9, after processing: 0 
INFO [runner] Processors filtering stat (out/in): skip_files: 7/7, exclude-rules: 2/2, cgo: 7/9, autogenerated_exclude: 2/7, path_prettifier: 7/7, skip_dirs: 7/7, identifier_marker: 2/2, exclude: 2/2, nolint: 0/2, filename_unadjuster: 7/7 
INFO [runner] processing took 5.337681ms with stages: nolint: 4.951235ms, path_prettifier: 102.678µs, exclude: 93.718µs, identifier_marker: 57.984µs, autogenerated_exclude: 51.174µs, skip_dirs: 48.385µs, filename_unadjuster: 15.858µs, cgo: 9.914µs, max_same_issues: 1.895µs, path_shortener: 798ns, max_from_linter: 757ns, diff: 704ns, source_code: 699ns, uniq_by_line: 587ns, skip_files: 564ns, max_per_file_from_linter: 368ns, exclude-rules: 363ns 
INFO [runner] linters took 1m5.185648428s with stages: goanalysis_metalinter: 58.910826734s, unused: 6.26925848s 
INFO File cache stats: 179 entries of total size 7.1MiB 
INFO Memory: 667 samples, avg is 575.3MB, max is 881.7MB 
INFO Execution took 1m7.628324203s   

[daixiang0]

@jirfag hi, any update?

[dixonwille]

I can't share the code I have that reproduced this, but I'll share what I can. Same linter error, but the reason is different. I have a strong hunch it is the caching layer that was added. I'll explain my reasoning after I dump my info.

Version of golangci-lint

$ golangci-lint --version
golangci-lint has version v1.21.0 built from (unknown, mod sum: "h1:HxAxpR8Z0M8omihvQdsD3PF0qPjlqYqp2vMJzstoKeI=") on (unknown)

Config file

$ cat .golangci.yml
run:
  timeout: 5m
  tests: false
  skip-dirs:
    - rpc$

linters-settings:
  govet:
    check-shadowing: true
  depguard:
    list-type: whitelist
    packages:
      - ac.st/
      - mesa.ac.st/
      - mono.ac.st/
      - golang.org/x/
      - github.com/joho/godotenv/autoload
      - github.com/ianschenck/envflag
      - github.com/couchbase/gocb
      - github.com/aws/aws-sdk-go
      - github.com/aws/aws-lambda-go/lambda
      - github.com/golang/protobuf/proto
      - github.com/RangelReale/osin
      - github.com/google/gofuzz
      - github.com/golang/protobuf/ptypes
      - gopkg.in/couchbase/gocbcore.v7
      - github.com/robfig/cron
      - github.com/stathat/go
      - github.com/dgrijalva/jwt-go
      - github.com/denisenkom/go-mssqldb
      - github.com/patrickmn/go-cache
      - gopkg.in/gomail.v2
      - github.com/pquerna/otp
      - github.com/pusher/pusher-http-go
      - github.com/sideshow/apns2
      - firebase.google.com/go
      - google.golang.org/api/option
      - github.com/smartystreets/smartystreets-go-sdk/us-street-api
      - github.com/smartystreets/smartystreets-go-sdk/wireup
      - github.com/smartystreets/smartystreets-go-sdk
      - github.com/microcosm-cc/bluemonday
      - github.com/aymerick/douceur/inliner
      - github.com/jhillyerd/enmime
      - github.com/google/go-cmp/cmp
      - github.com/google/go-cmp/cmp/cmpopts
      - github.com/docker/docker
      - github.com/docker/go-connections
      - github.com/speps/go-hashids
      - github.com/moov-io/ach
      - github.com/jackc/pgx
      - gopkg.in/jackc/pgx.v2
      - github.com/google/go-github
      - github.com/lib/pq
      - github.com/acst/realm-go-sdk/
      - github.com/Pallinder/go-randomdata
      - github.com/stretchr/testify

linters:
  enable:
    - unused
    - goconst
    - gocyclo
    - golint
    - ineffassign
    - interfacer
    - structcheck
    - varcheck
    - govet
    - depguard
    - goimports
    - misspell
  disable-all: true

issues:
  exclude-use-default: false

Go environment

$ go version && go env
go version go1.13.3 darwin/amd64
GO111MODULE=""
GOARCH="amd64"
GOBIN=""
GOCACHE="/Users/willdixon/Library/Caches/go-build"
GOENV="/Users/willdixon/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GONOPROXY="*.ac.st,ac.st,github.com/acst"
GONOSUMDB="*.ac.st,ac.st,github.com/acst"
GOOS="darwin"
GOPATH="/Users/willdixon/.gvm/pkgsets/go1.13.3/global"
GOPRIVATE="*.ac.st,ac.st,github.com/acst"
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/Users/willdixon/.gvm/gos/go1.13.3"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/Users/willdixon/.gvm/gos/go1.13.3/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD="/Users/willdixon/Projects/acst/mono.ac.st/go.mod"
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/0d/3m7jtb3n6m38r9j2kf9686c00000gn/T/go-build440485480=/tmp/go-build -gno-record-gcc-switches -fno-common"

Verbose output of running

$ golangci-lint run -v
INFO [config_reader] Used config file .ci/.golangci.yml
INFO [lintersdb] Active 12 linters: [depguard goconst gocyclo goimports golint govet ineffassign interfacer misspell structcheck unused varcheck]
INFO [loader] Go packages loading at mode 575 (exports_file|files|imports|compiled_files|deps|name|types_sizes) took 1.280307101s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 18.268349ms
INFO [runner/unused/goanalysis] analyzers took 142.124158ms with top 10 stages: buildssa: 106.625133ms, U1000: 35.499025ms
INFO [runner/goanalysis_metalinter/goanalysis] analyzers took 1.657572912s with top 10 stages: ctrlflow: 318.283588ms, printf: 292.376865ms, inspect: 275.397904ms, goimports: 243.844392ms, golint: 154.780209ms, buildssa: 149.552969ms, ineffassign: 85.899932ms, misspell: 50.46303ms, interfacer: 33.498591ms, varcheck: 11.124487ms
WARN [runner] Can't run linter goanalysis_metalinter: inspect: analysis skipped: errors in package: [/Users/willdixon/Projects/acst/mono.ac.st/pkg/realmsql/pool.go:331:34: cannot use context.Background() (value of type context.Context) as context.Context value in argument to p.shep.GetDatabase: wrong type for method Deadline /Users/willdixon/Projects/acst/mono.ac.st/pkg/realmsql/pool.go:316:36: cannot use context.Background() (value of type context.Context) as context.Context value in argument to p.shep.ListDatabases: wrong type for method Deadline /Users/willdixon/Projects/acst/mono.ac.st/pkg/realmsql/pool.go:280:34: cannot use context.Background() (value of type context.Context) as context.Context value in argument to p.shep.GetDatabase: wrong type for method Deadline /Users/willdixon/Projects/acst/mono.ac.st/pkg/realmsql/pool.go:212:34: cannot use context.Background() (value of type context.Context) as context.Context value in argument to p.shep.GetSiteByID: wrong type for method Deadline]
INFO [runner] processing took 7.123µs with stages: nolint: 2.176µs, max_same_issues: 2.004µs, skip_dirs: 459ns, cgo: 295ns, max_from_linter: 275ns, path_prettifier: 221ns, autogenerated_exclude: 211ns, filename_unadjuster: 207ns, skip_files: 206ns, exclude: 198ns, identifier_marker: 141ns, max_per_file_from_linter: 136ns, uniq_by_line: 136ns, diff: 133ns, path_shortener: 132ns, source_code: 131ns, exclude-rules: 62ns
INFO [runner] linters took 2.884034637s with stages: goanalysis_metalinter: 2.150029736s, unused: 733.935286ms
INFO File cache stats: 40 entries of total size 292.8KiB
INFO Memory: 44 samples, avg is 156.4MB, max is 270.8MB
INFO Execution took 4.213371167s

Verbose output of running right after the above

INFO [config_reader] Used config file .ci/.golangci.yml
INFO [lintersdb] Active 12 linters: [depguard goconst gocyclo goimports golint govet ineffassign interfacer misspell structcheck unused varcheck]
INFO [loader] Go packages loading at mode 575 (files|name|types_sizes|compiled_files|deps|exports_file|imports) took 1.323097512s
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 15.764565ms
INFO [runner/unused/goanalysis] analyzers took 0s with no stages
INFO [runner/goanalysis_metalinter/goanalysis] analyzers took 765.790541ms with top 10 stages: goimports: 221.566039ms, buildssa: 162.992434ms, golint: 127.599893ms, ineffassign: 80.914003ms, misspell: 57.103443ms, interfacer: 39.339366ms, printf: 14.050833ms, ctrlflow: 13.547298ms, varcheck: 8.203296ms, inspect: 5.692812ms
INFO [runner] Issues before processing: 1007, after processing: 2
INFO [runner] Processors filtering stat (out/in): skip_files: 1007/1007, autogenerated_exclude: 4/1007, exclude: 4/4, uniq_by_line: 4/4, cgo: 1007/1007, filename_unadjuster: 1007/1007, nolint: 4/4, path_prettifier: 1007/1007, identifier_marker: 4/4, max_from_linter: 2/2, path_shortener: 2/2, diff: 4/4, max_per_file_from_linter: 2/4, max_same_issues: 2/2, source_code: 2/2, skip_dirs: 1007/1007, exclude-rules: 4/4
INFO [runner] processing took 4.514408ms with stages: autogenerated_exclude: 1.872941ms, path_prettifier: 998.526µs, nolint: 703.673µs, source_code: 323.663µs, skip_dirs: 291.173µs, filename_unadjuster: 115.949µs, cgo: 100.536µs, identifier_marker: 94.799µs, max_same_issues: 3.471µs, uniq_by_line: 2.363µs, max_per_file_from_linter: 2.098µs, max_from_linter: 2.049µs, path_shortener: 1.487µs, exclude: 693ns, diff: 369ns, skip_files: 310ns, exclude-rules: 308ns
INFO [runner] linters took 902.522943ms with stages: goanalysis_metalinter: 574.583916ms, unused: 323.321398ms
services/realm/shepherd/internal/service/service.go:5: File is not `goimports`-ed (goimports)
    "github.com/golang/protobuf/ptypes"
services/realm/shepherd/internal/service/service.go:43:1: exported function `HelloWorld` should have comment or be unexported (golint)
func HelloWorld() {
^
INFO File cache stats: 41 entries of total size 299.2KiB
INFO Memory: 24 samples, avg is 80.6MB, max is 136.8MB
INFO Execution took 2.269677646s

It seems when I run golangci-lint cache clean and then rerun the lints, I get the Can't run linter goanalysis_metalinter. but after that execution (and sometimes it takes more than one run), it goes through. My guess is at a certain point, there is enough info in cache that it works. Sounds like a race condition.

[dixonwille]

I am tieing it into my CI workflow and I decided to change the version number (keeping go version at 1.13.3 on linux). I can reproduce with 1.20.1 and 1.20.0. Once I went to version 1.19.1 it worked as expected.

[dixonwille]

I think it may be the unused linter. Reading through the changes and all the memory improvements being made. removing it from the list of linters for me and I no longer get this warning.

[ihgann]

Confirming I am seeing this issue as well.

[ryboe]

Disabling unused also eliminated the warning for us.

[daixiang0]

Seems something wrong in unused linter.

[daixiang0]

@jirfag hi, any update?

[Rameshsachin]

@jirfag hi, any update?.
gofmt and misspell also have a same behaviour.

[duduwe]

Can you install another golangci-lint version, run the linter, and check the 1st two warning lines. There might be files that prevent golangci from executing properly. Compiling and building may work fine though. Here's probably a similar case as yours. https://github.com/golangci/golangci-lint/issues/827#issuecomment-616590392

[ernado]

Is this still relevant? @daixiang0