Golangci-lint: [QUESTION] Go packages loading is too slow in docker

[kpeu3i]

Hi! I am wondered why the time of loading modules in a docker container is significantly greater than on a host machine (macOS)? Yes, I know about poor file system performance on Mac in case of bind-mounted volumes. But I have a similar result on a Linux machine.

.golangci.yml

run:
    deadline: 2m
    skip-dirs:
        - "mock"
        - "features"
        - "scratch"
    skip-files:
        - "gen.go$"
        - ".*_test.go"

linters:
    enable-all: true
    disable:
        - gochecknoglobals
        - godox
        - maligned
        - wsl
        - interfacer
        - dupl
        - funlen

linters-settings:
    goimports:
        local-prefixes: github.com/golangci/golangci-lint

1. macOS, docker (bind-mounted volumes)

Go packages loading: 22.1404275s

CMD:

docker run --rm -v $(pwd):/app -v ${GOPATH}/pkg/mod:/go/pkg/mod -w /app golangci/golangci-lint:v1.24.0 golangci-lint run -v

Output:

level=info msg="[config_reader] Used config file .golangci.yml"
level=info msg="[lintersdb] Active 34 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd goprintffuncname gosec gosimple govet ineffassign lll misspell nakedret prealloc rowserrcheck scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]"
level=info msg="[lintersdb] Active 34 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd goprintffuncname gosec gosimple govet ineffassign lll misspell nakedret prealloc rowserrcheck scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]"
level=info msg="[loader] Go packages loading at mode 575 (compiled_files|exports_file|name|deps|files|imports|types_sizes) took 22.1404275s"
level=info msg="[runner/filename_unadjuster] Pre-built 0 adjustments in 183.3723ms"
level=info msg="[runner/unused/goanalysis] analyzers took 54.3041456s with top 10 stages: buildir: 54.129421s, U1000: 174.7246ms"
level=info msg="[runner/goanalysis_metalinter/goanalysis] analyzers took 1m23.6671276s with top 10 stages: buildir: 45.6301594s, buildssa: 3.982641s, goimports: 3.0779317s, ineffassign: 2.7826672s, misspell: 2.7148599s, gofmt: 2.6973646s, unparam: 2.6857799s, golint: 2.6506173s, isgenerated: 2.5840667s, lll: 2.2563254s"
level=info msg="[runner/skip dirs] Skipped 2 issues from dir scratch/1 by pattern scratch"
level=info msg="[runner/skip dirs] Skipped 5 issues from dir scratch by pattern scratch"
level=info msg="[runner/skip dirs] Skipped 3 issues from dir scratch/2 by pattern scratch"
level=info msg="[runner] Issues before processing: 989, after processing: 0"
level=info msg="[runner] Processors filtering stat (out/in): cgo: 989/989, identifier_marker: 860/860, nolint: 0/14, path_prettifier: 989/989, exclude-rules: 14/14, skip_files: 989/989, autogenerated_exclude: 860/979, filename_unadjuster: 989/989, skip_dirs: 979/989, exclude: 14/860"
level=info msg="[runner] processing took 688.7819ms with stages: path_prettifier: 324.7539ms, autogenerated_exclude: 279.9407ms, nolint: 30.4729ms, exclude: 24.5177ms, identifier_marker: 16.28ms, skip_dirs: 11.132ms, skip_files: 1.4366ms, cgo: 79µs, filename_unadjuster: 65.4µs, path_shortener: 17µs, uniq_by_line: 13.8µs, max_same_issues: 13.1µs, max_from_linter: 12.2µs, diff: 12.2µs, source_code: 11.9µs, exclude-rules: 11.8µs, max_per_file_from_linter: 11.7µs"
level=info msg="[runner] linters took 25.0067876s with stages: unused: 15.7876423s, goanalysis_metalinter: 8.5295692s"
level=info msg="File cache stats: 199 entries of total size 364.0KiB"
level=info msg="Memory: 422 samples, avg is 1000.2MB, max is 3705.2MB"
level=info msg="Execution took 47.3498858s"

2. macOS, host

CMD:

golangci-lint run -v

Go packages loading: 750.553194ms

INFO [config_reader] Used config file .golangci.yml
INFO [lintersdb] Active 34 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd goprintffuncname gosec gosimple govet ineffassign lll misspell nakedret prealloc rowserrcheck scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]
INFO [lintersdb] Active 34 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd goprintffuncname gosec gosimple govet ineffassign lll misspell nakedret prealloc rowserrcheck scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]
INFO [loader] Go packages loading at mode 575 (compiled_files|deps|files|imports|types_sizes|exports_file|name) took 750.553194ms
INFO [runner/filename_unadjuster] Pre-built 0 adjustments in 7.440391ms
INFO [runner/goanalysis_metalinter/goanalysis] analyzers took 0s with no stages
INFO [runner/unused/goanalysis] analyzers took 0s with no stages
INFO [runner/skip dirs] Skipped 5 issues from dir scratch by pattern scratch
INFO [runner/skip dirs] Skipped 2 issues from dir scratch/1 by pattern scratch
INFO [runner/skip dirs] Skipped 3 issues from dir scratch/2 by pattern scratch
INFO [runner] Issues before processing: 989, after processing: 0
INFO [runner] Processors filtering stat (out/in): identifier_marker: 860/860, exclude-rules: 14/14, skip_files: 989/989, path_prettifier: 989/989, cgo: 989/989, autogenerated_exclude: 860/979, exclude: 14/860, nolint: 0/14, filename_unadjuster: 989/989, skip_dirs: 979/989
INFO [runner] processing took 68.263429ms with stages: exclude: 29.962599ms, identifier_marker: 18.208828ms, nolint: 8.021528ms, autogenerated_exclude: 5.230585ms, path_prettifier: 4.633613ms, skip_files: 1.469733ms, skip_dirs: 602.784µs, cgo: 82.29µs, filename_unadjuster: 45.519µs, max_same_issues: 1.481µs, uniq_by_line: 1.115µs, exclude-rules: 704ns, diff: 669ns, source_code: 654ns, max_from_linter: 513ns, max_per_file_from_linter: 461ns, path_shortener: 353ns
INFO [runner] linters took 171.430888ms with stages: goanalysis_metalinter: 100.014066ms, unused: 2.981919ms
INFO File cache stats: 0 entries of total size 0B
INFO Memory: 12 samples, avg is 87.9MB, max is 137.8MB
INFO Execution took 1.014408857s

3. Linux, docker (gitlab-ci)

Go packages loading: 22.006963382s

lint:
  extends: .test
  image:
    name: golangci/golangci-lint:v1.22.2
    entrypoint: [""]
  script:
    - golangci-lint run -v

Output:
```
level=info msg="[config_reader] Used config file .golangci.yml"
level=info msg="[lintersdb] Active 32 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd gosec gosimple govet ineffassign lll misspell nakedret prealloc scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]"
level=info msg="[lintersdb] Active 32 linters: [bodyclose deadcode depguard dogsled errcheck gochecknoinits gocognit goconst gocritic gocyclo gofmt goimports golint gomnd gosec gosimple govet ineffassign lll misspell nakedret prealloc scopelint staticcheck structcheck stylecheck typecheck unconvert unparam unused varcheck whitespace]"
level=info msg="[loader] Go packages loading at mode 575 (exports_file|files|name|types_sizes|compiled_files|deps|imports) took 22.006963382s"
level=info msg="[runner/filename_unadjuster] Pre-built 0 adjustments in 24.720119ms"
level=info msg="[runner/unused/goanalysis] analyzers took 2.109622889s with top 10 stages: buildssa: 1.927047136s, U1000: 182.575753ms"
level=info msg="[runner/goanalysis_metalinter/goanalysis] analyzers took 33.316327212s with top 10 stages: buildssa: 18.773618487s, ctrlflow: 2.11388237s, printf: 2.075348031s, fact_purity: 1.890680495s, unconvert: 1.857950092s, fact_deprecated: 1.821037445s, inspect: 1.119102646s, goimports: 610.641329ms, unparam: 289.279668ms, gofmt: 268.70142ms"
level=info msg="[runner] Issues before processing: 1029, after processing: 0"
level=info msg="[runner] Processors filtering stat (out/in): skip_files: 974/1029, nolint: 0/19, cgo: 1029/1029, path_prettifier: 1029/1029, autogenerated_exclude: 855/974, exclude: 19/855, exclude-rules: 19/19, filename_unadjuster: 1029/1029, skip_dirs: 974/974, identifier_marker: 855/855"
level=info msg="[runner] processing took 77.764103ms with stages: exclude: 31.169762ms, identifier_marker: 22.437618ms, nolint: 11.380069ms, path_prettifier: 6.607466ms, skip_files: 2.862359ms, autogenerated_exclude: 2.349383ms, skip_dirs: 670.537µs, cgo: 143.8µs, filename_unadjuster: 136.431µs, max_same_issues: 2.099µs, uniq_by_line: 917ns, max_from_linter: 803ns, diff: 727ns, path_shortener: 651ns, source_code: 625ns, max_per_file_from_linter: 459ns, exclude-rules: 397ns"
level=info msg="[runner] linters took 10.801212344s with stages: goanalysis_metalinter: 9.278804731s, unused: 1.444438899s"
level=info msg="File cache stats: 196 entries of total size 357.5KiB"
level=info msg="Memory: 324 samples, avg is 162.7MB, max is 541.7MB"
level=info msg="Execution took 32.85263907s"

[dackroyd]

I think I'm having the same issue: https://github.com/golangci/golangci-lint/issues/807#issuecomment-605816631

What I found is that because the Go build cache ($GOCACHE) isn't populated in the Docker container, it's starting from scratch every time. This actually takes quite a bit to generate, but is (relatively) fast if you've already built those packages since it's pre-cached.

[damour]

@kpeu3i Try to add mod, build, linter caches.
Gitlab CI config example:

lint:
  stage: test
  image:
    name: golangci/golangci-lint:v1.21.0
    entrypoint: [""]
  script:
    - golangci-lint run -v --timeout=2m
  variables:
    GOCACHE: ${CI_PROJECT_DIR}/vendor/cache
    GOPATH: ${CI_PROJECT_DIR}/vendor/go
    GOLANGCI_LINT_CACHE: ${CI_PROJECT_DIR}/vendor/linter-cache
  cache:
    key: lintercache
    paths:
      - vendor/go/pkg/mod/
      - vendor/cache
      - vendor/linter-cache

[kpeu3i]

Thanks to @dackroyd and @damour I got ~2x speed boost on my Mac, using the following command:

docker run --rm -v $(pwd):/app -v $(go env GOCACHE):/cache/go -e GOCACHE=/cache/go -e GOLANGCI_LINT_CACHE=/cache/go -v ${GOPATH}/pkg:/go/pkg -w /app golangci/golangci-lint:v1.24.0 golangci-lint run -v

instead of command from doc (https://github.com/golangci/golangci-lint#docker):

docker run --rm -v $(pwd):/app -w /app golangci/golangci-lint:v1.24.0 golangci-lint run -v