I migrated away from govendor, and this tool is really nice to work with, everything seems really thought out, so thank you for making it. But now let me jump into my problem.
go version go1.8.1 linux/amd64, Ubuntu 16.04 -- 100mbit internet connection.
I realize that you guys are still yet to do a lot of optimization, and I'm not complaining, but would like to know if this is expected behaviour and not some type of bug. When running my initial dep ensure it takes 5 minutes 25 seconds to complete, seems pretty excessive.
Here's the output:
~/g/s/g/n/lolwtfspeed (master|…) $ time dep ensure -v
Root project is "github.com/nullbio/lolwtfspeed"
6 transitively valid internal packages
16 external packages imported from 9 projects
(0) ✓ select (root)
(1) ? attempt github.com/markbates/refresh with 1 pkgs; 3 versions to try
(1) try github.com/markbates/refresh@90f2daf1
(1) ✓ select github.com/markbates/refresh@90f2daf1 w/2 pkgs
(2) ? attempt github.com/mitchellh/go-homedir with 1 pkgs; 1 versions to try
(2) try github.com/mitchellh/go-homedir@master
(2) ✓ select github.com/mitchellh/go-homedir@master w/1 pkgs
(3) ? attempt gopkg.in/yaml.v2 with 1 pkgs; 1 versions to try
(3) try gopkg.in/yaml.v2@v2
(3) ✓ select gopkg.in/yaml.v2@v2 w/1 pkgs
(4) ? attempt github.com/unrolled/render with 1 pkgs; 2 versions to try
(4) try github.com/unrolled/render@master
(4) ✓ select github.com/unrolled/render@master w/1 pkgs
(5) ? attempt github.com/spf13/cobra with 1 pkgs; 3 versions to try
(5) try github.com/spf13/cobra@e606913c
(5) ✓ select github.com/spf13/cobra@e606913c w/1 pkgs
(6) ? attempt github.com/inconshreveable/mousetrap with 1 pkgs; 1 versions to try
(6) try github.com/inconshreveable/mousetrap@master
(6) ✓ select github.com/inconshreveable/mousetrap@master w/1 pkgs
(7) ? attempt github.com/markbates/going with 1 pkgs; 2 versions to try
(7) try github.com/markbates/going@master
(7) ✓ select github.com/markbates/going@master w/1 pkgs
(8) ? attempt go.uber.org/zap with 2 pkgs; 11 versions to try
(8) try go.uber.org/[email protected]
(8) ✓ select go.uber.org/[email protected] w/7 pkgs
(9) ? attempt github.com/volatiletech/abcweb with 6 pkgs; 5 versions to try
(9) try github.com/volatiletech/[email protected]
(9) ✓ select github.com/volatiletech/[email protected] w/6 pkgs
(10) ? attempt github.com/kat-co/vala with 1 pkgs; 2 versions to try
(10) try github.com/kat-co/vala@v1
(10) ✓ select github.com/kat-co/vala@v1 w/1 pkgs
(11) ? attempt github.com/satori/go.uuid with 1 pkgs; 3 versions to try
(11) try github.com/satori/[email protected]
(11) ✓ select github.com/satori/[email protected] w/1 pkgs
(12) ? attempt github.com/volatiletech/helpers with 1 pkgs; 2 versions to try
(12) try github.com/volatiletech/[email protected]
(12) ✓ select github.com/volatiletech/[email protected] w/1 pkgs
(13) ? attempt github.com/djherbis/times with 1 pkgs; 5 versions to try
(13) try github.com/djherbis/[email protected]
(13) ✓ select github.com/djherbis/[email protected] w/1 pkgs
(14) ? attempt github.com/volatiletech/mig with 1 pkgs; 4 versions to try
(14) try github.com/volatiletech/[email protected]
(14) ✓ select github.com/volatiletech/[email protected] w/1 pkgs
(15) ? attempt github.com/spf13/pflag with 1 pkgs; 3 versions to try
(15) try github.com/spf13/pflag@e57e3eeb
(15) ✓ select github.com/spf13/pflag@e57e3eeb w/1 pkgs
(16) ? attempt github.com/lib/pq with 1 pkgs; 6 versions to try
(16) try github.com/lib/pq@master
(16) ✓ select github.com/lib/pq@master w/2 pkgs
(17) ? attempt github.com/fatih/color with 1 pkgs; 10 versions to try
(17) try github.com/fatih/[email protected]
(17) ✓ select github.com/fatih/[email protected] w/1 pkgs
(18) ? attempt github.com/pkg/errors with 1 pkgs; 11 versions to try
(18) try github.com/pkg/[email protected]
(18) ✓ select github.com/pkg/[email protected] w/1 pkgs
(19) ? attempt go.uber.org/atomic with 1 pkgs; 6 versions to try
(19) try go.uber.org/[email protected]
(19) ✓ select go.uber.org/[email protected] w/1 pkgs
(20) ? attempt github.com/vattle/sqlboiler with 1 pkgs; 22 versions to try
(20) try github.com/vattle/[email protected]
(20) ✓ select github.com/vattle/[email protected] w/3 pkgs
(21) ? attempt github.com/nullbio/inflect with 1 pkgs; 1 versions to try
(21) try github.com/nullbio/inflect@master
(21) ✓ select github.com/nullbio/inflect@master w/1 pkgs
(22) ? attempt github.com/mattn/go-colorable with 1 pkgs; 9 versions to try
(22) try github.com/mattn/[email protected]
(22) ✓ select github.com/mattn/[email protected] w/1 pkgs
(23) ? attempt gopkg.in/redis.v5 with 1 pkgs; 22 versions to try
(23) try gopkg.in/[email protected]
(23) ✓ select gopkg.in/[email protected] w/6 pkgs
(24) ? attempt github.com/spf13/viper with 1 pkgs; 2 versions to try
(24) try github.com/spf13/viper@master
(24) ✓ select github.com/spf13/viper@master w/1 pkgs
(25) ? attempt github.com/mitchellh/mapstructure with 1 pkgs; 1 versions to try
(25) try github.com/mitchellh/mapstructure@master
(25) ✓ select github.com/mitchellh/mapstructure@master w/1 pkgs
(26) ? attempt github.com/spf13/afero with 1 pkgs; 1 versions to try
(26) try github.com/spf13/afero@master
(26) ✓ select github.com/spf13/afero@master w/2 pkgs
(27) ? attempt golang.org/x/text with 2 pkgs; 1 versions to try
(27) try golang.org/x/text@master
(27) ✓ select golang.org/x/text@master w/6 pkgs
(28) ? attempt github.com/spf13/jwalterweatherman with 1 pkgs; 2 versions to try
(28) try github.com/spf13/jwalterweatherman@master
(28) ✓ select github.com/spf13/jwalterweatherman@master w/1 pkgs
(29) ? attempt github.com/hashicorp/hcl with 1 pkgs; 4 versions to try
(29) try github.com/hashicorp/hcl@master
(29) ✓ select github.com/hashicorp/hcl@master w/9 pkgs
(30) ? attempt github.com/spf13/cast with 1 pkgs; 3 versions to try
(30) try github.com/spf13/[email protected]
(30) ✓ select github.com/spf13/[email protected] w/1 pkgs
(31) ? attempt github.com/pressly/chi with 2 pkgs; 9 versions to try
(31) try github.com/pressly/[email protected]
(31) ✓ select github.com/pressly/[email protected] w/2 pkgs
(32) ? attempt github.com/fsnotify/fsnotify with 1 pkgs; 32 versions to try
(32) try github.com/fsnotify/[email protected]
(32) ✓ select github.com/fsnotify/[email protected] w/1 pkgs
(33) ? attempt golang.org/x/sys with 1 pkgs; 1 versions to try
(33) try golang.org/x/sys@master
(33) ✓ select golang.org/x/sys@master w/1 pkgs
(34) ? attempt github.com/pelletier/go-toml with 1 pkgs; 13 versions to try
(34) try github.com/pelletier/[email protected]
(34) ✓ select github.com/pelletier/[email protected] w/1 pkgs
(35) ? attempt github.com/pelletier/go-buffruneio with 1 pkgs; 3 versions to try
(35) try github.com/pelletier/[email protected]
(35) ✓ select github.com/pelletier/[email protected] w/1 pkgs
(36) ? attempt github.com/magiconair/properties with 1 pkgs; 27 versions to try
(36) try github.com/magiconair/[email protected]
(36) ✓ select github.com/magiconair/[email protected] w/1 pkgs
(37) ? attempt github.com/go-sql-driver/mysql with 1 pkgs; 18 versions to try
(37) try github.com/go-sql-driver/[email protected]
(37) ✓ select github.com/go-sql-driver/[email protected] w/1 pkgs
(38) ? attempt github.com/denisenkom/go-mssqldb with 1 pkgs; 6 versions to try
(38) try github.com/denisenkom/go-mssqldb@master
(38) ✓ select github.com/denisenkom/go-mssqldb@master w/1 pkgs
(39) ? attempt golang.org/x/crypto with 1 pkgs; 1 versions to try
(39) try golang.org/x/crypto@master
(39) ✓ select golang.org/x/crypto@master w/1 pkgs
(40) ? attempt github.com/mattn/go-isatty with 1 pkgs; 4 versions to try
(40) try github.com/mattn/[email protected]
(40) ✓ select github.com/mattn/[email protected] w/1 pkgs
(41) ? attempt golang.org/x/net with 1 pkgs; 4 versions to try
(41) try golang.org/x/net@master
(41) ✓ select golang.org/x/net@master w/1 pkgs
✓ found solution with 76 packages from 41 projects
Solver wall times by segment:
b-source-exists: 2m42.365979365s
b-list-versions: 2m36.886150849s
b-gmal: 2.417451312s
b-deduce-proj-root: 1.834486434s
b-list-pkgs: 1.443041966s
select-root: 14.292374ms
select-atom: 6.752476ms
satisfy: 4.872988ms
new-atom: 3.812457ms
other: 71.056µs
b-matches: 4.077µs
b-pair-rev: 2.692µs
b-pair-version: 343ns
TOTAL: 5m24.976918389s
5.07user 1.37system 5:25.35elapsed 1%CPU (0avgtext+0avgdata 33572maxresident)k
89216inputs+104968outputs (75major+1135413minor)pagefaults 0swaps
It sat on no output at all for quite a long amount of time, and then the slowest parts were:
(3) try gopkg.in/yaml.v2@v2
(34) try github.com/pelletier/[email protected]
Here's my Gopkg.toml file:
[[constraint]]
revision = "90f2daf1"
name = "github.com/markbates/refresh"
[[constraint]]
version = "^0.8.0"
name = "github.com/pkg/errors"
[[constraint]]
version = "^2.1.0"
name = "github.com/pressly/chi"
[[constraint]]
revision = "e606913c"
name = "github.com/spf13/cobra"
[[constraint]]
revision = "e57e3eeb"
name = "github.com/spf13/pflag"
[[constraint]]
branch = "v1"
name = "github.com/unrolled/render"
[[constraint]]
version = "^2.0.0"
name = "github.com/vattle/sqlboiler"
[[constraint]]
version = "^2.0.0"
name = "github.com/volatiletech/abcweb"
[[constraint]]
version = "^1.0.0"
name = "github.com/volatiletech/helpers"
[[constraint]]
version = "^1.1.0"
name = "github.com/volatiletech/mig"
[[constraint]]
version = "^1.0.0"
name = "go.uber.org/zap"
[[constraint]]
branch = "master"
name = "gopkg.in/nullbio/null.v6"
Thank you.
nullbio
All 9 comments
@nullbio did you happen to do this during the github.com outage?
joegrasse
on 31 May 2017
@joegrasse I'm not sure, I didn't notice any outage. I've just run it again now and it's still taking about the same amount of time. If there was an outage perhaps Github is still experiencing issues? I can try it again tomorrow just to confirm if that helps.
nullbio
on 31 May 2017
hi, welcome!
I realize that you guys are still yet to do a lot of optimization
We definitely do 😄 https://github.com/golang/dep/blob/master/FAQ.md#why-is-dep-slow
I'm not complaining, but would like to know if this is expected behaviour and not some type of bug. When running my initial dep ensure it takes 5 minutes 25 seconds to complete, seems pretty excessive.
It's all good - that certainly does strike me as excessively long.
did you happen to do this during the github.com outage?
This was also my thought. That these two operations:
b-source-exists: 2m42.365979365s b-list-versions: 2m36.886150849s
Are what took so long is strongly suggestive that there was an issue communicating with github, as both of those operations involve relatively simple reach-out-and-touch that, for most repositories, have a response payload of <1k.
sdboyer
on 31 May 2017
@sdboyer Thanks Sam. After running it again today it took about 1min 30s. Seems Github was definitely causing problems. Is that more in line with expected values?
nullbio
on 1 Jun 2017
@nullbio sure! for 41 projects, 1m30s seems about right, for the current architecture. once we get our persistent caching regime going, the above solve run on a warm cache should be <1s 😄
(though writing out 41 repos to disk may take another couple seconds)
sdboyer
on 1 Jun 2017
@sdboyer Awesome :-) Thanks for making such a cool tool. Looking forward to seeing where you go from here. Take care.
nullbio
on 1 Jun 2017
i had similar issues today (~5 min ensure), turns out just my bad network. I would think this possible to detect. Any thoughts on warning for slow network tasks?
rileytg
on 2 Jun 2017
@rileytg how do you envision that working?
my first thought is that it seems like a fair bit of instrumentation work for marginal benefit. even if we tell the user that's what's going on, what would they do with that information that's different than what they already do? it also seems pretty uncharacteristic of CLI tools to try to do adaptive network quality detection and inform the user about it - scope creep.
sdboyer
on 2 Jun 2017
ill buy no b/c scope creep
rileytg
on 4 Jun 2017
Related issues
lwc
·
3Comments
sdboyer
·
3Comments
cemremengu
·
3Comments
carolynvs
·
3Comments
jjyr
·
3Comments
Most helpful comment
ill buy no b/c scope creep