We should be able to parallelize large parts of cmd/compile to make builds faster.
Parsing could be done in parallel on multiple files in a package.
Typechecking may be parallelizeable to some extent, I'm not sure.
Things like escape analysis might have to remain serial.
Compiling functions (order+walk+SSA+obj) can easily be done on each function in parallel. My only concern is making sure we don't use too much memory.
For all of these changes, we'd need to de-globalize the current compiler.
See also #9887 #15736
@josharian
randall77
All 91 comments
@griesemer @mdempsky @dr2chase
randall77
on 19 May 2016
CL https://golang.org/cl/26661 mentions this issue.
gopherbot
on 10 Aug 2016
CL https://golang.org/cl/38147 mentions this issue.
gopherbot
on 14 Mar 2017
CL https://golang.org/cl/38152 mentions this issue.
gopherbot
on 14 Mar 2017
CL https://golang.org/cl/38153 mentions this issue.
gopherbot
on 14 Mar 2017
CL https://golang.org/cl/38151 mentions this issue.
gopherbot
on 14 Mar 2017
CL https://golang.org/cl/38159 mentions this issue.
gopherbot
on 15 Mar 2017
CL https://golang.org/cl/38231 mentions this issue.
gopherbot
on 16 Mar 2017
CL https://golang.org/cl/38232 mentions this issue.
gopherbot
on 16 Mar 2017
CL https://golang.org/cl/38329 mentions this issue.
gopherbot
on 17 Mar 2017
CL https://golang.org/cl/38330 mentions this issue.
gopherbot
on 17 Mar 2017
CL https://golang.org/cl/38331 mentions this issue.
gopherbot
on 17 Mar 2017
CL https://golang.org/cl/38327 mentions this issue.
gopherbot
on 17 Mar 2017
CL https://golang.org/cl/38328 mentions this issue.
gopherbot
on 17 Mar 2017
CL https://golang.org/cl/38336 mentions this issue.
gopherbot
on 18 Mar 2017
CL https://golang.org/cl/38342 mentions this issue.
gopherbot
on 19 Mar 2017
CL https://golang.org/cl/38340 mentions this issue.
gopherbot
on 19 Mar 2017
CL https://golang.org/cl/38417 mentions this issue.
gopherbot
on 21 Mar 2017
CL https://golang.org/cl/38424 mentions this issue.
gopherbot
on 21 Mar 2017
CL https://golang.org/cl/38441 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38445 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38446 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38444 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38450 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38409 mentions this issue.
gopherbot
on 22 Mar 2017
CL https://golang.org/cl/38418 mentions this issue.
gopherbot
on 23 Mar 2017
CL https://golang.org/cl/38590 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38593 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38592 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38591 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38605 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38609 mentions this issue.
gopherbot
on 24 Mar 2017
CL https://golang.org/cl/38655 mentions this issue.
gopherbot
on 25 Mar 2017
CL https://golang.org/cl/38662 mentions this issue.
gopherbot
on 26 Mar 2017
CL https://golang.org/cl/38663 mentions this issue.
gopherbot
on 26 Mar 2017
CL https://golang.org/cl/38665 mentions this issue.
gopherbot
on 26 Mar 2017
CL https://golang.org/cl/38667 mentions this issue.
gopherbot
on 26 Mar 2017
CL https://golang.org/cl/38668 mentions this issue.
gopherbot
on 26 Mar 2017
CL https://golang.org/cl/38715 mentions this issue.
gopherbot
on 27 Mar 2017
CL https://golang.org/cl/38721 mentions this issue.
gopherbot
on 28 Mar 2017
CL https://golang.org/cl/38728 mentions this issue.
gopherbot
on 28 Mar 2017
CL https://golang.org/cl/38727 mentions this issue.
gopherbot
on 28 Mar 2017
CL https://golang.org/cl/38797 mentions this issue.
gopherbot
on 30 Mar 2017
CL https://golang.org/cl/38912 mentions this issue.
gopherbot
on 30 Mar 2017
CL https://golang.org/cl/38915 mentions this issue.
gopherbot
on 30 Mar 2017
CL https://golang.org/cl/38973 mentions this issue.
gopherbot
on 30 Mar 2017
CL https://golang.org/cl/38972 mentions this issue.
gopherbot
on 30 Mar 2017
CL https://golang.org/cl/39032 mentions this issue.
gopherbot
on 31 Mar 2017
CL https://golang.org/cl/39191 mentions this issue.
gopherbot
on 31 Mar 2017
CL https://golang.org/cl/39192 mentions this issue.
gopherbot
on 31 Mar 2017
CL https://golang.org/cl/39194 mentions this issue.
gopherbot
on 31 Mar 2017
CL https://golang.org/cl/39196 mentions this issue.
gopherbot
on 1 Apr 2017
CL https://golang.org/cl/39197 mentions this issue.
gopherbot
on 1 Apr 2017
CL https://golang.org/cl/39200 mentions this issue.
gopherbot
on 1 Apr 2017
CL https://golang.org/cl/39208 mentions this issue.
gopherbot
on 3 Apr 2017
CL https://golang.org/cl/39291 mentions this issue.
gopherbot
on 3 Apr 2017
CL https://golang.org/cl/39554 mentions this issue.
gopherbot
on 5 Apr 2017
CL https://golang.org/cl/39634 mentions this issue.
gopherbot
on 5 Apr 2017
CL https://golang.org/cl/39633 mentions this issue.
gopherbot
on 5 Apr 2017
CL https://golang.org/cl/39710 mentions this issue.
gopherbot
on 6 Apr 2017
CL https://golang.org/cl/39853 mentions this issue.
gopherbot
on 7 Apr 2017
CL https://golang.org/cl/39915 mentions this issue.
gopherbot
on 7 Apr 2017
CL https://golang.org/cl/39917 mentions this issue.
gopherbot
on 7 Apr 2017
CL https://golang.org/cl/39922 mentions this issue.
gopherbot
on 7 Apr 2017
CL https://golang.org/cl/39990 mentions this issue.
gopherbot
on 7 Apr 2017
CL https://golang.org/cl/39994 mentions this issue.
gopherbot
on 8 Apr 2017
@josharian is there a design document about this work? I'm especially interested in hearing why such a big effort is being attempted compared to just spawning multiple processes (like make -j). Also, it looks like there will be an ongoing cost on keeping the compiler concurrency-safe after this work is finished (e.g.: running trybots with -race, etc.), so I was wondering if there's a public log of the discussion.
rasky
on 8 Apr 2017
@rasky, cmd/go already spawns multiple processes for different packages. It works well, but not as well as if single packages were also parallelized. That's what Josh is doing.
bradfitz
on 8 Apr 2017
Also, the most common development cycle is incremental compilation, where the most important factor is recompiling ~1 package and linking. In that case, making the single package compile fast is very important.
For lots more discussion of this and related topics, start at #15734, #15681, #8893, #15752, #15736, and follow links from there. :)
josharian
on 8 Apr 2017
CL https://golang.org/cl/40172 mentions this issue.
gopherbot
on 10 Apr 2017
CL https://golang.org/cl/40254 mentions this issue.
gopherbot
on 10 Apr 2017
CL https://golang.org/cl/40252 mentions this issue.
gopherbot
on 11 Apr 2017
CL https://golang.org/cl/40313 mentions this issue.
gopherbot
on 11 Apr 2017
CL https://golang.org/cl/40370 mentions this issue.
gopherbot
on 11 Apr 2017
CL https://golang.org/cl/40371 mentions this issue.
gopherbot
on 11 Apr 2017
CL https://golang.org/cl/40372 mentions this issue.
gopherbot
on 11 Apr 2017
CL https://golang.org/cl/40408 mentions this issue.
gopherbot
on 12 Apr 2017
CL https://golang.org/cl/40496 mentions this issue.
gopherbot
on 12 Apr 2017
CL https://golang.org/cl/40495 mentions this issue.
gopherbot
on 12 Apr 2017
CL https://golang.org/cl/40502 mentions this issue.
gopherbot
on 13 Apr 2017
CL https://golang.org/cl/40507 mentions this issue.
gopherbot
on 13 Apr 2017
CL https://golang.org/cl/40509 mentions this issue.
gopherbot
on 13 Apr 2017
CL https://golang.org/cl/40693 mentions this issue.
gopherbot
on 14 Apr 2017
CL https://golang.org/cl/40697 mentions this issue.
gopherbot
on 14 Apr 2017
CL https://golang.org/cl/40853 mentions this issue.
gopherbot
on 16 Apr 2017
CL https://golang.org/cl/41050 mentions this issue.
gopherbot
on 21 Apr 2017
CL https://golang.org/cl/38713 mentions this issue.
gopherbot
on 21 Apr 2017
CL https://golang.org/cl/41820 mentions this issue.
gopherbot
on 26 Apr 2017
CL https://golang.org/cl/41503 mentions this issue.
gopherbot
on 26 Apr 2017
CL https://golang.org/cl/41819 mentions this issue.
gopherbot
on 26 Apr 2017
CL https://golang.org/cl/41192 mentions this issue.
gopherbot
on 26 Apr 2017
Related issues
mingrammer
路
3Comments
stub42
路
3Comments
natefinch
路
3Comments
dominikh
路
3Comments
bbodenmiller
路
3Comments
Most helpful comment
Also, the most common development cycle is incremental compilation, where the most important factor is recompiling ~1 package and linking. In that case, making the single package compile fast is very important.
For lots more discussion of this and related topics, start at #15734, #15681, #8893, #15752, #15736, and follow links from there. :)