As an experiment, I build "hello, world" at the release points for go 1.0.
1.1, and 1.2. Here are the binary's sizes:
% ls -l x.1.?
-rwxr-xr-x 1 r staff 1191952 Nov 30 10:25 x.1.0
-rwxr-xr-x 1 r staff 1525936 Nov 30 10:20 x.1.1
-rwxr-xr-x 1 r staff 2188576 Nov 30 10:18 x.1.2
% size x.1.?
__TEXT __DATA __OBJC others dec hex
880640 33682096 0 4112 34566848 20f72c0 x.1.0
1064960 94656 0 75952 1235568 12da70 x.1.1
1429504 147896 0 177440 1754840 1ac6d8 x.1.2
%
A near-doubling of the binary size in two releases is a bug of a kind. I will hold on to
the files so they can be analyzed more, but am filing this issue to get the topic
registered. We need to develop a better understanding of the problem and how to address
it.
Marking this 1.3 (not maybe) because I consider it a priority.
A few months ago I exchanged mail with Russ about this topic regarding a different, much
larger binary. To avoid him having to redo the analysis, here is what he said at the
time:
====
i sent CL 13722046 to make the nm -S output a bit more useful.
for the toy binary i now get
4a2280 1898528 D symtab
26f3a0 1405936 D type.*
671aa0 1058432 D pclntab
3c6790 598056 D go.string.*
4620c0 49600 D gcbss
7a7c20 45496 B runtime.mheap
46e280 21936 D gcdata
7a29e0 21056 b bufferList
1ed600 16480 T crypto/tls.(*Conn).clientHandshake
79eb20 16064 b semtable
1b3d90 14224 T net/http.init
that seems plausible to me. some notes:
symtab is the plan 9 symbol table. it in the binary but never referenced at run time. it
supports things like nm -S only. it needs to move into an unmapped section of the
binary, but it is only costing at most 8k at run time right now due to fragmentation and
it just wasn't worth the effort to try to move. the new linker will make this easier. of
course, moving it in the file doesn't shrink the file.
the thing named pclntab is a reencoding of the original pclntab and the parts of the
plan 9 symbol table that we did need at run time (mostly just a list of functions and
their names and addresses). as you can see, it is much smaller than the old form (the
symbol table dominates).
type.* is the reflect types and go.string.* is the static go string data. the *
indicates that i coalesced many symbols into one, to avoid useless individual names
bloating the symbol table. if we tried we could probably cut the reflect types by 2-4x.
it would mean packing the data a bit more compactly than an ordinary go data structure
would and then using unsafe to get it back out.
gcbss and gcdata are garbage collection bits for the bss and data segments. that's what
atom symbol did, and it's not clear whether it will last (probably not) and whether what
will replace it will be smaller. time will tell. i have a meeting with dmitriy, carl,
and keith next week to figure out what the plan is.
runtime.mheap, bufferList, and semtable are bss.
you're not seeing the gdb dwarf debug information here, because it's not a runtime
symbol.
g% otool -l $(which toy) | egrep '^ segname|filesize'
segname __PAGEZERO
filesize 0
segname __TEXT
filesize 7811072
segname __DATA
filesize 126560
segname __LINKEDIT
filesize 921772
segname __DWARF
filesize 2886943
g%
there's another 3 MB. you can build with -ldflags -w to get rid of that at least.
if you read the full otool -l output you will find
Load command 6
cmd LC_SYMTAB
cmdsize 24
symoff 10825728
nsyms 22559
stroff 11186924
strsize 560576
looks like another 1 MB or so (560576+11186924-10825728 or 22559*16+560576) for the
mach-o symbol table.
when we do the new linker we can make recording this kind of information in a useful
form a priority.
robpike
All 118 comments
Note: the binaries were build on amd64 10.7.5 (Lion), with gcc -version i686-apple-darwin11-llvm-gcc-4.2 For the record, I couldn't do this experiment on 10.9 with Xcode 5 because the older releases wouldn't build due to gcc/clang skew.
robpike
on 30 Nov 2013
Just for added reference, this are the size on go 1.1.2 vs 1.2 with OS X 10.9 and Xcode 5 (darwin gcc llvm 5.0 x86_64): $ ls -l *1.* -rwxr-xr-x 1 j staff 1525984 2 Dez 21:44 hello_1.1.2 -rwxr-xr-x 1 j staff 2192672 2 Dez 21:40 hello_1.2 $ size *1.* __TEXT __DATA __OBJC others dec hex 1064960 94720 0 76000 1235680 12dae0 hello_1.1.2 1433600 147896 0 177440 1758936 1ad6d8 hello_1.2
gopherbot
on 2 Dec 2013
This issue was updated by revision f238049a0073538caecfad1c60238a271426f43.
R=golang-dev, rsc CC=golang-dev https://golang.org/cl/35940047
randall77
on 3 Dec 2013
In this context, please reevaluate if the golang binaries can be changed to work with UPX (ultimate packer for executables).¹² For a small amount of computing power, upx can reduce the size of a binary down to a quarter of its original size. You can authenticate this using a existing example ›fixer‹ programm for golang binaries on linux/amd64.³ While this approach doesn't fix the root of the problem – only the symptoms – it would be nice to have this possibility always on hand. For technical background of this problem (PT_LOAD[0].p_offset==0), please look at the UPX bugtracker⁴. ¹ http://upx.sourceforge.net/ ² https://en.wikipedia.org/wiki/UPX ³ https://github.com/pwaller/goupx ⁴ http://sourceforge.net/p/upx/bugs/195/
zephyr
on 7 Dec 2013
minux
on 7 Dec 2013
minux: We may figure out why the binaries are growing fast first (lager runtime, optimization, etc.), before we claim that it isn't Go's problem.
leo-liu
on 8 Dec 2013
re #10, my #9 reply is to #8, which is about an entirely different problem. i'm not saying that the ever-growing binaries is not our problem, only that i don't believe that upx not accepting our binaries is our problem. it's clear that upx isn't able to handle all possible and correct ELF files (i.e. if the kernel can execute our binaries just fine, it's upx's problem to not be able to compress them).
minux
on 8 Dec 2013
More detail. The Plan 9 symbol table is about to be deleted. Here is a reference point, adding one new entry to the list above: $ ls -l *1.* -rwxr-xr-x 1 j staff 1525984 2 Dez 21:44 hello_1.1.2 -rwxr-xr-x 1 j staff 2192672 2 Dez 21:40 hello_1.2 -rwxr-xr-x 1 j staff 2474512 Feb 18 20:27 hello_1.2.x $ size *1.* __TEXT __DATA __OBJC others dec hex 1064960 94720 0 76000 1235680 12dae0 hello_1.1.2 1433600 147896 0 177440 1758936 1ad6d8 hello_1.2 1699840 160984 0 188944 2049768 1f46e8 hello_1.2.x Text has grown substantially, as has data. At least some of this is due to new annotations for the garbage collector.
robpike
on 19 Feb 2014
More detail. The Plan 9 symbol table is about to be deleted. Here is a reference point, adding one new entry to the list above: $ ls -l *1.* -rwxr-xr-x 1 r staff 1191952 Nov 30 10:25 x.1.0 -rwxr-xr-x 1 r staff 1525936 Nov 30 10:20 x.1.1 -rwxr-xr-x 1 r staff 2188576 Nov 30 10:18 x.1.2 -rwxr-xr-x 1 r staff 2474512 Feb 18 20:27 hello_1.2.x $ size *1.* __TEXT __DATA __OBJC others dec hex 880640 33682096 0 4112 34566848 20f72c0 x.1.0 1064960 94656 0 75952 1235568 12da70 x.1.1 1429504 147896 0 177440 1754840 1ac6d8 x.1.2 1699840 160984 0 188944 2049768 1f46e8 hello_1.2.x Text has grown substantially, as has data. At least some of this is due to new annotations for the garbage collector.
robpike
on 19 Feb 2014
More detail. The Plan 9 symbol table is about to be deleted. Here is a reference point, adding one new entry to the list above: $ ls -l *1.* -rwxr-xr-x 1 r staff 1191952 Nov 30 10:25 x.1.0 -rwxr-xr-x 1 r staff 1525936 Nov 30 10:20 x.1.1 -rwxr-xr-x 1 r staff 2188576 Nov 30 10:18 x.1.2 -rwxr-xr-x 1 r staff 2474512 Feb 18 20:27 hello_1.2.x $ size *1.* __TEXT __DATA __OBJC others dec hex 880640 33682096 0 4112 34566848 20f72c0 x.1.0 1064960 94656 0 75952 1235568 12da70 x.1.1 1429504 147896 0 177440 1754840 1ac6d8 x.1.2 1699840 160984 0 188944 2049768 1f46e8 x.1.2.x Text has grown substantially, as has data. At least some of this is due to new annotations for the garbage collector.
robpike
on 19 Feb 2014
This issue was updated by revision 964f6d3ec4c6e2bed377878bd2862767bfae463.
Nothing reads the Plan 9 symbol table anymore. The last holdout was 'go tool nm', but since being rewritten in Go it uses the standard symbol table for the binary format (ELF, Mach-O, PE) instead. Removing the Plan 9 symbol table saves ~15% disk space on most binaries. Two supporting changes included in this CL: debug/gosym: use Go 1.2 pclntab to synthesize func-only symbol table when there is no Plan 9 symbol table debug/elf, debug/macho, debug/pe: ignore final EOF from ReadAt LGTM=r R=r, bradfitz CC=golang-codereviews https://golang.org/cl/65740045
rsc
on 19 Feb 2014
After revision 737767dd81fd, I see a 25% reduction: Before: -rwxr-xr-x 1 bradfitz staff 23556028 Feb 18 20:47 bin/camlistored After: -rwxr-xr-x 1 bradfitz staff 17727420 Feb 18 20:48 bin/camlistored
bradfitz
on 19 Feb 2014
For my test case before/after deleting the Plan 9 symbol table: % ls -l ... -rwxr-xr-x 1 r staff 2474512 Feb 18 20:27 hello_1.2.x -rwxr-xr-x 1 r staff 2150928 Feb 18 22:28 hello_1.2.y % size ... __TEXT __DATA __OBJC others dec hex 1699840 160984 0 188944 2049768 1f46e8 hello_1.2.x 1376256 160984 0 188944 1726184 1a56e8 hello_1.2.x % So deleting the Plan 9 symbol table pretty close to exactly compensates for the GC information. We're back at Go 1.2 levels, still far too large but it's a start.
robpike
on 19 Feb 2014
This issue was updated by revision 2541cc81978dc5e41e2e2db6345d8ca7a365ca8.
Every function now has a gcargs and gclocals symbol holding associated garbage collection information. Put them all in the same meta-symbol as the go.func data and then drop individual entries from symbol table. Removing gcargs and gclocals reduces the size of a typical binary by 10%. LGTM=r R=r CC=golang-codereviews https://golang.org/cl/65870044
rsc
on 19 Feb 2014
This issue was updated by revision ae38b03f6cab6a25f9d8d34a39e33db9857dce2.
For an ephemeral binary - one created, run, and then deleted - there is no need to write dwarf debug information, since the binary will not be used with gdb. In this case, instruct the linker not to spend time and disk space generating the debug information by passing the -w flag to the linker. Omitting dwarf information reduces the size of most binaries by 25%. We may be more aggressive about this in the future. LGTM=bradfitz, r R=r, bradfitz CC=golang-codereviews https://golang.org/cl/65890043
rsc
on 19 Feb 2014
After removing gcargs from the symbol table (stepping across CL 65870044) % ls -l x.1.2.[yz] -rwxr-xr-x 1 r staff 2150928 Feb 18 22:28 hello_1.2.y -rwxr-xr-x 1 r staff 1932880 Feb 19 08:14 hello_1.2.z % size x.1.2.[yz] __TEXT __DATA __OBJC others dec hex 1376256 160984 0 188944 1726184 1a56e8 hello_1.2.y 1376256 160984 0 110160 1647400 192328 hello_1.2.z % It's now smaller than at 1.2 but still much bigger than 1.1, let alone 1.0.
robpike
on 19 Feb 2014
gopherbot
on 4 Mar 2014
The pclntab data should be reasonably compact, and note that fast access to the data is important, since it is used for runtime.Callers and friends. If you can find a significant reduction in size that would be great, but small tweaks are probably not desirable at this point.
ianlancetaylor
on 4 Mar 2014
Is there any documentation for what the pclntab contains and for what constraints its data structure must fullfill?
gopherbot
on 4 Mar 2014
Let's not use this issue as a discussion list. Please ask questions on golang-dev. Thanks.
ianlancetaylor
on 4 Mar 2014
From the peanut gallery, AFAICT this breaks pprof interactive command 'list'. At tip I get, e.g.: (pprof) list runner Total: 6424 samples objdump: syminit: Success no filename found in main.runner<400c40> Which works as expected with 1.2.1.
gopherbot
on 7 Mar 2014
Let's not use this issue as a discussion list. Please ask questions on golang-dev. Thanks. pprof not working is issue #7452.
_Labels changed: added restrict-addissuecomment-commit._
rsc
on 3 Apr 2014
This is as fixed as it is going to be for Go 1.3.
Right now at tip + CL 80370045 on darwin/amd64, compiling this program:
package main
import "fmt"
func main() {
fmt.Println("hello, world")
}
I get 1830352 bytes for the binary. Assuming this is the same case for which Rob's
numbers are reported, by this metric Go 1.3 will roll back more than half the size
increase caused by Go 1.2 (relative to Go 1.1). Will leave further improvement for Go
1.4.
_Labels changed: added release-go1.4, removed release-go1.3._
rsc
on 3 Apr 2014
This issue was updated by revision a26c01ad446d2853f0c6a7ddaacadb02efa00b7.
LGTM=khr R=khr CC=golang-codereviews https://golang.org/cl/80370045
rsc
on 4 Apr 2014
From Go 1.4 to Go tip on darwin/amd64, helloworld grows from 1927792 to 2344944 bytes (+22%).
Camlistore shows more modest increases, but they are non-trivial nevertheless:
| File | Go 1.4 | Go tip | Delta |
| --- | --- | --- | --- |
| camdeploy | 12750156 | 13512052 | +5.97% |
| camget | 12378300 | 13261956 | +7.13% |
| camlistored | 28374844 | 29469892 | +3.85% |
| cammount | 13586156 | 14496036 | +6.69% |
| camput | 14040524 | 14954180 | +6.50% |
| camtool | 23245596 | 24260084 | +4.36% |
| devcam | 19250156 | 20153892 | +4.69% |
| hello | 12925420 | 13816564 | +6.89% |
| publisher | 19047228 | 19964308 | +4.81% |
josharian
on 4 Jul 2015
One more data point. A program containing just a single non-linker-strippable string of length 500000 grows from 1.12mb in Go 1.4 to 1.56mb at tip (+39%). I tried bisecting and found (unfortunately) that the growth was pretty slow and steady: I saw file sizes along the way including at least 1.15, 1.25, 1.30, 1.34, and 1.47, and I wasn't looking hard.
package main
var s = "....very long...."
func main() { _ = s[0] }
Marked this as 1.6. I don't expect it to be fixed but I want to see us develop an understanding of what's going on.
robpike
on 5 Jul 2015
CL https://golang.org/cl/17398 mentions this issue.
gopherbot
on 4 Dec 2015
CL https://golang.org/cl/16610 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19692 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19697 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19695 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19694 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19696 mentions this issue.
gopherbot
on 19 Feb 2016
CL https://golang.org/cl/19698 mentions this issue.
gopherbot
on 20 Feb 2016
CL https://golang.org/cl/19768 mentions this issue.
gopherbot
on 21 Feb 2016
CL https://golang.org/cl/19770 mentions this issue.
gopherbot
on 21 Feb 2016
CL https://golang.org/cl/19766 mentions this issue.
gopherbot
on 21 Feb 2016
CL https://golang.org/cl/19769 mentions this issue.
gopherbot
on 21 Feb 2016
CL https://golang.org/cl/19767 mentions this issue.
gopherbot
on 21 Feb 2016
CL https://golang.org/cl/19790 mentions this issue.
gopherbot
on 22 Feb 2016
CL https://golang.org/cl/19813 mentions this issue.
gopherbot
on 22 Feb 2016
CL https://golang.org/cl/19852 mentions this issue.
gopherbot
on 23 Feb 2016
CL https://golang.org/cl/19916 mentions this issue.
gopherbot
on 25 Feb 2016
CL https://golang.org/cl/19987 mentions this issue.
gopherbot
on 4 Mar 2016
CL https://golang.org/cl/20334 mentions this issue.
gopherbot
on 8 Mar 2016
CL https://golang.org/cl/20335 mentions this issue.
gopherbot
on 8 Mar 2016
CL https://golang.org/cl/20483 mentions this issue.
gopherbot
on 9 Mar 2016
CL https://golang.org/cl/20701 mentions this issue.
gopherbot
on 14 Mar 2016
CL https://golang.org/cl/20709 mentions this issue.
gopherbot
on 15 Mar 2016
Progress update.
David Crawshaw's and others work has been paying off...
$ for x in go1.4 go1.5 go1.6 go; do ls -l $x/pkg/tool/linux_amd64/objdump; done
-rwxr-xr-x 1 bradfitz bradfitz 4408008 Feb 18 2015 go1.4/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 4885536 Nov 20 19:33 go1.5/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 4988208 Feb 17 20:37 go1.6/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 3582128 Mar 16 20:01 go/pkg/tool/linux_amd64/objdump
$ for x in go1.4 go1.5 go1.6 go; do ls -l $x/pkg/tool/linux_amd64/cgo; done
-rwxr-xr-x 1 bradfitz bradfitz 4560376 Feb 18 2015 go1.4/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 5111632 Nov 20 19:32 go1.5/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 5275056 Feb 17 20:36 go1.6/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 4161520 Mar 16 20:00 go/pkg/tool/linux_amd64/cgo
$ for x in go1.4 go1.5 go1.6 go; do ls -l $x/bin/go; done
-rwxr-xr-x 1 bradfitz bradfitz 9571864 Feb 18 2015 go1.4/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 11195936 Nov 20 19:33 go1.5/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 12523312 Feb 17 20:37 go1.6/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 9972464 Mar 16 20:08 go/bin/go
$ cat fmt_hello.go
package main
import "fmt"
func main() {
fmt.Println("Hello, world.")
}
$ ls -l fmt_hello_*
-rwxr-xr-x 1 bradfitz bradfitz 1941352 Mar 17 20:08 fmt_hello_14
-rwxr-xr-x 1 bradfitz bradfitz 2367120 Mar 17 20:08 fmt_hello_15
-rwxr-xr-x 1 bradfitz bradfitz 2288392 Mar 17 20:08 fmt_hello_16
-rwxr-xr-x 1 bradfitz bradfitz 1620472 Mar 17 20:08 fmt_hello_tip
You really should test a single program. I believe all of those except maybe objdump have changed significantly during the interval. My test with a more guaranteed stable source is seeing about 6% reduction from 1.4, which is good but still far from where it should be. Go back and read the first message to see how much bloat has arrived.
But progress is happening, that's for sure.
robpike
on 18 Mar 2016
CL https://golang.org/cl/20825 mentions this issue.
gopherbot
on 18 Mar 2016
@robpike objdump is shrinking more than most programs because the new dead code elimination in the linker can detect statically that it does not call methods via reflect. This lets it remove far more methods. For details see the CL description of https://golang.org/cl/20483
If you have a particular program you'd like me to look at for my next binary size pass, please send it to me.
crawshaw
on 18 Mar 2016
@robpike, the bottom of my comment contains the 1-line hello world program. It doesn't get much simpler than that.
bradfitz
on 18 Mar 2016
@bradfitz indeed, although it's not clear whether it's a typical Go program. Perhaps it is.
robpike
on 18 Mar 2016
Here's cmd/godoc at HEAD (fcde7743)
$ for x in go1.4 go1.5 go1.6 go; do $x/bin/go build -a -o ~/godoc_$x golang.org/x/tools/cmd/godoc; ls -l ~/godoc_$x; done
-rwxr-x--- 1 cbro eng 16267896 Mar 18 16:11 /usr/local/google/home/cbro/godoc_go1.4
-rwxr-x--- 1 cbro eng 17132840 Mar 18 16:11 /usr/local/google/home/cbro/godoc_go1.5
-rwxr-x--- 1 cbro eng 18468712 Mar 18 16:12 /usr/local/google/home/cbro/godoc_go1.6
-rwxr-x--- 1 cbro eng 15425920 Mar 18 16:12 /usr/local/google/home/cbro/godoc_go
broady
on 19 Mar 2016
CL https://golang.org/cl/20968 mentions this issue.
gopherbot
on 21 Mar 2016
CL https://golang.org/cl/21033 mentions this issue.
gopherbot
on 23 Mar 2016
CL https://golang.org/cl/21087 mentions this issue.
gopherbot
on 25 Mar 2016
CL https://golang.org/cl/20902 mentions this issue.
gopherbot
on 25 Mar 2016
CL https://golang.org/cl/21285 mentions this issue.
gopherbot
on 29 Mar 2016
CL https://golang.org/cl/21284 mentions this issue.
gopherbot
on 29 Mar 2016
CL https://golang.org/cl/21395 mentions this issue.
gopherbot
on 1 Apr 2016
CL https://golang.org/cl/21396 mentions this issue.
gopherbot
on 1 Apr 2016
CL https://golang.org/cl/21583 mentions this issue.
gopherbot
on 6 Apr 2016
CL https://golang.org/cl/21777 mentions this issue.
gopherbot
on 11 Apr 2016
CL https://golang.org/cl/21776 mentions this issue.
gopherbot
on 11 Apr 2016
CL https://golang.org/cl/22371 mentions this issue.
gopherbot
on 22 Apr 2016
CL https://golang.org/cl/22373 mentions this issue.
gopherbot
on 22 Apr 2016
CL https://golang.org/cl/22395 mentions this issue.
gopherbot
on 24 Apr 2016
Great progress this release cycle. Kicking open issue to next cycle because there's still a bit more to do. But at least the "and growing" has been reversed for one cycle.
rsc
on 17 May 2016
@rsc It seems like we haven't done any work on this for 1.8. I don't think keeping an umbrella issue like this open is useful. Is there anything in particular you had in mind for this issue, or should we close it now?
quentinmit
on 5 Oct 2016
I have some more typeOff work I was hoping to do in 1.8 (the continuation of the work linked here done in 1.7), though I am busy in other projects right now. Let's at least leave this open until the window closes in a few weeks.
crawshaw
on 5 Oct 2016
We have very few umbrella issues, but this one is fine to keep. It's good to keep in mind and collect all the work toward this. The binaries are still much bigger than we hope they would be.
rsc
on 6 Oct 2016
CL https://golang.org/cl/43090 mentions this issue.
gopherbot
on 10 May 2017
CL https://golang.org/cl/43190 mentions this issue.
gopherbot
on 11 May 2017
CL https://golang.org/cl/44007 mentions this issue.
gopherbot
on 24 May 2017
No notable wins (maybe a slight loss) at tip (to-be Go 1.9) compared to Go 1.8:
bradfitz@gdev:~$ for x in go1.4 go1.5 go1.6 go1.7 go1.8 go; do ls -l $x/bin/go; done
-rwxr-xr-x 1 bradfitz bradfitz 9571864 Feb 18 2015 go1.4/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 11290832 Jan 13 2016 go1.5/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 12534640 Jul 18 2016 go1.6/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 9953979 Aug 15 2016 go1.7/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 10068917 Feb 16 19:28 go1.8/bin/go
-rwxr-xr-x 1 bradfitz bradfitz 10346229 Jun 6 00:03 go/bin/go
bradfitz@gdev:~$ for x in go1.4 go1.5 go1.6 go1.7 go1.8 go; do ls -l $x/pkg/tool/linux_amd64/cgo; done
-rwxr-xr-x 1 bradfitz bradfitz 4560376 Feb 18 2015 go1.4/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 5111280 Jan 13 2016 go1.5/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 5279368 Jul 18 2016 go1.6/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 4114079 Aug 15 2016 go1.7/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 3914818 Feb 16 19:28 go1.8/pkg/tool/linux_amd64/cgo
-rwxr-xr-x 1 bradfitz bradfitz 4015485 Jun 6 00:03 go/pkg/tool/linux_amd64/cgo
bradfitz@gdev:~$ for x in go1.4 go1.5 go1.6 go1.7 go1.8 go; do ls -l $x/pkg/tool/linux_amd64/objdump; done
-rwxr-xr-x 1 bradfitz bradfitz 4408008 Feb 18 2015 go1.4/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 4889240 Jan 13 2016 go1.5/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 4988648 Jul 18 2016 go1.6/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 3622669 Aug 15 2016 go1.7/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 3717826 Feb 16 19:28 go1.8/pkg/tool/linux_amd64/objdump
-rwxr-xr-x 1 bradfitz bradfitz 3836105 Jun 6 00:03 go/pkg/tool/linux_amd64/objdump
bradfitz@gdev:~$ for x in go1.4 go1.5 go1.6 go1.7 go1.8 go; do ls -l $x/bin/gofmt; done
-rwxr-xr-x 1 bradfitz bradfitz 3594744 May 11 2016 go1.4/bin/gofmt
-rwxr-xr-x 1 bradfitz bradfitz 3944064 Jan 13 2016 go1.5/bin/gofmt
-rwxr-xr-x 1 bradfitz bradfitz 3895320 Jul 18 2016 go1.6/bin/gofmt
-rwxr-xr-x 1 bradfitz bradfitz 3036195 Aug 15 2016 go1.7/bin/gofmt
-rwxr-xr-x 1 bradfitz bradfitz 3481554 Feb 16 19:28 go1.8/bin/gofmt
-rwxr-xr-x 1 bradfitz bradfitz 3257512 Jun 6 00:03 go/bin/gofmt
I'm going to move this ongoing tracking bug to Go 1.10, since I don't see anything more happening for Go 1.9.
bradfitz
on 8 Jun 2017
Change https://golang.org/cl/57130 mentions this issue: cmd/compile/internal/ssa: combine consecutive loads and stores on amd64
gopherbot
on 24 Aug 2017
This issue reports slight increases between 1.8 and 1.9: #21653
mvdan
on 28 Aug 2017
Change https://golang.org/cl/61190 mentions this issue: cmd/compile: specialize map creation for small hint sizes
gopherbot
on 9 Sep 2017
Change https://golang.org/cl/88135 mentions this issue: cmd/compile: don't combine 64-bit loads/stores on amd64
gopherbot
on 17 Jan 2018
23934 reports an ~8% increase between 1.9.4 and 1.10 for default binaries, and a slight decrease with -ldflags="-s -w". (#11799 is relevant to debug info size.)
FiloSottile
on 20 Feb 2018
I'm seeing a significant regression in binaries sizes between go1.10.1 and the current tip on linux/amd64. A simple hello world is 26% bigger than it was on 1.10:
$ cat test.go
package main
import "fmt"
func main() {
fmt.Println("hi!")
}
$ go version
go version go1.10.1 linux/amd64
$ go build test.go
$ ls -l test
-rwxr-xr-x 1 alberto alberto 2011612 Apr 27 11:14 test
$ gotip version
go version devel +a3bafcf8cc Thu Apr 26 18:26:06 2018 +0000 linux/amd64
$ gotip build test.go
$ ls -l test
-rwxr-xr-x 1 alberto alberto 2543972 Apr 27 11:15 test
It's not just the hello world. The go binary went from 11MB to 15MB. gofmt from 3.4MB to 4.3MB.
Is this expected?
ALTree
on 27 Apr 2018
@ALTree have you tried the same tests with -ldflags="-w -s"? There have been many changes to debugging info recently, so perhaps it's just that the binaries contain more debug info.
mvdan
on 27 Apr 2018
Generally speaking we do not recommend stripping debug info (it's useful, after all), so that's only useful for figuring out why they grew, not a justification for the growth.
dominikh
on 27 Apr 2018
@mvdan Yes, and stripped binaries are the same size as in 1.10, but that's not the default...
But yeah, if this is an expected effect of the recent work on debug info then... ok, I guess? I just wanted to make sure this was intentional.
ALTree
on 27 Apr 2018
I should have been explicit; I meant it only as a way to quickly figure out where the growth was coming from.
mvdan
on 27 Apr 2018
When I looked a few weeks ago, all of the increase (and some) was dwarf. I think the best fix is probably #11799.
josharian
on 27 Apr 2018
Change https://golang.org/cl/118276 mentions this issue: cmd/link: compress DWARF sections in ELF binaries
gopherbot
on 15 Jun 2018
Change https://golang.org/cl/127075 mentions this issue: html: lazily populate Unescape tables
gopherbot
on 31 Jul 2018
related issue, specifically for wasm: #29478
katiehockman
on 2 Jan 2019
Go 1.11 got bigger, and Go 1.12 got even bigger: https://github.com/golang/go/issues/27266
bradfitz
on 10 Jan 2019
Change https://golang.org/cl/161337 mentions this issue: cmd/compile: reorganize init functions
gopherbot
on 6 Feb 2019
Change https://golang.org/cl/171760 mentions this issue: runtime, cmd/compile: re-order PCDATA and FUNCDATA indices
gopherbot
on 11 Apr 2019
Change https://golang.org/cl/171820 mentions this issue: cmd/link: fix off-by-one in ftabaddstring
gopherbot
on 12 Apr 2019
Change https://golang.org/cl/172079 mentions this issue: cmd/link: deduplicate pctab info in pclntab
gopherbot
on 15 Apr 2019
Change https://golang.org/cl/171860 mentions this issue: cmd/link: hide stkobj symbols
gopherbot
on 15 Apr 2019
Change https://golang.org/cl/172198 mentions this issue: cmd/compile: make stkobj symbols content-addressable
gopherbot
on 16 Apr 2019
Thanks for your effort, @josharian!
mvdan
on 16 Apr 2019
Change https://golang.org/cl/191198 mentions this issue: cmd/compile,runtime: generate hash functions only for types which are map keys
gopherbot
on 21 Aug 2019
Change https://golang.org/cl/205139 mentions this issue: net/http: support disabling built-in HTTP/2 with a new build tag
gopherbot
on 4 Nov 2019
Change https://golang.org/cl/231397 mentions this issue: cmd/link: don't mark a symbol's Gotype reachable
gopherbot
on 1 May 2020
For those of you following this issue, we now have a binary-size label to collect all related issues. You can see the list here: https://github.com/golang/go/labels/binary-size
mvdan
on 1 May 2020
Some good news: https://twitter.com/bradfitz/status/1256348714198654976
dev:~ $ cat go-many.go
package main
import (
"archive/zip"
"encoding/json"
"go/format"
"net/http"
"text/template"
)
func main() {
_ = http.ListenAndServe
_ = http.ListenAndServeTLS
_ = http.Get
_ = json.NewDecoder
_ = json.NewEncoder
_ = zip.NewReader
_ = zip.NewWriter
_ = format.Source
_ = template.HTMLEscape
}
dev:~ $ go version
go version devel +cb00d93431 Fri May 1 20:39:53 2020 +0000 linux/amd64
dev:~ $ go1.14.2 build -o go-many.go1.14 go-many.go
dev:~ $ go build -o go-many.go1.15 go-many.go
dev:~ $ ls -l go-many*
-rw-r--r-- 1 bradfitz bradfitz 307 May 1 15:19 go-many.go
-rwxr-xr-x 1 bradfitz bradfitz 8510732 May 1 15:21 go-many.go1.14
-rwxr-xr-x 1 bradfitz bradfitz 4055570 May 1 15:21 go-many.go1.15
Change https://golang.org/cl/261677 mentions this issue: cmd/compile: fix and improve alias detection
gopherbot
on 12 Oct 2020
If package / module paths are still taking up a big enough part of the binary, we could consider writing them in a more efficient format that splits paths into their individual components, for example "github.com/golang/go" becomes ["github.com", "golang", "go"], then we put all components from all package paths into a list, such that each component is only stored once. Then we write the string as a list of indexes into the list of path components. If we used varint, a 1 or 2 byte index could refer to a much longer 10 byte string such as "github.com", which would only ever have to be written once.
Keithcat1
on 15 Oct 2020
If package / module paths are still taking up a big enough part of the binary, we could consider writing them in a more efficient format that splits paths into their individual components,
While this might work for parts of the runtime where constructing the name is under control of Go would or is this more efficient format supported as standard ELF symbols or DWARF and tools like 'nm' would be able to read them?
If this is only for runtime e.g. get caller names note that it was pointed out that getting the names should still be fast and therefore not involve allocations.
martisch
on 15 Oct 2020
@Keithcat1 see related discussion in https://github.com/golang/go/issues/36313. This would help, but it's far from trivial to implement.
josharian
on 15 Oct 2020
Change https://golang.org/cl/263017 mentions this issue: cmd/link: emit include directories in DWARF line table prolog
gopherbot
on 16 Oct 2020
Change https://golang.org/cl/264459 mentions this issue: cmd/link: avoid exporting all symbols on windows buildmode=pie
gopherbot
on 22 Oct 2020
Change https://golang.org/cl/262797 mentions this issue: cmd/cgo: avoid exporting all symbols on windows buildmode=c-shared
gopherbot
on 22 Oct 2020
Related issues
griesemer
·
808Comments
DemiMarie
·
320Comments
jessfraz
·
154Comments
derekmarcotte
·
243Comments
tklauser
·
219Comments
Most helpful comment
Some good news: https://twitter.com/bradfitz/status/1256348714198654976