Sensu-go: Investigate binary size reduction

[annaplotkin]

This ticket is to investigate methods of reducing our binary size.

For reference:

https://blog.filippo.io/shrink-your-go-binaries-with-this-one-weird-trick/

Investigation details to be captured here and will inform a technical direction.

[maxadamo]

exactly!

go build -ldflags "-s -w"

and I didn't know the other trick:

upx --brute go

[junland]

This is actually a issue with upstream Golang in general. The devs are working on reducing binary size. See 6853 . Can't find it but I believe one of the main devs said that they are prioritizing binary size reduction for Go 1.12. (Don't quote me on that.)

For right now you can use the -s -w flags + using upx to reduce size. Or if your really adventurous you can compile it using gccgo which is just a alternative compiler using gcc.

[maxadamo]

But I'm also wondering if it is possible to release smaller rpm/Deb package.
I believe nobody needs the debug symbols.

[maxadamo]

@junland to be clear (correct me if you feel that this is wrong) even C binaries are stripped after the compilation.
Here we go:

file /bin/ls
/bin/ls: ELF 64-bit LSB shared object, x86-64, version 1 (SYSV), dynamically linked, interpreter /lib64/ld-linux-x86-64.so.2, for GNU/Linux 3.2.0, BuildID[sha1]=bf40cb84e7815de09fa792a097061886933e56fa, stripped

[junland]

@maxadamo Right, most C software is striped but C is fairly small anyways even with the debugging info included. However stripping debugging info makes it harder to debug if something happens which for a project like this that is still in it's early stages you would probably need to have the debug info included if something happens in production.

[maxadamo]

ok but, moment.
Majority of C programs (most of them) are stripped even if they're already small enough, but Go programs which are considerably big must be released with all the symbols?

[junland]

@maxadamo You bring up a good point. Maybe this would be a question for something on /r/golang on Reddit or the Golang IRC channel. I just assumed having symbols included for Go programs was common practice. If you could get a official statement from the Go dev's than that could help this issue for removing debug symbols for Sensu.

[maxadamo]

@junland just out of curiosity I ran some test with stripped binaries (either using strip or -ldflags "-s -w") and the results are good so far (of course: every C, C++ binary is stripped already).
Then I tested out a particular binary compressed with UPX and the result was quite bad.
I used a big Go binary, which is nomad, whose size is around 50 MB.
Running nomad -help compressed with UPX takes around 1 second!

[junland]

@maxadamo This should explain maybe why your seeing this.

I did my own test with nomad and here are my results:

time nomand --help (No stripping or compressed)
real    0m0.189s
user    0m0.000s
sys     0m0.109s

time nomad --help (Compressed with default settings using upx)
real    0m0.428s
user    0m0.156s
sys     0m0.141s

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[nikkictl]

I think this is still valid. Sounds like a spike though!

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

[maxadamo]

up, up :smile_cat: