Go: cmd/link: add a flag to the linker to do not write function names to runtime.pclntab

[xaionaro]

What version of Go are you using (go version)?

go version
go version devel +71154e061f Tue Jan 14 17:13:34 2020 +0000 linux/amd64

Does this issue reproduce with the latest release?

Yes.

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

GO111MODULE=""

GOARCH="amd64"

GOBIN=""

GOCACHE="/home/experiment0/.cache/go-build"

GOENV="/home/experiment0/.config/go/env"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="linux"

GOINSECURE=""

GONOPROXY=""

GONOSUMDB=""

GOOS="linux"

GOPATH="/home/experiment0/go"

GOPRIVATE=""

GOPROXY="https://proxy.golang.org,direct"

GOROOT="/home/experiment0/.gimme/versions/go"

GOSUMDB="sum.golang.org"

GOTMPDIR=""

GOTOOLDIR="/home/experiment0/.gimme/versions/go/pkg/tool/linux_amd64"

GCCGO="/usr/bin/gccgo"

AR="ar"

CC="gcc"

CXX="g++"

CGO_ENABLED="1"

GOMOD=""

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build891839909=/tmp/go-build -gno-record-gcc-switches"

What did you do?

go get github.com/u-root/u-root
u-root -o /dev/null -build bb core boot
go get github.com/u-root/u-root/bb
go tool nm -size -sort size "$(go env GOPATH)"/bin/bb | head -1

What did you expect to see?

  e02ba0    3836272 r runtime.pclntab

What did you see instead?

  e39500    4808314 r runtime.pclntab

Proposal

I found here https://groups.google.com/d/msg/golang-nuts/hEdGYnqokZc/zQojaoWlAgAJ that a binary size could be reduced by avoiding of wasting space on function names within runtime.pclntab. So I propose to add a linker flag (for example -stripfnnames) to do that. We need such feature for applying Golang into embedded environments (with very limited total space).

I would like to prepare a PR if it will be approved.

[networkimprov]

See #36313

[xaionaro]

@networkimprov : thank you. However I saw that ticket. It contains discussion how to reduce runtime.pclntab for a generic case. While my proposal is about a special case (embedded systems), so I created a new ticket :)

---

UPD: Wrote a comment to #36313 with a proposal to add an option to select a simple GC which does not require so complex runtime.pclntab. If the proposal will be approved then this ticket is indeed not actual.

[ianlancetaylor]

Removing function names from runtime.pclntab would break programs that use runtime.Callers, which is a lot of programs. For example, the testing and log packages would partially break.

[xaionaro]

Removing function names from runtime.pclntab would break programs that use runtime.Callers, which is a lot of programs. For example, the testing and log packages would partially break.

@ianlancetaylor :

1. It's only for embedded (where it's extremely important to reduce the size, and people are ready to sacrifice with log/testing for that). Moreover we can add a build-tag for -stripfnnames and use it to remove functions which uses runtime.Callers or to change their behavior. In my opinion it would good enough to just change the behavior to print <unknown> instead of function names.
2. If I understand correctly testing and log will actually continue to work even without modifications (but I did not look into to this part of the code, yet). They will just print emptiness instead of function names.

[ianlancetaylor]

I believe the testing.(*T).Helper function, for example, will break.

This will also likely break profiling information.

I understand that you would find it useful but I'm reluctant to endorse a change that will break code in ways that people will not know how to expect. I would rather find other ways to reduce binary size.

[xaionaro]

that will break code in ways that people will not know how to expect.

We can forbid using broken functions via the build-tag (added automatically by -stripfnnames).

[networkimprov]

Ian may be concerned that you'll eventually realize that Go isn't the best choice for an embedded app, and switch to Rust or http://ziglang.org ;-)

[beoran]

For embedded Go, where size is really important, for the time being, https://tinygo.org/ is the better solution.

Rather than having a specific build flag for stripping runtime.pclntab, I think that in the long run, what we need is new GOOS, such as "none" for running on raw iron, and "linux-embedded" for running an on embedded linux OS should be added. The "none" variants then allow building an OS kernel, and the -embedded variants then do a few things differently to save on space.

[xaionaro]

For embedded Go, where size is really important, for the time being, https://tinygo.org/ is the better solution.

I tried tinygo few times for u-root. And tinygo is a nice and interesting project, but it appears it has invalid CGo parser (which has problems with #define and other stuff), (if I remember correctly) does not support Golang's assembly language, it does not implement too much stuff from standard packages (which it overrides) and it even does not support something related to func() variables so it panics while compilation of u-root/bb. I've spent some time and realized that it's would require too much time to port the project to tinygo. Also tinygo's README.md says:

Non-goals: ...; Be able to compile every Go program out there.

---

Ian may be concerned that you'll eventually realize that Go isn't the best choice for an embedded app, and switch to Rust or http://ziglang.org ;-)

I believe the community of Golang is wise and will be able to find a solution for cases like mine :)
And I will be glad to somehow help to implement an approved solution when there will be one.

Anyway it would take much more time to port project to Rust than even to port it to tinygo :)

[embeddedgo]

@xaionaro I think we can try to introduce such -stripfnnames flag to the embeddedgo fork. Can you help?

I'd love to test such flag because I've just reached the limit of 1 MiB Flash in STM32L476RG:

$ size shell.elf
text data bss dec hex filename
1043876 3736 14004 1061616 1032f0 shell.elf

where pclntab is 345 KiB (34%):

$ nm --size-sort -S shell.elf|tail -5
08050104 00000fd4 T time.Time.AppendFormat
08059f98 00001444 T fmt.(*pp).printValue
200024e8 00001a74 B runtime.mheap_
08040ce0 00003728 T unicode.init
080a89c0 000565ac r runtime.pclntab

The embeddedgo fork adds support for noos/thumb, noos/riscv64 and linux/thumb.

I use linux/thumb only to run tests but it is usable and you can give it a try (no cgo support, sorry). It generates compressed Thumb2 instructions instead of ARM instructions so it produces smaller binaries that can be run on almost any ARMv7-A machine.

The noos ones are for bare metal programming.

[xaionaro]

I think we can try to introduce such -stripfnnames flag to the embeddedgo fork.

Sounds very interesting. It could become very useful for https://github.com/u-root/u-root.

But I do not understand: why do you do a fork instead of working with the upstream?

Can you help?

I would love to try. How can I help? :)

[embeddedgo]

why do you do a fork instead of working with the upstream?

You can much easily introduce new things in a fork. Even Go teem itself creates forks branches for for a bigger changes.

There is clearly no place for rarely used architectures or operating systems in the upstream but linux/thumb can be probably upstreamed as the 32-bit ARM seems to be now mainly used in embedded niche.

I would love to try. How can I help? :)

Simply send a pull request that implements -stripfnnames flag to the embeddedgo fork.

The embeddedgo compiler is released as patches to the original Go source so you can relatively easy see what was added/changed.

[embeddedgo]

@xaionaro I've just implemented stripfnn flag:

-stripfn int
    strip function names in pclntab, 1: remove package path, 2: blank names

Test results:

stripfn       text    data     bss
       0   1044754    3744   14004
       1   1026745    3744   14004
       2    973563    3744   14004

Sample panic outputs:

stripfn=1

panic: stripfn test

goroutine 1 [running]:
ramfs.(*dir).Close(0x2000b440, 0x20008f33, 0x0)
        dir.go:76 +0xaa
os.(*File).Close(0x2000a7c8, 0xffffffff, 0x20008070)
        file_noos.go:171 +0x1e
main.ls(0x20008050, 0x2, 0x2)
        ls.go:28 +0x70
main.runCmd(0x20008050, 0x2, 0x2)
        main.go:62 +0x80
main.main()
        main.go:25 +0x9e

stripfn=2

panic: stripfn test

goroutine 1 [running]:
(0x806d008, 0x8096f58)
        :1064 +0x3c6
(0x2000b600, 0x20008f33, 0x0)
        :76 +0xaa
(0x2000a7d8, 0xffffffff, 0x200080a0)
        :171 +0x1e
(0x20008080, 0x2, 0x2)
        :28 +0x70
(0x20008080, 0x2, 0x2)
        :62 +0x80
()
        :25 +0x9e
()
        :206 +0x11c
()
        :525 +0x2

In case of stripfn=2 panics are almost unusable. But in case of stripfn=1 there is enough information to find the problem.

Clone https://github.com/embeddedgo/go and try it.