Go: runtime/debug: document BuildInfo.Main.Version == "(devel)"

[mark-rushakoff]

What version of Go are you using (go version)?

$ go version
go version devel +571d93e977 Thu Dec 13 15:08:48 2018 +0000 darwin/amd64

What operating system and processor architecture are you using (go env)?

go env Output

$ go env

GOARCH="amd64"

GOBIN=""

GOCACHE="/Users/mr/Library/Caches/go-build"

GOEXE=""

GOFLAGS=""

GOHOSTARCH="amd64"

GOHOSTOS="darwin"

GOOS="darwin"

GOPATH="/Users/mr/go"

GOPROXY=""

GORACE=""

GOROOT="/Users/mr/gotip/src/github.com/golang/go"

GOTMPDIR=""

GOTOOLDIR="/Users/mr/gotip/src/github.com/golang/go/pkg/tool/darwin_amd64"

GCCGO="gccgo"

CC="clang"

CXX="clang++"

CGO_ENABLED="1"

GOMOD="/Users/mr/gomod/debug-module-version-demo/go.mod"

CGO_CFLAGS="-g -O2"

CGO_CPPFLAGS=""

CGO_CXXFLAGS="-g -O2"

CGO_FFLAGS="-g -O2"

CGO_LDFLAGS="-g -O2"

PKG_CONFIG="pkg-config"

GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/ct/bl4_z3g51ks8239_r2k07v_40000gn/T/go-build638578617=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

Repro case is https://github.com/mark-rushakoff/debug-module-version-demo. It's a module, and its main.go contents are a simple use of debug.ReadBuildInfo:

package main

import (
    "fmt"
    "runtime/debug"

    _ "rsc.io/quote"
)

func main() {
    bi, ok := debug.ReadBuildInfo()
    if !ok {
        panic("couldn't read build info")
    }

    fmt.Printf("%s version %s\n", bi.Path, bi.Main.Version)

    for _, d := range bi.Deps {
        fmt.Printf("\tbuilt with %s version %s\n", d.Path, d.Version)
    }
}

What did you expect to see?

I expected to see the first line print github.com/mark-rushakoff/debug-module-version-demo version v0.0.0-20181213... when checked out at an arbitrary commit, or github.com/mark-rushakoff/debug-module-version-demo version v0.0.1 when checked out at tag v0.0.1. I tried both go run . and go build . && ./debug-module-version-demo but both cases printed (devel).

What did you see instead?

github.com/mark-rushakoff/debug-module-version-demo version (devel)
    built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
    built with rsc.io/quote version v1.5.2
    built with rsc.io/sampler version v1.3.0

Based on the behavior I've observed, it looks as though the main module returned by debug.ReadBuildInfo is hardcoded to (devel) for the main module, which I assume is intended behavior. If so, that's unfortunate for use cases like mycmd version to easily print the module version of the software being built; but it should be documented.

The current documentation at https://tip.golang.org/pkg/runtime/debug/#ReadBuildInfo does not mention (devel) in any way, nor does it mention any special behavior of the Main module.

/cc @hyangah since you're on git blame for src/runtime/debug/mod.go.

[bcmills]

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag, although that might be misleading if some other module has a cyclic requirement specifying a higher version than the tag.

[stapelberg]

+1.

I thought being able to print the program’s main module’s version was the whole point of embedding build information into the binary.

[mark-rushakoff]

https://github.com/golang/go/issues/26404#issuecomment-451136809 is some separate uncertainty about when and why (devel) shows up. There are some linked issues there that seem to paint a fuller picture, but I don't have time to go through them right now.

I think this issue is still fully separate from #26404, as this is just about documenting (devel).

[mark-rushakoff]

It looks like one way to get the version to show up, is to make a new module referring to the one you actually want to build; and then build the target module.

bash-3.2$ gotip mod init ignore
go: creating new go.mod: module ignore

bash-3.2$ gotip get github.com/mark-rushakoff/debug-module-version-demo
go: finding github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: downloading github.com/mark-rushakoff/debug-module-version-demo v0.0.1
go: extracting github.com/mark-rushakoff/debug-module-version-demo v0.0.1

bash-3.2$ gotip build !$
gotip build github.com/mark-rushakoff/debug-module-version-demo

bash-3.2$ ./debug-module-version-demo
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
    built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
    built with rsc.io/quote version v1.5.2
    built with rsc.io/sampler version v1.3.0

[mholt]

@bcmills

We could perhaps detect the case where the main module happens to be a pristine checkout with a semantically-versioned tag,

Or even just any of the current VCS state. For Caddy's builds, we use tag if clean, or if not clean: nearest tag, commit SHA, and build date (although I can understand why build date is a bad idea if seeking reproducible builds).

For now, @mark-rushakoff's workaround is quite handy.

+1 to this though, some version info on the main module would be immensely helpful.

[hyangah]

As observed in #33926, currently it's possible to end up building a binary with the same main module version but with different dependency versions. People who intend to use the main module version as the binary version should be careful.

[bcmills]

People who intend to use the main module version as the binary version should be careful.

Using the “main module” version as the binary version (in the sense of https://tip.golang.org/cmd/go/#hdr-The_main_module_and_the_build_list) should be fine as long as the module doesn't have filesystem-local replace directives.

Unfortunately, that is not the version that the Main field indicates today (see #33975).

So I agree that the BuildInfo.Main field itself is not really reliable as an indication of the overall configuration in use, even when that version is not (devel).

[mholt]

Will this be a high priority for Go 1.14? It could really ease the burden of build-from-souce workflows. Like, _a lot_.

[bcmills]

@mholt, this is unlikely to happen in Go 1.14.

(We have a pretty full slate of issues already, and any time the git command is involved there are thorny security considerations that must be addressed with care.)

[mback2k]

Is using the git command really a requirement of this feature? Wouldn't it be possible to get the relevant metadata directly from .git/HEAD?

[bcmills]

@mback2k, .git/HEAD tells us the commit id of the HEAD revision. It does not tell us:

1. Whether the working directory is even in a Git repo, or perhaps in some other VCS nested within one.
2. Where to find the .git directory.
3. Whether the working tree contains ignored files or other differences from the commit indicated by .git/HEAD.

[mark-rushakoff]

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel). Whether it can be correctly set to a particular git sha seems to be a different topic in my opinion -- perhaps that should be another issue?

[mholt]

@mark-rushakoff

The original point of this issue was simply documenting what it means when BuildInfo.Main.Version returns the string (devel)

BuildInfo.Main.Version _always_ returns "(devel)".

I oppose documenting this, because I believe this is a bug: BuildInfo.Main.Version _always_ returns "(devel)" even when the main module is versioned at HEAD.

Documenting that it always returns "(devel)" makes it appear as correct behavior and makes it hard or impossible to change later.

Let's fix the actual problem, not document the bug as expected behavior.

[bcmills]

@mholt, we can document the current behavior in a way that does not promise that behavior for all time.

For example, we can document that Buildinfo.Main.Version is (devel) _when the go command does not know the version of the main module_, without promising that it will never be changed to discover that version.

[hyangah]

nit:

Buildinfo.Main.Version is (devel) when the go command does not know the version of the main module, without promising that it will never be changed to discover that version.

The intention was that the BuildInfo.Main is the module that contains the main package, not the main module described in https://golang.org/cmd/go/#hdr-The_main_module_and_the_build_list
issue 33975.

I hope correcting the documentation error will help explaining this BuildInfo.Main.Version == "(devel)" case.

[mholt]

Yeah, I think what we want is the version for the "main module" - the module that contains the main package.

[tv42]

While the main module still shows up as "(devel)" for just go build, the full workaround from https://github.com/golang/go/issues/29228#issuecomment-457390055 seems to be no longer necessary. Instead, you can just go get things outside of a current module:

$ GOBIN=$PWD GO111MODULE=on go get github.com/mark-rushakoff/debug-module-version-demo
$ ./debug-module-version-demo 
github.com/mark-rushakoff/debug-module-version-demo version v0.0.1
        built with golang.org/x/text version v0.0.0-20170915032832-14c0d48ead0c
        built with rsc.io/quote version v1.5.2
        built with rsc.io/sampler version v1.3.0

[ldemailly]

How would one build a binary (go service in docker image for instance) from ci/docker from a tag to get said tag in the binary being built?
(the source tree is already there (from ci source checkout step) and being copied (dockerfile COPY . .), not downloaded)

[zikaeroh]

As far as I know, there's no good way to do this (and it appears this issue is somewhat codifying that we'll never know). I'd love to be able to access my current version as it would appear had I created some other module and used @<commit ID> with a tidy, such that I could use that version in other places.

For my docker builds, I pass something in via --build-arg=version=..., then include it by doing things like -ldflags="-X path.to/version.verson=${version}".

[ldemailly]

I use the -ldflags="-X path.to/version.verson=${version}" too but I wanted the 2 versions to be in sync
for now I have a super ugly workaround which is to go get ...@tag but it's quite annoying specially on private repos to redownload something that is already there (it forces some very unsightly experimental docker support for ssh in docker etc)

putting it here in case it helps someone else:

DOCKER_BUILDKIT=1 docker build --ssh default ...

and dockerfile with

# syntax=docker/dockerfile:experimental
FROM golang as build
ARG VERSION
...
RUN git config --global url.ssh://git@somePrivateRepo/.insteadOf https://somePrivateRepo/
RUN mkdir -m 700 /root/.ssh && ssh-keyscan -4 -t rsa somePrivateRepo > /root/.ssh/known_hosts
RUN --mount=type=ssh go get -v -a -ldflags -s ... somePrivateRepo/pkg/path@${VERSION}

it would be so much easier for go build to have the option to check with git and/or some sort of --trust-me-the-version-is xxx flag