Confluent-kafka-go: go build -tags static

[nicbaz]

Hi,

I'm having trouble building a binary with librdkafka as a static library.

$ go build -tags static -ldflags -v bin/blu.go
a# command-line-arguments
HEADER = -H4 -T0x401000 -D0x0 -R0x1000
searching for runtime.a in $WORK/runtime.a
searching for runtime.a in /go/pkg/linux_amd64/runtime.a
searching for runtime.a in /usr/local/go/pkg/linux_amd64/runtime.a
 0.00 deadcode
 0.03 pclntab=1408982 bytes, funcdata total 187752 bytes
 0.04 dodata
 0.05 dwarf
 0.08 symsize = 0
 0.12 reloc
 0.13 asmb
 0.13 codeblk
 0.14 datblk
 0.15 sym
 0.15 symsize = 256320
 0.16 symsize = 258288
 0.16 dwarf
 0.18 headr
 0.19 host link: "gcc" "-m64" "-gdwarf-2" "-o" "/tmp/go-build416718997/command-line-arguments/_obj/exe/a.out" "-rdynamic" "/tmp/go-link-567465902/go.o" "/tmp/go-link-567465902/000000.o" "/tmp/go-link-567465902/000001.o" "/tmp/go-link-567465902/000002.o" "-g" "-O2" "-g" "-O2" "-Wl,-Bstatic" "-lrdkafka" "-Wl,-Bdynamic" "-lrdkafka" "-lpthread" "-lz" "-lcrypto" "-lssl" "-lsasl2" "-lrt" "-g" "-O2" "-lpthread"
 0.29 cpu time
94449 symbols
84360 liveness data
$ ldd blu
    linux-vdso.so.1 (0x00007ffda22d0000)
    librdkafka.so.1 => /usr/lib/x86_64-linux-gnu/librdkafka.so.1 (0x00007f3b0fc16000)
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f3b0f9f9000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007f3b0f7dd000)
    libcrypto.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libcrypto.so.1.0.0 (0x00007f3b0f3e1000)
    libssl.so.1.0.0 => /usr/lib/x86_64-linux-gnu/libssl.so.1.0.0 (0x00007f3b0f180000)
    libsasl2.so.2 => /usr/lib/x86_64-linux-gnu/libsasl2.so.2 (0x00007f3b0ef63000)
    librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007f3b0ed5b000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f3b0e9b0000)
    /lib64/ld-linux-x86-64.so.2 (0x0000562bb6602000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f3b0e7ab000)
    libresolv.so.2 => /lib/x86_64-linux-gnu/libresolv.so.2 (0x00007f3b0e594000)

As you can see librdkafka appears twice when linking.

I tried playing with pkg-config but could not get the correct LDFLAGS without -lrdfafka.
The only fix I could think of is manually setting LDFLAGS in kafka/build_static.go:

// +build static
// +build !static_all

package kafka

// #cgo LDFLAGS: -Wl,-Bstatic -lrdkafka -Wl,-Bdynamic -lpthread -lz -lcrypto -lssl -lsasl2 -lrt
import "C"

As I've never built with cgo before I might be doing something wrong, is this expected ?

[edenhill]

Will try to reproduce

[nicbaz]

To help you reproduce, I'm builduing in a docker container : https://github.com/docker-library/golang/blob/master/1.8/Dockerfile, and the librdkafka-dev version I'm using is the following:

Version: 0.9.4~1confluent3.2.0-1
Depends: librdkafka1 (= 0.9.4~1confluent3.2.0-1), librdkafka++1 (= 0.9.4~1confluent3.2.0-1)

[edenhill]

@nicbaz Thanks! 💯

[nicbaz]

@edenhill I know you're busy with librdkafka itself but is there anything I can do to help you make progress on this issue?

[edenhill]

Sorry for the long delay @nicbaz, I've been very busy with librdkafka and now final release work so hadn't had time to look at this. I'll take a look early next week.

My guess, after reading a bit more about pkg-config --static, is that we should remove the explicit LDFLAGS from build_static*.go

[nicbaz]

No worries, thanks for the update !

[jdgiotta]

@edenhill I was having a similar issue with building a "from scratch" docker image. I'm trying to understand the requirements here in order to static build a project using this library.

What I'm running in my build stage of my Dockerfile
RUN CGO_ENABLED=0 GOOS=linux go build -tags static_all -o trackservice .

Here is the error I receive:

config/events.go:9:2: build constraints exclude all Go files in /go/src/github.com/jdgiotta/myproject/package/vendor/github.com/confluentinc/confluent-kafka-go/kafka

EDIT

My build stage is using the golang:latest docker image. Which include the pkg-config.

[edenhill]

confluent-kafka-go uses cgo to interact with the underlying kafka client implementation in librdkafka (which is a C library), so you must not disable CGO (CGO_ENABLED=0).

[jdgiotta]

Ok, I guess what I'm not sure about is what dependencies I need in the building stage of the program.
This is what I getting without the CGO_ENABLED disabled

# github.com/jdgiotta/myproject/package/vendor/github.com/confluentinc/confluent-kafka-go/kafka
/usr/lib/librdkafka.a(rddl.o): In function `rd_dl_open0':
/librdkafka/src/rddl.c:80: warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/librdkafka.a(rdaddr.o): In function `rd_getaddrinfo':
/librdkafka/src/rdaddr.c:168: warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
# github.com/jdgiotta/myproject/package/vendor/github.com/confluentinc/confluent-kafka-go/kafka
cannot load imported symbols from ELF file $WORK/b137/_cgo_.o: no symbol section

Here is my complete Dockerfile build stage

FROM golang:latest as builder
RUN mkdir -p /go/src/github.com/jdgiotta/myproject
WORKDIR /
RUN git clone https://github.com/edenhill/librdkafka.git && \
    cd librdkafka && \
    ./configure --prefix /usr && \
    make && \
    make install
WORKDIR /go/src/github.com/jdgiotta/myproject
COPY . .
RUN GOOS=linux go build -a -tags static_all -o app .

[edenhill]

static_all is highly experimental, it is not likely to work with hand-holding.

If you're making a docker image there isn't much need for a completely static client anyway, right? You can simple include the librdkafka dependencies (libssl, libsasl2, etc).

[jdgiotta]

Sure, and I've done so already. I'm just not happy with the effort. I read the README and it says it supports static builds. That doesn't seem to be the case or at least not very well documented.

I'd be happy to document and create an example implementation. I'm not going to be only developer using kafka that will be looking for a "from scratch" approach to their go microservice consumer or producer.

Thank you for your time regardless.

[eliaslevy]

@jdgiotta This works for me:

FROM golang:1.10.1-alpine3.7

RUN apk add --no-cache  --repository http://dl-3.alpinelinux.org/alpine/edge/community/ \
      bash              \
      gcc               \
      git               \
      librdkafka-dev    \
      libressl-dev      \
      musl-dev          \
      zlib-dev          \
      wget          &&  \
      #
      # Install dep
      wget -nv -O /usr/local/bin/dep https://github.com/golang/dep/releases/download/v0.4.1/dep-linux-amd64 && \
      chmod a+rx /usr/local/bin/dep

docker build -f Dockerfile -t elevy/golang:1.10.1-alpine3.7-librdkafka .
docker run -it -v `pwd`:/go/src/app elevy/golang:1.10.1-alpine3.7-librdkafka bash -c 'cd /go/src/app; rm app; dep ensure; go test -coverprofile cover.out -v; go tool cover -html=cover.out -o coverage.html; go build -tags static_all .'

[jdgiotta]

@eliaslevy awesome! That worked like a charm.

[JoshKCarroll]

Note for posterity that the above method has been broken for a bit since alpine edge moved to 3.8 and 3.7 had old librdkafka version 0.11.1. At the current moment there is no golang:1.10-alpine3.8 but if you start with alpine:3.8 and install go / set $GOPATH you can get it working.

I came here with identical micro-service requirement as folks above, getting this working got my service docker image from ~400MB to ~12MB which has a big impact on boot time :)

This raises the question, any interest here to host an alpine image somewhere with golang and librdkafka installed to base static builds on? If not, I'll probably make one on our hub for quicker builds once the golang alpine 3.8 image is out and post back here if that's helpful.

[eliaslevy]

If Alpine doesn't have the version of librdkafka you'd like, it is easy to build it within the container. You can find the build file here. Also, as the output is a static binary, you can always build in 3.7 and deploy in 3.8, or vice versa. But, yes, it would be nice to have a prepackaged image in Docker Hub.

[aperrot42]

Hi, has anyone managed to produce a neat docker build image based on golang image ?