Confluent-kafka-go: A good template for dockerfile (Multistage) that includes librdkafka and go

Created on 17 Jun 2019 · 2Comments · Source: confluentinc/confluent-kafka-go

Description

I'm not able to find a good resource or a docker image that includes both go dependencies and llibrdkafka. I'd be interested in using multi-builds to reduce the size of such and image. Currently, I've the following docker file, however, it's quite fat take quite some time to download and install librdkafka.

````
FROM golang:1.12.4 as builder
ENV LIBRDKAFKA_VERSION 1.0.0

RUN apt-get -y update
&& apt-get install -y --no-install-recommends upx-ucl zip libssl-dev
&& apt-get clean
&& rm -rf /var/lib/apt/lists/*

RUN curl -Lk -o /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz https://github.com/edenhill/librdkafka/archive/v${LIBRDKAFKA_VERSION}.tar.gz &&
tar -xzf /root/librdkafka-${LIBRDKAFKA_VERSION}.tar.gz -C /root &&
cd /root/librdkafka-${LIBRDKAFKA_VERSION} &&
./configure --prefix /usr && make && make install && make clean && ./configure --clean

WORKDIR /app/
COPY . .
ENV GO111MODULE=on
RUN go build -o app main.go
ENTRYPOINT ["/bin/sh", "-c", "./app"]

````

How to reproduce

Checklist

Please provide the following information:

[x] confluent-kafka-go and librdkafka version (LibraryVersion(1.0.0)):
[x] Apache Kafka broker version: 2.2.0
[ ] Client configuration: ConfigMap{...}
[ ] Operating system:
[ ] Provide client logs (with "debug": ".." as necessary)
[ ] Provide broker log excerpts
[ ] Critical issue

Source

Gillani0

👍1

Most helpful comment

Found the hidden docker example!!

# build stage
FROM golang as builder
ARG MODULE
# librdkafka Build from source
RUN git clone https://github.com/edenhill/librdkafka.git

WORKDIR librdkafka

RUN ./configure --prefix /usr

RUN make

RUN make install

# Build go binary

WORKDIR /app/
COPY ./src/${MODULE} ${MODULE}

ENV GO111MODULE=on
WORKDIR /app/${MODULE}
RUN ls
RUN go mod download


RUN go build -o main
RUN ls
# final stage
FROM ubuntu
ARG MODULE
COPY --from=builder /usr/lib/pkgconfig /usr/lib/pkgconfig
COPY --from=builder /usr/lib/librdkafka* /usr/lib/
COPY --from=builder /app/${MODULE}/* /${MODULE}/
WORKDIR /${MODULE}
CMD ["./main"]

Gillani0 on 24 Jun 2019

👍5 🚀4 🎉3

All 2 comments

Found the hidden docker example!!

# build stage
FROM golang as builder
ARG MODULE
# librdkafka Build from source
RUN git clone https://github.com/edenhill/librdkafka.git

WORKDIR librdkafka

RUN ./configure --prefix /usr

RUN make

RUN make install

# Build go binary

WORKDIR /app/
COPY ./src/${MODULE} ${MODULE}

ENV GO111MODULE=on
WORKDIR /app/${MODULE}
RUN ls
RUN go mod download


RUN go build -o main
RUN ls
# final stage
FROM ubuntu
ARG MODULE
COPY --from=builder /usr/lib/pkgconfig /usr/lib/pkgconfig
COPY --from=builder /usr/lib/librdkafka* /usr/lib/
COPY --from=builder /app/${MODULE}/* /${MODULE}/
WORKDIR /${MODULE}
CMD ["./main"]

Gillani0 on 24 Jun 2019

👍5 🚀4 🎉3

I know this has been a while but this worked for me

```FROM golang:1.12.9-alpine AS build-stage

LABEL app="application_name"

ENV PATH=$PATH:$GOROOT/bin:$GOPATH/bin

RUN apk add --update --no-cache alpine-sdk bash python ca-certificates
libressl
tar
git openssh openssl yajl-dev zlib-dev cyrus-sasl-dev openssl-dev coreutils

WORKDIR /src/application_name

RUN git clone https://github.com/edenhill/librdkafka.git
WORKDIR /src/application_name/librdkafka
RUN /src/application_name/librdkafka/configure --prefix /usr
RUN make
RUN make install

WORKDIR /src/application_name
COPY . .
RUN GOOS=linux go build -a -o image-name .```

Got it from here https://stackoverflow.com/questions/58128541/build-golang-application-with-librdkafka-in-a-debian-docker-image?rq=1