Buildkit: how to share host system's composer cache docker build through dockerfile?

[vesper8]

I've been searching and trying for over an hour and I'm about to give us so thought I would ask here.

I'm willing to use the latest, experimental build, and enable buildkit via DOCKER_BUILDKIT=1

I want to do something that should be simple yet seems impossible

this is my dockerfile

FROM composer:1.8 as vendor

COPY database/ database/

COPY composer.json composer.json
COPY composer.lock composer.lock

RUN composer install \
    --ignore-platform-reqs \
    --no-interaction \
    --no-plugins \
    --no-scripts \
    --prefer-dist

This is a rather large composer.json so everytime I build it redownloads every package and takes close to a minute just for this part. When I run the same command on my host everything is already cached so it takes around 5 seconds.

I just want to share my host's ~/.composer/cache folder with the /tmp/cache folder on the image so that my builds run a lot faster and Composer can use a cache

I've tried using VOLUME but then found out this is not the intended use of VOLUME. I tried using ADD/COPY but found out that you can't ADD/COPY from files that are outside the relative path of the folder housing the Dockerfile

Finally I found a thread on SO that claims that BuildKit is the solution, using the --mount switch when using the RUN command. But I still can't figure out how to do it, it feels like this only allows to share a cache between build stages but doesn't help with sharing a cache from the host system

If I can't share the cache from the host system, I'd be happy if at least the cache persisted between builds (I don't mean between build stages inside the Dockerfile but I mean, if I run docker build multiple times, that it won't re-download half the internet every single time)

Hoping someone can clear this up for me.. many thanks

[FernandoMiguel]

The problem is that the upstream image has a VOLUME where the downloaded
data resides, losing it on next build.

You can create a multi layer docker image and override where the downloads
go, so next time you build but the composer-lock.json doesn't change, you
won't need to build that layer.
You then copy the vendor folder to the next layer.

We use this extensively, so if you have questions, feel free to ping me on
Docker slack for a sample

--
Fernando Miguel

On Wed, 15 May 2019, 19:12 C-Bass, [e-mail] wrote:

I've been searching and trying for over an hour and I'm about to give us
so thought I would ask here.

I'm willing to use the latest, experimental build, and enable buildkit via
DOCKER_BUILDKIT=1

I want to do something that should be simple yet seems impossible

this is my dockerfile

FROM composer:1.8 as vendor

COPY database/ database/

COPY composer.json composer.json
COPY composer.lock composer.lock

RUN composer install \
--ignore-platform-reqs \
--no-interaction \
--no-plugins \
--no-scripts \
--prefer-dist

This is a rather large composer.json so everytime I build it redownloads
every package and takes close to a minute just for this part. When I run
the same command on my host everything is already cached so it takes around
5 seconds.

I just want to share my host's ~/.composer/cache folder with the
/tmp/cache folder on the image so that my builds run a lot faster and
Composer can use a cache

I've tried using VOLUME but then found out this is not the intended use of
VOLUME. I tried using ADD/COPY but found out that you can't ADD/COPY from
files that are outside the relative path of the folder housing the
Dockerfile

Finally I found a thread on SO that claims that BuildKit is the solution,
using the --mount switch when using the RUN command. But I still can't
figure out how to do it, it feels like this only allows to share a cache
between build stages but doesn't help with sharing a cache from the host
system

If I can't share the cache from the host system, I'd be happy if at least
the cache persisted between builds (I don't mean between build stages
inside the Dockerfile but I mean, if I run docker build multiple times,
that it won't re-download half the internet every single time)

Hoping someone can clear this up for me.. many thanks

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/moby/buildkit/issues/1009?email_source=notifications&email_token=AABJDLUHAEWHGEINJTUQLYDPVRG73A5CNFSM4HNFXRG2YY3PNVWWK3TUL52HS4DFUVEXG43VMWVGG33NNVSW45C7NFSM4GT7V4RQ,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJDLTZVDW74UYYU3LLACDPVRG73ANCNFSM4HNFXRGQ
.

[vesper8]

thanks for the answer @FernandoMiguel, that's a start I guess.. but what if the composer.lock has changed slightly but 99% of the dependencies are still the same.. is there no way to have a cache persist?

I think I understand what you're offering as a solution.. that if nothing changes at all in that build stage then it will be skipped on future builds.. but if that's possible then it means that the vendor folder is somehow being persisted between builds? And if the vendor folder can be persisted, then why not the Composer cache folder too?

By the way if you could share a small example of how to skip this build stage on future builds when nothing has changed, that would be much appreciated

On a side note.. is there really no way to mount/inject the cache folder from the host with the new features offered by BuildKit?

[FernandoMiguel]

With Docker layers you can't really modify its content.

Maybe buildkit can indeed help a bit, but I have not tried.

But the all purpose of immutable infrastructure is to be able to build it
often.

A thing to consider is to have a local cache server, proxying your package
cache.

--
Fernando Miguel

On Wed, 15 May 2019, 20:21 C-Bass, [e-mail] wrote:

thanks for the answer @FernandoMiguel https://github.com/FernandoMiguel,
that's a start I guess.. but what if the composer.lock has changed slightly
but 99% of the dependencies are still the same.. is there no way to have a
cache persist?

I think I understand what you're offering as a solution.. that if nothing
changes at all in that build stage then it will be skipped on future
builds.. but if that's possible then it means that the vendor folder is
somehow being persisted between builds? And if the vendor folder can be
persisted, then why not the Composer cache folder too?

On a side note.. is there really no way to mount/inject the cache folder
from the host with the new features offered by BuildKit?

—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub
https://github.com/moby/buildkit/issues/1009?email_source=notifications&email_token=AABJDLTED44CWII34MCKAHLPVRPDHA5CNFSM4HNFXRG2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGODVPV4PA#issuecomment-492789308,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AABJDLRNKD542LAH62CODHLPVRPDHANCNFSM4HNFXRGQ
.

[tonistiigi]

# syntax=docker/dockerfile:experimental
FROM composer:1.8 as vendor

COPY database/ database/

COPY composer.json composer.json
COPY composer.lock composer.lock

RUN --mount=type=cache,target=/tmp/cache composer install \
    --ignore-platform-reqs \
    --no-interaction \
    --no-plugins \
    --no-scripts \
    --prefer-dist

[vesper8]

thank you for sharing this snippet @tonistiigi !

At first I couldn't get it to run despite being on the latest edge with experimental features turned on, but then I realized I needed to add that first comment line that provides the syntax in order for it to run

I'm still not sure I understand what this mount does however? Will it somehow persist the /tmp/cache inside the builder container for future build runs?

[tonistiigi]

At first I couldn't get it to run despite being on the latest edge with experimental features turned on,

You do not need to turn on experimental since docker 18.09

I'm still not sure I understand what this mount does however? Will it somehow persist the /tmp/cache inside the builder container for future build runs?

It creates a persistent directory in builder cache that you will write the cache on first run and then on subsequent ones composer can read back the files. The cache files will not be in the final image.

[vesper8]

great! well it works wonder!

I tried to implement something similar to speed up installing dependencies with apk on php:7.3-fpm-alpine's image

RUN --mount=type=cache,target=/var/cache/apk apk update && apk add \
    build-base shadow vim nano curl \
    php7 \
    php7-common \
    php7-dom \
    php7-fpm \
    php7-gd \
    php7-json \
    php7-mbstring \
    php7-mcrypt \
    php7-mysqli \
    php7-openssl \
    php7-pdo \
    php7-pdo_mysql \
    php7-phar \
    php7-session \
    php7-xml \
    php7-zip \
    php7-zlib

Unfortunately this doesn't seem to work.. either I'm using the wrong cache folder or apk doesn't is being pretty slow either way.. can't notice a speed improvement at all.. if you spot something I did wrong I would appreciate if you let me know.

In any case big thanks for helping out! You've solved my problem! I didn't need to upgrade to the Docker Edge release but I love the new progress bars and split windows during the build so I think I'll stay on Edge

[tonistiigi]

@vesper8 Look at https://wiki.alpinelinux.org/wiki/Local_APK_cache for apk. This should do it:

RUN --mount=type=cache,target=/var/cache/apk ln -s /var/cache/apk /etc/apk/cache && apk update && apk add \
    build-base shadow vim nano curl \
    php7 \
    php7-common \
    php7-dom \
    php7-fpm \
    php7-gd \
    php7-json \
    php7-mbstring \
    php7-mcrypt \
    php7-mysqli \
    php7-openssl \
    php7-pdo \
    php7-pdo_mysql \
    php7-phar \
    php7-session \
    php7-xml \
    php7-zip \
    php7-zlib

[FernandoMiguel]

@vesper8 Look at https://wiki.alpinelinux.org/wiki/Local_APK_cache for apk. This should do it:

RUN --mount=type=cache,target=/var/cache/apk ln -s /var/cache/apk /etc/apk/cache && apk update && apk add 

I'm very curious as to way this symlink would be needed @tonistiigi !

[FernandoMiguel]

anyway, for anyone finding this in the future, if you create the symbolic link, and later reuse that layer, DON'T try to recreate the symlink.... that cost me several minutes to debug

[tonistiigi]

I'm very curious as to way this symlink would be needed @tonistiigi !

It's just apk behavior documented in https://wiki.alpinelinux.org/wiki/Local_APK_cache . Presumably to avoid caching by default that would just bloat the images with extra tarballs.

[SaturnusDJ]

Would this be a fix for https://github.com/docker/buildx/issues/110?
I would not understand that then because I am using --no-cache everywhere to prevent cache ever being used.