Buildkit: exporter: support resetting timestamp for determinism

[AkihiroSuda]

[AkihiroSuda]

We also need to consider gzip determinism

For small images we might be able to just push the image without gzip and call it a day

[tonistiigi]

If we only reset times in differ it is little unsafe because snapshots in local build cache and remote build cache will have different timestamps. But if we reset in snapshot it will be 1) slow 2) confuse the containerd naive differ.

I guess the first is fine if this is opt-in from the user and clearly marked as an exporter feature.

With a custom (eg. fuse based) snapshotter+differ we could do this without the above limitations as well.

[ppiotrow]

Description

The Reproducible Builds guideline points that build tools should make build timestamp configurable https://reproducible-builds.org/docs/source-date-epoch/

While building docker image with docker build the image "created" property is set up to the docker daemon system timestamp. This is against reproducibility guidelines as even on the same host you're unable to build the same image again. It's even more difficult to do it on different hosts with docker daemon.

There is this cool blogpost that describes the simplest ever docker image ending up having different digests when running on two different hosts or without cache.

Workarounds
The JIB builds images on it's own, without using docker build command.
They create *.json and *.tar filles manually just to override "Created" property of the image.
It is described in their FAQ

Describe the results you expected:
I'd love to have additional option in the build command like
docker build --sourceDateEpoch='1970-01-01 00:00:00.0' .
or just --sourceDateEpoch=0 with default value to system.time().

This would enable to reuse layers cache while building docker images from various build tools plugins like this https://github.com/sbt/sbt-native-packager/issues/1321 or described here

[tonistiigi]

I'm fine with making the "created" configurable (it is already stable in buildkit if you get cache for a layer) but that on its own doesn't really solve this issue. The files generated in run commands still cause timestamps and gzip may not be deterministic. Resetting timestamps in snapshots is quite hard in current implementations. as explained before Some files are created by runc that is out of our control.

[Romain-Geissler-1A]

Hi,

Based on the previous answer, has this build flag to allow overriding just the "created" layer property rather than the layer files been implemented already ? That itself would already help the people who already do make layer storage stable already (ie not depending on current time).

Cheers,
Romain