Buildkit: `docker build` command omits some middle steps when buildkit is enabled

[beatak]

I have a Dockerfile like this:

# Step 1. Source fetching
FROM base as source
# fetch required source from multiple repositories

# Step 2. Building
FROM base-builder as builder
COPY --from=source ~~
# do some building

# Step 3. Run testing
FROM base-qa
# setup db and such
RUN rspec

# Step 4. Baking a running image
FROM base
COPY --from=builder ~~

Basically the following blog entry explains what I'm doing. My assumption is "_If any step of the build fails, no final image is generated._"
https://adilsoncarvalho.com/creating-multiple-images-from-a-single-dockerfile-3f69254b6137

So if I run this with DOCKER_BUILDKIT=0, It says there are 39 steps and it runs sequentially. But if I run this with DOCKER_BUILDKIT=1, It says there are 26 steps, and I don't see Step 3 runs in output.

With Buildkit enabled, it runs efficient orders and such and I love it, but it is troublesome if the Step 3 doesn't run. Is there any way to make sure to run the Step 3?

[tonistiigi]

These are not built because the final stage you are building has no dependency on the middle stages. If you don't want to use --target for the other stages you can make a stage buildandtestall and make it depend on everything you want to run. The easiest way to do that is to copy a file from that stage, for example, the test output or just copy --from=test test-completed .. If you use a custom frontend you can also use RUN --mount for the links and avoid actually moving any files. We could probably add some custom syntax for these links as well.

[beatak]

Thanks @tonistiigi for comments. I will try what you suggested. Also I really appreciate your work for Buildkit. This is really amazing!

[AkihiroSuda]

Is this issue closable now?

[beatak]

I decided to take the latter path and it works great.

I'm closing this issue but I guess one request: Can we have more documents? 😃

[gaggle]

WOW!, can I revive this old issue to ask where this is documented? Truly surprising that stages get skipped. I run tests and submit coverage reports in "middle stages", so this behavior has broken all our pipelines in a subtle way where no-one noticed.

(no disrespect intended, y'all are wonderful people for making your code and tools available ❤️)

I've added this to the final stage, which isn't so bad of a workaround:

RUN --mount=type=bind,from=middle,target=/middle echo "middle"
# ⬑ This is needed to ensure Docker runs the "middle stages" that don't directly contribute to the final result