Cli: docker builder prune should produce error when using "label" filter

[stevenengler]

When I attempt to clear select build cache files using the --filter option of docker builder prune, the filter option is seemingly ignored. For example if I use docker builder prune --filter "label=asdfasdfasdf", it removes all build cache files.

Steps to reproduce the issue:

1. Dockerfile:

FROM ubuntu:18.04
RUN echo "Hello World"

1. sudo DOCKER_BUILDKIT=1 docker build .
2. sudo docker builder prune --filter "label=asdfasdfasdf"

Describe the results you received:

WARNING! This will remove all dangling build cache. Are you sure you want to continue? [y/N] y
Deleted build cache objects:
ccsi71vdr28pnus85bldmoonf
hk1yxod8uv58i2tpui5it3rqn

Total reclaimed space: 42B

All cache files were deleted.

Describe the results you expected:

In the case of a non-existent label like described above (for example asdfasdfasdf), it should not delete anything.

Additional information you deem important (e.g. issue happens only occasionally):

Output of docker version:

Client: Docker Engine - Community
 Version:           19.03.8
 API version:       1.40
 Go version:        go1.12.17
 Git commit:        afacb8b7f0
 Built:             Wed Mar 11 01:25:46 2020
 OS/Arch:           linux/amd64
 Experimental:      false

Server: Docker Engine - Community
 Engine:
  Version:          19.03.8
  API version:      1.40 (minimum version 1.12)
  Go version:       go1.12.17
  Git commit:       afacb8b7f0
  Built:            Wed Mar 11 01:24:19 2020
  OS/Arch:          linux/amd64
  Experimental:     false
 containerd:
  Version:          1.2.13
  GitCommit:        7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc:
  Version:          1.0.0-rc10
  GitCommit:        dc9208a3303feef5b3839f4323d9beb36df0a9dd
 docker-init:
  Version:          0.18.0
  GitCommit:        fec3683

Output of docker info:

Client:
 Debug Mode: false

Server:
 Containers: 5
  Running: 0
  Paused: 0
  Stopped: 5
 Images: 48
 Server Version: 19.03.8
 Storage Driver: overlay2
  Backing Filesystem: <unknown>
  Supports d_type: true
  Native Overlay Diff: true
 Logging Driver: json-file
 Cgroup Driver: cgroupfs
 Plugins:
  Volume: local
  Network: bridge host ipvlan macvlan null overlay
  Log: awslogs fluentd gcplogs gelf journald json-file local logentries splunk syslog
 Swarm: inactive
 Runtimes: runc
 Default Runtime: runc
 Init Binary: docker-init
 containerd version: 7ad184331fa3e55e52b890ea95e65ba581ae3429
 runc version: dc9208a3303feef5b3839f4323d9beb36df0a9dd
 init version: fec3683
 Security Options:
  apparmor
  seccomp
   Profile: default
 Operating System: Ubuntu 18.04.4 LTS
 OSType: linux
 Architecture: x86_64
 Docker Root Dir: /var/lib/docker
 Debug Mode: false
 Registry: https://index.docker.io/v1/
 Labels:
 Experimental: false
 Live Restore Enabled: false

Additional environment details (AWS, VirtualBox, physical, etc.):

[NateBu]

Same issue. Also tried upgrading docker-ce (to version 20.10.1) and get the same results.

Client: Docker Engine - Community
 Version:           20.10.1
 API version:       1.41
 Go version:        go1.13.15
 Git commit:        831ebea
 Built:             Tue Dec 15 04:34:59 2020
 OS/Arch:           linux/amd64
 Context:           default
 Experimental:      true

Server: Docker Engine - Community
 Engine:
  Version:          20.10.1
  API version:      1.41 (minimum version 1.12)
  Go version:       go1.13.15
  Git commit:       f001486
  Built:            Tue Dec 15 04:32:40 2020
  OS/Arch:          linux/amd64
  Experimental:     true
 containerd:
  Version:          1.4.3
  GitCommit:        269548fa27e0089a8b8278fc4fc781d7f65a939b
 runc:
  Version:          1.0.0-rc92
  GitCommit:        ff819c7e9184c13b7c2607fe6c30ae19403a7aff
 docker-init:
  Version:          0.19.0
  GitCommit:        de40ad0

[thaJeztah]

Hmm.. yes, this should be documented. The label filter is not currently supported for buildkit build-cache (the build-caches do not have labels; only the final image that's produced could have a label); https://github.com/moby/moby/blob/46cdcd206c56172b95ba5c77b827a722dab426c5/builder/builder-next/builder.go#L158-L159

Ideally, this should produce an error when using docker builder prune, but the reason it currently "ignores" this filter, is that the same code-path is used when using docker system prune, which supports label filters for some object-types (images, volumes, etc); https://github.com/moby/moby/pull/38238

There was some discussion about that on https://github.com/moby/moby/pull/38238#discussion_r235280758, but I don't think work was done to make an exception for this in the cli

/cc @tiborvass

[henry701]

The absence of label filters also break cleaners that relied on labels previous to enabling BuildKit.

As an example, I use the label ephemeral with value true for parts of multi-stage dockerfiles that are very likely to change, and clean them passing the output of docker image list --all --quiet --filter label=ephemeral=true --filter dangling=true into rmi --no-prune --force $IMAGES (--no-prune being needed here so it does not cleanup more cache than it should) repeteadly inside a loop until no more images are deleted.

BuildKit uses the builder cache instead of creating random hashed images to represent build cache, so here I thought docker builder prune --filter label=ephemeral=true --force would be OK, but it actually cleaned every dangling build cache, including some very lengthy steps.

I will use the until filter as a workaround for now.

Ironically, I barely noticed this, because BuildKit builds so much faster than the original builder that the time of the full build was slower by only around 30 seconds or so :)