Minikube: "cache add" with sha256 sum: "Failed to cache... repository can only contain the runes"

[kameshsampath]

Right now the minikube cache command allows to cache only images with tag, it will be great to support caching images with SHAs

[tstromberg]

That sounds totally reasonable, and more like a bug than a feature request. What command-line are you using, and what currently happens when you try to add an image to a cache with a SHA?

[kameshsampath]

@tstromberg here is an example of the cache add with sha256 with error

minikube cache add gcr.io/kaniko-project/executor@sha256:d9fe474f80b73808dc12b54f45f5fc90f7856d9fc699d4a5e79d968a1aef1a72

Error

💣  Failed to cache and load images: caching images: caching image ~/.minikube/cache/images/gcr.io/kaniko-project/executor@sha256_d9fe474f80b73808dc12b54f45f5fc90f7856d9fc699d4a5e79d968a1aef1a72: creating docker image name: repository can only contain the runes `abcdefghijklmnopqrstuvwxyz0123456789_-./`: kaniko-project/executor@sha256

😿  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
👉  https://github.com/kubernetes/minikube/issues/new

[tstromberg]

Thanks for the additional information - it makes it much clearer what's going on.

I'm not sure what needs to be changed to make this work, but would be happy to approve any PR's that address this issue.

[kameshsampath]

if I get bandwidth will try sending PR your way .

[afbjorklund]

I think we should be using ParseReference, since that allows _both_ tags and digests ?

https://github.com/google/go-containerregistry/blob/master/pkg/name/ref.go#L41_L50

    tag, err := name.NewTag(image, name.WeakValidation)
    if err != nil {
        return errors.Wrap(err, "creating docker image name")
    }

https://github.com/kubernetes/minikube/blob/master/pkg/minikube/machine/cache_images.go#L300

[afbjorklund]

Gaah, unfortunately there is currently no tarball Write that accepts Digest (or Reference)...

    td := tarDescriptor{
        singleImageTarDescriptor{
            Config:   cfgName.String(),
            RepoTags: []string{tag.String()},
            Layers:   layerFiles,
        },
    }

For a digest tarball, this is supposed to say "RepoTags":null in JSON - doesn't work with tag

[afbjorklund]

@kameshsampath : Did a sample implementation here: https://github.com/kubernetes/minikube/compare/master...afbjorklund:reference

But those vendor changes should go upstream, and they added some more features to write.go.

[afbjorklund]

Added https://github.com/google/go-containerregistry/pull/404

[kameshsampath]

@afbjorklund - thanks, not sure I make sense here can we reuse this code ? https://github.com/knative/serving/blob/master/pkg/reconciler/v1alpha1/revision/resolve.go

[afbjorklund]

That code looks very similar, i.e. they are both using name.Tag and name.Digest ?

The main remaining problem with g/go-cr is that it is impossible to _load_ digests.
i.e. the API only takes Tag as parameters, and for a Digest that would be nil

But minikube doesn't really care, since we only use it to save tarballs anyway...

In case it wasn't obvious, the above code snippet does fix loading of digests.
It is just a matter of how to get the feature included in the vendored upstream.

[kameshsampath]

CC: @vdemeester, Vincent did patch the go/go-cr for another PR, he might be able to help us.

[vdemeester]

@afbjorklund upstream's PR got merged :angel: Do you want to open a PR here or should I go ahead ? :angel:

[afbjorklund]

I'm giving it a try: dep ensure -update github.com/google/go-containerregistry

[afbjorklund]

Oops, look like I broke upstream:

pkg/minikube/machine/cache_images.go:316:22: too many arguments in call to tarball.Write
    have (name.Tag, v1.Image, nil, *os.File)
    want (name.Reference, v1.Image, io.Writer)

My understanding of Go inheritance is limited.

---

EDIT: fortunately, it wasn't me. Read too fast, the ref is fine - it was someone else that broke API:

https://github.com/google/go-containerregistry/commit/03167950e20ac82689f50828811e69cdd9e02af2

[ImJasonH]

That was me. 👋

Luckily the fix is straightforward, just remove the nil argument and it should work again.

Sorry for the speedbump, we're hoping to stabilize the go-containerregistry library soon, and do real releases.

[afbjorklund]

Works as advertised, the only difference is in the RepoTags:

--- a/manifest.json 1970-01-01 01:00:00.000000000 +0100
+++ b/manifest.json 1970-01-01 01:00:00.000000000 +0100
@@ -1 +1 @@
-[{"Config":"sha256:025ab64f8cc830417dc6d85b1f2cbdff9030d1ba1bd781a44f3191f53450214b","RepoTags":["gcr.io/kaniko-project/executor:latest"],"Layers":["20aeb61b9e385a5ee65b8985abaa1b3dd01a301247c9ceb8048ff4dd70da81a9.tar.gz","76dd575757341869d3268e5a80f6492487a3ff4f2cf2fb220bf1280e99ce49a3.tar.gz","ec294f5524f9bb08d9e715bb4ff11c11d722fe46517408f8edc9c21378cca882.tar.gz","6352599e53cdf9f3c68b6f37ae5aa3e011f5f94386c839e63c5e2ca2b07eef8b.tar.gz","f5344ea33cc949b8da96ef152cc55add5428fdcace3436e14614c7833329398d.tar.gz","ec44e76f69b1fbdaa890e4724840a3c82c950680b78dc354ecdef905f9f9496e.tar.gz","ceb0b51a5b370acaa47fe3d52833b65252f3af2dc82bdcb3bd9619aafb6d5a25.tar.gz"]}]
\ No newline at end of file
+[{"Config":"sha256:025ab64f8cc830417dc6d85b1f2cbdff9030d1ba1bd781a44f3191f53450214b","RepoTags":null,"Layers":["20aeb61b9e385a5ee65b8985abaa1b3dd01a301247c9ceb8048ff4dd70da81a9.tar.gz","76dd575757341869d3268e5a80f6492487a3ff4f2cf2fb220bf1280e99ce49a3.tar.gz","ec294f5524f9bb08d9e715bb4ff11c11d722fe46517408f8edc9c21378cca882.tar.gz","6352599e53cdf9f3c68b6f37ae5aa3e011f5f94386c839e63c5e2ca2b07eef8b.tar.gz","f5344ea33cc949b8da96ef152cc55add5428fdcace3436e14614c7833329398d.tar.gz","ec44e76f69b1fbdaa890e4724840a3c82c950680b78dc354ecdef905f9f9496e.tar.gz","ceb0b51a5b370acaa47fe3d52833b65252f3af2dc82bdcb3bd9619aafb6d5a25.tar.gz"]}]
\ No newline at end of file

Similar to the output from docker save, identical RepoTags