Tilt: 401 Unauthorized when pulling an image from an authenticated Docker registry

[sssilver]

Tilt is failing with 401 Unauthorized when trying to pull an image from an authenticated Docker registry. Running tilt docker -- pull <image url> directly works. This caches the image, and subsequent tilt ups succeed because they use the cached image.

[sssilver]

Screenshot:

[sssilver]

After running:

$ tilt docker -- pull docker-registry.cfdata.org/stash/cf/cfsetup-images/stretch/master:16-0168a0d7b56d
Running Docker command as:
DOCKER_BUILDKIT=1 docker pull docker-registry.cfdata.org/stash/cf/cfsetup-images/stretch/master:16-0168a0d7b56d
---
16-0168a0d7b56d: Pulling from stash/cf/cfsetup-images/stretch/master
8176e34d5d92: Pull complete 
b99190f16537: Pull complete 
ff5f2c04b21c: Pull complete 
906a3808b97c: Pull complete 
Digest: sha256:f5dbeb24a832cd0b325da12d5325cc76b5a62fa38e0c8aaea80a6ab547aac28c
Status: Downloaded newer image for docker-registry.cfdata.org/stash/cf/cfsetup-images/stretch/master:16-0168a0d7b56d
docker-registry.cfdata.org/stash/cf/cfsetup-images/stretch/master:16-0168a0d7b56d

everything comes up.

[nicks]

Oof, sorry to hear about this.

I tried to reproduce this with both a gcr.io private registry and a hub.docker.com private registry, and both worked ok for me. Do you know what docker-registry.cfdata.org uses for auth?

[sssilver]

It's https://github.com/cesanta/docker_auth

[jazzdan]

@sssilver I've tried to reproduce with this example repo.

Do you know what credential helper you have configured for docker-registry.cfdata.org? (You can see the mapping of registry to credential helper in ~/.docker/config.json)

Something we thought of is that maybe there's a token that is expiring while Tilt is running. Does Tilt ever successfully pull down the image, even when you just start up Tilt?

Also, can you take a look in env for these environment variables? https://docs.docker.com/engine/reference/commandline/cli/ Do you have any of them set?

[jazzdan]

Also, what's your output for tilt doctor?

If you think it would be more helpful I'm also happy to jump on a call to help resolve this ([email protected] is my email address).

[jazzdan]

Hey @sssilver just checking in to see if this is still a problem for you!

[nicks]

Since we haven't heard any other reports of this, and haven't been able to repro with known authenticated registries, I'm going to move this ticket to low-priority. We can raise the priority again if we get more info