Renovate: Support ECR registry authentication

[rohansingh]

What would you like Renovate to be able to do?
Authenticate against private Amazon ECR registries to be able to check and pull images, using an AWS access key ID and AWS secret access key.

ECR credentials are very short-lived, and need to be refreshed by using AWS credentials. Using the existing support for Docker registry authentication (#798) doesn't work because any username and password for ECR expire after a few minutes.

In general, the username is always AWS, but a new password has to be retrieved every few minutes via an AWS API.

Describe the solution you'd like

Something like this:

hostRules: [
    {
      hostType: 'ecr',
      awsAccessKeyID: '<AWS access key ID>',
      awsSecretAccessKey: '<AWS secret access key>',
    },
  ],

In getAuthHeaders in the Docker datasource, we could check for a host rule of type ecr, and use that to retrieve the correct credentials for the request via an AWS API call.

Describe alternatives you've considered
For users that are self-hosting Renovate, it's trivial to use the aws CLI to retrieve a password and set it in DOCKER_PASSWORD. However, this doesn't work for users of the GitHub app.

Additional context
Authentication for ECR registries is described here:
https://docs.aws.amazon.com/AmazonECR/latest/userguide/Registries.html#registry_auth

[rohansingh]

@rarkins If the solution I outlined above sounds reasonable, I'd be happy to put together a PR this week.

[rarkins]

https://docs.aws.amazon.com/cli/latest/reference/ecr/get-authorization-token.html says that we can use HTTP Basic auth tokens that are valid for only 12 hours. Would that be a valid alternative?

[rarkins]

FYI we don't actually need to do a docker login or any other docker x commands - we hopefully can just use the Docker HTTP API.

So the next question is can we call aws ecr get-authorization-token via API rather than CLI? e.g. via https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_GetAuthorizationToken.html

[rohansingh]

https://docs.aws.amazon.com/cli/latest/reference/ecr/get-authorization-token.html says that we can use HTTP Basic auth tokens that are valid for only 12 hours. Would that be a valid alternative?

Totally. I was under the mistaken impression that they were valid for a shorter period of time (hence the "few minutes" I mentioned above).

So the next question is can we call aws ecr get-authorization-token via API rather than CLI? e.g. via https://docs.aws.amazon.com/AmazonECR/latest/APIReference/API_GetAuthorizationToken.html

Yes exactly, I think that's the best way to go. While the CLI approach is what most folks do when pushing from a terminal, Renovate could just use the API. To do that though, we need to provide an AWS access key ID and secret access key. I figured that a hostRule with hostType: "ecr" would be a good way to do that.

[rohansingh]

Also for the record, the HTTP basic auth token is just a base64-encoded version of AWS:<some password>. Once you have that, any existing Docker registry API's should "just work" (barring things like #3800).

[rarkins]

Great. Ideally I'd like to reuse username/password fields like so:

hostRules: [
    {
      hostType: 'ecr',
      username: '<AWS access key ID>',
      password: '<AWS secret access key>',
    },
  ],

But the problem is we actually can't use hostType=ecr because it will already be populated with hostType=docker by the docker datasource. I can't think of a way to overload the hostType field right now, which means either:

• we add another field, something like 'authType', which we could populate as "ecr" and maybe use later for other peculiar cases, or
• we hardcode some logic for ECR into our auth
• we add awsAccessKeyID and awsSecretAccessKey as you originally suggested

Currently our auth logic is done using a combination of https://github.com/renovatebot/renovate/blob/master/lib/util/got/host-rules.js and https://github.com/renovatebot/renovate/blob/master/lib/util/got/auth.js

[rarkins]

What's AWS's preferred approach for token retrieval? Should we cache it for 12 hours or is it ok to query it every time?

[rohansingh]

But the problem is we actually can't use hostType=ecr because it will already be populated with hostType=docker by the docker datasource.

Am I misunderstanding what's happening in the Docker datasource here then:
https://github.com/renovatebot/renovate/blob/714d4b77f9227941e05514a4c1110a20fff5e415/lib/datasource/docker/index.js#L52

Would it be possible to do something like this:

ecrOpts = hostRules.find({ hostType: 'ecr', url: apiCheckUrl });
dockerOpts  = hostRules.find({ hostType: 'docker', url: apiCheckUrl });

<somehow merge these>

[rohansingh]

What's AWS's preferred approach for token retrieval? Should we cache it for 12 hours or is it ok to query it every time?

There are limits on how many times that endpoint can be called:
https://docs.aws.amazon.com/AmazonECR/latest/userguide/service_limits.html

It's 20 requests per second per repository, with a burst up to 200 rps. So it would be nice to cache the credential for a few hours, but also wouldn't be the end of the world to not cache it.

[rarkins]

I think:

• set hostRule with hostType=ecr and username/password as I suggested
• docker datasource detects ecr hosts and if matching then does a special token lookup and populates the Authorization header
• cache the token in memory per run, but not between runs

[puug]

This is my basic take as I needed something ASAP - #4497

• Uses the existing host rules, with docker as hostType (no custom type) with access key / secret as username/password - uses the default credential provider chain otherwise.
• Region detection based on ECR host
• Doesn't yet cache the token