Aws-cli: An official Docker Image with the AWS CLI for use in CI/CD scenarios

[matti]

I read Issue #3529 and #3291 and saw those were closed, with the only reaction hinting it was 'not that complicated'. However, the comment also acknowledged that doing this yourself would run the risk of being out of date. Apart from exactly that point, I would also like to point out that, for commercial users, having an official Amazon image is hugely preferential to "/aws-cli:latest".

In my case, I would be using this in a Google Cloud Build because it is far superior than AWS CodeBuild.

[nscavell]

I am here because I too am looking for an official aws-cli docker image to use in my CI/CD environment. Instead I am now going to create ANOTHER pipeline to build a docker image which includes the aws cli when I could just reference the official one in my original pipeline.

Also...someone else gets it too https://hub.docker.com/r/google/cloud-sdk.

[justnance]

@matti and @nscavell, Thank you for following up on this topic. Sounds like there enough interest in this feature request to keep this open. This issue will be used to track an official Docker image for the AWS CLI. Please +1 it to show interest and help us prioritize this.

Thanks.

[matti]

Isn't +1 considered harmful ;)

Anyway this is the third (?) issue created about this...

[Deamon]

👍 add my +1

I have to create my own docker image to only include awsCLI in my CI. I would prefere an official one

[tim-palmer]

+1

[FraaJad]

+1

[mathiewz]

+1

[derickson82]

+1

[riznob]

+1

[robert-smith-nike]

+1

[harsha91]

+1

[ecimionatto]

+1

[shaunPDX]

+1

[B0yc3y]

+1

Heres an alpine (post fix) image with the most recent cli installed for anyone else looking for a recent version in the mean time.

[matti]

@justnance enough?

[nidi3]

+1

[ina-stoyanova]

+1

[lunarxlark]

+1

[davidconde]

+1

[nhoughto]

Another reason is when using an OS like coreOS that has no package management the idiomatic way of running things is via a container. I'm surprised the need for this would even be questioned, is an obvious omission. 👍

+1

[serlank]

:+1:

[cduris]

+1

[sigursoft]

+1

[jacky-tam]

+1

[dataplex]

+1

[kvc-code]

+1

[Rabadash8820]

As the opener of #3291 (the earliest of the three quoted Issues lol), I'm glad to see that this issue is finally gaining traction. To all future responders, when @justnance says "Please +1 it to show interest", he means add a like on the first comment, that's the proper way to +1 things on GitHub so that you're not spamming the maintainers' mailboxes.

[xplorld]

+1 to keep repo owners notified

[bert-laverman]

+1

[davidham]

When they say to +1 an issue, what they mean is to click the 👍 button. It doesn't help the developers to have 100 comments that each say "+1". The more you know!

[justnance]

@davidham - Thanks for clarifying and everyone for responding. I second that. Please use the GitHub reactions and click the 👍 button.

[Rabadash8820]

I said the same thing 2 days ago but hey we're all on the same side 🙃

[dirty49374]

+1

[lohmuller]

+1

[ragazzid]

+1

[saurav111]

+1

[NathanJewell]

+1

[Rakito]

+1

[sergey-vin]

+1

[pierreozoux]

Can you stop +1? if you want to +1, do it on the top.

You are wasting valuable engineer time. We are dozen subscribed to this issue...

[mikesir87]

I have been maintaining an aws-cli image for over two years now. Feel free to use it if needed (I use it several times a day). I receive updates on new version releases (via IFTT) and push updates fairly quickly. Despite the fame and glory of running my own image (ha!), I will _gladly_ defer and push folks to use an official image.

• GitHub repo
• Docker Hub

After having used my image for a long time, I will say that it would be _super_ helpful if the official image included jq (as the API is JSON heavy). It does make it super convenient to do things such as "grab latest ECS task definition, make a change, and push it back" all in a single CI/CD pipeline stage.

[isaiahgrant]

Yet another alternative solution to the problem: https://hub.docker.com/r/somedeveloper/aws-cli/

Reasons for using:

• Keeps it simple.
• Uses official python3.7-stretch as base image.
• Uses AWS recommended strategy for installation -- python + pip. see here.
• Includes aws-sam-cli for those Serverless nerds 8-).
• Its public and doesn't require a login.
• Great for CI/CD pipelines -- haven't used it for anything else so haven't weighed the pros and cons.

Still hoping for an official image. Think about the people now

[vehsakul]

https://hub.docker.com/r/google/cloud-sdk/ . Sorry, guys. It is such a hard task to do for a giant like AWS.

[edovale]

+1

[iancward]

Isn't +1 considered harmful ;)

Anyway this is the third (?) issue created about this...

Fourth, if you consider https://github.com/aws/amazon-linux-docker-images/issues/10.

[flaccid]

Can't we just put it in CircleCI and be done with it? Happy to help with the Dockerfile or pipeline.

[xplorld]

I wonder if there are any internal restrictions and/or paperworks inside Amazon that keeps the developers away from accomplishing this seemingly trivial work...

[matthewflannery]

k lol

[chuckwilbur]

There are a few variants that would be nice to have in an "official" image. For instance, I'm currently looking to grab a container with the aws cli and curl (to check the IAM metadata endpoint) and it would be really handy to find one I could just grab and plug into my k8s deployment.

[pawelprazak]

There is also a security reason for providing official images.

It makes the threat model simpler by removing a dependency on a "random person on the internet" maintaining the image that is used in high value targets like CI/CD pipelines that usually give a lot of access to your systems.

[nngo]

I would like an official aws-cli docker image based from the docker:18 (current stable) image (https://hub.docker.com/_/docker) - eg. aws-cli-docker-18 because I would like to build my docker image in my CI/CD environment that currently uses docker:18 image and publish it to AWS ECR.

[lgarvey]

+1

[aschoenenberg]

+1

[MatthiasKunnen]

It would be great if people would refrain from commenting when their comment does not contribute to the issue at hand. Comments such as "+1" just introduce spam for subscribers and make the issue lengthier than necessary. Instead, give a thumbs up to the first comment of the issue.

[cabrinha]

It would be great if people would refrain from commenting when their comment does not contribute to the issue at hand. Comments such as "+1" just introduce spam for subscribers and make the issue lengthier than necessary. Instead, give a thumbs up to the first comment of the issue.

This issue has been open since September of last year. I think we need to ask AWS to take a look at this issue again, since there is obviously interest in it. We should be told how much interest is "enough".

[lucasbasquerotto]

since there is obviously interest in it

Not just interest, but is the opened issue with more :+1: reactions:

https://github.com/aws/aws-cli/issues?q=is%3Aissue+is%3Aopen+sort%3Areactions-%2B1-desc

The 2nd with most reactions was opened in 2014 and the 3rd in 2015, while this issue was opened in September of 2018 (less than a year ago).

[seanturner83]

+1000

[DjThossi]

I have on my local machine all the time issues with setting up the right dependencies to make aws-cli work. Therefore I decided to switch to aws-cli inside docker. I found several publicly available images on docker-hub BUT because they are not official I don't trust them by default so every time there is an updated version available I have to search and find again an image which can be trusted. and is secure to use. Please AWS build, maintain and provide a secure aws-cli docker image via docker hub. Thanks in advance

[guice]

There's a huge fragmentation of community provided aws-cli images. But, as mentioned earlier: people would prefer one that's officially maintain and supported by Amazon. Several reasons why:

1) Won't stale out -- community images are notorious for eventually going stale;
2) More secure -- it's from a trusted resource, no matter how trusted a community member may be, they don't officially represent Amazon, therefore "all warranties void";
3) Official support means official support in the event of bugs, version conflicts, backwards compatibilities, etc.
4) AWS can update their image as they update the aws cli, including historical tags.

[joeinnyc]

+1

[JustAnotherSoftwareDeveloper]

+1 It's really unfortunate that Amazon does not provide this

[isaiahgrant]

I have since added another image that has been handy. It combines Docker in Docker with the AWS and SAM CLI which makes for a perfect ECR integration.

dind-aws-cli

[valll94]

+1

[JoeJenningsDMC]

Whilst there are loads of unofficial images out there that are well maintained, what's to stop them one day from saying "Pfft, what's the point of updating this anymore. This is Amazon's job!"

[zensam]

+1

[oded-dd]

+1

When it comes to helping people automate their work with AWS Amazon are failing big time, this is one of the reasons we are thinking of leaving them.

[chantzlarge]

Is there an official response from maintainers regarding whether or not this is on a road map yet? Like many others, I would prefer to use an official image as well...

[fv-robert]

+1

[ghost]

If there were an official image for the AWS CLI it would be an executable sitting inside a scratch container. Because that wouldn't be incredibly useful for individuals what might be best is to:

• Use an official python container to build the CLI from source
• Copy the resulting AWS CLI binary into a busybox or scratch container

Anything more would be over-engineering despite any debates occurring here.

[cabrinha]

AWS loves to demo all of it's ECR and CodeDeploy services. Why won't they point all that firepower at their own containerized CLI?

[ghost]

Because the offer on the table it seems is for someone from the community to maintain it.

[matti]

@balibebas somebody from the community is already doing it. There are a ton of images out there - but the point here is that there would be one by AWS, because I don't want to run randomguy/aws-cli:canbechangedanytime in my CI environment with all the secrets wide open.

[ghost]

What would you say about F-Droid then?

[matti]

it is as relevant to this issue as this cat:

[ghost]

You sound like a paying customer.

[matti]

well maybe that is because _I_ _am_

[ghost]

Preaching to the choir. Anyway the Brew formula had a better chance because it's been trolled longer, still with no movement. So cat photos become immediately counter productive when there's no general use case outside maybe a scratch or busybox container as I pointed out earlier. What's your design proposal?

[matti]

me and many others are currently doing "busybox containers" for example to run it with docker to get ECR auth credentials. given how many docker things aws has, it makes no sense that an official packaging does not exist.

[ghost]

Maybe they're just onto something else. ;)

[matti]

I'm glad we got this sorted out now. Back to +1 comments:

[ghost]

+1 for Dockerless

[radium-v]

@matti @balibebas Keep in mind that as of right now there are 64 people in this issue's conversation alone, and every response triggers an unhelpful email and notification that potentially gets sent to every one of those 64 people and more who are subscribed. This message will do the same, and I apologize for that.

But really, please keep things professional. While cat pictures are nice, the further this thread goes off the rails I feel the less likely it is to be taken seriously by the maintainers. Unfortunately, spam is spam is spam.

[ghost]

That's what the unsubscribe button does, the one I just clicked.

[beyondszine]

That would be a good thing to have. I am only surprised that no action is being taken or this is being maintained (includes being said NO).
Anyway people above have rightly pointed out the importance of an official aws cli image.
I think people are already using their own privately or via somebody else's built image / package managers route etc.
Yet Another Example Script for the same

But the simplest & non-avoidable problem remains the same.

• The never ending list of dependencies & uncertainties that hops in if it depends on developers to integrate things on their own. Will eventually lead to more issues in the repo as people will start with installing one thing then other and then a few other to get work done contaminating the environment which defeats the purpose of CI/CD or similar works of being isolated & pristine.
• Its hard to trust to implement & track anybody else's docker image for your work. This leads to making another pipeline & entry addition in docker registry for new awscli image, another repository i.e another thing to maintain.
• In CI/CD, my preference is to just use the image (our internal or official) & to avoid scripting lines to add things as much as possible.
• Problem with build & libs as official image can be right away build from source releases itself & have less dynamic dependencies wrt package manager & what not.

else
=> Everybody ends up making their own.

[jens-meiss]

same here, currently i use a self build image, but would prefer a official image under the

• https://hub.docker.com/_/awscli

namespace. I use it to build other docker images and push them to ECR, where the awscli is needed to get the credentials.

FROM docker:18.06

RUN apk update && \
apk -Uuv add python py-pip && \
pip install awscli && \
apk --purge -v del py-pip && \
rm /var/cache/apk/*

It shouldn't be that big deal to add this to your awscli build pipeline ... :)

--

i have updated the Dockerfile according to @mikesir87 suggestion, thx for that (thats another reason to have a standard image with contributions of the comunity to get the smallest image)

[mikesir87]

Sorry to spam everyone, but wanted to point out (in case anyone else wants to use the snippet from @jens-meiss) that you should _really_ consider chaining your three RUN statements into a single statement. Otherwise, you're still shipping around the contents of py-pip and the apk caches in the intermediate layers, even though your final container can't access them.

On another note, the comment brings up another valid use case... when you're using the aws-cli only to get credentials for ECR to push images. This sounds like a need for packaging up the ECR credential helper in an image too. It would certainly make it easy to build and push images without needing the full aws-cli.

[jamesls]

Hi everyone, maintainer here. Just wanted to clarify, this is happening, we're doing this.

We're in the process of building out better release infrastructure internally to be able to build/test/support more types of release artifacts, especially as we'll be shipping more release artifacts for cli v2.

We don't have an exact timeline at the moment, but it is happening.

[jvosantos]

Easy thing to do according to amazon devs and many others, yet still waiting after 2 years and no ETA 😂

[leonixyz]

+1

[xplorld]

Internal releasing infrastructure (2019 Q4)
legal team assessment (2020 Q1)
public beta under aws/cli-dev-test (2020 Q2)
final release (2020 Q3)

In this optimistic timeline, you will get what you want in less than 10 months. 🥇

[matti]

waiting for jeff barrs blog post

[inductor]

I'm damn waiting for this.

[bhmckendrick]

Can we at least get an official announcement or commitment. Maybe a target release ?

[joshsleeper]

@bhmckendrick is a commitment not exactly what this comment not too far above yours is?

https://github.com/aws/aws-cli/issues/3553#issuecomment-519280276

[chb0github]

over a year old and no updates? Image?

[joshsleeper]

hey @jamesls, would you consider hard-locking this thread until you have something to share?

the number of people wholly unwilling to read (hint hint) and instead choosing to spam 70+ watchers about how they _specifically_ are annoyed is, I'm sure, drastically lowering everyone's interest in following this thread.

also, thanks for your work towards making this happen!

[matti]

As the original author of this issue (well, I was just brave enough to copy/paste the previously closed "wontfix" ones) I have already unsubscribed from this issue because of the massive +1 spam and occasional fights with cat pictures (sorry).

Just adjust notification settings so that you only get notifications if this issue is closed.

[LongLiveCHIEF]

I'd just like to point out that aside from CI/CD, some developers (see @LongLiveCHIEF), prefer to have dockerized development environments, and don't like installing things natively, or having to deal with the ensuing version managers for the native languages those cli tools inevitably rely on.

It's easier to docker pull aws-cli than whatever the existing install steps are... not to mention if you're not a python developer, you have the overhead of setting up a good python version and environment for your user, or perhaps even each project.

Scale that to all the different tools a developer might use (ruby based cli's, node based cli's), and you have to learn environment setups for languages you might not ever code in.

The point I'm making here is that docker run is ubiquitous, and gets rid of any native language setup/configs, and makes things easy for users.

Even if users build their own docker images, they still have to struggle with those setup tasks.

I don't code in python, but I've been forced to learn the ins and outs of virtual environments and their best practices from various versions of python, purely because tool providers consider it "trivial".

Not all developers have the same background as those who built the tool, and providing a docker image is a sign of respect. The tool providers can take on the overhead of native language specific environment quirks very quickly and easily, whereas adopters have to spend far more time to manage this overhead through the various lifecycle stages of development with your product.

Food for though.

[ajhool]

@jamesls Thanks for listening to community feedback here. While waiting for the official hosted docker images, a helpful intermediate step might be to post install recommendations for a few popular base docker images here (eg. node, alpine, ubuntu, amazon2, python). This would be immediately valuable to us.

[blurayne]

As I work for big ass enterprise let me serve you with this:
https://github.com/aws/aws-codebuild-docker-images
https://hub.docker.com/r/amazon/aws-codebuild-local

[creduo]

This doesn't look like well maintained, but here's another one

https://hub.docker.com/r/amazon/lambda-build-node10.x

[pablosjv]

I have just spotted this into the wild: https://hub.docker.com/r/amazon/aws-cli

It seems that it's finally here :)

[anjakammer]

@pablosjv it is not an official or certified image. Be aware of that.

[lucasbasquerotto]

@anjakammer It seems it's an official amazon image, even though it's not an official image published by Docker.

I don't know if it's production ready tough, because they said nothing in this issue yet.

[scarytom]

Curious how this AWS image is 98.42 MB whereas others (e.g. atlassian/pipelines-awscli) are much smaller.

[mbaitelman]

https://aws.amazon.com/blogs/developer/aws-cli-v2-docker-image/

[kyleknap]

CLI team member here. Yep I can confirm we have officially launched a Docker image for the AWS CLI v2! I recommend reading the following:

• Blog post introducing it: https://aws.amazon.com/blogs/developer/aws-cli-v2-docker-image/
• User guide entry: https://docs.aws.amazon.com/cli/latest/userguide/install-cliv2-docker.html
• DockerHub repository: https://hub.docker.com/r/amazon/aws-cli

I'm going to close out this issue. If you have feedback or questions about the image, please open another GitHub issue in this repository.

[Rabadash8820]

As opener of the original issue #3291, those many years ago, my aching ❤️ is filled with such joy to see my concerns finally validated, and an official Docker image now available. Pot shots about how long it took to make this image aside, I'm sure this was much easier said than done, so a big thanks to the Amazon devs who made this happen. You all do excellent work. 👏🎉👏

_Okay Alexa, I thanked them, please let my family go now._

[zerkms]

Is the Dockerfile available anywhere?

[dbirks]

@zerkms Aha, found it in the v2 branch:
https://github.com/aws/aws-cli/blob/v2/docker/Dockerfile

[pSnehanshu]

I guess they are finally doing it, but I couldn't get it to run on Gitlab CI https://hub.docker.com/r/amazon/aws-cli

I'm instead using Gitlab's AWS CLI docker image, and it works perfectly. Just use

image: registry.gitlab.com/gitlab-org/cloud-deploy:latest

UPDATE:

The above image is deprecated, use the new one instead.

image: registry.gitlab.com/gitlab-org/cloud-deploy/aws-base:latest

[mikesir87]

Note that to use the image on GitLab CI, you will need to manually set an empty entrypoint, as the amazon/aws-cli image sets the entrypoint to /usr/local/bin/aws. An example...

image:
    name: amazon/aws-cli
    entrypoint: [""]

[pSnehanshu]

@mikesir87 why's that?

[lucasbasquerotto]

@pSnehanshu I think that's because you run the image as if it was the cli itself, like docker run --rm amazon/aws-cli <<command>>, which would be similar to running the cli with aws <<command>>, instead of docker run --rm amazon/aws-cli aws <<command>>. There are pros and cons with each approach, depending on what you prefer, and the way you run the image, but overriding the entrypoint should do the trick anyway.

[pSnehanshu]

@lucasbasquerotto I have settled for Gitlab's image anyway. Anyway, thanks for the explanation.

[blagerweij]

In case anyone is still interested to get AWS CLI v2 working on Alpine Linux, here's an example Dockerfile:

FROM alpine:3.11 AS builder

ENV GLIBC_VER=2.31-r0

# install glibc compatibility for alpine
RUN apk add --no-cache --virtual .build-deps \
        binutils \
        curl
RUN curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk
RUN curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk
RUN apk add --no-cache \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk

# install AWS CLI
RUN curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip
RUN unzip awscliv2.zip
RUN aws/install

FROM alpine:3.11
MAINTAINER Barry Lagerweij
RUN apk --update --no-cache --virtual .build-deps add \
    groff \
    && rm -rf /var/cache/apk/*
COPY --from=builder /usr/local/aws-cli/ /usr/local/aws-cli/
COPY --from=builder /usr/local/bin/ /usr/local/bin/
COPY --from=builder /usr/lib/ /usr/lib/
COPY --from=builder /lib64 /lib64
COPY --from=builder /usr/glibc-compat/ /usr/glibc-compat/
COPY --from=builder /lib/ld-linux-x86-64.so.2 /lib/

The problem was that AWS CLI v2 uses GLIBC, but Alpine Linux has limited GLIBC support (it uses 'musl', a light-weight alternative). The Dockerfile above installs the missing glibc libraries, and uses a multi-stage Dockerfile to keep the final image small. With a bit of effort, we can probably reduce the image size even further by only including the files from /usr/lib that are really required.

[blagerweij]

After some refactorings, I've managed to reduce the image size even further:

FROM alpine:3.11

ENV GLIBC_VER=2.31-r0

# install glibc compatibility for alpine
RUN apk --no-cache add \
        binutils \
        curl \
    && curl -sL https://alpine-pkgs.sgerrand.com/sgerrand.rsa.pub -o /etc/apk/keys/sgerrand.rsa.pub \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-${GLIBC_VER}.apk \
    && curl -sLO https://github.com/sgerrand/alpine-pkg-glibc/releases/download/${GLIBC_VER}/glibc-bin-${GLIBC_VER}.apk \
    && apk add --no-cache \
        glibc-${GLIBC_VER}.apk \
        glibc-bin-${GLIBC_VER}.apk \
    && curl -sL https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip -o awscliv2.zip \
    && unzip awscliv2.zip \
    && aws/install \
    && rm -rf \
        awscliv2.zip \
        aws \
        /usr/local/aws-cli/v2/*/dist/aws_completer \
        /usr/local/aws-cli/v2/*/dist/awscli/data/ac.index \
        /usr/local/aws-cli/v2/*/dist/awscli/examples \
    && apk --no-cache del \
        binutils \
        curl \
    && rm glibc-${GLIBC_VER}.apk \
    && rm glibc-bin-${GLIBC_VER}.apk \
    && rm -rf /var/cache/apk/*

The auto-complete index and example files are removed, and also 'groff' is removed (I'm assuming people don't need help pages in their Docker images)

[flaccid]

This is very simple, https://github.com/flaccid/docker-awscli/blob/master/Dockerfile and does the job but let me know via my github issues if anything else is needed in the image (valid use case).

[blagerweij]

This is very simple, https://github.com/flaccid/docker-awscli/blob/master/Dockerfile and does the job but let me know via my github issues if anything else is needed in the image (valid use case).

The above APK is based on AWS-CLI 1.18, not on the v2 CLI.

[keeganwitt]

Any chance Amazon would consider creating an image with version 1 of the CLI?

[Rabadash8820]

@keeganwitt You should open a new Issue for that request. :+1: