Virtual-environments: Update to Python3 on AWS-cli

[Santiago8888]

Tool information

• Tool name: aws-cli
• Tool license: Apache License
• Add or update? update
• Desired version: aws-cli/1.18.78 Python/3.7.6 Linux/5.3.0-53-generic botocore/1.17.1
• Approximate size:

Area for Triage: Python

Question, Bug, or Feature?:
Bug: 'ascii' codec can't decode byte 0xc3 in position 66: ordinal not in range(128)

Virtual environments affected

• [ ] macOS 10.15
• [ ] Ubuntu 16.04 LTS
• [x] Ubuntu 18.04 LTS
• [x] Ubuntu 20.04 LTS
• [ ] Windows Server 2016 R2
• [ ] Windows Server 2019

Can this tool be installed during the build?
Yes:
pip uninstall awscli
pip3 install awscli

Tool installation time in runtime
30 seconds

Are you willing to submit a PR?
Yes.

[maxim-lobanov]

It looks like AWS-CLI is installed with system version of Python by default and it is 2.x.
To install it with custom python, the following line can be used: sudo /usr/local/bin/python3.7 awscli-bundle/install -i /usr/local/aws -b /usr/local/bin/aws (from official guide: https://docs.aws.amazon.com/cli/latest/userguide/install-linux.html).
@Santiago8888 , do you have an idea if it could cause any regression on image side?

Also, instead of changing Python 2 to 3 for current AWS-CLI, I would consider switching from AWS-CLI 1 to AWS-CLI 2.

The AWS CLI version 2 has no dependencies on other Python packages. It has a self-contained, embedded copy of Python included in the installer.

@Santiago8888 , Will it work for you? Do you have any context if this update can be breaking change for anyone?

[NeilJed]

+1 on this. The current AWS CLI is out of date as per their releases: https://github.com/aws/aws-cli/releases. The current CLI version on the VMs doesn't support the new CodeArtifact commands.

Also, AWS are promoting moving the CLIv2 over CLIv1 now.

[Santiago8888]

@maxim-lobanov , yes I would expect breaking-changes for some the users when switching the CLI to Python3.

I do not know a lot about CLI v2, at the moment they seem to be equally maintained, later I can test my repo and report back. I would imagine it is a good opportunity to also make that transition.

To minimize impact, I imagine we could push the changes first to Ubuntu 20.04 LTS and, consider replicating them to Ubuntu 18 later.

@NeilJed, what command do you use to install V2?

[maxim-lobanov]

You can switch to V2 using the following code:

sudo rm -f "/usr/local/bin/aws"
sudo rm -rf "/usr/local/aws"
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install

Probably, there is a simpler way but it takes about 3 seconds.
V2 has a bunch of breaking changes in comparison with V1.

In my opinion, updating AWS CLI on Ubuntu images to V2 will be the better decision because we already have AWS-CLI V2 on macOS and Windows images so it makes sense to update Ubuntu images too. Also it is better from perspective side. But we definitely need some announcement for that.
We are discussing it internally. Will keep you posted

[NeilJed]

Just to add another motivation, if you look at the AWS CLI main page it only offers direct downloads to CLIv2: https://aws.amazon.com/cli/https://aws.amazon.com/cli/
Pretty much all their documentation has a banner header now telling you to use v2: https://docs.aws.amazon.com/cli/latest/reference/sts/assume-role.html

I agree, this is maybe a breaking change, but I think one of the main features of CLIv2 is that it's self contained with Python 3.8 and thus doesn't have any system dependencies.

Maybe we need an action to set-up the CLI with a specific version, kind of how you can pick a version when setting up Node or Python.

[maxim-lobanov]

@Santiago8888 , @NeilJed , thank you for your patience.
We have discussed this issue and we wouldn't like to switch AWS CLI V1 to Python 3.
Instead, we are going to update AWS CLI V1 to V2 on Ubuntu 20.04 image (Ubuntu 18.04 is still the most popular ubuntu version so we don't expect huge impact of this change). I have created announcement #1108

@Santiago8888 , I think this decision will resolve your issue on Ubuntu 20.04. But I see that you mentioned Ubuntu 18.04 in issue description too. Is it critical for you to get this update on Ubuntu 18.04 too? Update on Ubuntu 20 should be safe enough but I worry about impact on Ubuntu 18.04. What do you think about it?

[Santiago8888]

@maxim-lobanov, I use Ubuntu 18, so technically the issue is not fixed on my side. However, per commented, the work-around is easy enough. I agree with the decision as I believe is the responsible thing to do, it is also reassuring that when migrating to Ubuntu 20 we will not face this issue again.

Best,

[dibir-magomedsaygitov]

Hello!
AWS CLI has been updated to V2 on Ubuntu 20.04 image. Changes were merged and will be delivered to VMs with the next week's roll out. I will post update message.

[maxim-lobanov]

Hello!
AWS-CLI was updated to V2 on Ubuntu 20.04: aws-cli/2.0.29 Python/3.7.3 Linux/5.4.0-1020-azure botocore/2.0.0dev33
Closing the issue. Please let me know if you have any concerns