Aws-cli: Is there a reason you publish an up-to-date MSI but not an RPM which works with RHEL?

[michael-crawford]

I'd prefer to have an RPM instead of using the pip method to install AWS CLI on RHEL. This helps with insuring python packages installed by the OS are not replaced by pip which I've seen break other programs which depend on them.

Searching for issues or blog posts on this has turned up surprisingly un-useful results. I see that you package this for Amazon Linux and Windows, so it's surprising that you don't do what should be minimal work to package this for RHEL as well - sometimes we must use RHEL for support matrix reasons.

Is there a location where I can obtain awscli - in the most recent version or close to it - via RPM?

[dstufft]

It's not feasible for us to provide platform native packages for every OS so we have to pick and choose where we're likely to have the most impact. In particular on Windows there is no system Python provided so users need to get both Python and awscli which makes it an attractive OS to provide packages for.

I do not believe that we directly provide the Amazon Linux RPM and I think that is produced by the Amazon Linux team.

I'm going to go ahead and mark this as a feature request.

[michael-crawford]

It's quite difficult to replicate what's provided via RPM on RHEL, speaking as someone who has just attempted to do this. Here's some of what I noticed before I gave up this request as excessively complicated:

• aws-cli depends on a lot of python modules, some are not installed by default on RHEL, some are unique to aws-cli (i.e. jmespath, botocore) and developed in parallel. This creates a dependency chain that prevents an easy port of aws-cli to an RPM by customers, as we not only have to build an RPM for aws-cli, we have to build versions of about 7 other RPMs as well.
• But it's worse than that. The python module dependencies are based on a modification to how RHEL handles python. Where RHEL uses python-* for Python 2.7 and python3-* for Python 3.4, Amazon Linux supports both python26-* and python27-* as separate parallel versions, using alternates to switch between them. The SRPMs build both. Attempting to split these apart so that they can build properly on RHEL's python conventions was going to take a significant amount of time, and I wouldn't feel comfortable with the stability of the result. Plus, who knows how long such a port would take before it became obsolete.
• This is an effort which can be justified by the vendor of this software, but there's no way a customer using it can take on this effort. I really feel supporting the OS which is second only to Amazon Linux (above SUSE and Ubuntu!) in the "Launch Instance" pages of the Console is an AWS responsibility.
• This problem also exists with the awslogs and aws-cli-plugin-cloudwatch-logs RPMs. They can't be rebuilt due to these python problems.
• This problem also exists with the ec2-utils and ec2-net-utils plugins - dependent on upstart, not used in RHEL 7.

I know I'm asking you guys to take on more work, but in the interest of making your customer's experience with AWS as pleasant with RHEL - when we MUST use it for proprietary software vendor support matrix reasons - as it is with Amazon Linux, please realize this is a real pain point for us, and only you can address this if RedHat is not stepping up. They have their reasons for dragging their feet - it keeps people using their on-prem private cloud solutions instead of moving to AWS. Those are diametrically opposite to your interests in making this easier.

So, while I know this request extends beyond aws-cli, my request is to have an internal yum repo, usable by those running RHEL 6/7, CentOS 6/7 or variants of EL 6/7, containing RPMs which work for:

• aws-cli
• aws-cli-plugin-cloudwatch-logs
• awslogs
• aws-cfn-bootstrap
• ec2-utils
• ec2-net-utils

Many, MANY developers will thank you. This will certainly speed up the movement of on-prem RHEL-based workloads into AWS, built by CloudFormation with automation.

[lorengordon]

Ahem, https://github.com/ferricoxide/Lx-GetAMI-Utils

[nkadel-skyhook]

Tried this. It's a painful problem . The dependency tree isn't a tree, it's a forest by the time it's completed The dependencies of the dependencies accumulate, and have conflicting requirements, especially for "Sphinx" to generate the documents for the dependencies. The version dependencies of the subcomponents actively conflict. I had reached over 100 RPM's rebuilt for CentOS 6 to satisfy via RPM all the dependencies of the dependencies, and still could not resolve the conflicts, and had to surrender.

Frankly, for simple operations such as "aws s3 cp" and "aws s3 sync", it's much faster and more stable to use the "s3cmd" command available as an RPM from EPEL. Reserve awscli for tools that need more interaction with the ever-changing features of the AWS environment.

[ivanfetch]

I've solved this by downloading the AWS CLI bundle zip file, making that virtualenv relocatable, and using fpm to create an RPM of the virtualenv and a /usr/bin/aws symlink. This doesn't feel pretty, but it gives me a self-contained RPM which works on the major CentOS release for which it was created. Hopefully this will help someone else out there.

# Use the AWS CLI bundle zip file to create an RPM.
#
# Requirements: python-devel, and fpm which will require a newer ruby.
# We use fpm to create an RPM from a directory.
#
# DIR is normally set by the build system:
DIR=`pwd`
# Get and unzip the AWS CLI bundle,
# from https://s3.amazonaws.com/aws-cli/awscli-bundle.zip
# Unzip the bundle which creates an aws-cli directory.
#
# The build dir is what will be the root of the RPM
mkdir -p build/usr/local build/usr/bin
cd awscli-bundle
./install -i ${DIR}/build/usr/local/aws-cli
cd ..
# Extract the bundled virtualenv, and use it to create a relocatable virtualenv.
mkdir build/virtualenv
tar -C build/virtualenv -xzf awscli-bundle/packages/virtualenv-*.tar.gz
build/virtualenv/virtualenv*/virtualenv.py --relocatable build/usr/local/aws-cli
rm -rf build/virtualenv
# Since the AWS CLI will be installed to /usr/local/aws-cli,
# Create a symlink in /usr/bin.
# You could also just add /usr/local/aws-cli/bin to $PATH.
ln -s /usr/local/aws-cli/bin/aws build/usr/bin/aws
# Avoid RPM prelinking binaries which
# cause checksum mismatches - see https://github.com/jordansissel/fpm/issues/262
echo "=> Creating fake \$HOME and .rpmmacros to avoid RPM prelinking binaries, as an fpm work-around"
HOME=${DIR}/build
echo '%__prelink_undo_cmd     /bin/cat cat library' >${HOME}/.rpmmacros
#
# Create the RPM using FPM.
# Note the --description parameter does not work with \n as fpm docs describe.
# The RPM version is obtained from aws-cli.
fpm -s dir -t rpm \
-n aws-cli \
-v $(${DIR}/build/usr/local/aws-cli/bin/aws --version 2>&1 |awk '{print $1};' |cut -d/ -f2) \
--iteration 1 \
-a noarch \
--license 'ASL 2.0' \
--url 'http://aws.amazon.com/cli/' \
--vendor "YourDomain.com" \
-m "[email protected]" \
--description 'This package provides a unified command line interface to Amazon Web Services.

For more information and documentation please visit http://aws.amazon.com/documentation/cli/

Note this package is a Python virtualenv created from the aws-cli bundle zip file, with a symlink to the aws script.' \
--directories /usr/local/aws-cli \
-C ${DIR}/build \
usr/local/aws-cli usr/bin/aws

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

[jamesls]

Based on community feedback, we have decided to return feature requests to GitHub issues.

[nkadel]

RHEL 7 has reasonable up-to-date awscli backported from Fedora. RHEL 6, however, is a nightmare because python2.6 is so very, ver out of date that many modules simply cannot be compiled with the native python.

However, EPEL publishes the python34 squite of RPMs for RHEL 6, and I got it working by building up the extensive toolchain. My tools are at https://github.com/nkadel/awsclirepo . They can only build the python34-awscli RPM, which, which should have been named python2-awscli and an optional python36-awswcli RPM on RHEL 7 to follow the EPEL naming convention. I am exhausted with people naming Python packages as "package", rather than as "python2-package" or "python3-package" for the particular python release.

Anyway, it works well in my basic testing, and ideally all the components would be brought to EPEL.

[michael-crawford]

Great you went through all that work – but, I think you’re missing the point. AWS should be doing this, and publishing their RPMs in a public repo accessible to both instances and external VMs for consistency. Most other software vendors go through this trouble – it’s just expected for top-tier open source software packages. At least in EPEL, if not their own custom public YUM repo.

I use your packages, I just substitute a massive company that SHOULD be doing this for their own stuff, with one guy who, heroic as you may be, doing this. I can’t build a solution on one guy who could fail to keep these up-to-date. I don’t want to do it either.

AWS should be shamed into doing this – it’s just being a good open source citizen. They’re making stuff harder for their customers.

From: Nico Kadel-Garcia [e-mail]
Reply-To: aws/aws-cli [e-mail]
Date: Monday, April 29, 2019 at 10:12 PM
To: aws/aws-cli [e-mail]
Cc: Michael Crawford [e-mail], Author [e-mail]
Subject: Re: [aws/aws-cli] Is there a reason you publish an up-to-date MSI but not an RPM which works with RHEL? (#2501)

RHEL 7 has reasonable up-to-date awscli backported from Fedora. RHEL 6, however, is a nightmare because python2.6 is so very, ver out of date that many modules simply cannot be compiled with the native python.

However, EPEL publishes the python34 squite of RPMs for RHEL 6, and I got it working by building up the extensive toolchain. My tools are at https://github.com/nkadel/awsclirepo . They can only build the python34-awscli RPM, which, which should have been named python2-awscli and an optional python36-awswcli RPM on RHEL 7 to follow the EPEL naming convention. I am exhausted with people naming Python packages as "package", rather than as "python2-package" or "python3-package" for the particular python release.

Anyway, it works well in my basic testing, and ideally all the components would be brought to EPEL.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com/aws/aws-cli/issues/2501#issuecomment-487822934, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AADSAHI6DS4K3UBUFFXXFOLPS7ILJANCNFSM4DEGCGNA.

[nkadel]

awscli RPM building is working as is for Fedora and RHEL 7. RHEL 6 was.... a lot harder. It's not just a "publish a spec file" change, which I hae in fact published for awscli for RHEL 7. It involves updating or managing a set of related Python modules, such as s3transfer and botocore which aren't easily compatible with the very obsolete python 2.6 on RHEL 6. I only just figured out how to gracefully the the puthon34 hooks to support this. Without shaming anyone, I'm happy too cooperate or collaborate to get these into EPEL for RHEL 6.