Nixpkgs: AWS CLI version 2

[KyleOndy]

A few weeks ago [1] amazon released v2.0.0 of their CLI. I do not see any open issues or MR to try and package this tool yet.

---

Project description
The AWS Command Line Interface (CLI) is a unified tool to manage your AWS services. With just one tool to download and configure, you can control multiple AWS services from the command line and automate them through scripts.

Metadata

• homepage URL: https://aws.amazon.com/cli/
• source URL: https://github.com/aws/aws-cli
• license: apache2.0
• platforms: linux, darwin

[1] https://aws.amazon.com/blogs/developer/aws-cli-v2-is-now-generally-available/

[jonringer]

the aws python ecosystem is a monumental pain. Each "patch" release usually has breaking changes and it's hard to gauge what version the majority of the ecosystem supports.

That being said, the awscli package uses buidlPythonApplication which means that it can deviate from the normal python package set.

cc @muflax (maintainer)

[KyleOndy]

the aws python ecosystem is a monumental pain.

That doesn't surprise me very much. I am not in any immediate need for this package, but I can take a swing at trying to get in packaged when I have some free time.

[barakber]

ATM I'm using something like this if it helps:
~ nix
let
pkgs = import {};
awscli = with pkgs; stdenv.mkDerivation {
name = "awscli";
src = fetchzip {
url = https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip;
sha256 = "1pqvqhxbikqnji7nazvaqnk2czlmr3kvs1zyl13w96ym4if3np1v";
};
propagatedBuildInputs = with pkgs; [
];
installPhase = ''
mkdir -p $out/bin
./install -i $out -b $out/bin
'';
};
in pkgs.buildFHSUserEnv {
name = "awscli";
targetPkgs = pkgs: with pkgs; [
awscli
groff
];
profile = ''
export C_INCLUDE_PATH=/usr/include:$C_INCLUDE_PATH
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:${awscli}/v2/dist
'';
}
~

Then
~ shell
11:39 barak@berkos:/tmp/awscli $ nix-build
/nix/store/8kmna6m3d46n74ffd2lfq5808sw49c5j-awscli
11:39 barak@berkos:/tmp/awscli $ ./result/bin/awscli
awscli-chrootenv:barak@berkos:/tmp/awscli$
awscli-chrootenv:barak@berkos:/tmp/awscli$ aws cloudformation help
...
~

[dhess]

This is working for me in an overlay:

https://github.com/hackworthltd/hacknix/blob/master/pkgs/awscli/2.0/default.nix

[dhess]

It appears the bash completion is not working, but the utilities seem to work so far, including SSO support.

[surajbarkale]

The overlay proposed by @dhess worked for me on macOS as well. I bumped up the version to following:

      botocore = super.botocore.overridePythonAttrs (oldAttrs: rec {
        version = "2.0.0dev19";
        src = fetchFromGitHub {
          owner = "boto";
          repo = "botocore";
          rev = "ce7b76d296b345d7647f50557606947ad857a786";
          sha256 = "0g9pvcs94v2znjkzzjnz30021q9bm97fxmlhxvql6im3c6adrqb3";
        };
      });
-----
  pname = "awscli";
  version = "2.0.15";

  src = fetchFromGitHub {
    owner = "aws";
    repo = "aws-cli";
    rev = version;
    sha256 = "030pw7aa1y13awrpwlsq3ib633fh1z4rzkz8b31x7jg7s1hxrsvr";
  };

[jeffutter]

Hey, I'm pretty new to nix here. Does anyone have any suggestions on how to use @dhess 's code with home-manager until the official package gets updated?

[jonringer]

Hey, I'm pretty new to nix here. Does anyone have any suggestions on how to use @dhess 's code with home-manager until the official package gets updated?

This can be done using an overlay:
https://nixos.wiki/wiki/Overlays

I would just create something like a awscli.nix in or local overlay directory and follow the example in wiki article, but substituting the code from above.

Another option would be to just open a PR where you do the changes and merge it into master (although this won't be an immediate change)

[jeffutter]

Thanks for the help. Sorry I'm a little dense here and I realize this isn't a support forum, if there is somewhere better to ask this question I'd be glad to take it elsewhere.

I can't seem to get it to work as an overlay. I put dhess' code in ~/.config/nix/overlays/awscli/default.nix

and have this in my ~/.config/nix/default.nix:

self: super: {
  awscli = super.callPackage ./awscli { };
}

but when I home-manager build I get:

error: infinite recursion encountered, at undefined position
error: infinite recursion encountered, at undefined position

I'm sure I don't really understand how overlays work.

[rgoulter]

Inlining @dhess's code into the snippet from the wiki link above, I got https://gist.github.com/rgoulter/075268b116294b773a17aa4efe51e6c7

Putting the contents from this gist in ~/.config/nixpkgs/overlays/awscli.nix would let you run nix-env --install --attr nixpkgs.aws_cli_2_0. (Or presumably with home manager add pkgs.aws_cli_2_0 to the home.packages).

[jeffutter]

@rgoulter Thanks so much for the help. I was originally getting thrown by how to convert the { lib , python3 , groff , less , fetchurl }: in dhess' code.

Your version installed great. I had some trouble running it, aws --version would error about not being able to find abc. I switched all of the python stuff to use python3 and it works like a charm now.

Thanks a bundle.

[KyleOndy]

I am going to take a swing at packaging version 2 of this CLI and get it merged into master. It would be my first real contribution to this repo.

As v2 is not backwards compatible with v1 should the version 2 of the cli be packaged separate from the existing awscli?

I can foresee issues if we just bump the existing package to v2 all kinds of existing scripts and automatons _could_ break.

[jonringer]

for azure-cli, the previous version was deprecated (and eventually didn't even build), so i just replaced it. But, the acutal command changed from azure to az, so I wasn't worried about giving the false hope that previous workflows were still supported.

But for awscli, it just depends. Seems like they both use the aws command, so I would lean toward creating a new derivation awscli-v2 (or whatever makes sense). These implementation-specific details can also be discussed in a PR as well

[KyleOndy]

I spun my wheel locally trying to get something to work. I don't seem to grok the nix ecosystem enough yet. I'm gonna give another try this weekend when I find some time.

I'd love if someone else gets to this too.

[davegallant]

I spun my wheel locally trying to get something to work. I don't seem to grok the nix ecosystem enough yet. I'm gonna give another try this weekend when I find some time.

I'd love if someone else gets to this too.

I am also interested in having this package available in nixpkgs but have not yet contributed before.

I have an overlay working here with a slightly more recent version of awscli 2:
https://github.com/davegallant/dotfiles/blob/acf8681cf40742d185315e6703b439ba36dfabcd/.config/nixpkgs/overlays/awscli2.nix

EDIT: Took a stab at a derivation.

[austinbutler]

I think this issue can be closed now: https://nixos.org/nixos/packages.html?channel=nixpkgs-unstable&query=awscli2

[jonringer]

agreed