Sceptre: Enhancement: Cache assume-role credentials across sceptre invocations

[cyphus]

Currently, sceptre (more specifically, botocore) will cache assume-role credentials in-memory for the duration of the process, allowing multiple sessions to be created against the same profile. Botocore added support in 2017 (PR) for persisting assume-role credentials in the filesystem, compatible with awscli. However, it is not enabled by default.

Because credentials are not stored in a persistent cache, the user has to specify an MFA token for every sceptre command ran under an MFA-required assume-role profile, which is tedious for testing and development. Furthermore, the MFA requirement inadvertently rate-limits the user, as AWS only allows a token to be used once, even if the second command is ran within the same 30 second window. Once a token is used, the user has to wait for it to expire before running the next operation. The situation is even more aggravating when creating/updating stacks that reference outputs from stacks in other accounts, requiring the user to enter multiple unique MFA tokens in a row to complete the operation.

[craighurley]

For each environment, I tell sceptre which profile to use by setting the profile group_config item: https://sceptre.cloudreach.com/latest/docs/stack_group_config.html#profile

I use aws-mfa to manage credentials obtained from STS: https://github.com/broamski/aws-mfa

Before working on an environment or set of environments, I run aws-mfa --profile PROFILE_NAME for each environment then run sceptre commands until the credentials expire (default is 1h).

[ngfgrant]

Agreed with @craighurley on this. There are a multitude of utilities that can handle if you user wants.

[cyphus]

While there are definitely a multitude of ways to handle this, I developed this solution because it uses the method with official support from AWS, both in aws-cli and in botocore/boto3.

The PR I created simply enables a builtin feature in botocore. It is superior to alternatives because it doesn't require installing any third party tools, doesn't require you to re-run said tools every time your session expires (which in my case is hourly), and doesn't enforce a specific profile naming scheme like aws-mfa does.

@ngfgrant I respect your decision to close this issue and related PR, but for my use cases I am still going to pursue a solution that involves the officially supported credentials cache.

[mreeves1]

@cyphus - Agree with you. This would be really elegant and work just like aws cli itself does.

I set up a nice Role/Policy/User that enables mfa, etc. and was really confused why it works perfectly with aws cli and sceptre prompts me for MFA over and over.

Based on https://stackoverflow.com/questions/34795780/how-to-use-mfa-with-aws-cli

My IAM CF:

AWSTemplateFormatVersion: "2010-09-09"

Description: Global IAM

Parameters:
  WhitelistIp:
    Type: String
    Default: none
    Description: "Optional IP address that all API requests must come from. Leave as 'none' to disable"

Conditions:
  useWhitelistIp: !Not [ !Equals [ !Ref WhitelistIp, none ] ]

Resources:
  # IAM Convention: $vendor-$group(optional)-$domain(optional)-$purpose
  # Human IAM ##########################################################################################################
  HumanAssumePolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      ManagedPolicyName: human-assume
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: sts:GetCallerIdentity
            Resource: "*"
          - Effect: Allow
            Action: sts:AssumeRole
            Resource:
              - !Sub "arn:aws:iam::${AWS::AccountId}:role/human-power"
              - !Sub "arn:aws:iam::${AWS::AccountId}:role/human-admin"

  HumanAdminRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: human-admin
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal: { AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" }
            Condition: { Bool: { "aws:MultiFactorAuthPresent": true } }
            Action: sts:AssumeRole
          - !If
            - useWhitelistIp
            - Effect: Deny
              Principal: { AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" }
              Condition: { NotIpAddress: { "aws:SourceIp": [ !Ref WhitelistIp ] } }
              Action: sts:AssumeRole
            - !Ref AWS::NoValue
      ManagedPolicyArns:
        - !Ref HumanAdminPolicy
      MaxSessionDuration: 43200

  HumanAdminPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      ManagedPolicyName: human-admin
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Action: "*"
            Resource: "*"

  HumanPowerRole:
    Type: AWS::IAM::Role
    Properties:
      RoleName: human-power
      AssumeRolePolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            Principal: { AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" }
            Condition: { Bool: { "aws:MultiFactorAuthPresent": true } }
            Action: sts:AssumeRole
          - !If
            - useWhitelistIp
            - Effect: Deny
              Principal: { AWS: !Sub "arn:aws:iam::${AWS::AccountId}:root" }
              Condition: { NotIpAddress: { "aws:SourceIp": [ !Ref WhitelistIp ] } }
              Action: sts:AssumeRole
            - !Ref AWS::NoValue
      ManagedPolicyArns:
        - !Ref HumanPowerPolicy
      MaxSessionDuration: 43200

  HumanPowerPolicy:
    Type: AWS::IAM::ManagedPolicy
    Properties:
      ManagedPolicyName: human-power
      PolicyDocument:
        Version: 2012-10-17
        Statement:
          - Effect: Allow
            NotAction:
              - "iam:*"
              - "organizations:*"
              - "account:*"
            Resource: "*"
          - Effect: Allow
            Action:
              - account:ListRegions
              - iam:CreateServiceLinkedRole
              - iam:DeleteServiceLinkedRole
              - iam:ListRoles
              - organizations:DescribeOrganization
            Resource: "*"

  HumanMreevesUser:
    Type: AWS::IAM::User
    Properties:
      UserName: mreeves
      ManagedPolicyArns:
        - !Ref HumanAssumePolicy

Setup MFA for a new user

1. Create user/role/etc. via CFn

2. Create their access key

   aws iam create-access-key --user-name mreeves --profile $profile
   {
      "AccessKey": {
         "UserName": "mreeves",
         "AccessKeyId": "REDACTED",
         "Status": "Active",
         "SecretAccessKey": "REDACTED",
         "CreateDate": "2020-01-01T23:59:59+00:00"
     }
   }
   

3. Manually create their password in the console and save that to a password manager

   aws iam create-virtual-mfa-device --virtual-mfa-device-name mreeves \
   --outfile ./QRCode.png --bootstrap-method QRCodePNG --profile $profile
   {
      "VirtualMFADevice": {
          "SerialNumber": "arn:aws:iam::1234567890:mfa/mreeves"
      }
   }
   

4. Open the QRCode.png file and scan it into an authenticator app. Delete it afterwards!

5. Connect the MFA Device to the user. You will get the auth codes from the authenticator app.

   aws iam enable-mfa-device --user-name mreeves --serial-number arn:aws:iam::1234567890:mfa/mreeves \
   --authentication-code1 123456 --authentication-code2 987654
   

6. To wire up the user to the roles you will edit the ~/.aws/credentials file like so:

   [mreeves]
   aws_access_key_id = REDACTED
   aws_secret_access_key = REDACTED
   
   [mreeves_power]
   role_arn = arn:aws:iam::1234567890:role/human-power
   source_profile = mreeves
   mfa_serial = arn:aws:iam::1234567890:mfa/mreeves
   
   [mreeves_admin]
   role_arn = arn:aws:iam::1234567890:role/human-admin
   source_profile = mreeves
   mfa_serial = arn:aws:iam::1234567890:mfa/mreeves
   

7. Use one of the "role profiles" such as mreeves_power.
   aws s3 ls --profile mreeves_power
   The command will appear to hang (for some reason) but is waiting for you to enter your MFA token.
   This will be cached when used by awscli but for whatever reason sceptre will prompt you every time.

Then your sceptre file would look like this:

project_code: foobar
profile: mreeves_admin
region: us-west-1

Any interest @ngfgrant in reopening this if we update the PR?

This doc references the cache we are talking about:
https://docs.aws.amazon.com/cli/latest/userguide/cli-configure-role.html#cli-configure-role-cache