Aws-sdk-net: Add ability to prefer environment variable credentials over stored profile credentials

[imkheong]

Situation:
When specifying credential using environment variable, it relies on the FallbackCredentialsFactory to decide on a credential. For situation where the host has a stored profile credential, there is no way to configure the AWS SDK to prefer environment variable over stored profile credentials.

Problem:
It is difficult to run on a specified credential where host has a default profile credential defined.

Suggestion:
Add ability to specify use of environment variable credentials in AWS Options. How about a flag to indicate the choice to choose environment variable credential over stored profile credential?

[imkheong]

I have found a quick fix to the problem:
I was using the Amazon.Extensions.NETCore.Setup's AddDefaultAWSOptions and AddAWSService extension methods as shown in the .NET core recommendation.

// Configure Access key, secret key and region
services.AddDefaultAWSOptions(new AWSOptions());
// Register Aws clients
services.AddAWSService<IAmazonECS>();
services.AddAWSService<IAmazonS3>();
services.AddAWSService<IAmazonEC2>();

Having a second thought, it is not a must to use the above. Commenting out above code, replacing with:

var credentials = new EnvironmentVariablesAWSCredentials();
 // Custom AWS client registration
services.AddTransient<IAmazonECS>(serviceProvider => new AmazonECSClient(credentials));
services.AddTransient<IAmazonS3>(serviceProvider => new AmazonS3Client(credentials));
services.AddTransient<IAmazonEC2>(serviceProvider => new AmazonEC2Client(credentials));

// Register application Services
services.AddTransient<AwsEcsService>();

For my service code using the Amazon clients,

    public class AwsEcsService
    {
        private readonly IAmazonECS _amazonEcs;
        public AwsEcsService(IAmazonECS amazonEcs)
        {
            _amazonEcs = amazonEcs;
        }
        ...
    }

It will use the environment variable credentials over the stored profile credentials.

[sstevenkang]

I would like to understand the context a little more. It's unconventional that you have both a default/preferred credentials in the credential store and environment credentials. What's the reasoning behind this?

Also, I think a more general solution to this problem would be to allow clients to specify the fallback order. Would that functionality help in this case?

[imkheong]

The code was a continuous integration script to run on Jenkins slave to perform one click migration from one ECS cluster to another ECS cluster. It will check that both environment runs the same task definition image, cpu, ram etc.

The stored profile credential existed for some projects that has been using the AWS CLI to automate the ECS task registration in some Jenkins jobs.

The project was built using a different IAM user credential and it is not registered to the stored profile credential. It is specified through Jenkins bindings which sets the access key id and secret access key using Environment variable.

I do not want to trouble my Jenkins administrator as the Jenkins build binding can do the work to set the credentials over environment variables. It is also convenient to use Jenkins job configuration UI to change that without needing an administrator's intervention, which usually has a certain turn-around time involved.

Hoping this will justify the case.

[imkheong]

Continuing to answer the question...

The effort you guys put in to build the AWS SDK on .NET core has been WONDERFUL. However, I find it liberating to leave behind the Amazon.Extensions.NETCore.Setup package completely, which solved the issue when it comes to setting credentials using environment variable. A separate struggle when attempting to configure the environment variable credential previously in issue https://github.com/aws/aws-sdk-net/issues/494.

Personally, I would prefer to go direct without the abstracting package by Amazon.Extensions.NETCore.Setup.

Setting up the credential with Amazon.Extensions.NETCore.Setup is just not convenient at the moment for environment variable credentials in my opinion.

[normj]

Amazon.Extensions.NETCore.Setup is meant to be a helper for the common scenarios for setting up clients and credentials in an ASP.NET Core application. This seemed important since we no longer had the simple story of web.config for configuring your AWS clients. In no way should you feel required to use it.

I'm wondering if on the AWSOptions class if we should also expose a Credentials property of type AWSCredentials. So in your case you could have got the AWSOptions from Configuration and then set the Credentials property like this:

var options = Configuration.GetAWSOptions();      

// Assuming this property existed
options.Credentials = new Amazon.Runtime.EnvironmentVariablesAWSCredentials(); 

Then when it creates the client we would prefer the credentials property if set over profile or default profile.

[imkheong]

Hey normj

I was away on holidays so my apologies for the late response.

To me that would make AWSOptions a more complete options class. When specifying an option, it should be able to determine the credential 'strategy' without relying solely on fallback.

A user would be able to specify the credential either through code or through configuration.

[normj]

Version 3.3.0.3 of AWSSDK.Extensions.NETCore.Setup was released which allows you to set the credentials directly on the AWSOptions class.