Elasticsearch: Update AWS Java SDK to support ECS Task Roles

Created on 8 Feb 2017  路  4Comments  路  Source: elastic/elasticsearch

Describe the feature: Plugin repository-s3 currently use the the AWS Java SDK 1.10.69. Policy management for AWS resources used by Elasticsearch with AWS-plugins running on ECS would improve if the SDK version was upgraded to a newer release.

Background:
I noticed the old version due to lack of newer AWS IAM features. In this specific case:
Task IAM Roles.
IAM Roles for tasks require 1.11.16 or above.

IAM Roles for tasks are used as part of deployments to Amazon EC2 Container Service (ECS). Task roles allow specific containers, or set of containers, to run with specific Roles.
Task roles are similar to Instance Roles. In ECS the instances running in the cluster has an Instance role, typically allowing the cluster instances access to various ECS APIs, Log Management and Load-Balancer registration/deregistration. Containers running on the instance also inherits the policies of the Instance role.
If you wish the container to have access to resources without granting it to every container running in the cluster, you need to either use Task roles, or access keys.

We strive to avoid the usage of access keys for applications running on AWS.
Because of this our current workaround is to set the policies we require on the Instance role.

:DistributeSnapshoRestore >enhancement good first issue help wanted

All 4 comments

I took a brief look at how we might support this. One small trouble I had was identifying which credential provider to use, but I _think_ it would be ContainerCredentialsProvider. If using that provider is all that is necessary, then this should be a simple addition, and work similarly to the instance profile support.

The default chain should also cover ECS usage, as long as the SDK is new enough.
DefaultAWSCredentialsProviderChain

Credentials delivered through the Amazon EC2 container service if AWS_CONTAINER_CREDENTIALS_RELATIVE_URI" environment variable is set and security manager has permission to access the variable

@hmatland We no longer use the default chain because it allows for insecure ways of providing the credentials (sys props, environment variables, and profile files).

This would also be a very useful feature for the discover-ec2 plugin.

Was this page helpful?
0 / 5 - 0 ratings

Related issues

brwe picture brwe  路  3Comments

rjernst picture rjernst  路  3Comments

clintongormley picture clintongormley  路  3Comments

dadoonet picture dadoonet  路  3Comments

martijnvg picture martijnvg  路  3Comments