Elasticsearch: Add support for AWS STS credentials for snapshots

[jipperinbham]

The current setup does not support the ability to provide an AWS_SECURITY_TOKEN to be used with AWS STS authentication.

I believe the changes necessary would be to add an additional field to Settings for the cloud-aws plugin CLOUD_S3.TOKEN and if set, then return an instance of BasicSessionCredentials(accessKey, secretKey, sessionToken); instead of new BasicAWSCredentials(account, key).

[bleskes]

@dadoonet gracefully agreed to research and see what the implications of this are

[dadoonet]

Useful links to understand STS:

• http://docs.aws.amazon.com/STS/latest/APIReference/Welcome.html
• http://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html

If I'm not mistaken it means that we want to give a temporary credential to a S3 repository, right?

If so, I think it makes sense to support it only at a repository level but not in cluster/node settings. So having a temporary value for a setting like cloud.aws.token does not make sense.

If we want to implement it, I'd support it only when we create a repo. Like:

PUT _snapshot/tmp_repo
{
  "type": "s3",
  "settings": {
    "token": "temp-token-here"
  }
}

@jipperinbham WDYT?

[dadoonet]

No news on this. So I'm closing for now.
Feel free to comment and reopen.

[elliott-davis]

incase anyone else runs across this - It looks like it was resolved in #19556

[pheyos]

I would like to have support for AWS STS credentials (#19556 doesn't seem to resolve this).

As far as I understand, a session token is required when "MFA-Protected API Access" is activated on AWS [1]. Without support for session tokens all users affected by this policy are not able to access s3 via the plugin, so this would be an important improvement.
From my point of view the token should be in the elasticsearch-keystore, just like access_key and secret_key are [2].

[1] https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_mfa_configure-api-require.html
[2] https://www.elastic.co/guide/en/elasticsearch/plugins/6.1/repository-s3-client.html

[dadoonet]

I believe it's a valid ask. I believe that the implementation that described @jipperinbham is the right approach as well.
The question is more then about revocation of tokens. I'm not sure we can update live such a setting yet and it's probably something needed. Also the need of "reloading" a S3 client when credential are changing. Unless we just create a new client anytime we need to call S3...

[pheyos]

FWIW: The s3 input plugin for logstash seems to have support for session tokens [1].

[1] https://www.elastic.co/guide/en/logstash/current/plugins-inputs-s3.html