Terraform: S3 Provisioner

[SanchitBansal]

It is not actually an issue but more of a requirement. Can we have S3 provisioner to download content from S3 bucket just like file provisioner. My use-case is to upload private sshkeys to provisioned instance which I neither want to keep locally nor on git. Hence S3 is the best place to keep it as I am already using AWS S3 buckets. Can we have this provisioner? Or how can I handle it with current available resources.

[Puneeth-n]

doesn't remote-exec with awscli and ec2 instance profile suffice? Assuming that you start an instance with some keypair.

[SanchitBansal]

Hacks can always be applied, and for this unnecessarily I will have to configure awscli on provisioned instance and also have to pass aws credentials to make it work which again should not be there. Having one straightforward provisioner would be best. And I think someway or the other it must be useful for other terraform users.

[stack72]

Hi @SanchitBansal

thanks for the question here - I believe you can use the aws_s3_bucket_object data_source here

https://www.terraform.io/docs/providers/aws/d/s3_bucket_object.html

That allows you to retrieve the contents of an s3 bucket

Will this work for you?

Paul

[Puneeth-n]

@SanchitBansal you don't need credentials if you use roles ;)

[susannamartinelli]

HI @stack72,
correct me if I'm wrong but the page you suggested describe how to run a lambda function which resides on S3.
Does we need to download from S3 into the instance?

Thanks

[apparentlymart]

The example on that page is showing how to use the aws_s3_bucket_object data source to find the latest version of a zip file to use as the implementation of a lambda function.

The first part of the example is the relevant part for this request, since we actually want to retrieve the body (rather than the metadata) of the object in question:

data "aws_s3_bucket_object" "secret_key" {
  bucket = "awesomecorp-secret-keys"
  key    = "awesomeapp-secret-key"
}

resource "aws_instance" "example" {
  ## ...

  provisioner "file" {
    content = "${data.aws_s3_bucket_object.secret_key.body}"
  }
}

Please note the restriction described in the docs where body is available only if the object is configured with a Content-Type of something starting with text/ or is application/json. Unfortunately the usual MIME type of application/x-pem-file for a PEM key won't work here since the application/ prefix doesn't pass Terraform's whitelist of "safe" mime types to read.

[apparentlymart]

This data source is intended as the main way to download data from S3 to use in a Terraform config, so I'm going to close this issue since I think the original use-case has been met even though it was done in a different way than suggested.

If anyone has other use-cases that aren't met by this data source, I'd encourage opening a new issue in the AWS provider repository since that's where AWS provider development happens these days, since it was split out of this core Terraform repository.

Thanks for this feature request!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.