Algo: EC2: Encrypted EBS boot/root volumes

[defunctio]

It would be great to see support for both encrypted EBS root and boot volumes w/ KMS implemented for Algo.

References

• https://aws.amazon.com/blogs/aws/new-encrypted-ebs-boot-volumes/
• https://aws.amazon.com/blogs/aws/protect-your-data-with-new-ebs-encryption/

[dguido]

$500 bounty! Submit a pull request and email [email protected] to claim it. Partial solutions may be rewarded.

[jackivanov]

Either ansible or boto has a bug in the encrypted option and we forced to use awscli :\
We may use another way and make the encrypted image as a public image, but I think users actually want to use the trusted provider, like Canonical

[defunctio]

I'm not really keen on awscli being a requirement for this and we definitely want to continue to use images maintained by Canonical.

I decided to look into this and the problem is with boto. The API version declared in boto for ec2 connections is quite old. The encrypted parameter was only added more recently, unfortunately the older version of the API is used for backwards compatibility. Changing it to a version that supports the parameter would introduce breaking changes into boto I'm sure. So while the source code of boto had this feature added awhile ago, the API version it declares does not support it.

It looks like the problem is confirmed here but no one has publicly looked into why.

[defunctio]

I've re-written the ec2_ami_copy module to use boto3 and have successfully tested it with encrypted volumes. I need to finish it up and I'll issue a PR to ansible. I've added the following task list to keep track of the progress.

Additionally it seems Canonical only makes _snapshots_ of the latest AMIs public, so #140 actually ends up being a prerequisite for encrypted volumes since we need access to the snapshot for CopyImage.

• [x] Re-write ec2_amI_copy ansible module to use boto3
• [x] Issue a PR to ansible https://github.com/ansible/ansible/pull/20009
• [x] Merged into dev
• [x] Released (will be released in ansible 2.3)
• [x] #140 Resolved

Note: Encrypted AMIs will need to be tagged so that we can check for their existence and use them in future deployments without creating duplicates.

[jackivanov]

@defunctio In my experience the ansible team will not approve a PR that includes majority changes with modules (eg: boto->boto3).
Encrypted AMIs are tagged.

[defunctio]

@gunph1ld that's unfortunate, I guess we'll see. If not we can do as you said and include it within Algo

[aioue]

@defunctio I'd very much like to use your boto3 module if available, please. I absolutely can't shell out and do this via the CLI.

[defunctio]

@aioue sure, you can find the functional but unfinished module here: https://github.com/trailofbits/algo/blob/ec2encryption/ec2_ami_copy.py

I'll push the finished module when I get back to a computer.

[aioue]

Thank you very much @defunctio, I'll patiently for the finished one! 😄

[patmcd]

The t2 instance family does not actually support encrypted root/boot volumes via KMS.

(You can test this yourself by launching an instance in this family manually vs say a c4 instance).

[defunctio]

http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/EBSEncryption.html#EBSEncryption_supported_instances

According to Amazon it is unless I'm mistaken.

[patmcd]

I dug into this a bit more and my information was dated, you are correct. Previously, the smaller t2.* instances sizes did not support it at the root volume level. Now they all do.

[defunctio]

@aioue Apologies for the delay I've been away on holiday but you can find the finished module here https://github.com/trailofbits/algo/blob/ec2encryption/library/ec2_ami_copy.py

[aioue]

Thank you!