Terraform-provider-aws: Add support for Enhanced Networking to aws_instance resource.

[hashibot]

_This issue was originally opened by @kwilczynski as hashicorp/terraform#7815. It was migrated here as part of the provider split. The original body of the issue is below._

---

Amazon at the moment supports two flavours of Enhanced Networking feature:

Overview and documentation.

• SR-IOV (Intel 82599 VF)
  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/sriov-networking.html
• Elastic Network Adapter (ENA)
  http://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html

For any VPC-enabled and HVM-compatible EC2 instances.

We should allow users to enable Enhanced Network support and choose which flavour they would like to use. A very common use case, would be to run and span a Docker overlay network layer across multiple instances participating as hosts in the Docker Swarm. Such nodes do benefit from more linear and consistent performance which Enhanced Networking be bring, especially when latency is concerned and momentary spikes in such and "choking up" network throughput (e.g. exhausting the packer per-second rate, etc.) is not desired.

For this to work an AMI (Amazon Machine Image) needs to have moderately up-to-date Linux kernel and include supported version of both the ixgbevf[1] and ena[2] kernel driver, thus it would be prudent to to include latest drives and to enable support for the Enhanced Networking when creating (with Packer, etc.) custom AMIs for both the EBS and Instance Store type.

The Elastic Network Adapter is quite new (see the announcement) and it is not yet supported on Windows due to lack of available drivers for this platform (an on-going development effort by Amazon), plus the only instance type that supports ENA at the moment is X1, but hopefully Amazon would add ENA support to others later.

Worth noting is, that the Amazon Linux HVM-enabled AMIs already ship with both of the drivers included and have Enhanced Networking support enabled, thus are ready to use from the get-go.

API documentation:

• https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_DescribeInstanceAttribute.html
• https://docs.aws.amazon.com/AWSEC2/latest/APIReference/API_ModifyInstanceAttribute.html

Conceptually, this very similar to the way how the "SourceDestCheck" check is now supported in Terraform, as per: resource_aws_instance.go#L595-L603

Related to https://github.com/mitchellh/packer/issues/3688 in Packer.

1. Intel 82599 VF (SR-IOV) drivers.
2. Amazon Elastic Network Adapter (ENA) drivers.

[h4wkmoon]

It seems ENA is mandatory for m5 instances.

[tavisma]

Looks to be required for C5 as well

[tata2000]

Any expectations on that one?

[swibowo]

👍 on this feature, please.

Per AWS documentation (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking.html), here is the list of instances that use ENA driver for enhanced networking (definitely much more than just X1 instance as mentioned in this issue's original description).

C5, F1, G3, H1, I3, M5, P2, P3, R4, X1, and m4.16xlarge instances use the Elastic Network Adapter for enhanced networking.

I think it is desirable to have the ena-support flag configurable on AMI (aws_ami) and EC2 instance (aws_instance) resources.

[jmvbxx]

Has there been an update on this? Or, is the idea to just wait it out until ENA is automatically enabled on all instances?

[tavisma]

So the process to enable ENA is a bit odd, it doesn't seem like there is an API call to enable support on an AMI. The current process (verified with Amazon) has you booting up an instance, making sure the necessary drivers are installed, modifying the instance to support ENA with an API call and creating an AMI from that instance

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/enhanced-networking-ena.html

[jmvbxx]

Correct. The appropriate kernel has to be used and then and awscli enable
command and then restart. My solution was a quick Ansible role to make the
kernel adjustments and then manually enable. Luckily I didn't have many
older instances that didn't come preenabled.

On Jan 30, 2018 12:42, "Tavis" [e-mail] wrote:

So the process to enable ENA is a bit odd, it doesn't seem like there is
an API call to enable support on an AMI. The current process (verified with
Amazon) has you booting up an instance, making sure the necessary drivers
are installed, modifying the instance to support ENA with an API call and
creating an AMI from that instance

https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/
enhanced-networking-ena.html

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
https://github.com/terraform-providers/terraform-provider-aws/issues/206#issuecomment-361673882,
or mute the thread
https://github.com/notifications/unsubscribe-auth/AJryY-9SIrO-Rpo7_35FVuY33dXXIKZbks5tP1RvgaJpZM4N41Lk
.

[raddy]

Would love this included!

[paolo-rendano]

Yes, I would also love that. What about launch configurations with autoscaling? I would expect to be able to set this also there. is it?

[keien]

There is also an EnaSupport field for ec2.Image (doc) aka terraform aws_ami which would be lovely to have supported as well - newer instance types like m5's won't let you launch from an AMI without this enabled, which is currently what happens when you use terraform to manage your AMIs.

[bflad]

The aws_ami resource will have a new ena_support argument in version 1.30.0 of the AWS provider, releasing later today. 👍

[samm-git]

@bflad we got a problem due to that PR. After provider upgrade plan decided to re-create all ami-s because ena_support in defaults was not matching ena_support from aws. I dont think its a right behavior and could be very dangerous.

[bflad]

Hi @samm-git 👋 Sorry you are running into unexpected behavior. I'd suggest opening a new bug report issue filling out all the details if you feel its a bug so its not sending a notification to everyone following this separate feature request issue. 👍

[samm-git]

Hi, thank you for reply. @bflad already did in HashCorp bugtracker, so probably you will be contacted soon )

[CpuID]

Just to clarify - is this only on aws_ami and not aws_instance right now...?

[ianomad]

Would be nice to get an update on this..

[tavisma]

We ended up using Packer to create customised images (which is useful for a host of other reasons)

[pdomagala]

Any update on this? M5 instances were introduced in 2017...