Terraform-provider-aws: Investigate Upstream AppMesh API Changes

[bflad]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Potentially Affected Resource(s)

• aws_appmesh_mesh
• aws_appmesh_route
• aws_appmesh_virtual_node
• aws_appmesh_virtual_router

Description

AWS has publicly announced potentially breaking changes to the AppMesh service API: https://github.com/awslabs/aws-app-mesh-examples/issues/92

We will need to investigate the proposed upstream changes and determine a path forward for the existing Terraform AWS Provider resources.

[bcelenza]

I'd be happy to pull request the required changes for AWS App Mesh.

I would also love to understand how this change impacts current users of the Terraform integration with App Mesh, and will keep an eye on this issue during the comment period.

[ewbankkit]

We should probably start with a note in the documentation stating that the aws_appmesh_virtual_node and aws_appmesh_virtual_router resources will change in an upcoming release and link to the AWS GH issue.

[ewbankkit]

@bcelenza I'm totally happy for you to do the PR/collaborate as necessary.

[bcelenza]

Thanks @ewbankkit! Opened a PR for the doc changes.

[ewbankkit]

I added some comments to the announcement.

My thoughts:

• Add a new aws_appmesh_virtual_service resource (with terraform import functionality as Amazon have already created these)
• Modify the aws_appmesh_virtual_node and aws_appmesh_virtual_router resources with the relevant changes. State migration should be straightforward

My only concerns are around the short deprecation notice. I don't think we can follow the published best practice of doing a major release just for this and I'm not sure how this fits in with the provider _v2.0.0_ release timetable, but App Mesh is a preview service so there really should be reduced expectations around API stability.

[bcelenza]

Agree with your thoughts. I've read through the best practices, hoping to confirm my assumptions below.

Using VirtualNode as an example of a resource:

• spec.serviceDiscovery.dns.serviceName to hostname appears to be supported via Renaming a Required Attribute?
• spec.backends does not have a clear migration path given the structure change. Does Terraform have a mechanism for this sort of migration? It would likely be safe to assume that a string passed would be interpreted as a VirtualService name.

Alternatively, we could just update the resources to reflect the new APIs, and anyone on the current released integration will need to drop and recreate their App Mesh resources with the new structures. This is where my concern for existing adoption comes in to play.

[ewbankkit]

@bcelenza Terraform has robust state migration functionality so we should be able to rewrite state files created with the initial API version into a format that reflects the new API version:

• service_name -> hostname
• backends changes from list of virtual_services which are themselves a simple map with a single virtual_service_name string element

We should be able to find an suitable example in the code.

[ewbankkit]

Additional proposed API changes:

• https://github.com/awslabs/aws-app-mesh-examples/issues/94

[ewbankkit]

@bcelenza Is there a newer API model JSON with the latest proposed changes?

[bcelenza]

@ewbankkit Not yet -- we're still waiting for the last pieces of VirtualRouter listeners to reach production before we distribute an updated JSON. ETA Tuesday of next week, if I had to guess.

[bcelenza]

Updated Go SDK is now available as v1.17.13.

[ewbankkit]

@bcelenza I am working right now on a draft PR based on the original app-mesh-2019-01-25.trial.json model changes. I'll rebase it on the new AWS SDK.

[ewbankkit]

Will do, just wanted to get my current WIP pushed: https://github.com/terraform-providers/terraform-provider-aws/pull/7858.

[ewbankkit]

@bcelenza To clarify dependencies (I saw some error messages during testing):

• If a virtual service has a virtual node or virtual router provider then that virtual node or virtual router cannot be deleted before virtual service is deleted, correct?
• The dependency graph is then
  
  
  
  • _Mesh_ => _Virtual Node_ => _Virtual Service_ or
  
  
  • _Mesh_ => _Virtual Router_ => _Virtual Service_
  
  

[bcelenza]

@bcelenza To clarify dependencies (I saw some error messages during testing):

• If a virtual service has a virtual node or virtual router provider then that virtual node or virtual router cannot be deleted before virtual service is deleted, correct?

• The dependency graph is then

  • _Mesh_ => _Virtual Node_ => _Virtual Service_ or
  • _Mesh_ => _Virtual Router_ => _Virtual Service_

That's correct, the Virtual Service must be deleted before it's provider (VirtualNode or VirtualRouter). The thinking behind this is that the same actor in an organization likely owns both the Virtual Service and its providers, so it is there to protect them from creating an unintended outage of their own service.

[bflad]

These changes will go out in version 2.3.0 of the Terraform AWS Provider middle of next week unless we decide to release this as a 3.0.0 due to the breaking changes.

In the future, we may decide to relegate Public Preview and Beta APIs to separate Terraform AWS Provider(s) to prevent this situation in the future.

[bflad]

This has been released in version 2.3.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!