Aws-app-mesh-roadmap: Envoy configuration valid for 7 days after VirtualNode is deleted

[bcelenza]

Summary

When deleting a VirtualNode in a Mesh, the resulting Envoy configuration for that VirtualNode will remain available to an Envoy which identifies itself as that VirtualNode name (e.g. mesh/my-mesh/virtualNode/my-virtual-node). Envoys which are connected to the Envoy Management Service endpoint identified as that VirtualNode will remain connected and may receive improper configuration.

_Note:_ Other Envoys identified as separate VirtualNodes, who may have previously relied on the deleted VirtualNode as part of a backend definition, will be updated with the correct configuration.

The period that this configuration is available after deleting a VirtualNode is approximately 7 days. We are working to reduce this time.

Steps to reproduce
_Scenario 1: A connected Envoy remains connected after deletion of the VirtualNode_

1. Create a VirtualNode
2. Connect an Envoy which is identified as that VirtualNode
3. Delete the VirtualNode

Expected behavior: The Envoy no longer receives configuration updates and is disconnected from the Envoy Management Service endpoint.
Actual behavior: The Envoy remains connected and may receive improper configuration.

_Scenario 2: An Envoy connects after deletion of the VirtualNode__

1. Create a VirtualNode
2. Connect an Envoy which is identified as that VirtualNode and ensure it's working properly
3. Disconnect the Envoy
4. Delete the VirtualNode
5. Reconnect the Envoy

Expected behavior: The Envoy is not allowed to connect to the Envoy Management Service, and receives an appropriate error code (e.g. NOT_FOUND)
Actual behavior: The Envoy remains connected and may receive improper configuration.

Work-around

Make sure your Envoys are disconnected, and the associated ECS tasks, EKS pods, or applications running on EC2 are not serving traffic, then delete the VirtualNode.

[dastbe]

We've made changes to lower this validity period from ~7 days down to ~5 minutes during normal operation.