Terraform-provider-aws: Can't create RDS from snapshot when using shared subnet

[namliz]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment

Terraform Version

Terraform v0.11.14
provider.aws v2.10.0-v2.18.0

Terraform Configuration Files

resource "aws_db_instance" "default" {
  // allocated_storage      = "20"
  // engine                 = "mysql"
  // engine_version         = "5.6"
  identifier             = "main"
  snapshot_identifier    = "dev-baseline-auto"
  instance_class         = "db.t2.small"
  skip_final_snapshot    = true
  storage_encrypted      = false
  publicly_accessible    = false

  vpc_security_group_ids = ["${aws_security_group.rds.id}"]
  db_subnet_group_name   = "${aws_db_subnet_group.rds.name}"
}

Debug Output

2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: 2019/07/10 16:15:14 [DEBUG] DB Instance restore from snapshot configuration: {
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   AutoMinorVersionUpgrade: true,
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   CopyTagsToSnapshot: false,
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   DBInstanceClass: "db.t2.small",
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   DBInstanceIdentifier: "main",
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   DBSnapshotIdentifier: "dev-baseline-auto",
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   DBSubnetGroupName: "rds_main",
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   DeletionProtection: false,
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   Engine: "mysql",
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   PubliclyAccessible: false,
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   Tags: []
2019-07-10T16:15:14.775+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: }
2019-07-10T16:15:14.776+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: 2019/07/10 16:15:14 [DEBUG] [aws-sdk-go] DEBUG: Request rds/RestoreDBInstanceFromDBSnapshot Details:

2019-07-10T16:15:14.776+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: Action=RestoreDBInstanceFromDBSnapshot&AutoMinorVersionUpgrade=true&CopyTagsToSnapshot=false&DBInstanceClass=db.t2.small&DBInstanceIdentifier=main&DBSnapshotIdentifier=dev-baseline-auto&DBSubnetGroupName=rds_main&DeletionProtection=false&Engine=mysql&PubliclyAccessible=false&Tags=&Version=2014-10-31

2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: 2019/07/10 16:15:16 [DEBUG] [aws-sdk-go] <ErrorResponse xmlns="http://rds.amazonaws.com/doc/2014-10-31/">
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   <Error>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:     <Type>Sender</Type>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:     <Code>InvalidParameterValue</Code>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:     <Message>The specified VPC vpc-064827fe944738b6a is a shared VPC, please explicitly provide an EC2 security group.</Message>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   </Error>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4:   <RequestId>5768d5de-65a5-4c6c-8e8b-9e8e52ba9c13</RequestId>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: </ErrorResponse>
2019-07-10T16:15:16.385+0200 [DEBUG] plugin.terraform-provider-aws_v2.14.0_x4: 2019/07/10 16:15:16 [DEBUG] [aws-sdk-go] DEBUG: Validate Response rds/RestoreDBInstanceFromDBSnapshot failed, not retrying, error InvalidParameterValue: 

Expected Behavior

An RDS instance should have been created from the specified snapshot.

Actual Behavior

Failed to create.

Steps to Reproduce

Create a VPC and three subnets in account A
Share subnets with account B (via AWS RAM)

Create Subnet Group (aws_db_subnet_group) referencing the three shared subnets in account B
Create Security Group in account B

Create RDS from snapshot in account B - Fails
Create RDS in account B without referencing a snapshot - Works

Important Factoids

This is a multi-account setup using shared VPC resources via AWS RAM (https://docs.aws.amazon.com/ram/latest/userguide/getting-started-shared.html).

Temporary Workaround

Adding provider = "aws.vpc" and creating the aws_security_group, aws_db_subnet_group, and aws_db_instance in account A (the account where the VPC was created and the subnets are shared _from_) works.

All of that should be equally possible in account B and this is rather suboptimal.

Additional Info

Creating the RDS from snapshot in account B by hand works as expected, so we can establish this is possible.

In the debug logs the requests generated are identical when attempting to create the resources in account B or A but with A it simply succeeds with no errors.

[deuscapturus]

Ran into the same issue.

[mvillumsen]

This is still an issue in version 2.33.0 of the AWS provider (using Terraform v0.12.10) when using the terraform-aws-rds-aurora module (i.e. when creating an aws_rds_cluster_instance).

[Schinjo]

+1 when using terraform-aws-modules/rds/aws
version = "2.5.0"

[bailantilles7]

+1 when using terraform-aws-modules/rds/aws or the rds_instance resource

[ijaveed]

+1 Terraform v0.11.7 Are there any timelines when this will be fixed?

[msmagoo87]

+1 I just ran in to this issue myself.
Terraform v0.12.13 + provider.aws v2.31.0

Is there a fix in the works?

[jamengual]

please fix :)

[BenLeibig]

I don't understand the posted work around. You can't access security groups in other accounts with shared vpc. Has anyone figured out a way to work.

I manually created the database, then imported it into terraform without the snapshot set, and was able to create the security group, but we kind of need this to be repeatable for test environments...

[ps-pyoung]

My team and I are still affected by this issue. Is there any additional information we can provide to aid in identifying the root cause that will lead to a fix?

[bflad]

The fix for restoring RDS Database Instance snapshots into RAM shared Subnets has been merged and will release with version 2.55.0 of the Terraform AWS Provider, later this week. Thanks to @daniel-linhart for the fix. 👍

[hashibot[bot]]

This has been released in version 2.55.0 of the Terraform AWS provider. Please see the Terraform documentation on provider versioning or reach out if you need any assistance upgrading.

For further feature requests or bug reports with this functionality, please create a new GitHub issue following the template for triage. Thanks!

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!