Terraform-provider-aws: Terraform can't create RDS cluster instance

[leodamasceno]

Terraform version

➜  git terraform -v
Terraform v0.10.8

Terraform files:

• rds.tf:

resource "aws_rds_cluster_instance" "rds_cluster_instances" {
  engine               = "aurora-postgresql"
  count                = "${var.rds_instances_count}"
  identifier           = "${var.rds_cluster_name}-${count.index}"
  cluster_identifier   = "${aws_rds_cluster.rds_cluster.id}"
  instance_class       = "${var.rds_instance_type}"
}

resource "aws_db_subnet_group" "rds_private_db_subnet" {
  name        = "db-private-subnet"
  description = "Private subnets for RDS instance"
  subnet_ids  = [ "${aws_subnet.prod-private-subnet1.id}", "${aws_subnet.prod-private-subnet2.id}" ]
}

resource "aws_rds_cluster" "rds_cluster" {
  engine                  = "aurora-postgresql"
  cluster_identifier      = "${var.rds_cluster_name}"
  availability_zones      = "${var.rds_cluster_az}"
  vpc_security_group_ids  = ["${aws_security_group.application-rds-sg.id}"]
  database_name           = "${var.rds_db_name}"
  master_username         = "${var.rds_admin_user}"
  master_password         = "${var.rds_admin_pw}"
  db_subnet_group_name    = "${aws_db_subnet_group.rds_private_db_subnet.name}"
}

• vars.tf:

variable "rds_instances_count" {
  default = 1
}

variable "rds_cluster_name" {
  default = "application-db-cluster"
}
variable "rds_instance_type" {
  default = "db.t2.micro"
}

variable "rds_cluster_az" {
  default = ["us-east-1a", "us-east-1b"]
}

variable "rds_db_name" {
  default = "application_db"
}

variable "rds_admin_user" {
  default = "app_master"
}

variable "rds_admin_pw" {
  default = "ADMIN_PW"
}

Error:

Error: Error applying plan:

1 error(s) occurred:

* aws_rds_cluster_instance.rds_cluster_instances: 1 error(s) occurred:

* aws_rds_cluster_instance.rds_cluster_instances: InvalidParameterCombination: RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.micro, Engine=aurora-postgresql, EngineVersion=9.6.3, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see the documentation.
    status code: 400, request id: 9828fa34-bfa9-11e7-b50b-eb623c35da4b

Terraform does not automatically rollback in the face of errors.
Instead, your Terraform state file has been partially updated with
any resources that successfully completed. Please address the error
above and apply again to incrementally change your infrastructure.

It doesn't matter the instance type, I've tested it with t2.micro, t2.small, t2.medium, m4.large but it gives me the same error. I'm able to create an instance via AWS console so I can't understand why it complains about the combination.

[jpSimkins]

Error seems pretty clear:

RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.micro, Engine=aurora-postgresql, EngineVersion=9.6.3, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see the documentation.

That is from AWS so not a terraform issue.

It seems for a aurora-postgresql setup, the instance sizes available are:

• db.r4.large
• db.r4.xlarge
• db.r4.2xlarge
• db.r4.4xlarge
• db.r4.8xlarge
• db.r4.16xlarge

When building the terraform system, walk through the AWS system to ensure you don't miss any restraints.

[damascenorakuten]

Hello @jpSimkins, thanks for your reply.

I'm able to create an EC2 instance (t2.micro) using aurora-postgresql via AWS Console but I can't do the same via via Terraform. That's what I'm trying to understand.

[jpSimkins]

I was pointing out the error is from AWS so a limitation seems plausible.

Hmm, there is a difference between creating an EC2 instance running a DB vs creating a RDS cluster; which is what your response indicates to me. How are you creating it VIA aws-cli? That would help clear some things up.

[leodamasceno]

I'm sorry @jpSimkins I actually meant RDS instance, not EC2 instance. Here's how I can create a RDS instance using the AWS CLI:

aws rds create-db-instance --db-instance-identifier pine-production --allocated-storage 1 --db-instance-class db.t2.small  --engine postgres --master-username test --master-user-password test12345678 --allocated-storage 10

[jpSimkins]

Ok, so the issue here is that your aws-cli is a single instance and not a cluster. Your terraform code is a cluster so it won't work.

To create a cluster in aws-cli you would need to run create-db-cluster

To see the same issue in aws, run:

aws rds create-db-instance --db-instance-identifier pine-production2 --allocated-storage 1 --db-instance-class db.t2.small  --engine aurora-postgresql --master-username test --master-user-password test12345678 --allocated-storage 10

Which is giving the same error you are seeing in terraform.

An error occurred (InvalidParameterCombination) when calling the CreateDBInstance operation: RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.small, Engine=aurora-postgresql, EngineVersion=9.6.3, LicenseModel=postgresql-license. For supported combinations of instance class and database engine version, see thedocumentation.

Note the difference of the engine: aurora-postgresql (terraform) and postgres (aws-cli).

Terrafrom won't allow you to use postgres as a cluster (perhaps aws limitation?)

Error: aws_rds_cluster.rds_cluster: "engine" contains an invalid engine type "postgres". Valid types are either "aurora" or "aurora-postgresql".

I'm not sure of a work-a-round yet to get postgres working. To use aurora-postgresql you need the DB sizes I posted above.

This could be desired considering a small instance like that isn't necessary for a cluster as you could simply update the instance size to handle the load.

You could probably just use aws_db_instance instead of the cluster. I think this is more than likely what you need

[damascenorakuten]

The plan is actually to have a cluster. I've noticed what you said about the engine, I believe that's necessary for people using PostgreSQL. Can we tag it as a bug or improvement?

Thanks

[bflad]

Looking at the conversation above from a few months ago, this seems to be related to confusion between using Aurora and non-Aurora for PostgreSQL. The two versions have different hardware requirements.

The original error about cluster instance sizing was generated by the RDS API when attempting to use an unsupported instance size. Due to the complexity of attempting plan-time validation (and its required maintenance when its updated upstream), its very unlikely that we would implement validation for this type of problem.

If someone feels inclined to note what (if any) documentation updates might be needed here, we can take a look, but this issue is fairly stale as-is and doesn't clarify a definition of done, so I'm going to close it. 👍

[gomker]

Stumbled upon this issue while working on updating old code. The docs are listing the wrong instance sizes https://www.terraform.io/docs/providers/aws/r/rds_cluster_instance.html

Documentation says "Aurora" but that is Postgresql RDS : https://aws.amazon.com/rds/postgresql/pricing/

https://aws.amazon.com/rds/aurora/pricing/

Aurora only supports r4 types it seems

I would recommend updating the docs and list RDS supported vs. Aurora

[anisjonischkeit]

Had a lot of trouble finding this but Aurora does actually support more than just r4 types. This seems to be the list of instance types that work https://docs.amazonaws.cn/en_us/AmazonRDS/latest/AuroraUserGuide/Concepts.DBInstanceClass.html so the smallest instance for mysql seems to be a db.t2.small and a db.t3.medium for postgres

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!