Terraform: Support for Replication Groups for Elasticache

[zoltrain]

Amazon have support for Mutli-AZ deployments for Elasticache nodes, including Redis v2.8.6+.

Meaning you get automatic failover, via the master/slave replication via slave promotion.

http://docs.aws.amazon.com/AmazonElastiCache/latest/APIReference/API_CreateReplicationGroup.html

Currently according to the docs it isn't possible to create one of these via terraform, would be great if this was available for the next release.

[stack72]

@zoltrain just a FYI, I have started this and will finish it in a day or 2 - #4363

[zoltrain]

Amazing! HA Redis FTW

[aldarund]

Need this, any updates on it?

[wolfspyre]

+1 for this!

[yogin]

Need this also, any updates/ETA?

[Felivel]

Any news on this?

[iwashi]

+1!

[hatt]

+1

[sarkis]

+1

[chrisburrell]

+1 - any updates on this?

[vasiliyb]

+1

[bogus-py]

+1

[anttis]

+1

[bender-the-greatest]

+1

[kpumuk]

+1

[adammeghji]

+1

[jfromaniello]

:+1:

[gdxhsw]

+1

[mboret]

+1

[bfallik]

Wow, I can't tell if these +1s are trolling or not. Do the terraform developers actually use this information to prioritize work or are we all just spamming each other uselessly?

[mtougeron]

@catsby Any chance this is somewhere on Hashicorp's radar for implementation? Not being able to set a replication group is preventing us from being able to use Terraform to deploy our redis clusters in AWS.

sorry to ping you directly, I just know you've worked on some of the AWS providers :)

[Dchamard]

+1

[n8gard]

Also very much hoping to see this. We need multi-AZ Redis capabilities and it looks like we'll have to roll our own machines rather than using ElastiCache. :(

[arozwalak]

+1

[boxrick]

Maybe this can help someone?

resource "aws_cloudformation_stack" "default_elasticache" {
name = "${var.product}-${var.redis_dns}-${var.environment}-${var.region}-replgrp"
on_failure = "DELETE"
template_body = < {
"Resources" : {
"${var.product}${var.redis_dns}replgrp": {
"Type" : "AWS::ElastiCache::ReplicationGroup",
"Properties" : {
"AutomaticFailoverEnabled" : "true",
"AutoMinorVersionUpgrade" : "true",
"CacheNodeType" : "${var.reddis_instance_size}",
"CacheSubnetGroupName" : "${aws_elasticache_subnet_group.default_reddis_sg.name}",
"Engine" : "redis",
"NumCacheClusters" : "2",
"Port" : "6379",
"PreferredCacheClusterAZs" : ["eu-west-1a", "eu-west-1b"],
"ReplicationGroupDescription" : "${aws_elasticache_subnet_group.default_reddis_sg.name}",
"SecurityGroupIds" : ["${aws_security_group.default.id}"],
"PreferredMaintenanceWindow" : "wed:09:25-wed:22:30",
"SnapshotRetentionLimit" : "5",
"SnapshotWindow" : "03:30-05:30"
}
}
},
"Outputs" : {
"endpoint" : {
"Description": "Redis Cluster Endpoint",
"Value" : { "Fn::GetAtt" : [ "${var.product}${var.redis_dns}replgrp", "PrimaryEndPoint.Address"]}
}
}
}
STACK
}

Then I get the cache primary cluster endpoint using IAM roles on the specific box and this command. Basically I pull in the redis_cache cluster description and get the replication group and use that to find the endpoint and finally override my config.

redis_description=$(/usr/bin/curl -fs http://169.254.169.254/latest/user-data | jq -r '.host_variables.redis_description')
redis_endpoint=$(/usr/local/bin/aws --region eu-west-1 elasticache describe-replication-groups | jq -r ".ReplicationGroups[] | select(.Description == \"${redis_description}\") | .NodeGroups[].PrimaryEndpoint.Address")

[ricoli]

this is very much needed :)

[vshimoda]

+1

[hSATAC]

+1

[aq1018]

It's been 8 month and nothing...? Please add support!

[ajlanghorn]

+1

[hehachris]

+1

[MrPink]

+2

[Farhie]

+1

[guyon]

+1

[AndrewFarley]

Just saw 0.7 rc2 go out without this feature yesterday. Redundancy is quite important for a production-level architecture using Redis. I'm use non-redundancy at the moment via terraform to begin engineering on top of ElastiCache, but I hope to get this added soon. Any way we can bump the priority of this?

[CaseyLabs]

Any news on this? @mitchellh - we have a production roll out in the pipeline that I would love to use Terraform for, and Replication Groups for Elasticache would be a piece that we need.

[roblofthouse]

I'm shocked this isn't in here. +1 on getting this feature out. Currently working on a production rollout of some new systems and this is a huge blocker.

[MrPink]

Just a tip - We have been using the Terraform CloudFormation provider to build our Elasticache clusters in production and it works fine for the time being

[AndrewFarley]

@MrPink Yea I figured that is possible but didn't spend the time to do that. Mind gist-ing some example code to save some of us some time until this is implemented in Terraform?

[petestorey26]

+1

[brianbenns]

+1

[MrPink]

Here's the Gist for creating a Replication group with Cloudformation and controlling it with Terraform https://gist.github.com/MrPink/393840d460c0cb1caa5074d3fafd5657 - Unfortunately I have to modify it after with aws-cli to enable snapshots as its currently not possible with CF.

[jgruhl]

+1

[FlorinAndrei]

This is a blocker. It should be moved higher in the priority list.

[stack72]

Finally got back to doing the work here - #8275 should fix it up!

[stack72]

Closed via #8275

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.