Terraform-provider-aws: aws_autoscaling_schedule: implicitly-set start_time isn't re-calculated when you change the recurrence
_This issue was originally opened by @glasser as hashicorp/terraform#14023. It was migrated here as part of the provider split. The original body of the issue is below._
Terraform Version
0.9.3 (latest available on Homebrew; the relevant code in aws_autoscaling_schedule has not changed on HEAD).
Affected Resource(s)
- aws_autoscaling_schedule
Terraform Configuration Files
resource "aws_launch_configuration" "foobar" {
name = "tfbug-lc"
image_id = "ami-21f78e11"
instance_type = "t1.micro"
}
resource "aws_autoscaling_group" "foobar" {
availability_zones = ["us-west-2a"]
name = "tfbug-asg"
max_size = 0
min_size = 0
launch_configuration = "${aws_launch_configuration.foobar.name}"
}
resource "aws_autoscaling_schedule" "foobar" {
scheduled_action_name = "tfbug-sched"
autoscaling_group_name = "${aws_autoscaling_group.foobar.name}"
min_size = 0
max_size = 0
desired_capacity = 0
recurrence = "0 10 * * SAT"
}
Steps to Reproduce
Please list the steps required to reproduce the issue, for example:
terraform apply- Examine the StartTime of the created schedule
- Edit the recurrence; eg
perl -pi -e 's/0 10/0 8/' foo.tfto change it from 10:00 to 08:00 terraform apply- Examine the StartTime again
Expected Behavior
The implicitly-created StartTime should reflect the change to the Recurrence.
Actual Behavior
glasser@glasser-lyrid-2 1 /tmp/tfb $ AWS_DEFAULT_REGION=us-west-2 AWS_PROFILE=dev /usr/local/bin/terraform apply
aws_launch_configuration.foobar: Creating...
associate_public_ip_address: "" => "false"
ebs_block_device.#: "" => "<computed>"
ebs_optimized: "" => "<computed>"
enable_monitoring: "" => "true"
image_id: "" => "ami-21f78e11"
instance_type: "" => "t1.micro"
key_name: "" => "<computed>"
name: "" => "tfbug-lc"
root_block_device.#: "" => "<computed>"
aws_launch_configuration.foobar: Creation complete (ID: tfbug-lc)
aws_autoscaling_group.foobar: Creating...
arn: "" => "<computed>"
availability_zones.#: "" => "1"
availability_zones.2487133097: "" => "us-west-2a"
default_cooldown: "" => "<computed>"
desired_capacity: "" => "<computed>"
force_delete: "" => "false"
health_check_grace_period: "" => "300"
health_check_type: "" => "<computed>"
launch_configuration: "" => "tfbug-lc"
load_balancers.#: "" => "<computed>"
max_size: "" => "0"
metrics_granularity: "" => "1Minute"
min_size: "" => "0"
name: "" => "tfbug-asg"
protect_from_scale_in: "" => "false"
target_group_arns.#: "" => "<computed>"
vpc_zone_identifier.#: "" => "<computed>"
wait_for_capacity_timeout: "" => "10m"
aws_autoscaling_group.foobar: Creation complete (ID: tfbug-asg)
aws_autoscaling_schedule.foobar: Creating...
arn: "" => "<computed>"
autoscaling_group_name: "" => "tfbug-asg"
desired_capacity: "" => "0"
end_time: "" => "<computed>"
max_size: "" => "0"
min_size: "" => "0"
recurrence: "" => "0 10 * * SAT"
scheduled_action_name: "" => "tfbug-sched"
start_time: "" => "<computed>"
aws_autoscaling_schedule.foobar: Creation complete (ID: tfbug-sched)
Apply complete! Resources: 3 added, 0 changed, 0 destroyed.
The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.
State path:
glasser@glasser-lyrid-2 0 /tmp/tfb $ aws --region=us-west-2 --profile=dev autoscaling describe-scheduled-actions --scheduled-action-names=tfbug-sched
{
"ScheduledUpdateGroupActions": [
{
"MinSize": 0,
"DesiredCapacity": 0,
"AutoScalingGroupName": "tfbug-asg",
"MaxSize": 0,
"Recurrence": "0 10 * * SAT",
"ScheduledActionARN": "arn:aws:autoscaling:us-west-2:753487546527:scheduledUpdateGroupAction:7ff24fb4-f375-493e-94bb-9ebfc92fde67:autoScalingGroupName/tfbug-asg:scheduledActionName/tfbug-sched",
"ScheduledActionName": "tfbug-sched",
"StartTime": "2017-04-29T10:00:00Z",
"Time": "2017-04-29T10:00:00Z"
}
]
}
glasser@glasser-lyrid-2 0 /tmp/tfb $ perl -pi -e 's/0 10/0 8/' foo.tf
glasser@glasser-lyrid-2 0 /tmp/tfb $ AWS_DEFAULT_REGION=us-west-2 AWS_PROFILE=dev /usr/local/bin/terraform apply
aws_launch_configuration.foobar: Refreshing state... (ID: tfbug-lc)
aws_autoscaling_group.foobar: Refreshing state... (ID: tfbug-asg)
aws_autoscaling_schedule.foobar: Refreshing state... (ID: tfbug-sched)
aws_autoscaling_schedule.foobar: Modifying... (ID: tfbug-sched)
recurrence: "0 10 * * SAT" => "0 8 * * SAT"
aws_autoscaling_schedule.foobar: Modifications complete (ID: tfbug-sched)
Apply complete! Resources: 0 added, 1 changed, 0 destroyed.
The state of your infrastructure has been saved to the path
below. This state is required to modify and destroy your
infrastructure, so keep it safe. To inspect the complete state
use the `terraform show` command.
State path:
glasser@glasser-lyrid-2 0 /tmp/tfb $ aws --region=us-west-2 --profile=dev autoscaling describe-scheduled-actions --scheduled-action-names=tfbug-sched
{
"ScheduledUpdateGroupActions": [
{
"MinSize": 0,
"DesiredCapacity": 0,
"AutoScalingGroupName": "tfbug-asg",
"MaxSize": 0,
"Recurrence": "0 8 * * SAT",
"ScheduledActionARN": "arn:aws:autoscaling:us-west-2:753487546527:scheduledUpdateGroupAction:7ff24fb4-f375-493e-94bb-9ebfc92fde67:autoScalingGroupName/tfbug-asg:scheduledActionName/tfbug-sched",
"ScheduledActionName": "tfbug-sched",
"StartTime": "2017-04-29T10:00:00Z",
"Time": "2017-04-29T10:00:00Z"
}
]
}
The underlying PutScheduledUpdateGroupAction API has many optional parameters. For example, it's legitimate to specify just a Recurrence and no StartTime, in which case the StartTime will be specified based on the Recurrence.
Now let's say you change the Recurrence on an existing schedule. Since you didn't specify a start_time in your TF config file that should imply that you want to once again recalculate StartTime based on that Recurrence, and that's exactly what AWS does if you only specify Recurrence. But Terraform reads the StartTime into the start_time property in the Read callback and then passes it directly back to AWS (you can see this in the debug log below if you search for PutScheduledUpdateGroupAction — there's a StartTime in that URL). AWS interprets that as meaning "run this action next at the start time I was told, then recalculate based on the schedule".
In practice this means that if you change your cronjob from "run at 10 AM" to "run at 8 AM", it'll still run at 10 AM exactly once before updating to running at 8 AM the next day. Or if you change it from 8 AM to 10 AM, it'll run twice the first day (8 AM then 10 AM). This is super confusing.
I think that Terraform should never send a StartTime to AWS unless there's actually a start_time in the config file. I'm not actually sure how to implement this, though.
Debug Output
https://gist.github.com/glasser/0711dba33b6ed0c02e2158660287a129
(This is from the second run which mutates the action.)
hashibot
All 9 comments
:+1: :up:
BAlmeidaS
on 26 Sep 2017
+1
bhchew
on 15 Jan 2018
Maybe scheduled_action_name_prefix could be add and with the lifecycle create_before_destroy a new shcedule action will always be created and the StartTime will always be recalculated ?
What do you think ?
gboucherie
on 4 Apr 2018
Still having issues with this using the latest aws provider and terraform versions.
jseiser
on 13 Aug 2018
+1
itsnagaraj
on 28 Feb 2019
This is a blocker for me to use aws_autoscaling_schedule - I would need to destroy the resources before every re-run of the code when testing. Terraform also doesn't seem to handle converting local times to UTC.
Easier to use AWS CLI combined with some local time conversion code.
rdonkin
on 22 Aug 2019
I am using following workaround
resource "aws_autoscaling_schedule" "core_autoscaling_scheduler" {
scheduled_action_name = "terraform-demo-${lower("${random_id.random_id_autoscaling_scheduler.id}")}"
min_size = "${var.min_size}"
max_size = "${var.max_size}"
desired_capacity = "${var.desired_capacity}"
recurrence = "${var.recurrence}"
autoscaling_group_name = "${aws_autoscaling_group.asg.name}"
}
resource "random_id" "random_id_autoscaling_scheduler" {
keepers = {
asg_scheduler_action_id = "${var.recurrence}"
}
byte_length = 8
}
Every-time we change the recurrence variable it will destroy and build the resource again.
nitinda
on 4 Sep 2019
Bumping for visibility. Just hit that bug myself today. Please fix :)
arabek
on 1 Apr 2020
Bumb[2]. Had a cluster running all night because of this. Please fix!
vaghetti
on 20 May 2020
Related issues
ccslamstack
·
3Comments
hashibot
·
3Comments
carmas
·
3Comments
hashibot
·
3Comments
hashibot
·
3Comments
Most helpful comment
Maybe scheduled_action_name_prefix could be add and with the lifecycle create_before_destroy a new shcedule action will always be created and the StartTime will always be recalculated ?
What do you think ?