Terraform-provider-aws: Error: rpc error: code = Canceled desc = context canceled

[artem-tomyuk]

hi,

Terraform version: 0.12.8

• provider.archive v1.2.2
• provider.aws v2.11.0
• provider.null v2.1.2
• provider.random v2.1.2
• provider.template v2.1.2

this configuration generates errors during 'terraform plan' command:

resource "aws_alb" "alb" {
name = "${var.alb_name}"
subnets = "${flatten(var.alb_subnets)}"
security_groups = ["${var.lb_type == "application" ? "dwfwefwefe" : null}"]
internal = "${var.lb_internal}"
load_balancer_type = "${var.lb_type}"
}

[artem-tomyuk]

2019/09/12 11:02:20 [TRACE] Re-validating config for "module.alb_ecs_api_net.aws_alb.alb"
2019/09/12 11:02:20 [TRACE] GRPCProvider: ValidateResourceTypeConfig
2019-09-12T11:02:20.083+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: data.template_file.user_data_mon: Refreshing state...
data.template_file.user_data_rabbitmq: Refreshing state...

Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Unavailable desc = transport is closing



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled



Error: rpc error: code = Canceled desc = context canceled


panic: reflect: call of reflect.Value.Type on zero Value

[artem-tomyuk]

2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: 2019/09/12 11:02:20 [DEBUG] [aws-sdk-go] DEBUG: Response elasticloadbalancing/DescribeLoadBalancers Details:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: ---[ RESPONSE ]--------------------------------------
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: HTTP/1.1 400 Bad Request
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Connection: close
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Content-Length: 271
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Content-Type: text/xml
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Date: Thu, 12 Sep 2019 08:02:19 GMT
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: X-Amzn-Requestid: a50a881e-d533-11e9-8725-611784950610
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: -----------------------------------------------------
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: 2019/09/12 11:02:20 [DEBUG] [aws-sdk-go]
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Sender
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Throttling
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: Rate exceeded
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: a50a881e-d533-11e9-8725-611784950610
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: 2019/09/12 11:02:20 [DEBUG] [aws-sdk-go] DEBUG: Validate Response elasticloadbalancing/DescribeLoadBalancers failed, not retrying, error Throttling: Rate exceeded
2019-09-12T11:02:20.070+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: status code: 400, request id: a50a881e-d533-11e9-8725-611784950610

[artem-tomyuk]

i've tried with -parallelism=1 but error is the same
Error: rpc error: code = Canceled desc = context canceled
aws support recommends to implement Exponential Backoff and retries in tf code.

[artem-tomyuk]

also there is panic stack trace from log:
panic: reflect: call of reflect.Value.Type on zero Value
2019-09-12T11:02:20.083+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4:
2019-09-12T11:02:20.083+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: goroutine 6904 [running]:
2019-09-12T11:02:20.083+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: reflect.Value.Type(0x0, 0x0, 0x0, 0x10, 0xc000b07098)
2019-09-12T11:02:20.083+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/goenv/versions/1.12.2/src/reflect/value.go:1813 +0x169
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validatePrimitive(0xc0005877a0, 0xc0004b8500, 0x11, 0x0, 0x0, 0xc0005c2000, 0xc000ea1560, 0x1, 0x4635c20, 0xc000a2ddc0, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1706 +0xc8
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validateType(0xc0005877a0, 0xc0004b8500, 0x11, 0x0, 0x0, 0xc0005c2000, 0xc000ea1560, 0x10085dc, 0xc00001e000, 0x4c6f800, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1778 +0x56f
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validateList(0xc0005877a0, 0x53ffac3, 0xf, 0x4635c20, 0xc000a2ddc0, 0xc0005c2e00, 0xc000ea1560, 0x0, 0x4a99f60, 0xc000ea1590, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1507 +0x86c
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validateType(0xc0005877a0, 0x53ffac3, 0xf, 0x4635c20, 0xc000a2ddc0, 0xc0005c2e00, 0xc000ea1560, 0x0, 0x0, 0x0, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1774 +0x9e
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validate(0xc0005877a0, 0x53ffac3, 0xf, 0xc0005c2e00, 0xc000ea1560, 0x0, 0x0, 0x0, 0x0, 0x0, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1416 +0x21d
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.validateObject(0xc0005877a0, 0x0, 0x0, 0xc0005877a0, 0xc000ea1560, 0x53e8eb7, 0xc000707248, 0x7, 0xc000332601, 0x6b00000000000000, ...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:1673 +0x1c6
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.schemaMap.Validate(...)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/schema.go:705
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(Resource).Validate(0xc0005b5080, 0xc000ea1560, 0xc000707248, 0x7, 0xc0005e4198, 0x1, 0x40, 0x0)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/resource.go:368 +0x5e
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema.(Provider).ValidateResource(0xc0005d6480, 0xc000707248, 0x7, 0xc000ea1560, 0xc0008d9d70, 0xc000ea1560, 0xc000619ad8, 0x4a99f60, 0xc000ea1500, 0x0)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/schema/provider.go:242 +0x1d8
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/plugin.(GRPCProviderServer).ValidateResourceTypeConfig(0xc0000b4b18, 0x5c18dc0, 0xc000b859b0, 0xc000c78ac0, 0xc0000b4b18, 0xc000b859b0, 0xc0008a3bd0)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/helper/plugin/grpc_provider.go:226 +0x218
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/internal/tfplugin5._Provider_ValidateResourceTypeConfig_Handler(0x5302460, 0xc0000b4b18, 0x5c18dc0, 0xc000b859b0, 0xc000927720, 0x0, 0x5c18dc0, 0xc000b859b0, 0xc000f741a0, 0x195)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/github.com/hashicorp/terraform/internal/tfplugin5/tfplugin5.pb.go:2911 +0x23e
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc.(Server).processUnaryRPC(0xc000085200, 0x5c388e0, 0xc000001b00, 0xc000cbe800, 0xc0005cec90, 0x9775bd0, 0x0, 0x0, 0x0)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc/server.go:966 +0x470
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc.(Server).handleStream(0xc000085200, 0x5c388e0, 0xc000001b00, 0xc000cbe800, 0x0)
2019-09-12T11:02:20.084+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc/server.go:1245 +0xd25
2019-09-12T11:02:20.085+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc.(Server).serveStreams.func1.1(0xc000036120, 0xc000085200, 0x5c388e0, 0xc000001b00, 0xc000cbe800)
2019-09-12T11:02:20.085+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc/server.go:685 +0x9f
2019-09-12T11:02:20.085+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: created by github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
2019-09-12T11:02:20.085+0300 [DEBUG] plugin.terraform-provider-aws_v2.11.0_x4: /opt/teamcity-agent/work/5d79fe75d4460a2f/src/github.com/terraform-providers/terraform-provider-aws/vendor/google.golang.org/grpc/server.go:683 +0xa1

[artem-tomyuk]

@evanphx can you help please....?

[evanphx]

Do not tag random people.

[bflad]

Hi @artem-tomyuk 👋 Sorry you are running into trouble here.

If I recall correctly, the reflect: call of reflect.Value.Type on zero Value panic was fixed in more recent versions of the Terraform AWS Provider (after version 2.23.0 or so). Can you please try upgrading your provider and seeing if that resolves the issue or provide an updated panic output if it does not? Thanks.

[bflad]

Closing due to lack of response. If you are still having issues on recent versions of Terraform CLI (0.12+) and the Terraform AWS Provider, please create a new issue and we will take a fresh look. Thanks.

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you feel this issue should be reopened, we encourage creating a new issue linking back to this one for added context. Thanks!