Dependabot-core: Add support for Terraform 0.12 / HCL2

[marocchino]

Add support for Terraform 0.12 and HCL2 syntax. Dependabot currently fails with a cryptic error message when HCL2 syntax is parsed:

Dependabot couldn't parse the main.tf found at `/terraform/manifests/xxx/main.tf`.

You can mention @dependabot in the comments below to contact the Dependabot team.

[feelepxyz]

@marocchino are you by any chance using Terraform 0.12? We currently don't support it and don't have a good way of detecting the version being used 😢

[marocchino]

@feelepxyz
hmm. Actually, our project is using Terraform 0.12. and, Our main.tf contains below setting:

terraform {
  required_version = ">= 0.12.0"
  backend "local" {}
}

There seems to be no way to see the log, I'll take the core and run it locally and tell the result.

[marocchino]

Sorry for the newbie question. But, How can I point a private repo?
Solved it.

[marocchino]

It looks like we need to send additional parameters for terraform.

1

$ bin/dry-run.rb terraform vega-corporation/solomon-gamigin --dir=terraform/manifests/vpc

warning: parser/current is loading parser/ruby26, which recognizes

warning: 2.6.4-compliant syntax, but you are running 2.6.2.

warning: please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.

=> fetching dependency files

=> dumping fetched dependency files: ./dry-run/vega-corporation/solomon-gamigin/terraform/manifests/vpc

Initialized empty Git repository in /home/dependabot/dependabot-core/dry-run/vega-corporation/solomon-gamigin/terraform/manifests/vpc/.git/


* Please tell me who you are.

Run

git config --global user.email "[email protected]"

  git config --global user.name "Your Name"

to set your account's default identity.

Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'dependabot@21be5d37e14a.(none)')

=> parsing dependency files

Traceback (most recent call last):

    12: from bin/dry-run.rb:283:in <main>'
    11: from bin/dry-run.rb:185:incached_read'

    10: from bin/dry-run.rb:283:in block in <main>'
     9: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:inparse'

     8: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in each'
     7: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:inblock in parse'

     6: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:222:in parsed_file'
     5: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:34:inin_a_temporary_directory'

     4: from /usr/lib/ruby/2.6.0/tmpdir.rb:93:in mktmpdir'
     3: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:inblock in in_a_temporary_directory'

     2: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in chdir'
     1: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:inblock (2 levels) in in_a_temporary_directory'

/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:231:in block in parsed_file': unable to parse HCL: At 35:26: Unknown token: 35:26 IDENT var.vpc_name (Dependabot::SharedHelpers::HelperSubprocessFailed)
    7: from bin/dry-run.rb:283:in
'

    6: from bin/dry-run.rb:185:in cached_read'
    5: from bin/dry-run.rb:283:inblock in 
'

    4: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in parse'
    3: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:ineach'

    2: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in block in parse'
    1: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:219:inparsed_file'

/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:245:in `rescue in parsed_file': unable to parse HCL: At 35:26: Unknown token: 35:26 IDENT var.vpc_name (Dependabot::DependencyFileNotParseable)

    

2

$ bin/dry-run.rb terraform vega-corporation/solomon-gamigin --dir=terraform/manifests/eks

warning: parser/current is loading parser/ruby26, which recognizes

warning: 2.6.4-compliant syntax, but you are running 2.6.2.

warning: please see https://github.com/whitequark/parser#compatibility-with-ruby-mri.

=> fetching dependency files

=> dumping fetched dependency files: ./dry-run/vega-corporation/solomon-gamigin/terraform/manifests/eks

Reinitialized existing Git repository in /home/dependabot/dependabot-core/dry-run/vega-corporation/solomon-gamigin/terraform/manifests/eks/.git/


* Please tell me who you are.

Run

git config --global user.email "[email protected]"

  git config --global user.name "Your Name"

to set your account's default identity.

Omit --global to set the identity only in this repository.

fatal: unable to auto-detect email address (got 'dependabot@21be5d37e14a.(none)')

=> parsing dependency files

Traceback (most recent call last):

    12: from bin/dry-run.rb:283:in <main>'
    11: from bin/dry-run.rb:185:incached_read'

    10: from bin/dry-run.rb:283:in block in <main>'
     9: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:inparse'

     8: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in each'
     7: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:inblock in parse'

     6: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:222:in parsed_file'
     5: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:34:inin_a_temporary_directory'

     4: from /usr/lib/ruby/2.6.0/tmpdir.rb:93:in mktmpdir'
     3: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:inblock in in_a_temporary_directory'

     2: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:in chdir'
     1: from /home/dependabot/dependabot-core/common/lib/dependabot/shared_helpers.rb:37:inblock (2 levels) in in_a_temporary_directory'

/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:231:in block in parsed_file': unable to parse HCL: At 36:31: Unknown token: 36:31 IDENT data.terraform_remote_state.vpc.outputs.cluster_name (Dependabot::SharedHelpers::HelperSubprocessFailed)
    7: from bin/dry-run.rb:283:in
'

    6: from bin/dry-run.rb:185:in cached_read'
    5: from bin/dry-run.rb:283:inblock in 
'

    4: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:in parse'
    3: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:24:ineach'

    2: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:25:in block in parse'
    1: from /home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:219:inparsed_file'

/home/dependabot/dependabot-core/terraform/lib/dependabot/terraform/file_parser.rb:245:in `rescue in parsed_file': unable to parse HCL: At 36:31: Unknown token: 36:31 IDENT data.terraform_remote_state.vpc.outputs.cluster_name (Dependabot::DependencyFileNotParseable)

    

We keep the tfvar and tfstate file in a separate directory and use environment variables to determine which environment values ​​to read.

$ terraform plan -state "../../data/<module_name>/<env_name>.tfstate" \
  -var-file="../../data/<env_name>.tfvars" \
  -var-file="../../data/<module_name>/common.tfvars" \
  -var-file="../../data/<module_name>/<env_name>.tfvars"

[feelepxyz]

terraform {
  required_version = ">= 0.12.0"
  backend "local" {}
}

Ah nice one, we should be able to use this to return a better error. We put supporting terraform 0.12.0 on pause as we couldn't find a good way to parse the new HCL syntax. Looks like this is still experimental https://github.com/hashicorp/hcl2

[rebelagentm]

@feelepxyz I saw this PR (https://github.com/dependabot/dependabot-core/pull/1388) came in earlier today. Will this address the issue here?

[feelepxyz]

@rebelagentm yes possibly! @hmarr would love your 👀 on the above PR when you get a mo.

[edahlseng]

HCL 2 is no longer experimental! https://github.com/hashicorp/hcl/wiki/Version-Selection

[hazcod]

Copying from other issue:

Issue: https://github.com/ironPeakServices/infrastructure/issues/17
File in question: https://github.com/ironPeakServices/infrastructure/blob/master/main.tf

Terraform version:

terraform {
    required_version = ">= v0.12.19"
}

[infin8x]

Duplicate of #1176