Dependabot-core: Terraform 0.13 support

[greysteil]

Dependabot's terraform support doesn't work with HCL 2.0.

In particular, our logic for parsing HCL files is here and shells out to this tool which only support HCL 1.0 and, unfortunately, looks unmaintained.

To fix this we need a HCL -> soemthing-readable-by-ruby parser to replace the one we're currently using.

[greysteil]

@hmarr would love your eyes on this if there's an easy fix. There seems to be almost no support for HCL in Ruby which is how we ended up using a Go project that converts HCL to JSON. 😢

[hmarr]

Yep, I'll sort something out 👍

[hmarr]

Upgrading to HCL 2 turns out to be surprisingly difficult. I've tried several different approaches, but none of them have worked so far. The closest I've got is using github.com/hashicorp/terraform/config as a library, but that breaks Terragrunt support. Parsing HCL 2 seems to require a full specification of the HCL's structure, and I can't find one for Terraform's HCL that includes Terragrunt. We could start maintaining one, but that would be a fair amount of work.

I'm going to park this for now as we have several other high priority issues to address, and come back to it in a bit.

[domcleal]

Parsing HCL 2 seems to require a full specification of the HCL's structure, and I can't find one for Terraform's HCL that includes Terragrunt. We could start maintaining one, but that would be a fair amount of work.

Terragrunt itself doesn't support 0.12 yet either and they're making some changes over at https://github.com/gruntwork-io/terragrunt/pull/731 which may be relevant. The syntax in the current Terragrunt config file is known not to be HCL2 compatible, so they're moving it into a terragrunt.hcl file which will be HCL2 compatible.

Edit: Terragrunt 0.19 was released with these incompatible changes: migration guide

[wata727]

terraform-config-inspect might be a good replacement in the case.

[lorengordon]

@hmarr terragrunt now works with terraform 0.12. But, I think I'd suggest keeping the current parser as a fallback, if possible. That way the HCL1 syntax still works, either terraform or terragrunt.

[evenh]

Some alternatives to look into:

• https://github.com/tmccombs/hcl2json
• https://github.com/PetrusHahol/pyhcl2

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs within seven days. Thank you for your contributions.

[scottybrisbane]

Bump, this is an important issue to resolve.

[johncblandii]

Can we get this worked out soon? It makes our repos noisy and renders Dependabot, a tool we really like, useless for those specific TF 0.12 repos.

[tbugfinder]

This tool is very close compare to terraform-config-inspect: https://github.com/amplify-education/python-hcl2

[jakauppila]

Until an HCL2.0 -> JSON library is available, would it be feasible to add support for Terraform 0.12 configurations that are expressed as JSON?

[sheldonhull]

Idea on possible fix: specifying an exact file to match.

• Scenario: I place all my provider versions into versions.tf
• Dependabot directory filters to only *versions.tf
• Versions submitted for bump now is only against this versions.tf without problem of parsing the remaining files.

This actual requirement to me seems like it might be the fix needed to support HCL2 without more complicated parsing requirements for an entire new HCL2 format in the remaining content in a folder.

[patrickjahns]

@sheldonhull
There is already a pull request from the community at https://github.com/dependabot/dependabot-core/pull/1388

I've taken the liberty to convert it into a github action for the time-being - hopefully this is useful for anyone else as well https://github.com/marketplace/actions/terraform-dependabot

[sheldonhull]

I tried the scenario above and it doesn't work as expects a directory, not file path.
@patrickjahns EPIC! thanks for this. I am totally checking this out for sure!
Another great case for github actions!

[osterman]

Perhaps rename title now to Terraform 0.13 =)

[evenh]

There has been over a year since the last official response and it seems important to many users of Dependabot. Could we please get a rough time estimate on this issue @greysteil & @hmarr? Is there anything the community could help out with to speed up the process?

[greysteil]

I don't work on Dependabot anymore (I'm working on GitHub's code scanning features) but @infin8x is the product manager responsible here.

[bcomnes]

Even improving the error message would be an improvement (e.g. dependabot doesn't support terraform 0.12 or newer!). But also just adding support for the newer versions.

[estraph]

Hi @infin8x is a fix for this on the roadmap? I checked the public roadmap but couldn't find one.

[infin8x]

Hi @estraph - it is not, at present. We understand and are tracking the ask but don't have a timeline.

[tbugfinder]

My workaround is using https://github.com/renovatebot/renovate

[masutaka]

My workaround is using tfupdate (CircleCI orb)

[lorengordon]

We're using the github action based on the dependabot pr, https://github.com/dependabot/dependabot-core/pull/1388#issuecomment-610322918

[bogaertg]

Some news about this feature ?
Terraform with dependabot is useless without HCL 2 support.

[davidski]

This seems to be on the roadmap now for this quarter (Q1 2021)! 💥

https://github.com/github/roadmap/issues/150