Terraform: support for variable validation fails in a module that could be fully compatible with 0.13 and 0.12

[mariux]

Terraform Version

Terraform v0.13.0-beta1

and

Terraform v0.12.24

Terraform Configuration Files

terraform {
  required_version = ">= 0.12.20, < 0.14.0"

  experiments = [variable_validation]
}

Debug Output

terraform-0.13.0-beta1 plan

Error: Experiment has concluded

  on ../../versions.tf line 4, in terraform:
   4:   experiments = [variable_validation]

Experiment "variable_validation" is no longer available. Custom variable
validation can now be used by default, without enabling an experiment.

Expected Behavior

The concluded experiment should not block writing modules that support versions 0.12 and 0.13 of Terraform as the experiment is now part of the feature set.

Actual Behavior

• Enabling the experiment supports 0.12 terraform only.
• Disabling the experiment supports 0.13 terraform only.

Steps to Reproduce

• add validation to your variables.
• add validation experiment setting to your config and run successfully with Terraform 0.12
• remove the experiment setting and run successfully with Terraform 0.13
• add the experiment setting back in and fail running Terraform 0.13

Additional Context

I totally understand that the experiment is concluded. But from a UX point of view, it does not make sense to split the development of a module into two branches where the only difference is the setting of the experiment when no other breaking features are used.

My suggestion would be to just ignore the concluded experiment as the intention of enabling it is getting the support of validation. Alternatively, also conclude the experiment in Terraform 0.12 branch. Or add a setting to force ignore it in 0.13 branch.

If there is already a solution in place, just let me know. I tried to do my homework and searched for similar issues and solutions but this was not successful.

We would love to use variable validation in a module that supports terraform 0.12 and 0.13 at the same time. Thank you!

[apparentlymart]

Hi @mariux,

The intent of this error message is expressly to _prevent_ a never-ending accumulation of concluded experiments in every module, and the corresponding "experimental feature is in effect" warnings when they are used on older versions.

While I acknowledge that means it isn't possible to use new features in modules intended for older versions of Terraform, I would not expect any _released_ module intending to work with Terraform 0.12 to use it because released modules should generally not activate any experiments. Instead, as with all new language features, I suggest to wait to use this feature until you are ready to make your module require the version of Terraform where that feature was introduced.

[mariux]

Hi @apparentlymart,

Thank you for your reply. As we are developing modules for various customers some of them might not be able to upgrade to terraform 0.13 right away for whatever reasons. We already tested our modules with both versions and do not see any problems.

We will for sure at some point deprecate our support for 0.12.

Of course, we want to use as many new features as possible while keeping the overhead to maintain different branches low.

Is there any chance the experiment will be also concluded in the 0.12 branch and made available for 0.12? As far as I can see this would not mean any backward-incompatible change and could so be easily added to 0.12?

I guess other module developers could also make use of this as upgrading the IaC stacks on the user side to terraform 0.13 might take some time, once it is released.

Thanks for the great work and we would also be happy to help make this happen. Just let us know what we could do.

Best, Marius from @mineiros-io

[alexjurkiewicz]

I understand you don't want to accumulate old experiments forever. And that you don't want to guarantee behaviour around experiments so they don't lead to cruft for you as maintainer.

My suggestion is to soft-deprecate the experiment for one release before totally removing it. For instance:

• Terraform 0.12: works as currently
• Terraform 0.13: Raise a warning: "Warning: variable_validation experiment has concluded, please remove it from your config before 0.14."
• Terraform 0.14: Raise an error "Unknown experiment variable_validation".

This will allow smoother transition for people testing experiments. It increases the ease of testing experiments in situations of eg parallel deployment, where the same codebase is deployed in 0.12 and 0.13 environment (eg as part of preparing for 0.13 upgrade). I think this will lead to greater usage and therefore value from your experiments!

[alexjurkiewicz]

I wrote an example patch here to show my proposal: alexjurkiewicz/terraform@94649505c5c59e02e4e641e542b77db4559b7b0f

I think a possible objection is: "deprecation works when the experiment is accepted as-is. But what about the case where an experimental feature is removed or modified between the last experiment and next stable?".

There are two approaches:

1. Use the above patch as-is. The user will see a warning. The user is responsible for updating their code as required, this is part of the user's contract for using an experiment.

2. Concluded experiments can register whether they are blocking or non-blocking. In the case of a successful experiment like variable_validation, they follow the above proposed workflow (warning for one version, then removed). In the case of a failed experiment, they follow the currently-implemented workflow (error immediately).

As above, the goal is to simplify the user experience in trying experiments, so they are as broadly tested as possible.

[mariux]

My suggestion is to soft-deprecate the experiment for one release before totally removing it. For instance:

* Terraform 0.12: works as currently
* Terraform 0.13: Raise a warning: "Warning: `variable_validation` experiment has concluded, please remove it from your config before 0.14."
* Terraform 0.14: Raise an error "Unknown experiment `variable_validation`".

That would work for us, too. We are interested in using the actual feature to work around other issues we face. This would have a side effect on the user as a warning might be confusing.

So even when implementing the suggestion of @alexjurkiewicz, I would still vote to accept the experiment also in the 0.12.x version of Terraform as variable_validation is an added functionality not causing any backward incompatibilities. Thank you.