Atlantis: Release lock on no-op plans?

[mwarkentin]

Hi, I tried searching but didn't come across anything for this (#89 might be closest).

We have certain updates that we make where we can sometimes trigger a large number of plans across many projects. Sometimes these plans end up being "no-ops" (eg. "No changes. Infrastructure is up-to-date.") but Atlantis still stores the plan and locks the project.

This means we can run into some nasty contention that requires hopping back and forth and deleting plans or closing a PR temporarily to release locks. Would there be an issue with detecting a no-op plan and releasing the lock (or don't take the lock in the first place)?

[lkysow]

Hey Michael, sorry for the late response. I think this is something that's worth tackling but it's not super easy. We'd have to create a new representation of a no-op plan in atlantis because right now we just have a successful plan or an unsuccessful plan.

In addition, when you update an output variable, this generates a no-op plan that still needs to be applied to have the output in the state. We'd want a way for users to still apply these operations.

Would it be possible for you to describe how your TF is set up such that it generates a lot of no-op plans? This is just so I can understand how different people have their TF set up.

[nikovirtala]

In one of our deployment models each workspace is updated in separate pull request, and one application can have five (or more) workspaces. Thus one PR results one plan with changes and four no-op plans. If and when change is applied to all workspaces it results five plans with changes and 20 no-op plans.

[mwarkentin]

@lkysow Our terraform configs are broken down by environment and then smaller pieces for each application, each with their own state:

 -- app1
 -- app2
 -- ...
 -- app30

env2
 -- app1
 -- app2
 -- ...
 -- app30

...

env5
 -- app1
 -- app2
 -- ...
 -- app30

Within each env, there is also a shared.tf file which is symlinked into each of the application directories in order to reduce duplication (as we have >150 terraform states). Currently we re-plan all of our apps if there is a change to that shared.tf file which then can causes a cascade of no-op plans for a bunch of apps at times. If two developers are making changes to this file on separate branches, there are huge numbers of lock conflicts even if the plans themselves are "noops".

We used to re-plan all apps on any changes to the root environment *.tf however this was quickly disabled as it caused constant lock conflicts (again, even if they were all noops).

[mwarkentin]

In addition, when you update an output variable, this generates a no-op plan that still needs to be applied to have the output in the state. We'd want a way for users to still apply these operations.

@lkysow this actually just came up on our team - generally when doing this from the CLI we use terraform refresh to update the outputs without having to plan / apply - any thoughts on a atlantis refresh command as an alternative?

[lkysow]

In addition, when you update an output variable, this generates a no-op plan that still needs to be applied to have the output in the state. We'd want a way for users to still apply these operations.

@lkysow this actually just came up on our team - generally when doing this from the CLI we use terraform refresh to update the outputs without having to plan / apply - any thoughts on a atlantis refresh command as an alternative?

That's definitely a possible solution if we were to deal with no-op plans differently.

[majormoses]

I actually sometimes intentionally create no-op plans to create a lock before I start attempting to do state modifications such as running import, mv, or rm operations between one or more states. I certainly see the point but I generally prefer to apply even no-op plans to ensure the state is written to it since it's clearer from an auditing perspective what happened and why. That being said I can certainly see the value.

[mwarkentin]

@majormoses seems like a use case for a atlantis lock -p <project> command :)

[chadasapp]

Hey Michael, sorry for the late response. I think this is something that's worth tackling but it's not super easy. We'd have to create a new representation of a no-op plan in atlantis because right now we just have a successful plan or an unsuccessful plan.

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

[kipkoan]

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

I just checked, and this has no effect when an output changes, and therefore needs to be applied.

I do consider this to be a Terraform bug, though. Outputs are part of the state. If those change, then the plan needs to be applied. I'm not sure if v0.12 does this differently, but if not, we should open a bug report.

[chadasapp]

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

I just checked, and this has no effect when an output changes, and therefore needs to be applied.

So it returns 0? That's annoying to say the least.

I do consider this to be a Terraform bug, though. Outputs are part of the state. If those change, then the plan needs to be applied. I'm not sure if v0.12 does this differently, but if not, we should open a bug report.

I think here's our bug.

[kipkoan]

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

I just checked, and this has no effect when an output changes, and therefore needs to be applied.

So it returns 0? That's annoying to say the least.

Yes. :(

I do consider this to be a Terraform bug, though. Outputs are part of the state. If those change, then the plan needs to be applied. I'm not sure if v0.12 does this differently, but if not, we should open a bug report.

I think here's our bug.

I upvoted. 👍

[lkysow]

Hey Michael, sorry for the late response. I think this is something that's worth tackling but it's not super easy. We'd have to create a new representation of a no-op plan in atlantis because right now we just have a successful plan or an unsuccessful plan.

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

I meant that the Atlantis codebase has no internal representation of a no-op plan. It would require a sizeable refactor and also require thinking through a lot of edge cases since by the time Atlantis figures out that the plan is a no-op the rest of the code is expecting a success/failure result.

[nikovirtala]

Hey Michael, sorry for the late response. I think this is something that's worth tackling but it's not super easy. We'd have to create a new representation of a no-op plan in atlantis because right now we just have a successful plan or an unsuccessful plan.

terraform plan has -detailed-exitcode: 0 is no diff, 1 is error, 2 is ok-with-diff. Is this not usable?

I meant that the Atlantis codebase has no internal representation of a no-op plan. It would require a sizeable refactor and also require thinking through a lot of edge cases since by the time Atlantis figures out that the plan is a no-op the rest of the code is expecting a success/failure result.

That refactoring could make sense if we could be guaranteed that there are no changes in the plan ? - Currently that is not the case since a change to outputs is not considered as a change in the Terraform plan.