Terraform: Unlocking state

[victorspringer]

Hello, I am new at terraform. So please don't mind if my doubt is dumb but I didn't find the answer anywhere.
I need to unlock my configuration state (it is allocated at dynamodb). I tried the force-unlock command but I don't know what is/how to get the LockID. I tried to pass my configuration directory but I get "Failed to unlock state: failed to retrieve lock info: unexpected end of JSON input" as response.
I also get this response when I try this ID from the lock error:

Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed
    status code: 400, request id: IGUA4N3QNLOPSE5GHO6SH88EPVVV4KQNSO5AEMVJF66Q9ASUAAJG
Lock Info:
  ID:        40e844a6-ebb0-fcfd-da85-5cce5bd1ec89

Could anyone give me a clue?

[wchrisjohnson]

I'm getting this message as well.

[apparentlymart]

I'm not sure what exactly is going on here, but if all else fails, and you are sure there are no concurrent Terraform runs, you can look in the DynamoDB table and delete the corresponding record to clear the lock manually.

[victorspringer]

Yes, the solution was to do it manually
Thanks!

[apparentlymart]

Hi all,

It's dissatisfying to not reach an explanation for this, but it doesn't seem like we'll be able to repro and debug how this occurs with the information here, so I'm going to close this for now with the workaround of manually deleting the lock. Hopefully one day we can get to the bottom of the cause here, but it seems like it's some sort of edge-case and so it'll take some work to figure out exactly how to trigger it, so I'm going to close this for now until we are able to gather more information to work from.

Thanks for reporting it, and sorry there wasn't a smoother outcome here yet.

[unixhat]

Getting this message as well.

[CraigCottingham]

As am I. macOS 10.13.3, on a APFS-formatted volume.

$ terraform --version
Terraform v0.11.2
+ provider.aws v1.8.0

[CraigCottingham]

Update: It looks like the lock file is not being deleted when there's an error when running plan or apply (such as an error in a provider -- for example, just now I tried creating an S3 bucket with a name containing an underscore instead of a hyphen).

[dperetti]

It will also happen if you simply hit ctrl-C after a command.
We need a way to clear the lock with a terraform command !

[jvendryes]

This worked for me: https://www.terraform.io/docs/commands/force-unlock.html

[szwed]

Same issue here:

/usr/local/bin/terraform_0.11.3 force-unlock -force 0ade9cc2-ed43-1855-6084-d21abbd32bb1
Failed to unlock state: failed to retrieve lock info: unexpected end of JSON input

[Laxman-SM]

this work for me without any issue

terraform --version
Terraform v0.11.6

/usr/local/bin/terraform force-unlock 08bdeb97-8ba3-c4f4-08af-70ce6c63ec39
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may be still be in use. Only 'yes' will be accepted to confirm.

Enter a value: yes

Terraform state has been successfully unlocked!

The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.

Process finished with exit code 0

[kanawaden]

terraform force-unlock

This command gave me "Local state cannot be unlocked by another process" message. :(

[Laxman-SM]

Hi,

Can you please share your config and execution output ?

Regards,
Laxman singh

Get Outlook for Androidhttps://aka.ms/ghei36

---

From: kanawaden [e-mail]
Sent: Monday, May 28, 2018 12:05:11 PM
To: hashicorp/terraform
Cc: Laxman Singh; Comment
Subject: Re: [hashicorp/terraform] Unlocking state (#14513)

terraform force-unlock

This command gave me "Local state cannot be unlocked by another process" message. :(

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHubhttps://github.com/hashicorp/terraform/issues/14513#issuecomment-392432706, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AKHFng56UasY4w8lb8oLLASRRuVAux1tks5t25qfgaJpZM4NbiG3.

[dmcfranklin]

I was able to reproduce by hitting CTRL+Z while a plan operation was in progress.
For me, using force-unlock worked.

Lock Info:
ID: bcf8815f-53db-ad60-45d5-488829a83e5d
Path: ches-chc-prod-consumerengagement/terraform/dev/terraform.tfstate
Operation: OperationTypePlan
Who: dmceathron@x=x
Version: 0.11.7
Created: 2018-05-29 18:47:36.157851325 +0000 UTC
Info:

x=x:dev dmceathron$ terraform force-unlock bcf8815f-53db-ad60-45d5-488829a83e5d
Do you really want to force-unlock?
Terraform will remove the lock on the remote state.
This will allow local Terraform commands to modify this state, even though it
may be still be in use. Only 'yes' will be accepted to confirm.

Enter a value: yes

Terraform state has been successfully unlocked!

The state has been unlocked, and Terraform commands should now be able to
obtain a new lock on the remote state.

[gabnou]

same here after a crashed plan

[akhiljalan]

Same here, and force unlock gives

Local state cannot be unlocked by another process

[ddcdannyd]

I found that I need this flag -lock=false

[vibhuyadav]

vyadav@VYADAV-LT MINGW64 /c/infra-engineering-tools/terraform/ci
$ terraform refresh -lock=false

Error: Error loading state: open terraform.tfstate: Access is denied.

I want to delete the file but I can't. It let me create it in first place. Looks like Terraform is a virus.

vyadav@VYADAV-LT MINGW64 /c/infra-engineering-tools/terraform/ci
$ terraform state rm terraform.tfstate
Failed to load state: open terraform.tfstate: Access is denied.

vyadav@VYADAV-LT MINGW64 /c/infra-engineering-tools/terraform/ci
$ terraform state rm terraform.tfstate -force
Failed to load state: open terraform.tfstate: Access is denied.

[mattlawnz]

This occured for me under these conditions:

1) state file in AWS. I had access to this.
2) Terraforming a bucket in GCP. I did not have access to this when I did the plan

I ran the plan, after about 1 minute it came up with this error:

Error releasing the state lock!

Error message: RequestError: send request failed
caused by: Post https://dynamodb.ap-southeast-2.amazonaws.com/: dial tcp 52.94.13.64:443: i/o timeout
Lock Info:
ID: c5bf827f-XXXXX-77bab
Path: REDACTED
Operation: OperationTypePlan
Who: @jenkins-slave-3pn5c-2t2bw
Version: 0.11.8
Created: 2018-11-21 02:36:10.075141881 +0000 UTC
Info:

Terraform acquires a lock when accessing your state to prevent others
running Terraform to potentially modify the state at the same time. An
error occurred while releasing this lock. This could mean that the lock
did or did not release properly. If the lock didn't release properly,
Terraform may not be able to run future commands since it'll appear as if
the lock is held.

In this scenario, please call the "force-unlock" command to unlock the
state manually. This is a very dangerous operation since if it is done
erroneously it could result in two people modifying state at the same time.
Only call this command if you're certain that the unlock above failed and
that no one else is holding a lock.

Error: Error running plan: 1 error(s) occurred

[davehowell]

@mattlawnz That error looks like DynamoDB timed out. Perhaps you went over the provisioned writes limit for DynamoDB, or perhaps some other issue occurred causing DynamoDB the API Post to DynamoDB endpoint. Unless this is consistently reproducible? If it is then it could be a terraform issue with how it unwinds the lock. I'd rule out the first option though, given your error message explicitly mentions "caused by: Post ... dynamodb endpoint ... i/o timeout"

[chenrui333]

i had the issue with terragrunt plan-all, but got it fixed with:

• terragrunt plan-all -lock=false

And rerun terragrunt plan-all did not report issue anymore.

[kenorb]

I had this issue with aws_ses_domain_identity_verification, it was taking over 20 minutes (due to high timeout value: -lock-timeout), I had to stop it (Control-Z, kill %1), then when I tried to re-run, I got the _Error locking state_.

Running terraform force-unlock -force ID solved the problem.

[vinzzhang]

For anyone who get the output:

Local state cannot be unlocked by another process

after tried -force-unlock.
It means that you run terraform locally.
To address this issue: open the task manager / activity monitor and filter terraform and force exit all terraform related threads.

[AleksandarTokarev]

@kenorb 's solution fixed my problem ! Thanks a lot

[rastakajakwanna]

Faced too, today. It was most probably caused by impatient colleague who canceled some action before it unlocked the state.
I've compared DynamoDB content with another remote state and I found that healthy (and non-locked) DynamoDB has only *.tfstate-md5 records.
I was sure that *.tfstate (lock), with no "Digest" key, was not supposed to be there and force-unlock won't help with this state. After deleting *..tfstate record, all went well again.

I haven't tried -force though, but that would require checking the last state ID in remote tfstate file, not everybody knows where to find it.

[btgblink]

I had this problem (I cancelled a terraform plan that I ran locally because it was taking too long). Deleting all of the local .terraform directories allowed me to remove the lock.

[aryannava]

rename your terraform.tfstate file. Looks like it got corrupted
I got an alike problem and I renamed the file

Error: Error locking state: Error acquiring the state lock: open terraform.tfstate: permission denied

Terraform acquires a state lock to protect the state from being written

by multiple users at the same time. Please resolve the issue above and try

again. For most commands, you can disable locking with the "-lock=false"

flag, but this is not recommended.

aryan@aryan-VirtualBox:~/docker$ ls

main.tf terraform.tfstate

aryan@aryan-VirtualBox:~/docker$ ll

total 20

drwxr-xr-x 3 aryan aryan 4096 May 7 14:07 ./

drwxr-xr-x 21 aryan aryan 4096 May 7 13:57 ../

-rw-r--r-- 1 aryan aryan 279 May 7 13:57 main.tf

drwxr-xr-x 3 aryan aryan 4096 May 7 12:26 .terraform/

-rw-r--r-- 1 root root 1161 May 7 12:37 terraform.tfstate

aryan@aryan-VirtualBox:~/docker$ ls .terraform/

plugins

aryan@aryan-VirtualBox:~/docker$ less terraform.tfstate

aryan@aryan-VirtualBox:~/docker$ mv terraform.tfstate old_terraform.tfstate

[joebertj]

Local state cannot be unlocked by another process

Just stop the stale terraform process.

Gently:
ps -ef | grep terraform | grep -v grep | awk '{print $2}' | xargs -n 1 kill

Forcefully:
ps -ef | grep terraform | grep -v grep | awk '{print $2}' | xargs -n 1 kill -9

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.