Terraform: v0.9 Terraform s3 backed docs unclear

[eredi93]

If before was very clear how to store a state file using S3 now in not well documented, especially the locking mechanism.
I want to upgrade to 0.9 and use the Dynamo locking but i constantly get a ResourceNotFoundException: Requested resource not found on planning.

these are the step i took:

1. add the terraform block to my repo

terraform {
  // https://www.terraform.io/docs/backends/types/s3.html
  backend "s3" {
    bucket     = "terraform"
    key        = "test/terraform.tfstate"
    region     = "eu-east-1"
    encrypt    = "true"
    lock_table = "terraform_state"
  }
}

1. run a terraform init

$ terraform init
Initializing the backend...

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your environment. If you forget, other
commands will detect it and remind you to do so if necessary.

1. run a terraform plan

$ terraform plan                                                                                                                  
Error locking state: Error acquiring the state lock: 2 error(s) occurred:

* ResourceNotFoundException: Requested resource not found
    status code: 400, request id: D706BM06MD2TA51DQPVSF6OAVNVV4KQNSO5AEMVJF66Q9ASUAAJG
* ResourceNotFoundException: Requested resource not found
    status code: 400, request id: MU5GU4A1SLU5DSSIIMFUJBKQUFVV4KQNSO5AEMVJF66Q9ASUAAJG

Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"

i must be doing something wrong but definatly the docs ddi not make it easy to spot it

[jbardin]

Hi @eredi93,

Sorry you're having an issue with this. Did you create the terraform_state DynamoDB table with the correct primary key?

The S3 option have mostly stayed the same, and you can see documentation with "lock_table" here:

https://www.terraform.io/docs/backends/types/s3.html#lock_table

lock_table - (Optional) The name of a DynamoDB table to use for state locking. The table must have a primary key named LockID.

[plombardi89]

@jbardin Would help if the docs gave more guidance. For example, what is the type of the Primary Key? String, Binary or Number?

[plombardi89]

Nevermind. I might just be an idiot :) I was misreading the Dynamo docs since I've never used it and wanted to try it out for locking.

[plombardi89]

OK... Not sure honestly. I have a DynamoDB table

plombardi@palwork ~/d/b/loom-tf> aws dynamodb describe-table --table-name=terraform_state
{
    "Table": {
        "TableArn": "arn:aws:dynamodb:us-east-1:914373874199:table/terraform_state", 
        "AttributeDefinitions": [
            {
                "AttributeName": "LockID", 
                "AttributeType": "S"
            }
        ], 
        "ProvisionedThroughput": {
            "NumberOfDecreasesToday": 0, 
            "WriteCapacityUnits": 1, 
            "ReadCapacityUnits": 1
        }, 
        "TableSizeBytes": 0, 
        "TableName": "terraform_state", 
        "TableStatus": "ACTIVE", 
        "KeySchema": [
            {
                "KeyType": "HASH", 
                "AttributeName": "LockID"
            }
        ], 
        "ItemCount": 0, 
        "CreationDateTime": 1489688879.271
    }
}

I've configured Terraform to use this backend with this config

plombardi@palwork ~/d/b/loom-tf> cat backend.tf
terraform {
  backend "s3" {
    bucket     = "datawire-loom"
    region     = "us-east-1"
    key        = "test/terraform.tfstate"
    encrypt    = "true"
    acl        = "private"
    lock_table = "terraform_state"
  }
}

Should I be seeing any kind of items in the Dynamo table to ensure this feature is working?

[jbardin]

@plombardi89,

That table only has keys when there is a Terraform operation in progress. If it wasn't working, you would get an error from Terraform when it tried to lock the state.

[yoaquim]

I'm having this exact same issue.

Error locking state: Error acquiring the state lock: 2 error(s) occurred:

* ResourceNotFoundException: Requested resource not found
    status code: 400, request id: 5DI3M2N2OPRS9NSROMVDAOQO6BVV4KQNSO5AEMVJF66Q9ASUAAJG
* ResourceNotFoundException: Requested resource not found
    status code: 400, request id: NV9EQ0NSD3RUFMI2OFBJAJAKIBVV4KQNSO5AEMVJF66Q9ASUAAJG

Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.

I've confirmed that my tables are in the correct regions, that my user has access to DynamoDB (and all other needed resources) and that the key = LockID.

This is my backend config:

terraform {
  backend "s3" {
    bucket = "capp-terraform-state"
    key = "vpc/terraform.tfstate"
    region = "us-east-1"
    profile = "capp-terraform"
    lock_table = "terraform-state-lock"
  }
}

That AWS credentials profile is correct, and it's the same one I'm using to build with Terraform – which actually works if I ignore state locking.

Ideas?

[anthonyagresta]

@yoaquim @eredi93
I ran into this error today.
Odds are, you've either given up or figured it out by now, but I'll post how I fixed it in case anyone else runs into this.

In my case I had had accidentally made my DynamoDB table in the wrong AWS region.
I would double-check that you have the correct name and region for your DynamoDB table, and that the IAM user associated with your credentials is allowed to do DynamoDB things.

[keymon]

When you have the error:

* ResourceNotFoundException: Requested resource not found
    status code: 400, request id: DI3M2N2OPRS9NSROMVDAOQO6BVV4KQNSO5AEMVJF66Q9ASUAAJG

Very likely is the missing dynamodb table. The error is not clear and the documentation neither

You can try creating the table like this:

         aws dynamodb create-table \
         --region us-east-1 \
         --table-name terraform_locks \
         --attribute-definitions AttributeName=LockID,AttributeType=S \
         --key-schema AttributeName=LockID,KeyType=HASH \
         --provisioned-throughput ReadCapacityUnits=1,WriteCapacityUnits=1

Be sure the name matches the entry in the config dynamodb_table

Also, double check that the table is not already created in other region. You can use this script to list all the tables everywhere:

cat <<EOF | xargs -n1 aws dynamodb list-tables --region
us-east-2
us-east-1
us-west-1
us-west-2
ap-south-1
ap-northeast-2
ap-southeast-1
ap-southeast-2
ap-northeast-1
ca-central-1
eu-central-1
eu-west-1
eu-west-2
sa-east-1
EOF

[nic]

For other peoples locking for solution of lock-file when skip/cancel:

Acquiring state lock. This may take a few moments...

Error: Error locking state: Error acquiring the state lock: ConditionalCheckFailedException: The conditional request failed
    status code: 400, request id: D706BM06MD2TA51DQPVSF6OAVNVV4KQNSO5AEMVJF66Q9ASUAAJG
Lock Info:
  ID:        eeed251d-e050-49a5-a95a-133ac8da40d8
  Path:      sdffsdf/cddfd/terraform.tfstate
  Operation: OperationTypeApply
  Who:       nic@nic
  Version:   0.11.1
  Created:   2017-12-21 23:49:27.56572 +0000 UTC
  Info:


Terraform acquires a state lock to protect the state from being written
by multiple users at the same time. Please resolve the issue above and try
again. For most commands, you can disable locking with the "-lock=false"
flag, but this is not recommended.

terraform force-unlock eeed251d-e050-49a5-a95a-133ac8da40d8 .

[hmacias]

In my case the Dynamo table was correctly configured and still saw this issue. I got this to work by removing a hidden .terraform folder and ran _terraform init_ again. I am using Terraform v0.10.6...

[russmac]

It seems changing the s3 backend configuration to a different s3 backend config requires the .terraform directory to be removed, After doing this and initialising successfully a lock file (different to the ones from apply) remains in DynamoDB:

REDACTED-tf-remote-state/development/REDACTED/ecs-md5 | a24a01df33ddecaab28e76101290a2dc|

No errors reported, I remove the locks (directly in dynamo) and business as usual.

[maheshrajanna]

"lock_table" her

Thank You

[hashibot[bot]]

I'm going to lock this issue because it has been closed for _30 days_ ⏳. This helps our maintainers find and focus on the active issues.

If you have found a problem that seems similar to this, please open a new issue and complete the issue template so we can capture all the details necessary to investigate further.