Amplify-cli: UPDATE_ROLLBACK_FAILED state and can not be updated when pushing changes
Describe the bug
Getting the following error when I try to amplify push changes from the CLI. I'm using the CLI because my amplify console build is also failing.
Stack:arn:aws:cloudformation:us-east-1:473356752602:stack/modernapp-20190222141121/a3ea3540-36d5-11e9-8a22-121328bc7b22 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
To Reproduce
Steps to reproduce the behavior:
I recently attempted to change the PAY_PER_REQUEST to PROVISIONED DynamoDBBillingMode in parameters.json and when that failed I switched back. Then I added two new fields to my schema which also caused my amplify console build to fail.
I'm experiencing intermittent build errors in the amplify console for over a week and have someone looking into that well. I mention it because it could be related. I also upgraded to the latest cli last week.
Expected behavior
Ability to push schema changes without error.
Desktop (please complete the following information):
- OS: macOS
- Version 1.6.3
kldeb
All 9 comments
@aireater could you include the additional lines from the CloudFormation output.
yuth
on 29 Apr 2019
@yuth this is all i get from the cli:
⠼ Updating resources in the cloud. This may take a few minutes..✖ An error occurred when pushing the resources to the cloud
Stack:arn:aws:cloudformation:us-east-1:473356752602:stack/modernapp-20190222141121/a3ea3540-36d5-11e9-8a22-121328bc7b22 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
and here's the log from the amplify console failed build (latest):
2019-04-29T19:40:31.674Z [INFO]: Git SSH Key acquired
2019-04-29T19:40:31.773Z [INFO]: Cloning repository: [email protected]:modernutility/modern-app.git
2019-04-29T19:40:31.951Z [INFO]: Agent pid 69
2019-04-29T19:40:31.975Z [WARNING]: Identity added: /root/.ssh/git_rsa (/root/.ssh/git_rsa)
2019-04-29T19:40:32.111Z [WARNING]: Cloning into 'modern-app'...
2019-04-29T19:40:32.369Z [WARNING]: Warning: Permanently added the RSA host key for IP address '18.205.93.0' to the list of known hosts.
2019-04-29T19:40:35.498Z [INFO]: Switching to commit: ee2766820e04791899e5a5368b83f3ed33ba8361
2019-04-29T19:40:35.594Z [WARNING]: Note: checking out 'ee2766820e04791899e5a5368b83f3ed33ba8361'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at ee27668... format change
2019-04-29T19:40:35.594Z [INFO]: Checking for Git submodules at: /codebuild/output/src782203504/src/modern-app/.gitmodules
2019-04-29T19:40:35.626Z [INFO]: Retrieving cache...
2019-04-29T19:40:47.150Z [INFO]: Extracting cache...
2019-04-29T19:41:38.730Z [INFO]: Extraction done...
2019-04-29T19:41:38.928Z [INFO]: Retrieving environment cache...
2019-04-29T19:41:46.247Z [INFO]: ## Starting Backend Build
# Starting phase: build
# Executing command: amplifyPush --simple
2019-04-29T19:41:46.250Z [INFO]: # Getting Amplify CLI Cloud-Formation stack info from environment cache
2019-04-29T19:41:46.279Z [INFO]: # Start initializing Amplify environment: dev
2019-04-29T19:41:46.280Z [INFO]: STACKINFO={
"awscloudformation": {
"AuthRoleName": "modernapp-20190222141121-authRole",
"UnauthRoleArn": "arn:aws:iam::473356752602:role/modernapp-20190222141121-unauthRole",
"AuthRoleArn": "arn:aws:iam::473356752602:role/modernapp-20190222141121-authRole",
"Region": "us-east-1",
"DeploymentBucketName": "modernapp-20190222141121-deployment",
"UnauthRoleName": "modernapp-20190222141121-unauthRole",
"StackName": "modernapp-20190222141121",
"StackId": "arn:aws:cloudformation:us-east-1:473356752602:stack/modernapp-20190222141121/a3ea3540-36d5-11e9-8a22-121328bc7b22"
},
"categories": {
"auth": {
"cognito5cc85e6d": {}
}
}
}
# Importing Amplify environment: dev (amplify env import)
2019-04-29T19:41:56.739Z [INFO]: Successfully added environment from your project
2019-04-29T19:41:56.796Z [INFO]: # Initializing existing Amplify environment: dev (amplify init)
2019-04-29T19:41:58.251Z [INFO]: Note: It is recommended to run this command from the root of your app directory
2019-04-29T19:42:01.157Z [WARNING]: - Initializing your environment: dev
2019-04-29T19:42:01.805Z [WARNING]: ✔ Initialized provider successfully.
2019-04-29T19:42:02.250Z [INFO]: GraphQL schema compiled successfully.
Edit your schema at /codebuild/output/src782203504/src/modern-app/amplify/backend/api/ModernApp/schema.graphql or place .graphql files in a directory at /codebuild/output/src782203504/src/modern-app/amplify/backend/api/ModernApp/schema
2019-04-29T19:42:25.648Z [WARNING]: - Updating resources in the cloud. This may take a few minutes...
2019-04-29T19:42:25.921Z [WARNING]: Error updating cloudformation stack
2019-04-29T19:42:25.923Z [WARNING]: ✖ An error occurred when pushing the resources to the cloud
2019-04-29T19:42:25.923Z [WARNING]: ✖ There was an error initializing your environment.
2019-04-29T19:42:25.923Z [INFO]: init failed
2019-04-29T19:42:25.927Z [INFO]: { ValidationError: Stack:arn:aws:cloudformation:us-east-1:473356752602:stack/modernapp-20190222141121/a3ea3540-36d5-11e9-8a22-121328bc7b22 is in UPDATE_ROLLBACK_FAILED state and can not be updated.
at Request.extractError (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/protocol/query.js:50:29)
at Request.callListeners (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/sequential_executor.js:106:20)
at Request.emit (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/sequential_executor.js:78:10)
at Request.emit (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/request.js:683:14)
at Request.transition (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/request.js:22:10)
at AcceptorStateMachine.runTo (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/state_machine.js:14:12)
at /root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/state_machine.js:26:10
at Request.<anonymous> (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/request.js:38:9)
at Request.<anonymous> (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/request.js:685:12)
at Request.callListeners (/root/.nvm/versions/node/v8.12.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-provider-awscloudformation/node_modules/aws-sdk/lib/sequential_executor.js:116:18)
message: 'Stack:arn:aws:cloudformation:us-east-1:473356752602:stack/modernapp-20190222141121/a3ea3540-36d5-11e9-8a22-121328bc7b22 is in UPDATE_ROLLBACK_FAILED state and can not be updated.',
code: 'ValidationError',
time: 2019-04-29T19:42:25.919Z,
requestId: 'ea3ab9c0-6ab6-11e9-b81f-7744bf87e756',
statusCode: 400,
retryable: false,
retryDelay: 63.69543174526875 }
2019-04-29T19:42:25.934Z [ERROR]: !!! Build failed
2019-04-29T19:42:25.934Z [ERROR]: !!! Non-Zero Exit Code detected
2019-04-29T19:42:25.935Z [INFO]: # Starting environment caching...
2019-04-29T19:42:25.935Z [INFO]: Uploading environment cache artifact...
2019-04-29T19:42:26.099Z [INFO]: # Environment caching completed
Terminating logging...
Hey, @aireater, you will need to go to the AWS CoudFormation Console and look for the events of your stacks to find out what caused the UPDATE_ROLLBACK_FAILED. The logs of the Amplify Console doesn't help. Until you solve the UPDATE_ROLLBACK_FAILED state you won't be able to update it using the Amplify Console, CLI or any other method.
kstro21
on 30 Apr 2019
Thanks, @kstro21 I looked at the cloud formation console and I can see that UPDATE_ROLLBACK_FAILED is from Mon, 15 Apr.
The error is
Subscriber limit exceeded: Update to PayPerRequest mode are limited to once in 1 day(s). Last update at Mon Apr 29 17:25:33 UTC 2019. Next update can be made at Tue Apr 30 17:25:33 UTC 2019 (Service: AmazonDynamoDBv2; Status Code: 400; Error Code: LimitExceededException;
We are now past that time and I still can't push changes.
kldeb
on 30 Apr 2019
@aireater, you can use the option Continue update rollback in the stack. The state of the stack needs to be UPDATE_ROLLBACK_COMPLETE until you can update the stack again.
kstro21
on 30 Apr 2019
@kstro21 they are all nested stacks in UPDATE_ROLLBACK_FAILED state. The only non nested fail is from February.
kldeb
on 30 Apr 2019
@aireater, sorry it was not clear in my last comment. You can use the option Continue update rollback in the root stack. The state of the stack needs to be UPDATE_ROLLBACK_COMPLETE until you can update the stack again.
kstro21
on 30 Apr 2019
@kstro21 Thanks! That worked. Closing.
kldeb
on 30 Apr 2019
Whoops. Just had this problem and the failed ones were all real old but I hit continue rollback and after like 30 mins it still said in progress. Then I just decided they were so old I could probably just hit delete. Well I deleted my whole app accidently along with the 400 users I had gotten. Well there are good days and there's today. Time to restart. Be careful here
24jr
on 18 Apr 2020
Related issues
onlybakam
·
3Comments
jeanpaulcozzatti
·
3Comments
ffxsam
·
3Comments
jkeys-ecg-nmsu
·
3Comments
kstro21
·
3Comments