Sceptre: Update using changeset with dependencies doesn't apply changes

[ianburrell]

With Sceptre 2.2.1, updating a stack using changesets doesn't apply changes. It doesn't show the changes, and doesn't prompt if should apply. This behavior seems to happen when the stack has dependencies, and the dependencies don't have any changes.

$ sceptre update -c csv2dev/ingestion-dev-airflow.yaml
[2019-10-16 15:10:41] - csv2dev/ingestion-dev-db - Successfully initiated creation of Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'
[2019-10-16 15:10:42] - csv2dev/ingestion-dev-spark - Successfully initiated creation of Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'
[2019-10-16 15:10:44] - csv2dev/ingestion-dev-airflow - Successfully initiated creation of Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'
[2019-10-16 15:10:50] - csv2dev/ingestion-dev-db - Successfully deleted Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'
[2019-10-16 15:10:51] - csv2dev/ingestion-dev-spark - Successfully deleted Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'
[2019-10-16 15:10:51] - csv2dev/ingestion-dev-airflow - Successfully deleted Change Set 'change-set-c9a84920f06111e9b8ec14b1c80043dc'

Updating without changesets does apply the changes and shows that the dependent stacks don't have any changes.

$ sceptre update -c csv2dev/ingestion-dev-db.yaml
[2019-10-16 15:12:20] - csv2dev/ingestion-dev-db - Successfully initiated creation of Change Set 'change-set-04c5a78cf06211e9a3be14b1c80043dc'
[2019-10-16 15:12:20] - csv2dev/ingestion-dev-db - Successfully deleted Change Set 'change-set-04c5a78cf06211e9a3be14b1c80043dc'
(env) iburrell@csoriburrell:cloudformation(csv2dev)$ sceptre update csv2dev/ingestion-dev-db.yaml
Do you want to update 'csv2dev/ingestion-dev-db.yaml' [y/N]: y
[2019-10-16 15:12:28] - csv2dev/ingestion-dev-db - Updating Stack
[2019-10-16 15:12:29] - csv2dev/ingestion-dev-db - No updates to perform.
(env) iburrell@csoriburrell:cloudformation(csv2dev)$ sceptre update csv2dev/ingestion-dev-airflow.yaml
Do you want to update 'csv2dev/ingestion-dev-airflow.yaml' [y/N]: y
[2019-10-16 15:12:51] - csv2dev/ingestion-dev-db - Updating Stack
[2019-10-16 15:12:52] - csv2dev/ingestion-dev-db - No updates to perform.
upload s3://csv2dev-linear/cloudformation/phoenix-csv2dev-ingestion-dev-spark/files/setup-ingestion-spark-emr.sh
[2019-10-16 15:12:52] - csv2dev/ingestion-dev-spark - Updating Stack
[2019-10-16 15:12:53] - csv2dev/ingestion-dev-spark - No updates to perform.
upload s3://csv2dev-linear/cloudformation/phoenix-csv2dev-ingestion-dev-airflow/templates/redis-cache.yaml
upload s3://csv2dev-linear/cloudformation/phoenix-csv2dev-ingestion-dev-airflow/templates/rds-pg.yaml
[2019-10-16 15:12:53] - csv2dev/ingestion-dev-airflow - Updating Stack
[2019-10-16 15:12:55] - csv2dev/ingestion-dev-airflow phoenix-csv2dev-ingestion-dev-airflow AWS::CloudFormation::Stack UPDATE_IN_PROGRESS User Initiated
[2019-10-16 15:13:04] - csv2dev/ingestion-dev-airflow postgresStack AWS::CloudFormation::Stack UPDATE_IN_PROGRESS 
[2019-10-16 15:13:04] - csv2dev/ingestion-dev-airflow redisStack AWS::CloudFormation::Stack UPDATE_IN_PROGRESS 
[2019-10-16 15:13:04] - csv2dev/ingestion-dev-airflow redisStack AWS::CloudFormation::Stack UPDATE_COMPLETE 
[2019-10-16 15:13:04] - csv2dev/ingestion-dev-airflow postgresStack AWS::CloudFormation::Stack UPDATE_COMPLETE 
[2019-10-16 15:13:08] - csv2dev/ingestion-dev-airflow phoenix-csv2dev-ingestion-dev-airflow AWS::CloudFormation::Stack UPDATE_COMPLETE_CLEANUP_IN_PROGRESS 
[2019-10-16 15:13:12] - csv2dev/ingestion-dev-airflow redisStack AWS::CloudFormation::Stack UPDATE_IN_PROGRESS 
[2019-10-16 15:13:12] - csv2dev/ingestion-dev-airflow postgresStack AWS::CloudFormation::Stack UPDATE_IN_PROGRESS 
[2019-10-16 15:13:12] - csv2dev/ingestion-dev-airflow redisStack AWS::CloudFormation::Stack UPDATE_COMPLETE 
[2019-10-16 15:13:12] - csv2dev/ingestion-dev-airflow postgresStack AWS::CloudFormation::Stack UPDATE_COMPLETE 
[2019-10-16 15:13:12] - csv2dev/ingestion-dev-airflow phoenix-csv2dev-ingestion-dev-airflow AWS::CloudFormation::Stack UPDATE_COMPLETE 

Also, by modifying the stack to have dependencies, updating with changesets does work. It shows the changes and prompts.

[ngfgrant]

Thanks for raising. Will have a look in to this. I think it has been raised before so I’ll dig through the other issues and link them up.

[nunogrl]

I have the same issue. usually passing the parameter verbose forces the stack to update:

sceptre update -v <stack>

If you really want to create the Change-set, just create it:

sceptre create <stack> <change-set_name>

A change-set will be created to every dependency of your stack unfortunately, but this ensures that you'll be able to audit your changes prior to apply. You won't be able to run sceptre to execute your change-set from here, so proceed from the console.
I wrote something to de-clutter the aws environment of useless change-sets that can't be applied:
(it was used initially to execute all the pending change-sets as well, that bit is commented out now)

#!/bin/sh

CHANGESET=${2:-.}
STACK=${1:-prod}
# get list of stacks
sceptre list change-sets ${STACK} | jq -r '.[]|.[]| "\(.StackName) \(.ChangeSetName) \(.ExecutionStatus) \(.ChangeSetId)"' | egrep "${CHANGESET}" | while read line; do
    set -- $(echo $line)
    # echo $line
    if [ "${3}" = "UNAVAILABLE" ]; then
        echo "DELETE: CHANGE-SET \"${2}\" on \"${1}\""
        # echo "aws cloudformation delete-change-set --change-set-name ${4}"
        aws cloudformation delete-change-set --change-set-name ${4}
    else
        echo "EXECUTE: CHANGE-SET \"${2}\" on \"${1}\""
        echo " ===  DONT ==== " # aws cloudformation execute-change-set --change-set-name ${4}
    fi
done

[pipozzz]

I see the same issue here. Please can you take a look at this? if no than sceptre is not usable at all, I consider this as serious issue of basic functionality and no action from Oct 2019.
May I help you somehow?

Thank you.

[pipozzz]

Any update ?

[pipozzz]

Is this project alive?

[ngfgrant]

Yes - other priorities right now.

[ngfgrant]

AWS cli has a option ‘no-fail-on-empty-changeset’. I think we need to follow that example. This could be as simple as forcing our cli to return exit code 0 if the flag is set and that should allow change-sets with changes to be executed.

[ngfgrant]

this is a duplicate of https://github.com/Sceptre/sceptre/issues/846, closing this issue to track it in https://github.com/Sceptre/sceptre/issues/846

[ngfgrant]

#896 resolves