Aws-cli: cloudformation deploy fails if new parameters are not overridden even if they have default value

[kskory]

After adding new parameters to the template, it has to be deployed with --parameters-override for all the new parameters even if they have default value, otherwise deploy fails with:

An error occurred (ValidationError) when calling the CreateChangeSet operation: Invalid input for parameter key Param2. Cannot specify usePreviousValue as true for a parameter key not in the previous template

To reproduce, deploy simple template:

AWSTemplateFormatVersion : '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: A hello world application.
Parameters:
  Param1:
    Default: 123
    Type: String
Resources:
  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      Runtime: nodejs4.3
      CodeUri: ./hello.zip

add some parameter to it:

...
Parameters:
  Param1:
    Default: 123
    Type: String
  Param2:
    Default: 123
    Type: String
...

Try to deploy again, it fails with aforementioned error.

[JordonPhillips]

Thanks for the report, this seems to be related to a previous bugfix: #2573

[prokilogrammer]

Its not a regression to previous bugfix, but a new corner case that the code doesn't handle well.

Unfortunately, this is a tough one because I need to pull down the latest template from stack, parse parameters, diff with the new template and merge current-template, new-template and overrides to come up with list of accurate UsePreviousValues. For the majority use-case, this extra call is unnecessary.

Potential fix would be to use get-template-summary call instead of describe-stacks to check presence of stack AND fetch parameter information.

Until we fix this bug, you can override the parameter explicitly when you add a new parameter with default value. Just once when you first update the stack. Afterwards, UsePreviousValue should work as expected

[ffxsam]

This is still an issue as of awscli version 1.11.146.

[billnbell]

Just say you are adding ServiceCount, do it this way:

aws cloudformation deploy --template-file ./build/cfn-package.yml --stack-name dev-v2 --region us-west-2 --capabilities CAPABILITY_NAMED_IAM CAPABILITY_IAM --parameter-overrides ServiceCount=1

[btisdall]

In the modern world of infrastructure creates/updates happening via CI or some other automation one-time hacks to put the new parameter there really don't cut it.

[mga226]

Just adding another voice here.

This isn't a "corner case." The way this currently works doesn't play nice with CI at all, and it's something I have to hack around with manual workarounds on a regular basis. Using the default if no previous value exists should be the expected behavior.

[btisdall]

Shameless plug for #2943 which provides a alternate mode of operation (never re-use stack params because that's a deployment anti-pattern IMO) and thereby avoids this problem. I realise this won't be satisfactory for everyone but I thought I'd mention it.

[billnbell]

btisdall: this is not the same thing. I agree that we should not re-use stack params, but the default should work, and we should not have to override them.

[ffxsam]

I actually disagree. I think reusing params is extremely handy. No way would I want to have to specify every single parameter every time I wanted to re-deploy. Sure, in a CI/CD environment that would probably make sense. But often times I'm running build & deploy from the command line.

[dgcgh]

In particular when doing stack updates, where you might not be aware of all the parameters that were set at creation time. We currently have scripts for updates which pull all of the original parameters from the stack and then override the ones we're changing to prevent things like regression to defaults, but it would be nice to not have to do that.

Also because of the behavior cited here adding a parameter to a stack means breaking any code/procedure/deployment that does not know about the new parameter. That would make sense if the parameter did not have a default, but that's what defaults are supposed to do: make a selection when none is provided by the user.

[btisdall]

@billnbell and others: I'm fully aware it's not the same thing, I thought my comment made this clear. I absolutely agree that new template params with a default should just work with the command as it is. And I don't deny parameter re-use is handy, I just happen to think it's a bad choice for some situations because you're effectively pushing your infrastructure configuration out of code into a remote system.

[larryboymi]

👍 Any news on this?

[ldm314]

Ran into this same issue today.

[libert-xyz]

same issue here, any updates?

[foxx]

Same problem here :/

[ericzbeard]

This is still an issue with Cloud9 Lambda functions. It's not possible to add a parameter to the SAM template file. And the click-to-deploy functionality does not offer a way to fix it via the command line.

[aws-adjustmentbureau]

bug has still not been fixed on aws cli

[oxsannikova]

Have same issue with aws cli and Lambda.

[Smyja]

same issues

[mynamerahulkumar]

An error occurred (ValidationError) when calling the CreateStack operation: Parameters: [PrivateSubnet1CIDR, PublicSubnet2CIDR, PrivateSubnet2CIDR, PublicSubnet1CIDR] do not exist in the template
My solution-I have not proper indentation after proper indentation it works