Aws-cli: aws cloudformation deploy fails with nested stack: "TemplateURL must be an Amazon S3 URL."

[pedros007]

When I use aws cloudformation deploy on a master template with a nested stack, the CloudFormation console shows CREATE_FAILED with an error: TemplateURL must be an Amazon S3 URL.

This example from #2384 can be used to demonstrate the problem. When I run

$ aws cloudformation deploy --template-file master.yaml --stack-name test-aws-cfn-deploy --s3-bucket redacted --s3-prefix foo

The Security stack fails to create with TemplateURL must be an Amazon S3 URL.:

It may be worth noting that the cloudformation package subcommand seems to generate the proper TemplateURL and hash.template file in S3. That is, when I run

$ aws cloudformation package --template-file master.yaml --s3-bucket redacted --s3-prefix foo

The TemplateURL output is correctly set to https://s3.amazonaws.com/redacted/foo/1ba98b556f5708993212041be8ab1747.template.

Is this related to @sanathkr's comment in https://github.com/awslabs/serverless-application-model/issues/90?

$ aws --version
aws-cli/1.14.55 Python/2.7.10 Darwin/16.5.0 botocore/1.9.8

[joguSD]

@sanathkr Any ideas?

[pedros007]

One more thing that might be helpful. When I go to the CloudFormation console, select my stack, View the "Template" tab. Then click "View Processed template" and I get

{
  "AWSTemplateFormatVersion": "2010-09-09",
  "Description": "Pipeline CFN",
  "Resources": {
    "Security": {
      "Type": "AWS::CloudFormation::Stack",
      "Properties": {
        "TemplateURL": "security.yaml",
        "Parameters": {
          "EnvironmentName": {
            "Ref": "AWS::StackName"
          }
        }
      }
    }
  }
}

So it looks like TemplateURL is still pointing to my relative path security.yaml. I expected the Processed Template generated by aws cloudformation deploy ... to set something like "TemplateURL": "https://s3.amazonaws.com/redacted/foo/1ba98b556f5708993212041be8ab1747.template".

Digging around the code at https://github.com/aws/aws-cli/tree/master/awscli/customizations/cloudformation a little, I'm surprised the deploy module is not just a lightweight wrapper around thepackage module.

[sanathkr]

@pedros007 Are you using SAM or just plain cloudformation?

Either way, you should be first calling aws cloudformation package to upload your local files to S3 (including nested stack templates) before you deploy.

If you are using SAM, your nested stacks won't deploy because CloudFormation does not support transforms in nested stacks (https://github.com/awslabs/serverless-application-model/issues/90)

[pedros007]

Thanks @sanathkr. I am using vanilla CloudFormation. I wish the CLI could read a template from stdin. That would enable something like this:

$ aws cloudformation package [opts] | aws cloudformation deploy --template-file STDIN [opts]

There's some part of me that does not like having to make a temporary file, then delete it. I think this is the shortest possible workflow:

$ aws cloudformation package [package-options] > temporary_template.yml && \
   aws cloudformation deploy --template-file temporary_template.yml [deploy-options] && \
   rm temporary_template.yml

which feels a little tedious. If you agree, want me to open a new issue with the feature request?

[lorengordon]

Try this?

aws cloudformation deploy --template-file <(aws cloudformation package [package-options]) [deploy-options]

[lorengordon]

Some commands in the aws-cli also support the - notation to indicate read from stdin, so maybe:

aws cloudformation package [opts] | aws cloudformation deploy --template-file - [opts]

[felixbuenemann]

@lorengordon Neither of those works and also giving /dev/stdin doesn't work, it gives error like:

Invalid template path /dev/fd/11
Invalid template path /dev/stdin

This feature has been requested in #2504.

[pedros007]

Closing as this is covered by #2504

[Deliforce]

@sanathkr I am getting an error as "Circular dependency between resources: ". while using nested template for cloud formatioin deploy cli command
Even i removed transforms in nested stacks
Please help me with this
This is my template file
AWSTemplateFormatVersion: '2010-09-09'
Resources:
deliforceserver:
Type: AWS::CloudFormation::Stack
Properties:
TemplateURL: https://s3.ap-south-1.amazonaws.com/deliforcesck/d5b039ba137da028f009e478ecac3f1e.template
TimeoutInMinutes: '60'
Outputs:
StackRef:
Value: !Ref deliforceserver
OutputFromNestedStack:
Value: !GetAtt deliforceserver.Outputs.BucketName