Boto3: client.validate_template return error while aws cloudformation validate-template doesn't

[d2huynh]

python --version
Python 2.7.11 :: Anaconda 2.5.0 (64-bit)

print boto3.version
1.3.0

I'm seeing a problem where boto3 client.validate_template returns ValidationError consistently on any template I tried on; however, aws cloudformation validate-template reads the same templates ok.

In my test.py:

        cfclient = boto3.client('cloudformation')
       response = cfclient.validate_template(TemplateBody='file://mytemplate.json')

and this would return:
Traceback (most recent call last):
File "./test.py", line 207, in
response = cfclient.validate_template(TemplateBody='file:/mytemplate.json')
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 228, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 492, in _make_api_call
raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: JSON not well-formed. (line 1, column 5)
NOTE: the error always points to line 1, column 5.

When I run "aws cloudformation validate-template --template-body file://mytemplate.json. It appears to be okay.
{
"Description": "Template to launch an instance",
"Parameters": []
}

I have tried giving the TemplateBody absolute path (file:///root/development/mytemplate.json) or relative path (file://mytemplate.json) and the problem still persists. This lead me to believe that the cause is something else.

My ultimate goal being able to create a stack with the following code with Boto3:
response = cfclient.create_stack(StackName='my-stack', TemplateBody=tdmctemplate)

Currently, this statement will fail on the same error.

[d2huynh]

I also tried using the resource route and that also resulted in the same failure.

import boto3
cf = boto3.resource('cloudformation')
response = cf.meta.client.validate_template(
    TemplateBody='mybody',
    TemplateURL='myurl'
)

Traceback (most recent call last):
File "./tdmc_launch_tdstack.py", line 38, in
TemplateBody='file://mytemplate.json'
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 228, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 492, in _make_api_call
raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the ValidateTemplate operation: Template format error: JSON not well-formed. (line 1, column 5)

        cf = boto3.resource('cloudformation')
        resp = cf.meta.client.validate_template(
                        TemplateBody='file://mytemplate.json'
               )

[kyleknap]

So you cannot use the file:// from the CLI. We only have this in the CLI because there is not a good way of getting the contents of the file from the command line. For boto3, you will have to open a file yourself and read its contents. So something like this:

cfclient = boto3.client('cloudformation')
with open('mytemplate.json', 'r') as f:
    response = cfclient.validate_template(TemplateBody=f.read())

Let us know if that helps.

[d2huynh]

this works great thank you.

However, I'm getting the same validation error when I tried to create a new cloudformation stack with boto3.client.create_stack(...) call.

cf = boto3.client('cloudformation')
resp = cf.create_stack(StackName=stack_name, TemplateBody='mytemplate.json')

Traceback (most recent call last):
File "./tdmc_launch_tdmcstack.py", line 146, in tdmc_create_verify_stack
resp = cf.create_stack(StackName=stack_name, TemplateBody='mytemplate.json')
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 228, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/root/anaconda2/lib/python2.7/site-packages/botocore/client.py", line 492, in _make_api_call
raise ClientError(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (ValidationError) when calling the CreateStack operation: Template format error: JSON not well-formed. (line 1, column 11)

[kyleknap]

I think this a similar solution. You cannot provide the filename, you need to open the file and pass its contents to the parameter. So something like:

cf = boto3.client('cloudformation')
with open('mytemplate.json', 'r') as f:
    resp = cf.create_stack(StackName=stack_name, TemplateBody=f.read())

[d2huynh]

This also works great.
You can close this incident now.
Thanks!

[kyleknap]

Awesome. Glad to hear that you got it working.