Aws-sam-cli: sam validate fails if cf template was created with cdk

[codinpeat]

Description

After trying to validate template or execute a single lambda out of the cf template with sam cli, the follwing error appears:

Error: Failed to parse template: unacceptable character #x0000: special characters are not allowed
in "", position 3

It doesnt matter if i check out a example app or try it with my own.

Observed result

The last entries of the output in the console with the debug option are this ones:

Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\yamlhelper.py", line 64, in yaml_parse
return json.loads(yamlstr)
File "json__init__.py", line 354, in loads
File "json\decoder.py", line 339, in decode
File "json\decoder.py", line 357, in raw_decode
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "runpy.py", line 193, in _run_module_as_main
File "runpy.py", line 85, in _run_code
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli__main__.py", line 12, in
cli(prog_name="sam")
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 722, in __call__
return self.main(args, *kwargs)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 697, in main
rv = self.invoke(ctx)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 1066, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 895, in invoke
return ctx.invoke(self.callback, *ctx.params)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 535, in invoke
return callback(args, *kwargs)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\decorators.py", line 64, in new_func
return ctx.invoke(f, obj, *args[1:], *kwargs)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\click\core.py", line 535, in invoke
return callback(args, *kwargs)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\metrics.py", line 94, in wrapped
raise exception # pylint: disable=raising-bad-type
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\lib\telemetry\metrics.py", line 65, in wrapped
return_value = func(args, *kwargs)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\validate\validate.py", line 32, in cli
do_cli(ctx, template) # pragma: no cover
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\validate\validate.py", line 40, in do_cli
sam_template = _read_sam_file(template)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\commands\validate\validate.py", line 69, in _read_sam_file
sam_template = yaml_parse(sam_template.read())
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\samcli\yamlhelper.py", line 67, in yaml_parse
return yaml.safe_load(yamlstr)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\yaml__init__.py", line 94, in safe_load
return load(stream, SafeLoader)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\yaml__init__.py", line 70, in load
loader = Loader(stream)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\yaml\loader.py", line 24, in __init__
Reader.__init__(self, stream)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\yaml\reader.py", line 74, in __init__
self.check_printable(stream)
File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\yaml\reader.py", line 144, in check_printable
'unicode', "special characters are not allowed")
yaml.reader.ReaderError: unacceptable character #x0000: special characters are not allowed
in "", position 3

Expected result

When i invoke the following command:

sam local invoke MyFunction3BAA72D1 --no-event

i would expect that it will be executed normally.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

OS is Windows 10
Python is 3.8.0
nodejs is 12.13.0
SAM CLI, version 0.19.0

I hope this helps you to improve your very great software even more! Thanks!

[codinpeat]

When i convert the template with

iconv -f utf-16 -t utf-8 template.yaml > template.yaml

This error not come up again.

[sanathkr]

Template files are typically expected to be UTF-8 encoded. We haven't tested with utf-16. We have made several improvemtns to file handling. Can you upgrade SAM CLI to latest version and try?

[marcortw]

Ran into the same issue on powershell and was able to output in utf-8 by setting:
$PSDefaultParameterValues['Out-File:Encoding'] = 'utf8'
More information on SO: https://stackoverflow.com/a/40098904/4249811

[moscow4gh]

I ran into the same problem by SAM CLI 0.40.0 on Win10.
sam validate, template file is cp932 encoded:
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x8a in position 1874: invalid start byte
sam build, template file is utf-8 encoded:
Error: Failed to parse template: 'cp932' codec can't decode byte 0x86 in position 1879: illegal multibyte sequence

[JMRolf-Spok]

I've also encountered this issue with the latest version of the AWS SAM CLI (0.41.0 at time of writing).

What's curious is that @marcortw's suggesting of setting the out file encoding makes the validation pass. I re-synthesized my template and sam validate succeeded.

Unfortunately, this appears to surface another issue. When attempting to run some local SAM CLI commands I receive an error message that seems incorrect to me. The message reads

Error: [InvalidTemplateException("'Resources' section is required")] 'Resources' section is required

I say this seems incorrect because the template for which these commands were issued definitely does have a resources section. Here is a screen shot just in case I've done something obviously wrong. Thanks to the SAM team for all your hard work.

[jfuss]

@JMRolf-Spok Try sam local start-api -template.yml not template.yaml, since that is how you named the file.

[JMRolf-Spok]

@jfuss Yeah sorry that was a typo. The effect is the same, however.

[jfuss]

@JMRolf-Spok Please create a new issue, something else is going on which doesn't seem related to this issue.

[JMRolf-Spok]

@jfuss will do. Thanks.