Aws-sam-cli: Build Error: JavaMavenWorkflow:MavenBuild - 'utf-8' codec can't decode byte x in position z: invalid continuation byte

[mwysocki87]

Description

When initiating local Lambda run from IntelliJ IDEA (with AWS Toolkit plugin), I get the below error.

Steps to reproduce

1. Create a Java lambda function in AWS.
2. Create local run configuration.
3. Run the lambda.

Observed result

Note: to be able to use the --debug flag, I ran the command directly from terminal, but the error is the same as when run from IntelliJ IDEA.

PS C:\Users\YD9\source\repos\country-layer\integration-serverless\deployment> & "C:\Program Files\Amazon\AWSSAMCLI\bin\sam.cmd" build --template C:\Users\YD9\AppData\Local\Temp\template1.yaml --build-dir C:\Users\YD9\AppData\Local\T
emp\lambdaBuild1 --debug
2019-05-07 08:28:25 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-07 08:28:25 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-07 08:28:25 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-07 08:28:25 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-07 08:28:25 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-07 08:28:25 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-07 08:28:25 Changing event name from docs..autoscaling.CreateLaunchConfiguration.complete-section to docs..auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-07 08:28:25 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-07 08:28:25 Changing event name from docs..logs.CreateExportTask.complete-section to docs..cloudwatch-logs.CreateExportTask.complete-section
2019-05-07 08:28:25 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-07 08:28:25 Changing event name from docs..cloudsearchdomain.Search.complete-section to docs..cloudsearch-domain.Search.complete-section
2019-05-07 08:28:25 Changing event name from creating-client-class.iot-data to creating-client-class.iot-data-plane
2019-05-07 08:28:25 Changing event name from before-call.apigateway to before-call.api-gateway
2019-05-07 08:28:25 Changing event name from request-created.machinelearning.Predict to request-created.machine-learning.Predict
2019-05-07 08:28:25 Changing event name from before-parameter-build.autoscaling.CreateLaunchConfiguration to before-parameter-build.auto-scaling.CreateLaunchConfiguration
2019-05-07 08:28:25 Changing event name from before-parameter-build.route53 to before-parameter-build.route-53
2019-05-07 08:28:25 Changing event name from request-created.cloudsearchdomain.Search to request-created.cloudsearch-domain.Search
2019-05-07 08:28:25 Changing event name from docs..autoscaling.CreateLaunchConfiguration.complete-section to docs..auto-scaling.CreateLaunchConfiguration.complete-section
2019-05-07 08:28:25 Changing event name from before-parameter-build.logs.CreateExportTask to before-parameter-build.cloudwatch-logs.CreateExportTask
2019-05-07 08:28:25 Changing event name from docs..logs.CreateExportTask.complete-section to docs..cloudwatch-logs.CreateExportTask.complete-section
2019-05-07 08:28:25 Changing event name from before-parameter-build.cloudsearchdomain.Search to before-parameter-build.cloudsearch-domain.Search
2019-05-07 08:28:25 Changing event name from docs..cloudsearchdomain.Search.complete-section to docs..cloudsearch-domain.Search.complete-section
2019-05-07 08:28:25 'build' command is called
2019-05-07 08:28:25 No Parameters detected in the template
2019-05-07 08:28:25 1 resources found in the template
2019-05-07 08:28:25 Found Serverless function with name='Function' and CodeUri='C:/Users/YD9/source/repos/country-layer/integration-serverless/dea-windmill-integration/windmill-file-loader-lambda'
2019-05-07 08:28:25 Building resource 'Function'
2019-05-07 08:28:25 Looking for a supported build workflow in following directories: ['C:\Users\YD9\source\repos\country-layer\integration-serverless\dea-windmill-integration\windmill-file-loader-lambda', 'C:\Users\YD9\Ap
pData\Local\Temp']
2019-05-07 08:28:25 Loading workflow module 'aws_lambda_builders.workflows'
2019-05-07 08:28:25 Registering workflow 'PythonPipBuilder' with capability 'Capability(language='python', dependency_manager='pip', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'NodejsNpmBuilder' with capability 'Capability(language='nodejs', dependency_manager='npm', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'RubyBundlerBuilder' with capability 'Capability(language='ruby', dependency_manager='bundler', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'GoDepBuilder' with capability 'Capability(language='go', dependency_manager='dep', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'GoModulesBuilder' with capability 'Capability(language='go', dependency_manager='modules', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'JavaGradleWorkflow' with capability 'Capability(language='java', dependency_manager='gradle', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'JavaMavenWorkflow' with capability 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2019-05-07 08:28:25 Registering workflow 'DotnetCliPackageBuilder' with capability 'Capability(language='dotnet', dependency_manager='cli-package', application_framework=None)'
2019-05-07 08:28:25 Found workflow 'JavaMavenWorkflow' to support capabilities 'Capability(language='java', dependency_manager='maven', application_framework=None)'
2019-05-07 08:28:26 Running workflow 'JavaMavenWorkflow'
2019-05-07 08:28:26 Running JavaMavenWorkflow:CopySource
2019-05-07 08:28:26 JavaMavenWorkflow:CopySource succeeded
2019-05-07 08:28:26 Running JavaMavenWorkflow:MavenBuild
2019-05-07 08:28:37 JavaMavenWorkflow:MavenBuild raised unhandled exception
Traceback (most recent call last):
File "C:\Program Files\Amazon\AWSSAMCLI\runtimelib\site-packages\aws_lambda_builders\workflow.py", line 248, in run
action.execute()
File "C:\Program Files\Amazon\AWSSAMCLI\runtimelib\site-packages\aws_lambda_builders\workflows\java_maven\actions.py", line 40, in execute
self.subprocess_maven.build(self.scratch_dir)
File "C:\Program Files\Amazon\AWSSAMCLI\runtimelib\site-packages\aws_lambda_builders\workflows\java_maven\maven.py", line 32, in build
LOG.debug("Maven logs: %s", stdout.decode('utf8').strip())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xe6 in position 5837: invalid continuation byte

Build Failed
Error: JavaMavenWorkflow:MavenBuild - 'utf-8' codec can't decode byte 0xe6 in position 5837: invalid continuation byte

Expected result

Build completes and lambda runs locally.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Win 10sa
2. sam --version:
   SAM CLI, version 0.15.0
   aws-cli/1.16.152 Python/3.6.0 Windows/10 botocore/1.12.142

Add --debug flag to command you are running

[kmathmann]

I have the same issue with .netcore 2.1.

Description

sam build fails with the error DotnetCliPackageBuilder:RunPackageAction - 'utf-8' codec can't decode byte 0x81 in position 570: invalid start byte.

I'm using the example code for .net-core 2.1 from sam init without any modifications.

Steps to reproduce

1. sam init --runtime dotnetcore2.1
2. sam build

Observed result

...
2019-05-14 21:25:01 Running workflow 'DotnetCliPackageBuilder'
2019-05-14 21:25:01 Running DotnetCliPackageBuilder:GlobalToolInstall
2019-05-14 21:25:01 Installing Amazon.Lambda.Tools Global Tool
2019-05-14 21:25:01 executing dotnet: ['dotnet.exe', 'tool', 'install', '-g', 'Amazon.Lambda.Tools']
2019-05-14 21:25:01
2019-05-14 21:25:01 Error installing probably due to already installed. Attempt to update to latest version.
2019-05-14 21:25:01 executing dotnet: ['dotnet.exe', 'tool', 'update', '-g', 'Amazon.Lambda.Tools']
2019-05-14 21:25:04 Das Tool "amazon.lambda.tools" wurde in der neuesten stabilen Version neu installiert (Version 3.2.3).
2019-05-14 21:25:04 DotnetCliPackageBuilder:GlobalToolInstall succeeded
2019-05-14 21:25:04 Running DotnetCliPackageBuilder:RunPackageAction
2019-05-14 21:25:04 Running `dotnet lambda package` in D:\dev\local\SAM-Test\src\HelloWorld
2019-05-14 21:25:04 executing dotnet: ['dotnet.exe', 'lambda', 'package', '--output-package', 'D:\\dev\\local\\SAM-Test\\.aws-sam\\build\\HelloWorldFunction\\HelloWorld.zip']
2019-05-14 21:25:11 DotnetCliPackageBuilder:RunPackageAction raised unhandled exception
Traceback (most recent call last):
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflow.py", line 248, in run
    action.execute()
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\actions.py", line 83, in execute
    cwd=self.source_dir
  File "C:\Program Files\Amazon\AWSSAMCLI\runtime\lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\dotnetcli.py", line 60, in run
    LOG.info(out.decode('utf8').strip())
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x81 in position 570: invalid start byte

Build Failed
Error: DotnetCliPackageBuilder:RunPackageAction - 'utf-8' codec can't decode byte 0x81 in position 570: invalid start byte

2019-05-14 21:25:04 executing dotnet: ['dotnet.exe', 'lambda', 'package', '--output-package', 'D:\\dev\\local\\SAM-Test\\.aws-sam\\build\\HelloWorldFunction\\HelloWorld.zip'] creates the zip file successful.

Expected result

I expect the build to be successful.

Additional environment details (Ex: Windows, Mac, Amazon Linux etc)

1. OS: Windows 10
2. sam --version: SAM CLI, version 0.16.0
3. dotnet --version: 2.1.603

Add --debug flag to command you are running

[kmathmann]

Commenting out the line 60 LOG.info(out.decode('utf8').strip()) in the file dotnetcli.py solves the issue for me.

C:\Program Files\Amazon\AWSSAMCLI\runtime\Lib\site-packages\aws_lambda_builders\workflows\dotnet_clipackage\dotnetcli.py

[mwysocki87]

Confirmed - commenting out the corresponding logging line also solves the problem for the Maven worflow. Great catch, @kmathmann!;) I do hope, however, that a permanent solution will be found.

[madmox]

Having the same issue here with dotnet2.1. Looks like the warning message that should have been printed contains non-ascii characters because my workstation is set up in french.

If I replace err.decode('utf8').strip() with err.decode('iso-8859-1').strip(), the message is:

L'outil 'amazon.lambda.tools' a ‚t‚ r‚install‚ avec la derniŠre version stable (version '3.2.3').

which, if I fix the accent errors, should be:

L'outil 'amazon.lambda.tools' a été réinstallé avec la dernière version stable (version '3.2.3').

which, translated into english, is:

The tool 'amazon.lambda.tools' was reinstalled with the latest stable version (version '3.2.3').

and is apparently NOT utf8 encoded, but rather encoded in something close to iso-8859-1 (thus err.decode('utf8') fails).

[CoshUS]

Same issue as https://github.com/awslabs/aws-sam-cli/issues/2031