aws-cli s3 cp\sync 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

Created on 14 May 2014  路  11Comments  路  Source: aws/aws-cli

Hello!

aws-cli/1.3.10 Python/2.7.5 Windows/8

Today I tried upload my files to s3 and I have this error on all my files:
'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)

If --no-guess-mime-type enabled, is no errors.

Command: aws s3 cp . s3://...

Log with debug:

2014-05-14 13:55:05,302 - botocore.hooks - DEBUG - Event service-data-loaded.s3: calling handler
2014-05-14 13:55:05,305 - botocore.hooks - DEBUG - Event service-created: calling handler
2014-05-14 13:55:05,305 - botocore.handlers - DEBUG - Registering retry handlers for service: Service(s3)
2014-05-14 13:55:05,306 - awscli.customizations.s3.tasks - DEBUG - 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
Traceback (most recent call last):
File "awscli\customizations\s3\tasks.pyc", line 88, in _execute_task
File "awscli\customizations\s3\fileinfo.pyc", line 219, in upload
File "awscli\customizations\s3\fileinfo.pyc", line 191, in _handle_object_params
File "awscli\customizations\s3\fileinfo.pyc", line 227, in _inject_content_type
File "awscli\customizations\s3\utils.pyc", line 211, in guess_content_type
File "mimetypes.pyc", line 294, in guess_type
File "mimetypes.pyc", line 355, in init
File "mimetypes.pyc", line 259, in read_windows_registry
File "mimetypes.pyc", line 249, in enum_types
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe0 in position 0: ordinal not in range(128)
2014-05-14 13:55:05,309 - awscli.customizations.s3.executor - DEBUG - Submitting task:
2014-05-14 13:55:05,312 - botocore.service - DEBUG - Creating service object for: s3
2014-05-14 13:55:05,312 - awscli.customizations.s3.executor - DEBUG - Worker thread invoking task:
2014-05-14 13:55:05,312 - awscli.customizations.s3.executor - DEBUG - Received print task: {'message': u"upload failed: www.cdn.ecwid-tools.com\documentation\commercial_plugins\layer_slider\assets
\js\syntaxhighlighter\compass\shThemeDjango.scss to s3://cdn.ecwid-tools.com/documentation/commercial_plugins/layer_slider/assets/js/syntaxhighlighter/compass/shThemeDjango.scss 'ascii' codec can'
t decode byte 0xe0 in position 0: ordinal not in range(128)", 'error': True}

s3 s3sync
Source
picture kroniak

Most helpful comment

same problem:

# aws s3 ls s3://sa-prod-data/ 
'ascii' codec can't encode characters in position 31-40: ordinal not in range(128)

# aws --version
aws-cli/1.11.147 Python/2.6.6 Linux/2.6.32-642.el6.x86_64 botocore/1.7.5

# cat /etc/centos-release 
CentOS release 6.8 (Final)

# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
picture vbem on 7 Sep 2017
👍5

All 11 comments

I'm experiencing this issue as well, with a fresh install of aws-cli version 1.7.22 on Ubuntu 12 with Python 2.7.3. Here are the packages/dependencies that were pip installed in a clean virtual environment when I pip installed awscli:

argparse==1.2.1
awscli==1.7.22
bcdoc==0.13.0
botocore==0.103.0
colorama==0.3.3
docutils==0.12
jmespath==0.6.1
ply==3.4
pyasn1==0.1.7
python-dateutil==2.4.2
rsa==3.1.4
six==1.9.0
wsgiref==0.1.2

The command resulting in the same error as above was, in my case: aws s3 ls

I should also note that none of my bucket names have any special/non-ascii characters.

Any advice would be appreciated!

kerchner picture kerchner on 14 Apr 2015

@kerchner Could you post a scrubbed --debug log?

jamesls picture jamesls on 14 Apr 2015

@jamesls sure:

$ aws s3 ls --debug

2015-04-14 17:44:32,041 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.7.22 Python/2.7.3 Linux/3.2.0-76-generic, botocore version: 0.103.0
2015-04-14 17:44:32,042 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'ls', '--debug']
2015-04-14 17:44:32,042 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x1a87c08>
2015-04-14 17:44:32,042 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider at 0x1a4a500>
2015-04-14 17:44:32,044 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <function add_waiters at 0x1a522a8>
2015-04-14 17:44:32,045 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.s3.anonymous: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,046 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ls: calling handler <function add_waiters at 0x1a522a8>
2015-04-14 17:44:32,047 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.paths: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,048 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.summarize: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,048 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthand object at 0x1ac3590>
2015-04-14 17:44:32,048 - MainThread - awscli.argprocess - DEBUG - Detected structure: scalar
2015-04-14 17:44:32,049 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.anonymous: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,049 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.human-readable: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,050 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthand object at 0x1ac3590>
2015-04-14 17:44:32,050 - MainThread - awscli.argprocess - DEBUG - Detected structure: scalar
2015-04-14 17:44:32,050 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.page-size: calling handler <function uri_param at 0x1841d70>
2015-04-14 17:44:32,051 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2015-04-14 17:44:32,051 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2015-04-14 17:44:32,051 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2015-04-14 17:44:32,052 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2015-04-14 17:44:32,126 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2015-04-14 17:44:32,128 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2015-04-14 17:44:32,141 - MainThread - botocore.hooks - DEBUG - Event before-call.s3.ListBuckets: calling handler <function add_expect_header at 0x15c8f50>
2015-04-14 17:44:32,142 - MainThread - botocore.endpoint - DEBUG - Making request for <botocore.model.OperationModel object at 0x1adab50> (verify_ssl=True) with params: {'query_string': '', 'headers': {}, 'url_path': '/', 'body': '', 'method': u'GET'}
2015-04-14 17:44:32,142 - MainThread - botocore.hooks - DEBUG - Event request-created.s3.ListBuckets: calling handler <bound method s3._sign_request of <botocore.client.s3 object at 0x1dd4b50>>
2015-04-14 17:44:32,143 - MainThread - botocore.hooks - DEBUG - Event before-sign.s3.ListBuckets: calling handler <function fix_s3_host at 0x15c8c08>
2015-04-14 17:44:32,143 - MainThread - botocore.handlers - DEBUG - Checking for DNS compatible bucket for: https://s3.amazonaws.com/
2015-04-14 17:44:32,143 - MainThread - botocore.handlers - DEBUG - Not changing URI, bucket is not DNS compatible: 
2015-04-14 17:44:32,144 - MainThread - botocore.auth - DEBUG - Calculating signature using hmacv1 auth.
2015-04-14 17:44:32,144 - MainThread - botocore.auth - DEBUG - HTTP request method: GET
2015-04-14 17:44:32,145 - MainThread - botocore.auth - DEBUG - StringToSign:
GET


Tue, 14 Apr 2015 21:44:32 GMT
/
2015-04-14 17:44:32,146 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "awscli/clidriver.py", line 183, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "awscli/customizations/commands.py", line 190, in __call__
    parsed_globals)
  File "awscli/customizations/commands.py", line 187, in __call__
    return self._run_main(parsed_args, parsed_globals)
  File "awscli/customizations/s3/subcommands.py", line 324, in _run_main
    self._list_all_buckets()
  File "awscli/customizations/s3/subcommands.py", line 383, in _list_all_buckets
    response_data = self.client.list_buckets()
  File "botocore/client.py", line 191, in _api_call
    return self._make_api_call(operation_name, kwargs)
  File "botocore/client.py", line 235, in _make_api_call
    operation_model, request_dict)
  File "botocore/endpoint.py", line 173, in make_request
    return self._send_request(request_dict, operation_model)
  File "botocore/endpoint.py", line 199, in _send_request
    request = self.create_request(request_dict, operation_model)
  File "botocore/endpoint.py", line 183, in create_request
    operation_name=operation_model.name)
  File "botocore/hooks.py", line 226, in emit
    return self._emit(event_name, kwargs)
  File "botocore/hooks.py", line 209, in _emit
    response = handler(**kwargs)
  File "botocore/client.py", line 278, in _sign_request
    self._request_signer.sign(operation_name, request)
  File "botocore/signers.py", line 93, in sign
    signer.add_auth(request=request)
  File "botocore/auth.py", line 591, in add_auth
    "AWS %s:%s" % (self.credentials.access_key, signature))
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 4: ordinal not in range(128)
2015-04-14 17:44:32,152 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

'ascii' codec can't decode byte 0xe2 in position 4: ordinal not in range(128)
kerchner picture kerchner on 14 Apr 2015

Oh that's interesting. It looks like the access_key has a non ascii character in it. Are you able to make calls to other AWS services with the CLI? If not it would suggest it's an issue with the access_key.

jamesls picture jamesls on 14 Apr 2015
👍2

That's it! Thank you @jamesls. I opened ~/.aws/credentials and indeed there is a non-ascii character prefixing my access key value (a character which should not be there). Perhaps it got in there while cutting & pasting into the configuration dialog.

To make sure that aws config wasn't writing the configuration file wrong, I re-ran aws configure and re-entered the access_key value. It works now. Thanks again.

kerchner picture kerchner on 15 Apr 2015

Hello, I have the same issue when using awscli. I already tried the same thing as @kerchner but my config does not contain any non ascii char, so the solution didn't actually work.

I'm on a MacOS El Capitan (10.11.6) and awscli was installed via Homebrew:

$ aws --version
aws-cli/1.11.5 Python/2.7.10 Darwin/15.6.0 botocore/1.4.62

When running a simple aws s3 ls I get the following output (debug mode enabled):

$ aws s3 ls --debug
2016-10-17 16:46:02,285 - MainThread - awscli.clidriver - DEBUG - CLI version: aws-cli/1.11.5 Python/2.7.10 Darwin/15.6.0 botocore/1.4.62
2016-10-17 16:46:02,286 - MainThread - awscli.clidriver - DEBUG - Arguments entered to CLI: ['s3', 'ls', '--debug']
2016-10-17 16:46:02,286 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function add_scalar_parsers at 0x10675a668>
2016-10-17 16:46:02,286 - MainThread - botocore.hooks - DEBUG - Event session-initialized: calling handler <function inject_assume_role_provider_cache at 0x1064ee9b0>
2016-10-17 16:46:02,286 - MainThread - botocore.credentials - DEBUG - Skipping environment variable credential check because profile name was explicitly set.
2016-10-17 16:46:02,286 - MainThread - botocore.hooks - DEBUG - Event building-command-table.s3: calling handler <function add_waiters at 0x106762758>
2016-10-17 16:46:02,287 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.s3.anonymous: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,287 - MainThread - botocore.hooks - DEBUG - Event building-command-table.ls: calling handler <function add_waiters at 0x106762758>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.paths: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.human-readable: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthandParser object at 0x106760f10>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.page-size: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.anonymous: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.request-payer: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event load-cli-arg.custom.ls.summarize: calling handler <function uri_param at 0x106390848>
2016-10-17 16:46:02,288 - MainThread - botocore.hooks - DEBUG - Event process-cli-arg.custom.ls: calling handler <awscli.argprocess.ParamShorthandParser object at 0x106760f10>
2016-10-17 16:46:02,288 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: env
2016-10-17 16:46:02,289 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: assume-role
2016-10-17 16:46:02,289 - MainThread - botocore.credentials - DEBUG - Looking for credentials via: shared-credentials-file
2016-10-17 16:46:02,289 - MainThread - botocore.credentials - INFO - Found credentials in shared credentials file: ~/.aws/credentials
2016-10-17 16:46:02,289 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/data/endpoints.json
2016-10-17 16:46:02,299 - MainThread - botocore.loaders - DEBUG - Loading JSON file: /usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/data/s3/2006-03-01/service-2.json
2016-10-17 16:46:02,317 - MainThread - botocore.client - DEBUG - Registering retry handlers for service: s3
2016-10-17 16:46:02,319 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_post at 0x1060a9050>
2016-10-17 16:46:02,319 - MainThread - botocore.hooks - DEBUG - Event creating-client-class.s3: calling handler <function add_generate_presigned_url at 0x1060a2758>
2016-10-17 16:46:02,320 - MainThread - botocore.args - DEBUG - The s3 config key is not a dictionary type, ignoring its value of: None
2016-10-17 16:46:02,323 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/clidriver.py", line 186, in main
    return command_table[parsed_args.command](remaining, parsed_args)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/customizations/commands.py", line 190, in __call__
    parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/customizations/commands.py", line 187, in __call__
    return self._run_main(parsed_args, parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/customizations/s3/subcommands.py", line 455, in _run_main
    super(ListCommand, self)._run_main(parsed_args, parsed_globals)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/customizations/s3/subcommands.py", line 441, in _run_main
    parsed_globals.verify_ssl)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/awscli/customizations/s3/subcommands.py", line 434, in get_client
    config=config)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/session.py", line 824, in create_client
    client_config=config, api_version=api_version)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/client.py", line 69, in create_client
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/client.py", line 222, in _get_client_args
    verify, credentials, scoped_config, client_config, endpoint_bridge)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/args.py", line 70, in get_client_args
    timeout=(new_config.connect_timeout, new_config.read_timeout))
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/endpoint.py", line 277, in create_endpoint
    proxies=self._get_proxies(endpoint_url),
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/endpoint.py", line 286, in _get_proxies
    return get_environ_proxies(url)
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/vendored/requests/utils.py", line 534, in get_environ_proxies
    if should_bypass_proxies(url):
  File "/usr/local/Cellar/awscli/1.11.5/libexec/lib/python2.7/site-packages/botocore/vendored/requests/utils.py", line 511, in should_bypass_proxies
    if netloc.endswith(host) or netloc.split(':')[0].endswith(host):
UnicodeDecodeError: 'ascii' codec can't decode byte 0xe2 in position 3: ordinal not in range(128)
2016-10-17 16:46:02,325 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255

Shall I create a new issue for this or can be considered part of the same problem?

sapka12 picture sapka12 on 17 Oct 2016

Same problem as @sapka12 - seems to be a different problem to me.

AWS CLI in Alpine 3.4, aws-cli/1.11.20 Python/2.7.12 Linux/4.4.30-moby botocore/1.4.77

joerx picture joerx on 23 Nov 2016

Found it - env gave me this:

no_proxy=1锟斤拷v锟斤拷*.local, 169.254/16, localhost, 127.0.0.1, 192.168.99/16

There are my non-ascii characters. export no_proxy='169.254/16, localhost, 127.0.0.1, 192.168.99/16' fixed the problem.

Now I just need to figure out where this comes from. Not related to awscli though.

joerx picture joerx on 23 Nov 2016

same problem:

# aws s3 ls s3://sa-prod-data/ 
'ascii' codec can't encode characters in position 31-40: ordinal not in range(128)

# aws --version
aws-cli/1.11.147 Python/2.6.6 Linux/2.6.32-642.el6.x86_64 botocore/1.7.5

# cat /etc/centos-release 
CentOS release 6.8 (Final)

# locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
vbem picture vbem on 7 Sep 2017
👍5

I solved this problem by configuring aws config:
aws configure
Obvious, but may help to someone new in aws (like me)

x-bogdanovich-x picture x-bogdanovich-x on 19 Apr 2019

I encountered this issue and solved it by setting the locale to a locale that supports UTF-8.

Unix/Linux scripts which will run as super user will most likely run in the C locale.

The C locale is an ASCII 7 bit locale which doesn't support Unicode characters. The Python run time which gets used by the AWS CLI will throw an error message because it can't convert Unicode characters.

The simplest way to avoid this error is to start any script using AWS CLI with a set locale like:

export LC_ALL=en_US.UTF-8

source

anthonysgevans picture anthonysgevans on 10 Mar 2020
Was this page helpful?
0 / 5 - 0 ratings

Related issues

alexejk picture alexejk  路  3Comments
rahul003 picture rahul003  路  3Comments
braddr picture braddr  路  3Comments
KimberleySDU picture KimberleySDU  路  3Comments
ehammond picture ehammond  路  3Comments