Aws-cli: s3 sync/ls issues UnicodeWarning in Windows environment

[quiver]

What steps will reproduce the problem?

1. Log in to Windows(default language is non English(confirmed on Japanese))
2. Run `aws s3 ls s3:://bucketname/

What is the expected output? What do you see instead?

You will encounter UnicodeWarning as follows;

Bucket: bucketname
Prefix:

      LastWriteTime     Length Name
      -------------     ------ ----
                           PRE /
                           PRE test/
C:\Program Files\Amazon\AWSCLI\.\dateutil\parser.py:336: UnicodeWarning: Unicode  equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

2013-10-19 17:37:54         20 foobar
...

What version of the product are you using?

C:\>aws --version
aws-cli/1.2.0 Python/2.7.5 Windows/7

Additional information below.

This is not a bug of aws-cli per se and is already reported to python-dateutils.
https://bugs.launchpad.net/dateutil/+bug/1227221

But you encounter this warning every time you issue s3 commands, so it's a bit annoying.(awscli users are not always (Python) developers, right?)

Working around this warning would be appreciated. (When releasing awscli, suppressing warning with python -W ignore::UnicodeWarning is one option)

FYI, this warning also appears in #95(Which occurred in argparse module)

[ksdev-pl]

Same here (system language - Polish).

aws-cli/1.2.6 Python/2.7.5 Windows/7

c:\> aws s3 ls s3://bucket.pl
C:\Program Files\Amazon\AWSCLI\.\dateutil\parser.py:339: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal
2013-12-02 21:23:32        612 filename

[eerien]

I have same issue. (System language - Korean)

$ aws s3 ls
c:\Program Files\Amazon\AWSCLI\.\dateutil\parser.py:339: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

[skaurus]

+1. Russian.
Seems locale-related.

[JoernBerkefeld]

+1 german Win7, exactly the same message as eerien posted it after aws s3 sync s3://xxx s3://yyy
apart from the line being 428

aws-cli/1.7.41 Python/2.7.9 Windows/7

[quiver]

I found another workaround.

This issue is Python2(unicode nightmare!) specific and aws-cli installer is affected because it comes with Python 2.7.
So instead of the aws-cli bundled installer, first download Python3 and install aws-cli with $ pip install awscli.
Now you have no warnings :-)

On a side note, dateutil guys seem to have fixed the inherent bug:

• https://bugs.launchpad.net/dateutil/+bug/1227221
• dateutil/dateutil/issues/92

Hope this will solve the problem in the near future.

[Kainax]

+1 russian on CLI 1.9.7, Pyphon 2.7.9, Botocore 1.3.7, Win7 - @ 428.

[gustavohc91]

Hello there,

you guys can try to use " " instead ' ' between text clauses for example.

[Kainax]

More than one year past, and we still can't sync normally. Just reinstalled system from scratch, and still the same music.

aws --version
aws-cli/1.11.20 Python/2.7.9 Windows/8 botocore/1.4.77 (actually Windows 10, Russian)

C:\Program Files\Amazon\AWSCLI.dateutil\parser.py:605: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

Fix it, please!

[rmsys]

Kainax you are right. More than one year!

aws --version
aws-cli/1.11.21 Python/2.7.9 Windows/8 botocore/1.4.78 (actually Windows 10)

C:\Program Files\Amazon\AWSCLI.dateutil\parser.py:605: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

[arthurcohen]

three years! I can't believe that wasn't fixed.

How can I overpass this issue?

[souzabrunoftw]

Hi, I'm having the same issue.

aws --version
aws-cli/1.11.21 Python/2.7.9 Windows/8 botocore/1.4.78
Windows 10

Trying to sync a local folder to a S3 bucket.

C:\Program Files\Amazon\AWSCLI.dateutil\parser.py:605: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

[souzabrunoftw]

Any chance somebody is gonna fix it?

[arthurcohen]

well, my problem occurs when i use a boto3 package from python, that access the AWS CLI internally.

I solved this Warning putting this warning simple filter:

warnings.simplefilter("ignore", UnicodeWarning)

But I do not know how to solve this on AWS CLI, sorry guys.

[NK74]

+1 French

aws --version
aws-cli/1.11.35 Python/2.7.9 Windows/2012Server botocore/1.4.92

CLI x64 Windows Server 2012 R2

[ghenius]

Same problem with Windows 10 French. I am very very discouraged.

[NK74]

Up ?

[lenew]

Same problem with windows 10 zh-CN.

aws-cli/1.11.37 Python/2.7.9 Windows/8 botocore/1.5.0

Solved this problem by change dateutil with python-dateutil source code, and change parser.py at line 605 from
elif res.tzname and res.tzname in time.tzname:
to
elif res.tzname and res.tzname in [x.decode(sys.stdout.encoding) for x in time.tzname]:
and dont forget insert import sys at the head of this file

[enricforn]

I'm experiencing the same issue:

aws-cli/1.11.37 Python/2.7.9 Windows/8 botocore/1.5.0

[Sdcxv]

+1 Chinese

[statkclee]

+1 Korean

[psa-jforestier]

This bug is related to Python2 on Windows, when the timezone contain special characters (like accent, or asian glyph). Still not fixed : https://github.com/dateutil/dateutil/issues/289

[Kainax]

@psa-jforestier actually I have it both on Python 2 and on last tested Python 3.5.2 - also it is on Russian (cyrillic) not Asian.

[Sdcxv]

Here is my work around on Windows

1. Uninstall https://s3.amazonaws.com/aws-cli/AWSCLI64.msi
2. Install Python 3.6.1
3. Run pip install awscli
4. Run aws --version ,and you will get aws-cli/1.11.102 Python/3.6.1 Windows/10 botocore/1.5.65

[julianoappelklein]

My workaround - using Windows 10 - is just set the Windows user interface language to English.
This avoid the Python bug with timezone containing special characters.

[cglevican]

+1 Spanish

[rdricco]

+1 Brazilian Portuguese

[ASayre]

Good Morning!

We're closing this issue here on GitHub, as part of our migration to UserVoice for feature requests involving the AWS CLI.

This will let us get the most important features to you, by making it easier to search for and show support for the features you care the most about, without diluting the conversation with bug reports.

As a quick UserVoice primer (if not already familiar): after an idea is posted, people can vote on the ideas, and the product team will be responding directly to the most popular suggestions.

We’ve imported existing feature requests from GitHub - Search for this issue there!

And don't worry, this issue will still exist on GitHub for posterity's sake. As it’s a text-only import of the original post into UserVoice, we’ll still be keeping in mind the comments and discussion that already exist here on the GitHub issue.

GitHub will remain the channel for reporting bugs.

Once again, this issue can now be found by searching for the title on: https://aws.uservoice.com/forums/598381-aws-command-line-interface

-The AWS SDKs & Tools Team

[jamesls]

Based on community feedback, we have decided to return feature requests to GitHub issues.

[tinohager]

aws-cli/1.16.127 Python/2.7.15 Windows/10 botocore/1.12.117

C:\Program Files\Amazon\AWSCLI.dateutil\parser_parser.py:1177: UnicodeWarning: Unicode equal comparison failed to convert both arguments to Unicode - interpreting them as being unequal

Solve with install: https://s3.amazonaws.com/aws-cli/AWSCLI64PY3.msi

[github-actions[bot]]

Greetings! It looks like this issue hasn’t been active in longer than one year. We encourage you to check if this is still an issue in the latest release. Because it has been longer than one year since the last update on this, and in the absence of more information, we will be closing this issue soon. If you find that this is still a problem, please feel free to provide a comment to prevent automatic closure, or if the issue is already closed, please feel free to reopen it.