Salt: Importing 'unicode_literals' in utils/aws.py causes unicode error during s3.get module execution

[atoy3731]

When unicode_literals was imported from __futures__ in utils/aws.py, it caused the s3 module to break.

Setup

None

Steps to Reproduce Issue

Command to run: salt '*' s3.get

Leads to the following stacktrace:

Traceback (most recent call last):
  File "/home/adam/IdeaProjects/salt-stack/salt/minion.py", line 1622, in _thread_return
    return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
  File "/home/adam/IdeaProjects/salt-stack/salt/executors/direct_call.py", line 12, in execute
    return func(*args, **kwargs)
  File "/home/adam/IdeaProjects/salt-stack/salt/modules/s3.py", line 204, in get
    https_enable=https_enable)
  File "/home/adam/IdeaProjects/salt-stack/salt/utils/s3.py", line 144, in query
    payload_hash=payload_hash,
  File "/home/adam/IdeaProjects/salt-stack/salt/utils/aws.py", line 259, in sig4
    for header in sorted(new_headers.keys(), key=str.lower):
  TypeError: descriptor 'lower' requires a 'str' object but received a 'unicode'

Versions Report

Salt Version:
Salt: 2018.2.0-77-gbb94a21

Dependency Versions:
cffi: Not Installed
cherrypy: Not Installed
dateutil: Not Installed
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
ioflo: Not Installed
Jinja2: 2.10
libgit2: Not Installed
libnacl: Not Installed
M2Crypto: Not Installed
Mako: Not Installed
msgpack-pure: Not Installed
msgpack-python: 0.5.1
mysql-python: Not Installed
pycparser: Not Installed
pycrypto: 2.6.1
pycryptodome: Not Installed
pygit2: Not Installed
Python: 2.7.6 (default, Oct 26 2016, 20:30:19)
python-gnupg: Not Installed
PyYAML: 3.12
PyZMQ: 16.0.3
RAET: Not Installed
smmap: Not Installed
timelib: Not Installed
Tornado: 4.5.3
ZMQ: 4.1.6

System Versions:
dist: Ubuntu 14.04 trusty
locale: UTF-8
machine: x86_64
release: 4.4.0-66-generic
system: Linux
version: Ubuntu 14.04 trusty

[gtmanfred]

Can you provide the full stack trace? it looks like the end of it with the actual error that was thrown has been chopped off.

Thanks,
Daniel

[atoy3731]

Unfortunately that was all that was available in my minion log. I can dig a little more to try to get the full stack trace when I get some time.

[gtmanfred]

There should be one more line after that that has the Exception that was raised

>>> raise Exception
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
Exception

Yours just stops at the File that the exception was in, but not the exception

[atoy3731]

Ah, sorry about that. You're right:

Traceback (most recent call last):
  File "/Users/Adam/PycharmProjects/saltstack/salt/minion.py", line 1622, in _thread_return
    return_data = minion_instance.executors[fname](opts, data, func, args, kwargs)
  File "/Users/Adam/PycharmProjects/saltstack/salt/executors/direct_call.py", line 12, in execute
    return func(*args, **kwargs)
  File "/Users/Adam/PycharmProjects/saltstack/salt/modules/s3.py", line 204, in get
    https_enable=https_enable)
  File "/Users/Adam/PycharmProjects/saltstack/salt/utils/s3.py", line 153, in query
    payload_hash=payload_hash,
  File "/Users/Adam/PycharmProjects/saltstack/salt/utils/aws.py", line 259, in sig4
    for header in sorted(new_headers.keys(), key=str.lower):
TypeError: descriptor 'lower' requires a 'str' object but received a 'unicode'

[terminalmage]

This should be fixable using six.text_type.lower instead of str.lower. @twangboy is working on a fix now.

[rallytime]

@atoy3731 Can you give the fix in #45647 a try? That should fix this issue for you.

[atoy3731]

Yep, works perfectly now. Thanks!

[rallytime]

Great! Thanks for confirming.