Paramiko: UnicodeDecodeError: 'utf8' codec can't decode byte 0x83 in position 20: invalid start byte

[aliceinwire]

paramiko version 1.16.0
when reading the stdout of a windows machine i get this error:

Traceback (most recent call last):
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/pysoup/pysoup/build.py", line 67, in build
    mod(args)
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/pysoup/pysoup/env/WIN.py", line 19, in __init__
    self.start()
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/pysoup/pysoup/env/WIN.py", line 98, in start
    self.build_processMLwin(self.HOSTMLwin[cnt])
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/pysoup/pysoup/env/WIN.py", line 253, in build_processMLwin
    for line in stdout:
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/lib/python2.7/site-packages/paramiko/file.py", line 102, in next
    line = self.readline()
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/lib/python2.7/site-packages/paramiko/file.py", line 270, in readline
    return line if self._flags & self.FLAG_BINARY else u(line)
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/lib/python2.7/site-packages/paramiko/py3compat.py", line 53, in u
    return s.decode(encoding)
  File "/home/buildmaster/.virtualenvs/feature_virtualenv/lib/python2.7/encodings/utf_8.py", line 16, in decode
    return codecs.utf_8_decode(input, errors, True)
UnicodeDecodeError: 'utf8' codec can't decode byte 0x83 in position 20: invalid start byte
CRITICAL:pysoup.build:error: 
'
<type 'exceptions.UnicodeDecodeError'>

having a encoding option when calling ssh.exec_command(command)
would be nice.

[renpj]

I have the same problem when use sftp.listdir() from Windows server. Maybe a bug?

[sanseihappa]

With paramiko==2.0.2, I am also seeing this error on a Mac (and also using an implementation of an SFTP server):

File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/paramiko/transport.py", line 558, in start_server raise e UnicodeDecodeError: 'utf8' codec can't decode byte 0x8a in position 13: invalid start byte

[jmons]

So I think I'm seeing this today, when using a lib thats wrapping around paramiko. It looks like the code is using open(b'--redacted path--', 'r') and I think it needs to be open(path, 'rb') to allow for binary files, (based on logging output of paramiko.transport.sftp). Hope that helps.

[wzhongke]

The decode function has a parameter errors="ignore", so I think you can change py3compat.py. In function u(s, encoding='utf8'), there is s.decode(encoding). I think you can modify like this: s.decode(encoding, errors="ignore"). But this will throw the error bytes away.

[mnalsup]

@jmons solution worked for me