Telethon: can not encode to utf-8

[parazitt]

hi when i call the "get_dialogs" return an error:

Traceback (most recent call last):
File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
self.run()
File "/usr/lib/python3.5/threading.py", line 862, in run
self._target(self._args, *self._kwargs)
File "/usr/local/lib/python3.5/dist-packages/telethon/network/mtproto_sender.py", line 394, in updates_thread_method
reader)
File "/usr/local/lib/python3.5/dist-packages/telethon/network/mtproto_sender.py", line 225, in process_msg
return self.handle_gzip_packed(msg_id, sequence, reader, request)
File "/usr/local/lib/python3.5/dist-packages/telethon/network/mtproto_sender.py", line 356, in handle_gzip_packed
request)
File "/usr/local/lib/python3.5/dist-packages/telethon/network/mtproto_sender.py", line 243, in process_msg
return self.handle_update(msg_id, sequence, reader)
File "/usr/local/lib/python3.5/dist-packages/telethon/network/mtproto_sender.py", line 253, in handle_update
tlobject = reader.tgread_object()
File "/usr/local/lib/python3.5/dist-packages/telethon/utils/binary_reader.py", line 128, in tgread_object
result.on_response(self)
File "/usr/local/lib/python3.5/dist-packages/telethon/tl/types/updates_tg.py", line 64, in on_response
users_item = reader.tgread_object()
File "/usr/local/lib/python3.5/dist-packages/telethon/utils/binary_reader.py", line 128, in tgread_object
result.on_response(self)
File "/usr/local/lib/python3.5/dist-packages/telethon/tl/types/user.py", line 168, in on_response
self.last_name = reader.tgread_string()
File "/usr/local/lib/python3.5/dist-packages/telethon/utils/binary_reader.py", line 92, in tgread_string
return str(self.tgread_bytes(), encoding='utf-8')
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 6: invalid start byte

i check this problem and i found the "®" character in a user name.
i'll trying to resolve this problem and commit it

[Lonami]

UnicodeDecodeError: 'utf-8' codec can't decode byte 0xae in position 6: invalid start byte
i check this problem and i found the "®" character in a user name.

Using a Python shell:

>>> '®'.encode('utf-8')
b'\xc2\xae'

So yes, that's the issue :P, But it's definitely strange because I read all the bytes before actually converting it to a string. And I've never had issues sending or receiving any kind of strings?

i'll trying to resolve this problem and commit it

Awesome then, I'll leave it to you. Keep me updated :)

[parazitt]

tank you so much,
the Telethon is helpful for me. but i replace line 92 in utils/binary_reader.py with:

    try:
        return str(self.tgread_bytes(), encoding='utf-8')
    except:
        pass

and it worked fine :)

[Lonami]

I can't do that though. You should never simply ignore exceptions like that, and definitely not in a vital method like that. There has to be a better way to fix this.

[parazitt]

i know that, but for my target ignore is a best and shortest way. but for the others its may be Unpleasant,

[Lonami]

I tried adding ® to the name of one of my contacts, and I don't have such issue. Not sure how I could test this.

[Lonami]

Trying with the example given by #67, I tried copying the text of the offending message, for then pasting it back (on Telegram itself, for instance). I noticed that, instead of crashing, it shows a � character. So maybe I could remove the offending part and replace it?

[Lonami]

Every day I'm more surprised with Telegram :^)

[gtavasoli]

I found a solution. Simply add errors parameter to str object in method tgread_string.
The default value for this parameter is 'strict' that cause the exception.

def tgread_string(self):
    """Reads a Telegram-encoded string"""
    return str(self.tgread_bytes(), encoding='utf-8', errors='replace')

I guess it's better soloution than 3d66062. For more information (Unicode Python 3)

\ufffd is equivalent to �

[Lonami]

Absolutely, I came accross that but the code confused me so I ignored it. Would you like to make a pull request yourself @gtavasoli?

[ykaner]

Is this issue solved?
I am using the newest 1.0.3 version and still get a similar error. It's caused by the name of a channel that includes an emoji, in the stringify method.

Traceback (most recent call last):
File "Telethon1.py", line 12, in
print(contact.stringify())
File "C:\Users\ykaneAppData\Local\Programs\Python\Python35\lib\encodings\cp437.py", line 19, in encode
return codecs.charmap_encode(input,self.errors,encoding_map)[0]
UnicodeEncodeError: 'charmap' codec can't encode character '\U0001f30d' in position 597: character maps to

[Lonami]

That's an issue with your terminal, not the library.

[ykaner]

What do you mean my terminal? The platform is Python 3.5 on windows, what version I need to work clearly? Or maybe some dependencies?

[Lonami]

Look the error online. I'm sure it works without print.