Trying to reistablish connection with WhatsApp contacts now that whatsapp-purple is abandoned, I tried yowsup once more. Unfortunately, it crashes shortly after logon. Sometimes, a few messages are actually received prior to the UnicodeEncodeError. Revision is todays pip version installed by pip install yowsup2 --upgrade.
$ yowsup-cli demos -l REDACTED -y
yowsup-cli v2.0.15
yowsup v2.5.0
Copyright (c) 2012-2016 Tarek Galal
http://www.openwhatsapp.org
This software is provided free of charge. Copying and redistribution is
encouraged.
If you appreciate this software and you would like to support future
development please consider donating:
http://openwhatsapp.org/yowsup/donate
Yowsup Cli client
==================
Type /help for available commands
[offline]:/L
Auth: Logged in!
[connected]:WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
WARNING:yowsup.layers.axolotl.layer_receive:Received a message that we've previously decrypted, goint to send the delivery receipt myself
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
Traceback (most recent call last):
File "/usr/local/bin/yowsup-cli", line 368, in <module>
if not parser.process():
File "/usr/local/bin/yowsup-cli", line 268, in process
self.startCmdline()
File "/usr/local/bin/yowsup-cli", line 297, in startCmdline
stack.start()
File "/usr/local/lib/python2.7/dist-packages/yowsup/demos/cli/stack.py", line 26, in start
self.stack.loop(timeout = 0.5, discrete = 0.5)
File "/usr/local/lib/python2.7/dist-packages/yowsup/stacks/yowstack.py", line 188, in loop
asyncore.loop(*args, **kwargs)
File "/usr/lib/python2.7/asyncore.py", line 216, in loop
poll_fun(timeout, map)
File "/usr/lib/python2.7/asyncore.py", line 156, in poll
read(obj)
File "/usr/lib/python2.7/asyncore.py", line 87, in read
obj.handle_error()
File "/usr/lib/python2.7/asyncore.py", line 83, in read
obj.handle_read_event()
File "/usr/lib/python2.7/asyncore.py", line 449, in handle_read_event
self.handle_read()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/layer.py", line 102, in handle_read
self.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/layer.py", line 110, in receive
self.toUpper(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 29, in receive
self.processReceived()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 52, in processReceived
self.processReceived()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 49, in processReceived
self.toUpper(oneMessageData)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/auth/layer_crypt.py", line 65, in receive
self.toUpper(payload)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/coder/layer.py", line 35, in receive
self.toUpper(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/logger/layer.py", line 14, in receive
self.toUpper(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_control.py", line 44, in receive
self.toUpper(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 189, in receive
s.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 41, in receive
self.onMessage(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 74, in onMessage
self.handleEncMessage(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 86, in handleEncMessage
self.handlePreKeyWhisperMessage(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 129, in handlePreKeyWhisperMessage
self.parseAndHandleMessageProto(pkMessageProtocolEntity, plaintext[:-padding])
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 188, in parseAndHandleMessageProto
self.handleConversationMessage(node, m.conversation)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 217, in handleConversationMessage
self.toUpper(messageNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 189, in receive
s.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 125, in receive
recv(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_messages/layer.py", line 21, in recvMessageStanza
self.toUpper(entity)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/interface/interface.py", line 80, in receive
self.entity_callbacks[entityType](entity)
File "/usr/local/lib/python2.7/dist-packages/yowsup/demos/cli/layer.py", line 512, in onMessage
MESSAGE_ID = message.getId()
UnicodeEncodeError: 'ascii' codec can't encode characters in position 0-1: ordinal not in range(128)
this seems to happen for voice messages. It tries to decode it as ascii I think
I doubt it's just for voice messages. Here is another one:
$ yowsup-cli demos -l REDACTED -y
yowsup-cli v2.0.15
yowsup v2.5.0
Copyright (c) 2012-2016 Tarek Galal
http://www.openwhatsapp.org
This software is provided free of charge. Copying and redistribution is
encouraged.
If you appreciate this software and you would like to support future
development please consider donating:
http://openwhatsapp.org/yowsup/donate
Yowsup Cli client
==================
Type /help for available commands
[offline]:/L
Auth: Logged in!
[connected]:WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
WARNING:yowsup.layers.axolotl.layer_receive:Received a message that we've previously decrypted, goint to send the delivery receipt myself
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
[[email protected](03-06-2016 19:57)]:[B11319BA20EF7EF5251B9B0525E642] Flunky find ich auch gut
Message B11319BA20EF7EF5251B9B0525E642: Sent delivered receipt and Read
[connected]:
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
[[email protected](03-06-2016 19:57)]:[C1DDCE04FBE553BFDC31FEC0F1D2DE] kein Problem
Message C1DDCE04FBE553BFDC31FEC0F1D2DE: Sent delivered receipt and Read
[connected]:
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
[[email protected](03-06-2016 19:57)]:[A7A96B879A8FE32B981DAB264F514E] Top
Message A7A96B879A8FE32B981DAB264F514E: Sent delivered receipt and Read
[connected]:
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
[[email protected](03-06-2016 19:57)]:[0F69D97DCBED20090BF0069CFC8E0B] Wir maxbe und uns auf den Weg zum Fischers Park
Message 0F69D97DCBED20090BF0069CFC8E0B: Sent delivered receipt and Read
[connected]:
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
[[email protected](03-06-2016 19:57)]:[0AE69C5EF9D3192A4455D657233693] Geschmacklich aber himmelweiter Untefschied
Message 0AE69C5EF9D3192A4455D657233693: Sent delivered receipt and Read
[connected]:
WARNING:axolotl.sessionbuilder:We've already setup a session for this V3 message, letting bundled message fall through...
Traceback (most recent call last):
File "/usr/local/bin/yowsup-cli", line 368, in <module>
if not parser.process():
File "/usr/local/bin/yowsup-cli", line 268, in process
self.startCmdline()
File "/usr/local/bin/yowsup-cli", line 297, in startCmdline
stack.start()
File "/usr/local/lib/python2.7/dist-packages/yowsup/demos/cli/stack.py", line 26, in start
self.stack.loop(timeout = 0.5, discrete = 0.5)
File "/usr/local/lib/python2.7/dist-packages/yowsup/stacks/yowstack.py", line 188, in loop
asyncore.loop(*args, **kwargs)
File "/usr/lib/python2.7/asyncore.py", line 216, in loop
poll_fun(timeout, map)
File "/usr/lib/python2.7/asyncore.py", line 156, in poll
read(obj)
File "/usr/lib/python2.7/asyncore.py", line 87, in read
obj.handle_error()
File "/usr/lib/python2.7/asyncore.py", line 83, in read
obj.handle_read_event()
File "/usr/lib/python2.7/asyncore.py", line 449, in handle_read_event
self.handle_read()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/layer.py", line 102, in handle_read
self.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/network/layer.py", line 110, in receive
self.toUpper(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 29, in receive
self.processReceived()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 52, in processReceived
self.processReceived()
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/stanzaregulator/layer.py", line 49, in processReceived
self.toUpper(oneMessageData)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/auth/layer_crypt.py", line 65, in receive
self.toUpper(payload)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/coder/layer.py", line 35, in receive
self.toUpper(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/logger/layer.py", line 14, in receive
self.toUpper(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_control.py", line 44, in receive
self.toUpper(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 189, in receive
s.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 41, in receive
self.onMessage(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 74, in onMessage
self.handleEncMessage(protocolTreeNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 86, in handleEncMessage
self.handlePreKeyWhisperMessage(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 129, in handlePreKeyWhisperMessage
self.parseAndHandleMessageProto(pkMessageProtocolEntity, plaintext[:-padding])
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 188, in parseAndHandleMessageProto
self.handleConversationMessage(node, m.conversation)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/axolotl/layer_receive.py", line 217, in handleConversationMessage
self.toUpper(messageNode)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 189, in receive
s.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 125, in receive
recv(node)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/protocol_messages/layer.py", line 21, in recvMessageStanza
self.toUpper(entity)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/usr/local/lib/python2.7/dist-packages/yowsup/layers/interface/interface.py", line 80, in receive
self.entity_callbacks[entityType](entity)
File "/usr/local/lib/python2.7/dist-packages/yowsup/demos/cli/layer.py", line 512, in onMessage
MESSAGE_ID = message.getId()
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe4' in position 58: ordinal not in range(128)
As you can see, messages are received. But things fall apart as soon someone sends me a message containing a German umlaut (ä in this example). I assume, the error occurrs when writing messages to the terminal. Python might try to encode with ASCII although my terminal uses UTF-8.
@hoehermann are you sure if encrypted media in groups is supported by master code?
@jlguardi No, I am unaware aof the support of encrypted media in group chats. As you state in https://github.com/jlguardi/yowsup/issues/87#issuecomment-224777859 , there is none.
By looking at the trace, I cannot relate this problem to encrypted media in a group chat. If this is the case, I still would like yowsup to ignore the message and print a warning rather than crashing alltogether. Problem persists in f510d44 .
I am currently investigating this issue. For future reference, receiving an image in an encrypted group chat triggers an exception being handled and printed as
DUMP:
<binary data>
<list of char>
<list of int>
Traceback (most recent call last):
File "./yowsup-cli", line 368, in <module>
if not parser.process():
File "./yowsup-cli", line 268, in process
self.startCmdline()
File "./yowsup-cli", line 297, in startCmdline
stack.start()
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/demos/cli/stack.py", line 26, in start
self.stack.loop(timeout = 0.5, discrete = 0.5)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/stacks/yowstack.py", line 188, in loop
asyncore.loop(*args, **kwargs)
File "/usr/lib/python2.7/asyncore.py", line 216, in loop
poll_fun(timeout, map)
File "/usr/lib/python2.7/asyncore.py", line 156, in poll
read(obj)
File "/usr/lib/python2.7/asyncore.py", line 87, in read
obj.handle_error()
File "/usr/lib/python2.7/asyncore.py", line 83, in read
obj.handle_read_event()
File "/usr/lib/python2.7/asyncore.py", line 449, in handle_read_event
self.handle_read()
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/network/layer.py", line 102, in handle_read
self.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/network/layer.py", line 110, in receive
self.toUpper(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/stanzaregulator/layer.py", line 29, in receive
self.processReceived()
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/stanzaregulator/layer.py", line 49, in processReceived
self.toUpper(oneMessageData)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/auth/layer_crypt.py", line 65, in receive
self.toUpper(payload)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/coder/layer.py", line 35, in receive
self.toUpper(node)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/logger/layer.py", line 14, in receive
self.toUpper(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_control.py", line 44, in receive
self.toUpper(protocolTreeNode)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 76, in toUpper
self.__upper.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/__init__.py", line 189, in receive
s.receive(data)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_receive.py", line 41, in receive
self.onMessage(protocolTreeNode)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_receive.py", line 74, in onMessage
self.handleEncMessage(protocolTreeNode)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_receive.py", line 86, in handleEncMessage
self.handlePreKeyWhisperMessage(node)
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_receive.py", line 129, in handlePreKeyWhisperMessage
self.parseAndHandleMessageProto(pkMessageProtocolEntity, plaintext[:-padding])
File "/media/data/Eigene/Programmieren/Kompilieren/yowsup_tgalal/yowsup/layers/axolotl/layer_receive.py", line 171, in parseAndHandleMessageProto
m.ParseFromString(serializedData)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/message.py", line 185, in ParseFromString
self.MergeFromString(serialized)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 1080, in MergeFromString
if self._InternalParse(serialized, 0, length) != length:
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 1116, in InternalParse
pos = field_decoder(buffer, new_pos, end, self, field_dict)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/decoder.py", line 633, in DecodeField
if value._InternalParse(buffer, pos, new_pos) != new_pos:
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/python_message.py", line 1116, in InternalParse
pos = field_decoder(buffer, new_pos, end, self, field_dict)
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/decoder.py", line 502, in DecodeField
field_dict[key] = _ConvertToUnicode(buffer[pos:new_pos])
File "/usr/local/lib/python2.7/dist-packages/google/protobuf/internal/decoder.py", line 469, in _ConvertToUnicode
return local_unicode(byte_str, 'utf-8')
UnicodeDecodeError: 'utf8' codec can't decode byte 0xff in position 0: 'utf8' codec can't decode byte 0xff in position 0: invalid start byte in field: com.whatsapp.proto.UrlMessage.jpeg_thumbnail
This differs from my examples above.
I am now on f510d44 using python 2.7. I helped myself by editing yowsup/demos/cli/layer.py, surrounding the flaky call to self.__class__.MESSAGE_FORMAT.format with a try-except block. A less than ideal solution, but at least I can follow conversations now. Exceptions occur when receiving emoji in messeges like
Hot Town, Summer in da City 🍻🍻☀☀☀☀
I really needed those. :[
@hoehermann
I have a solution for you.
change line 511 of yowsup/demos/cli/layer.py file by:
MESSAGE = messageOut.encode('latin-1').decode() if sys.version_info >= (3, 0) else messageOut.encode('utf-8'),
if you want to see the nicknames or aliases in the "cli". add here the nickname and change this line by:
FROM = self.jidToAlias(sender),
I did not try the jidToAlias hint, but your proposal for layer.py is looking very good. I want to test it some more, but it looks stable.
god bless your soul! adding the .encode worked for me. been struggling the whole afternoon
MESSAGE = messageOut.encode('latin-1').decode() if sys.version_info >= (3, 0) else messageOut.encode('utf-8')
For me it worked changing line 511 as follows:
MESSAGE = messageOut.encode('utf-8').decode() if sys.version_info >= (3, 0) else messageOut
But it still crash when receiving files (image, video, PDF, etc)
I'm using Python version 3.6
Most helpful comment
@hoehermann
I have a solution for you.
change line 511 of
yowsup/demos/cli/layer.pyfile by:MESSAGE = messageOut.encode('latin-1').decode() if sys.version_info >= (3, 0) else messageOut.encode('utf-8'),if you want to see the nicknames or aliases in the "cli". add here the nickname and change this line by:
FROM = self.jidToAlias(sender),