Py3status: hddtemp utf-8 error

[cyrinux]

❯ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER="en_US.UTF-8"
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=en_US.UTF-8

❯ python -V
Python 3.6.5

❯ hddtemp -v
hddtemp version 0.3-beta15

❯ py3status -v
py3status version 3.11_rc0 (python 3.6.5)

'utf-8' codec can't decode byte 0x80 in position 51: invalid start byte (UnicodeDecodeError) hddtemp.py line 128.
Traceback
UnicodeDecodeError: 'utf-8' codec can't decode byte 0x80 in position 51: invalid start byte
  File "/usr/lib/python3.6/site-packages/py3status/module.py", line 732, in run
    response = method()
  File "/usr/lib/python3.6/site-packages/py3status/modules/hddtemp.py", line 128, in hddtemp
    line = Telnet('localhost', 7634).read_all().decode()[1:-1]

[lasers]

Can you try this?

diff --git a/py3status/modules/hddtemp.py b/py3status/modules/hddtemp.py
index 34ebde9e..02f30312 100644
--- a/py3status/modules/hddtemp.py
+++ b/py3status/modules/hddtemp.py
@@ -125,7 +125,7 @@ class Py3status:
         self.keys = ['path', 'name', 'temperature', 'unit']

     def hddtemp(self):
-        line = Telnet('localhost', 7634).read_all().decode()[1:-1]
+        line = Telnet('localhost', 7634).read_all().decode('utf-8', 'ignore')[1:-1]
         new_data = []

         for chunk in line.split('||'):

Would like to see an output of nc localhost 7634 too.

EDIT: Diff diff.

diff --git a/py3status/modules/hddtemp.py b/py3status/modules/hddtemp.py
index 34ebde9e..f6de21d9 100644
--- a/py3status/modules/hddtemp.py
+++ b/py3status/modules/hddtemp.py
@@ -125,7 +125,8 @@ class Py3status:
         self.keys = ['path', 'name', 'temperature', 'unit']

     def hddtemp(self):
-        line = Telnet('localhost', 7634).read_all().decode()[1:-1]
+        line = Telnet('localhost', 7634).read_all().decode(
+            'latin-1').encode('utf-8').decode()[1:-1]
         new_data = []

         for chunk in line.split('||'):

[cyrinux]

Hi @lasers,
good shot for the two try.

|/dev/sda|SK hynix SC311 SATA 512G B |36|C|

[lasers]

If first diff works okay, try replacing ignore with replace to see if the foreign marker shows up on the line (and where). If unrelated, we probably ought to strip the trailing space at end of your name.

[cyrinux]

with default "format" I have weird character with ignore or replace at the end of name

[cyrinux]

Moduletest_module: {'name': 'SK hynix SC311 SATA 512G B \x10', 'path': '/dev/sda', 'temperature': '40', 'unit': 'C'}

This fix:

❯ diff -u /tmp/hddtemp.py /usr/lib/python3.6/site-packages/py3status/modules/hddtemp.py
--- /tmp/hddtemp.py     2018-05-29 21:36:22.711105359 +0200
+++ /usr/lib/python3.6/site-packages/py3status/modules/hddtemp.py       2018-05-29 21:35:41.081344675 +0200
@@ -132,6 +132,7 @@
         for chunk in line.split('||'):
             hdd = dict(zip(self.keys, chunk.split('|')))
             self.py3.threshold_get_color(hdd['temperature'], u'temperature')
+            hdd['name'] = hdd['name'].rstrip('\x10').rstrip()
             new_data.append(self.py3.safe_format(self.format_hdd, hdd))

         format_separator = self.py3.safe_format(self.format_separator)

[lasers]

@cyrinux What about the latin-1 diff? You didn't say anything so I assume it did not show up.... And if it does not show up, then I assume we should use that along with a strip(). Otherwise, this diff might be better than stripping one thing. I'll ask @tobes tho.

diff --git a/py3status/modules/hddtemp.py b/py3status/modules/hddtemp.py
index 9b7d2163..74137928 100644
--- a/py3status/modules/hddtemp.py
+++ b/py3status/modules/hddtemp.py
@@ -109,6 +109,7 @@ compact
 """

 from telnetlib import Telnet
+from string import printable


 class Py3status:
@@ -126,12 +127,13 @@ class Py3status:
         self.keys = ['path', 'name', 'temperature', 'unit']

     def hddtemp(self):
-        line = Telnet('localhost', 7634).read_all().decode()[1:-1]
+        line = Telnet('localhost', 7634).read_all().decode('utf-8', 'ignore')
         new_data = []

         for chunk in line[1:-1].split('||'):
             hdd = dict(zip(self.keys, chunk.split('|')))
+            hdd['name'] = ''.join(
+                [x for x in hdd['name'] if x in printable]).strip()
             self.py3.threshold_get_color(hdd['temperature'], 'temperature')
             new_data.append(self.py3.safe_format(self.format_hdd, hdd))

@tobes What should we do here? Do we go with latin-1 diff + strip() or the diff above or do you have something else in mind? He have a trailing whitespace too.

[cyrinux]

Hi @lasers ,
for diff with latin-1 is was working like the other diff from my side.

In the last diff, which at my eyes seems clean,
you miss [1:-1] , like this it is working

diff --git a/py3status/modules/hddtemp.py b/py3status/modules/hddtemp.py
index 9b7d2163..74137928 100644
--- a/py3status/modules/hddtemp.py
+++ b/py3status/modules/hddtemp.py
@@ -109,6 +109,7 @@ compact
 """

 from telnetlib import Telnet
+from string import printable


 class Py3status:
@@ -126,12 +127,13 @@ class Py3status:
         self.keys = ['path', 'name', 'temperature', 'unit']

     def hddtemp(self):
-        line = Telnet('localhost', 7634).read_all().decode()[1:-1]
+        line = Telnet('localhost', 7634).read_all().decode('utf-8', 'ignore')[1:-1]
         new_data = []

         for chunk in line[1:-1].split('||'):
             hdd = dict(zip(self.keys, chunk.split('|')))
+            hdd['name'] = ''.join(
+                [x for x in hdd['name'] if x in printable]).strip()
             self.py3.threshold_get_color(hdd['temperature'], 'temperature')
             new_data.append(self.py3.safe_format(self.format_hdd, hdd))

[lasers]

Yeah, sorry. Diff on a diff in the last diff results in a bad diff. I moved it down to for chunk in line[1:-1].split('||'):. I hope we will close this issue soon enough. :-)

[cyrinux]

will for sure be closed before wwan one. ;)

[lasers]

will for sure be closed before wwan one. ;)

Nope. :-) Also, please close this. Xoxo. :heart:

[cyrinux]

Thanks, wwan win !