Core: 0.59: iOS updates failing

Created on 3 Dec 2017 · 14Comments · Source: home-assistant/core

Home Assistant release (hass --version):
0.59

Python release (python3 --version):
3.5.2

Component/platform:
iOS

Description of problem:
Since updating to 0.59 updates from iOS app do not work.

Traceback (if applicable):

Dec 03 18:15:56 hass hass[31276]: 2017-12-03 18:15:56 INFO (MainThread) [homeassistant.components.http] Serving /api/ios/identify to 10.1.10.146 (auth: True)
Dec 03 18:15:56 hass hass[31276]: 2017-12-03 18:15:56 ERROR (MainThread) [homeassistant.util.json] Failed to serialize to JSON: /srv/homeassistant/.homeassistant/.ios.conf
Dec 03 18:15:56 hass hass[31276]: Traceback (most recent call last):
Dec 03 18:15:56 hass hass[31276]:   File "/srv/homeassistant/lib/python3.5/site-packages/homeassistant/util/json.py", line 38, in save_json
Dec 03 18:15:56 hass hass[31276]:     data = json.dumps(config, sort_keys=True, indent=4)
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/__init__.py", line 237, in dumps
Dec 03 18:15:56 hass hass[31276]:     **kw).encode(obj)
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 200, in encode
Dec 03 18:15:56 hass hass[31276]:     chunks = list(chunks)
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 429, in _iterencode
Dec 03 18:15:56 hass hass[31276]:     yield from _iterencode_dict(o, _current_indent_level)
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
Dec 03 18:15:56 hass hass[31276]:     yield from chunks
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
Dec 03 18:15:56 hass hass[31276]:     yield from chunks
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 403, in _iterencode_dict
Dec 03 18:15:56 hass hass[31276]:     yield from chunks
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 436, in _iterencode
Dec 03 18:15:56 hass hass[31276]:     o = _default(o)
Dec 03 18:15:56 hass hass[31276]:   File "/usr/lib/python3.5/json/encoder.py", line 179, in default
Dec 03 18:15:56 hass hass[31276]:     raise TypeError(repr(o) + " is not JSON serializable")
Dec 03 18:15:56 hass hass[31276]: TypeError: datetime.datetime(2017, 12, 3, 18, 15, 56, 111095) is not JSON serializable

Additional info:

Seems to have been introduced in #10677. I have confirmed that pip install -U 'homeassistant<0.59' "fixes" the issue.

Source

wrboyce

All 14 comments

May be this is related, but I am seeing slightly different errors:

Failed to serialize to JSON: /home/homeassistant/.homeassistant/.ios.conf
Traceback (most recent call last):
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/util/json.py", line 38, in save_json
    data = json.dumps(config, sort_keys=True, indent=4)
  File "/usr/lib/python3.6/json/__init__.py", line 238, in dumps
    **kw).encode(obj)
  File "/usr/lib/python3.6/json/encoder.py", line 201, in encode
    chunks = list(chunks)
  File "/usr/lib/python3.6/json/encoder.py", line 430, in _iterencode
    yield from _iterencode_dict(o, _current_indent_level)
  File "/usr/lib/python3.6/json/encoder.py", line 404, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.6/json/encoder.py", line 404, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.6/json/encoder.py", line 404, in _iterencode_dict
    yield from chunks
  File "/usr/lib/python3.6/json/encoder.py", line 437, in _iterencode
    o = _default(o)
  File "/usr/lib/python3.6/json/encoder.py", line 180, in default
    o.__class__.__name__)
TypeError: Object of type 'datetime' is not JSON serializable

arsaboo on 3 Dec 2017

That is the same error, the important bit being:

TypeError: Object of type 'datetime' is not JSON serializable

When writing the updated .ios.conf, hass is trying to convert a datetime object into a string which is not supported by the builtin json encoder.

wrboyce on 3 Dec 2017

I thought so too 👍

pinging @mnigbur so that he can investigate and hopefully fix the same.

arsaboo on 3 Dec 2017

Specifically, it is the lastSeenAt attribute. I seem to have it fixed locally, will send a PR soon.

wrboyce on 3 Dec 2017

👍1

The iOS code was using JSONEncoder which has handling for dates, sets, and dict-like objects. Perhaps that code should move to json.py so all consumers get that behaviour.

mnoorenberghe on 3 Dec 2017

@mnoorenberghe yeah, that is the approach I took.

wrboyce on 3 Dec 2017

👍1

@wrboyce Can we use the fix in custom_components until the hotfix is out?

arsaboo on 3 Dec 2017

That needs the fix to be self-contained inside the ios.py component, right? To do that, take the current ios.py, remove the save_json import, and add (something like) this:

from homebridge.remote import JSONEncoder

def save_json(filename: str, config: Union[List, Dict]):
    try:
        data = json.dumps(config, sort_keys=True, indent=4, cls=JSONEncoder)
        with open(filename, 'w', encoding='utf-8') as fdesc:
            fdesc.write(data)
            return True
    except TypeError as error:
        _LOGGER.exception('Failed to serialize to JSON: %s',
                          filename)
        raise HomeAssistantError(error)
    except OSError as error:
        _LOGGER.exception('Saving JSON file failed: %s',
                          filename)
        raise HomeAssistantError(error)
    return False

I think that should do you.

wrboyce on 3 Dec 2017

That did not work. Here's my ios.py and here is the stack trace:

Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/lib/python3.6/asyncio/tasks.py", line 182, in _step
    result = coro.throw(exc)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/components/notify/__init__.py", line 87, in async_setup_platform
    hass, config, DOMAIN, p_type)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 255, in async_prepare_setup_platform
    hass, config, platform_path, platform.DEPENDENCIES)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 118, in _async_process_dependencies
    results = yield from asyncio.gather(*tasks, loop=hass.loop)
  File "/usr/lib/python3.6/asyncio/futures.py", line 332, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
    future.result()
  File "/usr/lib/python3.6/asyncio/futures.py", line 245, in result
    raise self._exception
  File "/usr/lib/python3.6/asyncio/tasks.py", line 180, in _step
    result = coro.send(None)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 49, in async_setup_component
    return (yield from setup_tasks[domain])
  File "/usr/lib/python3.6/asyncio/futures.py", line 334, in __iter__
    return self.result()  # May raise too.
  File "/usr/lib/python3.6/asyncio/futures.py", line 245, in result
    raise self._exception
  File "/usr/lib/python3.6/asyncio/tasks.py", line 182, in _step
    result = coro.throw(exc)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 60, in async_setup_component
    return (yield from task)
  File "/usr/lib/python3.6/asyncio/futures.py", line 332, in __iter__
    yield self  # This tells Task to wait for completion.
  File "/usr/lib/python3.6/asyncio/tasks.py", line 250, in _wakeup
    future.result()
  File "/usr/lib/python3.6/asyncio/futures.py", line 245, in result
    raise self._exception
  File "/usr/lib/python3.6/asyncio/tasks.py", line 180, in _step
    result = coro.send(None)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/setup.py", line 144, in _async_setup_component
    component = loader.get_component(domain)
  File "/srv/homeassistant/lib/python3.6/site-packages/homeassistant/loader.py", line 142, in get_component
    module = importlib.import_module(path)
  File "/srv/homeassistant/lib/python3.6/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 994, in _gcd_import
  File "<frozen importlib._bootstrap>", line 971, in _find_and_load
  File "<frozen importlib._bootstrap>", line 955, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 665, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 678, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/home/homeassistant/.homeassistant/custom_components/ios.py", line 204, in <module>
    def save_json(filename: str, config: Union[List, Dict]):
NameError: name 'Union' is not defined

arsaboo on 3 Dec 2017

All right adding from typing import Union, List, Dict at the top worked. No longer getting the error.

arsaboo on 3 Dec 2017

👍1

For anyone interested, here's the ios.py that should fix the error until the next bugfix release.

arsaboo on 3 Dec 2017

There are few different errors...are you on discord to debug in realtime?

arsaboo on 3 Dec 2017

I can be, got a link handy?

wrboyce on 3 Dec 2017

https://discordapp.com/channels/330944238910963714/377667559303938050

arsaboo on 3 Dec 2017