Core: notify's data dictionary is no longer valid as of 0.25

[robbiet480]

Home Assistant release (hass --version): 0.25

Python release (python3 --version): 3.5.1

Component/platform: notify

Description of problem: Passing in a data dictionary is no longer possible as of 0.25.

Using this example configuration:

input_boolean:
  input_test:
    name: An input test
    initial: off
    icon: mdi:car

automation:
  - alias: Automation test
    trigger:
      platform: state
      entity_id: input_boolean.input_test
      state: 'on'
    action:
      service: notify.RobbiesGrowl
      data_template:
        message: >
                Test 123 {{ states.input_boolean.input_test }}
        data:
          push:
            sound: US-EN-Morgan-Freeman-Roommate-Is-Arriving.wav

and flipping the input_boolean.input_test causes this error: 16-08-09 14:46:36 ERROR (ThreadPool Worker 4) [homeassistant.core] Invalid service data for notify.RobbiesGrowl: expected dict for dictionary value @ data['data']. Got "OrderedDict([('push', OrderedDict([('sound', 'US-EN-Morgan-Freeman-Roommate-Is-Arriving.wav')]))])".

I investigated the issue and believe that it was caused by @fabianhjr's work to add typing through HA. If I had to point the finger at a specific changeset, I would guess it was caused by #2735.

This can be fixed by changing the accepted type for the data dictionary from dict to object here. I would make this change myself but am unsure of the ramifications of changing from dict to object.

Expected: Notify payloads with a data dictionary should be allowed.

Additional info:

I got a more detailed error string from Voluptuous by using it's builtin humanize_error function. I changed this from:

except vol.MultipleInvalid as ex:
            _LOGGER.error('Invalid service data for %s.%s: %s',
                          call.domain, call.service, ex)

to:

except vol.MultipleInvalid as ex:
            from voluptuous.humanize import humanize_error
            _LOGGER.error('Invalid service data for %s.%s: %s',
                          call.domain, call.service,
                          humanize_error(call.data, ex))

It may be good to use humanize_error so that less skilled users can get a little more debugging information? Thoughts on that @balloob?

[fabianhjr]

Need to investigate, currently on phone on my way home. Just a heads up, Python does not check/enforce typing by default and OrderedDict subsets ABC.Dict, so it should be accepted by the Dict type.

[fabianhjr]

Are you 100% sure that the error has nothing to do with https://github.com/home-assistant/home-assistant/blame/dev/homeassistant/components/notify/__init__.py#L41 ? #2506

EDIT: Also, 0.25.2 is 8 days old, the PR you referenced was merged 2 days ago.

[robbiet480]

@fabianhjr Will look at the first part of your response in a few, but for now I should've mentioned that i'm running on latest dev now. I identified the issue started at the 0.25 merge.

[robbiet480]

@fabianhjr Also, check out the blame on the line you linked, all it changed was removing a comment. Hasn't changed since April before that.

[balloob]

Making it object would be wrong, that's pretty much a catch all. OrderedDict is a dict, so that should work…

[balloob]

I did not know about humanize error, I am down to put that into as many places as possible, starting with config_validation.log_exception error

[robbiet480]

@balloob Will make that change momentarily and link the PR here.

[fabianhjr]

Just to be clear, 0.24.1 works, 0.25 doesn't?

[robbiet480]

@fabianhjr I assume you mean 0.24.1?

[fabianhjr]

Yes

Diff 0.24 -> 0.25.txt

[balloob]

I wrote a test to verify it works 👍

However, I think what is the problem here is the fact that it is a script calling a service, which results in YAML data being passed to the service. And YAML gives us ordered dicts.

[robbiet480]

Agree with @balloob. I was busy today and think that I missed something, because it's not working on 0.24.1 either. Sorry to loop you in @fabianhjr!

[robbiet480]

@balloob I believe that this issue only occurs when using data_template btw.

[fabianhjr]

Yeah, this is probably the function involved.

https://github.com/home-assistant/home-assistant/blame/dev/homeassistant/util/yaml.py#L106

Though, it returned an OrderedDict before it was typed up.

[fabianhjr]

No problem, that is what typing is for: To nail down where typing errors come up and not when they explode in our faces.

[balloob]

So I managed to write a test to repro this.

Still confused, this works:

from collections import OrderedDict
import voluptuous as vol

schema = vol.Schema({})
print(schema(OrderedDict()))

schema2 = vol.Schema({'hello': {}})
print(schema2({'hello': OrderedDict()}))

[balloob]

On to something. Looks like the actual value gets translated to a string… (Note quote before OrderedDict)

{'data': "OrderedDict([('push', OrderedDict([('sound', 'US-EN-Morgan-Freeman-Roommate-Is-Arriving.wav')]))])", 'message': 'Test 123 None'}

[robbiet480]

Now that I look at the error coming out of humanize_error I noticed the same thing. I figured it was quoting just to delineate the output, not that it actually was the output haha.

[balloob]

Found it: data is a dictionary and you're passing that in to data_template. data_template will put each value through template renderer, making it a string.

[robbiet480]

So how can I accomplish what I showed above, if at all?

[balloob]

unindent the data attribute, no need to run it through templating

[robbiet480]

_headdesk_

Thanks all!

[balloob]

Actually, you need to have a double data: in that case

[robbiet480]

So you're saying like this?

service: notify.RobbieiOSApp
data_template:
  message: >
          {{ trigger.to_state.attributes.friendly_name }} has arrived at home.
data:
  data:
    push:
      sound: "US-EN-Morgan-Freeman-Roommate-Is-Arriving.wav"

[balloob]

yes