Core: Filter Sensor does not generate reasonable value in case of missing update.

[DieterChmod]

Home Assistant release with the issue: 0.85.1

Last working Home Assistant release (if known):

Operating environment (Hass.io/Docker/Windows/etc.):

Raspberry Pi 3B+
Raspbian 9.6 (stretch)
piVCCU 2.41.5-40 (homematic)
HmIP-SWO-B (weather sensor)

Component/platform:

Filter Sensor

Description of problem:
I'm using the filter sensor (time_simple_moving_average) for flattening my wind speed sensor values. When there's no wind at all, the sensor stays at a non-zero value until there is wind again, where it suddenly drops to zero and gets back to a reasonable value (see blue line in the attached screen shot).

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):
In my sensors.yaml I configure the filter sensor:

- platform: filter
  name: Windgeschwindigkeit Durchschnitt
  entity_id: sensor.wind_speed_ms
  filters:
    - filter: time_simple_moving_average
      window_size: 00:30
      precision: 2

In my configuration.yaml I add a history graph:

history_graph:
  wind_speed:
    name: Windgeschwindigkeit
    entities:
      - sensor.windgeschwindigkeit_durchschnitt
      - sensor.wind_speed_ms
      - sensor.wetter_heute_wind_speed
    hours_to_show: 12

Traceback (if applicable):

Additional information:
My personal guess is, that this has nothing to do with the filter component itself, but rather with missing updates, due to zero values or no changes.

[dgomes]

This is actually a feature of the Time SMA

It calculates the moving average of received values for a given time window. Since you have a very large time window with no received values, that average is obviously 0.

Probably you could use the Low Pass Filter to achieve better results, as it does not depend on the time.

[DieterChmod]

Maybe I wasn't explicit enough when I wrote the report at the first time. Sorry for that.

The blue line - which is actually the time SMA - is not 0.
The green line is the input data series.

If the blue line would reach 0 I would not write this bug report.

Please don't close an issue before asking back the author. I can imagine, that you have lots of bug reports, but I'm also spending my time to improve HASS.

[dgomes]

I understand your point is about the value of ~0.8m/s at ~8h30

The current implementation is built to not reset the values https://github.com/home-assistant/home-assistant/blob/master/homeassistant/components/sensor/filter.py#L473

It keeps the last_leak for indefinite amount of time in order not to reset the SMA if there are no values in the time window (this was purpose built).

I only see here the option to add a configuration parameter to disable this behavior.

[DieterChmod]

Alright, I think I got it. Maybe this additional parameter would be really nice. :)

However, it feels strange to me, that the input data curve falls down to zero, but the filter component does not recognize it. It should be able to handle zero values as well, don't you think so?

If you don't want to change or enhance the filter, you could maybe think about adding some information about this behavior to the documentation, because I guess that it is not obvious to the common user.

Thanks a lot. :)

[dgomes]

It never recognizes because of the last_leak property that was built to withstand void time windows, it behaves as a memory for the period of time the sensor got no updates.

Another way of "fixing" would be for the original sensor to publish the 0 values.

Would you like to have a go implementing/documenting :) ?

[DieterChmod]

Sure. I would be happy if I could contribute. Maybe I need some help from you... and some time. ;)
I'll get back to you, maybe we can have a chat.

[dgomes]

I'm available on discord using the same handle

[DieterChmod]

I think, fixing the sensor behavior does not make sense, since this would require all kinds of sensors potentially to be changed. This would probably also have an impact on other components.

I meanwhile added the lowpass filter. Interestingly, tt shows the same behavior during calm.

Thus, the time behavior of the SMA seems not to be the culprit.

[dgomes]

The LP filter does keep the bucket indefinitely too (I just checked the code)

I was not proposing changing sensor behavior, I was proposing change the TimeSMAFilter to drop the last_leak value and use 0 (per configuration option)

[stale[bot]]

There hasn't been any activity on this issue recently. Due to the high number of incoming GitHub notifications, we have to clean some of the old issues, as many of them have already been resolved with the latest updates.
Please make sure to update to the latest Home Assistant version and check if that solves the issue. Let us know if that works for you by adding a comment 👍
This issue now has been marked as stale and will be closed if no further activity occurs. Thank you for your contributions.