Core: Feedreader error on us-ascii/utf-8

[Mariusthvdb]

Home Assistant release with the issue:

102.2
Last working Home Assistant release (if known):

don't know
Operating environment (Hass.io/Docker/Windows/etc.):

hassio rpi4
Integration:

https://www.home-assistant.io/integrations/feedreader/

Description of problem:

trying the https://www.gdacs.org/xml/rss.xml feed in feedreader this is met with following error:

Error parsing feed https://www.gdacs.org/xml/rss.xml: document declared as us-ascii, but parsed as utf-8

seems the reader integration is too strict and needs some flexibility for the feeds to parse

Problem-relevant configuration.yaml entries and (fill out even if it seems unimportant):

feedreader:
  urls:
    - https://www.gdacs.org/xml/rss.xml

Traceback (if applicable):

Additional information:

[Mariusthvdb]

has there been any update on this issue please?
using 105.5 and still see:

2020-02-26 09:45:06 ERROR (SyncWorker_9) [homeassistant.components.feedreader] Error parsing feed https://www.gdacs.org/xml/rss.xml: document declared as us-ascii, but parsed as utf-8

sorry if I missed the obvious, please assist if I have to set a specific flag somewhere
thanks!
maybe can I tag you @exxamalte , since you've been developing the GDACS integration recently, would you know how to solve this too?

[exxamalte]

Quick explanation: That error message indicates that the library could not initially process the data correctly. However, the integration then takes a less strict approach to parse the information, and this often succeeds.
In the case of this GDCAS feed, the feed items are indeed processed and sent onto the event bus.

You could argue that we may want to change the log level from ERROR to WARNING, but that's as much as we can do in this case.

[Mariusthvdb]

Thanks, and please let me try to understand 100%:
are you saying the feedreader integration is setup, even with this 'error' message, and an automation like

automation:
  - alias: Notify RSS feed updated
    trigger:
      platform: event
      event_type: feedreader
    action:
      service: persistent_notification.create
      data_template:
        title: >
          {{ trigger.event.data.title }}
        message: >
          {{ trigger.event.data.author }} made a new Podcast on {{trigger.event.data.feed_url}}, see
          {{trigger.event.data.link}}
        notification_id: rss-feed

would still work on that feed? If so, lowering this to a warning would indeed be a good idea ;-) Do we need a FR for that?

btw, I now notice even the documentations own example for https://hasspodcast.io/feed/podcast causes the same error:

2020-02-26 23:00:44 ERROR (SyncWorker_1) [homeassistant.components.feedreader] Error parsing feed https://www.gdacs.org/xml/rss.xml: document declared as us-ascii, but parsed as utf-8
2020-02-26 23:00:44 ERROR (SyncWorker_15) [homeassistant.components.feedreader] Error parsing feed https://hasspodcast.io/feed/podcast: document declared as us-ascii, but parsed as utf-8

[exxamalte]

btw, I now notice even the documentations own example for https://hasspodcast.io/feed/podcast causes the same error:

Yeah, the underlying feedparser does a lot guessing when the character set is not explicitly set in the HTTP response. For some reason it registers this issue as an error even though it continues parsing the feed.

[Mariusthvdb]

ok cool, and btw, I can confirm the parsing of the feed while erroring on the feed itself (funny I should get my first persistent notification about HA 105 on the day 106 is released...)

changed the automation to:

automation:
  - alias: Notify RSS feed updated
    trigger:
      platform: event
      event_type: feedreader
    action:
      service: persistent_notification.create
      data_template:
        title: >
          {{ trigger.event.data.title }}
        message: >
          {% set url = trigger.event.data.feed_url.split('https://')[1] %}
          {% set source = {'www.home-assistant.io/atom.xml':'Home-assistant',
                           'hasspodcast.io/feed/podcast':'Hass podcast',
                           'www.gdacs.org/xml/rss.xml':'GDACS'} %}
          New Rss feed for {{source[url] if url in source else 'Unknown'}}, see
          {{trigger.event.data.link}},{{trigger.event.data.author}}
        notification_id: >
          {% set url = trigger.event.data.feed_url.split('https://')[1] %}
          {% set source = {'www.home-assistant.io/atom.xml':'Home-assistant',
                           'hasspodcast.io/feed/podcast':'Hass podcast',
                           'www.gdacs.org/xml/rss.xml':'GDACS'} %}
          rss-feed-{{source[url] if url in source else 'Unknown'}}

and the notification is:

the GDACS is active too:

note I have a {{trigger.event.data.author}} in the automation that isn't used (found that in an older community post). Isn't this data available on the feedreader events? I couldn't find which events are created, so tried it, apparently unsuccessfully

[Mariusthvdb]

just let me say thank you!

[Mariusthvdb]

might have to re-open this, as I still see the error on the Hass podcast, but also have to report it doesn't process the feed after all:

2020-03-05 16:24:34 ERROR (SyncWorker_14) [homeassistant.components.feedreader] Error parsing feed https://hasspodcast.io/feed/podcast: document declared as us-ascii, but parsed as utf-8

not a single podcast feed is reported.
now on 106.5, config unchanged from the above.

would appreciate if anyone could have a look and maybe test the podcast feed?

[exxamalte]

The fix for this has not yet been released.

I just tried the following configuration, and it processes the first 20 feed entries without problems.

feedreader:
  urls:
    - https://hasspodcast.io/feed/podcast

[Mariusthvdb]

Thanks for trying and reporting back Malte.

how odd though, and you're seeing this on current (non fixed)106.5? just retried that and no feed at all.

even added the exact automation from the docs, to be sure my templates don't cause any issues. Nothing.

[Omirax]

I have docker-ce and Home Assistant 0.107.4 with debian 9
When I use

feedreader:
  urls:
    - https://hasspodcast.io/feed/podcast

HA created new file feedreader.pickle in config directory with content:
_'utf-8' codec can't decode byte 0x80 in position 0: invalid start byte_