News: A (mostly) valid feed that can't be read

Created on 2 May 2019  路  8Comments  路  Source: nextcloud/news

Explain the Problem

When trying to add
http://yatuu.fr/feed/ which w3 say it's a valid feed
I get this error in the left panel, just where "Yatuu" appeared

LogicException: this document is not a XML stream in /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader/Document.php:95
Stack trace:
#0 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader/Document.php(71): FeedIo\Reader\Document->loadDomDocument()
#1 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Standard/Atom.php(54): FeedIo\Reader\Document->getDOMDocument()
#2 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(167): FeedIo\Standard\Atom->canHandle(Object(FeedIo\Reader\Document))
#3 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(153): FeedIo\Reader->getAccurateParser(Object(FeedIo\Reader\Document))
#4 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(138): FeedIo\Reader->parseDocument(Object(FeedIo\Reader\Document), Object(FeedIo\Feed))
#5 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(117): FeedIo\Reader->handleResponse(Object(FeedIo\Adapter\Guzzle\Response), Object(FeedIo\Feed))
#6 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/vendor/debril/feed-io/src/FeedIo/FeedIo.php(286): FeedIo\Reader->read('http://yatuu.fr...', Object(FeedIo\Feed), Object(DateTime))
#7 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/lib/Fetcher/FeedFetcher.php(77): FeedIo\FeedIo->read('http://yatuu.fr...')
#8 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/lib/Fetcher/Fetcher.php(68): OCA\News\Fetcher\FeedFetcher->fetch('http://yatuu.fr...', true, NULL, NULL, NULL)
#9 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/lib/Service/FeedService.php(116): OCA\News\Fetcher\Fetcher->fetch('http://yatuu.fr...', true, NULL, NULL, NULL)
#10 /var/snap/nextcloud/12753/nextcloud/extra-apps/news/lib/Controller/FeedController.php(157): OCA\News\Service\FeedService->create('http://yatuu.fr...', 7, 'refhi', NULL, NULL, NULL)
#11 /snap/nextcloud/12753/htdocs/lib/private/AppFramework/Http/Dispatcher.php(166): OCA\News\Controller\FeedController->create('http://yatuu.fr...', 7, NULL, NULL, NULL)
#12 /snap/nextcloud/12753/htdocs/lib/private/AppFramework/Http/Dispatcher.php(99): OC\AppFramework\Http\Dispatcher->executeController(Object(OCA\News\Controller\FeedController), 'create')
#13 /snap/nextcloud/12753/htdocs/lib/private/AppFramework/App.php(118): OC\AppFramework\Http\Dispatcher->dispatch(Object(OCA\News\Controller\FeedController), 'create')
#14 /snap/nextcloud/12753/htdocs/lib/private/AppFramework/Routing/RouteActionHandler.php(47): OC\AppFramework\App::main('OCA\\News\\Contro...', 'create', Object(OC\AppFramework\DependencyInjection\DIContainer), Array)
#15 [internal function]: OC\AppFramework\Routing\RouteActionHandler->__invoke(Array)
#16 /snap/nextcloud/12753/htdocs/lib/private/Route/Router.php(297): call_user_func(Object(OC\AppFramework\Routing\RouteActionHandler), Array)
#17 /snap/nextcloud/12753/htdocs/lib/base.php(987): OC\Route\Router->match('/apps/news/feed...')
#18 /snap/nextcloud/12753/htdocs/index.php(42): OC::handleRequest()
#19 {main}

System Information

  • News app version: 13.1.6
  • Nextcloud version:15.0.7
  • Distribution and version: ubuntul LTS 18.04, nextcloud snap 15
bug stale
Source
picture Refhi
👍1

Most helpful comment

I did some more digging and I've found out that the 3 characters at the begining are the Byte Order Mark. This has been fixed in the latest versions of the vendor library but not in the 3.x versions.

Modifying the constructor within /vendor/debril/feed-io/src/FeedIo/Reader/Document.php to match the one here addresses this problem as it removes the Byte order mark (and other invalid characters that may occur).

public function __construct(string $content)
{
    $invalid_characters = '/[^\x9\xa\x20-\xD7FF\xE000-\xFFFD]/';
    $content = preg_replace($invalid_characters, '', $content);
    $this->content = trim(str_replace("\xEF\xBB\xBF", '', $content));
}
picture alnixon on 5 May 2019
🎉1 👍1

All 8 comments

Same problem here. I added the feed of a local newspaper (www.tagblatt.de/Kreis%20T%C3%BCbingen.rss) and it's valid (used the validator from OP) but feedio fails.

LogicException: this document is not a XML stream in /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader/Document.php:95
Stack trace:
#0 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader/Document.php(71): FeedIo\Reader\Document->loadDomDocument()
#1 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Standard/Atom.php(54): FeedIo\Reader\Document->getDOMDocument()
#2 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(167): FeedIo\Standard\Atom->canHandle(Object(FeedIo\Reader\Document))
#3 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(153): FeedIo\Reader->getAccurateParser(Object(FeedIo\Reader\Document))
#4 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(138): FeedIo\Reader->parseDocument(Object(FeedIo\Reader\Document), Object(FeedIo\Feed))
#5 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/Reader.php(117): FeedIo\Reader->handleResponse(Object(FeedIo\Adapter\Guzzle\Response), Object(FeedIo\Feed))
#6 /var/www/html/custom_apps/news/vendor/debril/feed-io/src/FeedIo/FeedIo.php(286): FeedIo\Reader->read('https://www.tag...', Object(FeedIo\Feed), Object(DateTime))
#7 /var/www/html/custom_apps/news/lib/Fetcher/FeedFetcher.php(77): FeedIo\FeedIo->read('https://www.tag...')
#8 /var/www/html/custom_apps/news/lib/Fetcher/Fetcher.php(64): OCA\News\Fetcher\FeedFetcher->fetch('https://www.tag...', false, '', NULL, NULL)
#9 /var/www/html/custom_apps/news/lib/Service/FeedService.php(228): OCA\News\Fetcher\Fetcher->fetch('https://www.tag...', false, '', NULL, NULL)
#10 /var/www/html/custom_apps/news/lib/Service/FeedService.php(184): OCA\News\Service\FeedService->update(76, 'vdr')
#11 /var/www/html/custom_apps/news/lib/Utility/Updater.php(49): OCA\News\Service\FeedService->updateAll()
#12 /var/www/html/custom_apps/news/lib/Cron/Updater.php(52): OCA\News\Utility\Updater->update()
#13 /var/www/html/lib/private/BackgroundJob/Job.php(61): OCA\News\Cron\Updater->run(NULL)
#14 /var/www/html/cron.php(123): OC\BackgroundJob\Job->execute(Object(OC\BackgroundJob\JobList), Object(OC\Log))
#15 {main}"
v1r0x picture v1r0x on 3 May 2019

Sorry to chime in with a "and me", but another example if it helps is https://www.eveonline.com/rss
Same stack trace as above examples.

I've tried updating the vendor library to v3.1.5 but get the same result.

edit:
Interestingly if I copy the raw xml, obtained via wget, and copy it to a file on a server and try to add that then it's fine. So either the request headers via wget are different or the response headers from my server are.

edit:
Looking at the xml News obtains further, it's being prefixed with "茂" which trips up the detection of { or < for the content type. Stripping the first 3 characters from the content results in it working ok. I'm guessing this is an encoding flag but I don't know why it's getting in the content.

alnixon picture alnixon on 5 May 2019

I did some more digging and I've found out that the 3 characters at the begining are the Byte Order Mark. This has been fixed in the latest versions of the vendor library but not in the 3.x versions.

Modifying the constructor within /vendor/debril/feed-io/src/FeedIo/Reader/Document.php to match the one here addresses this problem as it removes the Byte order mark (and other invalid characters that may occur).

public function __construct(string $content)
{
    $invalid_characters = '/[^\x9\xa\x20-\xD7FF\xE000-\xFFFD]/';
    $content = preg_replace($invalid_characters, '', $content);
    $this->content = trim(str_replace("\xEF\xBB\xBF", '', $content));
}
alnixon picture alnixon on 5 May 2019
🎉1 👍1

@alnixon Thanks for investigating!

@ devs: Is there a reason you picked the 3.x branch in favor of the 4.x branch? The only reason I see is support of PHP 7 (as it is the min-version in info.xml), but it reached EOL in january...

v1r0x picture v1r0x on 5 May 2019

4.x doesn't work for NC 14 and 15 atm you can check https://github.com/nextcloud/server/issues/15225

If you are interested and using NC16 you can install a alpha version. It's in the release section and the app store

Grotax picture Grotax on 5 May 2019

Ok, makes sense. Didn't think of server-related issues.

I'm using the patched constructor and it works fine for me :)

v1r0x picture v1r0x on 5 May 2019

Agree that latest Alpha resolves this issue.

mentalinc picture mentalinc on 30 May 2019

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

stale[bot] picture stale[bot] on 20 Jun 2019
Was this page helpful?
0 / 5 - 0 ratings

Related issues

flurischt picture flurischt  路  3Comments
janumix picture janumix  路  7Comments
BlackIkeEagle picture BlackIkeEagle  路  3Comments
Emporea picture Emporea  路  8Comments
amacchietto picture amacchietto  路  7Comments