News: Feed processing regression on 13.1.0

[maralorn]

IMPORTANT

Read and tick the following checkbox after you have created the issue or place an x inside the brackets ;)

• [x] I have read the CONTRIBUTING.md and followed the provided tips

Explain the Problem

I was really, really happy and actually recommended nextcloud-news widely, when I started using it a few weeks ago and worked really, really nice with all the feeds I like to read (and they are a lot).

After updating to 13.1.0 this got much worse for a lot of feeds. I can only give examples:

• https://xkcd.com/rss.xml: Before I saw Image+Caption, now it doesn't show the caption anymore.
• http://blog.fefe.de/rss.xml?html: Before it rendered the html perfectly, now it displays the html source.
• https://www.lawblog.de/index.php/feed/: Before it showed the whole article. Now it only shows 3-lines and a "continue reading …" link.
• https://www.kuketz-blog.de/feed/: Same problem, it's not the full text anymore.
• https://matrix.org/blog/feed/: Same here (but there are still feeds, which show fulltext)

This might be a configuration problem on my side for every individual feed. But it worked so nicely out of the box before, so this regression is a real bummer. (Just a few weeks after finding the rss reader of my dreams ending years of suffering.)

Steps to Reproduce

Explain what you did to encounter the issue

1. Update to 13.1.0
2. Look at the above feeds

System Information

• News app version: 13.1.0
• Nextcloud version: 15.0.4
• PHP version: 7.1.26
• Database and version: postgresql-9.6.11
• Browser and version: Firefox and in the Android App
• Distribution and version: nixos 18.09

[fran-penedo]

Same issue here. Every feed with long articles is cut, and articles with html won't display properly.

[kernel1024]

Same here. A lot of feeds from blogs on FC2 and Blogger shows only title with hyperlink. No article text and no pictures.

[Grotax]

Hey there thank you for your post we know that stuff is broken.

After the discontinuation of a very important library picofeed we switched to feed-io which caused some problems that we didn't and in part couldn't catch before the release.

But don't worry we will continue to work on this app to end your suffering.

Please wait for the next release, also try the "Full article" feature in the web GUI.

No more "Same" posts please, it doesn't get more attention for your problem but is rather annoying for the maintainers of this app. If you have more information you we are happy to hear it. Otherwise please use the reactions thingy (:+1:) from GitHub to express your support for a certain post.

[thibaultamartin]

@Grotax would it be possible to switch back to picofeed in a 13.1.1 version so users can get back a functional app in its latest version? I would both allow admins to deploy the app in a working state without manually installing a previous version, and allow new installs of the app to work properly

[Luticus]

@Grotax, I actually get an internal server error when I try to toggle the "Full article" feature. I'll post the log if you like when I get home. Hope you guys get the issues sorted out. This is one of my favorite Nextcloud apps.

[Grotax]

@thibaultamartin I don't think that this would be a sensible thing to do, as picofeed is unmaintained for quite some time.

@Luticus That would be great.

[thibaultamartin]

@Grotax well, picoffeed being unsupported, things might break in the future. As of today, things are deeply broken for many users.

I do agree next versions should not be based on picofeed. But at least rolling back to the application based on picofeed during the stabilisation period of new iterations makes a lot of sense to me :)

[SMillerDev]

While I'd love to support everyones options, I had this running without issues for 2 months. There's simply too much variation in feeds to test everything and in that case the only option is to release.

[maralorn]

I must admit, I hadn’t read about the maintenance situation of this app before posting the bug.

So let me first emphasize, that I am really glad, that there is work being done on this app and I support dropping unmaintained libraries.

More to the point: I realized that my first complaint (xkcd) was a weak one. The image actually has the image caption in the feed. So I just need to hover the mouse over it. (as opposed to the caption being just written below the image like it was before.) That's a change, but it's reasonable.

So the remaining issues are:

• Wrong HTML handling in one feed. (http://blog.fefe.de/rss.xml?html). I know very little about rss but it reminds me of Mimetype problems in E-Mails. Any idea what could be going wrong?
• Only showing the first few lines in a lot of feeds. I am under the impression, that what picoFeed displayed was actually not the content of the feeds (I had a look at a sample and the full text was not in the feed) but already the scraped article (even though that option was not enabled.) So even though it felt like a regression to me, it might actually be a bug fix. I am speculating a bit here, but does this make sense?

So @SMillerDev, all my "bugs" seems to be boiling down to just one buggy feed and a change in default behaviour. No blame on you at all.

[maralorn]

If my speculations about the other bad behaving feeds is correct (It'd be awesome if somebody could confirm that picofeed did some unrequested magic there) this bug could be closed as a duplicate of #425.

[SMillerDev]

Seems like all your feeds were (unexpectedly) in full text mode. Though, as an xkcd fan myself I might make a special parser for it. The other issue seems to match #425 indeed

[maralorn]

I am very sure, that I didn't put my feeds in full text mode. And they weren't. So it is weird behavior. But since it's about a past version, we should just leave it at that.

[maralorn]

I just wanted to remark that as of 13.1.4 (http://blog.fefe.de/rss.xml?html) is now working fine again.

Thank you so much for working on this!

[jreybert]

I updated to last news 13.1.4, and restarted apache. The two last images of XKCD are not displayed, but older ones are displayed. FTR, I use ACPu cache, but I do not find any way to clear the cache (except restarting apache).

Is there still an issue with the news plugin?

[Grotax]

@jreybert that feed is still broken.

[maralorn]

@Grotax I want to it that it is not _still_ broken but just broke with 13.1.4. It worked before. I mentioned this in #445 to. Because this issue is closed and we shouldn't track open issues here.

[maralorn]

With 13.1.5. most of the feeds I compained above (mainly the wordpress-feeds I guess) now have working full text again (without having enabled the not-implemeted full-text-scraping feature of course) because of the content tag that is now respected again. That means nearly all my problems above have been addressed in just 2 weeks. You guys rock!

[Luticus]

@Grotax Well it seems one of the versions released since I posted this fixed the error I was seeing regarding the full text issue. Still have odd parsing errors with some engadget articles and it doesn't seem to show full text in the articles even though the full text menu option is enabled.

[Grotax]

Full text is currently not implemented.
You may create a new issue for your parsing errors.