News: can not add valid feed

[retarepo]

What problem did you encounter?
after last update the "getHashUrl()" error disappeared but a new "Can not add feed: Unable to parse feed" appeared, when I try to add any arxiv.org feed, even with different rss versions, which they offer.

Steps to Reproduce

Explain what you did to encounter the issue

1. add new feed, e.g.: http://export.arxiv.org/rss/nucl-th
2. get error message
3. check validity of feed: https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fexport.arxiv.org%2Frss%2Fnucl-th
4. get frustrated...

System Information

• News app version: 14.1.4
• Nextcloud version: 18.0.4
• Cron type: system cron
• PHP version: 7.3
• Database and version: 10.0.38-MariaDB-0ubuntu0.16.04.1
• Browser and version: chromium Version 81.0.4044.122 (Official Build) snap (64-bit)
• Distribution and version: ubuntu 19.10 (client) Ubuntu 16.04 LTS (cloud server)

Contents of nextcloud/data/nextcloud.log

nothing related

Contents of Browser Error Console

Read http://ggnome.com/wiki/Using_The_Browser_Error_Console if you are unsure what to put here

nothing interesting

[Grotax]

My first guess is that feed-io can't deal with the "Implausible date: 1901-01-01T00:00+00:00"
Which of course shouldn't be there. If you want to analyze this more and achieve a fix the right place would be https://github.com/alexdebril/feed-io.

But remember that there are so many little bugs in feeds produced by the authors and their systems we can't fix them all. We are not Google or feedly :D

[retarepo]

after many years of programming and using any kind of software this is the very first time I ever filed a bug report and I wouldn't have done it before an excessive use of any kind of search engine.

arxiv.org offers the possibility to choose the feed version, where most of them don't show this "recommendation" in the validator, but others. see:
https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fexport.arxiv.org%2Frss%2Fnucl-th%3Fversion%3D0.91
https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fexport.arxiv.org%2Frss%2Fnucl-th%3Fversion%3D1.0
https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fexport.arxiv.org%2Frss%2Fnucl-th%3Fversion%3D2.0
https://validator.w3.org/feed/check.cgi?url=http%3A%2F%2Fexport.arxiv.org%2Frss%2Fnucl-th%3Fversion%3Datom_1.0

anyway none of the versions work in nc news.
So feed-io might not be able to handle any of those issues?

[Grotax]

Yea that's very good thank you for not just dumping your problem but doing some research.
And yea seems like the feed is failing I just have no idea why I tried using feed-io on the cli and the return value is empty so yea news can't do anything with that and fails ...

➜  feed-io ./vendor/bin/feedio read http://export.arxiv.org/rss/nucl-th\?version\=atom_1.0

---
➜  feed-io ./vendor/bin/feedio read http://export.arxiv.org/rss/nucl-th        

And that's where my knowledge ends :/

[pirovc]

I also have the same issues with arxiv feeds. I think arxiv is quite used in the scientific community and would be very nice to have it working on the app.

[retarepo]

So i made some tests (downloading the xml file and changing it until it works) and it turns out to be the date in the line as suggested

<syn:updateBase>1901-01-01T00:00+00:00</syn:updateBase>

so while just modifying the date I figured that the validator complains about everything before 1990, what is probably due to the fact that xml is from 1999. regardless, the parser works when the year is after 1970. So I assume that feed-io is converting it into a timestamp and can not deal with a negative value.
since I do not know xml enough to know the purpose of that particular element, I can not judge how much non-sense this line is, but I am going to report this problem to both parties:

• feed-io, since they should catch it as exception on not let the entire thing crash
• arxiv, since the value is obviously non-sense and should be changed to something more compatible.

as a workaround for now I wrote a small php wrapper which reads the entire file and changes the one line on-the-fly, so I can use it for now.

thanks for the input.

[Grotax]

@retarepo I appreciate your investigation it really helps to make OpenSource better. No matter if axiv fixes it or feed-io can catch it.

If you want you can share your wrapper here.

[SMillerDev]

Can someone try contacting arxiv about this?

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.

[pirovc]

@retarepo could you please share your php wrapper?

[retarepo]

@pirovc I am actually kinda embarrassed by it's crudeness ;) Just for now it fulfills its purpose and unfortunately I do not find any time to help solving the actual problem.

<?php
header( "Content-type: text/xml");
$file = file_get_contents ($_GET["feed"]);
$file = str_replace("<syn:updateBase>1901-01-01T00:00+00:00</syn:updateBase>", "<syn:updateBase>1991-01-01T00:00+00:00</syn:updateBase>", $file);
echo $file;
?>

So it simply replaces the given date, afair I was testing the dates a bit, and everything from 1970 works without error (so obviously a negative timestamp is the problem) and everything beyond 1990 doesnt alarm the w3c validator.

[SMillerDev]

Did anyone ever try and contact the feed creators?

[trougnouf]

Did anyone ever try and contact the feed creators?

I sent [email protected] an email today.

edit:

Dear Benoit Brummer,

Thank you for the notification. We are aware of this issue, and at some stage in the future we hope to have more feature-rich feeds.

Regards,
arXiv admin

[alexdebril]

it's fixed with feed-io version 4.7.12 which now supports items published before the industrial revolution. Please note that I refuse to go beyond Gutenberg's printing press.

[SMillerDev]

Thanks @alexdebril !