Freshrss: Cannot add this feed - validator say is ok

[planetahuevo]

https://ayudawp.com/feed/

http://validator.w3.org/feed/check.cgi?url=https://ayudawp.com/feed/

Not sure why...

Any ideas?

Thanks!

[planetahuevo]

Another one: same issue.
http://validator.w3.org/feed/check.cgi?url=https://tellado.es/feed/

[Frenzie]

Hah, it sure has a very helpful error in the log.

2017-08-23 15:12:08Unknown error for feed [https://ayudawp.com/feed/]

You could try adding #force_feed to the end of the URL (see #1410). However, the mimetype on both of them seems to be application/rss+xml so the root cause is different.

[planetahuevo]

Tried and not working but at least I have a nicer error log. :)
I have the same problem with this other feed from the same person (server issue?)
```
2017-08-23 14:22:29

cURL error 22: The requested URL returned error: 400 Bad Request [https://ayudawp.com/feed/]

2017-08-23 12:55:30

cURL error 22: The requested URL returned error: 400 Bad Request [https://tellado.es/feed/]```

[Frenzie]

It seems fine when I try something like curl -v -A "SimplePie/1.7" https://ayudawp.com/feed/ so I'm not really sure what could be happening.

It works in rss-bridge on my same server, so if it is SimplePie-related then it's only in the FreshRSS version. /rss-bridge/?action=display&bridge=Filter&url=https%3A%2F%2Fayudawp.com%2Ffeed%2F&filter=&filter_type=permit&format=Atom, you can try it on a public instance here.

[planetahuevo]

No idea, this goes far beyond my knowledge on code.
I have found other issues adding sites but those didn´t pass the validator, so I didn´t think the problem is related.
Is there anything I can test or help?

[Frenzie]

You could play around with the curl parameters but right now I don't have the time to investigate which ones are relevant or even where they're located. And maybe use var_dump() or print_r() on the output, as well. I like to use error_log(print_r($relevant_var, true)) but if all of that is meaningless to you then the fastest option is probably to wait for @Alkarex to notice this. :-P

[planetahuevo]

Not a hurry at all. Thank you for looking into it. 👍
I will try to understand what you wrote there but maybe I should wait. :)

[planetahuevo]

One more feed with issues, so you can test it:
https://maugelves.com/feed/

[Frenzie]

That page is different. It blocks SimplePie.

curl -v -A "SimplePie/1.7" https://maugelves.com/feed/

            <h1>The page you are trying to access is restricted due to a security rule.</h1>
    <br>
            <h2>If you believe the security rule is affecting the normal operation of your website, contact your host support team and provide detailed instructions how to recreate this error.
<br>
They will be able to assist you with rectifying the problem and adjusting the security configuration if needed.</h2>

[Alkarex]

Related fix https://github.com/FreshRSS/FreshRSS/issues/1627

[planetahuevo]

Another one. https://enlacepermanente.es/feed/

Not sure where the issue is, but for me 70% of my feeds are not working.

[Frenzie]

Another security rule.

$ curl -v -A "SimplePie/1.3" https://enlacepermanente.es/feed/
*   Trying 37.60.237.110...
* TCP_NODELAY set
* Connected to enlacepermanente.es (37.60.237.110) port 443 (#0)

[…]

            <h1>The page you are trying to access is restricted due to a security rule.</h1>
    <br>
            <h2>If you believe the security rule is affecting the normal operation of your website, contact your host support team and provide detailed instructions how to recreate this error.
<br>
They will be able to assist you with rectifying the problem and adjusting the security configuration if needed.</h2>

[planetahuevo]

Hi,
I found a "solution". I am using this free rss aggregator to mix all the feeds that freshrss do not like and creating a "mixed rss".
http://www.rssmix.com/
That mixed rss is accepted on freshrss so Problem solved for me.
Not the best but until this is resolved (maybe using another system and not simplepie) is the only solution I think.

[Frenzie]

I just noticed that the feed (at least the last one) also works on rss-bridge. The workaround is basically the one already proposed in #1627:

https://github.com/RSS-Bridge/rss-bridge/blob/e30ad3feb44a5a55ce687583a5744f4cce5978a0/index.php#L87-L89

PS You can also put rss-bridge on the same server as FreshRSS (or another one :-P) and simply pass it through a filter-less filter like so: rss-bridge/?action=display&bridge=Filter&url=https%3A%2F%2Fenlacepermanente.es%2Ffeed%2F&filter=&filter_type=permit&format=Atom

PPS Of course I wrote the Filter for filtering. One of the feeds I subscribe to is rss-bridge/?action=display&bridge=Filter&url=http%3A%2F%2Fwww.arte.tv%2Fpapi%2Ftvguide-flow%2Ffeeds%2Fvideos%2Ffr.xml%3Ftype%3DARTE_PLUS_SEVEN%26player%3Dtrue&filter=Karambolage&filter_type=permit&format=Atom (i.e., the whole Arte feed but only the Karambolage program). But if you don't pass a filter it can serve as a proxy of sorts, just like that rssmix website.

I think rss-bridge is a great companion to FreshRSS, or any feed reader for that matter. :-)

[planetahuevo]

Well, I agree that is another workaround, but having to install another service and make sure is updated and secure is not something I am looking for.
This should be incorporated into FreshRSS somehow (no idea how, out of my knowledge I am afraid :( ).
But thank you for the alternative! :)

[Frenzie]

Sure, it's not intended for this purpose. That's just a side-effect.

[Alkarex]

There are different categories of problems, such as:

1. Servers blocking some user-agents, typically "SimplePie". This is likely due to abuses, with people refreshing too often. This seems to be the larger category of problems and will be addressed with https://github.com/FreshRSS/FreshRSS/issues/1627 and I will propose another patch to remove SimplePie from the user-agent globally.
2. Servers blocking normal HTTP requests (e.g. cURL) with CloudFlare. This seems to be a relatively small number of feeds, and I am not so inclined to implement any workaround for this case.
3. Server blocking your IP. Seems to be rare, and probably a sign that the user has been refreshing to often. A fix is to use a proxy / bridge on another IP.
4. Invalid feeds, for various reasons.
5. Feed that can work with cURL, but not with the cURL parameters used by FreshRSS. This seems to be rare, and requires further investigation. E.g. https://github.com/FreshRSS/FreshRSS/issues/1494

[Alkarex]

@planetahuevo Please give a try to this patch if you can https://github.com/FreshRSS/FreshRSS/pull/1656

[planetahuevo]

@Alkarex thanks! I am starting a long trip right now so I wont be able to test it soon, but that seems a good solution for most of the problems.
I will test it as soon as I am back!

[kevinpapst]

I can confirm that all feeds which were mentioned here as problematic, are now working with the latest dev version:

• https://ayudawp.com/feed/
• https://tellado.es/feed/
• https://enlacepermanente.es/feed/
• https://maugelves.com/feed/

Can probably be closed.

[Frenzie]

Alright, closing. :-)

[planetahuevo]

Thank you! I had no time to test. :(