_Original issue located at https://github.com/akrennmair/newsbeuter/issues/518._
kutsan
All 7 comments
I'd like to say sorry for reopening that but I am looking forward to this feature literally for years.
kutsan
on 12 Nov 2017
I don't mind reopening that much, but I don't think that'll change anything either. If noone is working on this feature, it won't be implemented.
Might get easier once #35 is done though.
Minoru
on 12 Nov 2017
So sorry to necrobump, but has there been any development on this? Thanks a lot for your work btw!
matcharles
on 10 May 2020
No, sorry, nothing yet. The source is open, though, so if you're in the mood to write some C++, I'm happy to show you around the codebase :)
Minoru
on 10 May 2020
Haha thanks! Unfortunately, my programming knowledge starts and ends with
print("hello world")
In python and I don't even know if it's 2 or 3 haha.
matcharles
on 10 May 2020
I've made a start on this here #961
SeanBE
on 25 May 2020
961 could be used by an individual, but it uses a developer access token that's severely rate-limited: just 250 or 500 requests per day (for Free and Pro plans, respectively). To merge that PR, we needed to get a "production" token. There are a couple smaller problems that need to be overcome as well. @SeanBE discussed this with Feedly developers, and what follows is my overview of the situation.
There are three main problems.
OAuth tokens have to be private. They can't be part of the source code, otherwise anyone can just copy them and impersonate Newsboat. If an impostor does something bad (e.g., DDoSes Feedly), the token will be revoked, rendering Newsboat unusable with Feedly. Furthermore, we probably won't be able to get a new token, since we won't be able to prove it wasn't us who did the DDoS.
All of us (@SeanBE, Feedly devs, me) researched potential solutions, and there seem to be only two:
- each user gets their own tokens and puts them into Newsboat config. Feedly finds this unacceptable, stating that their keys are designed for apps, not individual users.
- we as a project could operate a proxy that adds our keys to queries that go through the proxy. This proxy has to provide some anti-abuse features, which I think means it would have to require registration by users. Neither I nor @SeanBE are willing to develop and maintain such a proxy, so this solution is unacceptable as well.
The bottom line is: we don't have a solution for this.
Feedly TOS requires no-compete for features already present in Newsboat. Search is a premium feature in Feedly, so Newsboat would have to limit its own search and query feeds (Feedly devs suggested limiting results to the last 31 days). This is fairly inconvenient to us, but if it was the only roadblock, I'd consider this.
Newsboat generates too many queries to remote APIs. Newsbeuter, and consequently Newsboat, are optimized to fetch many different feeds from many different sites, and to keep read/unread state etc. in a local database. When it talks to a remote API, it uses the same model: each feed is basically a separate request. Most APIs, including Feedly, aren't like that; they prefer to bundle things, e.g. to fetch all recent items of all feeds in one request. This already gives us headaches with other remote APIs (#16); Feedly just devs reiterated that 1 request per minute per feed (Newsboat's top speed) is not okay.
In summary, Newsboat can't support Feedly because there is no way to securely store OAuth tokens. The other two problems are pretty severe too, but OAuth thing is a deal-breaker. "Invent a way to store tokens" isn't very actionable, so there is no clear path forward for this issue. Thus, I'm closing it. Feel free to comment if you have ideas; this can always be re-opened if there's a plan.
Thanks to @SeanBE for all the work he sunk into this, and my apologies to all the users who waited for this for so many years.
Minoru
on 3 Jun 2020
Related issues
Minoru
路
3Comments
qazip
路
3Comments
colpshift
路
4Comments
dennisschagt
路
4Comments
suroa
路
4Comments
Most helpful comment
961 could be used by an individual, but it uses a developer access token that's severely rate-limited: just 250 or 500 requests per day (for Free and Pro plans, respectively). To merge that PR, we needed to get a "production" token. There are a couple smaller problems that need to be overcome as well. @SeanBE discussed this with Feedly developers, and what follows is my overview of the situation.
There are three main problems.
OAuth tokens have to be private. They can't be part of the source code, otherwise anyone can just copy them and impersonate Newsboat. If an impostor does something bad (e.g., DDoSes Feedly), the token will be revoked, rendering Newsboat unusable with Feedly. Furthermore, we probably won't be able to get a new token, since we won't be able to prove it wasn't us who did the DDoS.
All of us (@SeanBE, Feedly devs, me) researched potential solutions, and there seem to be only two:
The bottom line is: we don't have a solution for this.
Feedly TOS requires no-compete for features already present in Newsboat. Search is a premium feature in Feedly, so Newsboat would have to limit its own search and query feeds (Feedly devs suggested limiting results to the last 31 days). This is fairly inconvenient to us, but if it was the only roadblock, I'd consider this.
Newsboat generates too many queries to remote APIs. Newsbeuter, and consequently Newsboat, are optimized to fetch many different feeds from many different sites, and to keep read/unread state etc. in a local database. When it talks to a remote API, it uses the same model: each feed is basically a separate request. Most APIs, including Feedly, aren't like that; they prefer to bundle things, e.g. to fetch all recent items of all feeds in one request. This already gives us headaches with other remote APIs (#16); Feedly just devs reiterated that 1 request per minute per feed (Newsboat's top speed) is not okay.
In summary, Newsboat can't support Feedly because there is no way to securely store OAuth tokens. The other two problems are pretty severe too, but OAuth thing is a deal-breaker. "Invent a way to store tokens" isn't very actionable, so there is no clear path forward for this issue. Thus, I'm closing it. Feel free to comment if you have ideas; this can always be re-opened if there's a plan.
Thanks to @SeanBE for all the work he sunk into this, and my apologies to all the users who waited for this for so many years.