Freshrss: A new FreshRSS project?

[marienfressinaud]

Context

As you may know, I started my own company (flus.io, aimed at French people) at the end of November. The actual service is a basic installation of FreshRSS with some specific additions (published here). My idea was to earn a living, and consequently to develop FreshRSS. This was fine to start my activity, but FRSS is mainly intended for tech people, or at least for people who are at ease with concepts of RSS feeds, URLs, etc. It doesn't have a sufficient added value to convince enough people to pay for such a service (there's too much competition).

This situation made me think that I need to develop a whole new product, totally different. I spent time since January to imagine this new solution and I'm now confident enough to say, considering my constraints, that it'll be almost impossible for the features to fit in the current version of FreshRSS (not even in an extension, which has been my option n°1 for a long time). I won't go into detail and present all the options I considered (Flus is not the subject of my ticket :)). The option I finally chose is to develop my product separately from the community so I can make the choices that I consider the most pertinent for me. Also I might be able to go faster during conception and development if I don't have all the historical code on my path.

That being said, I still plan to work for the FreshRSS community, and that's the point of my ticket!

A new project?

I can't count the number of times I said to myself (or openly): this feature/improvement would be great to have, but it would be too complicated to develop considering the current implementation. The most obvious example that come to my mind is to give access to the settings on mobile. There's also this old ticket about re-designing the architecture which suggestions were pertinent for a new project, but not that much considering the state of FRSS. My point was at this time we didn't have time to work on a new big milestone (i.e. a v2), but, hey, I now have time to start _something_ new! :)

Basically, an aggregator:

1. fetches content, mainly RSS and Atom feeds but we can imagine other kind of sources (server-to-server backend)
2. serves the content based on users requests e.g. the main feed or filtered by favourites (server-to-client backend)
3. presents the content (client frontend)

In my opinion, points 2 and 3 are specific to FRSS (or to the future Flus). But we don't have to duplicate efforts to fetch content. So my idea is to develop this piece of code as a project of the FRSS community. It would be a first step in direction of an hypothetical v2. At least, it could help us to experiment new ideas in small projects since we won't have to worry about how to fetch and store content.

The features I can think about:

• fetch several URLs in parallel
• cache URL content
• extract information for a given URL (title, auto-discovered RSS URL**s**, opengraph info, etc.)
• extract articles from a RSS or Atom feed
• manage a list of sources to fetch on a specified frequency
• store information and articles in DB
• archive and cleaning systems

This would be a kind of SimplePie on steroids (but not based on it). It's not totally clear if it'll be a library, or a standalone application, or a bit of both. It will be user-agnostic since it will only be the server-to-server part.

I would be happy to have your insights about this project (especially if you think it's not interesting or a waste of time). Let me know if you need precision on a particular point.

[Frenzie]

The most obvious example that come to my mind is to give access to the settings on mobile.

Really? That seems like it's pretty much just a few lines of CSS. :-)

Btw, perhaps also see https://github.com/simplepie/simplepie-ng

I would be happy to have your insights about this project (especially if you think it's not interesting or a waste of time).

I assume there's something about SimplePie that doesn't suit your purposes? I'd be interested to hear what's suboptimal about it, because it does sound a lot like a few extra features around SimplePie-type functionality. And "extract articles/enclosures" is quite a complex task.

[marienfressinaud]

Really? That seems like it's pretty much just a few lines of CSS. :-)

You're right, it's maybe not the best example because it can probably be done in this v1. But I maintain it's more complicated than just few lines of CSS but I don't want to go into details here. In brief, a first step would more probably be to get an overview of the implications than a random PR adding a button somewhere ^^

I assume there's something about SimplePie that doesn't suit your purposes?

First thing is that it's almost not maintained anymore (same thing with simplepie-ng, but I might find some interesting things here, thanks!) I could propose to maintain it myself, but I don't want to spend weeks or months to understand the codebase while I have other priorities.

I didn't look recently, but I have in mind that SimplePie wasn't able to fetch multiple URLs in parallel so it would require some weird tricks to make simultaneous calls. Also, in the ticket I pointed, @Alkarex was saying he wanted to provide an alternative to SimplePie, which was at this time "too heavy, too large, too much legacy"

Anyway, I don't set my mind on this particular subject yet. I might use SimplePie (most probably not), feed-io (used in Nextcloud News app), or my own library (based on what exists elsewhere).

it does sound a lot like a few extra features around SimplePie-type functionality

I would rather say "alongside SimplePie-type functionality" :) I mean parsing RSS/Atom feeds would not be more important than other features. I see it like a set of separated tools with the same objective of scraping the Web.

Although I do realize that I'll have to spend more time to see what already exists!

---

I forgot to mention that I plan to maintain an instance of the current FreshRSS version while I have customers paying for it! So I'll probably continue to make improvements and PRs on the current repository :)

[ddeimeke]

So, it is more or less about a new library to replace SimplePie and better support of mobile devices?

For me, FreshRSS is a really good fit to arrange my information workflow with around 700 feeds. A really big part of them come from RSS-Bridge.

What I like to see in FreshRSS is a native RSS-Bridge (maybe a plugin), so that there is no need to maintain two services and two-factor-authentification.

But, for sure, I am very limited to my own needs and I do not look in the machine room to see what SimplePie does not do well. And the mobile support is good enough for me, it works well on an Android tablet. For reading on a mobile phone I would use an App and not the website.

[Alkarex]

Short feedback, and I hope to come back to it (or ask if there is any specific point in particular for me to focus on).

• We can already fetch several URLs in parallel (this is what the Web UI does, and a cron can also do it if one starts while the previous one is not completed)
• We already have a feed cache (on disk) shared between users (it would require a few improvements though, for instance to disable it in some cases). It could relatively easily be changed to use a DB if wanted
• SimplePie, after a period of near abandon, is again OK-maintained (e.g. recent updates for PHP 7.4)
• We could make a new front-end (e.g. progressive Web app) without changing much of the existing back-end (alternatively using our Google Reader API like Google Reader did for offline reading synchronisation etc.).
• Watch out of the typical 80-20% rule, and fragmentation of efforts. Several new functions (e.g. better sharing, new sorting, better filters, better search, etc.) can be done with comparable efforts regardless of the rest of the back-end, so it is more about getting the job done, which is best addressed by a consolidation of efforts. Green field might be more fun for developers, but it risks leading to a disservice for the users and for the supported values :-)

[marienfressinaud]

We could make a new front-end (e.g. progressive Web app) without changing much of the existing back-end […]
Watch out of the typical 80-20% rule, and fragmentation of efforts. […]

Yes I know, and that's why I considered the extension option for a long time: I thought it would make it. I don't want to detail yet what I'm going to propose as features in the future version of Flus because nothing is really fixed, but I assure you it's not just about a new interface or improving current features. It's a totally new and different project, almost not even a RSS aggregator (well… it will… in a different way ;)). To describe it in few words, it will be a mix between an aggregator and a Shaarli but for groups of people. I also want to experiment drastic choices (e.g. by not implementing some features) that wouldn't match with the spirit of a traditional aggregator. I don't want to impose these choices to FreshRSS because it wouldn't make any sense and I would feel bad if they don't work.

To present it differently: I will develop this new application and I will need some specific backend-to-backend functionalities that are not present in FreshRSS today. What I was suggesting is to make it usable for both projects and to put the repo under the FRSS community. I also can start it under my own organization and transfer it once its purpose becomes more clear and if it appears interesting to the community :)

Green field might be more fun for developers, but it risks leading to a disservice for the users and for the supported values :-)

Sure! I considered that carefully and I'm sure to bring more values with something completely new and different :)

[Alkarex]

Sorry for being a bit slow at the moment; I hope it will soon be better.

I am not sure of how extensive the features you want to implement are, @marienfressinaud , but there at at least a few things that I believe would fit nicely inside the core of FreshRSS.
In particular, improved sharing from FreshRSS has been requested for a long time, and I have actually tried to find time to start implementing it. I have not come to it just yet, but I do have some initial thoughts, though, and I would gladly contribute some implementation efforts:

• Allow removing feeds while keeping some articles: modelled with a default feed, and a new JSON attributes field in the article containing optional feed information.
  
  
  
  • This allows keeping labelled and favourite articles even when removing feeds
  
  
  • It is a relatively small DB change
  
  
• Using the same mechanism as above, UI to add a link to an article in DB, without having to subscribe to its corresponding feed
• UI to attach a comment to an article
• UI to delete an article
• Improve the token mechanism to re-share FreshRSS outputs, to allow having multiple tokens
  
  
  
  • Produce an HTML view in addition of the RSS view when sharing from FreshRSS
  
  
• Based on the new tokens, expose a page to allow clients consuming FreshRSS shares to select additional filters server-side, e.g. filtering on a custom selection of tags
  
  
  
  • We already have most of the underlying mechanism to do that
  
  
• (Slightly unrelated, but natural follow-up) New mechanism to highlight articles , which have been shared by more than one source (à la Fever, but decentralised), e.g. when subscribing to a Planet, to the original site, and to multiple FreshRSS / Shaarli / ... instances, which have all re-shared the same article

[ColasNahaboo]

Features idea: Forums-friendly RSS

Hi, I don't know if this feature is interesting to enough people to develop it, but I scan a lot of forums to look for interesting information, but I am not a committed regular member of them, I just scan from time to time. I have used RSS readers for this (most forums do not have an easy way to remember the unseen posts), and FreshRSS which is the one I use now, but I found that a regular RSS reader was not adapted to reading forums this way. So I toyed with what could be a good FreshRSS feature for this, and here is my idea:

• Make FreshRSS show only one post for each forum thread, the first unread post in it. No need to show me the 100+ replies to it on active threads as first-class citizen in the FreshRSS UI.

I though about implementing this as a FreshRSS extension, but since I do not know a lot the FreshRSS code, and have not a lot of practice in php, I decided to prototype a separate RSS filter so that I could toy out and experiment ideas. My current prototype (with a mini doc) is usable at https://forumsfeed.colaz.com/ if you want to toy with it (suggestions welcome).

Note: the sources are not yet available (links from the page do not work), as they are changing a lot, I do not want to commit them to their definitive GitHub place and maintain compatibility yet. But I can publish them in my work repos if you are interested. They are in bash, as this is where I am most efficient in fast prototyping.

[ainola]

Thanks for the transparency!

This may be unrelated but I feel that knowing this project's existence might be helpful for conceptualization.

https://fraidyc.at/

[marienfressinaud]

Thanks for the pointer @ainola ! I saw it few months ago but I forget about it. Definitely something that could inspire me! I actually plan special support for platforms such as Twitter or Youtube, might be useful for FreshRSS too :)

For full transparency, I started the development of https://github.com/flusio/flusio in May and I keep separated a "SpiderBits" library which is aimed at simplifying Web crawling. Nothing fancy but I hope to add some more features in the next months (for now I'm focusing on getting a coherent beta version).

[oupala]

I've commented #956 then I saw this issue. Please tell me what you think of this last comment.

[Alkarex]

Now that Flusio is up and running, we can probably close here :-) (But synergies should continue!)