Mastodon: Make public streaming api not require user credentials

Created on 17 Apr 2017 · 8Comments · Source: tootsuite/mastodon

This is related to #1291 and maybe #1053. To enable third party app to browse public timeline on any instance, we should be able to open a streaming connection without requiring authentication token.

[x] I searched or browsed the repo’s other issues to ensure this is not a duplicate.

api

Source

siuying

👍4

Most helpful comment

If the server is streaming a lot of message, more than the API can handle, we can limit the output by sampling it, like twitter firehose. That would be in separate in issue then.

siuying on 26 Apr 2017

👍2

All 8 comments

Isn't this exactly what #1291 fixes tho? And it's been merged?

wxcafe on 17 Apr 2017

No. #1291 fixes REST API, where I'm talking about streaming api at https://github.com/tootsuite/mastodon/blob/master/streaming/index.js

siuying on 17 Apr 2017

Alright makes sense, sorry

wxcafe on 17 Apr 2017

This would be absolutely amazing to have this: I want to start gathering toot / hashtag data (toot length, most common hashtags, toots languages...) in MNM, and to do this in a reliable manner, I need to subscribe to instances public timeline, which is not possible at the moment because of required authentication.

I understand it can be a performance / DDoS concern, since anybody being able to register to the public streaming API will definitely increase the load on instances. Maybe it can be adressed or mitigated by limiting it in duration, and by processing message delivery with a really low priority?

agateblue on 17 Apr 2017

👍1

I understand it can be a performance / DDoS concern, since anybody being able to register to the public streaming API will definitely increase the load on instances.

Not if the alternative is that they just poll it. Even if they're good about using etags and since_id, which not everyone will be, polling is almost certainly going to hit the server harder.

This is a political issue. Twitter lost a lot of fans over charging people for the firehose.

sandhawke on 26 Apr 2017

👍1

If the server is streaming a lot of message, more than the API can handle, we can limit the output by sampling it, like twitter firehose. That would be in separate in issue then.

siuying on 26 Apr 2017

👍2

Not if the alternative is that they just poll it. Even if they're good about using etags and since_id, which not everyone will be, polling is almost certainly going to hit the server harder.

Another approach would be to have at least ONE STREAM go to another big pipe resource which, in turn, is used by clients... offloading some work while still providing firehose for free. Maybe other benefits, too.

[working on an idea]