Beaker: Feature request: add dats with no peers to a watchlist

[sgwilym]

It's great that there are so many initiatives to make dats more available, e.g. hashbase, but it seems like there will always be times when you request a dat that has no peers, and that it shouldn't be presented as a failure or an unexpected outcome.

After requesting a dat with no peers, it would be great if I could add that dat to a watchlist which would occasionally check to see if that dat has become available, and notify me of it.

As a dat could become available long after someone tried to request it, it would also be great if you could add a description of what you expected to see, e.g. an essay about a new kind of paintbrush, so that when it finally does reappear, you remember why you tried to request it in the first place.

Something like this could contribute towards making the current 'Timed out' page a little less 'exceptional', and a little more 'this is normal -- and here's something cool you can do with Beaker Browser and dats!'

Would a feature like this even be feasible?

[pfrazee]

This would be feasible and I think it's a really interesting idea. I'll talk about it with @taravancil!

[roscoevanderboom]

This sounds like an amazing idea! I'm sure It won't be long before there could be push notifications sent to mobile that a site you've been waiting for is back online.

[roscoevanderboom]

As an extension of this idea....in the opposite direction, would it be possible to track the uptime of a site?

I love the function of seeding sites. I was trying to get a system like it going for IPFS. But if a site is well seeded with 100% uptime, there is no need for me to also seed it. I would rather seed a different site that needs to be kept online. The network shouldn't know who is seeding ...that's too much information. In this way we could distribute the web without wasting data.

I hope the culture of distributing data and conserving energy can be fundamental in all the projects we create and engage in. I got the idea from the diaspora network with their distributed pods.

[pfrazee]

@roscoevanderboom "uptime" or some kind of health-metric would be an interesting stat to track, yeah

[sgwilym]

I was thinking about this idea a little more, and some of the problems it doesn't solve -- one of them being that even if you add a dat to a watchlist, that doesn't mean you'll have beaker browser open when another peer finally appears. In which case this feature is a bit like having a watchdog who becomes catatonic for several hours at a time.

But pinning services could mitigate this weakness. A dat which is added to this watchlist could also be added to your pinning service of choice, which I assume would accept the peerless dat and keep retrying until it eventually reappears (?). Then when you'd come back online, the pinning service would be able to serve the dat to you. Admittedly this has a little bit of a _recentralising_ aspect to it, and also makes it so that your pinning service fills up with a lot of dats you might not particularly care for in the end.

Sorry to pile another idea into this thread, I will contribute with some renderings of how this feature could look soon!

[RangerMauve]

This sounds like a good idea that an extension would solve. 😁

[pfrazee]

@sgwilym I've been thinking about how we could do things like that using the user-profile dats without creating some kind of centralization risk. You'd have to publicly announce what you're watching for until we have at-rest encryption, but your pinning service(s) could watch your user-dat and fulfill that request for you. The plus-side of the user-dat is that it's very easy to migrate between pinning services.

[sgwilym]

Some jottings of what I think this feature could roughly look like.

Firstly, the replacement for the current 'Timed out' page:

In case it's not clear: de-emphasise the timing out and emphasise the no peers bit, and add a prominent action that the user can perform to not forget about this page forever. Still present the firewall dialog, but push it down a little bit.

Secondly, accessing the watchlist from the burger menu:

Add a new item to the current burger menu, which could badge itself with a number of newly resolved pages. Could also possibly badge the burger menu itself.

Finally, the watchlist page:

Same layout and grammar as the library page, but with two sections showing pages that have been resolved by the watchlist, and those that are still unreachable, showing the user's own title for the page and when they added it.

[pfrazee]

@sgwilym I love that! I'm completely on board with that design.

[pfrazee]

Here are the pieces that need to get done for this:

• [ ] Add a new DB table in beaker-core for tracking the watchlist
• [ ] Add a new internal Web API, beaker.watchlist, for managing the data
• [ ] Create a new builtin page, beaker://watchlist, which provides the UI of active watches and resolves
• [ ] Add the new HTML to the error page that's served by dat://
• [ ] Update the shell-window to provide the watchlist UIs

[Septimus]

@sgwilym I love the titles you put in the drawing. "video about new esport for dogs" 😂

Anyways, I'm getting started on this today!

[roscoevanderboom]

This relates to the DB for the watchlist.
Might be a naive suggestion...or too complicated.

What about MultiChain as a distributed DB. It's a highly customizable blockchain, easy installation and well documented API. They also have an electron dependency module npm install mutichain It would be running as a background process with Beaker and a webapp creating/querying the assets (json with dat archive metadata) on the chain.

The first Beaker Chain node would set up the permissions (default: anyone can connect; anyone can create asset (json with dat archive metadata) and create the Root Stream. This essentially becomes the search engine DB. Next, create a webapp for users to query the Root Stream. After that, installing Beaker (or another client for the chain) instantiates a new node, downloads permissions settings and subscribes to the Root Stream. Publishing a dat creates an asset that is published to the Root Stream. Remember to add good metadata for better search results.

Now it should be possible to query the stream assets in all kinds of ways. File size, author, date, category, title...whatever is written in the json. Also watching for content or a node to be online.

There are more functions that could be included in the chain like wallets or users creating their own stream with custom permissions and publish their links there. But I think a simple start is the best.

My main concern would be the updating of MultiChain protocol and the migration of the Root Stream. If it breaks....

Is this possible?

[pfrazee]

No we’re not really looking to add any new db or blockchain techs. (Let’s not clutter this issue with that discussion. Open another issue if you want to discuss further.)

On Oct 7, 2018, at 1:30 AM, roscoevanderboom [e-mail] wrote:

This relates to the DB for the watchlist.
Might be a naive suggestion...or too complicated.

What about MultiChain as a distributed DB. It's a highly customizable blockchain, easy installation and well documented API. They also have an electron dependency module npm install mutichain It would be running as a background process with Beaker and a webapp creating/querying the assets (json with dat archive metadata) on the chain.

The first Beaker Chain node would set up the permissions (default: anyone can connect; anyone can create asset (json with dat archive metadata) and create the Root Stream. This essentially becomes the search engine DB. Next, create a webapp for users to query the Root Stream. After that, installing Beaker (or another client for the chain) instantiates a new node, downloads permissions settings and subscribes to the Root Stream. Publishing a dat creates an asset that is published to the Root Stream. Remember to add good metadata for better search results.

Now it should be possible to query the stream assets in all kinds of ways. File size, author, date, category, title...whatever is written in the json. Also watching for content or a node to be online.

There are more functions that could be included in the chain like wallets or users creating their own stream with custom permissions and publish their links there. But I think a simple start is the best.

My main concern would be the updating of MultiChain protocol and the migration of the Root Stream. If it breaks....

Is this possible?

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

[roscoevanderboom]

no worries. just a thought. discard idea

[Septimus]

Getting there. Gimme feedback on the style if you don't like :)

[pfrazee]

Looking like a good start!

[sgwilym]

Very cool @Septimus!

Is everyone happy with the phrase 'watchlist'? There's not some other obvious term I'm missing that could be used for this feature? Similarly, is everyone happy with the (somewhat) ominous eye used to represent it? 😄

One phrase which I'm not satisfied by is 'resolved pages' in the watchlist page. I can't think of what else to call pages which have been caught by the watchlist, though. Found pages? Downloaded pages?

@Septimus one (tiny) thing about the screenshot you posted of the burger menu: that badged number's thinnest state should be totally round.

[pfrazee]

I'll tweak the UX and copy before merging, but I do like watchlist.

[Septimus]

1210 Pull Request started! WIP right now, but should be ready very soon, hopefully in time for 0.8!

[pfrazee]

This feature has been landed in master.