Status-react: Add unfurling of URLs in chats

[pedropombeiro]

User Story

As a user, I want the app to unfurl URLs so that I can easily see what they refer to without needing to open the URL.

Description

_Type_: Feature

_Summary_: If another user shares a URL with me (maybe from a 3rd party app), it would be great if I could see the summary of the content of the URL so I have a quick idea of what it's about.

Expected behavior

1. User receives URL
2. App recognizes URL and uses an unfurlingeloper-blog/everything-you-ever-wanted-to-know-about-unfurling-but-were-afraid-to-ask-or-how-to-make-your-e64b4bb9254) library to generate an image (or HTML) content that represents the summary of the URL.

Acceptance Criteria

• [ ] Links starting with http:// or https:// are unfurled on the receiving client side
• [ ] Performance is not noticeably negatively impacted by the new functionality

Security implications

We have to think about the security implications if we do this on the client side (will it be acceptable to access a random URL which might leak metadata about us?)

[liamzebedee]

I've just started working on this.

[liamzebedee]

I think it's a very real security concern, and it would have to be done client-side unless Status served the content.

Would it be a better idea to keep it as a setting? ie. "automatically generate previews"

[pedropombeiro]

Yeah, this would need to be done on the client-side. Hopefully the library we end up using for this does what it can to prevent any security issues arising from generating a preview from any random site (maybe starting by whitelisting safe sites).

cc @adambabik regarding security

[liamzebedee]

Yeah @PombeirP I think it'd be best to start off with a list of the common social sites (Twitter, YouTube, Facebook, Spotify, etc) and go from there.

Another question is about the UX scope - are we fine with including the following?

• Title
• Favicon
• Image
• Description

[pedropombeiro]

Ping @andmironov for design

[adambabik]

Do we want to unfurl any URL on the client side? So, I will send a link to a public channel and now each client will make a call (or more calls) to get details about the content of link? And some random site will know IPs of many Status users?

I don't think it's a good idea. I hate that stuff actually. I think we can do better and do unfurling only when a user requests for that by taping a button or something.

[liamzebedee]

@adambabik so what do we think of the following: an option about loading link previews that is either an on/off or something like "for trusted sites only" vs "never"

[adambabik]

@liamzebedee personally, I would love to have it disabled by default but have an option to preview each link. Maybe something similar to how Slack is doing that? E.g. I get a link from https://github.com for the first time and there is a message below saying "(1) always preload from https://github.com, (2) preload this time, (3) never preload" and my answer is remembered.

We can, of course, fill in the list of trusted URLs by default with some common websites.

Also, there must be a way to remove already whitelisted URLs.

GitHub

Build software better, together

GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects.

GitHub

Build software better, together

GitHub is where people build software. More than 28 million people use GitHub to discover, fork, and contribute to over 85 million projects.

[andytudhope]

@liamzebedee are you still working on this?

[liamzebedee]

@andytudhope no not anymore!

[StatusSceptre]

@PombeirP Seems like there was a bit of back and forth with what we'd do with this option, any finality / clarity around scope on this one if it's still being considered for a bounty.

[pedropombeiro]

I think if we do it like the Messages app in Android, where it just adds a "Generate Preview" button to unfurl on-demand, it would be less contentious.

[StatusSceptre]

@PombeirP Cleaning up the bounty tracking board. Is this still relevant for bounties, and being worked on? If so, It needs a bounty.

[mandrigin]

@StatusSceptre I think it is still relevant.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

__This issue now has a funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to it.__

• If you would like to work on this issue you can 'start work' on the Gitcoin Issue Details page.
  
  
  
  • Want to chip in? Add your own contribution here.
  
  
  • Questions? Checkout Gitcoin Help or the Gitcoin Slack
  
  
  • $100,266.23 more funded OSS Work available on the Gitcoin Issue Explorer
  
  

[mandrigin]

Note to a future bounty hunter:
There are a few things to take into account while doing this bounty:
(1) Previews should be shown un-furled only when sent from contacts
(2) Un-furling should happen on the sender side (cc @corpetty, what do you think about it?)
(3) The cookie storage/cookie header should be EMPTY when doing request to unfurl

I think we can add an additional field to a whisper message that has this unfurled metadata.

I think we can start with just displaying the document title there.

[corpetty]

Going off what he proposes in terms of what to include, I don't see any problem right now with the content (it's what slack does, afterall).

We do need to be cognizant of what the user is sending out in terms of personal information. I'm looking further into it, but start with what @mandrigin mentions.

furthermore, this should be an option for the user.

I think if we do it like the Messages app in Android, where it just adds a "Generate Preview" button to unfurl on-demand, it would be less contentious.

[gitcoinbot]

@developerkumar Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

__Workers have applied to start work__.

These users each claimed they can complete the work by 11 months, 2 weeks from now.
Please review their action plans below:

1) e18r has applied to start work _(Funders only: approve worker | reject worker)_.

I read your comments and, after some research, here are my conclusions:

1. I'll use this unfurl library. I checked it and it makes requests with empty cookie headers, as @mandrigin proposes. The data it returns is as follows:
   {
   :url - The url of the resource, according to the server
   :title - The title of the given url
   :description - The description of the given url
   :preview-url - The url of a preview image for the given url
   }
2. When a user adds a link to a message, the unfurling is automatically done. This will happen every time, but on the sender side only, as @mandrigin suggests.
3. It is safe to show the sanitized title and description fields to the recipient without compromising her privacy, so I propose to show them to the recipient every time.
4. The request to the preview-url field, done with empty cookie headers, should only happen if the recipient expressly chooses to do so by means of a button, as @adambabik, @PombeirP and @corpetty suggest.
5. If the sender is in the recipient's contacts, the image request is done automatically, as @mandrigin proposes.

My development environment is ready and I already started to look at the code. I'm fairly new to Clojure, but I already made a contribution to Status. I'm ready to work full-time on this.

Learn more on the Gitcoin Issue Details page.

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

__Work has been started__.

These users each claimed they can complete the work by 11 months, 1 week from now.
Please review their action plans below:

1) e18r has been approved to start work.

I read your comments and, after some research, here are my conclusions:

1. I'll use this unfurl library. I checked it and it makes requests with empty cookie headers, as @mandrigin proposes. The data it returns is as follows:
   {
   :url - The url of the resource, according to the server
   :title - The title of the given url
   :description - The description of the given url
   :preview-url - The url of a preview image for the given url
   }
2. When a user adds a link to a message, the unfurling is automatically done. This will happen every time, but on the sender side only, as @mandrigin suggests.
3. It is safe to show the sanitized title and description fields to the recipient without compromising her privacy, so I propose to show them to the recipient every time.
4. The request to the preview-url field, done with empty cookie headers, should only happen if the recipient expressly chooses to do so by means of a button, as @adambabik, @PombeirP and @corpetty suggest.
5. If the sender is in the recipient's contacts, the image request is done automatically, as @mandrigin proposes.

My development environment is ready and I already started to look at the code. I'm fairly new to Clojure, but I already made a contribution to Status. I'm ready to work full-time on this.

Learn more on the Gitcoin Issue Details page.

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[e18r]

Hi, I'll start working on this in a few hours.

[e18r]

Hi, this a quick report on my progress so far in this issue. In the past days I've been working on the unfurl library. It turns out that library is a Clojure, not a ClojureScript library. Given there aren't other URL unfurling libraries for ClojureScript, I decided to make it compatible. I opened an issue in their Github repo, and the main developer of that library gave me helpful advice on how to do it. The main issue is to replace the HTTP client. The two candidates are cljs-http and cljs-ajax. I've made some tests with them, but I've encountered a few unexpected issues, including the Single-origin policy.

So I will keep working on making this library compatible with ClojureScript as a prerequisite to start working on this issue. Any feedback much appreciated.

[e18r]

Hi, I'm still working on getting the unfurling library to work with ClojureScript. I have to manage the asynchrony of ClojureScript's HTTP clients, which is tricky if one wants to stay compatible with Clojure HTTP clients, which aren't asynchronous. I'll keep you guys informed.

[e18r]

I've been researching about how asynchrony works in JavaScript. Now that I have a better understanding of it, I will finally be able to port the unfurling library to ClojureScript. I'm sorry this is taking so long - it will still take me a while.

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[e18r]

Hi, I already submitted a PR to the unfurl library. As soon as it gets approved, I'll start woring on incorporating it in Status.

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[e18r]

I haven't heard back from the developer of the unfurl library. For now I'll upload my own version to clojars and incorporate that into Status.

[e18r]

I uploaded my own version of the unfurl library to clojars, but adding it to Status has been difficult. As soon as I :require it in my code, Status gets stuck on the initial logo forever. I just opened issue #8373 with all the details.

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[gitcoinbot]

@e18r Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[e18r]

Hello, I stopped working on this issue. Feel free to apply on Gitcoin. Also, if you have any questions you think I can answer, don't hesitate.

[krisc]

if no one else is interested, I may take on this bounty. But I do have other issues to resolve first. So I'll hold off on applying

[StatusSceptre]

Hey @krisc - I see you're also interested in working on some other issues. Thanks for your interest! Let me know if I can help in anyway.

[rachelhamlin]

@thecyberd3m0n I know it's been ages - but are you still interested in this bounty? I'm cleaning up all our bounties now and would love to have you take a stab if so.

[thecyberd3m0n]

I'm sorry I missed some notifications from Github (it's spamming a lot). Yes. I'll try to do it.

[rachelhamlin]

@thecyberd3m0n nah, nothing missed - just on our side :) You're approved so let us know if you have any questions. Thanks!

[thecyberd3m0n]

it will take me about the week

[rachelhamlin]

Sounds good.

[gitcoinbot]

@thecyberd3m0n Hello from Gitcoin Core - are you still working on this issue? Please submit a WIP PR or comment back within the next 3 days or you will be removed from this ticket and it will be returned to an ‘Open’ status. Please let us know if you have questions!

• [x] reminder (3 days)
• [ ] escalation to mods (6 days)

Funders only: Snooze warnings for 1 day | 3 days | 5 days | 10 days | 100 days

[thecyberd3m0n]

@gitcoinbot yes

[thecyberd3m0n]

my current version from develop can't build

error: bundling failed: Error: Unable to resolve module `qrcode` from `/home/tcd/hdd/gitcoin-bounties/status-im/status-react/index.desktop.js`: Module `qrcode` does not exist in the Haste module map

I did like documentation describes
https://status.im/build_status/desktop.html

Any suggestions what can I do wrong?

OS: KDE Neon Linux x64 5.**
Node: v10.16.0 (from nvm)

[yenda]

@thecyberd3m0n the desktop build is behind mobile atm, you should develop from android or iOS

[thecyberd3m0n]

ok I'm sorry, but I had failed to do this task (due to lack of ClosureJS knowledge)

[yenda]

@thecyberd3m0n what did you try at this point? did you find a suitable js or react-native library for url unfurling?

[thecyberd3m0n]

I didn't try to find unfurling library (my approach was to GET it as it is and just read html's metadata, started by oc). But I failed to do also this, due to unknown language (I thought it was JS or TS). I'm highly unfamiliar with CJS (like, see this first time, and didn't even know such language exists :)

I dismissed this task at gitcoin side so others can join it.

[yenda]

@thecyberd3m0n clojurescript is an amazing and quite simple language, if you want to try it out and need guidance as of where to start you can join #status-core channel on status.

[rachelhamlin]

Thanks for opting out on Gitcoin for us @thecyberd3m0n - if you have any feedback on the clarity of the gitcoin posting I'm happy to hear it. And thanks @yenda for helping to trouble shoot. I'm going to let this one drop for now, as it's not a real priority.

[thecyberd3m0n]

I need tasks in known environments, where I can put predictable amount of time to get the results. Unfortunately, learning whole new environment is not one of them :( I'm trying to earn more while already having commercial contracts.

I'm not saying it's bad, I'm just total noob in it, seeing it first time, and can't afford time for learning such a lot about it. Still, thanks for support and big sorry for taking your time.

[ScyDev]

@rachelhamlin Would this one be a priority? If so, I'm ready to start work on it right away and not afraid to dig into the framework and learn all about it.

[rachelhamlin]

Sorry for my delay @ScyDev - also no. We have quite a few issues open that don't have bounties applied yet, I'm looking at our v1 release backlog. How about this small fix to get started?

https://github.com/status-im/status-react/issues/8818

Or this one?
https://github.com/status-im/status-react/issues/8837

[ScyDev]

@rachelhamlin Alright, I'll gladly take both of them. Thanks!

[rachelhamlin]

@ScyDev I'll put a price on them and invite you - they both look small to me but will be good to get you started. Welcome aboard!

Edit: update - Gitcoin is not recognizing that Metamask is unlocked and is preventing me from creating bounties...I'm going to put each of them at 60 DAI though, so you're aware. Will try again later.

[ScyDev]

@rachelhamlin Thanks! I've been trying to get in touch with you through Status. Is there a Discord or Slack channel for easier discussion?

[rachelhamlin]

@ScyDev oh, sorry about that! I've been on and off chat today. Just got your Status messages, we can chat there :)

[gitcoinbot]

Issue Status: 1. Open 2. Started 3. Submitted 4. Done

---

__Workers have applied to start work__.

These users each claimed they can complete the work by 4 months, 2 weeks from now.
Please review their action plans below:

1) jezsmith720 has applied to start work _(Funders only: approve worker | reject worker)_.

Shouldn't be a problem, I would like to work on this if it is still open? Cheers.

Learn more on the Gitcoin Issue Details page.

[StatusSceptre]

@errorists I believe you already have designs for this. Could you add these here or create a new issue?

[errorists]

yes, the chat part is here and the settings part is here
It's work in progress, I've yet to document how the message bubble is presented.

This needs to be discussed with regards of security policy and technical feasibility. The security notice I used there also needs an update.
imo a new issue is the better way to go forward here

[vkjr]

Closed in favor of #11158

[gitcoinbot]

Issue Status: 1. Open 2. Cancelled

---

__The funding of 250.0 DAI (250.0 USD @ $1.0/DAI) attached to this issue has been cancelled by the bounty submitter__

• Questions? Checkout Gitcoin Help or the Gitcoin Chat
• $736,488.02 more funded OSS Work available on the Gitcoin Issue Explorer