Signal-android: YouTube link previews not loading.

[schmeat]

• [X] I have searched open and closed issues for duplicates
• [X] I am submitting a bug report for existing functionality that does not work as intended
• [X] I have read https://github.com/signalapp/Signal-Android/wiki/Submitting-useful-bug-reports
• [X] This isn't a feature request or a discussion topic

---

Bug description

Link preview fails to load every time a youtube video is pasted in.

Steps to reproduce

Copy a youtube link and wait for preview to fail loading.

Screenshots

https://gfycat.com/wateryunhappycommabutterfly

Device info

Time : 1570681676210
Device : samsung SM-G975W (beyond2qltecs)
Android : 9 (G975WVLS2ASI6, PPR1.180610.011.G975WVLS2ASI6)
ABIs : arm64-v8a, armeabi-v7a, armeabi
Memory : 68M (23.08% free, 512M max)
Memclass : 256
OS Host : 21HHAD01
First Version: 530
App : Signal 4.48.16 (5452)

Link to debug log

https://debuglogs.org/f132028ab42abbc37ed834e8a8b7d276d43d6f970f34db28694284c58186f69a

[schmeat]

Seems to have started working again. I'll assume temporary back end issues?

Edit: I spoke too soon, stopped working again.

[chynric]

Same for me. It is neither loading in Signal Android, nor in Signal Desktop.
Signal Android: 4.48.14
Signal Desktop: 1.27.1

[Flupkees]

Same for me

[mikeymop]

I am experiencing this on 4.47.7 so I think it is in the backend.

[iomintz]

YouTube likely blocked Signal's crawlers. This happened to Discord and Telegram recently, as well.

[Flupkees]

Ah, that's good to know. I assumed all previews didn't work anymore, but I just tested and reddit previews work, so it seems to only be youtube. Does anyone know or can anyone point me to why signal is doing this in a centralised way? If I remember correctly, WhatsApp clients generate link previews locally and that seems to work really well and not only for 4 websites.

[u32i64]

@Flupkees
Does anyone know or can anyone point me to why signal is doing this in a centralised way?

https://signal.org/blog/i-link-therefore-i-am/

In fact it's 5 websites now, Pinterest was added by PRs from a user (who apparently works for Pinterest) as well: #8760.

[Flupkees]

@Flupkees
Does anyone know or can anyone point me to why signal is doing this in a centralised way?

https://signal.org/blog/i-link-therefore-i-am/

In fact it's 5 websites now, Pinterest was added by PRs from a user (who apparently works for Pinterest) as well: #8760.

Thanks for that!
I also found this as to why one may want to avoid whatsapp's implementation: https://techcrunch.com/2017/06/15/should-whatsapp-let-you-disable-url-previews/

[thesurg3on]

I also have this issue. Are there any solutions for this?

[iomintz]

@thesurg3on Not for us users. The Signal team has to work it out with YouTube.

[schmeat]

Can we please keep discussion off this ticket.

Any discussion should happen on the signal forums.

If this happens to you as well, please thumbs up the post but don't add an additional comment.

[iomintz]

This issue appears to be resolved.

[greyson-signal]

Ok, so tentatively YouTube is working again. Fingers crossed it stays that way :)

[nanowinner]

I'd hate to do this, but the issue is present still both on the Desktop app (latest) and Android app (latest).

[chynric]

Ok, so tentatively YouTube is working again. Fingers crossed it stays that way :)

Yes it was working again a short time until stopping once again.

[KommX]

@greyson-signal Should we reopen this issue or create a new one?

[greyson-signal]

Yeah, we had something that was working for a while, but it stopped again. This is a server-side issue with our proxy, but I'm ok with tracking it here. TL;DR is that we're being rate-limited by YouTube.

[nutts0]

YouTube doesn't work here

[wwwizzarrdry]

How does duck duck go proxy their YouTube videos? Is that an option here?

[JonasNord]

Issue persists for me as well, omitting device info as issues are server side.

EDIT: What's the reason behind the thumbs down? Am i missing some etiquette or rule?

Yeah, we had something that was working for a while, but it stopped again. This is a server-side issue with our proxy, but I'm ok with tracking it here. TL;DR is that we're being rate-limited by YouTube.

[jgspratt]

Could this be re-written so that the client makes the request?

Could your proxy be altered so that it uses many IPs?

[asbjornu]

Why are you using a proxy to implement link previews? For privacy protection? What about those of us who value phishing protection over privacy? Having link previews that work are more important to me than my IP address being masked to the website being previewed.

The Open Graph Protocol exists to allow link previews regardless of website and should be used to implement this feature, without a hard coded list of supported domain names. If people want to restrict the domains that offer link previews, that can be configurable. The current implementation is almost completely useless to me as I never share URLs for any of the supported domains except YouTube.

Please reconsider the current implementation.

[iomintz]

Yes, it's for privacy protection. Doing the link previews client side amounts to leacking the contents of (some of) your messages to third parties, which Signal is obviously against.

[jgspratt]

Yes, it's for privacy protection. Doing the link previews client side amounts to leacking the contents of (some of) your messages to third parties, which Signal is obviously against.

In most cases (99% or better), I'm sharing a link I've already opened in my browser, so the security point is moot.

But an option is to add an option: "Generate previews locally". That makes everyone happy.

Or how about "Automatically generate previews locally if proxied preview fails"?

Or "Offer to generate previews locally if proxied preview fails"?

[Ralms]

@iomintz "Doing the link previews client side amounts to leaking the contents of (some of) your messages to third parties"
LOL What? You know that is not true right?
If you try to develop something and interact with other APIs you might be enlighten.

[iomintz]

@Ralms what? I have used third party APIs. How is that relevant here? Signal doesn't use any.

[iomintz]

Other than, I think Giphy, which it uses in a clever way.

[Ralms]

@Ralms what? I have used third party APIs. How is that relevant here? Signal doesn't use any.

Right, in fact it does, is using on the servers right now for this feature.
So, I would love to see your arguments / proof of how using Open Graph would result on a data leak of messages content, being it is literally a Get request which can be isolated.

[iomintz]

Alice sends "check this out http://example.com/foo" to Bob
example.com receives GET request from Signal app
suppose Alice already signed in to example.com before
example.com now knows that Alice sent a link to /foo on Signal

[iomintz]

At least the current implementation preserves Alice's anonymity

[jgspratt]

@iomintz, that's why a "Failed to load preview via proxy. Do you want to load it locally? This may tell example.com that you're using Signal. (click here to never show this for example.com again" message would be handy.

[Ralms]

Alice sends "check this out http://example.com/foo" to Bob
example.com receives GET request from Signal app
suppose Alice already signed in to example.com before
example.com now knows that Alice sent a link to /foo on Signal

Not true, like I said, you can isolate it and for the perspective of example.com would be a generic chrome or safari browser access.
And on top of that Alice was already signed in and most likely was on example.com where she obtained the link.

This level of paranoia is what will inhibit us to convince our friends to get out of things such as WhatsApp.
Worst case scenario should be optimal to be enabled by whoever doesn't mind to use it.

[jonest]

I think there's another issue where Alice send a link to Bob. Bob has not been to www.example.com. Bob's Signal app loads a preview and now that website knows Bob.

Am I mistaken here?

[iomintz]

@jonest, Alice's client would generate the link preview and include it in the message sent to Bob.

[jonest]

@iomintz cool, thanks! Wasn't sure if that would be delivered as part of the payload or not 👍

[mueller-ma]

Please use the forum to discuss changes of this feature: https://community.signalusers.org/t/link-preview-when-pasting-or-typing-links/2370

[asbjornu]

Yes, it's for privacy protection. Doing the link previews client side amounts to leacking the contents of (some of) your messages to third parties, which Signal is obviously against.

This is blatantly false. Performing an HTTP GET request towards a URL does not expose anything but my IP address. And if the sender (as you suggest, @iomintz) is the one that actually bundles up the preview, the receiver's IP address wouldn't even be exposed, it would be the sender, who would already have had to visit the page for it to be shared in the first place.

I don't really understand what there is to protect by using a proxy here. It's the wrong solution to a non-existing problem, and exactly like @Realms writes; the level of paranoia that makes it impossible to convince our friends and family to get out of things such as WhatsApp, Facebook Messenger, etc.

[greyson-signal]

This is blatantly false. Performing an HTTP GET request towards a URL does not expose anything but my IP address.

Not true, like I said, you can isolate it and for the perspective of example.com would be a generic chrome or safari browser access.

Nope, this does leak some message contents. Just requesting the HTML content of a page is not what a normal web browser does. They make tons of requests for linked assets and stuff. Just requesting HTML would be easily identifiable as a request for a link preview, at which point you've leaked part of your message -- namely, the link that you're sending.

When trying to make this feature, I had even come up with ideas to do a full webpage load in an offscreen webview to make it look like legitimate web traffic... but then you need to download the image for the thumbnail, which often isn't an image a web browser would usually load, at which point you definitely look like a link preview request.

This is something where the devil is really in the details. We are looking into alternative ways to do link previews more generically, but it's tricky.

But as previously noted, people should be discussing this on the forum. We use github for bugs, not general discussion.

[mikeymop]

What is you opinion on removing the link preview code until a solution is found and developed?

Currently, users can see the link preview attempt to generate and then it disappears.

This can look to the user like something in Signal is broken. Then this can be revisited in a PR discussion and the issue won't continue to be bumped.

[Ralms]

What is you opinion on removing the link preview code until a solution is found and developed?

Currently, users can see the link preview attempt to generate and then it disappears.

This can look to the user like something in Signal is broken. Then this can be revisited in a PR discussion and the issue won't continue to be bumped.

Doing that will result on this feature most likely never being added again.

[asbjornu]

@greyson-signal,

Nope, this does leak some message contents.

That depends on your interpretation of "message contents". Most people will read that as "the private message the sending party writes to the receiving party", which under no circumstance will be leaked to a website being previewed, unless the preview mechanism is exceptionally broken.

Just requesting the HTML content of a page is not what a normal web browser does. They make tons of requests for linked assets and stuff.

So don't use a full-blown browser to create the preview. Just do an HTTP request with URLSession or HttpURLConnection, fetch the Open Graph Protocol metadata from the <head> section and create a preview based on that. Worst case fallback: use the <title> element to at least provide a human readable title of the link.

Just requesting HTML would be easily identifiable as a request for a link preview, at which point you've leaked part of your message -- namely, the link that you're sending.

What would be identifiable is that the user agent isn't executing JavaScript. The fact that there wasn't made a request to "linked assets and stuff" is something most sites don't correlate, as data about these requests are stored server-side on CDNs and not souped up by the JavaScript-based trackers used everywhere.

So yes, the shared site will "at best" know that a request was made from a user agent that didn't execute JavaScript. Just like any web crawler, bot, or similar, in existence.

All of these points are moot if the link is shared from Safari, though. Safari's share sheet will expose all of the shared site's metadata through a PKLinkMetadata object (I suppose something similar is possible with Intent.ACTION_SEND on Android), making a second preview request unnecessary.

This is something where the devil is really in the details.

To guarantee 100% privacy and anonymity; yes. But most (potential) users of Signal don't care about that to such an extreme extent. Some do, so please make sure they can turn link preview off, but for the rest of us who care much more about Signal being a product we in good faith can recommend to our friends and family – working and ubiquitous link preview is essential. Not having them feels like being thrown back into the early 2000's before link previews existed.

We are looking into alternative ways to do link previews more generically, but it's tricky.

PKLinkMetadata and The Open Graph Protocol are two generic ways to do this. I don't think you're using any of them.

But as previously noted, people should be discussing this on the forum.

We are, but as the Signal developers don't seem to be present there, the discussion isn't of much value, tbh.

We use github for bugs, not general discussion.

I think the title of this issue – "YouTube link previews not loading" – is a bug, and most of the comments I've read on here are discussing viable solutions to the problem, which I think is pretty much on topic.

[navid-zamani]

But most (potential) users of Signal don't care about that to such an extreme extent.

That is patently false, and a dangerous mindset. Such ”extreme“ (read: actually 100%, so actually having a point at all) privacy is the whole point of using Signal over e.g. just usinf WhatsApp. It is why users install it, and what they expect.

That meme of the careless ”average user“ is also very dead. Especially nowadays in the EU. Since the GDPR, even average grandparents have become very wary of their privacy not being protected. It has become fashionable.

I just had to add that, so Signal does not veer off in a suicidal direction of half-assing what its whole point is.

[appdoo]

First of all I think this feature should work for all links - if the user opts in.

Implementation:
I understand the privacy concerns, so does anyone see a problem with the following implementation:

1. Sender posts a link.
2. If he opted in to link previews, a link preview is generated just for himself

3. Depending on the implementation, the message is tagged with a link preview tag.

4. The receiver receives the link in a message

5. If the Receivers opted in to „load link previews“ a link preview is generated locally for him.

This way link previews will never be transferred, they‘ll always stay local on the users device.
The sender sends just a „load_link_preview“ tag. If the receiver wants to see it from youtube or any other site, he generates it‘s own preview tag ..

Information is never transferred.

Another solution is also available:

1. same as above
2. same as above
3. Sender sends the image to the receiver, so the receiver doesn't have to download the link preview.
   Even simpler solution.

Just wanted to add a note:
Basically the problem can be reduced to email applications loading remote content. Personally I've opted out from loading remote content in my mail programs, but I can select "load remote content" separately for each mail.

Link previews/document preview are a great feature to know what is behind the link and helps the sender and the receiver to organise its content.

[trafenrest]

I prefer your second approach.

[mueller-ma]

As Greyson, one of the maintainers, noted above:

But as previously noted, people should be discussing this on the forum. We use github for bugs, not general discussion.

Please use https://community.signalusers.org/t/link-preview-when-pasting-or-typing-links/2370 to discuss how this feature should be improved.

[sougatc]

This issue is occurring again. Can someone please help debug this.

[jgspratt]

That's WhatsApp. Signal's text bar says "Signal message".

On Tue, Jul 21, 2020, 17:38 sougatc [e-mail] wrote:

This issue is occurring again. Can someone please help debug this.

[image: Screenshot_20200722-030650.jpg]
https://user-images.githubusercontent.com/64637913/88109611-754b7a80-cbc8-11ea-9b31-10d061888d7c.jpg

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/9086#issuecomment-662119917,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI63KZC7XBBXKH6VGXZ3Z3R4YDDTANCNFSM4I7HW5VA
.

[sougatc]

Sorry can you please elaborate more on this ? Do you mean the issue is at WhatsApp end ?

[jgspratt]

Your screenshot is from WhatsApp.

On Tue, Jul 21, 2020, 18:20 sougatc [e-mail] wrote:

Sorry can you please elaborate more on this ? Do you mean the issue is at
WhatsApp end ?

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/9086#issuecomment-662136293,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI63KZLULLAEFBKAAGIQS3R4YIEFANCNFSM4I7HW5VA
.

[trafenrest]

This issue is occurring again. Can someone please help debug this.

It was never gone

[jgspratt]

It started happening to WhatsApp users a few days ago, and some of them are
finding their way here because they don't know which app they are using.

On Wed, Jul 22, 2020, 12:37 trafenrest [e-mail] wrote:

This issue is occurring again. Can someone please help debug this.

It was never gone

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/9086#issuecomment-662558871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI63K4EVBDZDDRUOFVGMJDR44IS7ANCNFSM4I7HW5VA
.

[sougatc]

It started happening to WhatsApp users a few days ago, and some of them are
finding their way here because they don't know which app they are using.

On Wed, Jul 22, 2020, 12:37 trafenrest [e-mail] wrote:

This issue is occurring again. Can someone please help debug this.

It was never gone

—
You are receiving this because you are subscribed to this thread.
Reply to this email directly, view it on GitHub
https://github.com/signalapp/Signal-Android/issues/9086#issuecomment-662558871,
or unsubscribe
https://github.com/notifications/unsubscribe-auth/AAI63K4EVBDZDDRUOFVGMJDR44IS7ANCNFSM4I7HW5VA
.

I dropped a tweet to the Head of Products at YouTube and Whatsapp in hope that they might notice but it didn't yield any attention unfortunately.

[an0n7mous3]

This isn't the place to talk about Whatsapp.

This issue is still ongoing on Signal. Could I suggest giving users the option to source link previews from their own IP, rather than the Signal proxy which hasn't worked for over 10 months now?

[iomintz]

@an0n7mous3 this has been discussed before

[an0n7mous3]

@an0n7mous3 this has been discussed before

I'm aware, I read through this thread. This reply rebuts that stance quite well. Giving users the option to source previews locally could easily come with a privacy disclaimer. I don't see how it could be an issue.

[asbjornu]

@iomintz, it has also been strongly rebutted before: https://github.com/signalapp/Signal-Android/issues/9086#issuecomment-620171884. The argumentation has no technical merit and in the end should be a user choice.

[mueller-ma]

This can be closed now. Link previews don't use the proxy anymore that caused this issue.

[trafenrest]

When will this be effective?
I don't see any change yet and propose to close it when the effect is visible.

[nanowinner]

It has finally happened!

[Ralms]

It has finally happened!

What OS and Signal version is that?
I've checked both Windows and Android and the issue persists.

[Ralms]

Ok, had an update for Android, I can confirm that it's working on Signal version 4.69.4 on Android.
On Windows Desktop still waiting on the update, works receiving previews from Youtube, but other websites doesn't work receiving on Windows Desktop.

[nanowinner]

It has finally happened!

What OS and Signal version is that?
I've checked both Windows and Android and the issue persists.

[mueller-ma]

AFAIK support for link previews of all pages is in

• Android 4.69 https://community.signalusers.org/t/beta-feedback-for-the-upcoming-android-4-69-release/16219
• (receive only) iOS 3.15 or 3.14 (not exactly sure)
• currently not in Desktop

[iomintz]

Hopefully YouTube specifically doesn't break again.

[iomintz]

@iomintz, it has also been strongly rebutted before: #9086 (comment). The argumentation has no technical merit and in the end should be a user choice.

I gave it some more thought and I agree with you. Ultimately it's better for the user this way. I noticed that if I wanted to send someone a link to e.g. a Twitter post
, given the choice between telegram and signal I picked the less private option. For a lot of people I imagine it's the same. Anyway thanks for your arguments 👍

[Mhowser]

YouTube preview are not working again
Signal version 4.72.5

I still think we should have the option to load the preview locally, with a privacy disclaimer. Help us casual users out!

[iomintz]

They are loaded locally now @Mhowser. It didn't work for you because SMS doesn't support link previews at all.

[Mhowser]

Wow didn't know that, thanks for clearing that up.

[sougatc]

Hi
The issue seems pop up again. Found this on twitter it seems the issue is at Whatsapp end.

[Ralms]

Hi
The issue seems pop up again. Found this on twitter it seems the issue is at Whatsapp end.

LOOOOL and what does WhatsApp bugs have to do with Signal?

[trafenrest]

I have no problems with that on signal.
I don't use whatsapp.
This isn't the place to talk about Whatsapp.

[Ralms]

Probably you should go through the thread to understand what is being discussed.

You clearly need to do that, as you are clearly very confused on how software works.
This issue doesn't happen anymore in Signal for a few months since the apps where updated with a new implementation.

On top of that, Signal and WhatsApp have NOTHING to do with each other, they are completely independent applications.

Sorry of these couple lines was too much for your understanding.

[sougatc]

Probably you should go through the thread to understand what is being discussed.

You clearly need to do that, as you are clearly very confused on how software works.
This issue doesn't happen anymore in Signal for a few months since the apps where updated with a new implementation.

On top of that, Signal and WhatsApp have NOTHING to do with each other, they are completely independent applications.

Sorry of these couple lines was too much for your understanding.

I have removed the post. Hope that helps. For future, a word of advice, you could have conveyed the same in your first response rather than making fun. But I guess it's too much to ask for !

[sougatc]

I have no problems with that on signal.
I don't use whatsapp.
This isn't the place to talk about Whatsapp.

Have removed the post. Apologies for the inconvenience. Hope that helps

[Ralms]

Probably you should go through the thread to understand what is being discussed.

You clearly need to do that, as you are clearly very confused on how software works.
This issue doesn't happen anymore in Signal for a few months since the apps where updated with a new implementation.
On top of that, Signal and WhatsApp have NOTHING to do with each other, they are completely independent applications.
Sorry of these couple lines was too much for your understanding.

I have removed the post. Hope that helps. For future, a word of advice, you could have conveyed the same in your first response rather than making fun. But I guess it's too much to ask for !

And I did, I asked what does WhatsApp has to do with Signal.
Remember what your reply was? As I was the one confused.

[greyson-signal]

I believe the original issue has been resolved.